Athlete outlier interpretation method and device based on FOLOF algorithm

Abstract

The present invention discloses a method and device for explaining athlete outliers based on a FOLOF algorithm. The method comprises: detecting outliers in an athlete data set by using the FOLOF algorithm: firstly, using the elbow rule to determine the optimal clustering number of the athlete data set; then, pruning the athlete data set by using the objective function of FCM to obtain an athlete outlier candidate set; finally, using a weighted local outlier factor detection algorithm to calculate the degree of outlier of each athlete outlier in the athlete outlier candidate set to obtain an athlete outlier set; the athlete data set comprises an NBA player data set; and dividing the degree of outlier of the athlete outliers in each athlete outlier set into different levels by using the golden section method, and performing athlete outlier analysis according to the division result. The present invention can better explain the outlier causes of athlete outliers and explore the meaning behind them.

Description

Technical Field

[0001] The invention belongs to the technical field of athlete outlier mining, and in particular relates to an athlete outlier interpretation method and device based on a FOLOF algorithm. Background Art

[0002] Outliers are points that deviate far from most of the data, and they are often suspected to be generated by different mechanisms. Outlier detection plays an important role in data processing. It aims to mine meaningful potential information in the data set and is a hot topic in current research. It has been widely used in fraud detection, network intrusion detection, market analysis, point cloud denoising and medical diagnosis. Its detection methods can be roughly divided into statistical, distance, density and clustering-based methods. In addition to outlier detection, outlier analysis is also an important part of outlier mining. The main task of detection is to discover outliers from the data set, and analysis is to explore the connotation knowledge behind the outlier data detected, including classifying outliers to draw valuable conclusions or rules. Analyzing outliers often discovers more useful knowledge than analyzing regular data. Explaining outlier data can not only help users evaluate the quality of the overall data, but also enable users to have a further understanding of the causes, meanings and behavioral characteristics of outlier data. This is an issue that needs to be paid attention to in practical applications, and it is also the ultimate goal of outlier mining. Reference [Ding Tianyi, Zhang Min, Fang Shengliang. An outlier detection algorithm based on similarity pruning [J]. Small and Micro Computer Systems, 2018, 39(08): 1680-1684.] proposed an outlier detection algorithm based on similarity pruning. By constructing a similarity matrix and calculating the similarity between sample points, the non-outlier points are pruned, and then the outlier factors of all points in the outlier candidate set are calculated by the LOF algorithm. Reference [Bowen Zheng, Piervincenzo Rizzo, Amir Nasrollahi. Outlier analysis of nonlinear solitary waves for health monitoring applications [J]. Structural Health Monitoring, 2020, 19(4).] uses the inconsistency test and Mahalanobis square distance of outlier analysis to improve the damage detection capability of the monitoring system based on highly nonlinear solitary waves. They extracted features from the time waveform and input them into univariate and multivariate analysis. The test data was compared with a set of baseline data. The results show that outlier analysis significantly improves the ability of solitary waves to detect damage.Zhang Zhongping et al. [Zhang Zhongping, Liu Weixiong, Zhang Yuting, Deng Yu, Wei Mianxin. ERDOF: An outlier detection algorithm based on relative entropy weighted density outlier factor [J]. Journal of Communications, 2021, 42(09): 133-143.] proposed an outlier detection method based on relative entropy weighted density outlier factor, which improved the accuracy of the algorithm on complex data distribution and high-dimensional data sets. First, the entropy weighted distance was introduced to replace the Euclidean distance, and then the Gaussian kernel density of the data object was estimated in combination with the concept of natural neighbors. At the same time, the relative distance was proposed to characterize the degree to which the data object deviated from the neighborhood. Finally, the relative entropy weighted density outlier factor was used to characterize the degree of outlier of the data object. Knorr and Ng [Knorr EM, Ng R T. Finding intensional knowledge of distance-based outliers [C]. Proc. of the 25th International Conference on Very Large DataBases. New York: Morgan Kaufmann. 1999: 211-222.] divided distance-based outliers into two categories: trivial and non-trivial. According to their different characteristics, they further divided non-trivial outliers into strong and weak categories. They analyzed the outlier characteristics of the data from bottom to top according to the subspace composed of different attributes. Chen et al. [Z Chen, J Tang, A Fu. Modeling and Efficient Mining of Intentional Knowledge of Outliers [C]. Proc. of the 7th International Database Engineering and Applications Symposium. IEEE Computer Society. 2003: 1-10.] proposed a related search algorithm based on the above method. They believed that the difference in attribute subspaces affects the formation of outliers, and divided the attribute space into identification subspace, indicator subspace and observation subspace. However, this algorithm requires certain domain knowledge when selecting parameters, which may lead to different results for different observation points. Reference [Deng Yujie, Zhu Qingsheng. Clustering-based outlier analysis method [J]. Computer Application Research, 2012, 29 (03): 865-868.] introduced the concepts of outlier attributes and outlier clusters, proposed a method for outlier analysis based on outlier classification, implemented a clustering-based outlier classification method, and revealed the connotation knowledge of outliers.

[0003] However, there are few studies that apply outlier detection to athlete research, especially fewer studies on the interpretation and analysis of athlete outliers. Summary of the invention

[0004] Aiming at the problem that the existing outlier detection cannot mine the connotation knowledge behind the outlier data of athletes, that is, the cause of outliers, the present invention proposes an athlete outlier interpretation method and device based on FOLOF algorithm.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] On the one hand, the present invention proposes a method for interpreting athlete outliers based on the FOLOF algorithm, comprising:

[0007] Detect outliers in the player data set by using the FOLOF algorithm: first, use the elbow rule to determine the optimal number of clusters of the player data set; then prune the player data set by using the objective function of FCM to obtain a player outlier candidate set; finally, use the weighted local outlier factor detection algorithm to calculate the outlier degree of each player outlier in the player outlier candidate set to obtain a player outlier set; the player data set includes an NBA player data set;

[0008] The outlier degrees of the athlete outliers in each athlete outlier set are divided into different levels through the golden section method, and the athlete outlier analysis is performed based on the division results.

[0009] Furthermore, the detecting of outliers in the athlete data set by the FOLOF algorithm includes:

[0010] Step 1.1: Use the elbow rule to determine the optimal number of clusters c for the athlete data set D;

[0011] Step 1.2: Input parameter c, use FCM algorithm to cluster the athlete data set D, list the change of the objective function value after each athlete data is removed and compare them, and prune the athlete data set D according to the pruning threshold to obtain the athlete outlier candidate set D0;

[0012] Step 1.3: Perform Z-score standardization on the athlete dataset D to obtain D′. The weight P of the j-dimensional attribute of the i-th data in the standardized athlete dataset D′ is calculated by formula (8): ij :

[0013]

[0014] Among them, 1≤i≤n, 1≤j≤m, x′ ij is the j-th dimension attribute value of the i-th data of D′;

[0015] Step 1.4: Use formula (9) to calculate the information entropy E of the j-th dimension attribute in the data set D′ j :

[0016]

[0017] Where p = 1 / ln n;

[0018] Step 1.5: Use formula (10) to calculate the weight ω of the j-th dimension attribute j :

[0019]

[0020] After the above steps, we finally get the m-dimensional attribute weight set W = {ω1,ω2,...,ω m}, where 0≤ω j ≤1, and

[0021] Step 1.6: After using the entropy weight method to assign weights to each dimension of the attribute, the LOF algorithm is then executed on each dimension of the attribute of D0 separately, and weighted according to formula (11) to obtain the final LOF value as the anomaly score of the overall data:

[0022]

[0023] Where LOF ij Indicates the LOF value of the jth dimension of the i-th data;

[0024] Step 1.7: Arrange the LOF values ​​of each athlete data in descending order, output the first p data objects, and form the athlete outlier set of the athlete data set D.

[0025] Furthermore, the outlier degree of the athlete outlier points in each athlete outlier point set is divided into different levels by the golden section method, and the athlete outlier point analysis is performed according to the division result, including:

[0026] Assume that the outlier factor of the athlete data, i.e., the LOF value, ranges from [a, b], and divides it into n different levels. First, the entire area is subjected to the golden section, and then the remaining areas are subjected to the golden section in turn, and the corresponding levels are defined as T1, T2, ..., T n , define the upper limit of LOF value of each level as t1, t2, ..., t n ,have to:

[0027] t1=(1-0.618)a+[(1-0.618) 0 ]0.618b;

[0028] t2=(1-0.618) 2 a+[(1-0.618) 1 +(1-0.618) 0]0.618b;

[0029] t3=(1-0.618) 3 a+[(1-0.618) 2 +(1-0.618) 1 +(1-0.618) 0 ]0.618b;

[0030] …

[0031] t n =(1-0.618) n a+[(1-0.618) n-1 +(1-0.618) n-2 +...+(1-0.618) 1 +(1-0.618) 0 ]0.618b;

[0032] The value range of each outlier level is obtained in turn.

[0033] Furthermore, the outlier degree of the athlete outlier points in each athlete outlier point set is divided into different levels by the golden section method, and the athlete outlier point analysis is performed according to the division result, including:

[0034] The outlier levels are divided into three categories: weak outliers, ordinary outliers, and strong outliers. Among them, the outlier factor LOF(T1) value of weak outliers, that is, T1 level outliers, conforms to the following formula:

[0035] a≤LOF(T1)<(1-0.618)a+0.618b

[0036] For common outliers, i.e., T2-level outliers, the outlier factor LOF(T2) value meets the following conditions:

[0037] (1-0.618)a+0.618b≤LOF(T2)<(1-0.618) 2 a+[(1-0.618) 1 +(1-0.618) 0 ]0.618b

[0038] Strong outliers, i.e. T3 level outliers, have outlier factors LOF(T3) that meet the following criteria:

[0039] (1-0.618) 2 a+[(1-0.618) 1 +(1-0.618) 0 ]0.618b≤LOF(T3) <b。

[0040] Another aspect of the present invention provides an athlete outlier interpretation device based on the FOLOF algorithm, comprising:

[0041] An outlier detection module is used to detect outliers in a player data set by using a FOLOF algorithm: first, the elbow rule is used to determine the optimal number of clusters of the player data set; then, the player data set is pruned by using the objective function of FCM to obtain a player outlier candidate set; finally, a weighted local outlier factor detection algorithm is used to calculate the degree of outlier of each player outlier in the player outlier candidate set to obtain a player outlier set; the player data set includes an NBA player data set;

[0042] The outlier level classification module is used to classify the outlier degrees of the athlete outliers in each athlete outlier set into different levels by using the golden section method, and perform athlete outlier analysis based on the classification results.

[0043] Furthermore, the outlier detection module is specifically used for:

[0044] Step 1.1: Use the elbow rule to determine the optimal number of clusters c for the athlete data set D;

[0045] Step 1.2: Input parameter c, use FCM algorithm to cluster the athlete data set D, list the change of the objective function value after each athlete data is removed and compare them, and prune the athlete data set D according to the pruning threshold to obtain the athlete outlier candidate set D0;

[0046] Step 1.3: Perform Z-score standardization on the athlete dataset D to obtain D′. The weight P of the j-dimensional attribute of the i-th data in the standardized athlete dataset D′ is calculated by formula (8): ij :

[0047]

[0048] Among them, 1≤i≤n, 1≤j≤m, x′ ij is the j-th dimension attribute value of the i-th data of D′;

[0049] Step 1.4: Use formula (9) to calculate the information entropy E of the j-th dimension attribute in the data set D′ j :

[0050]

[0051] Where p = 1 / ln n;

[0052] Step 1.5: Use formula (10) to calculate the weight ω of the j-th dimension attribute j :

[0053]

[0054] After the above steps, we finally get the m-dimensional attribute weight set W = {ω1,ω2,...,ω m}, where 0≤ω j ≤1, and

[0055] Step 1.6: After using the entropy weight method to assign weights to each dimension of the attribute, the LOF algorithm is then executed on each dimension of the attribute of D0 separately, and weighted according to formula (11) to obtain the final LOF value as the anomaly score of the overall data:

[0056]

[0057] Where LOF ij Indicates the LOF value of the jth dimension of the i-th data;

[0058] Step 1.7: Arrange the LOF values ​​of each athlete data in descending order, output the first p data objects, and form the athlete outlier set of the athlete data set D.

[0059] Furthermore, the outlier level classification module is specifically used for:

[0060] Assume that the outlier factor of the athlete data, i.e., the LOF value, ranges from [a, b], and divides it into n different levels. First, the entire area is subjected to the golden section, and then the remaining areas are subjected to the golden section in turn, and the corresponding levels are defined as T1, T2, ..., T n , define the upper limit of LOF value of each level as t1, t2, ..., t n ,have to:

[0061] t1=(1-0.618)a+[(1-0.618) 0 ]0.618b;

[0062] t2=(1-0.618) 2 a+[(1-0.618) 1 +(1-0.618) 0 ]0.618b;

[0063] t3=(1-0.618) 3 a+[(1-0.618) 2 +(1-0.618) 1 +(1-0.618) 0 ]0.618b;

[0064] …

[0065] t n=(1-0.618) n a+[(1-0.618) n-1 +(1-0.618) n-2 +...+(1-0.618) 1 +(1-0.618) 0 ]0.618b;

[0066] The value range of each outlier level is obtained in turn.

[0067] Furthermore, the outlier level classification module is specifically used for:

[0068] The outlier levels are divided into three categories: weak outliers, ordinary outliers, and strong outliers. Among them, the outlier factor LOF(T1) value of weak outliers, that is, T1 level outliers, conforms to the following formula:

[0069] a≤LOF(T1)<(1-0.618)a+0.618b

[0070] For common outliers, i.e., T2-level outliers, the outlier factor LOF(T2) value meets the following conditions:

[0071] (1-0.618)a+0.618b≤LOF(T2)<(1-0.618) 2 a+[(1-0.618) 1 +(1-0.618) 0 ]0.618b

[0072] Strong outliers, i.e. T3 level outliers, have outlier factors LOF(T3) that meet the following criteria:

[0073] (1-0.618) 2 a+[(1-0.618) 1 +(1-0.618) 0 ]0.618b≤LOF(T3) <b。

[0074] Compared with the prior art, the present invention has the following beneficial effects:

[0075] The present invention can quickly and accurately detect outliers in the athlete data set through the FOLOF algorithm, and classify the athlete outliers according to the golden section method based on the size of the outlier factor obtained during the detection process. Through the outlier factor of each dimensional attribute, the outlier attributes of each athlete outlier data are mined to reveal the cause of the outlier. The proposed method is experimented in the NBA player data set to prove its effectiveness. The present invention can better explain the outlier cause of the athlete outlier and explore the meaning behind it. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 A basic flow chart of a method for interpreting athlete outliers based on the FOLOF algorithm according to an embodiment of the present invention;

[0077] Figure 2 This is a schematic diagram of the golden section method according to an embodiment of the present invention;

[0078] Figure 3 This is a schematic diagram of an outlier factor line in an embodiment of the present invention;

[0079] Figure 4 Schematic diagram of outlier classification in two cases of an embodiment of the present invention;

[0080] Figure 5 This is a schematic diagram of an artificial dataset dataset1 according to an embodiment of the present invention;

[0081] Figure 6 A classification diagram of outliers according to outlier factors in an embodiment of the present invention;

[0082] Figure 7 A classification diagram of outliers according to the first dimension outlier factor in an embodiment of the present invention;

[0083] Figure 8 A classification diagram of outliers according to the second dimension outlier factor in an embodiment of the present invention;

[0084] Fig. 9 The present invention is a schematic diagram of the architecture of an athlete outlier interpretation device based on the FOLOF algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0085] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:

[0086] like Figure 1 As shown, a method for explaining athlete outliers based on the FOLOF algorithm includes:

[0087] The outliers in the player data set are detected by using the FOLOF algorithm (FCM Objective function-based LOF, an outlier detection method based on an objective function): first, the elbow rule is used to determine the optimal number of clusters of the player data set; then, the player data set is pruned by using the FCM objective function to obtain a player outlier candidate set; finally, the weighted local outlier factor detection algorithm is used to calculate the outlier degree of each player outlier in the player outlier candidate set to obtain a player outlier set; the player data set includes an NBA player data set;

[0088] The outlier degrees of the athlete outliers in each athlete outlier set are divided into different levels through the golden section method, and the athlete outlier analysis is performed based on the division results.

[0089] Specifically, the FOLOF algorithm first uses the elbow rule to determine the optimal number of clusters for the athlete dataset; then prunes the athlete dataset through the objective function of FCM to obtain a candidate set of athlete outliers; finally, the weighted local outlier factor detection algorithm is used to calculate the outlier degree of each point in the candidate set of athletes.

[0090] Information entropy is often used to characterize the degree of discreteness of data. The larger the entropy value, the greater the discreteness of the variable, and the less information it provides; the smaller the entropy value, the smaller the discreteness of the variable, and the more information it provides. Therefore, the features of different dimensions of data provide different amounts of information, and different weights need to be assigned accordingly. For attributes with large entropy, the weight should be higher, so that the degree of discreteness can be more clearly reflected. In the LOF algorithm, each dimension of the attribute is assigned a weight, and then the FOLOF algorithm is proposed.

[0091] Specifically, the FOLOF algorithm is as follows:

[0092] Input: Athlete dataset D, number of neighborhood queries k, pruning threshold T, number of outliers p;

[0093] Output: Top p athlete outliers;

[0094] Step 1: Use the elbow rule to determine the optimal number of clusters c for the athlete data set D to obtain the best clustering effect;

[0095] Step 2 Input parameter c, use FCM algorithm to cluster the athlete data set D, list the change of objective function value after each athlete data is removed and compare them, according to the pruning threshold T (AvgDOF i ), prune the athlete dataset D to obtain the athlete outlier candidate set D0;

[0096] Step 3: Perform Z-score normalization on the athlete dataset D to obtain D′. The weight P of the j-dimensional attribute of the i-th data in the standardized athlete dataset D′ is calculated by formula (8): ij :

[0097]

[0098] Among them, 1≤i≤n, 1≤j≤m, x′ ij is the j-th dimension attribute value of the i-th data of D′;

[0099] Step 4: Use formula (9) to calculate the information entropy E of the j-th dimension attribute in the athlete data set D′.j :

[0100]

[0101] Where p = 1 / ln n;

[0102] Step 5: Use formula (10) to calculate the weight ω of the j-th dimension attribute. j :

[0103]

[0104] After the above steps, we finally get the m-dimensional attribute weight set W = {ω1,ω2,...,ω m}, where 0≤ω j ≤1, and

[0105] Step 6 uses the entropy weight method to assign weights to each dimension of the attribute, and then executes the traditional LOF algorithm separately for each dimension of the attribute of D0, weighting it according to formula (11) to obtain the final LOF value as the anomaly score of the overall data.

[0106]

[0107] Where LOF ij Indicates the LOF value of the jth dimension of the i-th data;

[0108] Step 7 arranges the LOF values ​​of each athlete data in descending order, outputs the first p data objects, and constitutes the athlete outlier set of the athlete data set D.

[0109] The golden section is also called the best section. It means dividing the whole into two parts. The ratio of the larger part to the whole part is equal to the ratio of the smaller part to the larger part. The ratio is about 0.618. This ratio is recognized as the most aesthetically pleasing ratio, so it is called the golden section. The golden section has strict proportionality and artistry, and is widely used in aesthetics, music, architecture, transportation and other fields. Figure 2 As shown, let the length of a line segment AB be a, point C is at the golden section point close to point B, and AC is b, then the ratio of b to a is called the golden ratio, and its length conforms to the following formula reasoning:

[0110]

[0111] That is b 2 =a(ab)=a 2 -ab(13)

[0112]

[0113]

[0114] After simple arrangement,

[0115] Assuming that the length of AB is 1, the position of point C is approximately 0.618, which shows that point C is the optimal segmentation position in interval AB. Based on the above analysis, the present invention proposes an outlier factor classification method based on the golden section method, which can accurately mine the connotation knowledge behind the outlier points and find the cause of the outlier.

[0116] The FOLOF algorithm can be used to obtain the outlier factor of each data in the athlete outlier candidate set and the outlier factor of each dimensional feature. The size of the outlier factor, i.e., the LOF value, represents the degree of outlier of the data. The golden section method can be used to divide the degree of outlier into different levels according to its value range. Since the number of outliers at lower levels is often more than that at higher levels, the idea of ​​the golden section is completely reasonable.

[0117] Assume that the outlier factor of the athlete data, i.e., the LOF value, ranges from [a, b] (i.e., a = LOF min , indicating the minimum value of LOF; b = LOF max , indicating the maximum value of LOF value), divide it into n different levels, first perform golden section on the entire area, then perform golden section on the remaining areas in turn, and define the corresponding levels as T1, T2, ..., T n , and define its upper limit as t1,t2,...,t n , after summarizing and sorting, we get:

[0118] t1=(1-0.618)a+[(1-0.618) 0 ]0.618b;

[0119] t2=(1-0.618) 2 a+[(1-0.618) 1 +(1-0.618) 0 ]0.618b;

[0120] t3=(1-0.618) 3 a+[(1-0.618) 2 +(1-0.618) 1 +(1-0.618) 0 ]0.618b;

[0121] …

[0122] t n =(1-0.618) na+[(1-0.618) n-1 +(1-0.618) n-2 +...+(1-0.618) 1 +(1-0.618) 0 ]0.618b;

[0123] The value range of each outlier level can be obtained in turn.

[0124] Assume that n is 3, and divide the outliers into three categories: weak, normal, and strong. The definitions of outliers in each category are as follows:

[0125] Definition 1: Weak outlier: An outlier at level T1, whose outlier factor LOF(T1) value conforms to the following formula:

[0126] LOF min ≤LOF(T1)<(1-0.618)LOF min +0.618LOF max

[0127] Definition 2: Common outliers: Outliers of level T2, whose outlier factor LOF(T2) value meets,

[0128] (1-0.618)LOF min +0.618LOF max ≤LOF(T2)<(1-0.618) 2 LOF min +[(1-0.618) 1 +(1-0.618) 0 ]0.618LOF max

[0129] Define three strong outliers: T3 level outliers, whose outlier factor LOF(T3) meets,

[0130] (1-0.618) 2 LOF min +[(1-0.618) 1 +(1-0.618) 0 ]0.618LOF max ≤LOF(T3) <LOF max

[0131] The introduction of equal outlier factor lines illustrates the application of the golden section method in outlier classification. Figure 3As shown in the figure, the center of the circle represents the minimum outlier factor a, and the outermost circumference represents the maximum outlier factor b. The same circumference represents the same outlier factor. The length of this range is taken as the radius and divided into three areas T1, T2, and T3 according to the golden section method mentioned above, representing three outlier levels. The outlier factor gradually increases along the dotted line direction.

[0132] In addition, in real data sets, there are many distributions of outliers. For example, Figure 4 As shown:

[0133] exist Figure 4 (a) shows that there is only one cluster, and the red points are outliers. The farther away from the cluster, the larger the outlier factor, that is, the higher the outlier level. The boundaries of the regions of each level are marked in turn according to the golden section method. After excluding normal points, the three outliers belong to levels T1, T2, and T3 respectively. Figure 4 The blue points in (b) are the k-neighborhoods of the corresponding outliers. The points around the k-neighborhoods of the T1-level outliers are relatively sparse, that is, the neighborhood density of their k-neighborhoods is small, so the outlier factor of the point is small and the outlier level is low. The points around the k-neighborhoods of the T2-level outliers are relatively dense, that is, the neighborhood density of their k-neighborhoods is large, so the outlier factor of the point is large and the outlier level is high. The lower the outlier level, the closer it is to the average level of the cluster to which it belongs, and the higher the level, the farther it deviates from the average level of the cluster to which it belongs.

[0134] To verify the effect of the present invention, this example uses the 2020-2021 NBA player data set for outlier analysis, explains the causes of the corresponding outliers, and mines the connotation knowledge behind them. All the parameters mentioned in the previous section are set uniformly in the following experiment, the threshold T used in pruning is 1, and the number of neighbors k in the FOLOF algorithm is 5.

[0135] NBA Players Dataset

[0136] This part uses the outlier classification method to classify all the experience, second-year, and rookie player data of the NBA 2020-2021 regular season to explore the connotation knowledge behind the outlier players. The data set has a total of 150 data, including 5-dimensional features such as average points per game, average rebounds per game, free throws, offense, and defense.

[0137] First, the outlier factors of the top 10 outliers and the outlier factors of each dimension are detected by the FOLOF detection algorithm, as shown in Table 1.

[0138] Table 1 Outlier factors of the top 10 outliers and outlier factors of each dimension (outlier factors are arranged in descending order)

[0139]

[0140] The 10 players corresponding to the above points are LD, DF, GA, JH, MB, BT, LJ, SC, DL, and DG. The outlier factors of each dimension of these ten points are classified, and five representative players are selected for analysis, namely LD (6), GA (5), JH (21), MB (79), and SC (1). The classification results are shown in Table 2.

[0141] Table 2 Classification of representative player outliers

[0142]

[0143] Obviously, for 6 and 21, the second dimension feature, average rebounds per game, is the main reason for their outliers. Combined with the actual data, LD and JH have 8 and 7.9 rebounds per game in the series, while the average rebounds per game of stars of the same level is about 5. LD and JH are more outstanding in average rebounding ability compared with other guards, which is completely consistent with the reality.

[0144] For 5, the third dimension feature, free throws, is the main reason for his outlier. Combined with the actual data, GA's free throw percentage in the series is only 68.5, while the free throw percentages of the stars of the same level are around 85 or even higher, indicating that GA's free throws are his biggest weakness, far below the NBA average. In future training, he can practice more free throws.

[0145] For 79, the fourth dimension feature, namely the offensive value, is the main reason for his outlier. Combined with the actual data, MB's offensive value is as high as 3.6, while the offensive values ​​of the stars of the same level are all around 1, which means that MB's offensive efficiency this season is very high. He is a more outstanding player among the second-year players and deserves the team's key training.

[0146] For 1, the first dimension feature, namely scoring, is the main reason for his outlier status. According to the actual data, SC's average score per game is as high as 32, while the average score per game of the stars of the same level is around 25. SC is very outstanding in scoring ability and is one of the best guards today. In terms of other abilities, SC is not very outstanding, which makes his outlier level not as high as LD.

[0147] In the NBA, all players are carefully selected outstanding players whose comprehensive abilities are already far superior to ordinary people, and the limits of human beings are often difficult to surpass. Therefore, in the classification, weak outliers account for the majority of cases, and there are very few players who can reach the ordinary outlier level in a certain ability. For players with better statistics, players who reach the strong outlier level already represent the highest level of the NBA in terms of corresponding abilities.

[0148] In summary, in order to mine the connotation knowledge behind the outlier data, that is, the reasons causing the outliers, the present invention proposes a method for classifying the detected outliers according to the size of the outlier factor and the golden section method. First, the FOLOF algorithm is used to detect the top p athlete outliers, and then the overall outlier factor and each dimension outlier factor of each athlete outlier are recorded, and the golden section method is used to segment its regions and record them as T1, T2, …, T n Finally, the outlier reason is obtained according to the outlier level of each dimension of outlier factor. The method proposed by the present invention can well explain the outlier reason of the athlete outlier point and explore the meaning behind it. Analyzing the outlier data of athletes can obtain more useful knowledge and rules than analyzing the normal data.

[0149] It is worth noting that the method of the present invention is also applicable to other data sets, such as artificial data sets (point sets) and UCI data sets.

[0150] On the basis of the above embodiments, in order to verify the versatility of the present invention, the following experimental design is carried out:

[0151] (1) Artificial dataset dataset1

[0152] In order to visualize the whole process of outlier detection and interpretation, experiments are conducted using artificial data dataset1. Dataset1 has 70 data points, including 10 outliers, namely the 6th, 7th, 19th, 30th, 40th, 45th, 52nd, 58th, 69th, and 70th points. Figure 5 shown.

[0153] Through pruning, we get 11 points, namely the 6th, 7th, 19th, 30th, 40th, 45th, 52nd, 58th, 60th, 69th, and 70th. The top 10 points with the largest outlier values ​​are output, namely the 58th, 69th, 60th, 6, 40th, 19th, 7, 52nd, 30th, and 45th points. Compared with the manually set outliers, the outlier factor of the 60th point detected by the algorithm is much larger than that of the 70th point, so it is excluded from the top 10 outliers. LOF values ​​of 11 points in the outlier candidate set and LOF of each dimension i The values ​​are shown in Table 3.

[0154] Table 3 Outlier factors of each point and each dimension in the outlier candidate set of dataset1 (outlier factors are arranged in descending order)

[0155]

[0156] From the table, we can conclude that the outlier factors of the 11 points in the first dimension are not much different. The reason why the 60th point is more outlier than the 70th point is mainly in the second dimension. The outlier factor of the 60th point in the second dimension is much larger than that of the 70th point.

[0157] For the outlier factor:

[0158] Weak outliers = {6, 7, 19, 30, 40, 45, 52, 70}, normal outliers = {60}, strong outliers = {58, 69}.

[0159] The classification results are as follows Figure 6 shown.

[0160] Similarly, for the first dimension outlier factor, weak outliers = {7, 19, 30, 45, 52, 58, 60, 69, 70}, common outliers = {40}, strong outliers = {6}, the classification results are shown in the figure below. Figure 7 shown.

[0161] For the second dimension outlier factor, weak outliers = {6, 7, 19, 30, 40, 45, 45, 52, 70}, common outliers = {60}, strong outliers = {58, 69}, and the classification results are shown in the figure below. Figure 8 shown.

[0162] From the classification results, we can see that the 70th point is a weak outlier in both dimensions, while the 60th point is a weak outlier in the first dimension but a common outlier in the second dimension. Therefore, in general, the outlier degree of the 60th point is much higher than that of the 70th point. In addition, in the outlier factor classification, the 6th point is a weak outlier, while in the first-dimensional outlier factor, the 6th point is a strong outlier. Therefore, the first-dimensional outlier factor is the main reason for the outlier of the 6th point, which directly leads to its high outlier degree.

[0163] The direct cause of the outlier factor is the reachable density, i.e., the lrdk value. The reachable densities of the 11 points in the candidate set are listed in Table 4, and the specific reasons for the data outliers can be clearly observed.

[0164] Table 4. The reachable density of each point in the outlier candidate set of dataset1 and the reachable density of each dimension

[0165]

[0166] From the table, we can see that the reachable density of the 11 points in the first dimension is not much different, while the reachable density of the 60th point in the second dimension is much smaller than that of the 70th point, which leads to a larger outlier factor. Therefore, the reason for the outlier of the 60th point is mainly in the second dimension, which is consistent with the analysis of the outlier factor.

[0167] (2) UCI ​​Dataset

[0168] This part classifies outliers in Iris, Wine, Yeast and UKM datasets, showing that the proposed method is feasible. The value of n is 3 for outlier classification in all four datasets.

[0169] Table 5 Experimental data statistics

[0170]

[0171]

[0172] The Iris dataset selects the first two categories of data as normal data, and randomly selects 10 data points from the third category as outlier data. Similarly, the first two categories of the Wine dataset, totaling 130 data points, are selected as normal data, and 10 data points are randomly selected from the third category as outlier data. For the Yeast dataset, it is not necessary to use all the data. A total of 1,136 data points from three major categories, including CYT, NUC, and MIT, are selected as normal data, and 20 data points from the POX category are selected as outlier data. In addition, since its sixth dimension attribute contains a 0 value, when running the attribute weighting algorithm to calculate the proportion, the existence of a 0 value is not allowed. The sixth dimension attribute is deleted, and the remaining 7-dimensional attributes are used for outlier detection. Similarly, for the UKM dataset, since very_low contains a 0 value, a total of 224 data points from High and Middle are selected as normal data, and 5 data points from Low are randomly selected as outlier data.

[0173] In the Iris dataset, 9 outliers were detected, which were the 101st, 103rd to 110th points. The first-dimensional outlier factors of these 9 points were classified. In the Wine dataset, 10 outliers were detected, which were the 131st to 140th points. The third-dimensional outlier factors of these 10 points were classified. In the Yeast dataset, 18 outliers were detected, which were the 1117th, 1119th, 1121st to 1136th points. The fifth-dimensional outlier factors of these 18 points were classified. In the UKM dataset, 5 outliers were detected, which were the 225th to 229th points. The second-dimensional outlier factors of these 5 points were classified. The classification results are listed in Table 6.

[0174] Table 6 Classification of outliers in four data sets

[0175]

[0176] From the classification results, we can see that in the Iris dataset, the first-dimensional feature calyx length is one of the main reasons for the outlier of the 107th point, and its calyx length is small. Similarly, in the Wine dataset, the third-dimensional feature Ash is one of the main reasons for the outliers of the 135th, 136th, and 139th points, and the Ash values ​​of these three data are small. In the Yeast dataset, the fifth-dimensional feature erl is one of the main reasons for the outliers of the 1123rd, 1129th, 1130th, 1131st, and 1132nd points, among which the corresponding attribute values ​​of 1123 and 1130 are small, and the rest are large. In the UKM dataset, the second-dimensional feature SCG is one of the main reasons for the outlier of the 229th point, and its attribute value is small. By classifying the outlier data, we can know the specific reasons for the outlier data, and combine it with the actual data to deeply explore the characteristics behind the data.

[0177] Based on the above embodiments, Fig. 9 As shown, the present invention also proposes an athlete outlier interpretation device based on the FOLOF algorithm, comprising:

[0178] An outlier detection module is used to detect outliers in a player data set by using a FOLOF algorithm: first, the elbow rule is used to determine the optimal number of clusters of the player data set; then, the player data set is pruned by using the objective function of FCM to obtain a player outlier candidate set; finally, a weighted local outlier factor detection algorithm is used to calculate the degree of outlier of each player outlier in the player outlier candidate set to obtain a player outlier set; the player data set includes an NBA player data set;

[0179] The outlier level classification module is used to classify the outlier degrees of the athlete outliers in each athlete outlier set into different levels by using the golden section method, and perform athlete outlier analysis based on the classification results.

[0180] Furthermore, the outlier detection module is specifically used for:

[0181] Step 1.1: Use the elbow rule to determine the optimal number of clusters c for the athlete data set D;

[0182] Step 1.2: Input parameter c, use FCM algorithm to cluster the athlete data set D, list the change of the objective function value after each athlete data is removed and compare them, and prune the athlete data set D according to the pruning threshold to obtain the athlete outlier candidate set D0;

[0183] Step 1.3: Perform Z-score standardization on the athlete dataset D to obtain D′. The weight P of the j-dimensional attribute of the i-th data in the standardized athlete dataset D′ is calculated by formula (8): ij :

[0184]

[0185] Among them, 1≤i≤n, 1≤j≤m, x′ ij is the j-th dimension attribute value of the i-th data of D′;

[0186] Step 1.4: Use formula (9) to calculate the information entropy E of the j-th dimension attribute in the data set D′ j :

[0187]

[0188] Where p = 1 / ln n;

[0189] Step 1.5: Use formula (10) to calculate the weight ω of the j-th dimension attribute j :

[0190]

[0191] After the above steps, we finally get the m-dimensional attribute weight set W = {ω1,ω2,...,ω m}, where 0≤ω j ≤1, and

[0192] Step 1.6: After using the entropy weight method to assign weights to each dimension of the attribute, the LOF algorithm is then executed on each dimension of the attribute of D0 separately, and weighted according to formula (11) to obtain the final LOF value as the anomaly score of the overall data:

[0193]

[0194] Where LOF ij Indicates the LOF value of the jth dimension of the i-th data;

[0195] Step 1.7: Arrange the LOF values ​​of each athlete data in descending order, output the first p data objects, and form the athlete outlier set of the athlete data set D.

[0196] Furthermore, the outlier level classification module is specifically used for:

[0197] Assume that the outlier factor of the athlete data, i.e., the LOF value, ranges from [a, b], and divides it into n different levels. First, the entire area is subjected to the golden section, and then the remaining areas are subjected to the golden section in turn, and the corresponding levels are defined as T1, T2, ..., T n , define the upper limit of LOF value of each level as t1, t2, ..., t n ,have to:

[0198] t1=(1-0.618)a+[(1-0.618) 0 ]0.618b;

[0199] t2=(1-0.618) 2 a+[(1-0.618) 1 +(1-0.618) 0 ]0.618b;

[0200] t3=(1-0.618) 3 a+[(1-0.618) 2 +(1-0.618) 1 +(1-0.618) 0 ]0.618b;

[0201] …

[0202] t n =(1-0.618) n a+[(1-0.618) n-1 +(1-0.618) n-2 +...+(1-0.618) 1 +(1-0.618) 0 ]0.618b;

[0203] The value range of each outlier level is obtained in turn.

[0204] Furthermore, the outlier level classification module is specifically used for:

[0205] The outlier levels are divided into three categories: weak outliers, ordinary outliers, and strong outliers. Among them, the outlier factor LOF(T1) value of weak outliers, that is, T1 level outliers, conforms to the following formula:

[0206] a≤LOF(T1)<(1-0.618)a+0.618b

[0207] For common outliers, i.e., T2-level outliers, the outlier factor LOF(T2) value meets the following conditions:

[0208] (1-0.618)a+0.618b≤LOF(T2)<(1-0.618) 2 a+[(1-0.618) 1 +(1-0.618) 0 ]0.618b

[0209] Strong outliers, i.e. T3 level outliers, have outlier factors LOF(T3) that meet the following criteria:

[0210] (1-0.618) 2a+[(1-0.618) 1 +(1-0.618) 0 ]0.618b≤LOF(T3) <b。

[0211] In summary, the present invention can quickly and accurately detect outliers in the athlete data set through the FOLOF algorithm, classify the athlete outliers according to the golden section method based on the size of the outlier factor obtained during the detection process, and mine the outlier attributes of each athlete outlier data through the outlier factor of each dimension attribute, revealing the cause of the outlier, and the proposed method is experimented in the NBA player data set to prove its effectiveness. The present invention can better explain the outlier causes of athlete outliers and explore the meaning behind them.

[0212] The above is only a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for explaining athlete outliers based on the FOLOF algorithm, characterized in that: include: The outliers in the athlete dataset are detected by the FOLOF algorithm: first, the elbow rule is used to determine the optimal number of clusters for the athlete dataset; Then, the player data set is pruned by the objective function of FCM to obtain a player outlier candidate set; finally, the outlier degree of each player outlier in the player outlier candidate set is calculated by using a weighted local outlier factor detection algorithm to obtain a player outlier set; the player data set includes an NBA player data set; The outliers detected in the athlete data set by the FOLOF algorithm include: Step 1.1: Use the elbow rule to determine the optimal number of clusters c for the athlete data set D; Step 1.2: Input parameter c, use FCM algorithm to cluster the athlete data set D, list the change of the objective function value after each athlete data is removed and compare them, and prune the athlete data set D according to the pruning threshold to obtain the athlete outlier candidate set D0; Step 1.3: Perform Z-score standardization on the athlete dataset D to obtain D′. The weight P of the j-dimensional attribute of the i-th data in the standardized athlete dataset D′ is calculated by formula (8): ij : Among them, 1≤i≤n, 1≤j≤m, x′ ij is the j-th dimension attribute value of the i-th data of D′; Step 1.4: Use formula (9) to calculate the information entropy E of the j-th dimension attribute in the data set D′ j : Where p = 1 / ln n; Step 1.5: Use formula (10) to calculate the weight ω of the j-th dimension attribute j : After the above steps, we finally get the m-dimensional attribute weight set W = {ω1,ω2,...,ω m }, where 0≤ω j ≤1, and Step 1.6: After using the entropy weight method to assign weights to each dimension of the attribute, the LOF algorithm is then executed separately for each dimension of the attribute of D0, and the weights are added according to formula (11) to obtain the final LOF value as the anomaly score of the overall data: Where LOF ij Represents the LOF value of the jth dimension of the i-th data; Step 1.7: Arrange the LOF values ​​of each athlete's data in descending order, output the first p data objects, and form the athlete outlier set of the athlete data set D; The outlier degrees of the athlete outliers in each athlete outlier set are divided into different levels through the golden section method, and the athlete outlier analysis is performed based on the division results.

2. The athlete outlier interpretation method based on the FOLOF algorithm according to claim 1, characterized in that: The method of dividing the outlier degree of the athlete outlier points in each athlete outlier point set into different levels by the golden section method, and performing athlete outlier point analysis according to the division result includes: Assume that the outlier factor of the athlete data, i.e., the LOF value, ranges from [a, b], and divides it into n different levels. First, the entire area is subjected to the golden section, and then the remaining areas are subjected to the golden section in turn, and the corresponding levels are defined as T1, T2, ..., T n , define the upper limit of LOF value of each level as t1, t2, ..., t n ,have to: t1=(1-0.618)a+[(1-0.618) 0 ]0.618b; t2=(1-0.618) 2 a+[(1-0.618) 1 +(1-0.618) 0 ]0.618b; <h2 style=";text-align:left;direction:ltr">t3 = (1-0.618)<h2 style=";text-align:left;direction:ltr"> 3 <h2 style=";text-align:left;direction:ltr"> a+[(1-0.618)<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +(1-0.618)<h2 style=";text-align:left;direction:ltr"> 1 <h2 style=";text-align:left;direction:ltr"> +(1-0.618)<h2 style=";text-align:left;direction:ltr"> 0 <h2 style=";text-align:left;direction:ltr"> ]0.618b; … t n =(1-0.618) n a+[(1-0.618) n-1 +(1-0.618) n-2 +...+(1-0.618) 1 +(1-0.618) 0 ]0.618b; The value range of each outlier level is obtained in turn.

3. The athlete outlier interpretation method based on the FOLOF algorithm according to claim 2, characterized in that: The method of dividing the outlier degree of the athlete outlier points in each athlete outlier point set into different levels by the golden section method, and performing athlete outlier point analysis according to the division result includes: The outlier levels are divided into three categories: weak outliers, ordinary outliers, and strong outliers. Among them, the outlier factor LOF(T1) value of weak outliers, that is, T1 level outliers, conforms to the following formula: a≤LOF(T1)<(1-0.618)a+0.618b For common outliers, i.e., T2-level outliers, the outlier factor LOF(T2) value meets the following conditions: (1-0.618)a+0.618b≤LOF(T2)<(1-0.618) 2 a+[(1-0.618) 1 +(1-0.618) 0 ]0.618b Strong outliers, i.e. T3 level outliers, have outlier factors LOF(T3) that meet the following criteria: <h2 style=";text-align:left;direction:ltr">(1-0.618)<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> a+[(1-0.618)<h2 style=";text-align:left;direction:ltr"> 1 <h2 style=";text-align:left;direction:ltr"> +(1-0.618)<h2 style=";text-align:left;direction:ltr"> 0 <h2 style=";text-align:left;direction:ltr"> ]0.618b≤LOF(T3) <b。 4. A device for explaining athlete outliers based on the FOLOF algorithm, characterized in that: include: The outlier detection module is used to detect outliers in the athlete data set using the FOLOF algorithm: first, the elbow rule is used to determine the optimal number of clusters for the athlete data set; Then, the player data set is pruned by the objective function of FCM to obtain a player outlier candidate set; finally, the outlier degree of each player outlier in the player outlier candidate set is calculated by using a weighted local outlier factor detection algorithm to obtain a player outlier set; the player data set includes an NBA player data set; The outlier detection module is specifically used for: Step 1.1: Use the elbow rule to determine the optimal number of clusters c for the athlete data set D; Step 1.2: Input parameter c, use FCM algorithm to cluster the athlete data set D, list the change of the objective function value after each athlete data is removed and compare them, and prune the athlete data set D according to the pruning threshold to obtain the athlete outlier candidate set D0; Step 1.3: Perform Z-score standardization on the athlete dataset D to obtain D′. The weight P of the j-dimensional attribute of the i-th data in the standardized athlete dataset D′ is calculated by formula (8): ij : Among them, 1≤i≤n, 1≤j≤m, x′ ij is the j-th dimension attribute value of the i-th data of D′; Step 1.4: Use formula (9) to calculate the information entropy E of the j-th dimension attribute in the data set D′ j : Where p = 1 / ln n; Step 1.5: Use formula (10) to calculate the weight ω of the j-th dimension attribute j : After the above steps, we finally get the m-dimensional attribute weight set W = {ω1,ω2,...,ω m }, where 0≤ω j ≤1, and Step 1.6: After using the entropy weight method to assign weights to each dimension of the attribute, the LOF algorithm is then executed separately for each dimension of the attribute of D0, and the weights are added according to formula (11) to obtain the final LOF value as the anomaly score of the overall data: Where LOF ij Represents the LOF value of the jth dimension of the i-th data; Step 1.7: Arrange the LOF values ​​of each athlete's data in descending order, output the first p data objects, and form the athlete outlier set of the athlete data set D; The outlier level classification module is used to classify the outlier degrees of the athlete outliers in each athlete outlier set into different levels by using the golden section method, and perform athlete outlier analysis based on the classification results.

5. The athlete outlier interpretation device based on the FOLOF algorithm according to claim 4, characterized in that: The outlier level classification module is specifically used for: Assume that the outlier factor of the athlete data, i.e., the LOF value, ranges from [a, b], and divides it into n different levels. First, the entire area is subjected to the golden section, and then the remaining areas are subjected to the golden section in turn, and the corresponding levels are defined as T1, T2, ..., T n , define the upper limit of LOF value of each level as t1, t2, ..., t n ,have to: t1=(1-0.618)a+[(1-0.618) 0 ]0.618b; t2=(1-0.618) 2 a+[(1-0.618) 1 +(1-0.618) 0 ]0.618b; <h2 style=";text-align:left;direction:ltr">t3 = (1-0.618)<h2 style=";text-align:left;direction:ltr"> 3 <h2 style=";text-align:left;direction:ltr"> a+[(1-0.618)<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +(1-0.618)<h2 style=";text-align:left;direction:ltr"> 1 <h2 style=";text-align:left;direction:ltr"> +(1-0.618)<h2 style=";text-align:left;direction:ltr"> 0 <h2 style=";text-align:left;direction:ltr"> ]0.618b; … t n =(1-0.618) n a+[(1-0.618) n-1 +(1-0.618) n-2 +...+(1-0.618) 1 +(1-0.618) 0 ]0.618b; The value range of each outlier level is obtained in turn.

6. The athlete outlier interpretation device based on the FOLOF algorithm according to claim 5, characterized in that: The outlier level classification module is specifically used for: The outlier levels are divided into three categories: weak outliers, ordinary outliers, and strong outliers. Among them, the outlier factor LOF(T1) value of weak outliers, that is, T1 level outliers, conforms to the following formula: a≤LOF(T1)<(1-0.618)a+0.618b For common outliers, i.e., T2-level outliers, the outlier factor LOF(T2) value meets the following conditions: (1-0.618)a+0.618b≤LOF(T2)<(1-0.618) 2 a+[(1-0.618) 1 +(1-0.618) 0 ]0.618b Strong outliers, i.e. T3 level outliers, have outlier factors LOF(T3) that meet the following criteria: <h2 style=";text-align:left;direction:ltr">(1-0.618)<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> a+[(1-0.618)<h2 style=";text-align:left;direction:ltr"> 1 <h2 style=";text-align:left;direction:ltr"> +(1-0.618)<h2 style=";text-align:left;direction:ltr"> 0 <h2 style=";text-align:left;direction:ltr"> ]0.618b≤LOF(T3) <b。

Images