Traffic flow data anomaly identification method based on fuzzy outlier detection

Abstract

The invention discloses a traffic flow data anomaly identification method based on fuzzy outlier detection, and the method comprises the steps: obtaining a traffic flow data set, constructing delta B + tree indexes based on the traffic flow data set, constructing a first boundary search algorithm and a second boundary search algorithm, and constructing an MBAFOM search algorithm which is used for setting limiting parameters, all leaf nodes are obtained from the delta B + tree indexes, the number of first neighbors and the number of second neighbors of all the leaf nodes are calculated according to a first boundary search algorithm and a second boundary search algorithm, and the types of the leaf nodes are calculated according to limiting parameters and the number of the first neighbors and the number of the second neighbors corresponding to the leaf nodes; all potential outliers are obtained, the fuzzy outlier degree of the potential outliers is calculated, and all determined outliers, the potential outliers and the fuzzy outlier degree of the potential outliers serve as abnormal traffic data. According to the method, the retrieval efficiency of the data nodes and the detection precision of the abnormal traffic data are improved.

Description

Technical Field

[0001] The present invention relates to the field of traffic flow prediction, and in particular to a method for identifying anomalies in traffic flow data based on fuzzy outlier detection. Background Art

[0002] Traffic flow forecasting is a core task in intelligent transportation systems (ITS). Its purpose is to predict future traffic flow changes by analyzing historical traffic data and real-time monitoring data. These predictions have a wide range of applications in traffic management, infrastructure planning, signal control optimization, and emergency response. Accurate traffic flow forecasting can effectively alleviate traffic congestion, improve road utilization, reduce traffic accidents, and provide a scientific basis for urban planning and policy making. However, traffic flow variations are extremely complex and are affected by multiple factors such as time of day, weather conditions, special events (such as traffic accidents and road construction), socioeconomic activities (such as holidays and peak periods), and driving behavior. The interaction of these factors results in a high degree of uncertainty and nonlinear changes in traffic flow. Therefore, accurately predicting traffic flow in such a complex environment has become a challenging problem.

[0003] In traffic flow prediction, outliers usually refer to traffic data that deviates significantly from the normal pattern. These abnormal data may be caused by emergencies (such as traffic accidents), data collection errors, abnormal weather conditions or other unconventional factors. Traditional outlier detection methods mainly include algorithms based on statistics, distance, density and machine learning. Although traditional outlier detection methods are effective to a certain extent, they often have the limitation of rigid thresholds when dealing with complex traffic flow data, especially in high-dimensional data and fuzzy environments: traditional methods usually rely on fixed thresholds to distinguish normal data from abnormal data. However, the boundaries of traffic flow data are often not absolute, but fuzzy and dynamically changing.

[0004] In summary, the existing technology cannot effectively divide potential outliers and has low computational efficiency, resulting in inaccurate and incomplete detection results when detecting outliers in traffic flow data. Summary of the Invention

[0005] The present invention provides a traffic flow data anomaly identification method based on fuzzy outlier detection to overcome the above technical problems.

[0006] Traffic flow data anomaly recognition method based on fuzzy outlier detection includes:

[0007] S1. Obtain a traffic flow dataset, where the traffic flow dataset includes N pieces of traffic data, each piece of traffic data including a traffic volume and an average vehicle speed;

[0008] S2, build δ B based on traffic flow dataset + Tree index, δ represents the dimension of the traffic flow dataset, constructs the first boundary search algorithm and the second boundary search algorithm, the first boundary search algorithm is used to obtain the B where the xth traffic data is located + The first search range of the tree index is used to obtain the first neighbor number of the xth traffic data according to the first search range. The second boundary search algorithm is used to obtain δ-1 Bs other than the index where the xth traffic data is located. + The second search range of the tree index is used to obtain the number of second neighbors of the x-th traffic data according to the second search range;

[0009] S3, construct MBAFOM search algorithm, the MBAFOM search algorithm is used to set the limit parameters, from δ B + Obtain all leaf nodes from the tree index, calculate the number of first neighbors and the number of second neighbors of all leaf nodes according to the first boundary search algorithm and the second boundary search algorithm, and calculate the type of the leaf node according to the limited parameters and the number of first neighbors and the number of second neighbors corresponding to the leaf node, wherein the type of the leaf node includes a regular data point, a confirmed outlier, and a potential outlier;

[0010] S4. Obtain all potential outliers and calculate the fuzzy outlier degree of the potential outliers;

[0011] S5. All confirmed outliers, potential outliers, and fuzzy outlier degrees of the potential outliers are taken as abnormal traffic data.

[0012] Preferably, the calculation of the fuzzy outlier degree of the potential outlier includes constructing an outlier degree function OF p , the outlier function OF p According to formula (1),

[0013]

[0014] Among them, OF p represents the outlier function, p is the potential outlier point, r min 、r max is the minimum search radius and the maximum search radius, r is a discrete value, r∈[r min ,r max ], OF p (r) represents the outlier degree of r, N up is the upper bound of the number of neighbors of p, N low is the lower bound of the number of neighbors of p, and N is the number of data points whose distance from p is no greater than r.

[0015] Preferably, the value of the outlier function is obtained by numerically integrating formula (3):

[0016]

[0017] Among them, [r min ,r max ] is divided into m parts, each part has a distance of (r max -r min ) / m, let r be equal to r min +i*(r max -r min ) / m, i∈[0,m-1], calculate m OFs according to formula (1) p (r) and then perform cumulative summation. Represents the outlier degree calculated based on numerical integration.

[0018] Preferably, the constructing of the MBAFOM search algorithm comprises:

[0019] S21, given 4 limiting parameters r min 、r max 、k min 、k max , where r min <r max , k min <k max ,

[0020] S22, initialize three data tables to record potential outliers, determine outliers and regular data points, and use the created B + Tree index, save all leaf nodes leave into leaveList,

[0021] S23. For each leaf node,

[0022] Calculate r min and r max The number of first neighbors of the leaf node leave under the radius, the first number of neighbors includes r min The number of local neighbors corresponding to the radius N loc (r min ) and r max The number of local neighbors corresponding to the radius N loc (r max ),

[0023] Calculate r min and r max Under the radius, the number of second neighbors of the leaf node leave, the second number of neighbors includes r min The number of other neighbors corresponding to the radius N(r min ) and r max The number of other neighbors corresponding to the radius N(r max ),

[0024] According to r min 、r max 、k min 、k max With N loc (r min ), N loc (r max )、N(r min )、N(r max ) relationship to determine the type of leaf node, and store the type of leaf node in the corresponding data table,

[0025] S24. Return to S23, traverse all leaf nodes and determine the type of the leaf nodes.

[0026] Preferably, the min 、r max 、k min 、k max With N loc (r min ), N loc (r max )、N(r min )、N(r max ) relationship to determine the type of leaf node includes when N loc (r min )>k max , indicating that the leaf node is a regular data point, when N(r max ) is empty, let N(r max )=N loc (r max ), when N(r max ) <k min , indicating that the leaf node is a definite outlier. If N(r max ) is not empty and N(r max )>k min And N(r min ) is empty, it means that the leaf node is a potential outlier. If N(r min ) is not empty, judge N(r min ) is greater than k max , if so, the leaf node is a regular data point, otherwise it is a potential outlier.

[0027] The present invention provides a method for identifying anomalies in traffic flow data based on fuzzy outlier detection. First, according to the distribution of traffic flow data, multiple B + Then, according to the MBAFOM search algorithm, the search results are sorted from multiple B + The tree index is used to quickly find outliers and fuzzy outliers, which improves the detection accuracy of outliers. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0029] Figure 1 is a flow chart of the traffic flow data anomaly identification method according to the present invention;

[0030] Figure 2 is a flow chart of two parts in the embodiment of the present invention;

[0031] Figure 3 is a two-dimensional data point distribution diagram including traffic flow and average vehicle speed in the embodiment of the present invention;

[0032] Figure 4 is a segmentation diagram of five data points in the embodiment of the present invention;

[0033] Figure 5 2. It is an example diagram of the search range of two-dimensional data points in the embodiment of the present invention. DETAILED DESCRIPTION

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0035] Figure 1 Flow chart of the method of the present invention, such as Figure 1 As shown, the method of this embodiment includes:

[0036] S1. Obtain a traffic flow data set, wherein the traffic flow data set includes N pieces of traffic data, each piece of traffic data includes traffic volume and average speed.

[0037] S2, build δ B based on traffic flow dataset + Tree index, δ represents the dimension of the traffic flow dataset, constructs the first boundary search algorithm and the second boundary search algorithm, the first boundary search algorithm is used to obtain the B where the i-th traffic data is located +The first search range of the tree index is used to obtain the number of first neighbors of the i-th traffic data according to the first search range. The second boundary search algorithm is used to obtain δ-1 Bs other than the index where the i-th traffic data is located. + The second search range of the tree index is used to obtain the number of second neighbors of the i-th traffic data.

[0038] S3, construct MBAFOM search algorithm, the MBAFOM search algorithm is used to set the limit parameters, from δ B + Obtain all leaf nodes from the tree index, calculate the number of first neighbors and second neighbors of all leaf nodes according to the first boundary search algorithm and the second boundary search algorithm, calculate the type of the leaf node according to the limited parameters and the number of first neighbors and second neighbors corresponding to the leaf node, and the type of the leaf node includes regular data points, confirmed outliers and potential outliers,

[0039] S4. Obtain all potential outliers and calculate the fuzzy outlier degree of the potential outliers.

[0040] S5. All confirmed outliers, potential outliers, and fuzzy outlier degrees of the potential outliers are taken as abnormal traffic data.

[0041] Based on the above content, the present invention provides a traffic flow data anomaly identification method based on fuzzy outlier detection. First, according to the distribution of traffic flow data, multiple B + Then, according to the MBAFOM search algorithm, the B + The tree index is used to quickly find outliers and fuzzy outliers, which improves the detection accuracy of outliers in traffic flow data.

[0042] Specifically, the first part of this embodiment first gives the relevant content of fuzzy mathematics and traditional fuzzy outliers, and the second part of this embodiment gives the details of the traffic flow data anomaly identification method. The flowcharts of the two parts are as follows: Figure 2 shown.

[0043] The first part specifically includes:

[0044] Fuzzy mathematics is a mathematical tool for dealing with uncertainty and ambiguity. By introducing the concept of fuzziness, it can model and analyze fuzzy phenomena in the real world. Unlike traditional binary logic, fuzzy logic uses a membership degree to represent the degree of belonging for data located on the boundary. This degree, referred to as the outlier degree in outlier detection, can be used. Fuzzy logic addresses uncertainty and ambiguity in data by introducing fuzzy sets, enabling detection results to better reflect the fuzzy characteristics of reality. Fuzzy outlier detection can capture data points that are close to anomalies and may contain important predictive information. It can also integrate the complex interrelationships between multidimensional data to provide more comprehensive anomaly detection results for high-dimensional data. Furthermore, fuzzy outlier detection methods can improve model robustness, reduce false positives, and enhance prediction accuracy and reliability in dynamic and complex traffic environments. By incorporating fuzzy outlier detection technology, traffic flow prediction systems can better cope with complex traffic flow data, improving anomaly detection accuracy and model reliability.

[0045] The traditional definition of fuzzy outliers is:

[0046] User-given parameter r min 、r max 、k min 、k max , where r min 、r max is the minimum search radius and the maximum search radius, k min 、k max It refers to the minimum and maximum thresholds for determining outliers, satisfying the condition r min <r max , k min <k max For any point p, the following rules exist:

[0047] 1) If N(p,r min )≥k max , then p exceeds the upper bound, it must not be an outlier, and p is called a regular data point, N(p,r min ) means that the circle has p as the center and radius r min The number of data points included in the circle;

[0048] 2) If N(p,r max )≤k min , then p is lower than the lower bound, it must be an outlier, and p is called a definite outlier. N(p,r max ) means that the circle has p as the center and radius r max The number of data points included in the circle;

[0049] 3) If N(p,r min ) <kmax And N(p,r max )>k min , then p is between the upper and lower bounds and may be an outlier. This p is called a potential outlier.

[0050] like Figure 3 As shown, a two-dimensional spatial coordinate system is constructed by taking two-dimensional data points as an example. The x-axis and y-axis of the two-dimensional spatial coordinate system represent the traffic volume and average speed respectively. Under the given definition of fuzzy outliers, the data points p1, p2, and p3 are redefined. p1 and p3 meet the definition criteria of regular data points and confirmed outliers, while p2 meets the definition of potential outliers. When the radius is 0.1, the number of neighbors is 2, which is less than k. max ; When the radius is 0.2, the number of neighbors is 3, which is greater than k min Therefore, the data point p2 meets the definition criteria of a fuzzy outlier and is a potential outlier.

[0051] Based on the traditional distance-based fuzzy outlier, this embodiment redefines fuzzy outliers. Specifically, a reasonable measurement method is required for potential outliers. The purpose of this indicator is to measure data between regular data points and confirmed outliers. For any potential outlier p, there should be a quantitative standard OF. The outlier function OF(p) corresponding to p should satisfy the following properties:

[0052] 1) When N(p,r min )→k max When , OF(p)→0, that is, p tends to the regular data point;

[0053] 2) When N(p,r max )→k min When , OF(p)→1, that is, p tends to determine the outlier;

[0054] 3) When the radius r remains unchanged, the larger the number of neighbors N(p,r) within the radius, the smaller the outlier OF(p).

[0055] That is, the OF(p) value decreases monotonically as N(p,r) increases;

[0056] Therefore, the data point p is located at <r min ,r max > range, when the search radius is r, according to property 3, the size of OF(p) should change according to the number of neighbors within a certain radius r; according to properties 1 and 2, the outlier function OF(p) can be determined, which has practical significance in the range of <0,1>. Based on the range of outliers, the point is linearly scaled to obtain the value of the outlier function. <r min ,r max >The range of the number of inner neighbors.

[0057] According to Property 1, when r→r min , k→k max When the data point p tends to the regular data point, it can be stipulated that the data point p is located at <r min ,r max >Within the range, when the search radius is r, the upper bound of the number of neighbors is N up :

[0058]

[0059] According to Property 2, when r→r max , k→k min When p tends to determine the outlier, it can be stipulated that the data point p is located at <r min ,r max >Within the search radius r, there is a lower bound N on the number of neighbors. low :

[0060]

[0061] As an outlier function, it needs to be in the range of 0 to 1. To satisfy the above properties, the expression of the outlier function OF(p) is constructed. In this invention, the most basic case is adopted, that is, OF(p) is linearly related to N(p,r), so the following functional relationship can be constructed:

[0062]

[0063] When a potential outlier p is given, for any r∈[r min ,r max ], the corresponding n value can be calculated, that is, the number of data points N whose distance from p is not greater than r p (r), from formulas 1 and 2, we can see that N low and N up Related to the variable r, the outlier degree of p can be calculated using Formula 3. In other words, when the data point p is determined, Formula 2 can be transformed into a unary function of OF and r.

[0064] It can be seen that for point p, whenever a value of r is given, a corresponding outlier value OF can be estimated. p (r), and the overall outlier degree of p is OF p Should be in the value range of r [r min ,r max ]Internal function OF p The mean of (r) is written as a definite integral as follows

[0065]

[0066] In the actual calculation process, r is a discrete value, so OF p The exact value of OF cannot be obtained by calculation. The numerical integration method can be used to calculate OF p The approximate value of [r min ,r max ] is divided into m parts, each part has a distance of (r max -r min ) / m, let d be equal to r min +i*(r max -r min ) / m(i∈[0,m-1]), find the corresponding m OF p (r) value, estimated using the rectangular formula, then this OF p The mean of (r) is the OF in formula 4 p The approximate value of , the derived formula is:

[0067]

[0068] After obtaining the definition standard of fuzzy outlier degree and the quantitative standard of outlier degree, the data set can be calculated. The final query goal is to find all the confirmed outliers and potential outliers in the data set P, and calculate the outlier degree of all potential outliers. The definition of fuzzy outliers in this embodiment is a new extension based on the traditional distance outlier definition, with reference to the concept of fuzziness in fuzzy mathematics. In the traditional outlier definition, data points are simply divided into outliers and regular data points. This "black or white" judgment method often appears too arbitrary and one-sided when faced with the complexity of traffic flow data. Traffic flow data is affected by many factors, such as traffic volume, vehicle speed, weather conditions, etc., which leads to the ambiguity of its abnormal patterns. By introducing fuzzy logic, the fuzzy outlier definition can more flexibly handle the uncertainty in these data, identify those points close to abnormal states, and thus provide more accurate and comprehensive anomaly detection results.

[0069] In the second part, this embodiment provides details of the traffic flow data anomaly identification method, including:

[0070] S1. Obtain a traffic flow dataset. The dataset consists of N traffic data points, each of which includes traffic volume and average speed. Specifically, dataset P contains traffic flow data for a city's main road network at different time periods, used to study and test fuzzy outlier detection technology. The data covers indicators such as traffic volume, speed, traffic density, and weather conditions for multiple road segments, and simulates traffic flow characteristics under normal and abnormal conditions.

[0071] S2, build δ B based on traffic flow dataset +Tree index, where δ represents the dimension of the traffic flow data set, construct the first boundary search algorithm and the second boundary search algorithm. The first boundary search algorithm is used to obtain the B where the i-th traffic data is located + The first search range of the tree index, and obtain the first neighbor count of the i-th traffic data according to the first search range. The second boundary search algorithm is used to obtain δ - 1 Bs except the index where the i-th traffic data is located + The second search range of the tree index, and obtain the second neighbor count of the i-th traffic data according to the second search range

[0072] Specifically, first create an index for the data dimension of the data set P. The algorithm uses δ Bs + Tree BP0, BP1, …, BP δ-1 ,BP i is an index with the value of the i-th dimension of the data as the keyword. For any data point p in the data set P, since the dimension of the data set is δ, the attribute values of each dimension of p are p[0], p[1], …, p[δ - 1]. It is necessary to sort the values of each dimension of p in descending order. If the value p[i] of the i-th dimension is the largest, then it is stipulated that p uses p[i] as the key and the coordinates of p as the value, and stores it in BP in the form of key-value i This method can ensure that when dealing with complex traffic flow data, it can quickly and efficiently search for and identify fuzzy outliers, improving the accuracy and efficiency of anomaly detection.

[0073] Such as Figure 4 shown, take the two-dimensional data of traffic flow and average vehicle speed as an example. The two-dimensional space is divided into two parts by y = x, that is, the two cases of x > y and x < y. For the area where x > y, it usually represents traffic congestion or high-density traffic conditions. This is because in the case of high traffic flow, the distance between vehicles becomes shorter, resulting in a decrease in vehicle speed. Sudden events such as traffic accidents or road construction can also cause high traffic flow but extremely low vehicle speed, which will also appear in this area; while for the area where x > y, in this area, the traffic flow is relatively low and the average vehicle speed is relatively high. It usually represents smooth traffic and vehicles can travel at a relatively high speed. When the traffic flow is low, the distance between vehicles is large and it is not easy to cause congestion. Therefore, there are two B + tree indexes, storing BP0 that satisfies the coordinate condition x > y and BP1 that satisfies the coordinate condition x < y. Figure 4 Five two-dimensional data points p1, p2, p3, p4, p5 are given in. According to the division rule, since the data points p1, p2, p3 satisfy the coordinate condition x > y, the value of x is used as the key and the coordinates of the point are used as the value and stored in B +In the tree index BP0, since p2 and p3 satisfy the condition x < y, the value of y is used as the key, and the coordinates of the point are stored as the value in B + In the tree index BP1.

[0074] Specifically, as Figure 4 shown, taking two-dimensional data as an example, there are two B + tree indexes, storing BP0 that satisfies the coordinate condition x > y and BP1 that satisfies the coordinate condition x < y. The data points p1, p2, p3 are stored in the index BP1, while the data points p4, p5 are stored in the index BP0. The data point p1 has intersections with the two indexes BP0 and BP1 within the circular range covered by the search radius r. This means that there may be neighbor counts within both indexes for this point. Therefore, in addition to searching for the neighbor count within the local index BP1 storing the p1 point, it is also necessary to search in the other index BP0. Thus, two search boundary algorithms are designed: calculate_locate_Boundary, which is the first boundary search algorithm, and calculate_Boundary, which is the second boundary search algorithm, to respectively give the search ranges of the index storing its own data point and the search ranges of other indexes.

[0075] The main purpose of the calculate_locate_Boundary algorithm is to find the search range of this data point under its own B + tree index, and obtain the B + tree index within the corresponding dimension range to search for the neighbor count in this dimension. There are two cases when searching for data in its own index. The first case is that the entire search range is within the index range; the second case is that the search range has intersections with other indexes.

[0076] In Figure 5 , taking two-dimensional data as an example, for a certain data point p2, with the data point p2 as the center, the search range should be a circle with the search radius fr. This search range not only intersects with other indexes in addition to being within its own index. When searching for its own index range, it can be seen that the search range of p2 in its own index should be [y min , y max . For y max , it is relatively easy to calculate. Just add the value of p2 on the y-axis to the radius r; while for y min , this value needs to calculate the intersection point of the circle function with radius r and the line y = x, and the value of this point on the y-axis is the required y min .

[0077] When searching for points p1 and p3, the search ranges of p1 and p3 with radius r are both within their own B +In a tree index, there is no intersection with other ranges. Therefore, when calculating the two points, you only need to add or subtract the radius r from the coordinates of the two points to get the search range. The calculate_locate_Boundary algorithm returns the upper and lower boundaries of the local index through the above calculation.

[0078] The main purpose of the calculate_Boundary algorithm is to find the upper and lower boundaries of the data point in other dimensions. After obtaining the boundaries, the B + In the tree index, find the number of neighbors in this dimension.

[0079] exist Figure 5 In this paper, taking two-dimensional data as an example, according to B + Tree index, storing different points in different B + In the tree index, it is required to find the number of neighbors of three points p1, p2, and p3. The search range of p1 and p3 with radius r is within their own B + In the tree index, there is no intersection with other ranges, so when searching for neighbor ranges, the calculate_locate_Boundary algorithm returns the search range [y min ,y max ]. The search range of p2 with r as radius has intersections with other spaces, indicating that there are neighbors in other dimensional ranges. Therefore, in addition to returning the search range of the local index through the calculate_locate_Boundary algorithm, calculate_Boundary should also return other B + The search range under the tree index, that is, there are neighbors in the BP0 index, and the search range is determined to be [x min , x max After searching, p1 has two neighbors in the y dimension and one neighbor point in the x dimension.

[0080] The search strategy of the calculate_Boundary algorithm is to sort the data points from large to small according to their dimension values ​​and record each dimension. Taking two-dimensional data as an example, if the circular function of the data point p with a radius of r has no intersection with the line y=x, then it proves that the search range of the point is only within its own index range and there are no neighbors in other indexes; if it is high-dimensional data, it is necessary to calculate whether there is an intersection within the range of each dimension. If there is an intersection, there may be neighbors. The dimension is added to the calculation to calculate the upper and lower boundaries of different dimensions.

[0081] S3, construct MBAFOM search algorithm, the MBAFOM search algorithm is used to set the limit parameters, from δ B +Obtain all leaf nodes from the tree index, calculate the number of first neighbors and second neighbors of all leaf nodes according to the first boundary search algorithm and the second boundary search algorithm, calculate the type of the leaf node according to the limited parameters and the number of first neighbors and second neighbors corresponding to the leaf node, and the type of the leaf node includes regular data points, confirmed outliers and potential outliers,

[0082] Specifically, the construction of the MBAFOM search algorithm includes:

[0083] S21, given 4 limiting parameters r min 、r max 、k min 、k max , where r min <r max , k min <k max ,

[0084] S22, initialize three data tables to record potential outliers, determine outliers and regular data points, and use the created B + Tree index, save all leaf nodes leave into leaveList,

[0085] S23. For each leaf node,

[0086] Calculate r min and r max The number of first neighbors of the leaf node leave under the radius, the first number of neighbors includes r min The number of local neighbors corresponding to the radius N loc (r min ) and r max The number of local neighbors corresponding to the radius N loc (r max ),

[0087] Calculate r min and r max Under the radius, the number of second neighbors of the leaf node leave, the second number of neighbors includes r min The number of other neighbors corresponding to the radius N(r min ) and r max The number of other neighbors corresponding to the radius N(r max ),

[0088] According to r min 、r max 、k min 、k max With N loc (r min ), N loc (rmax )、N(r min )、N(r max ) relationship to determine the type of leaf node, and store the type of leaf node in the corresponding data table, min 、r max 、k min 、k max With N loc (r min ), N loc (r max )、N(r min )、N(r max ) relationship to determine the type of leaf node includes when N loc (r min )>k max , indicating that the leaf node is a regular data point, when N(r max ) is empty, let N(r max )=N loc (r max ), when N(r max ) <k min , indicating that the leaf node is a definite outlier. If N(r max ) is not empty and N(r max )>k min And N(r min ) is empty, it means that the leaf node is a potential outlier. If N(r min ) is not empty, judge N(r min ) is greater than k max If yes, then the leaf node is a regular data point, otherwise it is a potential outlier.

[0089] S24. Return to S23, traverse all leaf nodes and determine the type of the leaf nodes.

[0090] S4. Obtain all potential outliers and calculate the fuzzy outlier degree of the potential outliers.

[0091] Specifically, according to the definition of fuzzy outliers, there are three types of points, namely potential outliers, regular data points and definite outliers. + In the tree index, for a data point p in δ dimension, to calculate the number of all neighbors, it is actually necessary to find all B from 0 to δ-1 + Tree index. Therefore, when calculating the boundaries of each dimension, we can determine whether there are any intersections to determine the number of neighbors in that dimension. We can then perform batch filtering, retaining only the dimensions where neighbors are likely to exist, and then find the number of neighbors in those dimensions.

[0092] For the data set P, given 4 limiting parameters r min 、rmax 、k min 、k max , where r min <r max , k min <k max The goal is to find all the confirmed outliers and potential outliers in the dataset P and calculate the outlier degrees of all potential outliers. Because it is necessary to calculate the confirmed outliers and the outlier degrees of potential outliers, preliminary pruning calculations can be performed during the calculation process.

[0093] The specific implementation process of the MBAFOM search algorithm is as follows:

[0094] Initialize three data tables that record potential outliers and determine outliers and common data points. + Tree index, save all leaf nodes leave into leaveList, because the neighbors of each data must be calculated eventually.

[0095] Calculate r min and r max The number of neighbors of the data point leave under the local index under the radius. The data points are distributed in different B + In the tree index, we need to calculate all B + The number of neighbors under the tree, which naturally includes the local B + The index of the tree, first calculate the B where leave is located + The number of neighbors in the tree index. If the condition is met, there is no need to calculate other B + The number of neighbors of the tree.

[0096] In r min Under the radius, calculate the B + The search range of the tree index, B in this range + The number of leaves in the tree is the number of leaves in r min The number of neighbors of its own index in the radius N loc (r min ), the actual calculation process is B + The search of the tree, because B + All data of the tree is stored in the leaves and is ordered, so given a range, B can be calculated + The number of leaves in the tree within this range is the number of neighbors of the index. loc (r min )>k max, then the data point meets the definition of a normal data point and the point is saved in the list safe_List (normal data points).

[0097] In r max Under the radius, calculate the B + The search range of the tree index, B in this range + The number of leaves in the tree is the number of leaves in r max The number of neighbors of its own index in the radius N loc (r max ), save the number of neighbors of the data point in its own index for subsequent calculations.

[0098] Calculate r min and r max Under the radius, the data point leave is in other B + The tree index search range. max is the radius, calculated by the calculate_Boundary algorithm in other B + The search range under the tree is max_boundary; if max_boundary is empty, it means that the data point is in other B + The number of neighbors does not exist in the tree index. At this time, N(r max )=N loc (r max ), if the condition N(r max ) <k min , then the data point meets the definition of an outlier, and the point is saved in outlier_List (determined outliers).

[0099] With r min The radius is calculated according to the calculate_Boundary algorithm to calculate the search range min_boundary in other B+ trees; if min_boundary is empty, the point is saved to outlier_CoordinateList (outlier candidate point). If min_boundary is not empty, the number of neighbors in other indexes needs to be calculated. When calculating the number of neighbors, when N(r min )>k max If the condition is met, stop calculating and save the point to safe_List (regular data point), otherwise save the data point to outlier_CoordinateList (outlier candidate point).

[0100] In the above specific implementation part, only N(r min ) but did not calculate N(rmax ), so outlier_CoordinateList (outlier candidate points) contains some potential outliers and confirmed outliers, so it is necessary to calculate N(r max ), if the condition N(r max ) <k min , save the data point to the determined outliers. The remaining points are the potential outliers for which the fuzzy outlier degree needs to be calculated. The fuzzy outlier degree of each potential outlier is calculated according to Formula 4.

[0101] Overall beneficial effects:

[0102] The present invention provides a method for identifying anomalies in traffic flow data based on fuzzy outlier detection. First, according to the distribution of traffic flow data, multiple B + Then, according to the MBAFOM search algorithm, the search results are sorted from multiple B + The tree index is used to quickly find outliers and fuzzy outliers, which improves the detection accuracy of outliers.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A traffic flow data anomaly identification method based on fuzzy outlier detection, characterized in that: include: S1. Obtain a traffic flow data set, where the traffic flow data set includes N pieces of traffic data, each piece of traffic data includes a vehicle flow rate and an average vehicle speed; S2. Construct δ B based on traffic flow dataset + Tree index, δ represents the dimension of the traffic flow data set, constructs a first boundary search algorithm and a second boundary search algorithm, the first boundary search algorithm is used to obtain the B where the xth traffic data is located + The first search range of the tree index is used to obtain the number of first neighbors of the xth traffic data according to the first search range, and the second boundary search algorithm is used to obtain δ-1 Bs other than the index where the xth traffic data is located. + The second search range of the tree index, obtaining the number of second neighbors of the x-th traffic data according to the second search range; S3, constructing MBAFOM search algorithm, the MBAFOM search algorithm is used to set the limiting parameters, from δ B + Obtain all leaf nodes from the tree index, calculate the number of first neighbors and the number of second neighbors of all leaf nodes according to the first boundary search algorithm and the second boundary search algorithm, and calculate the type of the leaf node according to the limited parameters, the number of first neighbors and the number of second neighbors corresponding to the leaf node, wherein the type of the leaf node includes a regular data point, a determined outlier point, and a potential outlier point; S4, obtaining all potential outliers and calculating the fuzzy outlier degree of the potential outliers; S5. All confirmed outliers, potential outliers and fuzzy outlier degrees of the potential outliers are taken as abnormal traffic data.

2. The traffic flow data anomaly identification method based on fuzzy outlier detection according to claim 1 is characterized in that: The calculation of the fuzzy outlier degree of the potential outlier point includes constructing an outlier degree function OF p , the outlier function OF p It is expressed by formula (1): Among them, OF p represents the outlier function, p is the potential outlier point, r min 、r max is the minimum search radius and the maximum search radius, r is a discrete value, r∈[r min ,r max ],OF p (r) represents the outlier degree of r, N up is the upper bound of the number of neighbors of p, N low is the lower bound of the number of neighbors of p, and N represents the number of data points whose distance to p is no greater than r.

3. The traffic flow data anomaly identification method based on fuzzy outlier detection according to claim 2 is characterized in that: The value of the outlier function is obtained by numerically integrating formula (3): Among them, [r min ,r max ] is divided into m parts, each part has a distance of (r max -r min ) / m, let r be equal to r min +i*(r max -r min ) / m, i∈[0,m-1], calculate m OFs according to formula (1) p (r) and then perform cumulative summation. Represents the outlier degree calculated based on numerical integration.

4. The traffic flow data anomaly identification method based on fuzzy outlier detection according to claim 1 is characterized in that: The construction of MBAFOM search algorithm comprises: S21, given 4 limiting parameters r min 、r max , k min , k max , where r min <r max , k min <k max , S22, initialize three data tables that record potential outliers, determine outliers and regular data points, and use the created B + Tree index, save all leaf nodes leave into leaveList, S23. For each leaf node, Calculate r min and r max The number of first neighbors of the leaf node leave under the radius, the first number of neighbors includes min The number of local neighbors corresponding to the radius N loc (r min ) and r max The number of local neighbors corresponding to the radius N loc (r max ), Calculate r min and r max The number of second neighbors of the leaf node leave under the radius, the second number of neighbors includes min The number of other neighbors corresponding to the radius N(r min ) and r max The number of other neighbors corresponding to the radius N(r max ), According to r min 、r max , k min , k max With N loc (r min ), N loc (r max )、N(r min )、N(r max ) relationship to determine the type of leaf node, and store the type of leaf node in the corresponding data table, S24. Return to S23, traverse all leaf nodes and determine the type of the leaf nodes.

5. The method for identifying anomalies in traffic flow data based on fuzzy outlier detection according to claim 4, characterized in that: According to the min 、r max , k min , k max With N loc (r min ), N loc (r max )、N(r min )、N(r max ) The types of leaf nodes include: loc (r min )>k max , indicating that the leaf node is a regular data point, when N(r max ) is empty, let N(r max )=N loc (r max ), when N(r max ) <k min , indicating that the leaf node is a definite outlier; if N(r max ) is not empty and N(r max )>k min And N(r min ) is empty, it means that the leaf node is a potential outlier; if N(r min ) is not empty, judge N(r min ) is greater than k max , if so, the leaf node is a regular data point, otherwise it is a potential outlier.

Images