An edge computing device traffic anomaly detection method and system based on RRCF and TAD

Abstract

The application relates to the technical field of edge computing and artificial intelligence, in particular to an edge computing device traffic anomaly detection method and system based on RRCF and TAD. final Finally, a standard deviation-based adaptive threshold is generated, the edge computing device traffic anomaly detection method and system based on RRCF and TAD provide an efficient and reliable solution for intelligent maintenance of edge devices through innovative algorithm design and actual scene verification, and the stability and resource utilization of the system are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of edge computing and artificial intelligence, specifically to a method and system for detecting abnormal traffic in edge computing devices based on RRCF and TAD. Background Technology

[0002] Currently, traditional edge computing devices are typically deployed in outdoor or field locations, with their LAN ports often left open, posing a high security risk. Malicious devices can access these LAN ports to perform network scanning and attacks on private or intranet networks, seriously threatening network and data security. Furthermore, edge computing devices are often used to collect information from PLCs (Programmable Logic Controllers) or serial devices, and their traffic typically exhibits periodic fluctuations. However, when abnormal devices connect, this normal traffic periodicity is disrupted, resulting in significant abnormal traffic fluctuations. Therefore, real-time monitoring and anomaly detection of edge computing device traffic are crucial for ensuring network security and stable operation.

[0003] Traditional traffic anomaly detection methods are mainly based on a single algorithm. Common algorithms include: (1) Statistical methods: such as mean, variance, and Z-score, which judge anomalies by statistical traffic characteristics. (2) Machine learning methods: such as Isolation Forest and Support Vector Machine (SVM), which identify anomalies by training a model. These methods usually rely on a single algorithm to analyze traffic data, attempting to identify anomalies through a single dimension or feature.

[0004] Existing traffic anomaly detection methods based on single algorithms have the following shortcomings: (1) High false alarm rate: Single algorithms are difficult to fully capture complex traffic characteristics, especially when facing high-dimensional and dynamic network traffic, which easily leads to false alarms. For example, statistical methods are sensitive to noisy data, while machine learning methods are highly dependent on the quality of training data. (2) Poor real-time performance: Traditional methods have high computational complexity when processing large-scale traffic data, which is difficult to meet the real-time detection needs of edge computing devices. For example, time series analysis methods have a large computational overhead on long-period data. (3) Insufficient robustness: Single algorithms are poorly adaptable to anomaly patterns and are difficult to cope with traffic fluctuations in complex environments. For example, machine learning methods show a significant decrease in detection performance when facing new attacks or anomaly patterns. (4) Lack of multidimensional analysis capabilities: Traditional methods usually only analyze from a single dimension (such as statistical features or time series), which cannot fully capture the multidimensional characteristics of traffic. For example, statistical methods cannot effectively handle periodic fluctuations in time series. (5) Poor adaptability to edge computing devices: Traditional methods are usually designed for centralized servers and are difficult to deploy directly on resource-constrained edge computing devices. For example, deep learning-based methods have high requirements for computing and storage resources, making them unsuitable for edge computing scenarios.

[0005] Therefore, in view of the above situation, there is an urgent need to develop a method and system for detecting abnormal traffic in edge computing devices based on RRCF and TAD, so as to overcome the shortcomings in current practical applications. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for detecting traffic anomalies in edge computing devices based on RRCF and TAD. By collecting and analyzing traffic data in real time on the edge device, a multi-dimensional anomaly detection model is constructed. The optimal detection strategy is calculated collaboratively by dual algorithms and deployed on the edge device. The device itself executes the optimal strategy to dynamically optimize the current traffic status and detect anomalies, thereby solving the problems of high false alarm rate, low real-time performance, insufficient robustness, lack of multi-dimensional analysis capabilities, and poor adaptability of edge devices in the prior art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A method for detecting traffic anomalies in edge computing devices based on RRCF and TAD includes the following steps: (1) Collect raw traffic data in real time from the network interface of the edge computing device; (2) Clean and standardize the collected raw flow data; (3) Extract multidimensional traffic features from the preprocessed data. The multidimensional traffic features include at least time dimension features and data packet quantity dimension features; (4) Input the multidimensional traffic features in parallel into the RRCF algorithm engine and the TAD algorithm engine to generate the first anomaly score respectively. Second abnormal score ; (5) Through weighted fusion formula Generate a comprehensive anomaly score, in which, These are adjustable weighting coefficients; (6) Based on the most recent A comprehensive anomaly score for each time window is used to dynamically calculate the adaptive alarm threshold. ; (7) When Real-time alarms are generated.

[0008] As a further aspect of the present invention: In step (4), the RRCF algorithm engine models the flow characteristics based on a robust random cutting forest model, and its anomaly score is obtained by calculating the structural displacement caused by the data points in the random cutting tree structure, specifically including: For the data points to be detected In the In a randomly cut tree, the displacement is defined as: ; in, For the first From the root node to the data point in the tree leaf node path, For nodes In containing The corresponding bounding box, For nodes In removal The bounding box after, The volume of the bounding box is equal to the product of the differences between the maximum and minimum values ​​in each dimension; Then, calculate the data points. The total displacement throughout the forest is used as its anomaly score: in, For data points In the The displacement caused by the trees; The score ranges from [0, +∞). The larger the value, the greater the impact of the data point on the forest structure and the higher the degree of anomaly.

[0009] As a further aspect of the present invention: in step (4), the TAD algorithm engine calculates S using the following formula. TAD : in, The index of the current time window; This represents the number of traffic data packets within the current time window. This represents the average number of traffic data packets within a historical time window. This represents the standard deviation of the number of traffic data packets within a historical time window.

[0010] As a further aspect of the present invention: the mean ) and standard deviation Through recent The number of traffic data packets per time window is calculated as follows: recent Mean of scores for each time window The calculation formula is: recent The standard deviation of the scores within a time window The calculation formula is: Where, x i The number of traffic data packets within the i-th time window.

[0011] As a further aspect of the present invention: in step (6), the adaptive alarm threshold... Calculated using the following formula: ; in, The standard deviation is denoted as .

[0012] As a further aspect of the present invention: the adjustable weighting coefficient The value range is 0.5-0.8.

[0013] As a further aspect of the present invention: the coefficient The value can be 2-3.

[0014] An anomaly detection system for edge computing device traffic anomaly detection methods based on RRCF and TAD as described above includes: Data acquisition module: Used to collect traffic data in real time from the network interface of edge computing devices; Preprocessing module: used to clean and standardize traffic data; Feature extraction module: used to extract multidimensional traffic features in terms of time and quantity dimensions; RRCF and TAD dual algorithm engines: used for parallel calculation of anomaly scores and generation of weighted comprehensive scores; Adaptive alarm module: used to dynamically adjust thresholds and trigger alarms.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention tests five datasets collected from edge devices and shows that the model's decision accuracy remains at a high level, regardless of whether it is normal traffic (Benign) or various tools (such as DoS, DDOS, Botnet). 2. Through innovative algorithm design and real-world scenario verification, this invention provides an efficient and reliable solution for the intelligent maintenance of edge devices, significantly improving system stability and resource utilization, and has important practical application significance and promotion value. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the edge computing device traffic anomaly detection method based on RRCF and TAD in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0019] Please see Figure 1 This invention provides a method and system for detecting traffic anomalies in edge computing devices based on RRCF (Randomly Cut Forest) and TAD (Time Series Detection). Through the synergy of dual algorithms and multi-dimensional feature analysis, it achieves high-precision, low-latency, and highly robust traffic anomaly detection, while optimizing resource consumption to meet the needs of edge computing scenarios and providing strong protection for network security. Specifically, it includes the following: The system architecture of this invention is as follows: Data acquisition module: Collects raw traffic data in real time from the network interface of edge computing devices.

[0020] Data preprocessing module: Cleans and standardizes the collected raw traffic data.

[0021] Traffic Feature Export Module: Extracts multidimensional features from the processed data and uses them as input to the algorithm engine.

[0022] RRCF and TAD Algorithm Engines: The core algorithm engine, which ultimately generates a comprehensive anomaly score through a weighted fusion strategy. Alarm Engine: Based on the output of the algorithm engine, it adaptively adjusts the threshold and generates real-time alarm information.

[0023] like Figure 1 As shown, this invention collects traffic data in real time through edge computing devices and extracts traffic status from the raw data. The data then enters the RRCF algorithm engine and the TAD algorithm engine, which use the Robust Random CutForest algorithm and the Time-series Anomaly Detection algorithm, respectively, to analyze the traffic characteristics and generate a comprehensive anomaly score. Finally, the alarm threshold is dynamically adjusted according to the recent score trend, and the alarm mechanism is triggered.

[0024] The core elements of traffic anomaly detection are defined as follows: 1. Set of traffic characteristics This represents a multi-dimensional feature vector extracted from the traffic data generated by edge devices during operation, covering two major dimensions: time and quantity. Time is a time series dimension; quantity is a step size of [missing information]. Number of data packets per second.

[0025] 2. The calculation formula for the RRCF engine is as follows: This invention employs the Robust Random Cut Forest (RRCF) algorithm to construct a system containing... A forest model with [number] trees. For the data points to be detected... The anomaly score is measured by co-displacement, which is obtained by measuring the degree of disturbance of data points to the randomly cut forest structure.

[0026] Data points The displacement in a single tree is defined as: along the path from the root node to the containing node. On the path of the leaf node, each bounding box is due to The sum of volume increments resulting from the existence of the anomalies; the co-displacement is the average value of the displacement throughout the forest, used as the final anomaly score.

[0027] Based on the above principles, this invention formalizes the co-displacement quantity into the following implementable calculation expression: First, calculate the data points. In the Displacement in the tree: ; in, For the first From the root node to the data point in the tree leaf node path, For nodes In containing The corresponding bounding box, For nodes In removal The bounding box after, The volume of the bounding box is equal to the product of the differences between the maximum and minimum values ​​in each dimension; Then, calculate the data points. The total displacement throughout the forest is used as its anomaly score: in, For data points In the The displacement caused by the trees; The score ranges from [0, +∞). The larger the value, the greater the impact of the data point on the forest structure and the higher the degree of anomaly.

[0028] 3. The calculation formula for TAD engine (Transition Probability) is as follows: in, The index of the current time window; This represents the number of traffic data packets within the current time window. This represents the average number of traffic data packets within a historical time window. This represents the standard deviation of the number of traffic data packets within a historical time window.

[0029] The implementation mechanism is as follows: Data Acquisition: Collects the number of traffic data packets within a time window from edge computing devices. The length of each time window can be set according to the acquisition period, such as 30 seconds, 1 minute, etc. Calculate the mean and standard deviation: Calculate the mean for the current time window based on historical data. ) and standard deviation ; Calculate the TAD value: Based on the above The formula calculates the value of the current time window. .

[0030] Example: For time window 4: Historical data: , , ; mean 4): ; Standard deviation : Calculate the TAD value: For time window 4, if the current time window x(4) = 120, then: ; 4. The formula for the comprehensive anomaly scoring (Immediate Reward) is as follows: ; Among them, the adjustable weighting coefficient Its purpose is to balance the contributions of RRCF and TAD in the overall anomaly score. The selection is adjusted based on algorithm performance, application scenario, and data characteristics.

[0031] Example: RRCF rating: TAD's rating: ;coefficient If it is 0.75; then: .

[0032] 5. Adaptive threshold based on standard deviation (Discount Factor); This invention designs an adaptive threshold formula based on standard deviation, assuming the nearest... The comprehensive anomaly score data for each time window is as follows: Threshold dynamic adjustment formula: ; in, For the most recent The average score for each time window; For the most recent The standard deviation of the scores within each time window; , where is a coefficient used to control the sensitivity of the threshold; recent Mean of scores for each time window The calculation formula is: recent The standard deviation of the scores within a time window The calculation formula is: Where, x i The number of traffic data packets within the i-th time window.

[0033] Example: Suppose the rating data for the most recent 24 hours is as follows: [0.70,0.72,0.75,0.78,0.80,0.82,0.84,0.85,0.86,0.87,0.88,0.89,0.90,0.91,0.92,0.93,0.94,0.95,0.96,0.97,0.98,0.99,1.00,1.01]; Calculate the mean : ; Calculate the standard deviation : ; Calculate threshold : .

[0034] Anomaly detection: If the score of the current time window... If 1.10 > 1.06, it is considered abnormal; if the score of the current time window is... If 1.05 < 1.06, then it is considered normal.

[0035] In summary, this invention tested the states of five datasets collected from edge devices, and the resulting validation data is shown in the table below: Among the aforementioned verification data, the accuracy rate of the Webattack dataset is relatively low. This is because web attack traffic is highly diverse and covert, and some attack payloads are very similar to normal business traffic in terms of traffic statistics, making it difficult for a single detection method to identify this type of attack. However, the dual-algorithm collaborative mechanism of this invention still achieved an accuracy rate of 0.68, which is better than traditional single-algorithm methods, and still has important early warning value in practical edge computing scenarios.

[0036] As can be seen, the model's decision accuracy remains at a high level regardless of normal traffic (Benign) or various tools (such as DoS, DDOS, Botnet).

[0037] This invention relates to a method and system for detecting traffic anomalies in edge computing devices based on RRCF and TAD. First, traffic data is collected from the edge computing device; then, the data is preprocessed and features are extracted; finally, a comprehensive anomaly score is calculated. Ultimately, an adaptive threshold based on standard deviation is generated. Therefore, through innovative algorithm design and practical scenario verification, this invention provides an efficient and reliable solution for the intelligent maintenance of edge devices, significantly improving system stability and resource utilization, and has important practical application significance and promotion value.

[0038] It should be noted that, in this invention, although the specification describes the embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for detecting abnormal traffic flow in edge computing devices based on RRCF and TAD, characterized in that, Includes the following steps: (1) Collect raw traffic data in real time from the network interface of the edge computing device; (2) Clean and standardize the collected raw flow data; (3) Extract multidimensional traffic features from the preprocessed data. The multidimensional traffic features include at least time dimension features and data packet quantity dimension features; (4) Input the multidimensional traffic features in parallel into the RRCF algorithm engine and the TAD algorithm engine to generate the first anomaly score respectively. Second abnormal score ; (5) Through weighted fusion formula Generate a comprehensive anomaly score, in which, These are adjustable weighting coefficients; (6) Based on the most recent A comprehensive anomaly score for each time window is used to dynamically calculate the adaptive alarm threshold. ; (7) When Real-time alarms are generated.

2. The method for detecting abnormal traffic flow in edge computing devices based on RRCF and TAD according to claim 1, characterized in that, In step (4), the RRCF algorithm engine models the flow characteristics based on a robust random cut forest model. Its anomaly score is obtained by calculating the structural displacement caused by data points in the random cut tree structure, specifically including: For the data points to be detected In the In a randomly cut tree, the displacement is defined as: in, For the first From the root node to the data point in the tree leaf node path, For nodes In containing The corresponding bounding box, For nodes In removal The bounding box after, The volume of the bounding box is equal to the product of the differences between the maximum and minimum values ​​in each dimension; Then, calculate the data points. The total displacement throughout the forest is used as its anomaly score: in, For data points In the The displacement caused by the trees; The score ranges from [0, +∞). The larger the value, the greater the impact of the data point on the forest structure and the higher the degree of anomaly.

3. The method for detecting abnormal traffic flow in edge computing devices based on RRCF and TAD according to claim 1, characterized in that, In step (4), the TAD algorithm engine calculates using the following formula. : in, The index of the current time window; This represents the number of traffic data packets within the current time window. This represents the average number of traffic data packets within a historical time window. This represents the standard deviation of the number of traffic data packets within a historical time window.

4. The method for detecting abnormal traffic flow in edge computing devices based on RRCF and TAD according to claim 3, characterized in that, The mean ) and standard deviation Through recent The number of traffic data packets per time window is calculated as follows: recent Mean of scores for each time window The calculation formula is: recent The standard deviation of the scores within a time window The calculation formula is: Where, x i The number of traffic data packets within the i-th time window.

5. The method for detecting abnormal traffic flow in edge computing devices based on RRCF and TAD according to claim 4, characterized in that, In step (6), the adaptive alarm threshold Calculated using the following formula: ； in, The standard deviation is denoted as .

6. The method for detecting abnormal traffic flow in edge computing devices based on RRCF and TAD according to claim 1, characterized in that, The weighting coefficient The value range is 0.5-0.

8.

7. The method for detecting abnormal traffic flow in edge computing devices based on RRCF and TAD according to claim 5, characterized in that, The coefficient The value can be 2-3.

8. An anomaly detection system for an edge computing device traffic anomaly detection method based on RRCF and TAD as described in any one of claims 1-7, characterized in that, include: Data acquisition module: Used to collect traffic data in real time from the network interface of edge computing devices; Preprocessing module: used to clean and standardize traffic data; Feature extraction module: used to extract multidimensional traffic features in terms of time and quantity dimensions; RRCF and TAD dual algorithm engines: used for parallel calculation of anomaly scores and generation of weighted comprehensive scores; Adaptive alarm module: used to dynamically adjust thresholds and trigger alarms.

Images