Bot Traffic Detection Using Dynamic Clean Traffic Distributions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Detecting bot traffic in network traffic is challenging due to the absence of labeled training data and the non-stationarity of attack traffic, making it difficult for supervised machine learning models to keep up with constantly changing bot activities, and unsupervised methods require parameter tuning based on network characteristics.
Innovation Solution
An unsupervised machine learning approach that identifies clean distributions of human traffic using conditional independence relationships to detect bot traffic by quantizing feature values and identifying clean buckets, allowing for real-time detection with low latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If supervised machine learning models are used to detect bot traffic, then detection accuracy may be improved with labeled training data, but the models cannot keep up with constantly changing bot activities due to non-stationarity of attack traffic
Solution Approach 1:
Instead of trying to model bot traffic directly, the patent inverts the approach by modeling clean human traffic and using that as a reference. The system identifies deviations from human behavior patterns to detect bots, rather than attempting to classify bot behavior directly. This inversion allows the system to adapt to new bot types without retraining, as long as they deviate from human patterns.
Solution Approach 2:
The system automatically updates its understanding of human traffic patterns over time without requiring external retraining or parameter adjustment. By continuously learning from incoming human traffic and maintaining dynamic baselines, the system serves itself by adapting to changes in legitimate user behavior while detecting anomalies that indicate bot activity.
2Adaptability or versatility
If unsupervised machine learning methods are used to detect bot traffic, then adaptability to changing traffic patterns is improved, but parameter tuning is required based on network characteristics
Solution Approach 1:
The system automatically determines optimal parameter values by analyzing the actual traffic data. It computes statistics from observed human traffic to set thresholds and parameters dynamically, eliminating the need for manual tuning. The system serves itself by deriving its own configuration from the data it processes.
Solution Approach 2:
The patent dynamically adjusts detection parameters based on the statistical properties of observed traffic. Rather than using fixed thresholds, the system adapts parameters like mean and standard deviation of traffic features based on current human behavior patterns, allowing automatic adaptation to different network environments without manual intervention.
3Measurement precision
If traditional bot detection methods are used, then some bot traffic can be detected, but false alarms are generated against intended human users
Solution Approach 1:
The patent applies different detection strategies to different segments of traffic based on their characteristics. By identifying clean human traffic segments and using them to establish local baselines, the system tailors detection sensitivity to specific traffic patterns. This localized approach reduces false alarms by adapting detection thresholds to the specific context of each traffic segment rather than using uniform thresholds.
Solution Approach 2:
The system continuously monitors traffic patterns and uses feedback from observed human behavior to adjust its detection model. By comparing actual traffic against dynamically updated baselines of human behavior, the system can distinguish between legitimate variations in human traffic and actual bot activity, reducing false positives while maintaining detection effectiveness.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A bot traffic detection system detects scripted network traffic. The bot traffic detection system may use a one-sided unsupervised machine learning technique to estimate distributions for human, non-scripted traffic (clean distributions). The clean distributions may be dynamically updated based on the latest traffic patterns. To estimate the clean distributions the bot traffic detection system may identify, for a certain subset of network traffic, feature values of the certain subset of network traffic that do not include bot traffic (clean buckets). Using clean traffic may provide more robust and stable behavior that can be tracked over time. Using the clean distributions, the bot traffic detection system may generate a rules table that indicates a likelihood that network traffic with a given combination of feature values is scripted network traffic. The bot traffic detection system may apply the rules table in real time to identify scripted network traffic.