LSTM Web Access Anomaly Detection for Endpoint Security
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Organizational environments face challenges in detecting malicious activity on endpoint computers due to similarities in web access patterns, as malicious codes deviate from regular usage patterns, making it difficult to distinguish between legitimate and malicious web activities.
Innovation Solution
A system utilizing LSTM algorithms for anomaly detection in web access patterns, which preprocesses raw data into fixed-size sequences, calculates loss scores, classifies websites as benign or suspicious, and calculates severity scores to identify suspicious machines, while creating a similarity graph to investigate infected users and alert on malicious activity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If web access patterns are monitored to detect malicious activity, then detection capability is improved, but false positives increase due to similarities in usage patterns between legitimate and malicious activities
Solution Approach 1:
The patent segments web access patterns into distinct categories (legitimate vs. malicious) by analyzing multiple dimensions including timing, frequency, and content characteristics. This segmentation allows the system to differentiate between similar-looking patterns by examining them from multiple angles simultaneously, thereby improving detection precision while maintaining reliability.
Solution Approach 2:
The patent introduces additional analytical dimensions beyond simple pattern matching, including temporal analysis (when accesses occur), frequency analysis (how often accesses occur), and content analysis (what is being accessed). By examining patterns across multiple dimensions simultaneously, the system can distinguish malicious activities from legitimate ones even when surface-level patterns appear similar, resolving the contradiction between detection capability and false positive rate.
2Measurement precision
If anomaly detection algorithms are applied to all web access data, then malicious activity detection is improved, but computational complexity increases
Solution Approach 1:
The patent applies preliminary filtering and preprocessing to web access data before applying complex anomaly detection algorithms. By first organizing and categorizing data into structured formats with relevant features extracted in advance, the system reduces the computational burden on subsequent anomaly detection steps while maintaining detection effectiveness.
Solution Approach 2:
The patent applies different levels of analytical complexity to different types of web access patterns based on their risk characteristics. High-risk patterns receive more intensive analysis while low-risk patterns receive streamlined processing. This localized approach to quality of analysis maintains high detection precision for malicious activities while reducing overall computational complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system for detecting malicious activity on endpoint computers utilizing anomaly detection in web access patterns in organizational environments, comprising data collection and preprocessing module for collecting, aggregating and transforming received log data into temporal website sequences of length n, such that each sequence is from a specific user; an LSTM training module, for feeding each sequence, including the sequence's machine ID to a neural network for performing a training phase and predicting the next website in the sequence; an anomaly detection module that uses LSTM for feeding all the sequences into the LSTM once again, after the LSTM is trained and is able to predict the next token correctly, the LSTM provides a "probability score" representing how probable the sequence is for every sequence, such that if the probability score is above a predetermined threshold t, deciding that the sequence is suspicious; a benign URL's remover being a pre-trained classifier, for classifying each website as malicious or benign; filtering the benign website out, if a benign website appears in a malicious sequence and if most of the suspicious websites of a user are classified as benign, ignoring the user; an alerting module, for proving alerts upon detecting malicious activity.