ML Query Vector Analysis for Database Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer security systems are inadequate in detecting and mitigating insider threats, particularly when user accounts with database access are compromised, leading to potential data breaches and loss.
Innovation Solution
A machine learning and artificial intelligence-based system that profiles normal user behavior by mapping database queries to a vector space, allowing for the identification of anomalies that may indicate compromised accounts or data breaches, and triggers remedial actions such as increased logging or account locking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional security systems are used to monitor database access, then system simplicity is maintained, but detection precision of insider threats and compromised accounts is insufficient
Solution Approach 1:
The patent replaces traditional rule-based security monitoring systems with a machine learning-based anomaly detection system. The ML model automatically learns normal user behavior patterns from historical query data and identifies deviations, substituting manual security rule configuration and interpretation with automated intelligent analysis, thereby improving detection precision without requiring proportional increases in operational complexity
Solution Approach 2:
The system transforms security monitoring from static rule-based parameters to dynamic behavioral parameters. By analyzing multiple query parameters simultaneously (query structure, data access patterns, timing, frequency) and using ML to learn their normal relationships, the system achieves higher detection precision. The complexity is managed through automated feature engineering and model training rather than manual parameter tuning
2Reliability
If comprehensive user behavior profiling is implemented to detect anomalies, then detection capability is improved, but processing time and computational resources increase
Solution Approach 1:
The system performs preliminary action by pre-training the machine learning model on extensive historical query data during an offline training phase. This pre-processing creates a robust anomaly detection model that can quickly evaluate new queries in real-time. The computationally intensive behavior profiling and pattern learning are done in advance, allowing fast online detection without processing delays
Solution Approach 2:
The system implements partial monitoring by focusing computational resources on the most critical anomaly detection tasks. Rather than analyzing every single query parameter in equal depth, the ML model identifies and focuses on the most discriminative features for detecting compromised accounts and insider threats, achieving high reliability with optimized processing time
3Speed
If automated remedial actions are triggered based on anomaly detection, then response speed is improved, but risk of false positives and unnecessary disruptions increases
Solution Approach 1:
The system implements feedback mechanisms where security analysts can review detected anomalies and provide feedback on false positives or confirmed threats. This feedback is used to continuously refine and retrain the machine learning model, improving its ability to distinguish true threats from normal variations in user behavior. The feedback loop reduces false positives over time while maintaining rapid automated response capabilities for confirmed anomalies
Solution Approach 2:
The remedial action system is dynamic and adaptive rather than static. The machine learning model continuously learns from new data and adjusting its detection thresholds and anomaly criteria. This allows the system to maintain high response speed while adapting to legitimate changes in user behavior patterns, reducing false positives as the model becomes more sophisticated
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
Within an organization, numerous different persons can access data. But a user account with database access may be compromised, leading to data theft and data destruction. Database queries used to access data may vary in length, content, and formatting. Features of these queries can be extracted to train a machine learning classifier. Queries for users can be mapped to a vector space and when a new sample query is received, it can be assessed using the classifier to determine its level of similarity with previous queries by that user and other users. By analyzing the results of this assessment on the new query, it can be determined if this new query represents a data access anomaly-e.g. a particularly unusual query for a user, given his or her past, that may indicate user credentials have been compromised. When a data access anomaly exists, a remedial action may be taken.