Subject Logging for Real-Time Database Access Auditing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing big data processing systems face challenges in auditing user data access due to the large number of transactions, making it difficult to track what data has been accessed and when, especially for large organizations, and lack effective methods for proactive monitoring and near real-time detection of anomalous behavior.
Innovation Solution
A system for subject logging that includes real-time logging of user activity with indexed records, allowing for efficient storage and analysis of user access patterns, including anomaly detection using machine learning, and providing searchable logs for auditing and breach detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional data processing systems are used to handle large numbers of transactions, then the system can process five million transactions per day, but it becomes difficult to audit and track what data has been accessed and when
Solution Approach 1:
The patent segments the data access tracking function by creating separate logging tables (e.g., query_logs, access_logs) that are distinct from the main data tables. Each log entry captures specific access information (user ID, timestamp, query details) independently, allowing the system to maintain high transaction processing capacity while separately tracking access patterns for auditing purposes.
Solution Approach 2:
The patent introduces logging mechanisms as intermediary components between the data processing system and the auditing function. The logging system captures access information without interfering with the main transaction processing flow, enabling both high productivity and effective monitoring through this intermediate layer.
2Reliability
If comprehensive logging of all user activities is implemented, then complete audit trails are available, but system complexity and storage requirements increase significantly
Solution Approach 1:
The patent applies local quality by implementing selective logging that captures different levels of detail for different types of access. Critical access patterns are logged with full detail while less important accesses use simplified logging, reducing overall system complexity while maintaining audit trail completeness for security-critical operations.
Solution Approach 2:
The patent implements partial logging by focusing on capturing the most essential access information (user ID, timestamp, resource accessed) rather than logging every possible detail of each transaction. This partial action approach provides sufficient audit capability without the overwhelming complexity of comprehensive logging.
3Loss of information
If detailed access information is stored for every transaction, then complete audit capability is achieved, but processing speed and system performance decrease
Solution Approach 1:
The patent uses copying by creating simplified log entries that replicate only the essential access information from each transaction. Instead of storing complete transaction data, the system creates lightweight copies containing user ID, timestamp, and resource accessed, which preserves audit information while minimizing performance impact.
Solution Approach 2:
The patent implements disposable logging by using lightweight, simple data structures for log entries that require minimal storage and processing resources. These simple log records can be quickly created and discarded or archived without burdening the main transaction processing system, maintaining both information retention and processing speed.
Data Source
AI summary
The present application discloses a method, system, and computer system for monitoring tasks with respect to information stored in a database system. The method includes receiving a request to execute a task with respect to a database, wherein the request is associated with an identifier corresponding to a user that inputs a query for the request; determining whether the task is authorized for the user; in response to a determination that the task is authorized for the user, obtaining a set of information that is to be returned for the task; determining a subset of the set of information, wherein the subset of the set of information comprises one or more parts of the set of information for which the user has access permission; and storing a record of the request to execute the task, wherein the record comprises an indication of the user, and an indication of subset of the set of information.