Sensitive Data Write Detection via Log Analysis

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

In large distributed computing systems, tracking the storage and access of sensitive data across multiple users, services, and storage locations is challenging, leading to potential data security breaches due to resource-intensive searches and increased complexity.

Innovation Solution

Implementing data monitoring techniques through software plugins, API controls, and logging mechanisms to track sensitive data access, usage, and storage, combined with policy and pattern matching systems to identify unauthorized access and storage, and utilizing a rules engine to analyze logs and enforce data access rules.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If comprehensive searches are conducted across all potential data storage locations to identify improperly stored sensitive data, then data security detection capability is improved, but system resources are excessively consumed and search time increases

Engineering Contradiction:
Improvedata security detection capabilityVSAvoidsystem resource consumption
Core Design Contradiction:
ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent implements preliminary action by establishing data loss prevention policies and instrumentation before sensitive data is written to storage locations. The system pre-configures monitoring rules, defines sensitive data patterns, and sets up logging mechanisms in advance, so that when data access or write operations occur, the pre-established policies can immediately evaluate and detect potential security issues without requiring comprehensive post-hoc searches across all storage locations.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent applies segmentation by dividing the monitoring system into separate components: policy definition modules, instrumentation modules that intercept specific data operations, logging modules that record only relevant events, and analysis modules that evaluate logs against policies. This segmentation allows the system to focus computational resources on evaluating specific data access events against predefined policies rather than conducting exhaustive searches across entire storage systems.

Inventive Principle:
Principle #1Segmentation

2Reliability

If comprehensive searches are conducted across all potential data storage locations to identify improperly stored sensitive data, then data security detection capability is improved, but search time increases

Engineering Contradiction:
Improvedata security detection capabilityVSAvoidsearch time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary action by pre-configuring monitoring policies and instrumentation before data operations occur. This allows real-time or near-real-time detection of sensitive data writes as they happen, eliminating the need for time-consuming retrospective searches across storage locations. The policies are established in advance with defined patterns for sensitive data, enabling immediate evaluation when data is accessed or written.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary logging and analysis layer between data operations and security evaluation. Instead of directly searching storage locations, the system uses instrumentation that intercepts data operations and logs them, then analyzes the logs against policies. This intermediary approach significantly reduces search time by evaluating data operations as they occur rather than retrospectively scanning entire storage systems.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Measurement precision

If extensive searches are performed across complex systems to locate improperly stored data, then detection thoroughness is improved, but error rates increase due to the extensive search space

Engineering Contradiction:
Improvedetection thoroughnessVSAvoidlocating accuracy
Core Design Contradiction:
Measurement precisionVSReliability

Solution Approach 1:

The patent applies local quality by making the monitoring system sensitive to specific local conditions - predefined policies that define what constitutes sensitive data and where it should not be stored. The instrumentation and log analysis are configured with local knowledge about data classification, storage policies, and compliance requirements, enabling precise detection of violations without the errors associated with blanket searches across entire systems.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The system implements feedback through its log analysis capability, which continuously evaluates logged data operations against predefined policies and provides feedback about potential security violations. This feedback mechanism enables the system to learn from detected patterns and adjust monitoring focus, improving detection accuracy over time while reducing false positives that would occur in exhaustive search approaches.

Inventive Principle:
Principle #23Feedback

4Reliability

If resource-intensive searches are conducted to identify improperly stored sensitive data, then data security monitoring is improved, but user experience deteriorates due to diverted computing resources

Engineering Contradiction:
Improvedata security monitoringVSAvoiduser experience
Core Design Contradiction:
ReliabilityVSEase of operation

Solution Approach 1:

The system performs preliminary action by establishing monitoring policies and instrumentation in advance, enabling lightweight real-time evaluation of data operations as they occur. This approach requires minimal computational resources during normal operations compared to exhaustive searches, maintaining system performance and user experience while providing continuous security monitoring. The heavy lifting of policy definition and pattern matching is done in advance, not during data operations.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS10114960B1Identifying sensitive data writes to data stores
Publication Date: 2018.10.30 AMAZON TECH INC
  • US10114960B1 patent drawing
  • US10114960B1 patent drawing
  • US10114960B1 patent drawing

AI summary

Techniques for detecting access to computer system data by applications running on a computer system are described herein. Data access event log entries are recorded, the log entries including one or more metadata items associated with how the computer system application accessed the computer system data. The log entries are analyzed using correlations with other computer system events and, if improper access is detected, one or more operations relating to the type of data accessed and the type of violation are performed to mitigate the improper data access.