Query-Based Database Redaction Without Data Export
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems often require exporting sensitive data for redaction, which exposes it to potential compromise and complicates data governance, and existing mechanisms for in-database redaction can be complex and resource-intensive.
Innovation Solution
A system that performs redaction of sensitive data within the database using SQL-based user-defined functions (UDFs) and machine learning, allowing in-situ analysis without data export, using cloud computing platforms like AWS, Azure, or Google Cloud to identify and mask sensitive information using standard database queries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If sensitive data is exported from the database for redaction, then redaction can be performed, but data exposure risk increases and data governance becomes complex
Solution Approach 1:
The patent introduces an intermediary redaction layer that sits between the database and users. This layer intercepts query results, identifies sensitive data using regex patterns and machine learning, and redacts it before returning data to users. The sensitive data never leaves the database, eliminating export risks while maintaining governance through centralized redaction control.
Solution Approach 2:
The database system performs redaction automatically through integrated machine learning models and regex patterns without requiring manual data export or external processing. The system self-identifies sensitive information and applies redaction rules autonomously, simplifying governance while maintaining security.
2Reliability
If existing in-database redaction mechanisms are used, then data can be redacted without export, but the system becomes complex and resource-intensive
Solution Approach 1:
The redaction system is segmented into distinct components: regex pattern matching for structured data, machine learning models for unstructured data, and configurable redaction rules. Each component handles specific types of sensitive information independently, reducing overall system complexity while maintaining comprehensive redaction capability.
Solution Approach 2:
The system uses configurable parameters including sensitivity thresholds, redaction patterns, and machine learning confidence levels. These parameters can be adjusted to balance redaction thoroughness with system resource consumption, allowing optimization based on specific organizational needs without fundamentally changing system architecture.
3Reliability
If comprehensive rediction of all sensitive data is performed, then data protection is maximized, but processing time and computational resources increase
Solution Approach 1:
The system applies redaction selectively based on configurable sensitivity thresholds and data types. Rather than redacting all possible sensitive information uniformly, it focuses computational resources on high-priority sensitive data identified through regex patterns and machine learning, achieving adequate protection with reduced processing overhead.
Solution Approach 2:
The system pre-loads and caches regex patterns and machine learning models before data processing begins. This preliminary preparation reduces computational overhead during actual redaction operations, improving processing efficiency while maintaining comprehensive protection capabilities.
Data Source
AI summary
Embodiments of the present disclosure describe systems, methods, and computer program products for redacting sensitive data within a database. An example method can include receiving a data query referencing unredacted data of a database, wherein the data query that is received comprises a value identifying a type of sensitive data to be redacted from the unredacted data, responsive to the data query, executing, by a processing device, a redaction operation to identify candidate sensitive data that matches the type of sensitive data to be redacted within the unredacted data of the database, and returning a redacted data set in which the candidate sensitive data that is provided is based on an authentication level utilized for execution of the redaction operation.