MapReduce Data Segmentation for Secure Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data warehouses face challenges in securely managing and accessing private data due to the need for role-based security and protection of sensitive information, as existing solutions do not adequately isolate and protect private data from unauthorized access.
Innovation Solution
The MapReduce framework identifies private data, encrypts it, and stores it separately, using a reference to the encrypted data within the original data, allowing for secure access through a system that verifies user authorization and decrypts the data based on permissions, ensuring that only authorized users can access the private information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If private data is stored together with received data in the data warehouse, then data access and processing are simplified, but data security and protection from unauthorized access are compromised
Solution Approach 1:
The patent segments private data from received data by storing encrypted private data in a separate location within the data warehouse, while maintaining references in the original data structures. This allows the system to maintain simplified access procedures while ensuring security through physical separation of sensitive information.
Solution Approach 2:
The patent introduces encrypted private data as an intermediary between the original private data and storage. By encrypting private data and storing it separately with references, the system mediates between the need for simple access and the requirement for security, allowing authorized access while protecting against unauthorized exposure.
2Reliability
If private data is encrypted and stored separately, then data security is improved, but system complexity and processing overhead increase
Solution Approach 1:
The patent applies preliminary action by encrypting private data before storage and pre-computing references to the encrypted data locations. This preliminary processing reduces the complexity during data access operations, as the encryption and reference creation are performed once during data ingestion rather than repeatedly during access operations.
Solution Approach 2:
The patent uses copying by creating reference copies that point to the encrypted private data locations. Instead of moving or duplicating the actual encrypted data, the system creates lightweight reference structures that simplify access while maintaining security, reducing the overall system complexity.
3Productivity
If all data is processed in parallel using Map Reduce, then processing speed is improved, but security control and authorization verification become more difficult
Solution Approach 1:
The patent applies local quality by implementing different processing characteristics for different data types. Received data can be processed in parallel using Map Reduce for high productivity, while private data locations are marked with special properties that trigger security verification procedures. This allows the system to maintain parallel processing efficiency while enforcing security controls where needed.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for securing data. One of the methods includes receiving, by the map reduce framework, data for analysis. The method includes identifying, by the map reduce framework, private data in received data. The method includes encrypting the private data. The method includes storing the encrypted private data in a location separate from the received data. The method includes obfuscating the private data by adding a reference to the location of the encrypted private data in the received data.