Group-Based Row-Level Security for Big Data Platforms
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Big data platforms face challenges in efficiently retrieving and filtering large datasets due to slow whole data set scanning and tag matching processes, especially when dealing with numerous security tags, which hinders fast data retrieval and access control.
Innovation Solution
Implementing a group-based row-level security mechanism that organizes data in columnar format with bit map tags for faster filtering, using bit operations to determine access rights, and storing data in a compressed form to reduce I/O volume and improve query performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional string-based security tag matching is used, then data security is maintained, but data retrieval speed deteriorates due to slow whole data set scanning
Solution Approach 1:
The patent segments the security tag matching process by introducing bit map representations of user groups and security tags. Instead of scanning and comparing entire data sets with string-based methods, the system divides the data into manageable units with associated bit map tags, enabling parallel processing and rapid filtering of authorized data records.
Solution Approach 2:
The patent replaces the mechanical string-based comparison system with a bitwise operation system. By converting security tags and group memberships into bit map representations, the system substitutes slow string matching with fast bitwise AND operations, achieving significant performance improvements in data retrieval speed.
2Productivity
If bit map tags are used for security filtering, then data retrieval speed improves, but system complexity increases due to additional data structures
Solution Approach 1:
The patent applies universality by designing the bit map data structure to serve multiple functions simultaneously. The same bit map tags that enable fast security filtering also support group management, user authorization, and data organization. This multi-functionality reduces the need for separate complex mechanisms while maintaining high query performance.
3Measurement precision
If data is stored in columnar format with security tags, then access control precision is improved, but storage overhead increases
Solution Approach 1:
The patent changes the parameter representation from string-based security tags to compact bit map integers. This parameter transformation reduces the storage space required for security tags while maintaining precise access control. The bit map representation converts potentially long string identifiers into compact binary forms, significantly reducing storage overhead while preserving the ability to precisely control data access.
Data Source
AI summary
Methods, apparatus, systems and articles of manufacture are disclosed to facilitate electronic data security. An example apparatus includes a data storage including a memory adjusted to store data organized according to a data table including columns identifying a first data record and a first security tag associated with the first data record. In the example apparatus, retrieval of data from the data storage involves a bit operation comparing the first security tag with a first privilege tag. In the example apparatus, the data storage provides the first data record when the bit operation comparing the first security tag with the first privilege tag has a non-zero result, and the data storage does not provide the first data record when the bit operation comparing the first security tag with the first privilege tag has a zero result.