Data Replication for Granular Table Permissioning and Query Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data querying systems face inefficiencies when enforcing granular permissioning on large data tables, requiring access controls for each individual cell, which hinders query performance and security.
Innovation Solution
The system employs data table replication and a query writing service to generate queries based on sensitivity level permissions, allowing users to access specific portions of the data without needing permissioning at the individual cell level, using a replicated table data layer that integrates with existing search services.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If granular permissioning is enforced at the individual cell level, then data security is improved, but query performance deteriorates
Solution Approach 1:
The patent segments the data table into multiple partitions, each with its own permission set. Instead of enforcing permissions at the individual cell level across the entire table, the system divides the table into manageable segments (partitions) and applies permissioning at the partition level. This segmentation reduces the complexity of permission checks during queries, improving performance while maintaining security through controlled access to specific data segments.
Solution Approach 2:
The patent introduces an intermediary permissioning layer that sits between the query execution engine and the underlying data cells. This intermediary mechanism evaluates permissions at the partition level before allowing access to individual cells, avoiding the need for repeated cell-level permission checks. The intermediary layer caches and manages permission information, reducing the overhead during query execution while maintaining granular security controls.
2Manufacturing precision
If detailed cell-level permissioning is implemented, then access control precision is improved, but system complexity increases
Solution Approach 1:
The patent segments the complex permissioning task into manageable units by organizing data into partitions with associated permission sets. Each partition contains a subset of data cells with related permission requirements. This segmentation reduces system complexity by breaking down the monolithic cell-level permissioning problem into smaller, more manageable partition-level permissioning tasks, while still achieving detailed access control precision within each partition.
Solution Approach 2:
The patent introduces a new dimension of organization by adding partition identifiers and permission set identifiers as additional indexing dimensions. Instead of managing permissions solely at the cell level (one dimension), the system adds partition-level organization (second dimension), creating a hierarchical structure that simplifies permission management. This dimensional change allows the system to achieve detailed access control without the full complexity of flat cell-level permissioning across the entire table.
3Reliability
If permissioning is applied to large data tables, then data security is improved, but querying efficiency deteriorates
Solution Approach 1:
The patent divides large data tables into smaller partitions, each with its own permission set and metadata. This segmentation allows the query execution engine to quickly identify and access only the relevant partitions based on user permissions, avoiding the need to scan or check permissions across the entire large table. The segmentation improves querying efficiency by reducing the search space while maintaining data security through partition-level access control.
Solution Approach 2:
The patent performs preliminary organization of data into permissioned partitions before query execution. Permission sets and partition metadata are pre-configured and cached, allowing the system to quickly evaluate user permissions and identify accessible partitions without performing complex permission checks during query execution. This preliminary action reduces the overhead during querying while maintaining robust security controls across large data tables.
Data Source
Figure 1A~1B
Figure 2
Figure 3A~3B
AI summary
A data querying system is disclosed that provides improved computer functionality that enables efficient permissioning and querying of specific portions of a data table, such that users (e.g., based on user roles or user attributes) are only allowed access to specific portions (e.g., particular data items / rows, and particular data items attributes / columns) of the data. The system advantageously provides efficient and improved querying and permissioning of specific portions of a data table through replication of the data table, or portions of the data table, and does not require permissioning of each individual cell of the data table. Further, the data table replication, querying, and permissioning techniques of the present disclosure, according to various implementations, advantageously integrate with a wide variety of data table query or search services to provide improved functionality, efficiency, and data permissioning.