Data Partitioning via Authorization Rules
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage solutions lack effective partitioning and security measures, leading to inefficiencies in query times and potential security breaches by requiring users to access entire datasets, wasting computing resources and risking unauthorized data access.
Innovation Solution
Implementing a method to partition data based on authorization rules by initiating data nodes corresponding to specific user permissions, using an ETL application service to extract, transform, and load data only to authorized nodes, thereby improving security and reducing query times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If users access entire datasets to ensure data availability, then data accessibility is improved, but security deteriorates due to potential unauthorized access
Solution Approach 1:
The patent segments the entire dataset into multiple partitioned datasets based on authorization rules. Each user or application receives access only to their authorized partition rather than the complete dataset. This segmentation maintains data accessibility for authorized users while inherently improving security by limiting exposure to only necessary data portions.
2Quantity of substance
If users access entire datasets to ensure complete data availability, then data completeness is improved, but computational overhead increases due to processing unnecessary data
Solution Approach 1:
The patent extracts and isolates only the necessary data subsets that each user or application is authorized to access. By extracting relevant data from the complete dataset and presenting only those portions, the system maintains data completeness for authorized operations while eliminating computational waste from processing unauthorized or unnecessary data.
3Reliability
If data is partitioned at the query stage to improve security, then data security is improved, but query time increases due to later partitioning operations
Solution Approach 1:
The patent performs data partitioning in advance at the ETL (Extract, Transform, Load) stage rather than at query time. By pre-partitioning data according to authorization rules before queries are executed, the system eliminates runtime partitioning overhead. This preliminary action ensures both security and optimal query performance, as queries operate on already-segmented data without additional partitioning delays.
4Ease of operation
If entire datasets are stored and accessed, then data availability is improved, but storage efficiency deteriorates due to redundant data storage
Solution Approach 1:
The patent segments the monolithic dataset into multiple specialized partitions, each stored separately and optimized for specific authorization contexts. This segmentation eliminates the need to store and maintain redundant copies of entire datasets for different users, as each partition contains only the data necessary for its authorized users. The result is improved storage efficiency while maintaining data availability for each user group.
Data Source
AI summary
A system and method for partitioning data based on authorization rules. The method includes initiating a plurality of data nodes, each data node including a dataset generated based on a shared data model, wherein each of the plurality of data nodes corresponds to at least one authorization rule, wherein each authorization rule authorizes at least one user identifier to access a respective data node of the plurality of data nodes; and configuring an extract, transform, and load (ETL) application service to extract data from a data source and to load data from the data source to the dataset stored on a first data node of the plurality of data nodes when the at least one first authorization rule authorizes a requesting user to access the first data node.