Hybrid Sharing Model Query Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In business applications like CRM, efficiently managing data access requests for millions of records with complex policies and rules is challenging due to the need for rapid evaluation of security permissions at multiple levels, leading to inefficiencies in data access operations.
Innovation Solution
Implementing a system where sharing rules associated with users are evaluated in advance and at the time of data access requests, allowing for efficient processing by determining shared records granted by both materialized and non-materialized sharing rules based on user visibility statistics, optimizing the execution order of sub-queries to enhance performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If security applications evaluate policies and rules for each data access request in real-time, then data access security is ensured, but processing time and computational overhead increase significantly
Solution Approach 1:
The system pre-evaluates sharing rules and determines shared records for users before actual data access requests are processed. Materialized sharing rules are evaluated in advance and stored, so when a data access request comes in, the security application can quickly retrieve pre-computed results instead of re-evaluating all rules from scratch, significantly reducing processing time while maintaining security
Solution Approach 2:
The evaluation process is divided into two distinct segments: materialized sharing rules (evaluated in advance and stored) and non-materialized sharing rules (evaluated at request time). This segmentation allows the system to handle the bulk of security evaluations beforehand, reserving only necessary dynamic evaluations for actual access requests, thereby balancing security with performance
2Reliability
If security applications perform operations on millions of records with complex policies, then comprehensive security coverage is achieved, but operational efficiency decreases
Solution Approach 1:
Materialized sharing rules are evaluated in advance across all relevant records, and the results are stored for rapid retrieval. This preliminary evaluation covers millions of records once, rather than repeatedly evaluating the same complex rules for every access request, maintaining comprehensive security coverage while dramatically improving operational efficiency
Solution Approach 2:
The system merges the results of materialized sharing rule evaluations with non-materialized sharing rule evaluations to produce the final set of shared records. This combination approach allows the system to leverage pre-computed results while still applying necessary dynamic security checks, achieving both comprehensive coverage and high efficiency
Data Source
AI summary
Systems and methods for processing requests for shared records are described. A server computing system receives a data access request associated with a user. The server determines shared records granted by a first sharing rule associated with the user in response to receiving a data access request. The server processes the data access request based on the shared records granted by the first sharing rule and shared records granted by a second sharing rule associated with the user. The shared records granted by the second sharing rule having been determined prior to receiving the data access request, the first sharing rule and the second sharing rule generated prior to receiving the data access request.