Query Reconstruction Processor for Distributed Storage Policy Enforcement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data access query languages in distributed storage and processing environments, such as Apache Hadoop, Apache Hive, and Apache Spark, are not easily broken down into components for re-engineering or policy enforcement, making it difficult to parse and reconstruct queries across different access methods.
Innovation Solution
A query reconstruction processor that tokenizes query statements, categorizes and modifies data elements while preserving computational and control logic, to enforce access control and policy compliance, utilizing lexical, translation, and execution algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If query languages are designed to work with distributed storage and processing resources, then data access capability is improved, but the ability to parse and reconstruct queries for policy enforcement deteriorates
Solution Approach 1:
The query statement is segmented into discrete tokens representing different components (data elements, computational logic, control logic). This tokenization allows the system to process and reconstruct queries by manipulating individual tokens rather than treating the entire query as an indivisible string, thereby enabling policy enforcement while maintaining data access capability
Solution Approach 2:
The patent introduces an intermediary processing layer that receives queries, tokenizes them, applies policy rules to modify specific tokens, and reconstructs the query. This intermediary mechanism enables policy enforcement without requiring fundamental changes to the underlying query languages or distributed processing systems
2Adaptability or versatility
If query languages are designed to be domain-specific (HiveQL, Spark SQL), then specialized data access functionality is improved, but ease of parsing and reconstruction across different access methods deteriorates
Solution Approach 1:
The patent creates a universal tokenization framework that can process queries from multiple domain-specific languages (HiveQL, Spark SQL, and others) using the same token types and processing mechanisms. This universal approach allows the system to handle specialized data access functionality while maintaining ease of parsing and reconstruction across different access methods through a common processing interface
3Adaptability or versatility
If query languages behave like set theory operations, then data manipulation capability is improved, but ease of breaking down into components for re-engineering deteriorates
Solution Approach 1:
The patent segments set theory operations into discrete token types that represent specific operations (selection, projection, join, aggregation). By breaking down abstract set operations into concrete token components, the system maintains data manipulation capability while enabling easier re-engineering through structured token manipulation and transformation
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A non-transitory computer readable storage medium has instructions executed by a processor to receive a query statement. The query statement is one of many distributed storage and distributed processing query statements with unique data access methods. Token components are formed from the query statement. The token components are categorized as data components or logic components. Modified token components are formed from the token components in accordance with a policy. The query statement is reconstructed with the modified token components and original computational logic and control logic associated with the query statement.