Event-Version Data Storage with Encoded Permissions for Fast Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Contemporary financial data management systems face challenges in optimizing storage and retrieval processes to be both storage-space efficient and retrieval-time efficient, particularly in handling vast datasets with dynamic user-specific visibility and permissions, and lack effective mechanisms for data exchange while maintaining consistency in real-time permissions.
Innovation Solution
A method and system for data storage and retrieval that generates encoded values for each event version indicating user-specific access permissions, records deviations, and creates a dataset to optimize storage and retrieval, ensuring fast and secure access to comprehensive order book information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If financial data is stored with full detail and version history, then data completeness and accuracy are improved, but storage space requirements increase significantly
Solution Approach 1:
The patent extracts only the necessary data fields and version information required for financial data management, separating essential data from redundant information. By extracting only the necessary elements (data entries, versions, encoded permission values) and storing them in an optimized structure, the system maintains data completeness while significantly reducing storage requirements compared to storing full detailed history of all data.
Solution Approach 2:
The patent changes the storage parameters by using encoded values for access permissions instead of storing full permission matrices, and by storing deviations from base data rather than complete data copies. This parameter transformation allows the system to maintain data completeness through encoded representations while reducing the actual storage space required.
2Loss of information
If comprehensive data is stored for all events and users, then data retrieval completeness is improved, but retrieval speed decreases due to larger datasets
Solution Approach 1:
The patent applies local quality by creating user-specific visibility and permission encodings that allow each user to access only the data relevant to them. Instead of storing and retrieving complete comprehensive data for all users, the system stores encoded permission values that enable fast retrieval of only the necessary data portions for each specific user, maintaining retrieval completeness for authorized data while significantly improving retrieval speed.
Solution Approach 2:
The patent performs preliminary encoding of access permissions and creation of versioned data structures during data ingestion, so that when retrieval is needed, the system can quickly decode and filter data based on pre-computed encoded values. This preliminary preparation avoids the need to scan through entire comprehensive datasets during retrieval operations, maintaining completeness for authorized data while dramatically improving retrieval speed.
3Reliability
If dynamic user-specific permissions are implemented, then data security and access control are improved, but system complexity increases
Solution Approach 1:
The patent merges multiple functions into a unified system: data storage, permission encoding, version management, and access control are combined into a single integrated framework. By combining these functions and using a unified encoded value structure that encapsulates both data and permission information, the system achieves reliable dynamic user-specific access control without proportionally increasing system complexity.
Solution Approach 2:
The patent uses encoded value copies that represent permission states rather than storing full permission matrices or complex access control logic for each data element. These encoded copies can be quickly generated and applied during data ingestion, providing dynamic user-specific permissions without requiring complex real-time access control computations, thus improving security while managing system complexity.
4Measurement precision
If real-time data ingestion with permission recording is implemented, then data accuracy and permission consistency are improved, but processing time and computational load increase
Solution Approach 1:
The patent performs preliminary encoding of permission values and recording of data versions during the data ingestion process itself, rather than performing these operations later during retrieval or query processing. By completing these time-consuming operations upfront during ingestion, the system ensures permission consistency and data accuracy are established immediately, while minimizing the processing time required for subsequent operations.
Solution Approach 2:
The patent replaces complex mechanical permission checking and data filtering operations with encoded value comparisons and decoding operations. Instead of performing complex queries to determine permissions and filter data, the system uses pre-encoded permission values that can be quickly compared and decoded, significantly reducing processing time while maintaining permission consistency and data accuracy.
Data Source
AI summary
Disclosed is a method for data storage and retrieval. The method comprising receiving data entries for events, wherein each event amongst events comprises versions having different values of data entries stored therein; generating and storing encoded value for each version of each event indicative of access permissions exclusive for user(s) amongst users for each version of each event; recording deviations, in each data entry; creating and storing dataset of recorded deviations for each data entry; receiving user query for data retrieval, wherein user query comprises parameter(s) identifying event(s) amongst events associated with user query; determining version amongst versions of event(s) indicative of access permissions for user; identifying data entry(s) data entries that is corresponding to determined version of event(s); retrieving recorded deviation in data entry(s), based on stored dataset of recorded deviations for data entry(s); and providing response to user, based on data entry(s) and recorded deviation in data entry(s).