Multi-Buffering System for Unified Data Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data Warehouse Systems face challenges in achieving high-performance unified read/write access across different data source types, particularly when dealing with multidimensional and non-multidimensional data sources, where technical and logical keys may differ, and aggregation rules vary, affecting data consistency and concurrency during planning sessions.
Innovation Solution
A multi-buffering system is introduced in the application layer, comprising an abstract buffer component specialized into delta and after-image buffers, and a storage mechanism with cache and calculation scenario stores, leveraging in-memory database capabilities to support unified read/write access and manage data consistency across various data sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a unified buffer mechanism is used for different data source types, then the system complexity is reduced and ease of operation is improved, but data consistency and aggregation rules cannot be properly maintained across diverse data sources
Solution Approach 1:
The patent divides the unified buffer mechanism into specialized buffer types (delta buffer for cube data sources, after image buffer for DSO data sources) that are segmented according to data source characteristics. Each buffer type implements specific aggregation rules and consistency mechanisms appropriate for its data source type, resolving the contradiction by maintaining unity at the access level while implementing segmentation at the implementation level.
2Reliability
If specialized buffering mechanisms are used for different data source types, then data consistency and aggregation rules are maintained, but the system complexity increases and ease of operation deteriorates
Solution Approach 1:
The patent creates a universal buffer interface that can handle multiple data source types through polymorphism. The abstract buffer class defines a common interface for read/write operations, while specialized buffer implementations (delta buffer, after image buffer) provide type-specific behavior. This allows the system to maintain data consistency through specialized mechanisms while presenting a unified, simple interface to users, effectively reducing perceived complexity.
Solution Approach 2:
The patent introduces an abstract buffer class as an intermediary layer between the unified access interface and specialized buffer implementations. This mediator handles the complexity of type-specific operations internally while presenting a consistent interface to users, allowing specialized buffering mechanisms to operate without increasing user-facing system complexity.
3Reliability
If conventional database access methods are used, then data consistency is maintained through traditional locking mechanisms, but performance is reduced due to unnecessary database requests and lack of in-memory capabilities
Solution Approach 1:
The patent implements preliminary action by loading data into specialized buffers in advance of actual processing needs. The delta buffer and after image buffer pre-load and pre-process data according to their specific aggregation rules, so that when read/write operations occur, the data is already prepared and available in memory. This eliminates unnecessary database requests during operations while maintaining consistency through the buffer's built-in mechanisms.
Solution Approach 2:
The patent replaces traditional mechanical database locking mechanisms with an in-memory buffer-based consistency model. Instead of relying on database-level locks and traditional concurrency control, the specialized buffers implement their own consistency mechanisms in memory, substituting the mechanical locking system with a software-based approach that provides both consistency and high performance through in-memory operations.
Data Source
AI summary
Embodiments provide a multi-buffering system in an application layer to support unified read/write access to different data source types of an underlying database layer. In a particular embodiment, the database layer comprises both a multi-dimensional data source (e.g. a cube), and another data source type (e.g. a Data Store Object—DSO). An abstract buffer component of the multi-buffering system is specialized into a delta buffer configured to read/write data of the cube, and into an after image buffer configured to read/write data of the DSO. A storage mechanism component of the multi-buffering system is specialized into a cache store configured to read/write data of the cube, and a calculation scenario store configured to read/write data of the DSO. The after image buffer may support certain aggregation rules during a DSO read/write operation. The calculation scenario store may leverage capabilities of an in memory database during a DSO read/write operation.