Tree Object Data Processing for Complex OLAP Calculations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current relational databases and OLAP technologies are limited in performing complex data calculations and manipulations, requiring users to write programs for basic operations like summation, subtraction, multiplication, and division, making it difficult to handle intricate calculations effectively.
Innovation Solution
A large-scale data processing system that converts a multi-dimensional data model into an N-level tree data structure, allowing for easy calculation and manipulation of data through a processor and interface, enabling operations such as Select, Split, Join, Replicate, Merge, Transform, Aggregate, Distribute, arithmetic, logical, comparison, λ, age, and update processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If relational databases and OLAP are used for data storage and retrieval, then data can be organized and accessed systematically, but complex data calculations and manipulations require users to write programs themselves, increasing operational difficulty
Solution Approach 1:
The patent segments complex data processing operations into atomic Tree Objects (TOs) that represent individual measures. Each TO is an independent, self-contained unit that can be easily manipulated. By breaking down complex calculations into sequences of simple TO operations (select, split, join, aggregate, etc.), the system enables users to perform data manipulations without writing complex programs, thus improving ease of operation while maintaining systematic data organization.
Solution Approach 2:
The patent introduces Tree Objects as intermediary entities between the multi-dimensional data model and user operations. TOs serve as a mediator that simplifies the interface between users and complex database operations. Instead of directly manipulating raw database data, users work with TOs that encapsulate measurement data and its relationships, making complex calculations and manipulations more accessible and easier to perform.
2Adaptability or versatility
If users write custom programs for data manipulation operations, then specific calculation requirements can be met, but the complexity of writing and maintaining feasible programs increases
Solution Approach 1:
The patent enables self-service data processing by providing a standardized set of Tree Object operations that users can compose to meet specific calculation requirements. The system includes built-in operations (select, split, join, aggregate, distribute, transform, etc.) that cover common data manipulation needs. Users can adaptively process data by combining these operations without needing to write custom programs from scratch, thus maintaining versatility while reducing programming complexity and maintenance burden.
Solution Approach 2:
The patent creates a universal Tree Object framework that can handle diverse data processing tasks through a common set of operations. The TO structure and its associated operations serve multiple functions: data selection, aggregation, transformation, joining, and manipulation. This multi-functional approach allows the same framework to adapt to various calculation requirements without requiring separate custom programs for each task, enhancing versatility while simplifying the user interface.
3Reliability
If multi-dimensional data models are used for organizing data, then data can be structured with dimensions and measures, but performing calculations requires conversion to tree structures, adding processing steps
Solution Approach 1:
The patent applies preliminary action by pre-converting the multi-dimensional data model into a tree-structured format during data loading or model definition. The system transforms the conceptual multi-dimensional model (with dimensions, members, and measures) into a concrete tree structure where measures are organized as Tree Objects. This preliminary conversion ensures data organization reliability is maintained while reducing processing time during actual calculations, as the tree structure is already prepared and ready for efficient TO operations without requiring repeated conversions during query execution.
Data Source
AI summary
A large-scale data processing system, a large-scale data processing method, and a non-transitory tangible machine-readable medium are provided. The large-scale data processing system comprises an interface and a processor. The interface accesses a multi-dimensional data model, wherein the multi-dimensional data model comprises a plurality of dimensions, the dimensions form a multi-dimensional space of measures, each dimension is a single space comprising a plurality of members with a common set of attributes, and each measure is a data element organized and accessible through the multi-dimensional space of the cross-product of all dimensions. The processor builds at least one Tree Object (TO), wherein the TO is derived by converting the multi-dimensional data model into an N-level tree data structure according to a level order of N attributes, each tree node in the TO meets all conditions of attributes for all ancestor nodes, and N is a positive integer.