Measure Tree Object for Real-Time Large-Scale Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current relational databases and OLAP systems are unable to provide real-time responses when processing large volumes of data, leading to intolerable processing times for business managers who need to make decisions based on big data.
Innovation Solution
A large-scale data processing apparatus and method that creates a measure tree object by following a level order of attributes, allowing for efficient querying and retrieval of data through a view path, using a storage unit, interface, and processor to access and extend a measure table via the TransJoin operation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional relational databases and OLAP systems are used to process huge amounts of data, then data storage and basic processing capabilities are provided, but the processing time becomes significantly long and real-time response cannot be achieved
Solution Approach 1:
The patent segments the data processing task by creating a measure tree object that hierarchically organizes measures and dimensions. The tree structure divides the huge data set into manageable segments that can be processed and queried more efficiently, allowing the system to handle large volumes of data without proportionally increasing processing time
Solution Approach 2:
The patent performs preliminary action by pre-building the measure tree object from the measure table before queries are executed. This pre-processing step organizes the data into an optimized structure that enables faster query execution and real-time analysis, eliminating the need to process raw data from scratch for each query
2Adaptability or versatility
If more data is stored in the database to support comprehensive analysis, then the analytical capability is improved, but the access and processing time increases significantly
Solution Approach 1:
The patent introduces a new dimensional structure by creating a measure tree object with hierarchical levels representing different dimensions (measures, dimensions, and their relationships). This dimensional transformation allows the system to maintain comprehensive analytical capability across multiple data dimensions while enabling efficient navigation and query execution through the hierarchical structure
3Quantity of substance
If the database processes millions or billions of records, then comprehensive data coverage is achieved, but the response time becomes intolerable for real-time decision-making
Solution Approach 1:
The measure tree object serves as an intermediary structure between the raw measure table and the query processing system. It acts as a mediator that pre-organizes and indexes the comprehensive data, enabling the system to access and process millions or billions of records efficiently without directly querying the underlying raw data for each operation
Data Source
AI summary
A large-scale data processing apparatus, method, and non-transitory tangible machine-readable medium are provided. The large-scale data processing apparatus includes a storage unit, an interface, and a processor. The storage unit is stored with a measure table comprising at least one measure. Each of the at least one measure includes a value corresponding to a key attribute and a piece of data corresponding to a data field. The interface is configured to receive a level order of N attributes, wherein N is a positive integer and the N attributes comprise the key attribute. The processor is configured to create a measure tree object for the measure table by following the level order so that the measure tree object has N levels corresponding to the N attributes in a one-to-one fashion.