Hierarchical Data Processing System with Aggregation Layer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database servers are inflexible in responding to changes in required analyses, as they are designed for specific analyses and require significant modifications or new system construction when analysis needs change.
Innovation Solution
A data processing system with a hierarchical database structure including a data warehouse, integration layer, aggregation layer, and analysis layer, allowing for the integration and aggregation of data to generate versatile analysis data that can be extracted based on changing conditions, reducing processing load and enabling flexible analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a data warehouse stores all input data to enable various analyses, then analysis versatility is improved, but processing load and processing time increase significantly
Solution Approach 1:
The patent segments the data warehouse into multiple layers (ODS layer, aggregation layer, and analysis layer) with different granularities. The ODS layer stores detailed transactional data, the aggregation layer stores pre-aggregated data by various dimensions, and the analysis layer stores analysis results. This segmentation allows the system to handle both detailed and summarized data needs without processing the entire data warehouse for every query.
Solution Approach 2:
The patent implements preliminary action by pre-aggregating data in the aggregation layer before analysis is required. Data is aggregated by various dimensions (time, product, customer, etc.) in advance, so when analysis queries are executed, the system can retrieve pre-processed data instead of performing complex aggregations in real-time, significantly reducing processing time.
2Productivity
If a data mart is created by selecting only necessary information from the data warehouse, then processing load is reduced, but the system cannot flexibly adapt to changes in analysis requirements
Solution Approach 1:
The patent creates a multi-functional aggregation layer that serves multiple purposes: it acts as an intermediate storage between the ODS and analysis layers, provides pre-aggregated data for various analysis dimensions, and supports different types of analysis queries. This universal layer can adapt to different analysis requirements without requiring complete system redesign, as it maintains aggregated data across multiple dimensions that can be accessed flexibly.
Solution Approach 2:
The patent implements dynamics by making the aggregation layer configurable and adaptable to changing analysis requirements. The system can dynamically select which aggregated data to retrieve based on the specific analysis needs, and the aggregation dimensions can be adjusted to match evolving business requirements without reconstructing the entire data warehouse.
3Productivity
If the database structure is designed for specific analyses in advance, then processing efficiency for those analyses is improved, but the system requires significant modifications when analysis needs change
Solution Approach 1:
The patent segments the database into distinct layers with clear responsibilities: the ODS layer handles data ingestion, the aggregation layer handles data summarization by various dimensions, and the analysis layer handles query processing. This segmentation allows each layer to be optimized independently and enables flexible reconfiguration of the aggregation layer to meet new analysis requirements without affecting the entire system architecture.
Data Source
AI summary
A database of a data processing system includes a data warehouse that stores all of input data that are input. In the data processing system, an integration layer stores an integrated data after the input data are integrated to generate the integrated data, and an aggregation layer stores aggregated data after the integrated data are aggregated by at least the number of addition items or the number of non-addition items for each of one or more combinations of the non-addition items to generate the aggregated data. An analysis layer stores an analysis data after one aggregated data is selected from the aggregated data based on a condition necessary for generation of the analysis data set by a setting section. The analysis data are further extracted from the one aggregated data.