Asynchronous Extended Table Updating for Project Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing project management systems face challenges in providing real-time reporting due to the need for daily or weekly ETL processes, which lead to outdated operational reporting and inefficient resource utilization, as well as difficulties in SQL reporting, data extraction, and long-term archiving without affecting underlying data fields.
Innovation Solution
The system employs asynchronous updating of extended tables associated with base tables, allowing for near real-time reporting and combined views without altering existing application code or data storage, using an asynchronous updater module to process data in logical units and differentiate between demand and non-demand projects for efficient processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If daily or weekly ETL processes are used for data updating, then data consistency between base tables and extended tables is maintained, but reporting timeliness deteriorates and operational reporting becomes outdated
Solution Approach 1:
The system transitions from static batch ETL processes to dynamic asynchronous updating. The asynchronous updater module continuously monitors base tables and automatically triggers extended table updates when data changes occur, making the system adaptive and responsive to real-time data changes while maintaining consistency.
Solution Approach 2:
The system pre-establishes the asynchronous updating mechanism and triggers update operations in advance when base table changes are detected. This preliminary action ensures that extended tables are updated before reporting queries execute, eliminating delays associated with traditional batch processing.
2Ease of manufacture
If traditional ETL processes are used, then data processing is performed in batches, but resource utilization becomes inefficient and productivity decreases
Solution Approach 1:
The asynchronous updater module operates autonomously to monitor base tables and trigger extended table updates without requiring manual intervention or centralized batch scheduling. This self-service mechanism optimizes resource utilization by performing updates only when necessary, eliminating wasted computational resources on unnecessary batch operations.
Solution Approach 2:
The system maintains continuous monitoring and updating capability rather than relying on periodic batch processes. The asynchronous updater continuously watches for data changes and performs updates immediately when triggered, ensuring continuous data freshness and efficient resource usage throughout system operation.
3Speed
If base tables are altered to improve reporting capabilities, then real-time reporting may be achieved, but existing application code and data storage structures are affected
Solution Approach 1:
The system segments data storage into base tables for raw data and extended tables for processed data. This segmentation allows the asynchronous updater to work on extended tables independently without affecting base table structures or existing application code that relies on base tables, thereby maintaining system stability while enabling real-time reporting.
Solution Approach 2:
The asynchronous updater module acts as an intermediary between base tables and extended tables. It translates data changes from base tables into corresponding updates in extended tables without requiring direct modifications to base table structures or existing application code, thus maintaining system architecture integrity while achieving real-time reporting capabilities.
Data Source
AI summary
A system for defining a plurality of extended tables, wherein each extended table of the plurality of extended tables is associated with a corresponding base table of a plurality of base tables. The system updates data contained by the plurality of extended tables in accordance with data contained by the plurality of base tables, wherein the updating is performed asynchronously across logical units of data contained by the plurality of extended tables. The system provides a combined view of the data contained by the plurality of extended tables and the data contained by the plurality of base tables.