Legacy Data Catalog for Efficient Reporting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Archived legacy system data, stored in files with serialized objects, becomes inefficient for querying and reporting due to its structure, where multiple files contain copies of similar records but not complete tables, making data retrieval cumbersome and inefficient for analytical reporting.
Innovation Solution
Creating a report template to identify relevant data objects, obtaining hierarchical lists of tables and fields, and using a processing device to extract and merge data from legacy system archive files, combining it with data from non-legacy systems to generate a consolidated report, while utilizing an in-memory database system like HANA for efficient data processing and reporting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Duration of action of stationary object
If legacy system data is archived in serialized object files for long-term retention, then data preservation and cost-effectiveness are improved, but data retrieval efficiency and reporting capability deteriorate
Solution Approach 1:
The patent segments the archived data into two components: the original serialized object files for long-term preservation, and a newly created data catalog that indexes the contents of these files. The data catalog contains metadata about each serialized file including the tables and records it contains, enabling efficient querying without reading the entire serialized files. This segmentation allows the system to maintain both long-term retention and efficient retrieval.
Solution Approach 2:
The patent introduces a data catalog as an intermediary layer between the query interface and the archived serialized object files. When a reporting query is received, the system first queries the data catalog to identify which serialized files contain the requested data, then retrieves only those specific files rather than scanning all archived data. This intermediary enables efficient data location and retrieval while preserving the original archived structure.
2Reliability
If multiple archive files contain copies of similar data records, then data completeness for individual objects is improved, but query efficiency and reporting performance deteriorate
Solution Approach 1:
The data catalog serves as an intermediary that tracks which serialized archive files contain copies of specific data records. The catalog maintains indexes that map data identifiers to the archive files containing them, allowing the system to efficiently locate and retrieve only the necessary file copies for a given query, rather than scanning all archive files containing similar data.
Solution Approach 2:
Instead of retrieving all archive files that might contain similar data records (excessive action), the system uses the data catalog to identify and retrieve only the specific files needed to answer the query (partial action). This selective retrieval approach maintains data completeness while significantly improving reporting performance by avoiding unnecessary data transfer and processing.
3Quantity of substance
If archived data maintains its original serialized structure, then long-term storage efficiency is improved, but analytical reporting and data aggregation capability deteriorate
Solution Approach 1:
The patent segments the data access function into two parts: the original serialized archive files that maintain compact storage efficiency, and a separate data catalog that provides adaptive querying capabilities. The catalog stores metadata in a query-friendly format that enables analytical reporting, while the actual data remains in the space-efficient serialized format. This segmentation allows both storage efficiency and reporting versatility to coexist.
Solution Approach 2:
The data catalog provides universal access capabilities that work with all archived serialized files regardless of their original structure or content. It creates a unified interface for querying, aggregating, and reporting on archived data while preserving the diverse formats and structures of the underlying serialized files. This multi-functional catalog enables analytical reporting without requiring changes to the archived data structure.
Data Source
AI summary
Legacy computing system archive files may have unique data structures designed for long-term data retention purposes and not particularly suited for data mining, querying, and other data collection functions needed to prepare certain types of reports. A report specific template may be created to facilitate the extract of specific legacy system data necessary to complete report that is stored in legacy system archive files. Once the data objects containing information relevant to the report have been identified, a hierarchical list of tables and fields of each table associated with each identified data object may be obtained from an external data source for further selection. Copies of the selected table and/or field data may then be extracted, compared, and merged with other data, such as other field and table data, obtained from other sources.