VSAM Analytics Framework for On-Demand Querying
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
VSAM methods face limitations such as performance constraints compared to RDBMS, lack of concurrent access and conditional querying support, and absence of true metadata or data dictionary concepts, which hinder efficient analytics and data management on mainframes.
Innovation Solution
A VSAM analytics framework is introduced that enables self-serve on-demand analytics by discovering metadata, parsing data into tables, and executing conditional queries on VSAM data, all while operating on Unix System Services (USS) to reduce MIPS consumption and cycle time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If VSAM files are analyzed using traditional methods (COBOL, JCL), then data can be processed, but cycle time is long and MIPS consumption is high
Solution Approach 1:
The patent creates a copy of the VSAM file data in a relational database table, allowing analytics to be performed on the copied data rather than the original VSAM files. This copying approach enables parallel processing and eliminates the time-consuming sequential access patterns of traditional VSAM analysis, directly reducing cycle time while maintaining data integrity.
Solution Approach 2:
The patent replaces the mechanical processing of VSAM files through COBOL/JCL programs with a database-driven approach using SQL queries and relational database management systems. This substitution enables more efficient data access patterns, concurrent query execution, and reduced MIPS consumption by leveraging the optimized query execution engines of RDBMS.
2Productivity
If VSAM files are analyzed using traditional methods, then data processing is possible, but MIPS consumption is high
Solution Approach 1:
By creating a copy of VSAM data in a relational database, the system enables analytics to run on the copied data structure which is more efficiently queryable. This reduces the computational overhead and MIPS consumption associated with traditional VSAM file processing while maintaining equivalent analytical capabilities.
Solution Approach 2:
The patent changes the data structure parameters from VSAM file format to relational database table format, enabling more efficient query optimization, indexing, and execution plans. This parameter change allows the database engine to utilize sophisticated query optimization techniques that reduce MIPS consumption compared to traditional VSAM processing.
3Speed
If data is moved to RDBMS for analytics, then query performance improves, but data security and integrity may be compromised
Solution Approach 1:
The patent creates a copy of the VSAM data in a relational database table specifically for analytics purposes. The original VSAM files remain intact and unchanged on the mainframe, preserving data security and integrity. The copied data in the RDBMS enables high-performance querying without compromising the source system's security model.
Solution Approach 2:
The patent segments the data system into two distinct components: the original VSAM files for production and security-critical operations, and a copied relational database table for analytics and reporting. This segmentation allows each component to operate with its own optimized characteristics while maintaining clear boundaries that preserve security and integrity of the source data.
4Reliability
If VSAM files are accessed sequentially, then data integrity is maintained, but analytics performance is limited
Solution Approach 1:
The patent copies VSAM file data into a relational database table structure that supports efficient random access and parallel query execution. The copy enables analytics workloads to leverage the RDBMS's optimized access paths, indexes, and query execution engines, dramatically improving analytics throughput while the original VSAM files maintain their sequential access integrity for production operations.
Data Source
AI summary
Implementations are directed to receiving, by an application, query data, the query data indicating a conditional query and a selection of files on which to execute the conditional query; identifying a subset of selected files for which copies of the selected files are not stored in a file database of the application; extracting, from a file system, a copy of each file of the subset of selected files into the file database of the application; parsing the copies of the selected files from the file database into a dataframe; executing the conditional query on the selected files in the dataframe; and providing a result of the conditional query. Actions include executing, by a discovery engine, a discovery process to provide metadata comprising, for a file in the file system: a name of the file; corresponding column names and data types; and a file alias.