Event Record Format Conversion via Virtual Machine Mediator
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In production environments, event records in fixed length formats, such as those used on mainframe systems with EBCDIC encoding, require manual conversion to target formats like those used in distributed systems like Kafka, which can be complex and burdensome for developers, as they need to understand specific message formats and syntax for database queries.
Innovation Solution
A virtual machine is used to automatically convert event records from an initial system to a target system by employing a conversion file that maps fields and schema, communicates with an authentication server for encryption, and streams the records to the target system, simplifying the process and eliminating the need for developers to write custom code.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual conversion methods are used to transform event records from mainframe formats to distributed system formats, then developers can access information in target systems, but the process becomes complex and burdensome requiring knowledge of specific message formats and database syntax
Solution Approach 1:
The patent introduces an intermediary conversion system that automatically transforms event records from mainframe formats (EBCDIC, fixed-length) to distributed system formats (UTF-8, variable-length). This intermediary layer handles the complex conversion logic, field mapping, and schema translation, shielding developers from format complexities while enabling seamless data access across systems.
Solution Approach 2:
The conversion system performs self-service by automatically detecting source format characteristics, applying appropriate transformation rules, and generating target format records without requiring developer intervention. The system autonomously manages encoding conversion, field repositioning, and schema adaptation, eliminating the burden of manual format knowledge requirements.
2Productivity
If developers write custom code or scripts to process event records in specific formats, then they can access and manipulate data, but the development time and effort increase significantly
Solution Approach 1:
The system performs preliminary actions by pre-configuring conversion rules, field mappings, and schema transformations in a centralized configuration. This upfront preparation enables automatic conversion without requiring developers to write custom code for each format transformation, significantly reducing development time and effort while maintaining high productivity.
Solution Approach 2:
The conversion system provides universal functionality by handling multiple source formats (EBCDIC, fixed-length, variable-length) and multiple target formats (UTF-8, JSON, Avro) through a single unified platform. This multi-functional approach eliminates the need for separate custom scripts for each format pair, streamlining the development process and improving overall productivity.
3Ease of manufacture
If event records are converted automatically using a virtual machine with conversion files, then the conversion process is simplified and standardized, but the system requires additional infrastructure components
Solution Approach 1:
The system simplifies conversion setup by parameterizing the conversion process through configuration files that define source and target format parameters, field mappings, and transformation rules. By externalizing these parameters, the system becomes easily configurable without modifying core code, reducing the complexity of setup while maintaining standardized conversion processes.
Solution Approach 2:
The conversion infrastructure is segmented into modular components: a virtual machine execution environment, conversion file storage, authentication server interface, and target system connector. This segmentation allows each component to be independently configured and managed, reducing overall system complexity while enabling standardized, reusable conversion processes across different environments.
Data Source
AI summary
Systems and methods for streaming event records in one record format to a target system by converting the event record into a new format that is compatible with the target system. A conversion file may be used to translate the contents of the event record into an updated event record that may include schema or topics. In addition, an authentication server is used to obtain an authentication token and an encryption server may be used to manage the encryption of event records as they are streamed into a target computing system. Some embodiments involve streaming mainframe event records into a Kafka-based server.