Transformer Rule Repository for Structured Data Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Converting large text-based human-readable data formats, such as JSON, into a form suitable for processing by applications is resource-intensive in terms of time and processing power due to the inefficiencies of existing methods.
Innovation Solution
The implementation of transformer embodiments that operate on blocks of data, using transformer rules to convert JSON-formatted documents into arrays of JAVA objects, reducing memory consumption and processing requirements by handling data in a block-by-block manner.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire text-based human-readable document is loaded into memory for conversion, then the conversion can be performed completely, but the memory consumption becomes excessively high for large documents
Solution Approach 1:
The patent divides the large text-based document into smaller blocks or chunks that can be processed individually. Instead of loading the entire document into memory at once, the converter processes one block at a time, converting each block from text-based format to binary format separately. This segmentation approach maintains conversion completeness while dramatically reducing peak memory consumption to only the size of individual blocks rather than the entire document.
2Reliability
If conventional conversion methods are used on large text-based documents, then the data can be converted to a suitable form, but the time and processing power required become excessively high
Solution Approach 1:
By segmenting the conversion process into independent block-level operations, the system can process multiple blocks in parallel or sequentially without waiting for the entire document to be loaded. This approach maintains conversion accuracy for each block while significantly improving overall conversion throughput and reducing total processing time for large documents.
Solution Approach 2:
The patent performs preliminary parsing and structuring of the text-based document into standardized blocks before conversion. This preliminary action organizes the data in a way that facilitates efficient binary conversion, reducing the computational complexity of the actual conversion process and improving overall productivity while maintaining accuracy.
Data Source
AI summary
A system includes a processor configured to create a rule repository instance. The rule repository instance specifies a plurality of different procedures that facilitate specifying a sequence of transformer rules by cascading each of the procedures together using a dot notation format. The processor configures the rule repository instance with a plurality of transformer rules using the dot notation format. The processor receives data from a file arranged according to a first structured data format. The processor executes the sequence of transformer rules to convert data elements in the file to a second structured data format. The processor then provides for display or storage the data as converted into the second structured data format by the sequence of transformer rules.