XML to SDML Conversion for Parallel Parsing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies fail to efficiently process huge XML documents in parallel, leading to performance bottlenecks due to time-consuming parsing operations, which hinder high performance in multicore environments and across servers.
Innovation Solution
Converting XML documents into Modified Simple Dependency Markup Language (SDML) format, allowing for high degree of parallelism and efficient processing across multiple cores and servers, leveraging string manipulation for simplified parsing and data representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional XML parsing methods (SAX, DOM) are used on huge XML documents, then parsing completeness and accuracy are maintained, but parsing time becomes excessively long (dozens of seconds) and performance is bottlenecked
Solution Approach 1:
The patent divides the huge XML document into multiple smaller fragments based on XML element boundaries. Each fragment can be parsed independently by different processing units, enabling parallel processing. This segmentation reduces the time complexity from O(n) sequential parsing to O(n/p) parallel parsing where p is the number of processing units, directly addressing the performance bottleneck of parsing huge XML documents.
Solution Approach 2:
The patent performs preliminary actions by pre-processing the XML document to identify suitable split points and prepare fragments before actual parsing. It also pre-allocates multiple processing units and establishes parallel processing pipelines in advance. This preliminary preparation eliminates runtime overhead and enables immediate parallel processing, significantly reducing total parsing time for huge documents.
2Productivity
If XML documents are processed sequentially in traditional manner, then processing logic is simple to implement, but multicore processing capabilities are not utilized and performance remains limited
Solution Approach 1:
The patent segments both the XML document and the processing architecture. The document is divided into fragments, and the processing system is divided into multiple independent processing units. Each unit handles specific fragments with simplified logic, while a coordination layer manages distribution and result aggregation. This segmentation enables linear scaling with added cores without proportionally increasing individual processor complexity.
Solution Approach 2:
The patent creates universal processing units that can handle any XML fragment with the same processing logic. These multi-functional units can be dynamically allocated to different fragments based on workload, making the system adaptable to various document sizes and structures. The same processing template is reused across multiple units, simplifying implementation while enabling parallel execution.
3Quantity of substance
If huge XML documents are parsed without fragmentation, then data integrity and context preservation are ensured, but memory consumption and processing time increase significantly
Solution Approach 1:
The patent segments the XML document into fragments that preserve complete XML elements and their hierarchical relationships. Each fragment maintains self-contained data integrity with proper opening and closing tags. This segmentation reduces memory consumption from loading the entire document into memory to loading only necessary fragments, while maintaining data integrity through careful boundary selection at XML element levels.
Solution Approach 2:
The patent introduces an intermediary coordination layer that manages fragment boundaries and ensures data integrity across partitions. This mediator tracks element relationships that span fragment boundaries and reconstructs complete data context in the final aggregation phase. It acts as a buffer between fragmented processing and whole-document requirements, preserving reliability while enabling memory-efficient processing.
Data Source
AI summary
The present invention relates to the field of high performance computation. Particularly, the invention relates to converting a huge XML document into SDML format which can be processed with high degree of parallelism to achieve high performance. In addition also SDML can be used as a standalone protocol for data representation. SDML deals with one time write and many times read. Further, SDML files can be splitted on number of lines which makes it easier to distribute among multi cores and even distributing across servers.