Compact XML Tree Node Representation for Memory-Constrained Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Legacy XML document management technologies, such as DOM processing, face memory constraints when handling large volumes of data, leading to crashes and inefficiencies, as they require the entire XML document to be stored in memory, making it inconvenient and unwieldy, especially in kernel processing applications.
Innovation Solution
A compact tree node representation method that uses an XML tree index data structure and hash tables to store XML document nodes, allowing for discrete memory allocations and paging, enabling efficient memory usage by loading only necessary portions into memory while storing the rest in external storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If DOM processing is used to access XML documents, then the entire XML document can be stored in memory for random access, but memory consumption becomes excessive and causes crashes when handling large volumes of data
Solution Approach 1:
The patent divides the XML document into a compact tree index structure stored in memory and the actual document content stored separately on disk. The index is segmented into fixed-size blocks that can be independently loaded and paged, allowing random access to specific nodes without loading the entire document into memory.
Solution Approach 2:
The patent introduces a compact tree index as an intermediary data structure between the XML document content and the processing application. This index contains only essential node information (tags, attributes, element types) and serves as a navigation layer that enables random access to document nodes without requiring the full document to reside in memory.
2Loss of information
If the entire XML document is loaded into memory for processing, then complete data availability is achieved, but memory allocation requirements become uncontrollable and lead to system crashes
Solution Approach 1:
The patent segments the XML document representation into two distinct parts: a compact index structure in memory and the actual document content on disk. This segmentation ensures that only the essential structural information resides in memory, while the bulk data remains on storage, preventing memory exhaustion while maintaining data accessibility.
Solution Approach 2:
The patent implements partial loading of XML data into memory by loading only the compact index structure and specific index blocks as needed. The full document content remains on disk and is loaded into memory only when required for processing, avoiding the excessive memory allocation that would occur with complete document loading.
3Adaptability or versatility
If legacy DOM processing is used, then standard XML access methods are available, but the approach becomes inconvenient and unwieldy when dealing with large data volumes requiring job splitting
Solution Approach 1:
The patent implements a dynamic memory management system where index blocks are loaded into and paged out of memory based on processing needs. The system can dynamically allocate and deallocate index blocks as the processing progresses through different portions of the XML document, eliminating the need for static job splitting while maintaining ease of operation.
Data Source
AI summary
Methods and systems for creating a compact tree node representation of an XML document. One implementation commences by allocating memory for storing an XML tree index data structure, then allocating another separate portion of memory to store a hash table. Then, traversing an XML document to process the traversed nodes as follows: (a) when the traversed node is an element node, then adding the element node to the XML tree index data structure (b) when the traversed node is a text node, then populating a text node index into the XML tree index data structure and copying the text node values to the hash table, and (c) when the traversed node is an attribute node, then populating an attribute node index into the XML tree index data structure. Such a structure supports fast index-based tree restructuring, and permits very large XML document to be accessed within tight memory size constraints.