XML Query Result Locators for Memory-Constrained Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing approaches for XML data generation and aggregation are inefficient and do not scale well for queries that return large results, primarily due to the serialization and storage of entire query results, leading to significant performance issues and memory consumption.
Innovation Solution
The method involves representing XML query results using compact in-memory data structures that store locators pointing to XML fragments rather than the fragments themselves, allowing for on-demand generation of serialized representations, thereby reducing memory usage and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire query result is serialized and stored in a temporary LOB, then the XML data can be aggregated and returned, but the memory consumption and I/O operations increase significantly
Solution Approach 1:
The patent extracts only the essential locator information from XML fragments rather than extracting and storing the entire fragment content. The data structure stores locators that point to the actual XML data in external storage, separating the indexing function from the data storage function. This reduces memory consumption while maintaining the ability to retrieve and aggregate XML query results accurately.
2Reliability
If all intermediate results are serialized during query processing, then the aggregation can be performed, but the processing time and I/O operations increase
Solution Approach 1:
The patent performs preliminary organization of XML data into a locator-based data structure during the query execution process, but delays the actual serialization and materialization of intermediate results until they are needed for aggregation. This preliminary structuring allows for efficient in-memory manipulation of locators without the overhead of serializing large XML fragments at each intermediate step, reducing processing time while ensuring complete query results.
3Speed
If XML fragments are stored directly in memory during query evaluation, then the data can be accessed quickly, but the memory usage increases for large result sets
Solution Approach 1:
The patent introduces locators as intermediary elements that bridge in-memory data structures and external XML storage. Instead of storing actual XML fragment content in memory, the system stores compact locator objects that contain references to the external storage locations. These locators enable fast in-memory comparison, sorting, and aggregation operations while the actual XML data remains in external storage, achieving both fast access and low memory usage.
Data Source
AI summary
Techniques for fast and scalable generation and aggregation of XML data are described. In an example embodiment, an XML query that requests data from XML documents is received. The XML query is evaluated to determine one or more XML results. For each particular XML result, evaluating the XML query comprises: instantiating a particular data structure that represents the particular XML result, where the particular data structure is encoded in accordance with tags specified in the XML query but does not store the tags; and storing, in the particular data structure, one or more locators that respectively point to one or more fragments in the XML documents, where the particular data structure stores the one or more locators but does not store the one or more fragments. On demand, in response to a request indicating the particular XML result, a serialized representation of the particular XML result is generated based at least on the particular data structure.