Streaming XPath Evaluation Skipping Irrelevant XML Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Streaming XPath evaluation on large XML documents is inefficient due to the need to traverse the entire document, even when only a subset of nodes matches the query, especially when large subtrees do not match the given XPath expression.
Innovation Solution
A summary of the XML document is created using start and end marks for nodes that satisfy specific criteria, allowing the streaming evaluation engine to skip irrelevant nodes and their subtrees by tracking these marks during the evaluation process, thereby reducing traversal time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a linear scan is performed on the entire XML document for streaming XPath evaluation, then all nodes can be checked for matches, but the evaluation time increases significantly for large documents with non-matching subtrees
Solution Approach 1:
The patent creates a summary of the XML document structure before query evaluation, pre-identifying nodes and their hierarchical relationships. This preliminary action allows the evaluation engine to skip entire non-matching subtrees during query processing, avoiding the need to traverse every node while maintaining evaluation completeness.
Solution Approach 2:
The patent segments the XML document into a hierarchical summary structure where nodes are divided into matching and non-matching portions. By representing the document as discrete node segments with start and end positions, the system can selectively evaluate only relevant segments against the XPath query, eliminating unnecessary traversal of large non-matching subtrees.
2Reliability
If the streaming evaluation engine traverses the entire document each time a query is evaluated, then no matching nodes are missed, but the processing efficiency decreases when large subtrees do not match the given XPath
Solution Approach 1:
The summary structure is built in advance, pre-organizing node information including start positions, end positions, and hierarchical relationships. This preliminary organization enables the evaluation engine to quickly determine which nodes to evaluate and which to skip, maintaining accurate matching while dramatically improving processing efficiency for large documents.
Solution Approach 2:
The patent implements a skipping mechanism where the evaluation engine uses the summary to identify and bypass large non-matching subtrees. By jumping over portions of the document that cannot possibly contain matching nodes, the system maintains complete node matching accuracy while achieving significant speedups in processing efficiency.
3Productivity
If start and end marks are used to represent nodes in a summary, then irrelevant nodes can be skipped during evaluation, but additional memory is required to store the summary structure
Solution Approach 1:
The patent segments the XML document into a compact summary representation using start and end marks for each node. This segmentation creates a lightweight metadata structure that is significantly smaller than the full document, enabling fast evaluation while consuming minimal additional memory compared to the original document size.
Data Source
AI summary
A method and apparatus are described for summarizing a document. For each node in the document that satisfies a marking criteria, a start and end mark pair is stored in a summary in document order. The start mark specifies a location in the document where the node starts, and the end mark specifies a location in the document where the node ends. When evaluating a query for a hierarchical path, the document is streamed into memory until the mark of a tag matches a start mark in the summary. If that tag does not fit within the path, then streaming of the document may resume at the end mark, thereby skipping the node during streaming evaluation. Translation information may be used to indicate a logical position relative to the marks in the summary when the document is modified.