XML Data Search Using SIMD Packed Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The 'list of structures' representation of XML data in memory is inefficient for hierarchical searches due to sequential access, leading to slower search times compared to the 'tree of structures' representation, which requires more memory and has random access capabilities.
Innovation Solution
The XML data is stored in a format suitable for Single Instruction Multiple Data (SIMD) instructions by splitting the list of structures into a primary list for key information in parallel layout and a secondary list for residual data in serial form, allowing SIMD instructions to perform parallel searches and reducing scanning costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If XML data is stored as a list of structures for sequential access, then memory usage is reduced and ease of automatic processing is improved, but search speed deteriorates compared to tree of structures representation
Solution Approach 1:
The patent divides the list of structures into multiple segments or chunks that can be processed in parallel. Each segment contains a portion of the XML data structures, allowing simultaneous processing of multiple segments using SIMD instructions. This segmentation enables the system to maintain the memory efficiency of list of structures while achieving parallel processing capabilities that improve search speed.
Solution Approach 2:
The patent introduces an intermediary data structure or indexing mechanism that bridges the sequential list of structures and the hierarchical search requirements. This intermediary layer allows the system to process data in a hierarchical manner without converting to a full tree structure, maintaining memory efficiency while enabling faster search operations through the intermediary access patterns.
2Speed
If XML data is stored as a tree of structures with pointers for random access, then search speed is improved, but memory usage increases
Solution Approach 1:
The patent extracts only the essential pointer-like functionality needed for hierarchical searching from the full tree of structures representation. Instead of implementing complete tree structures with all their memory overhead, the system extracts and implements only the critical access patterns and indexing mechanisms required for fast hierarchical searches, discarding the redundant memory structures.
Solution Approach 2:
The patent changes the parameters of the data structure representation by modifying how data is organized and accessed. It transforms the traditional sequential list into a parallelizable format with modified access patterns, changing parameters such as data layout, access methodology, and processing mode to achieve both memory efficiency and search speed improvements.
3Quantity of substance
If hierarchical search is performed on list of structures data, then memory efficiency is maintained, but search time increases due to sequential access requirements
Solution Approach 1:
The patent introduces dynamic processing capabilities to the list of structures representation by enabling the data to be processed in multiple modes depending on the search requirements. The system can dynamically switch between sequential processing for simple operations and parallel SIMD processing for hierarchical searches, making the data structure adaptive and flexible to different operational demands.
Solution Approach 2:
The patent substitutes the traditional sequential mechanical processing of list of structures with parallel processing using SIMD (Single Instruction Multiple Data) instructions. This replacement transforms the processing mechanism from sequential byte-by-byte scanning to simultaneous multi-byte processing, dramatically reducing search time while maintaining the memory-efficient list of structures format.
Data Source
AI summary
Extensible Markup Language (XML) data is represented as a list of structures with each structure in the list representing an aspect of the XML. A set of frequently used elements is extracted from the list of structure representation and stored in packed vectors. The packed vector representation allows Single Instruction Multiple Data (SIMD) instructions to be used directly on the XML data to increase the speed at which the XML data may be searched while minimizing the memory needed to store the XML data.