XML Data Compression with Tag Bit Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data warehousing systems face performance issues and increased costs due to the need for long-term storage and retrieval of large volumes of XML data, as existing methods like indexing and parallel processing are burdensome and costly, especially when business practices change, requiring frequent design modifications.
Innovation Solution
A data management apparatus that compresses data groups with associated tagging information, allowing for efficient storage and retrieval by using tag bit maps and a tag judge to localize access, reducing the need for manual design changes and parallel access across multiple disks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Duration of action of stationary object
If XML data is stored directly as columns in RDB for long-term storage, then data retention capability is improved, but data search performance deteriorates and disk storage costs increase
Solution Approach 1:
The patent segments XML data into hierarchical structures with root elements and child elements, creating a tree-like organization that enables selective access to specific data segments. This segmentation allows the system to retrieve only necessary portions of data rather than scanning entire documents, thereby improving search performance while maintaining long-term storage capability.
Solution Approach 2:
The patent implements preliminary indexing by storing not only the XML data but also metadata about the data structure, element types, and relationships before actual search operations. This preliminary organization of data information enables faster retrieval operations by pre-establishing access pathways through the hierarchical structure, reducing the need for full document scanning during search operations.
2Productivity
If indexes with XML-like structure are added to localize records, then data search performance is improved, but design modification costs increase when business practices change
Solution Approach 1:
The patent creates a dynamic data management system where the hierarchical XML structure serves as a flexible framework that can adapt to changing business requirements. The system maintains the core XML structure while allowing dynamic addition of new elements, attributes, and relationships without requiring complete redesign of the indexing mechanism. This dynamic capability enables the system to accommodate business practice changes while preserving search performance benefits.
Solution Approach 2:
The patent designs a universal indexing mechanism that works across different XML document types and business scenarios. The hierarchical structure and indexing approach are designed to be generally applicable, allowing the same fundamental mechanism to handle various data structures and business practices. This universality reduces the need for custom index designs for each business requirement, thereby lowering modification costs when practices change.
3Productivity
If parallel processing is used to speed up searches, then search performance is improved, but resource usage and system complexity increase
Solution Approach 1:
The patent segments the search operation into hierarchical levels, where different parts of the XML tree can be searched independently. By leveraging the structured nature of XML data, the system can perform searches at different levels of the hierarchy simultaneously, achieving parallel processing without requiring complex distributed systems. This segmentation enables search optimization through natural parallelism in the data structure itself.
Data Source
AI summary
A data management apparatus includes a storage device; and a processor that executes a procedure, the procedure including selecting a data group, each data in the data group including one of a plurality of tags, among a plurality of data, compressing the data group into a compressed data group, and storing the compressed data group in the storage device, the stored compressed data group being associated with tagging information which indicates that each data of the data group includes the certain tag.