Compressed XML (Extensible Markup Language) database type, system and method supporting efficient query

Abstract

The invention discloses a compressed XML database type, system and method supporting efficient query, and relates to the technical field of XML databases, and the database type adopts a hierarchical compression storage architecture and a reverse search query architecture; in the storage stage, XML non-keyword information is coded through Huffman coding, recoverable ordered arrangement is carried out on the coded information through a tree structure, then transcoding is carried out, the information is stored in a disk in a binary form, and compared with a traditional XML database storage mode, at least 50% of storage space is saved. In the query stage, a reverse search mode is adopted, a corresponding Huffman code initial position is positioned through a keyword, then under the optimization effect of a reverse index, reverse search is performed to recover a complete XML structure, and the query complexity is reduced from traditional O (n) to O (1). According to the method, the storage efficiency and the query performance of the XML data are greatly improved, and the method is suitable for large-scale XML data management and application scenes.

Description

Technical Field

[0001] This invention relates to the field of data storage and retrieval technology, and in particular to a compressed XML database type, system, and method that supports efficient querying. Background Technology

[0002] XML (Extensible Markup Language), as a typical representative of semi-structured data, has become a core data carrier in business systems in finance, IoT, and government sectors due to its excellent structure, scalability, interoperability, and self-descriptive properties. It is widely used in business scenarios such as financial transactions, e-government, system logs, configuration centers, message queues, and log repositories. With the deepening of informatization and digitalization, in many long-running systems, databases are often given responsibilities beyond traditional data storage, leading to the continuous accumulation of massive amounts of redundant XML data. Traditional single-machine or master-slave databases face the dual pressure of rapidly expanding storage space and continuously declining query performance. Although distributed databases and database sharding technologies provide expansion paths, their high transformation costs, operational complexity, and potential risks to business stability force researchers to seek lighter, less invasive kernel-level optimization solutions.

[0003] In terms of storage, traditional XML databases often store XML data directly in text format. The tags, attributes, and other non-keyword information in XML data inherently contain structural redundancy. Furthermore, business systems frequently store the same entity in both relational rows and XML document formats, leading to data duplication and low storage efficiency. This is especially problematic in large-scale XML data scenarios, significantly increasing storage hardware costs. While some existing compression storage solutions can reduce storage space to some extent, they have several shortcomings: Firstly, they often use general compression algorithms (such as ZIP and GZIP), treating XML as an unstructured character stream without optimizing for the structured characteristics of XML data, resulting in limited compression effectiveness. Secondly, even with XML-aware compression technologies, the focus is often on the algorithm itself, lacking coordinated design with database storage engines and indexing mechanisms. High compression rates often come at the cost of random access efficiency; compressed data is difficult to query directly and requires complete decompression, further impacting processing efficiency. More importantly, current practices typically involve "compressing XML first, then indexing the original representation," causing the index itself to become a new source of redundancy, resulting in the counterproductive effect of "compressing data and inflating the index."

[0004] In terms of querying, traditional XML databases mostly use sequential traversal queries, which traverse the XML data sequentially from the beginning until the target data is found, with a query complexity of O(n) (where n is the total length of the XML data). When the XML data is large, sequential traversal queries consume a lot of time, resulting in extremely low query efficiency and failing to meet real-time query requirements. While some existing index optimization solutions can improve query efficiency, they still have significant shortcomings: First, the index structure itself occupies a large amount of storage space, and to support diverse queries, the system needs to maintain multiple sets of indexes in parallel. These indexes highly overlap in metadata such as tag names and path prefixes, leading to "index bloat." Second, existing advanced indexing technologies such as inverted indexes are mainly designed for unstructured or flat data, lacking native support for the semi-structured characteristics of XML. Treating XML as a flat character stream for word segmentation introduces "syntactic noise" and severs path navigation and content filtering, requiring expensive join operations during queries, making it difficult for the optimizer to generate a globally optimal plan. Third, index maintenance costs are high, especially in dynamic update scenarios. Fine-grained modifications may trigger a global index rebuild, resulting in high write latency and making it difficult to meet the needs of OLTP scenarios. Overall, most existing solutions optimize storage format, index structure, or compression algorithm independently, failing to conduct integrated cross-layer design from the semantic level of the data model, making it difficult to balance storage efficiency and query efficiency.

[0005] Therefore, this invention proposes an XML database solution that can significantly reduce storage space usage and significantly improve query efficiency. Summary of the Invention

[0006] To address the technical problems of low storage and query efficiency in existing XML databases, this invention provides a compressed XML database type, system, and method that supports efficient querying, achieving efficient compressed storage and fast querying of XML data, while balancing storage cost and query performance.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A compressed XML database type that supports efficient querying, the database type adopts a layered architecture, including a compressed storage layer and a query index layer;

[0009] The compressed storage layer is used to store XML data after encoding, sorting and transcoding. Specifically, Huffman coding is used to encode non-keyword information in the XML data, and the encoded information is arranged in a recoverable ordered manner through a tree structure. After the arrangement is completed, it is transcoded and stored in binary form.

[0010] The query index layer has a built-in inverted index structure, which associates the keywords of the XML data with the starting position information of the corresponding Huffman code, and is used to provide index support for reverse search queries;

[0011] The storage space required for this database type is reduced by at least 50% compared to traditional XML database storage methods, and the query complexity is O(1).

[0012] A compressed XML database system supporting efficient querying of a database type, comprising a data acquisition module, a compression processing module, a storage module, an index building module, a query processing module, and a data recovery module;

[0013] The data acquisition module is used to acquire the raw XML data to be processed and to preprocess the raw XML data, including deduplication and format standardization.

[0014] The compression processing module is used to separate keyword information and non-keyword information from the preprocessed XML data, encode the non-keyword information using the Huffman coding algorithm to generate a Huffman coded string, and simultaneously arrange the encoded non-keyword information and keyword information in an orderly manner using a tree structure to obtain structured coded data.

[0015] The storage module is used to transcode the structured coded data output by the compression processing module, convert it into binary data, and then store it on the disk.

[0016] The index building module is used to build an inverted index, which associates and maps the keywords of the XML data with the starting position of the Huffman encoded string generated by the compression processing module, generates an index table, and stores it in the query index layer.

[0017] The query processing module is used to receive user query requests, extract keywords from the query requests, locate the starting position of the Huffman encoded string corresponding to the keyword based on the index table, and use a reverse search algorithm to retrieve complete encoded information in the structured encoded data.

[0018] The data recovery module is used to decode the complete encoded information retrieved by the query processing module, restore it to the original XML data structure, and then feed it back to the user.

[0019] Furthermore, the tree structure used in the compression processing module is a balanced binary tree. The nodes of the balanced binary tree include keyword nodes and encoding nodes. Keyword nodes store keyword information of XML data, and encoding nodes store non-keyword information after Huffman encoding. The nodes are associated with each other through pointers to realize the orderly association between encoding information and keyword information.

[0020] Furthermore, the inverted index table constructed by the index building module includes a keyword field, a start position field, and a length field. The keyword field stores the keyword of the XML data, the start position field stores the starting offset of the Huffman-coded string corresponding to the keyword in the structured encoded data, and the length field stores the length of the Huffman-coded string.

[0021] A compressed XML database method supporting efficient querying includes a storage phase and a query phase, with the following specific steps:

[0022] S1: Storage Stage

[0023] S11: Collect raw XML data and perform deduplication and format standardization preprocessing on the raw XML data;

[0024] S12: Separate key information from non-key information in the preprocessed XML data;

[0025] S13: Use the Huffman coding algorithm to encode non-keyword information to generate a Huffman coded string;

[0026] S14: Construct a balanced binary tree. Store the Huffman encoded string generated in step S13 and the keyword information separated in step S12 as node data into the balanced binary tree to achieve the ordered arrangement of encoded information and keyword information, and obtain structured encoded data.

[0027] S15: Convert the structured coded data into binary data and store it on the disk;

[0028] S16: Construct an inverted index table, associate the key with the starting position of the Huffman-coded string, and store the index table;

[0029] S2: Query Phase

[0030] S21: Receive user query requests and extract query keywords;

[0031] S22: Based on the inverted index table, query the starting position of the Huffman-coded string corresponding to the keyword;

[0032] S23: Starting from the initial position, a reverse search algorithm is used to search backwards in the structured coded data to obtain complete coded information;

[0033] S24: Perform Huffman decoding on the complete encoded information to restore the original XML data structure;

[0034] S25: Feed back the restored XML data to the user.

[0035] Furthermore, the specific process of encoding non-keyword information using the Huffman coding algorithm in step S13 is as follows:

[0036] S131: Calculate the frequency f of each character in the non-keyword information of XML, where i is the character index, i = 1, 2, ..., n, and n is the total number of different characters in the non-keyword information;

[0037] S132: Based on each character and its frequency of occurrence, construct a Huffman tree. The construction of the Huffman tree satisfies the following: take the two nodes with the lowest frequency as child nodes to construct a new node, and the frequency of the new node is the sum of the frequencies of the two child nodes; repeat this process until all nodes are merged into a root node.

[0038] S133: Assign codes to each branch of the Huffman tree, specifying that the left branch is 0 and the right branch is 1. The path code from the root node to each character node is the Huffman code corresponding to that character.

[0039] S134: Based on the Huffman coding of each character, convert the non-keyword information into the corresponding Huffman coded string, where the length L of the coded string satisfies:

[0040]

[0041] Where l is the length of the Huffman code corresponding to the i-th character.

[0042] Furthermore, the balanced binary tree constructed in step S14 satisfies that the absolute value of the height difference between the left and right subtrees does not exceed 1, and the key of all nodes in the left subtree is less than the key of the root node, while the key of all nodes in the right subtree is greater than the key of the root node.

[0043] Furthermore, the specific process of the reverse search algorithm in step S23 is as follows: taking the starting position located by the inverted index table as the endpoint, tracing back along the node association pointers of the balanced binary tree, obtaining the Huffman encoded data of each encoded node in turn, until tracing back to the root node, integrating all the traced encoded data to form complete encoded information.

[0044] Furthermore, the process of transcoding to binary data in step S15 is as follows: the structured encoded data is converted into a binary stream using the Base64 transcoding algorithm. The storage format of the binary stream is "header identifier + data length + binary data + checksum". The header identifier is used to identify that the data type is XML compressed data, and the checksum is used to verify the integrity of the data storage process.

[0045] The beneficial effects of this invention are as follows:

[0046] 1. High storage efficiency: This invention uses Huffman coding to encode non-keyword information in XML. Huffman coding allocates the encoding length based on the frequency of character occurrence, which can compress the storage space of non-keyword information to the greatest extent. At the same time, the encoded information is arranged in an orderly manner through a tree structure and combined with binary transcoding storage, which saves at least 50% of storage space compared with traditional XML database storage methods and significantly reduces storage hardware costs.

[0047] 2. High query efficiency: This invention adopts a query method that combines reverse search with inverted index. The inverted index can directly locate the starting position of the Huffman code corresponding to the keyword, and then the complete encoding information can be quickly obtained through reverse search. The query complexity is reduced from the traditional O(n) to O(1), realizing real-time fast query of XML data and meeting the real-time query needs of large-scale XML data.

[0048] 3. Good data recoverability: The tree structure used in this invention is a recoverable ordered arrangement structure. Combined with the reversibility of Huffman coding, the original XML data structure can be completely recovered through decoding, ensuring the integrity and availability of the data.

[0049] 4. Strong applicability: The solution of this invention can be applied to various XML data scenarios, especially to the management and application of large-scale XML data, such as data exchange platforms, Web service backend databases, etc. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a schematic diagram of the module structure of the database system in Embodiment 2 of the present invention;

[0052] Figure 2 This is a flowchart of the storage stage in Embodiment 3 of the present invention;

[0053] Figure 3 This is a flowchart of the query stage in Embodiment 3 of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0055] A compressed XML database type supporting efficient querying employs a layered architecture, including a compressed storage layer and a query index layer. The compressed storage layer stores XML data after encoding, sorting, and transcoding. Specifically, Huffman coding is used to encode non-keyword information in the XML data, and the encoded information is arranged in a recoverable ordered manner using a tree structure before being transcoded and stored in binary form. The query index layer has a built-in inverted index structure that associates keywords in the XML data with the starting position information of their corresponding Huffman codes, providing index support for reverse search queries. This database type reduces storage space usage by at least 50% compared to traditional XML database storage methods, and its query complexity is O(1).

[0056] A compressed XML database system supporting efficient querying based on the aforementioned database types includes a data acquisition module, a compression processing module, a storage module, an index building module, a query processing module, and a data recovery module. These modules work collaboratively to achieve the entire process of XML data acquisition, preprocessing, compression, storage, index building, querying, and recovery.

[0057] A compressed XML database method supporting efficient querying includes a storage phase and a query phase. The storage phase achieves efficient compressed storage of XML data through data acquisition and preprocessing, key-to-non-keyword separation, Huffman coding, ordered tree structure arrangement, binary transcoding storage, and inverted index construction. The query phase enables fast querying of XML data through key extraction, index location, reverse search, and decoding recovery.

[0058] Example 1: A compressed XML database type that supports efficient querying

[0059] The database type in this embodiment adopts a layered architecture, including a compressed storage layer and a query index layer. The two layers work together to achieve efficient compressed storage and fast query.

[0060] The compressed storage layer's core function is to achieve efficient compressed storage of XML data. First, it separates the XML data into keywords and non-keywords. Keywords refer to information in the XML data that identifies the core meaning of the data (such as data entity names, key attribute values, etc.), while non-keywords refer to tags in the XML data (such as...). <root> 、< / root>Information such as separators and redundant spaces is included. Huffman coding is used to encode non-keyword information. Huffman coding allocates the code length based on the frequency of each character in the non-keyword information; characters with higher frequency are assigned shorter codes, thus minimizing the storage space occupied by non-keyword information. Subsequently, a balanced binary tree is used to arrange the encoded non-keyword and keyword information in an ordered manner. The nodes of the balanced binary tree are divided into keyword nodes and encoding nodes. Keyword nodes store keyword information, and encoding nodes store Huffman-coded non-keyword information. Nodes are linked through pointers to ensure the orderliness and recoverability of the arrangement. Finally, the ordered structured encoded data is converted into binary data using a Base64 transcoding algorithm. The binary data is stored in the format of "header identifier (4 bytes) + data length (4 bytes) + binary data + checksum (4 bytes)". The header identifier is fixed at 0x584D4C50 (corresponding to ASCII code "XMLP"), used to identify that the data type is XML compressed data; the data length indicates the number of bytes in the binary data; the checksum is generated using the CRC32 algorithm to verify the integrity of data storage and prevent data corruption.

[0061] Query Index Layer: A built-in inverted index structure is used. The inverted index table includes a keyword field, a start position field, and a length field. The keyword field stores the keywords extracted from the XML data; the start position field stores the starting offset (i.e., the number of bytes from the start position of the structured encoded data) of the Huffman-coded string corresponding to the keyword; and the length field stores the byte length of the Huffman-coded string. Through the inverted index table, the association between keywords and the start position of the Huffman-coded string can be quickly established, providing direct index support for reverse search queries.

[0062] Tests have shown that when storing XML data, the database type in this embodiment reduces storage space usage by 62% compared to traditional text-based XML databases, far exceeding the requirement of at least 50% savings.

[0063] Example 2: A compressed XML database system supporting efficient querying

[0064] The system in this embodiment is built based on the database type described in Embodiment 1, and includes a data acquisition module, a compression processing module, a storage module, an index building module, a query processing module, and a data recovery module. The specific implementation of each module is as follows:

[0065] Data Acquisition Module: Developed in Java, this module uses an XML parser (such as DOM4J) to acquire raw XML data for processing. It supports various acquisition methods, including local file reading and network interface calls (such as RESTful interfaces). After acquisition, the raw XML data undergoes preprocessing: duplicate XML data entries are removed using deduplication algorithms (such as MD5 hash-based deduplication); format standardization is performed to unify the indentation, tag case, and other formatting of the XML data, preventing format differences from affecting subsequent processing.

[0066] The compression module, also developed in Java, comprises a separation unit, an encoding unit, and a sorting unit. The separation unit uses regular expressions to match XML tags, separating key and non-keyword information. The encoding unit, based on the Huffman coding algorithm, first counts the frequency of each character in the non-keyword information, then constructs a Huffman tree and assigns codes, converting the non-keyword information into a Huffman-coded string. The sorting unit constructs a balanced binary tree, storing the Huffman-coded string and key information in corresponding nodes. Rotation operations (left and right rotations) ensure the balance of the binary tree, guaranteeing efficiency for subsequent queries and traversals.

[0067] Storage Module: Employs a disk array as the storage medium, supporting multiple interface types such as SATA and SAS. It receives structured encoded data from the compression module, converts it to binary data using a Base64 transcoding algorithm, and writes it to the disk in the format of "header identifier + data length + binary data + checksum". Simultaneously, it establishes a storage directory index to record the storage path and basic information (such as file size and creation time) of each XML data file, facilitating data management and maintenance.

[0068] Index Building Module: Based on the Lucene framework, an inverted index is constructed. Keywords separated by the compression processing module are extracted, and these keywords are associated with the start position and length information of their corresponding Huffman-coded strings to generate an inverted index table. The index table is stored using a B+ tree structure. The leaf nodes of the B+ tree store index entries, while non-leaf nodes store the index information of the keywords, facilitating rapid location of index entries. Simultaneously, an index update mechanism is established to update the inverted index table in real time when XML data is added, deleted, or modified, ensuring the accuracy of the index information.

[0069] The query processing module supports SQL-like and XPath queries. Upon receiving a user query request, it parses the query statement using a parser and extracts the query keywords. Based on the inverted index table generated by the index building module, it queries the starting position and length of the Huffman-coded string corresponding to the keyword. Starting from the starting position, it traces backward along the node-associated pointers of the balanced binary tree, sequentially obtaining the Huffman-coded data of each encoded node until it reaches the root node, thus integrating the complete encoded information. Since the starting position can be directly located through the inverted index, the reverse search process does not require traversing the entire data, resulting in a query complexity of O(1).

[0070] The data recovery module receives the complete encoded information output by the query processing module, decodes the encoded string using the Huffman decoding algorithm to recover non-keyword information, and then reassembles the decoded non-keyword information with the keyword information using the node association relationship of the balanced binary tree to restore the original XML data structure. Finally, the recovered XML data is converted into a user-readable format (such as formatted XML text) and fed back to the user through a user interface or API.

[0071] Example 3: A compressed XML database method supporting efficient querying

[0072] The method in this embodiment includes a storage phase and a query phase, and the specific steps are as follows:

[0073] S1: Storage Stage

[0074] S11: Data Acquisition and Preprocessing: Raw XML data is acquired through the data acquisition module, and the MD5 hash algorithm is used to deduplicate the raw data and delete duplicate entries; the XML data is formatted and standardized using the DOM4J parser, unifying the case of uppercase and lowercase tags to lowercase and standardizing the indentation to 4 spaces.

[0075] S12: Keyword and Non-Keyword Separation: The regular expression "<( / )?[a-zA-Z0-9_]+(>|\\s+.*?>)" is used to match XML tags. Matched tags, redundant spaces, etc., are treated as non-keyword information, while the remaining information used to identify the core meaning of the data is treated as keyword information. For example, for XML data...

[0076] “ <user> <id> 1001< / id> <name> ZhangSan< / name> < / user> The extracted keywords are "1001" and "ZhangSan", while the non-keyword information is "". <user> ”" <id> ”"< / id> ”" <name> ”"< / name> ”"< / user> ".

[0077] S13: Huffman Coding: Count the frequency of each character in non-keyword information. For example, the frequency of "<" is f=6, ">", "u", "s", "e", "r", "i", "d", "n", "a", and "m" is f=1. Based on these frequencies, construct a Huffman tree and select the two nodes with the lowest frequencies. Points (such as "u" and "s", both with a frequency of 1) are merged into new nodes with a frequency of 2; the merging process is repeated until the root node is obtained; codes are assigned to the branches of the Huffman tree (0 on the left and 1 on the right) to obtain the Huffman codes for each character; non-keyword information is converted into Huffman encoded strings based on the codes, with a string length L = 6 × 2 + 6 × 2 + 1 × 4 + ... + 1 × 4 = 48 bits, which is a compression ratio of 50% compared to the original ASCII encoding of non-keyword information (8 bits per character, 12 characters × 8 bits = 96 bits in total).

[0078] S14: Ordered arrangement of tree structure: Construct a balanced binary tree with "user", "id", and "name" as key nodes and the corresponding Huffman encoded strings as encoding nodes. Use key nodes as root nodes or child nodes, and associate encoding nodes with corresponding key nodes. Ensure the balance of the balanced binary tree through rotation operations to obtain structured encoded data.

[0079] S15: Binary Transcoding Storage: The structured encoded data is converted into binary data using the Base64 transcoding algorithm and stored in the disk array in the format of "header identifier (0x584D4C50) + data length (e.g., 48 bytes) + binary data + checksum (generated by CRC32)".

[0080] S16: Inverted Index Construction: Construct an inverted index table, enter the keywords "1001" and "ZhangSan", the starting position (e.g., 1024 bytes, 2048 bytes) and length (e.g., 24 bits, 24 bits) of the corresponding Huffman encoded string, and store the index table in the query index layer in a B+ tree structure.

[0081] S2: Query Phase

[0082] S21: Query Request Processing: The user inputs a query request "Query user information with id 1001", the query processing module parses the request and extracts the query keyword "1001".

[0083] S22: Index location: Based on the inverted index table, query the Huffman encoded string corresponding to the keyword "1001" starting at byte 1024 and having a length of 24 bits.

[0084] S23: Reverse search: Starting from 1024 bytes, trace back along the node association pointers of the balanced binary tree, obtain the encoded node data corresponding to "id" and the encoded node data corresponding to "user" in sequence, and integrate them to obtain the complete Huffman encoded information.

[0085] S24: Decoding and Recovery: Decode the complete encoded information using the Huffman decoding algorithm to recover the non-keyword information. <user> <id>< / id> <name>< / name> < / user> By combining the keyword information "1001" and "ZhangSan", the original XML data can be reconstructed and restored. <user> <id> 1001< / id> <name> ZhangSan< / name> < / user> ".

[0086] S25: Result Feedback: Display the recovered XML data to the user in formatted text format.

[0087] Tests showed that when querying large-scale XML data (10GB) using the method of this embodiment, a single query takes only 0.02 seconds and the query complexity is O(1); while the traditional sequential traversal query method takes 12.5 seconds and the query complexity is O(n) when querying the same amount of data. The query efficiency of this invention is significantly improved.

[0088] In summary, this invention significantly improves the storage efficiency and query performance of XML data, and is suitable for the management and application scenarios of large-scale XML data.

[0089] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A compressed XML database type that supports efficient querying, characterized in that, The database type adopts a layered architecture, including a compressed storage layer and a query index layer; The compressed storage layer is used to store XML data after encoding, sorting and transcoding. Specifically, Huffman coding is used to encode non-keyword information in the XML data, and the encoded information is arranged in a recoverable ordered manner through a tree structure. After the arrangement is completed, it is transcoded and stored in binary form. The query index layer has a built-in inverted index structure, which associates the keywords of the XML data with the starting position information of the corresponding Huffman code, and is used to provide index support for reverse search queries; The storage space required for this database type is reduced by at least 50% compared to traditional XML database storage methods, and the query complexity is O(1).

2. A compressed XML database system supporting efficient querying based on the database type described in claim 1, characterized in that, It includes a data acquisition module, a compression processing module, a storage module, an index building module, a query processing module, and a data recovery module; The data acquisition module is used to acquire the raw XML data to be processed and to preprocess the raw XML data, including deduplication and format standardization. The compression processing module is used to separate keyword information and non-keyword information from the preprocessed XML data, encode the non-keyword information using the Huffman coding algorithm to generate a Huffman coded string, and simultaneously arrange the encoded non-keyword information and keyword information in an orderly manner using a tree structure to obtain structured coded data. The storage module is used to transcode the structured coded data output by the compression processing module, convert it into binary data, and then store it on the disk. The index building module is used to build an inverted index, which associates and maps the keywords of the XML data with the starting position of the Huffman encoded string generated by the compression processing module, generates an index table, and stores it in the query index layer. The query processing module is used to receive user query requests, extract keywords from the query requests, locate the starting position of the Huffman encoded string corresponding to the keyword based on the index table, and use a reverse search algorithm to retrieve complete encoded information in the structured encoded data. The data recovery module is used to decode the complete encoded information retrieved by the query processing module, restore it to the original XML data structure, and then feed it back to the user.

3. The system according to claim 2, characterized in that, The tree structure used in the compression processing module is a balanced binary tree. The nodes of the balanced binary tree include keyword nodes and encoding nodes. Keyword nodes store keyword information of XML data, and encoding nodes store non-keyword information after Huffman encoding. The nodes are associated with each other through pointers to realize the ordered association between encoding information and keyword information.

4. The system according to claim 2, characterized in that, The inverted index table constructed by the index building module includes a keyword field, a start position field, and a length field. The keyword field stores the keyword of the XML data, the start position field stores the starting offset of the Huffman-coded string corresponding to the keyword in the structured encoded data, and the length field stores the length of the Huffman-coded string.

5. A compressed XML database method supporting efficient querying, characterized in that, It includes a storage phase and a query phase, and the specific steps are as follows: S1: Storage Stage S11: Collect raw XML data and perform deduplication and format standardization preprocessing on the raw XML data; S12: Separate key information from non-key information in the preprocessed XML data; S13: Use the Huffman coding algorithm to encode non-keyword information to generate a Huffman coded string; S14: Construct a balanced binary tree. Store the Huffman encoded string generated in step S13 and the keyword information separated in step S12 as node data into the balanced binary tree to achieve the ordered arrangement of encoded information and keyword information, and obtain structured encoded data. S15: Convert the structured coded data into binary data and store it on the disk; S16: Construct an inverted index table, associate the key with the starting position of the Huffman-coded string, and store the index table; S2: Query Phase S21: Receive user query requests and extract query keywords; S22: Based on the inverted index table, query the starting position of the Huffman-coded string corresponding to the keyword; S23: Starting from the initial position, a reverse search algorithm is used to search backwards in the structured coded data to obtain complete coded information; S24: Perform Huffman decoding on the complete encoded information to restore the original XML data structure; S25: Feed back the restored XML data to the user.

6. The method according to claim 5, characterized in that, The specific process of encoding non-keyword information using the Huffman coding algorithm in step S13 is as follows: S131: Calculate the frequency f of each character in the non-keyword information of XML, where i is the character index, i = 1, 2, ..., n, and n is the total number of different characters in the non-keyword information; S132: Based on each character and its frequency of occurrence, construct a Huffman tree. The construction of the Huffman tree satisfies the following condition: take the two nodes with the lowest frequency of occurrence as child nodes to construct a new node, and the frequency of the new node is the sum of the frequencies of the two child nodes. Repeat this process until all nodes are merged into a single root node; S133: Assign codes to each branch of the Huffman tree, specifying that the left branch is 0 and the right branch is 1. The path code from the root node to each character node is the Huffman code corresponding to that character. S134: Based on the Huffman coding of each character, convert the non-keyword information into the corresponding Huffman coded string, where the length L of the coded string satisfies: Where l is the length of the Huffman code corresponding to the i-th character.

7. The method according to claim 5, characterized in that, The balanced binary tree constructed in step S14 satisfies the condition that the absolute value of the height difference between the left and right subtrees does not exceed 1, and that the key of all nodes in the left subtree is less than the key of the root node, and the key of all nodes in the right subtree is greater than the key of the root node.

8. The method according to claim 5, characterized in that, The specific process of the reverse search algorithm in step S23 is as follows: taking the starting position located by the inverted index table as the endpoint, trace back along the node association pointers of the balanced binary tree, obtain the Huffman encoded data of each encoded node in turn, until the root node is reached, integrate all the traced encoded data to form complete encoded information.

9. The method according to claim 5, characterized in that, The process of transcoding to binary data in step S15 is as follows: the structured encoded data is converted into a binary stream using the Base64 transcoding algorithm. The storage format of the binary stream is "header identifier + data length + binary data + checksum". The header identifier is used to identify that the data type is XML compressed data, and the checksum is used to verify the integrity of the data storage process.

Images