Data compression method and device and data decompression method and device

Abstract

The invention discloses a data compression method and device and a data decompression method and device, and relates to the technical field of data storage. The data compression method comprises the steps of obtaining second data corresponding to first data, and then compressing the second data to obtain compressed data corresponding to the first data. The storage structure of the second data is a tree structure, and the bottommost layer node of the tree structure comprises a root-child node and a leaf-child node having an index relationship with the root-child node; the compressed data includes the data indicated by the root-child node and the compressed data indicated by the leaf-child node. When the data of the tree structure is queried, the data indicated by the child nodes are sequentially queried from the data indicated by the root nodes of the tree structure, and the data indicated by the root-child nodes are not compressed, so that when the computing device queries the data hierarchy indicated by the root-child nodes in the compressed data, the data hierarchy indicated by the root-child nodes in the compressed data is not compressed. And the compressed data does not need to be decompressed, so that the data query efficiency can be improved.

Description

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on November 9, 2023, with application number 202311489545.9 and application name “Data Compression Method and Computing Device”, all contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the technical field of data storage, and in particular to a data compression method, a data decompression method and a device. Background Art

[0003] Data compression technology has become a key technology in today's storage and other fields. In order to improve the query efficiency of the storage system for compressed data, the storage device can construct the data into a B-Tree or B+-Tree storage structure for storage, and then compress the data stored in the B-Tree or B+-Tree storage structure. For example, a prefix encoding algorithm that compresses the common prefix of the data, or a general compression algorithm that compresses the data as a whole. However, in the scheme using the above compression algorithm, when it is necessary to query specific data, the specific location of the specific data in the compressed data cannot be known, resulting in the need to decompress all the compressed data in order to query the specific data, and the query efficiency is low. Summary of the invention

[0004] The present application provides a data compression method, a data decompression method and a device to solve the problem that when specific data needs to be queried, the specific location of the specific data in the compressed data cannot be known, resulting in the need to decompress all the compressed data in order to query the specific data, resulting in low query efficiency.

[0005] In the first aspect, the present application provides a data compression method, which can be applied to a computer system or to a computing device that supports the computer system to implement the data compression method, for example, the computing device is a storage device. Here, the data compression method provided by the present embodiment is described as an example in which a computing device executes the data compression method, and the data compression method includes: obtaining second data corresponding to the first data, and then compressing the second data to obtain compressed data corresponding to the first data. Among them, the storage structure of the second data is a tree structure, and the bottom-level nodes of the tree structure include a root-child node and a leaf-child node having an index relationship with the root-child node, the first data includes the root-child node and the data indicated by the leaf-child node, and the compressed data includes the data indicated by the root-child node and the data indicated by the compressed leaf-child node.

[0006] Since when querying the data of the tree structure, the data indicated by the root node of the tree structure is queried in sequence to the data indicated by the child node, and in the present application, the data indicated by the root-child node is not compressed, therefore, when the data level indicated by the root-child node in the compressed data is queried, the compressed data does not need to be decompressed, which can improve the query efficiency of the data. In addition, the data indicated by the leaf-child node is compressed, which reduces the storage space occupied by the data and improves the utilization rate of the storage space.

[0007] In a possible scenario, the tree structure may be a B+-Tree.

[0008] In a possible scenario, the data format of the first data is specifically a key-value pair Key-Value, and the data indicated by the leaf-child node is stored in the storage area of ​​the leaf-child node in sequence according to the Key.

[0009] In the present application, storing data in sequence according to Key is conducive to determining that the data to be decompressed belongs to the Nth group according to the data indicated by the root-child node, and then decompressing the N groups to obtain the data to be decompressed, which can improve the query efficiency of the data.

[0010] In a possible scenario, the storage structure of the first data may be one or more of the following: a tree structure, a hash table, and a skip table.

[0011] In a possible implementation, the storage area of ​​the root-child node is used to store the first sub-data that meets the condition, and the storage area of ​​the leaf-child node is used to store the second sub-data that does not meet the condition. The first data includes the first sub-data and the second sub-data.

[0012] Exemplarily, first sub-data satisfying the condition is determined from the first data, and the first sub-data is migrated to the storage area of ​​the root-child node. Also, second sub-data satisfying the condition is determined from the first data, and the second sub-data is migrated to the storage area of ​​the leaf-child node.

[0013] In the present application, data of different attributes are respectively allocated to the storage area corresponding to the root-child node or the leaf-child node according to the conditions, and then when the second data is compressed, the data indicated by the root-child node will not be compressed, and the data indicated by the leaf-child node will be compressed. Therefore, when the data level indicated by the root-child node in the compressed data is queried, the compressed data does not need to be decompressed, which can improve the query efficiency of the data. In addition, compressing the data indicated by the leaf-child node reduces the storage space occupied by the data and improves the utilization rate of the storage space.

[0014] In a possible scenario, the above condition is used to indicate that the access index of the data is greater than or equal to a threshold.

[0015] Exemplarily, the access index includes: access frequency or access frequency. Furthermore, the first sub-data can be used to indicate hot data, and the second sub-data can be used to indicate cold data.

[0016] In this application, the hot data and cold data in the data indicated by the bottom-level node are determined based on the access index, and the hot data is migrated to the storage area of ​​the root-child node without compression. Furthermore, when the Key-Value is stored in the storage area of ​​the root-child node, since the data level indicated by the root-child node in the compressed data is queried, the compressed data does not need to be decompressed, so the Key-Value can be directly obtained, which improves the efficiency of querying hot data.

[0017] Exemplarily, the access frequency or access frequency of each data in the first data is determined from the metadata corresponding to the first data, and then the first sub-data whose access frequency or access frequency meets the threshold and the second sub-data whose access frequency or access frequency does not meet the threshold are determined.

[0018] In another possible scenario, the interval between the identification numbers of two adjacent data indicated by the root-child node is K plus 1, where K is an integer greater than or equal to 1.

[0019] Exemplarily, K may indicate the number of second sub-data included in a set of second sub-data, that is, the amount of data. The above identification number may be Key.

[0020] In the present application, by limiting the identification numbers of two adjacent data in the storage area of ​​the root-child node, the data in the root-child node can regularly indicate the data in the leaf-child node, so that the data in the leaf-child node can be indexed faster based on the data in the root-child node, thereby improving the efficiency of data query.

[0021] Exemplarily, K satisfies the following formula: K(K+X) ≥ N. Where X is a constant and N is the amount of data included in a bottom-level node.

[0022] In a possible implementation, the storage area of ​​the bottom-level node in the compressed data is also used to store: the right boundary or the left and right boundaries of each group of data indicated by the leaf-child node in the storage area of ​​the leaf-child node. The left boundary of the first group of data is used to indicate the minimum byte length from the first address of the storage area corresponding to the leaf-child node to the location where the first group of data is stored, and the right boundary of the first group of data is used to indicate the maximum byte length from the first address of the storage area corresponding to the leaf-child node to the location where the first group of data is stored. The first group of data is any group of data indicated by the leaf-child node.

[0023] In a possible scenario, the minimum byte length and the maximum byte length of the first set of data match the Key in the first set of data.

[0024] In a possible scenario, the data indicated by the root-child node indicates the right boundary or the left and right boundaries of each group of data.

[0025] Exemplarily, since there is an index relationship between the root-child node and the leaf-child node, and the minimum byte length and the maximum byte length of the first group of data match the Key in the first group of data, the right boundary or the left and right boundaries of each group of data in the leaf-child node can be determined based on the data in the root-child node.

[0026] In the present application, when determining that data belongs to the Nth group of data from the index relationship between the root-child node and the leaf-child node, the right boundary or left boundary of the Nth group of data can be determined based on the right boundary or left boundary of each group of data stored above. Then, the right boundary or left boundary of the Nth group of data is passed to the compression engine for decompression, avoiding decompression of the entire bottom-level node, reducing the decompression area passed to the compression engine, thereby significantly improving the decompression efficiency and increasing the speed of data query.

[0027] In the second aspect, the present application provides a data decompression method, which can be applied to a computer system or to a computing device that supports the computer system to implement the data decompression method, for example, the computing device is a storage device. Here, the data decompression method provided by the present embodiment is described as an example in which a computing device executes the data decompression method, and the data decompression method includes: obtaining a data decompression request including an identifier of the first data, and querying the compressed data according to the identifier of the first data. Then, a target storage area in the compressed data that matches the identifier of the first data is determined, and the data in the target storage area is decompressed to obtain the first data. Among them, the storage structure of the compressed data is a tree structure, and the bottom-level nodes of the tree structure include a root-child node and a leaf-child node having an index relationship with the root-child node. The compressed data includes the data indicated by the root-child node and the data indicated by the compressed leaf-child node, and the target storage area is the storage area of ​​the leaf-child node.

[0028] Since when querying the data of the tree structure, the data indicated by the root node of the tree structure is queried in sequence to the data indicated by the child node, and in this application, the data indicated by the root-child node is not compressed, therefore, when the data level indicated by the root-child node in the compressed data is queried, the compressed data does not need to be decompressed, which can improve the query efficiency of the data. In addition, when querying data, according to the root-child node and the index relationship, the smaller area (target storage area) where the data to be queried is located in the storage area of ​​the leaf-child node can be accurately located, and then only the compressed data of the smaller area needs to be decompressed, and all the data does not need to be decompressed, which reduces the amount of decompressed compressed data, thereby further improving the query efficiency of the data.

[0029] In a possible scenario, the data format of the first data is a key-value pair Key-Value, and the data indicated by the leaf-child node is stored in the storage area of ​​the leaf-child node in sequence according to the Key.

[0030] In one possible scenario, the tree structure comprises a B-tree.

[0031] In a possible implementation, the storage area of ​​the bottom-level node of the compressed data is also used to store: the right boundary or the left and right boundaries of each group of data indicated by the leaf-child node in the storage area of ​​the leaf-child node. The left boundary of the first group of data is used to indicate the minimum byte length from the first address of the storage area corresponding to the leaf-child node to the location where the first group of data is stored, and the right boundary of the first group of data is used to indicate the maximum byte length from the first address of the storage area corresponding to the leaf-child node to the location where the first group of data is stored. The first group of data is any group of data indicated by the leaf-child node.

[0032] In a possible scenario, the data indicated by the root-child node indicates the right boundary or the left and right boundaries of each group of data.

[0033] In a possible implementation, determining a target storage area in the compressed data that matches an identifier of the first data includes: querying from the root node of the compressed data to the bottom-level node in sequence according to the identifier of the first data, and then comparing the identifier of the first data with the data in the root-child node to determine the right boundary or the left and right boundaries of the second group of data. The first data or the address information of the first data is stored in the storage area of ​​the leaf-child node in the bottom-level node. The second group of data includes the first data, and the right boundary or the left and right boundaries of the second group of data are used to indicate the target storage area.

[0034] In the present application, since there is an index relationship between the root-child node and the leaf-child node, the smaller area (that is, the target storage area) where the first data is located in the storage area of ​​the leaf-child node can be accurately located, and then only the compressed data in the smaller area needs to be decompressed, without decompressing all the data, thereby reducing the amount of decompressed compressed data and improving the efficiency of data queries.

[0035] In one possible implementation, the identifier of the first data is compared with the data in the root-child node to determine the right boundary or the left and right boundaries of the second group of data, including: determining the first identifier and / or the second identifier of the data indicated by the root-child node that satisfies the first strategy with the identifier of the first data. Then, from the index relationship, determine the second group of data in the leaf-child node corresponding to the first identifier and / or the second identifier, so as to obtain the right boundary or the left and right boundaries of the second group of data in the storage area of ​​the leaf-child node. Among them, the first strategy is used to indicate: the identifier of the first data is less than the first identifier, or the identifier of the first data is greater than the second identifier, or the identifier of the first data is greater than or equal to the first identifier and less than or equal to the second identifier, and the first identifier is less than the second identifier.

[0036] In the present application, the identifier indicated by the root-child node is queried according to the identifier of the first data, thereby determining the first identifier and / or the second identifier. Since there is an index relationship between the root-child node and the leaf-child node, the smaller area (that is, the target storage area) where the first data is located in the storage area of ​​the leaf-child node is accurately located, and then only the compressed data in the smaller area needs to be decompressed, without decompressing all the data, thereby reducing the amount of decompressed compressed data, thereby improving the efficiency of data query.

[0037] In a possible implementation, the storage area of ​​the root-child node is used to store the first sub-data that meets the condition, and the storage area of ​​the leaf-child node is used to store the second sub-data that does not meet the condition. The first data includes the first sub-data and the second sub-data.

[0038] In one possible scenario, the condition is used to indicate that an access index of data is greater than or equal to a threshold.

[0039] Exemplarily, the access index includes: access frequency or access frequency; the first sub-data is used to indicate hot data, and the second sub-data is used to indicate cold data.

[0040] In another possible scenario, the interval between the identification numbers of two adjacent data indicated by the root-child node is K plus 1, where K is an integer greater than or equal to 1.

[0041] Exemplarily, K satisfies the following formula, K(K+X) ≥ N. Where X is a constant and N is the amount of data included in a bottom-level node.

[0042] For more details of the data decompression method provided in this aspect, please refer to the description of the first aspect above, which will not be repeated here.

[0043] In a third aspect, the present application provides a data compression device. The data compression device is applied to a computer system or a computing device that supports the computer system to implement a data compression method, and the data compression device includes various modules for executing the data compression method in the first aspect or any optional implementation of the first aspect. The data compression device includes:

[0044] The acquisition module is used to acquire the second data corresponding to the first data. The storage structure of the second data is a tree structure, and the bottom-level nodes of the tree structure include a root-child node and a leaf-child node having an index relationship with the root-child node; the first data includes the root-child node and the data indicated by the leaf-child node.

[0045] The compression module is used to compress the second data to obtain compressed data corresponding to the first data, wherein the compressed data includes data indicated by the root-child node and data indicated by the compressed leaf-child node.

[0046] The data compression device provided in this aspect may include a module for executing the method of any one of the implementation modes in the first aspect.

[0047] In a fourth aspect, the present application provides a data decompression device. The data decompression device is applied to a computer system or a computing device that supports the computer system to implement a data decompression method, and the data decompression device includes various modules for executing the data decompression method in the first aspect or any optional implementation of the first aspect. The data decompression device includes:

[0048] The acquisition module is used to acquire a data decompression request, where the data decompression request includes an identifier of the first data.

[0049] The query module is used to query the compressed data according to the identifier of the first data. The storage structure of the compressed data is a tree structure, the bottom-level nodes of the tree structure include a root-child node and a leaf-child node having an index relationship with the root-child node, and the compressed data includes data indicated by the root-child node and data indicated by the compressed leaf-child node.

[0050] The determination module is used to determine a target storage area in the compressed data that matches the identifier of the first data, where the target storage area is a storage area of ​​a leaf-child node.

[0051] The decompression module is used to decompress the data in the target storage area to obtain the first data.

[0052] The data decompression device provided in this aspect may include a module for executing the method of any one of the implementation modes in the second aspect.

[0053] In a fifth aspect, the present application provides a computing device. The computing device includes a memory and a processor, the memory is used to store instructions; the processor executes the instructions to implement the method in the first aspect or any possible implementation of the first aspect. For more details of the computing device in this aspect, please refer to the description of the storage device or processing device below, which will not be repeated here.

[0054] In a sixth aspect, the present application provides a computer-readable storage medium. The storage medium stores a computer program or instruction, and when the computer program or instruction is executed by a computing device, the method in the first aspect or any optional implementation of the first aspect is implemented.

[0055] In a seventh aspect, the present application provides a computer program product. The computer program product includes a computer program or instructions, and when the computer program or instructions are executed by a computing device, the method in the first aspect or any optional implementation of the first aspect is implemented.

[0056] The beneficial effects of the second to fifth aspects above can refer to the description of the first aspect or any implementation of the first aspect, and will not be repeated here. Based on the implementations provided by the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A structural diagram of a data storage system provided by this application;

[0058] Figure 2 A flowchart of the data compression method provided in this application;

[0059] Figure 3 A schematic diagram of the storage structure of the bottom-level nodes of the first data and the second data provided in this application;

[0060] Figure 4 Schematic diagram of the structure of the B+-tree based on HOT provided in this application Figure 1 ;

[0061] Figure 5 Schematic diagram of the structure of the B+-tree based on HOT provided in this application Figure 2 ;

[0062] Figure 6 The schematic diagram of the IOT structure provided for this application;

[0063] Figure 7 A flowchart of the data decompression method provided in this application;

[0064] Figure 8A schematic diagram of the structure of a data compression device provided by the present application;

[0065] Fig. 9 A schematic diagram of the structure of a data decompression device provided in this application. DETAILED DESCRIPTION

[0066] In the field of data storage, in order to improve the efficiency of data query, data indexing is often used to quickly query the target data. For example, in the database, tree structure indexing is often used. Since the data corresponding to the entire tree structure often occupies a large storage space, the data corresponding to the tree structure can be compressed through data compression technology to reduce the storage space occupied. However, before querying or using the compressed data, the compressed data needs to be decompressed, resulting in device performance loss.

[0067] In order to reduce the device performance loss caused by the need to decompress the compressed data before querying or using the compressed data, two possible processing solutions are provided below.

[0068] Processing solution 1, compressing data using lightweight data encoding, which may include prefix encoding (also called prefix encoding algorithm), run length encoding (RLE), suffix encoding, and the like.

[0069] The following uses the prefix encoding in lightweight data encoding to compress data as an example: according to the order of data in the tree structure, the common prefix of adjacent data is extracted and only stored in one copy, and the remaining suffix parts are stored separately, thereby achieving the purpose of data compression.

[0070] However, when querying specific data in the compressed data obtained by the above prefix encoding, since the compressed data is divided into a common prefix and a suffix part, the data is incomplete, and the specific position of the specific data in the compressed data cannot be determined. Therefore, it is necessary to decompress all the compressed data before querying to obtain the specific data, resulting in low query efficiency for the specific data. The above decompression of the compressed data is to splice the above common prefix and suffix parts to obtain the complete data.

[0071] The above scheme has low query efficiency for compressed data, and the compression rate of data is unstable because the proportion of common prefix and suffix parts of data changes according to the similarity between data. In other words, the compression rate of data is directly related to the distribution of data.

[0072] Processing solution 2: compressing data using a general compression algorithm, which may include LZ77 (Lempel-Ziv 77).

[0073] LZ77 usually compresses all data within a single node in a tree structure index. Compared with the above processing scheme 1, the general compression algorithm can provide a more stable compression rate, thereby significantly reducing the storage space occupied by the compressed data, thereby reducing storage costs.

[0074] However, when querying specific data in the compressed data obtained using the above-mentioned LZ77, the specific location of the specific data in the compressed data is still unknown, and the data in the entire index node needs to be decompressed to query the aforementioned specific data, resulting in low query efficiency for specific data.

[0075] Based on Solution 2, a general compression algorithm is used to selectively compress data in the tree structure index. For example, data in relatively inactive index nodes is compressed, and data in relatively active index nodes is not compressed.

[0076] However, when the specific data to be queried is located in a relatively inactive index node, the data in the entire relatively inactive index node must still be decompressed in order to query the aforementioned specific data, resulting in low query efficiency for the specific data.

[0077] Based on this, the present application provides a data processing method. In the method, the data is compressed data, the storage structure of the compressed data is a tree structure, and the bottom-level nodes of the tree structure include a root-child node and a leaf-child node having an index relationship with the root-child node, and the compressed data includes the data indicated by the root-child node and the compressed data indicated by the leaf-child node. The method includes: compressing the data to obtain the above-mentioned compressed data. Alternatively, the method includes: decompressing the above-mentioned compressed data to obtain the data indicated by the data decompression request.

[0078] For example, the data processing method is a data compression method. The data compression method includes: obtaining second data corresponding to the first data, and then compressing the second data to obtain compressed data corresponding to the first data. The storage structure of the second data is consistent with the storage structure of the compressed data. The first data includes data indicated by the root-child node and the leaf-child node.

[0079] For another example, the data processing method is a data decompression method. The data decompression method includes: obtaining a data decompression request including an identifier of the first data, querying the decompressed data according to the identifier of the first data. Then, determining a target storage area in the compressed data that matches the identifier of the first data, decompressing the data in the target storage area, and obtaining the first data. The target storage area is a storage area of ​​a leaf-child node.

[0080] Since when querying the data of the tree structure, the data indicated by the root node of the tree structure is queried in sequence to the data indicated by the child node, and in this application, the data indicated by the root-child node is not compressed, therefore, when the data level indicated by the root-child node in the compressed data is queried, the compressed data does not need to be decompressed, which can improve the query efficiency of the data. In addition, when querying data, according to the root-child node and the index relationship, the smaller area (target storage area) where the data to be queried is located in the storage area of ​​the leaf-child node can be accurately located, and then only the compressed data of the smaller area needs to be decompressed, and all the data does not need to be decompressed, which reduces the amount of decompressed compressed data, thereby further improving the query efficiency of the data.

[0081] The terms used in the implementation method of this application are only used to explain the specific embodiments of this application, and are not intended to limit this application. The following is a brief introduction to some concepts that may be involved in this application.

[0082] B-Tree is a multi-branch balanced tree. The storage area corresponding to B-Tree stores an ordered set of keys. B-Tree consists of multiple index nodes, which can be divided into at least two layers. Each index node stores multiple keys. When querying data in B-Tree, based on the given key, starting from the top index node of B-Tree, binary search is performed layer by layer until the bottom index node is queried.

[0083] B+-Tree, the difference between B+-Tree and B-Tree is that in B+-Tree, all keys are only stored in the bottom-level index nodes, and the middle index nodes only store keys used for indexing.

[0084] An index node is a physical storage unit for data. A B+-Tree or B-Tree consists of multiple index nodes. An index node is also called an index block or index page, and its size is fixed, usually 8KB or 16KB.

[0085] The following describes the application scenarios of the above data compression method and data decompression method in conjunction with the accompanying drawings. Figure 1 As shown, Figure 1 1 is a schematic diagram of a data storage system provided in the present application. The data storage system includes a host 110 and a storage device 120. Figure 1 In the application scenario shown, users access data through applications. The computer running these applications can be called a "host". The host 110 can be a physical machine or a virtual machine. Physical machines include but are not limited to desktop computers, servers, laptops, and mobile devices.

[0086] In a possible example, the host 110 accesses the storage device 120 through a network to access data. For example, the network may include a switch 130 .

[0087] In another possible example, the host 110 may also communicate with the storage device 120 through a wired connection, such as a universal serial bus (USB) or a peripheral component interconnect express (PCIe) bus.

[0088] Figure 1 The storage device 120 shown may be a centralized storage system. The centralized storage system is characterized by having a unified entrance through which all data from external devices must pass, and this entrance is the engine 121 of the centralized storage system. The engine 121 is the core component of the centralized storage system, and many advanced functions of the storage system are implemented in it.

[0089] like Figure 1 As shown, the engine 121 may have one or more controllers. Figure 1 The example of the engine 121 including one controller is used for explanation. In a possible example, if the engine 121 has multiple controllers, any two controllers may have a mirror channel to achieve the function of any two controllers backing up each other, thereby avoiding the unavailability of the entire storage device 120 due to hardware failure. It should be understood that if the engine 121 includes multiple controllers, the engine 121 may also be called an array controller of the storage device 120.

[0090] The engine 121 further includes a front-end interface 1211 and a back-end interface 1214, wherein the front-end interface 1211 is used to communicate with the host 110, thereby providing data access services for the host 110. The back-end interface 1214 is used to communicate with the hard disk to expand the capacity of the storage device 120. Through the back-end interface 1214, the engine 121 can connect more hard disks to form a very large storage resource pool.

[0091] In hardware, such as Figure 1As shown, the controller includes at least a processor 1212 and a memory 1213. The processor 1212 is a central processing unit (CPU) for processing data access requests from outside the storage device 120 (server or other storage system), and is also used to process requests generated inside the storage device 120. Exemplarily, when the processor 1212 receives a data compression request or a data decompression request sent by the host 110 through the front-end interface 1211, the data in these data compression requests will be temporarily stored in the memory 1213. When the total amount of data in the memory 1213 reaches a certain threshold, the processor 1212 sends the data stored in the memory 1213 to at least one of the mechanical hard disk 1221, the mechanical hard disk 1222, the solid state drive (SSD) 1223 or other hard disks 1224 through the back-end port for persistent storage.

[0092] Memory 1213 refers to an internal memory that directly exchanges data with the processor. It can read and write data at any time and at a very fast speed. It serves as a temporary data storage for the operating system or other running programs. The memory includes at least two types of memories. For example, the memory can be either a random access memory or a read-only memory (ROM). For example, the random access memory is DRAM or SCM. DRAM is a semiconductor memory, and like most random access memories (RAM), it is a volatile memory device. However, DRAM and SCM are only exemplary descriptions in this embodiment, and the memory can also include other random access memories, such as static random access memory (SRAM), etc. As for the read-only memory, for example, it can be a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), etc.

[0093] In addition, the memory 1213 may also be a dual in-line memory module or a dual in-line memory module (DIMM), that is, a module composed of a dynamic random access memory (DRAM), or an SSD. In practical applications, multiple memories 1213 and different types of memories 1213 may be configured in the controller. This embodiment does not limit the number and type of memory 1213. In addition, the memory 1213 may be configured to have a power-saving function. The power-saving function means that when the system loses power and then powers on again, the data stored in the memory 1213 will not be lost. A memory with a power-saving function is called a non-volatile memory.

[0094] The memory 1213 stores a software program, and the processor 1212 runs the software program in the memory 1213 to implement writing and reading of compressed data. For example, according to the first address space indicated by the read request and the preset address mapping table, the second compressed data is read from the second address space, and the second compressed data is decompressed according to the compression information indicated by the read request to obtain the first compressed data.

[0095] like Figure 1 As shown, in this system, the engine 121 may not have a hard disk slot, and the hard disk needs to be placed in the hard disk frame 122, and the back-end interface 1214 communicates with the hard disk frame 122. The back-end interface 1214 exists in the form of an adapter card in the engine 121, and two or more back-end interfaces 1214 can be used simultaneously on one engine 121 to connect multiple hard disk frames. Alternatively, the adapter card can also be integrated on the motherboard, in which case the adapter card can communicate with the processor 1212 via the PCIe bus.

[0096] It should be noted that Figure 1 Only one engine 121 is shown in the figure. However, in actual applications, the storage system may include two or more engines 121, and redundancy or load balancing is performed between the multiple engines 121.

[0097] The hard disk enclosure 122 includes a control unit 1225 and a plurality of hard disks. The control unit 1225 may have various forms. In one case, the hard disk enclosure 122 is an intelligent disk enclosure, such as Figure 1As shown, the control unit 1225 includes a CPU and a memory. The CPU is used to perform operations such as address conversion and reading and writing data. The memory is used to temporarily store data to be written to the hard disk, or read from the hard disk to send data to the controller. In another case, the control unit 1225 is a programmable electronic component, such as a data processing unit (DPU). The DPU has the versatility and programmability of the CPU, but is more specialized and can run efficiently on network data packets, storage requests or analysis requests. The DPU is distinguished from the CPU by a large degree of parallelism (needing to process a large number of requests). Optionally, the DPU here can also be replaced by a processing chip such as a graphics processing unit (GPU) and an embedded neural network processor (NPU). Generally, the number of control units 1225 can be one, or two or more. The functions of the control unit 1225 can be offloaded to the network card 1226. In other words, in this embodiment, the hard disk frame 122 does not have a control unit 1225 inside, but the network card 1226 completes data reading and writing, compression, decompression and other computing functions. At this time, the network card 1226 is an intelligent network card. It can include a CPU and a memory. The CPU is used to perform operations such as compression, decompression, and reading and writing data. The memory is used to temporarily store data to be written to the hard disk, or read from the hard disk to be sent to the controller. It can also be a programmable electronic component, such as a DPU. There is no ownership relationship between the network card 1226 and the hard disk in the hard disk frame 122. The network card 1226 can access any hard disk in the hard disk frame 122 (such as Figure 1 The mechanical hard disk 1221, mechanical hard disk 1222, solid state disk 1223 and other hard disks 1224 are shown, so it is more convenient to expand the hard disk when the storage space is insufficient.

[0098] According to the type of communication protocol between the engine 121 and the hard disk frame 122, the hard disk frame 122 may be a serial attached small computer system interface (SAS) hard disk frame, or it may be an NVMe (Non-Volatile Memory express) hard disk frame and other types of hard disk frames. The SAS hard disk frame adopts the SAS3.0 protocol, and each frame supports 25 SAS hard disks. The engine 121 is connected to the hard disk frame 122 through the onboard SAS interface or the SAS interface module. The NVMe hard disk frame is more like a complete computer system, and the NVMe hard disk is inserted in the NVMe hard disk frame. The NVMe hard disk frame is then connected to the engine 121 through the RDMA port.

[0099] In an optional implementation, the storage device 120 is a centralized storage system with integrated disk and controller, and the storage device 120 does not have the above-mentioned hard disk frame 122, and the engine 121 is used to manage multiple hard disks connected through the hard disk slots. The function of the hard disk slots can be implemented by the back-end interface 1214.

[0100] In another optional implementation, Figure 1 The storage device 120 shown is a distributed storage system, which includes a computing device cluster and a storage device cluster. The computing device cluster includes one or more computing devices, and each computing device can communicate with each other. The computing device can be a physical computing device, such as a server, a desktop computer, or a controller of a storage array. In terms of hardware, the computing device may include a processor, a memory, and a network card. Among them, the processor is a CPU, which is used to process data access requests from outside the computing device, or requests generated inside the computing device. Exemplarily, when the processor receives a write data request sent by a user, the data in these write data requests will be temporarily stored in the memory. When the total amount of data in the memory reaches a certain threshold, the processor sends the data stored in the memory to the storage device for persistent storage. In addition, the processor is also used for data calculation or processing, such as metadata management, deduplication, data compression, decompression, virtualized storage space, and address conversion. In an example, any computing device can access any storage device in the storage device cluster through a network. The storage device cluster includes multiple storage devices. A storage device includes one or more controllers, a network card, and multiple hard disks, and the network card is used to communicate with the computing device.

[0101] Exemplarily, the storage structure of the data stored in the storage device 120 may be a tree structure, such as a B-Tree or a B+-Tree.

[0102] For example, in Figure 1In a data storage system, a user creates a compressed file in an industry standard format (referred to as first compressed data) on a host 110, such as a JPEG (Joint Photographic Experts Group) file. The storage device 120 compresses the JPEG file using a preset compression algorithm to obtain a file in a private format (referred to as second compressed data), and the storage device 120 stores the second compressed data in the storage device. The file in the private format refers to the format of the file obtained after the storage device 120 compresses the JPEG file according to a preset compression algorithm. When the user initiates a data decompression request on the host 110, the storage device 120 obtains the corresponding second compressed data according to the data decompression request, and decompresses the second compressed data to obtain the above-mentioned first compressed data. Finally, the storage device 120 returns the first compressed data to the host 110.

[0103] In the present application, the storage device 120 compresses and decompresses data, which can be processed by a compression engine in the storage device, and the compression engine can be a compression engine that supports a high compression rate. The function of the compression engine can be implemented by the processor 1212 in the storage device 120. For example, the compression engine can support the use of the above-mentioned general compression algorithm (such as LZ77) or a lightweight compression algorithm to compress data.

[0104] The following describes how the processing device compresses or decompresses data.

[0105] The first possible scenario: Figure 1 Based on the data storage system shown, a possible implementation method of compressing data in a processing device is provided, such as Figure 2 As shown, Figure 2 The content shown in this embodiment can be executed by the processing device 200, which can be Figure 1 The storage device 120 in FIG. Figure 2 , the data compression method provided in this embodiment includes steps S210 to S230.

[0106] S210: The processing device 200 obtains first data.

[0107] The storage structure of the first data may be a tree structure, such as a B+-Tree or a B-Tree. The data included in each index node in the tree structure is arranged in sequence. For example, the data is arranged in sequence according to the value of the Key in the data. The data format of the first data may be a key-value pair (Key-Value) form.

[0108] In other examples of the present application, the storage structure of the first data may also be a hash table. In the hash table, a hash function is used to map the key to a bucket (hash bucket), and the key-value pair is stored in the bucket. Through the calculation of the hash function, a specific Key-Value can be quickly found and accessed.

[0109] The storage structure of the first data described above is only an example provided in the present application and should not be construed as a limitation of the present application. In other examples in the present application, the storage structure of data that can store the Key-Value format can be the storage structure of the first data. For example, a skip list. Alternatively, the storage structure of the first data may not be limited, and the format of the first data only needs to be Key-Value.

[0110] In a possible implementation, the processing device 200 obtains the first data, including: the processing device 200 obtains the first data from the mechanical hard disk 1221, the mechanical hard disk 1222, the solid state disk 1223 or the other hard disk 1224. In other words, the processing device 200 obtains the first data locally.

[0111] For example, the processing device 200 obtains a data compression request a sent by the host 110, the data compression request carrying the specific location of the first data in the hard disk, such as a logical block address (LBA). The processing device 200 obtains the first data from the hard disk according to the LBA.

[0112] In a possible implementation manner, the processing device 200 obtains the first data, including: the processing device 200 obtains the first data sent by the host 110 .

[0113] For example, the processing device 200 obtains a data compression request forwarded by the host 110 through the switch 130 , where the data compression request carries the first data.

[0114] S220: The processing device 200 obtains second data corresponding to the first data.

[0115] The storage structure of the second data is a tree structure, and the bottom-level nodes of the tree structure include a root-child node and a leaf-child node having an index relationship with the root-child node. The first data includes the root-child node and the data indicated by the leaf-child node.

[0116] In a possible scenario, the tree structure may be a B+-Tree.

[0117] like Figure 2As shown, the storage structure of the second data is a B+-Tree, which includes multiple index nodes. The multiple index nodes can be divided into at least two layers. The data (Value) corresponding to all keys is only stored in the bottom index node (bottom node), and the middle index node only stores part of the key.

[0118] Exemplarily, the storage area corresponding to the bottom-level node includes storage data corresponding to the root-child node and the leaf-child node.

[0119] In a possible implementation, the processing device 200 obtains the second data corresponding to the first data, including: the processing device 200 organizes the storage structure of the first data to obtain the second data.

[0120] Regarding the processing device 200 organizing the storage structure of the first data to obtain the content of the second data, three possible scenarios are provided below.

[0121] In a first possible scenario, the storage structure of the first data is a B+-Tree.

[0122] Since the overall storage structure of the first data and the second data is B+-Tree, the processing device 200 only needs to organize the data indicated by the bottom-level node in the first data to obtain the second data.

[0123] Take a bottom-level node (such as node a) among the multiple bottom-level nodes included in the first data as an example for description. The processing device 200 determines the first sub-data that meets the condition from the data indicated by node a, and migrates the first sub-data to the storage area of ​​the root-child node. And, the processing device 200 determines the second sub-data that meets the condition or does not meet the condition from the data indicated by node a, and migrates the second sub-data to the storage area of ​​the leaf-child node.

[0124] The first sub-data is arranged in order in the storage area of ​​the root-child node according to the key of the first sub-data, for example, K3 / K6. The second sub-data is arranged in order in the storage area of ​​the leaf-child node according to the key of the second sub-data, for example, K1 / K2 / K4 / K5 / K7 / K8.

[0125] In the present application, the processing device 200 allocates data of different attributes to the storage area corresponding to the root-child node or the leaf-child node according to the conditions, and then when compressing the second data, the data indicated by the root-child node will not be compressed, and the data indicated by the leaf-child node will be compressed. Therefore, when the processing device 200 queries the data level indicated by the root-child node in the compressed data, it does not need to decompress the compressed data, which can improve the query efficiency of the data. In addition, compressing the data indicated by the leaf-child node reduces the storage space occupied by the data and improves the utilization rate of the storage space.

[0126] In a possible example, the above condition is used to indicate that an access index of the data is greater than or equal to a threshold. The access index includes: access frequency or access frequency.

[0127] The processing device 200 determines the access frequency or access frequency of each data (Key-Value) from the metadata of node a, and then determines the first sub-data whose access frequency or access frequency meets the threshold, and the second sub-data whose access frequency or access frequency does not meet the threshold.

[0128] The first sub-data may be hot data, the second sub-data may be cold data, and the access frequency or the thresholds corresponding to the access frequency are different. For example, the access frequency corresponds to threshold a (such as 50 times), and the access frequency corresponds to threshold b (such as 0.2).

[0129] Exemplarily, the data indicated by node a includes K(Key)1-V(Value)1, K2-V2, K3-V3, K4-V4, K5-V5. The corresponding access frequencies of K1-V1, K2-V2, K3-V3, K4-V4, K5-V5 are 40, 20, 60, 20, 40 respectively. The processing device 200 compares the access frequencies corresponding to K1-V1, K2-V2, K3-V3, K4-V4, K5-V5 with the threshold a, and determines that K1-V1, K2-V2, K4-V4, K5-V5 are the second sub-data, and K3-V3 is the first sub-data. Furthermore, the processing device 300 migrates K2-V2 to the storage area corresponding to the root-child node, and migrates K1-V1, K2-V2, K4-V4, K5-V5 to the storage area corresponding to the leaf-child node.

[0130] It is worth noting that the second sub-data on the left and right sides of the Key of the first sub-data can be used as two groups of second sub-data, for example, K1-V1 and K2-V2 can be used as one group of second sub-data, and K4-V4 and K5-V5 can be used as another group of second sub-data.

[0131] In the present application, the processing device 200 determines the hot data and cold data in the data indicated by the bottom-level node according to the access index, and migrates the hot data to the storage area of ​​the root-child node without compression. Furthermore, when the Key-Value is stored in the storage area of ​​the root-child node, since the processing device 200 does not need to decompress the compressed data when querying the data level indicated by the root-child node in the compressed data, the Key-Value can be directly obtained, which improves the efficiency of the processing device 200 when querying hot data.

[0132] In a second possible example, the above condition is used to indicate that the interval between the identification numbers of two adjacent data indicated by the root-child node is K plus 1, where K is an integer greater than or equal to 1.

[0133] K may indicate the number of second sub-data included in a set of second sub-data, i.e., the amount of data. Since the data in the bottom-level node in the first data is arranged in sequence according to the Key, after the processing device 200 determines a set of second sub-data, it obtains the largest Key in the set of second sub-data, and uses the Key-Value corresponding to the largest Key+1 as the first sub-data. The above identification number is also the Key.

[0134] A possible calculation method is provided below for the above K. K satisfies the following formula: K(K+X) ≥ N. Where X is a constant and N is the amount of data included in a bottom-level node.

[0135] The above constants can be configured by the user, and X is an integer greater than or equal to 1. N is the number of Key-Values ​​included in a bottom-level node.

[0136] Exemplarily, K is the smallest positive integer that satisfies the above formula.

[0137] For example, Figure 3 As shown, Figure 3 A schematic diagram of the storage structure of the bottom-level nodes of the first data and the second data provided in this application. Figure 3 , a bottom-level node (node ​​b) among a plurality of bottom-level nodes included in the second data is shown in FIG. Figure 3 Figure a shows the arrangement of data in the bottom node of the first data, that is, the data in node b of the first data is arranged in order from small to large according to Key. The processing device 200 receives X configured by the user as 2, and determines that the amount of data in node b is 8, and then determines K as 2 according to the above formula.

[0138] Furthermore, if Figure 3 As shown in b, Figure 3Middle b shows the logical structure of the root-child node and the leaf-child node in the second data. The processing device 200 takes the data corresponding to K1 / K2 as a group of first sub-data, the data corresponding to K3 as the second sub-data, the data corresponding to K4 / K5 as another group of first sub-data, the data corresponding to K6 as the second sub-data and the data corresponding to K7 / K8 as another group of first sub-data.

[0139] Thus, if Figure 3 As shown in c, Figure 3 Figure c shows the storage structure of the compressed second data. The processing device 300 migrates the data corresponding to K3 and K6 to the storage area corresponding to the root-child node, and migrates the data corresponding to K1, K2, K3, K4, K7, and K8 to the storage area corresponding to the leaf-child node. In this storage area, the data is stored in sequence according to the Key, and the data corresponding to K1, K2, K3, K4, K7, and K8 are compressed.

[0140] In the present application, storing data in sequence according to Key is beneficial for the processing device 200 to determine that the data to be decompressed belongs to the Nth group according to the data indicated by the root-child node, and then decompress the N groups to obtain the data to be decompressed, which can improve the query efficiency of the data.

[0141] It is worth noting that the above Figure 3 As shown in the figure, only K3 and K6 in K3-V3 and K6-V6 are migrated to the storage area corresponding to the root-child node for storage, and the complete data of K3-V3 and K6-V6 are still stored in the storage area corresponding to the leaf-child node, which is not shown in the figure. Furthermore, the processing device 200 can use the data corresponding to K1, K2 and K3 as a group of first sub-data, and the data corresponding to K4, K5 and K6 as another group of first sub-data.

[0142] In a second possible scenario, the storage structure of the first data is a B-Tree.

[0143] The processing device 200 converts the storage structure of the first data from B-Tree to B+-Tree, and then organizes the data indicated by the bottom-level nodes in the B+-Tree to obtain the second data.

[0144] Since each layer of nodes in the B-Tree stores Key-Value, the processing device 200 needs to migrate the Key-Value stored in each layer of nodes to the bottom layer of nodes for storage, and only retain the Key used for indexing in the aforementioned layers of nodes.

[0145] Regarding the processing device 200 organizing the data indicated by the bottom-level node in the B+-Tree to obtain the content of the second data, reference may be made to the description in the first possible scenario above, which will not be described in detail here.

[0146] In a third possible scenario, the storage structure of the first data is a skip table.

[0147] The processing device 200 converts the storage structure of the first data from the skip list to the B+-Tree, and then organizes the data indicated by the bottom-level node in the B+-Tree to obtain the second data.

[0148] Exemplarily, the processing device 200 reads all the data in the skip list, and then stores the read data into the constructed B+-Tree.

[0149] Regarding the processing device 200 organizing the data indicated by the bottom-level node in the B+-Tree to obtain the content of the second data, reference may be made to the description in the first possible scenario above, which will not be described in detail here.

[0150] The above contents are only possible scenarios provided by the present application and should not be construed as limiting the present application. In other embodiments of the present application, the processing device 200 additionally constructs a B+-Tree according to the first data to index the first data.

[0151] S230: The processing device 200 compresses the second data to obtain compressed data corresponding to the first data.

[0152] The compressed data includes the data indicated by the root-child node and the compressed data indicated by the leaf-child node.

[0153] In a possible implementation, the processing device 200 compresses the second data to obtain compressed data corresponding to the first data, including: the processing device 200 only compresses the data indicated by the leaf-child node in the second data to obtain compressed data corresponding to the first data.

[0154] The processing device 200 may use a compression algorithm to compress the data indicated by the leaf-child node to obtain compressed data corresponding to the first data.

[0155] For example, the compressed data obtained by using the above compression algorithm can support partial decompression. The above compression algorithm can be an LZ77 algorithm, and the LZ77 algorithm can realize partial decompression based on the right boundary of the data.

[0156] In one possible scenario, the above compression algorithm can implement partial decompression based on the left and right boundaries of the data.

[0157] For the description of the left boundary or the right boundary, please refer to the content of the storage area of ​​the bottom-level node described below, which will not be described in detail here.

[0158] In a possible embodiment, the storage area of ​​the bottom-level node in the compressed data is also used to store: the right boundary or the left and right boundaries of each group of data indicated by the leaf-child node in the storage area of ​​the leaf-child node.

[0159] The left boundary of the first group of data is used to indicate the minimum byte length from the first address of the storage area corresponding to the leaf-child node to the location where the first group of data is stored. The right boundary of the first group of data is used to indicate the maximum byte length from the first address of the storage area corresponding to the leaf-child node to the location where the first group of data is stored, and the first group of data is any group of data indicated by the leaf-child node.

[0160] The minimum byte length and the maximum byte length of the first set of data match the Key in the first set of data.

[0161] In a possible scenario, the data indicated by the root-child node indicates the right boundary or the left and right boundaries of each group of data.

[0162] In a possible example, the storage area of ​​the above-mentioned bottom-level node stores the space (length) occupied by each data (Key-Value), and the right boundary of the Nth group of data in the multiple groups of data in the leaf-child node can be determined according to the sum of the space occupied by the Key-Value in the first N groups of data, and the left boundary of the Nth group of data in the multiple groups of data in the leaf-child node can be determined according to the sum of the space occupied by the Key-Value in the first N-1 groups of data.

[0163] Since there is an index relationship between the root-child node and the leaf-child node, and the minimum byte length and the maximum byte length of the first group of data match the Key in the first group of data, the processing device 200 can determine the right boundary or the left and right boundaries of each group of data in the leaf-child node based on the data in the root-child node.

[0164] Exemplarily, the data indicated by the root-child node is the data corresponding to K3 and K6, and the data indicated by the corresponding leaf-child node only includes three groups of data: K1 / K2, K4 / K5, and K7 / K8. The processing device 200 can determine the right boundary or left and right boundaries of the first group of data K1 / K2 in the leaf-child node based on K3 in the root-child node; can determine the right boundary or left and right boundaries of the second group of data K4 / K5 in the leaf-child node based on K3 and K6 in the root-child node; and can determine the right boundary or left and right boundaries of the third group of data K7 / K8 in the leaf-child node based on K6 in the root-child node.

[0165] It is worth noting that the right boundary or the left and right boundaries of the first group of data are used to indicate the right boundary or the left and right boundaries before the first group of data is compressed.

[0166] In the present application, when the processing device 200 determines that the data belongs to the Nth group of data from the index relationship between the root-child node and the leaf-child node, it can determine the right boundary or left boundary of the Nth group of data based on the right boundary or left boundary of each group of data stored above, and then pass the right boundary or left boundary of the Nth group of data to the compression engine for decompression, avoiding decompression of the entire bottom-level node and reducing the decompression area passed to the compression engine, thereby significantly improving the decompression efficiency and increasing the speed of querying the data.

[0167] In a possible embodiment, Figure 4 As shown, Figure 4 Schematic diagram of the structure of the B+-tree based on HOT provided in this application Figure 1 The first data type is a heap-organized table (HOT). HOT stores data in an independent heap. The heap logic is a collection of data. Each data stored in the heap has a unique address, namely, the data address (row identifier, RID). A B+-tree can be established based on the Key on the HOT. The bottom-level node of the B+-tree includes multiple Key-RIDs, which are arranged in order according to the Key.

[0168] Based on the content in S220, the processing device 200 divides the bottom-level nodes of the B+-tree into root-child nodes and leaf-child nodes. The data indicated by the root-child nodes include K3 and K6, and the data indicated by the leaf-child nodes include K1-R(RID)1, K2-R2, K3-R3, K4-R4, K5-R5, K6-R6, K7-R7, and K8-R8. The data indicated by the leaf-child nodes are stored in sequence according to the Key in the storage area corresponding to the leaf-child nodes.

[0169] The processing device 200 compresses the data indicated by the leaf-child nodes as a whole, so that the obtained compressed data includes the data indicated by the root-child nodes (K3 and K6) and the compressed data indicated by the leaf-child nodes (compressed K1-R1, K2-R2, K3-R3, K4-R4, K5-R5, K6-R6, K7-R7, K8-R8).

[0170] In a possible example, the processing device 200 may still store the compressed data in the storage area corresponding to the bottom-level node. The storage area of ​​the bottom-level node includes the storage area indicated by the leaf-child node.

[0171] In another possible example, the processing device 200 stores the compressed data in other storage areas, or stores the root-child node in a storage area other than the storage area of ​​the bottom-level node.

[0172] As for the content of the overall compression of the data indicated by the leaf-child node by the processing device 200, reference may be made to the description of S230 above, which will not be elaborated here.

[0173] In a possible scenario, the processing device 200 compresses the data indicated by the leaf-child nodes under all the bottom-level nodes in the above B+-tree.

[0174] In another possible scenario, the processing device 200 determines to compress the data indicated by the leaf-child nodes under some of the bottom-level nodes in the above B+-tree according to the screening condition.

[0175] The above-mentioned screening condition may be that the access frequency or access frequency of the data in each bottom-level node is less than or equal to the threshold value c. If the access frequency of the data in the bottom-level node a is 100 and less than the threshold value c (120), the processing device 200 compresses the data indicated by the leaf-child node under the bottom-level node a. If the access frequency of the data in the bottom-level node a is 150 and greater than the threshold value c (120), the processing device 200 does not compress the data in the bottom-level node a.

[0176] If the access frequency of the data in the bottom node b is 0.2, which is less than the threshold value c (0.4), the processing device 200 compresses the data indicated by the leaf-child node under the bottom node b. If the access frequency of the data in the bottom node b is 0.5, which is greater than the threshold value c (0.4), the processing device 200 does not compress the data in the bottom node b.

[0177] In one possible scenario, the root-child nodes in a B+-tree can store data in the form of Key-Value. Figure 5 As shown, Figure 5 Schematic diagram of the structure of the B+-tree based on HOT provided in this application Figure 2 , the processing device 200 can divide the bottom-level nodes of the B+-tree into root-child nodes and leaf-child nodes, the data indicated by the root-child nodes include K3-R3 and K6-R6, and the data indicated by the leaf-child nodes include K1-R1, K2-R2, K4-R4, K5-R5, K7-R7, and K8-R8. The data indicated by the leaf-child nodes are stored in sequence according to the Key in the storage area corresponding to the leaf-child nodes.

[0178] In the present application, the processing device 300 stores data such as RID that occupies less space in the storage area corresponding to the root-child node. Without occupying a large amount of storage space, the efficiency of querying the RID indicated in the root-child node can be improved, and the data indicated by the RID can be read, thereby improving the efficiency of the entire data query process.

[0179] In another possible embodiment, Figure 6 As shown, Figure 6 The structural diagram of the IOT provided for this application. The type of the first data is an index-organized table (IOT). The IOT itself is a B+-Tree. The data is directly stored in the bottom-level nodes of the B+-Tree. Each bottom-level node contains multiple Key-Values ​​and is arranged in order according to the Key. The other nodes of the B+-Tree are only used to store the Key used for indexing.

[0180] Based on the content in S220, the processing device 200 divides the bottom-level nodes of the B+-tree into root-child nodes and leaf-child nodes. The data indicated by the root-child nodes include K3 and K6, and the data indicated by the leaf-child nodes include K1-V1, K2-V2, K3-V3, K4-V4, K5-V5, K6-V6, K7-V7, and K8-V8. The data indicated by the leaf-child nodes are stored in sequence according to the Key in the storage area corresponding to the leaf-child nodes.

[0181] The processing device 200 compresses the data indicated by the leaf-child node as a whole, so that the obtained compressed data includes the data indicated by the root-child node (K3 and K6) and the compressed data indicated by the leaf-child node (compressed K1-V1, K2-V2, K3-V3, K4-V4, K5-V5, K6-V6, K7-V7, K8-V8). The storage area of ​​the root-child node is only used to store the key of the data for indexing the data indicated by the leaf-child node.

[0182] For more details of this embodiment, please refer to the above Figure 4 The description of the illustrated embodiments will not be repeated here.

[0183] In this application, only Key is stored in the root-child node for indexing, which avoids the possibility that the amount of Value data is large, resulting in more storage space being occupied, thereby improving the utilization of storage space. In addition, Key-Value is stored in the storage area of ​​the leaf-child node, and the data indicated by the leaf-child node is compressed as a whole, further reducing the storage space occupied and improving the utilization of storage space.

[0184] In one possible scenario, the root-child node in the B+-tree can store data in the form of Key-Value. For example, the processing device 200 can divide the bottom-level nodes of the B+-tree into root-child nodes and leaf-child nodes. The data indicated by the root-child node includes K3-V3 and K6-V6, and the data indicated by the leaf-child node includes K1-V1, K2-V2, K4-V4, K5-V5, K7-V7, and K8-V8. The data indicated by the leaf-child node is stored in sequence according to the Key in the storage area corresponding to the leaf-child node.

[0185] A second possible scenario: After the processing device 200 compresses the second data to obtain compressed data, the processing device 200 receives a data decompression request from the host 110 and decompresses part of the compressed data. Figure 7 As shown, Figure 7 The content shown in this embodiment can be executed by the processing device 200, which can be Figure 1 The storage device 120 in FIG. Figure 7 , the data decompression method provided in this embodiment includes steps S710 to S740.

[0186] S710: The processing device 200 obtains a data decompression request.

[0187] The data decompression request includes an identifier of the first data. For example, the identifier of the first data is a Key of the first data.

[0188] In a possible implementation, the processing device 200 obtains the data decompression request, including: the processing device 200 obtains the data decompression request sent by the host 110 .

[0189] The data decompression request may be generated by a user operation on the host 110. For example, the host 110 receives a click operation, a slide operation, or a voice operation of the user on the front display interface of the host 110, and generates a data decompression request.

[0190] Exemplarily, the host 110 receives a signal from a user clicking a control component corresponding to the first data on the front-end display interface of the host 110 , thereby determining an identifier of the first data, and encapsulating the identifier of the first data into a data decompression request, thereby sending the data decompression request carrying the identifier of the first data to the processing device 200 .

[0191] S720: The processing device 200 queries the compressed data according to the identifier of the first data.

[0192] Among them, the storage structure of compressed data is a tree structure, the bottom-level nodes of the tree structure include a root-child node and a leaf-child node having an index relationship with the root-child node, and the compressed data includes the data indicated by the root-child node and the compressed data indicated by the leaf-child node.

[0193] For the data indicated by the root-child node and the compressed data indicated by the leaf-child node, refer to the above Figure 2 The contents of the data stored in the storage area of ​​the root-child node and the storage area of ​​the leaf-child node are not described in detail here.

[0194] Since the storage structure of the compressed data is a tree structure, the processing device 200 sequentially queries the data indicated by the root node in the compressed data to the data indicated by the child node. For example, the processing device 200 starts to query from the first-level index node of the compressed data, and then indexes the index node of the second level according to the content queried in the first level, and then queries in the index node of the second level, and indexes the index node of the third level according to the content queried in the index node of the second level, and repeats the above steps until the Nth group of data where the first data is located is queried in the compressed data. N is an integer greater than or equal to 1.

[0195] When the processing device 200 searches for data indicated by the root-child node of the bottom-level node in the compressed data, it first compares the identifier of the first data with the data in the root-child node to determine the second group of data to which the first data belongs.

[0196] In one possible implementation, the processing device 200 determines, among the identifiers of the data indicated by the root-child node, a first identifier and / or a second identifier that satisfies the first strategy with the identifier of the first data, and then determines, from the index relationship, a second set of data in the leaf-child node corresponding to the first identifier and / or the second identifier.

[0197] For example, the first strategy is used to indicate that the identifier of the first data is smaller than the first identifier.

[0198] In this example, the first identifier is the minimum identifier indicated by the root-child node. Exemplarily, the storage area of ​​the root-child node includes K3 and K6, and the first identifier is K3.

[0199] For example, Figure 3 As shown in b, Figure 3 b in the figure shows a node at the bottom of the tree structure. That is, the processing device 200 determines that the first data is located at Figure 3The lowest level node shown in b. If the identifier of the first data is K1, the processing device 200 performs a binary search in the root-child node and determines that K1 is less than K3 (the first identifier) ​​in the root-child node. Therefore, based on the index relationship between the root-child node and the leaf-child node, it is determined that the data corresponding to K1 is located in the first set of data indicated by the leaf-child node. The first set of data can also be referred to as the data indicated by the first leaf-child node among multiple leaf-child nodes.

[0200] For another example, the first strategy may also be used to indicate that the identifier of the first data is greater than the second identifier.

[0201] In this example, the second identifier is the maximum identifier indicated by the root-child node. Exemplarily, the storage area of ​​the root-child node includes K3 and K6, and the second identifier is K6.

[0202] For example, Figure 3 As shown in b, if the identifier of the first data is K7, the processing device 200 performs a binary search in the root-child node and determines that K7 is greater than K6 (the second identifier) ​​in the root-child node. Therefore, based on the index relationship between the root-child node and the leaf-child node, it is determined that the data corresponding to K6 is located in the third group of data indicated by the leaf-child node. The third group of data can also be referred to as the data indicated by the third leaf-child node among multiple leaf-child nodes.

[0203] For another example, the first strategy is used to indicate that: the identifier of the first data is greater than or equal to the first identifier and less than or equal to the second identifier, and the first identifier is less than the second identifier.

[0204] In this example, the first identifier and the second identifier are the identifiers of the identifiers indicated by the root-child node that are adjacent to the identifier of the first data, or the first identifier or the second identifier is the identifier of the identifier indicated by the root-child node that is consistent with the identifier of the first data. Exemplarily, the storage area of ​​the root-child node includes K3, K6, and K9. If the identifier of the first data is K5, the first identifier is K3 and the second identifier is K6.

[0205] For example, Figure 3 As shown in b, if the identifier of the first data is K5, the processing device 200 performs a binary search in the root-child node and determines that K5 is greater than K3 (the first identifier) ​​in the root-child node, and K5 is less than K6 (the second identifier) ​​in the root-child node. Therefore, based on the index relationship between the root-child node and the leaf-child node, it is determined that the data corresponding to K5 is located in the second group of data indicated by the leaf-child node.

[0206] If the identifier of the first data is K6, the processing device 200 performs a binary search in the root-child node to determine that K6 is equal to K6 (first identifier) ​​in the root-child node. Therefore, based on the index relationship between the root-child node and the leaf-child node, it is determined that the data corresponding to K6 is located in the second set of data indicated by the leaf-child node.

[0207] It is worth noting that the first group of data, the second group of data, etc. indicated by the leaf-child node are arranged in sequence according to the data decompression direction. In other examples of the present application, other names may also be used, which are not limited by the present application.

[0208] In the present application, the processing device 200 queries the identifier indicated by the root-child node according to the identifier of the first data, and then determines the first identifier and / or the second identifier. Since there is an index relationship between the root-child node and the leaf-child node, the smaller area (that is, the target storage area) where the first data is located in the storage area of ​​the leaf-child node is accurately located, and then only the compressed data in the smaller area needs to be decompressed, and all data does not need to be decompressed, which reduces the amount of decompressed compressed data, thereby improving the efficiency of data query.

[0209] S730: The processing device 200 determines a target storage area in the compressed data that matches the identifier of the first data.

[0210] In this embodiment, the target storage area is the storage area of ​​the leaf-child node.

[0211] The storage area of ​​the leaf-child node stores the first data or the address information of the first data.

[0212] The following description is made by taking the storage area of ​​a leaf-child node storing the first data as an example.

[0213] In one possible implementation, the processing device 200 determines a target storage area in the compressed data that matches an identifier of the first data, including: the processing device 200 determines the right boundary or the left and right boundaries corresponding to the Nth group of data in which the first data is located from the metadata corresponding to the bottom-level node, thereby obtaining a target storage area that matches the identifier of the first data.

[0214] In a possible scenario, metadata is stored in the storage area of ​​the lowest node of the compressed data, and the metadata may include the right boundary or the left and right boundaries of each group of data. The right boundary or the left and right boundaries of each group of data are stored in sequence in the storage area of ​​the lowest node according to the arrangement of each group of data. For example, the right boundary or the left and right boundaries of each group of data are arranged in the storage area of ​​the lowest node as follows: the right boundary or the left and right boundaries of the first group of data, the right boundary or the left and right boundaries of the second group of data, and the right boundary or the left and right boundaries of the third group of data are arranged in sequence.

[0215] Exemplarily, the processing device 200 obtains the Nth right boundary or the left and right boundaries from the metadata according to the Nth group of data in which the first data is located, thereby obtaining a target storage area that matches the identifier of the first data.

[0216] In another possible scenario, the storage area of ​​the bottom-level node of the compressed data stores the space occupied by each group of data, such as length, or the space occupied by each Key-Value.

[0217] Exemplarily, when the storage area of ​​the bottom-level node of the compressed data stores the space occupied by each group of data, the processing device 200 determines that the length of the first address a of the storage area corresponding to the leaf-child node and the right boundary of the first data is N times the space occupied by each group of data, and then adds the aforementioned first address a to N times the space occupied by each group of data to obtain the right boundary of the first data.

[0218] The processing device 200 determines that the length of the first address a of the storage area corresponding to the left boundary of the first data and the leaf-child node is (N-1) multiplied by the space occupied by each group of data, and then adds the aforementioned first address a plus (N-1) multiplied by the space occupied by each group of data to obtain the left boundary of the first data.

[0219] Exemplarily, in the case where the space occupied by each Key-Value is stored in the storage area of ​​the bottom node of the compressed data, the processing device 200 determines that the length of the first address a of the storage area corresponding to the right boundary of the first data and the leaf-child node is N multiplied by the number of Key-Values ​​included in each group of data and then multiplied by the space occupied by each Key-Value, and then adds the aforementioned first address a plus the length of the first address a of N multiplied by the number of Key-Values ​​included in each group of data and then multiplied by the space occupied by each Key-Value to obtain the right boundary of the first data.

[0220] The processing device 200 determines that the left boundary of the first data and the length of the first address a of the storage area corresponding to the leaf-child node are (N-1) multiplied by the number of Key-Values ​​included in each group of data and then multiplied by the space occupied by each Key-Value, and then the left boundary of the first data is obtained by adding the aforementioned first address a plus the length of the first address a (N-1) multiplied by the number of Key-Values ​​included in each group of data and then multiplied by the space occupied by each Key-Value.

[0221] Exemplarily, the storage area of ​​the bottom-level node of the compressed data stores the space occupied by each Key-Value, and the space occupied by each Key-Value is inconsistent. The processing device 200 determines that the length from the right boundary of the first data to the first address a of the storage area corresponding to the leaf-child node is the sum of the space occupied by each Key-Value in the first N groups of data, and then adds the first address a to the sum of the space occupied by each Key-Value in the first N groups of data to obtain the right boundary of the first data.

[0222] The processing device 200 determines that the length from the left boundary of the first data to the starting address a of the storage area corresponding to the leaf-child node is the sum of the space occupied by each Key-Value in the first N-1 groups of data, and then adds the aforementioned starting address a to the sum of the space occupied by each Key-Value in the first N-1 groups of data to obtain the left boundary of the first data.

[0223] In a possible example, the metadata is stored in a fixed position in the storage area of ​​the bottom-level node, such as at the beginning of the storage area.

[0224] For the content of the Nth group of data where the first data is located, reference can be made to the description of S920 above, which will not be described in detail here. For the content of the right boundary or the left and right boundaries, reference can be made to the embodiment in which the storage area of ​​the bottom node in the compressed data is also used to store the right boundary or the left and right boundaries, which will not be described in detail here.

[0225] S740: The processing device 200 decompresses the data in the target storage area to obtain first data.

[0226] The target storage area, that is, the above-mentioned right boundary or left and right boundaries are the boundaries before data compression.

[0227] With respect to the processing device 200 decompressing the data in the target storage area to obtain the content of the first data, two possible implementations are provided below.

[0228] In a first possible implementation, the processing device 200 decompresses the data in the leaf-child node starting from the first address a of the storage area corresponding to the leaf-child node until the decompressed data reaches the right boundary. The processing device 200 queries the decompressed data for the first data corresponding to the identifier of the first data.

[0229] In a second possible implementation, the processing device 200 determines the left boundary of the data a corresponding to the left boundary of the first data from the correspondence between the boundary of the data before compression and the boundary of the data after compression, and then decompresses the compressed data from the left boundary of the data a until the decompressed data reaches the right boundary. The processing device 200 obtains the first data from the decompressed data according to the identifier of the first data.

[0230] The correspondence between the boundaries of the above data before compression and the boundaries after compression is metadata stored in the bottom-level node.

[0231] In view of the above content, a possible example is provided below.

[0232] like Figure 6 As shown, when the data decompression request received by the processing device 200 carries the identifier K4, the compressed IOT is queried, that is, from the root node of the compressed IOT to the bottom node, and it is determined that the data corresponding to K4 is located in the bottom node a. Then, the processing device 200 queries the root-child node (K3 / K6) in the bottom node a according to K4, and determines that K4 is located between K3 and K6, thereby determining that the data corresponding to K4 is located in the second group of data (that is, the second leaf node). The processing device 200 determines the space occupied by the Key-Value in the first two groups of data according to the space occupied by each Key-Value maintained in the bottom node, thereby obtaining the right boundary of the second group of data. The processing device 200 decompresses the data in the leaf-child node starting from the first address a of the storage area corresponding to the leaf-child node until the decompressed data reaches the above right boundary, and then queries the Value corresponding to K4 in the aforementioned decompressed data.

[0233] In a possible embodiment, if the storage area of ​​the leaf-child node stores the address information of the first data, that is, the data stored in the leaf-child node is Key-RID.

[0234] The processing device 200 decompresses the data in the target storage area to obtain the first data, including: the processing device 200 decompresses the data in the leaf-child node starting from the first address a of the storage area corresponding to the leaf-child node until the decompressed data reaches the above right boundary. The processing device 200 queries the identifier of the first data from the decompressed data, queries the RID of the corresponding first data, and then reads the Value indicated by the RID.

[0235] Alternatively, the processing device 200 determines the left boundary of the first data corresponding to the left boundary of data a from the correspondence between the boundary of the data before compression and the boundary after compression, and then decompresses the compressed data from the left boundary of data a until the decompressed data reaches the above right boundary. The processing device 200 queries the identifier of the first data from the decompressed data, queries the RID of the corresponding first data, and then reads the Value indicated by the RID.

[0236] In a possible embodiment, data is stored in the storage area of ​​the root-child node in a Key-Value format, and then, after the processing device 200 finds that the data in the root-child node matches the identifier of the first data, the matching data can be directly returned.

[0237] It is understandable that in order to implement the functions in the above embodiments, the processing device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0238] Combined with the above Figures 1 to 6 , describes in detail the data compression method provided by this embodiment, and will now be combined with Figure 8 , describing the data compression device provided according to this embodiment.

[0239] Figure 8 The present application provides a schematic diagram of a data compression device. The data compression device can be used to implement the functions of the processing device in the above method embodiment, and thus can also achieve the beneficial effects of the above method embodiment. In this embodiment, the data compression device can be a module (such as a chip) of the processing device.

[0240] like Figure 8 As shown, the data compression device 800 includes an acquisition module 810 and a compression module 820. The data compression device 800 is used to implement the above Figures 1 to 6 The functions in the method embodiments shown in .

[0241] Acquisition module 810 is used to acquire second data corresponding to the first data; the storage structure of the second data is a tree structure, and the bottom-level nodes of the tree structure include root-child nodes and leaf-child nodes having an index relationship with the root-child nodes; the first data includes the root-child nodes and the data indicated by the leaf-child nodes.

[0242] The compression module 820 is used to compress the second data to obtain compressed data corresponding to the first data, where the compressed data includes data indicated by the root-child node and data indicated by the compressed leaf-child node.

[0243] It should be noted that, in other embodiments, the acquisition module 810 can be used to execute any step in the management method, and the compression module 820 can be used to execute any step in the management method. The steps that the acquisition module 810 and the compression module 820 are responsible for implementing can be specified as needed, and the acquisition module 810 and the compression module 820 respectively implement different steps in the data compression method to achieve all the functions of the data compression device 800.

[0244] It is worth noting that the processing device according to the embodiment of the present application may correspond to the data compression device 800 in the embodiment of the application, and may correspond to the method for executing the method according to the embodiment of the present application. Figures 1 to 6 The corresponding corresponding subjects, and the operations and / or functions of each module in the data compression device 800 are respectively to achieve Figures 1 to 6 For the sake of brevity, the corresponding processes of each method in the corresponding embodiment are not repeated here.

[0245] Combined with the above Figure 7 , describes in detail the data decompression method provided by this embodiment, and will now be combined with Fig. 9 , describing the data decompression device provided according to this embodiment.

[0246] Fig. 9 The present application provides a schematic diagram of the structure of a data decompression device. The data decompression device can be used to implement the functions of the processing device in the above method embodiment, and thus can also achieve the beneficial effects of the above method embodiment. In this embodiment, the data decompression device can be a module (such as a chip) of the processing device.

[0247] like Fig. 9 As shown, the data decompression device 900 includes an acquisition module 910, a query module 920, a determination module 930 and a decompression module 940. The data decompression device 900 is used to implement the above Figure 7 The functions in the method embodiments shown in .

[0248] An acquisition module 910 is used to acquire a data decompression request, where the data decompression request includes an identifier of the first data;

[0249] A query module 920 is used to query the compressed data according to the identifier of the first data; wherein the storage structure of the compressed data is a tree structure, the bottom-level nodes of the tree structure include a root-child node and a leaf-child node having an index relationship with the root-child node, and the compressed data includes data indicated by the root-child node and data indicated by the compressed leaf-child node;

[0250] A determination module 930 is used to determine a target storage area in the compressed data that matches the identifier of the first data, where the target storage area is a storage area of ​​a leaf-child node;

[0251] The decompression module 940 is used to decompress the data in the target storage area to obtain first data.

[0252] It should be noted that, in other embodiments, the acquisition module 910 can be used to execute any step in the management method, the query module 920 can be used to execute any step in the management method, the determination module 930 can be used to execute any step in the management method, and the decompression module 940 can be used to execute any step in the management method. The steps that the acquisition module 910, the query module 920, the determination module 930, and the decompression module 940 are responsible for implementing can be specified as needed, and the acquisition module 910, the query module 920, the determination module 930, and the decompression module 940 respectively implement different steps in the data decompression method to realize all the functions of the data decompression device 900.

[0253] It is worth noting that the processing device according to the embodiment of the present application may correspond to the data decompression device 900 in the embodiment of the application, and may correspond to the method for executing the method according to the embodiment of the present application. Figure 7 The corresponding corresponding subjects, and the operations and / or functions of each module in the data decompression device 900 are respectively to achieve Figure 7 For the sake of brevity, the corresponding processes of each method in the corresponding embodiment are not repeated here.

[0254] When the data compression device or the data decompression device implements the data compression method or the data decompression method shown in any of the above-mentioned figures through software, the data compression device and its various units or the data decompression device and its various units can also be software modules. The above-mentioned data compression method or data decompression method is implemented by calling the software module through a processor. The processor can be a CPU, an ASIC implementation, or a programmable logic device (PLD), and the above-mentioned PLD can be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0255] For a more detailed description of the above-mentioned data compression device or data decompression device, please refer to the relevant description in the embodiment shown in the above-mentioned figures, which will not be repeated here. It can be understood that the data compression device or data decompression device shown in the above-mentioned figures is only an example provided by this embodiment. Depending on the data type or compression method, the data compression device or data decompression device may include more or fewer units, and this application is not limited to this.

[0256] When the data compression device or the data decompression device is implemented by hardware, the hardware may be implemented by a processor or a chip. The chip includes an interface circuit and a control circuit. The interface circuit is used to receive data from other devices outside the processor and transmit it to the control circuit, or to send data from the control circuit to other devices outside the processor.

[0257] The control circuit and the interface circuit are used to implement any possible implementation method in the above embodiments through logic circuits or execution code instructions. The beneficial effects can be found in the description of any aspect of the above embodiments, which will not be repeated here.

[0258] It is understood that the processor in the embodiments of the present application may be a CPU, NPU or GPU, or other general-purpose processors, digital signal processors (DSP), ASICs, FPGAs or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0259] in addition, Figure 8 The data compression device 800 shown and Fig. 9 The data decompression device 900 shown can also be implemented by a storage device, such as Figure 1 The storage device 120 shown in FIG. 1 , or a data storage system including a storage device, etc.

[0260] The method steps in the embodiments of the present application can also be implemented by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in RAM, flash memory, ROM, PROM, EPROM, electrically erasable programmable read-only memory (electrically EPROM, EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a communication device. Of course, the processor and the storage medium can also be present in a communication device as discrete components.

[0261] The present application also provides a chip system, which includes a processor for implementing the function of the storage device in the above method. In one possible design, the chip system also includes a memory for storing program instructions and / or data. The chip system can be composed of a chip or a chip and other discrete devices.

[0262] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the above-mentioned compression method or decompression method.

[0263] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute a compression method or a decompression method.

[0264] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).

[0265] In each embodiment of the present application, if there is no special explanation and logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships. The various digital numbers involved in the embodiments of the present application are only used to distinguish for the convenience of description and are not used to limit the scope of the embodiments of the present application. The size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic.

Claims

1. A data compression method, characterized in that: The method comprises: Acquire second data corresponding to the first data; the storage structure of the second data is a tree structure, the bottom-level nodes of the tree structure include a root-child node and a leaf-child node having an index relationship with the root-child node; the first data includes the root-child node and the data indicated by the leaf-child node; The second data is compressed to obtain compressed data corresponding to the first data, wherein the compressed data includes the data indicated by the root-child node and the compressed data indicated by the leaf-child node.

2. The method according to claim 1, characterized in that: The storage area of ​​the root-child node is used to store the first sub-data that meets the condition, and the storage area of ​​the leaf-child node is used to store the second sub-data that does not meet the condition, and the condition is used to indicate that the access index of the data is greater than or equal to a threshold, or, the identification number interval of two adjacent data indicated by the root-child node is K plus 1, where K is an integer greater than or equal to 1.

3. The method according to claim 2, characterized in that The access index includes: access frequency or access frequency; the first sub-data is used to indicate hot data, and the second sub-data is used to indicate cold data.

4. The method according to claim 2, characterized in that: K satisfies the following formula, K(K+X) ≥ N; Among them, X is a constant, and N is the amount of data included in a bottom-level node.

5. The method according to any one of claims 1 to 4, characterized in that The storage area of ​​the bottom-level node in the compressed data is also used to store: the right boundary or the left and right boundaries of each group of data indicated by the leaf-child node in the storage area of ​​the leaf-child node; The left boundary of the first group of data is used to indicate the minimum byte length from the first address of the storage area corresponding to the leaf-child node to the location where the first group of data is stored, and the right boundary of the first group of data is used to indicate the maximum byte length from the first address of the storage area corresponding to the leaf-child node to the location where the first group of data is stored. The first group of data is any group of data indicated by the leaf-child node.

6. The method according to claim 5, characterized in that The data indicated by the root-child node indicates the right boundary or the left and right boundaries of each group of data.

7. The method according to any one of claims 1 to 6, characterized in that The data format of the first data is a key-value pair Key-Value, and the data indicated by the leaf-child node is stored in the storage area of ​​the leaf-child node in sequence according to the Key.

8. The method according to any one of claims 1 to 7, characterized in that The tree structure is a B+-tree.

9. A data decompression method, characterized in that: The method comprises: Obtaining a data decompression request, where the data decompression request includes an identifier of the first data; According to the identifier of the first data, the compressed data is queried; wherein the storage structure of the compressed data is a tree structure, the bottom-level nodes of the tree structure include a root-child node and a leaf-child node having an index relationship with the root-child node, and the compressed data includes the data indicated by the root-child node and the compressed data indicated by the leaf-child node; Determine a target storage area in the compressed data that matches the identifier of the first data, the target storage area being a storage area of ​​a leaf-child node; The data in the target storage area is decompressed to obtain the first data.

10. The method according to claim 9, characterized in that The storage area of ​​the bottom-most node of the compressed data is also used to store: the right boundary or the left and right boundaries of each group of data indicated by the leaf-child node in the storage area of ​​the leaf-child node; The left boundary of the first group of data is used to indicate the minimum byte length from the first address of the storage area corresponding to the leaf-child node to the location where the first group of data is stored, and the right boundary of the first group of data is used to indicate the maximum byte length from the first address of the storage area corresponding to the leaf-child node to the location where the first group of data is stored. The first group of data is any group of data indicated by the leaf-child node.

11. The method according to claim 10, characterized in that The data indicated by the root-child node indicates the right boundary or the left and right boundaries of each group of data.

12. The method according to claim 11, characterized in that The determining, in the compressed data, a target storage area that matches the identifier of the first data includes: According to the identifier of the first data, query the bottom-level nodes in sequence from the root node of the compressed data; the first data or the address information of the first data is stored in the storage area of ​​the leaf-child node in the bottom-level node; The identifier of the first data is compared with the data in the root-child node to determine the right boundary or the left and right boundaries of the second group of data; the second group of data includes the first data, and the right boundary or the left and right boundaries of the second group of data are used to indicate the target storage area.

13. The method according to claim 12, characterized in that The step of comparing the identifier of the first data with the data in the root-child node to determine the right boundary or the left and right boundaries of the second set of data includes: Determine, among the identifiers of the data indicated by the root-child node, a first identifier and / or a second identifier that satisfies a first policy with the identifier of the first data; the first policy is used to indicate that: the identifier of the first data is less than the first identifier, or the identifier of the first data is greater than the second identifier, or the identifier of the first data is greater than or equal to the first identifier and less than or equal to the second identifier, and the first identifier is less than the second identifier; Determine, from the index relationship, a second group of data in a leaf-child node corresponding to the first identifier and / or the second identifier; The second set of data is obtained at the right boundary or the left boundary of the storage area of ​​the leaf-child node.

14. The method according to any one of claims 9 to 13, characterized in that The storage area of ​​the root-child node is used to store the first sub-data that meets the condition, and the storage area of ​​the leaf-child node is used to store the second sub-data that does not meet the condition, the first data includes the first sub-data and the second sub-data, and the condition is used to indicate that the access index of the data is greater than or equal to the threshold, or, the identification number interval of two adjacent data indicated by the root-child node is K plus 1, and K is an integer greater than or equal to 1.

15. The method according to claim 14, characterized in that The access index includes: access frequency or access frequency; the first sub-data is used to indicate hot data, and the second sub-data is used to indicate cold data.

16. The method according to claim 14, characterized in that K satisfies the following formula, K(K+X) ≥ N; Among them, X is a constant, and N is the amount of data included in a bottom-level node.

17. The method according to any one of claims 9 to 16, characterized in that The data format of the first data is a key-value pair Key-Value, and the data indicated by the leaf-child node is stored in the storage area of ​​the leaf-child node in sequence according to the Key.

18. The method according to any one of claims 9 to 17, characterized in that The tree structure includes a B+-tree.

19. A data compression device, characterized in that: The device comprises: an acquisition module, configured to acquire second data corresponding to the first data; the storage structure of the second data is a tree structure, the bottom-level nodes of the tree structure include a root-child node and a leaf-child node having an index relationship with the root-child node; the first data includes the root-child node and the data indicated by the leaf-child node; A compression module is used to compress the second data to obtain compressed data corresponding to the first data, wherein the compressed data includes the data indicated by the root-child node and the compressed data indicated by the leaf-child node.

20. The device according to claim 19, characterized in that The storage area of ​​the root-child node is used to store the first sub-data that meets the condition, and the storage area of ​​the leaf-child node is used to store the second sub-data that does not meet the condition, and the condition is used to indicate that the access index of the data is greater than or equal to a threshold, or, the identification number interval of two adjacent data indicated by the root-child node is K plus 1, where K is an integer greater than or equal to 1.

21. The device according to claim 20, characterized in that The access index includes: access frequency or access frequency; the first sub-data is used to indicate hot data, and the second sub-data is used to indicate cold data.

22. The device according to claim 20, characterized in that K satisfies the following formula, K(K+X) ≥ N; Among them, X is a constant, and N is the amount of data included in a bottom-level node.

23. The device according to any one of claims 19 to 22, characterized in that The storage area of ​​the bottom-level node in the compressed data is also used to store: the right boundary or the left and right boundaries of each group of data indicated by the leaf-child node in the storage area of ​​the leaf-child node; The left boundary of the first group of data is used to indicate the minimum byte length from the first address of the storage area corresponding to the leaf-child node to the location where the first group of data is stored, and the right boundary of the first group of data is used to indicate the maximum byte length from the first address of the storage area corresponding to the leaf-child node to the location where the first group of data is stored. The first group of data is any group of data indicated by the leaf-child node.

24. The device according to claim 23, characterized in that The data indicated by the root-child node indicates the right boundary or the left and right boundaries of each group of data.

25. The device according to any one of claims 19 to 24, characterized in that The data format of the first data is a key-value pair Key-Value, and the data indicated by the leaf-child node is stored in the storage area of ​​the leaf-child node in sequence according to the Key.

26. The device according to any one of claims 19 to 25, characterized in that The tree structure is a B+-tree.

27. A data decompression device, characterized in that: The device comprises: An acquisition module, configured to acquire a data decompression request, wherein the data decompression request includes an identifier of the first data; A query module, configured to query compressed data according to the identifier of the first data; wherein the storage structure of the compressed data is a tree structure, the bottom-level nodes of the tree structure include a root-child node and a leaf-child node having an index relationship with the root-child node, and the compressed data includes data indicated by the root-child node and compressed data indicated by the leaf-child node; A determination module, configured to determine a target storage area in the compressed data that matches the identifier of the first data, wherein the target storage area is a storage area of ​​a leaf-child node; A decompression module is used to decompress the data in the target storage area to obtain the first data.

28. The device according to claim 27, characterized in that The storage area of ​​the bottom-most node of the compressed data is also used to store: the right boundary or the left and right boundaries of each group of data indicated by the leaf-child node in the storage area of ​​the leaf-child node; The left boundary of the first group of data is used to indicate the minimum byte length from the first address of the storage area corresponding to the leaf-child node to the location where the first group of data is stored, and the right boundary of the first group of data is used to indicate the maximum byte length from the first address of the storage area corresponding to the leaf-child node to the location where the first group of data is stored. The first group of data is any group of data indicated by the leaf-child node.

29. The device according to claim 28, characterized in that The data indicated by the root-child node indicates the right boundary or the left and right boundaries of each group of data.

30. The device according to claim 29, characterized in that The determining, in the compressed data, a target storage area that matches the identifier of the first data includes: According to the identifier of the first data, query the bottom-level nodes in sequence from the root node of the compressed data; the first data or the address information of the first data is stored in the storage area of ​​the leaf-child node in the bottom-level node; The identifier of the first data is compared with the data in the root-child node to determine the right boundary or the left and right boundaries of the second group of data; the second group of data includes the first data, and the right boundary or the left and right boundaries of the second group of data are used to indicate the target storage area.

31. The device according to claim 30, characterized in that The step of comparing the identifier of the first data with the data in the root-child node to determine the right boundary or the left and right boundaries of the second set of data includes: Determine, among the identifiers of the data indicated by the root-child node, a first identifier and / or a second identifier that satisfies a first policy with the identifier of the first data; the first policy is used to indicate that: the identifier of the first data is less than the first identifier, or the identifier of the first data is greater than the second identifier, or the identifier of the first data is greater than or equal to the first identifier and less than or equal to the second identifier, and the first identifier is less than the second identifier; Determine, from the index relationship, a second group of data in a leaf-child node corresponding to the first identifier and / or the second identifier; The second set of data is obtained at the right boundary or the left boundary of the storage area of ​​the leaf-child node.

32. The device according to any one of claims 27 to 31, characterized in that The storage area of ​​the root-child node is used to store the first sub-data that meets the condition, and the storage area of ​​the leaf-child node is used to store the second sub-data that does not meet the condition, the first data includes the first sub-data and the second sub-data, and the condition is used to indicate that the access index of the data is greater than or equal to the threshold, or, the identification number interval of two adjacent data indicated by the root-child node is K plus 1, and K is an integer greater than or equal to 1.

33. The device according to claim 32, characterized in that The access index includes: access frequency or access frequency; the first sub-data is used to indicate hot data, and the second sub-data is used to indicate cold data.

34. The device according to claim 32, characterized in that K satisfies the following formula, K(K+X) ≥ N; Among them, X is a constant, and N is the amount of data included in a bottom-level node.

35. The device according to any one of claims 27 to 34, characterized in that The data format of the first data is a key-value pair Key-Value, and the data indicated by the leaf-child node is stored in the storage area of ​​the leaf-child node in sequence according to the Key.

36. The device according to any one of claims 27 to 35, characterized in that The tree structure includes a B+-tree.

37. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory is used to store computer instructions; when the processor executes the computer instructions, the method according to any one of claims 1 to 18 is implemented.

38. A computer-readable storage medium, characterized in that: The storage medium stores a computer program or instruction, and when the computer program or instruction is executed by a processing device, the method according to any one of claims 1 to 18 is implemented.

39. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed on a processing device, the method according to any one of claims 1 to 18 is implemented.

Images