Query compression method for database two-dimensional table

Abstract

The embodiment of the invention discloses a queriable compression method for a database two-dimensional table. Wherein the database is additionally provided with a data structure to compress and represent the two-dimensional table; the data structure comprises a value array, a field value learning type mapping tree graph and a primary key learning type mapping tree graph; the method comprises the following steps: executing construction operation to construct value arrays of all byte array type fields, field value learning type mapping tree graphs of all byte array type fields and a unique primary key learning type mapping tree graph; and performing addition, deletion, modification and query operations on the major key learning type mapping tree graph according to a received operation instruction, wherein the addition, deletion, modification and query operations comprise major key single-point query, major key range query, two-dimensional table record addition operation, two-dimensional table record deletion operation and two-dimensional table record modification operation. According to the method, the storage space of the primary key and the byte array type field value can be greatly compressed, and the addition, deletion, modification and check operation of the two-dimensional table record can be more efficiently executed.

Description

Technical Field

[0001] The present application belongs to the field of database technology, and in particular relates to a queryable compression method for a two-dimensional table in a database. Background Art

[0002] With the advent of the big data era, real-world data is growing dramatically. Structured data, as a high-quality component of this data, is also increasing. When storing massive amounts of structured data, database management systems inevitably face the potential problem of insufficient storage space. Therefore, data compression and storage have become one of the core functions of today's database management systems.

[0003] General compression methods focus primarily on the statistical characteristics of data, without considering the structural features of the data. These methods can be applied to various types of data, from text to binary files. The core idea of ​​general compression is to use the statistical laws and redundant information in the data to reduce the storage space of the data. There are many common general compression techniques, such as null value compression to save space by eliminating null values; dictionary coding to replace repeated data items with short codes; block compression to split data blocks and compress them separately to improve access efficiency; run-length encoding to simplify continuous repeated data; Huffman coding to assign variable-length codes based on frequency to achieve the theoretically optimal compression rate; and the LZ series of algorithms to compress by referencing historical data. Because general compression methods lose the structural characteristics of the data after compressing the data, when querying the compressed data, all or part of the data needs to be decompressed in order to properly process the user query and return the query results.

[0004] Compression methods that consider the data characteristics of two-dimensional database tables are mainly divided into row-based compression and column-based compression. Row-based compression primarily leverages the dependencies between tuples to reduce redundant tuples, thereby saving storage space. However, row-based compression struggles with dynamically changing data content because changes to the data can cause dependencies between tuples to no longer hold. Column-based compression primarily leverages the similarity of data within a column to achieve efficient compression, reducing storage space and optimizing query performance.

[0005] Currently, existing column mapping compression methods support queries without decompression, but these methods significantly impact data writing and updating performance. Furthermore, the cost of converting between source and target column values ​​is a significant factor affecting query performance. Therefore, data compression methods that support both non-decompression queries and efficient add, delete, modify, and query operations have long been a key goal in the database compression field. More efficiently executing add, delete, modify, and query operations on two-dimensional table records is a pressing technical challenge facing those skilled in the art.

[0006] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0007] Based on this, it is necessary to propose a queryable compression method for database two-dimensional tables to address the above problems, which can more efficiently perform the addition, deletion, modification and query operations of two-dimensional table records.

[0008] This application adopts the following technical solutions to solve the above technical problems: The present application provides a queryable compression method for a two-dimensional table of a database, which is applied to a database. The database is used to store a two-dimensional table. The rows of the two-dimensional table are composed of records, and the columns of the two-dimensional table are composed of primary keys and other column fields. The primary key is represented by a byte array, and the data type of the other column fields is one of a byte array type and a numeric type. The value of one other column field in each record is called the field value of the field; the database adds a data structure to compress and represent the two-dimensional table; the data structure includes a value array, a field value learning mapping tree diagram, and a primary key learning mapping tree diagram; the database includes multiple value arrays, and the value arrays have a one-to-one correspondence with the fields of the byte array type. The value arrays store the corresponding byte array types. All different field values ​​of the field in the two-dimensional table; the database includes multiple field value learning mapping tree diagrams, the field value learning mapping tree diagram has a one-to-one correspondence with the field of byte array type, the field value learning mapping tree diagram is used to store the mapping relationship between field values ​​and field value numbers, and the field value number is the sequence number of the field value in the corresponding value array; the field value learning mapping tree diagram includes multiple nodes; each node includes a discretization model, a common prefix and an entry list; the discretization model is used to map the field value after clipping the upper common prefix to the entry sequence number in the entry list; the common prefix is ​​the common part of all field values ​​covered in the subtree with the corresponding node as the root; the entry list includes node entries The primary key learning mapping tree diagram is used to store the mapping relationship between a given primary key and other column value tuples. The field values ​​of the byte array type in the other column value tuples are represented as serial numbers in the corresponding value arrays, while the field values ​​of the numerical type in the other column value tuples are represented as the field values ​​themselves. The primary key learning mapping tree diagram includes multiple nodes. Each node includes a discretization model, a common prefix, and an entry list. The discretization model is used to map the primary key after clipping the upper common prefix to the entry serial number in the entry list. The common prefix It is the common part of all primary keys covered in the subtree with the corresponding node as the root; the entry list includes node entries and data entries; the node entry is used to store child node pointers; the data entry is used to store the suffix of the corresponding primary key and other column value tuples; the method includes the following steps: executing a construction operation to construct a value array of all byte array type fields, a field value learning mapping tree diagram of all byte array type fields and a unique primary key learning mapping tree diagram; performing add, delete, modify and query operations on the primary key learning mapping tree diagram according to the received operation instructions, and the add, delete, modify and query operations include primary key single point query, primary key range query, two-dimensional table record addition operation, two-dimensional table record deletion operation and two-dimensional table record modification operation.

[0009] In an optional embodiment of the present application, when the construction operation is used to construct a field value learning mapping tree diagram of a byte array type field, the construction operation includes: obtaining all field values ​​of the byte array type field stored in the two-dimensional table; determining the first k-key number N according to all field values ​​respectively. H(a[1:k]) and the second k-bond number N H(a[1:k]),b ; The first k-key number N H(a[1:k]) Defined as the number of field values ​​whose prefix H function value is equal to H(a[1:k]); the second k-key number N H(a[1:k]),b The value of the H function defined as a prefix is ​​equal to the number of field values ​​in H(a[1:k]) whose k+1th byte is equal to b, among all field values ​​in the string H(a[1:k]); k is the number of bytes, b is the byte; H is a preset function that maps character arrays to non-negative numbers; the byte conditional probability P(b|H(a[1:k])) is determined based on the first k-key number and the second k-key number, and is calculated as follows: ; The byte array probability P(a[1:k]) is defined iteratively based on the byte conditional probability: P(a[1:0])=1; and for any k≥0, P(a[1:k+1])=P(a[1:k])·P(a[k+1]|H(a[1:k])); The conversion function F(a[1:k]) is iteratively defined based on the byte conditional probability P(b|H(a[1:k])) and the byte array probability P(a[1:k]): F(a[1:0])=0; and for any k≥0, F(a[1:k+1])=F(a[1:k])+P(a[1:k])∑ b∈{0,1,…,a[k+1]-1} P(b|H(a[1:k])); Construct the input set {v1,…,v according to all the field values ​​in the corresponding value array of the byte array type field N}, N is the number of different field values ​​of the byte array type field, v i is the field value, i∈[1,N], v1,…,v N Arrange in ascending order; set the common prefix of the root node of the field value learning mapping tree diagram of the byte array type field to an empty byte array; determine the number of entries n of the root node according to the number of field values ​​N and the preset expected number of entries E; use the input set as the input of the conversion function F, and according to the first processing result F(v i)Construct a discretization model M; use the first processing result corresponding to each field value in the input set as the input of the discretization model M and substitute it for processing to obtain the corresponding second processing result; determine an entry list according to the second processing result; determine the entry list according to the second processing result, including: assuming M(F(v0)) < M(F(v1)) and M(F(v N )) < M(F(v N+1 )) is always true, traverse all the second processing results M(F(v i )); if there exists an i such that M(F(v i-1 )) < M(F(v i )) < M(F(v i+1 )), then determine the entry with the serial number M(F(v i )) as a data entry, and the data entry is used to save the suffix of v i and the number of v i ; if there exists an i < j such that M(F(v i-1 )) < M(F(v i )) = M(F(v i+1 )) = … = M(F(v j )) < M(F(v j+1 )), then recursively call the new node construction operation with {v i ,…,v j} as the input, and determine the entry with the serial number M(F(v i )) as a node entry, and the node entry is used to store the memory address pointer of the new node; summarize all the nodes for which the discretization model M, the common prefix, and the entry list are determined to obtain a field value learning mapping tree diagram of the byte array type field.

[0010] In an optional embodiment of the present application, when the construction operation is used to construct a primary key learning mapping tree diagram, the construction operation includes: obtaining all the primary keys stored in the two-dimensional table; respectively determining the third k-key number N H(a[1:k]) and the fourth k-key number N H(a[1:k]),b ; the third k-key number N H(a[1:k]) is defined as the number of primary keys whose H function value of the prefix in all primary keys is equal to H(a[1:k]); the fourth k-key number N H(a[1:k]),b is defined as the number of primary keys whose k + 1-th byte is equal to b among all primary keys whose H function value of the prefix is equal to H(a[1:k]); k is the number of bytes, b is a byte; H is a preset function that maps from a character array to a non-negative number; determine the byte conditional probability P K (b|H(a[1:k])) according to the third k-key number and the fourth k-key number, and the calculation method is: ; Iteratively define the byte array probability P based on the byte conditional probability K (a[1:k]): P K (a[1:0])=1; and for any k≥0, P K (a[1:k+1])=P K (a[1:k])·P K (a[k+1]|H(a[1:k])); According to the byte conditional probability P K (b|H(a[1:k])) and byte array probability P K (a[1:k]) iteratively defines the transformation function F K (a[1:k]): F K (a[1:0])=0; and for any k≥0, F K (a[1:k+1])=F K (a[1:k])+P K (a[1:k])∑ b∈{0,1,…,a[k+1]-1} P K (b|H(a[1:k])); Construct the input set {key1,…,key M}, M is the number of all primary keys, key i is the primary key, i∈[1,M], key1,…,key M Ascending order; Determine the discretization model of the primary key learning mapping tree graph, the process of common prefix and entry list, except using the input set to replace the input set, the monotonic conversion function F K Except for the replacement conversion function F and the data entries used to save the suffix of the corresponding primary key and other column value tuples, the remaining steps are consistent with the process of determining the discretization model, common prefix and entry list of the field value learning mapping tree graph; all nodes with determined discretization models, common prefixes and entry lists are summarized to obtain the primary key learning mapping tree graph.

[0011] In an optional embodiment of the present application, the number of entries n of the root node is determined according to the number of field values ​​N and the preset expected number of entries E; the input set is used as the input of the conversion function F, and the first processing result F(v i ) Constructing a discretization model M, including: determining the discretization interval size s according to the number of field values ​​N and the preset expected number of entries E. The calculation process is: s=ceil(N / E); ceil(x) returns the smallest integer not less than x; introduce v N1 And assume that v N1It is an empty byte array, where N1 is any integer greater than N; determine the partition points of the discretization model M one by one: the left neighbor v of the first partition point s1 is v1, ..., v s In the equation “v1,…,v s1 The longest common prefix of is not empty, and s1=s or v1, ..., v s1+1 The longest common prefix of the "is empty" field value with the smallest sequence number, and the right neighbor of the first partition point is v s1+1 , determine the first dividing point as (F(v s1 )+F(v s1+1 )) / 2; the left neighbor v of the k≥2 partition point sk It is v i ,…,v i+s In the i ,…,v sk The longest common prefix of is not empty, and s k =i+s or v i ,…,v sk+1 The longest common prefix is ​​empty" and the field value with the smallest sequence number, where v i is the right neighbor of the k-1th partition point; if s k =N, then n=k and M(x) has k-1 partition points, otherwise the right neighbor of the kth partition point is v sk+1 , the kth partition point is determined as (F(v sk )+F(v sk+1 )) / 2; Summarize all the division points to obtain the discretization model M.

[0012] In an optional embodiment of the present application, a primary key single-point query operation includes: determining a target key to be queried in the instruction content of the primary key single-point query operation, calling a locate(node, key) function, wherein the first parameter node of the locate(node, key) function is a node in the primary key learning mapping tree diagram, which is set as the root node of the primary key learning mapping tree diagram; the second parameter key is a byte array to be queried, which is set as the target key; the locate(node, key) function locates the node with the sequence number M(F K (key)) is marked as a query entry, where M is the discretized model of node, F KIt is a monotonic conversion function of the primary key learning mapping tree graph; judge the entry type of the query entry; when the query entry is a data entry, mark the query entry as a positioning entry, and concatenate the common prefix of all nodes in the path from the root node to the positioning entry and the suffix stored in the positioning entry into a comparison key; judge whether the comparison key is the same as the target key; if the comparison key is not equal to the target key, return the information that the target key cannot be found; if the comparison key is equal to the target key, decode the other column value tuples stored in the positioning entry, and return the decoded data tuple. The decoding process for other column value tuples is to replace the field value numbers of all byte array type fields in the other column value tuples with byte arrays with field value numbers in the corresponding value array; when the entry type of the query entry is a node entry, access the child node through the child node pointer stored in the query entry; judge whether the common prefix stored in the child node is the same as the prefix of the target key; if the common prefix stored in the child node is the same as the prefix of the target key, trim the common prefix of the child node from the target key to obtain the remaining target key, and return locate(child node, remaining target keys); if the common prefix stored in the child node is different from the prefix of the target key, the query entry is marked as a positioning entry, and the information that the target key cannot be found is returned; a primary key range query, including: determining the range lower limit query key and range upper limit query key to be queried in the instruction content of the primary key range query; executing a primary key point query according to the range lower limit query key, determining the positioning entry of the primary key point query, and marking it as the start entry; executing a primary key point query according to the range upper limit query key, determining the positioning entry of the primary key point query, and marking it as the end entry; traversing the data entries in all nodes between the start entry and the end entry in a depth-first traversal; obtaining the primary key and other column value tuples stored in each data entry, and returning the primary key and data tuple, the data tuple is obtained by decoding the other column value tuples, and the decoding process for the other column value tuples is to replace the field value numbers of all byte array type fields in the other column value tuples with byte arrays with field value numbers as serial numbers in the corresponding value array.

[0013] In an optional embodiment of the present application, the two-dimensional table record adding operation includes: determining the two-dimensional table record to be added in the instruction content of the two-dimensional table record adding operation, recording the two-dimensional table record to be added as record new_1 , record new_1 The primary key is denoted as key new_1 ; for key new_1 Execute a primary key single-point query to determine the primary key single-point query location entry, called the first target entry; determine each node in the path from the root node to the first target entry, marked as a path node; concatenate the common prefixes of all path nodes and mark the result as prefix; key new_1The result after trimming the prefix is ​​marked as suffix new ; If the first target entry is a data entry, get the suffix originally stored in the first target entry, recorded as suffix old ; If suffix old with suffix new If they are consistent, then according to record new_1 Construct a new value tuple and update the other column value tuples originally stored in the first target entry to the new value tuple; if the first target entry is a node entry, mark the child node pointed to by the first target entry as the first invalid node; all remaining primary keys and suffixes covered in the subtree with the first invalid node as the root new as input, a new node construction operation is called to construct a first new node, where each remaining primary key is concatenated from the common prefix of all nodes in the path from the first failed node to a data entry and the suffix of the data entry; the child node pointer stored in the first target entry is replaced with the memory address pointer of the first new node, and the first failed node is deleted; whether each path node meets a preset reconstruction condition is determined, where the total number of nodes in the subtree where the path node is located is not less than a preset node number threshold, the number of entries in the entry list of the path node is not less than a preset multiple, and the proportion of data entries in all entries covered by the subtree where the path node is located is not less than a preset proportion; when any node in the path node meets the reconstruction condition, the path node that meets the reconstruction condition is marked as a reconstructed node; a new node construction operation is called to construct a second new node, where each remaining primary key is concatenated from the common prefix of all nodes in the path from the reconstructed node to a data entry and the suffix of the data entry; the child node pointer stored in the node entry pointing to the reconstructed node is replaced with the memory address pointer of the second new node, and the reconstructed node is deleted.

[0014] In an optional embodiment of the present application, the two-dimensional table record deletion operation includes: determining the primary key of the two-dimensional table record to be deleted in the instruction content of the two-dimensional table record deletion operation, recording the two-dimensional table record to be deleted as record old_1 , record old_1 The primary key is denoted as key old_1 ; for key old_1 Execute the primary key single point query to determine the positioning entry of the primary key single point query, which is called the second target entry; if the primary key single point query returns key old_1If no information is found, the deletion operation is terminated; otherwise, the node where the second target entry is located is marked as the second failed node, and the set of all remaining primary keys that do not involve the second target entry and are covered in the subtree with the second failed node as the root is used as input, and the new node construction operation is called to construct a second new node. Each remaining primary key is composed of the common prefix of all nodes in the path from the second failed node to a data entry different from the second target entry and the suffix of the data entry; the child node pointer stored in the node entry pointing to the second failed node is replaced with the memory address pointer of the second new node, and the second failed node is deleted.

[0015] In an optional embodiment of the present application, the two-dimensional table record modification operation includes: determining the original primary key and the modified two-dimensional table record in the instruction content of the two-dimensional table record modification operation, recording the original primary key as key old_2 , key old_2 The corresponding two-dimensional table record is recorded as record old_2 , record the modified two-dimensional table record as record new_2 , record new_2 The primary key is denoted as key new_2 ;Judge key old_2 and key new_2 Are they the same? If key old_2 and key new_2 If they are not the same, first use record new_2 As the instruction content, perform the two-dimensional table record addition operation, and then use record old_2 As the instruction content, execute the two-dimensional table record deletion operation; if key old_2 and key new_2 If they are consistent, then the key new_2 Execute a primary key single-point query to determine the primary key single-point query positioning entry, called the third target entry; if the primary key single-point query returns key new_2 If the information is not found, record new_2 As the instruction content, perform the two-dimensional table record addition operation, otherwise according to the record new_2 Construct a new value tuple and change the other column value tuples stored in the third target entry to the new value tuple.

[0016] In an optional embodiment of the present application, according to record new Construct a new value tuple, including: according to record new Determine the other columns except the primary key and perform the following processing on the byte array type fields in the other columns: Get record newThe field value of the middle byte array type field and the value array corresponding to the byte array type field; if the field value is not in the value array, mark the field value as a new field value, add the new field value to the value array, and add the mapping relationship between the new field value and the number in the field value learning mapping tree diagram; the process of adding a new field value in the field value learning mapping tree diagram is basically the same as the process of adding a new primary key in the two-dimensional table record addition operation, except that the generated data entry stores the number of the new field value; finally, the new numerical tuple consists of record new The field value numbers of the byte array type fields in other column fields in new and the original field values of the numerical type fields.

[0017] In an optional embodiment of the present application, the new node construction operation includes: denoting the new node as node, obtaining the input set required to construct node, and the input set is the set of all input field values required to construct the node, denoted as {v1[1:m1], …, v n [1:m n}; setting the common prefix of node to the longest common prefix v1[1:p] of the input set; trimming the input set by the common prefix of node, denoted as {v1[p + 1:m1], …, v n [p + 1:m n}; substituting {v1[p + 1:m1], …, v n [p + 1:m n} into the conversion function F to determine the number of entries of node and construct the discretization model of node. The determination methods of the number of entries and the discretization model of node are the same as those of the field value learning mapping tree diagram; determining the entry list of node according to the input set and the discretization model of node. The determination method of the entry type of the entries in the entry list is the same as that of the entries in the root node; when a certain entry of node is determined to be a data entry, the byte array obtained by trimming the corresponding input field value v i by the common prefix of node, that is, v i [p + 1:m i is determined as the suffix stored in this entry; when a certain entry of node is determined to be a node entry, assuming that the input field values located by the discretization model of node for this node entry are v i [1:m i , …, v j [1:m j , where i < j, then use {v i [p + 1:m i , …, v j [p + 1:m j]} is used as input to recursively call the new node construction operation, and the memory address pointer of the constructed new node is determined as the child node pointer stored in the entry.

[0018] The embodiments of the present application have the following beneficial effects: The two-dimensional table compression representation proposed in this application can directly perform single-point query and range query of primary keys without decompressing data, so it belongs to a queryable compression method. In a single-point query operation, locating the primary key only requires accessing a certain data entry from top to bottom along a single path starting from the root node, and each node along the way only accesses one entry. Therefore, the queryable compression method proposed in this application has the ability to accurately locate. In addition, the construction, addition, deletion and modification of the above-mentioned compression representation are all carried out around the learning-type mapping tree graph. The specific technical effect can be explained by comparing it with the mapping compression method based on the prefix tree: both methods support the same prefix of different primary keys or field values ​​to share storage space, so both can greatly compress the storage space of primary keys and byte array type field values; in the learning-type mapping tree graph proposed in this application, each node covers a common prefix composed of multiple bytes, while each node in the traditional prefix tree only stores a single byte. Therefore, the depth of the learning-type mapping tree graph proposed in this application is smaller than that of the traditional prefix tree, and can more efficiently perform the addition, deletion, modification and query operations of two-dimensional table records.

[0019] The above description is only an overview of the technical solution of this application. In order to more clearly understand the technical means of this application, which can be implemented in accordance with the contents of the description, and to make the above and other purposes, features and advantages of this application more obvious and easy to understand, the following preferred embodiments are specifically described in detail with reference to the accompanying drawings. It should be understood that the above general description and the detailed description below are only exemplary and explanatory and do not limit this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] To more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be derived from these drawings without inventive effort.

[0021] in: Figure 1 A flowchart of a queryable compression method for a two-dimensional table in a database provided by an embodiment; Figure 2 A schematic diagram of the structure of a field value learning mapping tree diagram provided by an embodiment; Figure 3 A schematic diagram of the structure of a primary key learning mapping tree diagram provided by an embodiment; Figure 4 The present invention is a schematic block diagram of the structure of a computer device provided by an embodiment. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0023] Column-based compression primarily leverages the similarity of data within a column to achieve efficient compression, reducing storage space and optimizing query performance. Currently, a wide range of column-based compression methods exist, including prefix compression, which uses common prefixes within column values ​​to reduce storage; bit vector encoding, which represents column values ​​using bitmaps; dictionary encoding, which replaces the original value with an integer ID; difference encoding, which stores the difference between adjacent values ​​and is suitable for increasing or decreasing sequences; run-length encoding, which replaces consecutive repeated values ​​with a pair consisting of a value and a repeating number; and reference encoding, which selects the reference value and only stores the difference value, which takes up less space. Column-based compression methods other than run-length encoding can be viewed as mapping methods for single-column data. Their goal is to map the space-consuming source column values ​​to the space-saving target column values, hence the term column-mapping compression methods. Most column-mapping compression methods, such as bit vector encoding, dictionary encoding, and reference encoding, support queries without decompression: when querying compressed data, simply convert the source column values ​​mentioned in the query statement into the target column values. After querying the compressed data, the target column values ​​in the answer are then restored to the source column values ​​to obtain the correct answer.

[0024] The DeepMapping method recently published at the ICDE2024 conference is a feasible solution that supports both non-decompression queries and efficient add, delete, modify and query operations. This solution uses a neural network to learn the mapping relationship between the primary key and the values ​​of other columns in a two-dimensional table, and uses the trained neural network and the primary key list as a compressed representation of the two-dimensional table. However, the neural network obtained by DeepMapping cannot guarantee the correct prediction of the mapping value of the primary key, so it also introduces an error table to store the records of the original two-dimensional table that cannot be mapped correctly. DeepMapping cannot theoretically guarantee the upper limit of the size of the error table. In the worst case of multiple changes to the two-dimensional table data, the error table can be close to the size of the two-dimensional table, resulting in the failure to achieve data compression. In order to continuously control the size of the error table, the neural network can only be updated frequently. Therefore, DeepMapping is still not a relatively complete data query compression method.

[0025] Based on DeepMapping's mapping compression concept, a readily conceivable improvement is to use a common tree graph structure to store the mapping relationship between primary keys and other column values ​​in a two-dimensional table. One approach uses internal nodes of a balanced binary tree to store the primary key comparison objects, and leaf nodes to store the row records containing the primary keys. This approach allows for a compressed representation of each row in the two-dimensional table, but requires storing all primary keys. Another approach uses prefix tree nodes to store the components of the primary key. The components stored along the path from the root node to other internal or leaf nodes form the primary key prefix. Nodes where the primary key prefix equals the complete primary key also store the corresponding column values. In this approach, identical prefixes for different primary keys can share storage space, thus compressing the primary key storage space. Furthermore, both approaches correctly locate the primary key mapping value through top-down tree graph node access, eliminating the need for a location correction mechanism similar to DeepMapping's misclassified tables. However, the performance of add, delete, modify, and query operations in these approaches is dominated by the depth of the tree graph. Therefore, further efforts are needed to control the tree graph depth to adapt to dynamic data updates and maintain stable high performance. In summary, there is currently a lack of a more complete data compression method that supports both non-decompression query and efficient add, delete, modify and query operations. To this end, this application proposes a queryable compression method for a two-dimensional table in a database. To clearly describe the method provided in this embodiment, please refer to Figures 1 to 3 , including steps S110~S120.

[0026] The method provided in this application is applied to a database that stores a two-dimensional table, wherein the rows of the two-dimensional table are composed of records. The columns of the two-dimensional table are composed of a primary key and other column fields. The primary key is represented by a byte array, and the data type of the other column fields is either a byte array type or a numeric type. The value of one of the other column fields in each record is called the field value of the field. The byte array type is a universal data representation type; any data type, including a combination of multiple field values, can be represented as a byte array. Therefore, using a byte array to specify the data type of the primary key does not essentially impose any restrictions on the primary key. In addition, the classification of the other column data into two types, numeric and byte array, rather than simply specifying it as a byte array type, is because the field compression method proposed in this application is more suitable for byte array types. Numeric field values ​​can be compressed using other mapping methods such as dictionary encoding and reference system encoding. This application does not limit the compression of numeric field values. However, considering the compression of numeric field values ​​based on the technical solution of this application is intuitive and natural, and should not be considered to exceed the technical protection scope of this application.

[0027] This application compresses and represents two-dimensional tables by adding three new data structures, including value arrays, field value learning mapping tree graphs, and primary key learning mapping tree graphs.

[0028] The database includes multiple value arrays, and the value arrays have a one-to-one correspondence with the fields of the byte array type. The value arrays store all different field values ​​of the corresponding byte array type fields in the two-dimensional table.

[0029] The database includes multiple field value learning mapping tree graphs. The field value learning mapping tree graphs have a one-to-one correspondence with the byte array type fields. The field value learning mapping tree graph is used to store the mapping relationship between the field value and the field value number. The field value number is the sequence number of the field value in the corresponding value array. It is also worth noting that only the byte array type field has a value array and a field value learning mapping tree graph, while the numeric type field does not. For the data structure of the field value learning mapping tree graph, please refer to Figure 2 As shown. The field value learning mapping tree diagram includes multiple nodes, each node includes a discretization model, a common prefix and an entry list. The discretization model is used to map the field value after clipping the upper-level common prefix to the entry number in the entry list. The common prefix is ​​the common part of all field values ​​covered in the subtree with the corresponding node as the root. The entry list includes two types of entries: node entries and data entries. Among them, the node entry is used to store the child node pointer; the data entry is used to store the suffix and field value number of the corresponding field value.

[0030] The database also includes a primary key learning type mapping tree diagram, which is used to store the mapping relationship between a given primary key and other column value tuples. The byte array type field values ​​in other column value tuples are represented as the sequence number in the corresponding value array, while the numeric type field values ​​in other column value tuples are represented as the field value itself. For the data structure of the primary key learning type mapping tree diagram, please refer to Figure 3 As shown. The structure of the primary key learning mapping tree diagram is similar to that of the field value learning mapping tree diagram. The primary key learning mapping tree diagram includes multiple nodes, each of which includes a discretization model, a common prefix, and an entry list. The discretization model is used to map the primary key after clipping the upper common prefix to the entry number in the entry list. The common prefix is ​​the common part of all primary keys covered in the subtree with the corresponding node as the root. The entry list also includes node entries and data entries. Among them, the node entry is used to save the child node pointer; the difference is that the data entry in the primary key learning mapping tree diagram is used to save the suffix of the corresponding primary key and other column value tuples. The field value of the byte array type in the value tuple is represented by the field value number.

[0031] Step S110: Execute a construction operation to construct a value array of all byte array type fields, a field value learning mapping tree diagram of all byte array type fields, and a unique primary key learning mapping tree diagram.

[0032] In one embodiment, the overall execution process of the construction operation is to first read the different values ​​of each byte array type field to construct a value array for each byte array type field, then use these value arrays to construct a learning mapping tree graph for each field value, and finally construct a primary key learning mapping tree graph.

[0033] In one embodiment, when the construction operation is used to construct a field value learning mapping tree diagram of a byte array type field, the construction operation specifically includes: First, obtain all the field values ​​of the byte array type fields stored in the two-dimensional table. Determine the first k-key number N according to all the field values H(a[1:k]) and the second k-bond number N H(a[1:k]),b The first k-key number N H(a[1:k]) Defined as the number of field values ​​whose prefix H function value is equal to H(a[1:k]); the second k-key number N H(a[1:k]),b The function H defined as a prefix is ​​equal to the number of field values ​​in H(a[1:k]) whose k+1th byte is equal to b. Here, k is the number of bytes, b is a byte, and H is a predefined function that maps character arrays to non-negative numbers.

[0034] Determine the byte conditional probability P(b|H(a[1:k])) based on the first k-key number and the second k-key number. The calculation method can be referred to: (1) The byte array probability P(a[1:k]) is defined iteratively based on the byte conditional probability: P(a[1:0])=1; and for any k≥0, P(a[1:k+1])=P(a[1:k])·P(a[k+1]|H(a[1:k])).

[0035] The conversion function F(a[1:k]) is iteratively defined based on the byte conditional probability P(b|H(a[1:k])) and the byte array probability P(a[1:k]): F(a[1:0])=0; and for any k≥0, F(a[1:k+1])=F(a[1:k])+P(a[1:k])∑ b∈{0,1,…,a[k+1]-1} P(b|H(a[1:k])).

[0036] Therefore, it can be proved that the transfer function F is a monotonic transfer function. Assume that the value range of the H function is 0 to N max -1, for all different H(a[1:k]) and b, N H(a[1:k]) and N H(a[1:k]),b All of them can be obtained by traversing all the field values ​​of the target field once, where N H(a[1:k]) Occupancy Nmax Storage space for a floating - point number, N H(a[1:k]),b Occupies 256N max Storage space for floating - point numbers. In a specific embodiment, H(a[1:k]) is defined as 256·a[1]+a[m], and when m < 2, 0 - byte padding is added after a[1:m] to make up 2 characters. Then the N corresponding to this H function max = 65536.

[0037] The following proves that the conversion function F is a monotonic conversion function: For any character arrays a1[1:m] and a2[1:n] that satisfy a1 < a2, there must exist an index k≥1 such that a1[1:k - 1]=a2[1:k - 1] and 0≤a1[k]<a2[k]≤255. Here, for any k > m, a1[k] is defined as 0, and for any k > n, a2[k] is defined as 0. Denote the byte array obtained by concatenating a1[1:p] with 256 as a3[1:p + 1], that is, a3[1:p]=a1[1:p] and a3[p + 1]=256. Then we have: F(a3[1:k + 1]) = F(a1[1:k])+P(a1[1:k]) = F(a1[1:k - 1])+P(a1[1:k - 1])∑ b∈{0,…,a1[k]-1} P(b|H(a[1:k - 1]))+P(a1[1:k - 1])P(a1[k]|H(a1[1:k - 1])) = F(a2[1:k - 1])+P(a2[1:k - 1])∑ b∈{0,…,a1[k]} P(b|H(a[1:k - 1])) <F(a2[1:k - 1])+P(a2[1:k - 1])∑ b∈{0,…,a2[k]-1} P(b|H(a[1:k - 1])) = F(a2[1:k]).

[0038] Similarly, it can be proved that F(a3[1:k + 2])<F(a3[1:k + 1]), ……, F(a3[1:m])<F(a3[1:m - 1]). Therefore, F(a1[1:m])<F(a3[1:m])<……<F(a3[1:k + 1])<F(a2[1:k]) ≤ F(a2[1:n]), and thus it can be known that F is a monotonic conversion function.

[0039] After determining F, call the construction process of the root node to complete the construction of the field - value learning - type mapping tree diagram. Construct the input set {v1, …, v N} according to all field values in the corresponding value array of the byte - array - type field, where N is the number of different field values of the byte - array - type field, and v iis the field value, i∈[1,N], v1,…,v N Arrange in ascending order; the construction process is based on the set of different field values ​​{v1,…,v N} as input, field value v1<…… <v N , where v i <v j means that there exists k such that v i [1:k]=v j [1:k], and m i <km j or v i [k] <v j [k], where m i and m j They are v i and v j The length of v i =v i [1:m i ] and v j =v j [1:m j ].

[0040] Set the common prefix of the root node of the field value learning mapping tree graph of the byte array type field to an empty byte array.

[0041] Determine the number of entries n of the root node according to the number of field values ​​N and the preset expected number of entries E; use the input set as the input of the conversion function F, and according to the first processing result F(v i )Construct a discretized model M.

[0042] Since the number of partition points in M(x) is n-1, we only need to determine these n-1 partition points to determine M(x). n and M(x) are determined together as follows.

[0043] First, the discretization interval size s is determined based on the number of field values ​​N and the preset expected number of entries E. The calculation process is: s = ceil(N / E) (2) In the above formula, ceil(x) returns the smallest integer not less than x.

[0044] To simplify the description, we introduce v N1 And assume that v N1 is an empty byte array, where N1 is any integer greater than N.

[0045] Determine the division points of the discretization model M one by one: The left neighbor v of the first partition point s1 is v1, ..., v s In the equation “v1,…,vs1 The longest common prefix is non-empty, and s1 = s or v1, …, v s1+1 The field value with the smallest serial number for which the longest common prefix is empty, and the right neighbor of the first division point is v s1+1 , determine the first division point as (F(v s1 )) + F(v s1+1 )) / 2; By analogy, the left neighbor v sk of the k≥2-th division point is v i , …, v i+s satisfying "v i , …, v sk The longest common prefix is non-empty, and s k = i + s or v i , …, v sk+1 The field value with the smallest serial number for which the longest common prefix is empty, where v i is the right neighbor of the (k - 1)-th division point.

[0046] After that, it can be directly judged: If s k = N, then determine n = k and M(x) has k - 1 division points; otherwise, the right neighbor of the k-th division point is v sk+1 , and the k-th division point is determined as (F(v sk )) + F(v sk+1 )) / 2. Aggregate all division points to obtain the discretization model M.

[0047] Use the first processing result corresponding to each field value in the input set as the input of the discretization model M and substitute it for processing to obtain the corresponding second processing result, and the second result can be expressed as M(F(v i )). Determine the entry list according to the second processing result.

[0048] Determine the entry list according to the second processing result, including: Assuming M(F(v0)) < M(F(v1)) and M(F(v N )) < M(F(v N+1 )) is always true, traverse all the second processing results M(F(v i )) and construct the entry list of the root node according to the following method, where the entry serial number M(F(vi)) is between 0 and n - 1: If there exists an i such that M(F(v i-1 )) < M(F(v i )) < M(F(v i+1 )), then determine the entry with the serial number M(F(v i )) as a data entry, and the data entry is used to save the suffix of v i and the number of v i ; If there exists \(i < j\) such that \(M(F(v i-1 )) < M(F(v i )) = M(F(v i+1 )) = \(\cdots\) = M(F(v j )) < M(F(v j+1 )),then recursively call the new node construction operation with \(\{v i ,\cdots,v j \}\) as the input, and determine the entry with the serial number \(M(F(v i ))\) as the node entry, and the node entry is used to store the memory address pointer of the new node.

[0049] Furthermore, through the above-described method, summarize all the nodes that have determined the discretization model \(M\), the common prefix, and the entry list to obtain the learning mapping tree diagram of the field value of the byte array type field.

[0050] The learning mapping tree diagram of the byte array type field proposed in this application can be easily adapted to the secondary index structure of this field, and only need to additionally store the list of primary keys corresponding to the corresponding field values in the data entries. Therefore, considering the secondary index of the byte array type field based on the technical solution of this application is also simple and intuitive, and should not be considered to exceed the technical protection scope of this application.

[0051] In one embodiment, when the construction operation is used to construct the primary key learning mapping tree diagram, the construction operation is similar to the construction process of the field value learning mapping tree diagram, and the differences between the two will be described later. For the parts not described, the process of constructing the field value learning mapping tree diagram in the previous text can be directly referred to.

[0052] First, obtain all the primary keys stored in the two-dimensional table, and respectively determine the third \(k -\) key number \(N H(a[1:k]) \) and the fourth \(k -\) key number \(N H(a[1:k]),b \). The third \(k -\) key number \(N H(a[1:k]) \) is defined as the number of primary keys whose \(H\) function value of the prefix in all primary keys is equal to \(H(a[1:k])\); the fourth \(k -\) key number \(N H(a[1:k]),b \) is defined as the number of primary keys whose \(k + 1\) - th byte is equal to \(b\) among all primary keys whose \(H\) function value of the prefix is equal to \(H(a[1:k])\). Similarly, \(k\) is the number of bytes, \(b\) is a byte, and \(H\) is a preset function that maps from a character array to a non - negative number. Determine the byte conditional probability \(P K \)(b|H(a[1:k]))\), and the calculation method is: (3) Iteratively define the byte array probability \(P K \)(a[1:k])\) according to the byte conditional probability: PK (a[1:0])=1; and for any k≥0, P K (a[1:k+1])=P K (a[1:k])·P K (a[k+1]|H(a[1:k])).

[0053] According to the byte conditional probability P K (b|H(a[1:k])) and byte array probability P K (a[1:k]) iteratively defines the transformation function F K (a[1:k]): F K (a[1:0])=0; and for any k≥0, F K (a[1:k+1])=F K (a[1:k])+P K (a[1:k])∑ b∈{0,1,…,a[k+1]-1} P K (b|H(a[1:k])).

[0054] Then, the construction process of the root node is called. This construction process is basically the same as the construction process of the field's learning mapping tree diagram, which is still a process using the conversion function F K The recursive process includes determining the common prefix, determining the number of entries, constructing a discretization model and an entry list. The only difference is that the set of all primary keys in ascending order {key1,…,key N} as input. The conversion function F is used to construct the discretization model. K Instead of F; after the entry in the entry list is determined as a data entry, the entry storage stores the suffix corresponding to the primary key and the other column value tuples, wherein the values ​​corresponding to each byte array type field in the value tuple are replaced with the number of the field value through the single-point query operation of the field value in the corresponding learning mapping tree diagram. The single-point query operation of the field value is basically the same as the primary key single-point query operation explained below. The only difference is that when the query is successful, the number of the field value, that is, the sequence number of the field value in the value array, is returned instead of returning a data tuple consisting of a value and a byte array.

[0055] Finally, all nodes with determined discretization models, common prefixes, and entry lists are aggregated to obtain a primary key learning mapping tree graph.

[0056] According to the construction operation in the previous article, the construction of two learning mapping tree graphs mainly relies on the monotonic conversion function. The conversion function F from the primary key (or field value) byte array to floating point number introduced in the application is K(or F) all use a preset mapping function H from byte arrays to non-negative integers. The specific definition of this mapping function does not affect the monotonicity of the conversion function. This application does not further define this mapping function H, but only provides a specific calculation formula as an example. Modifying this calculation method should not be considered beyond the technical protection scope of this application.

[0057] Step S120: performing add, delete, modify, and query operations on the primary key learning mapping tree diagram according to the received operation instructions. The add, delete, modify, and query operations include primary key single point query, primary key range query, two-dimensional table record addition operation, two-dimensional table record deletion operation, and two-dimensional table record modification operation.

[0058] This application proposes a queryable compression solution for byte arrays. Therefore, it focuses only on the compression of primary keys and byte array fields in two-dimensional tables and does not involve the compression of numeric fields. The specific execution process of the add, delete, modify, and query operations will be described later.

[0059] In one embodiment, when the operation instruction is a primary key single-point query operation, the specific execution process may include: Determine the target key to be queried in the instruction content of the primary key single-point query operation, call the locate(node, key) function, the first parameter node of the locate(node, key) function is a node in the primary key learning mapping tree diagram, set as the root node of the primary key learning mapping tree diagram; the second parameter key is the byte array to be queried, set as the target key. Return locate(root node, target key), the locate(node, key) function will be the node with the serial number M(F K (key)) is marked as a query entry, where M is the discretized model of node, F K is a monotonic transformation function of the primary key learning mapping tree graph.

[0060] Determine the type of query entry: When the query entry is a data entry, the query entry is marked as a positioning entry, and the common prefixes of all nodes in the path from the root node to the positioning entry and the suffix stored in the positioning entry are concatenated into a comparison key.

[0061] Determine whether the comparison key is the same as the target key: If the comparison key is not equal to the target key, the target key cannot be found information is returned; If the comparison key is equal to the target key, the other column value tuple stored in the located entry is decoded and the decoded data tuple is returned. The decoding process for the other column value tuple is to replace the field value number of all byte array type fields in the other column value tuple with the byte array numbered by the field value number in the corresponding value array. Decoding refers to replacing the field value number corresponding to each byte array type field in the value tuple with the corresponding byte array in the field value array. A data tuple is a tuple composed of elements of different types.

[0062] When the query entry is a node entry, access the child node through the child node pointer stored in the query entry. Determine whether the common prefix stored in the child node is the same as the prefix of the target key: If the common prefix stored in the child node is the same as the prefix of the target key, the target key is pruned of the common prefix of the child node to obtain the remaining target key, and the locate(child node, remaining target key) function is returned for the primary key single-point query operation. If the common prefix stored in the child node is different from the prefix of the target key, the query entry is marked as a located entry and the information that the target key cannot be found is returned.

[0063] In one embodiment, a primary key range query can be considered as executing a series of primary key single-point query operations. The specific execution process may include: First, determine the lower limit query key and upper limit query key in the primary key range query instruction. Execute a primary key point query based on the lower limit query key, determine the primary key point query's location entry, and mark it as the start entry. Execute a primary key point query based on the upper limit query key, determine the primary key point query's location entry, and mark it as the end entry.

[0064] Traverse the data entries in all nodes between the start and end entries using a depth-first traversal of nodes and from left (small) to right (large) traversal. Collect data entries from the set of all traversed entries, including the start and end entries, and return the corresponding primary keys and other column data tuples for these data entries. Obtain the primary key and other column value tuples stored in each data entry and return the primary key and data tuple. The data tuple is obtained by decoding the other column value tuple. The decoding process for the other column value tuple is to replace the field value numbers of all byte array type fields in the other column value tuple with byte arrays in the corresponding value array with the field value number as the sequence number.

[0065] In one embodiment, when the operation instruction is an operation of adding a record to a two-dimensional table, the specific execution process may include: Determine the two-dimensional table record to be added in the instruction content of the two-dimensional table record addition operation, and record the two-dimensional table record to be added as recordnew_1 , record new_1 The primary key is denoted as key new_1 . For key new_1 Execute a primary key single-point query to determine the positioning entry of the primary key single-point query, which is called the first target entry.

[0066] Determine each node in the path from the root node to the first target entry and mark it as a path node. Concatenate the common prefixes of all path nodes and mark the result as prefix. new_1 The result after trimming the prefix is ​​marked as suffix new .

[0067] Determine the entry type of the first target entry: If the first target entry is a data entry, get the suffix originally stored in the first target entry, recorded as suffix old If suffix old with suffix new If they are consistent, then according to record new_1 Construct a new value tuple, and update the other column value tuples originally stored in the first target entry to the new value tuple; If the first target entry is a node entry, the child node pointed to by the first target entry is marked as the first invalid node. All remaining primary keys and suffixes covered in the subtree with the first invalid node as the root new The new node construction operation is called with the set as input to construct the first new node. Each remaining primary key is formed by concatenating the common prefix of all nodes in the path from the first failed node to a data entry and the suffix of the data entry. The new node construction operation will be explained in detail later and is not described in detail here. The child node pointer stored in the first target entry is replaced with the memory address pointer of the first new node, and the first failed node is deleted.

[0068] After the above tree graph is updated, there must be a data entry corresponding to the key in the tree graph. The data entry stores the numerical tuple of other columns constructed by the following method: If a byte array type field value corresponding to the key is not in the value array of the corresponding field, first add the new field value to the value array of the corresponding field, and then add the mapping relationship between the new field value and its number in the learning mapping tree graph of the corresponding field. The process of adding a new field value in the learning mapping tree graph of the field is basically the same as the process of adding a new primary key in the learning mapping tree graph of the primary key. The only difference is that the generated data entry stores the number of the field value instead of the numerical tuple.

[0069] Finally, consider each node starting from the root node in the path from the root node to the node where the target entry is located: determine whether each path node meets the preset reconstruction condition. When any node in the path node meets the reconstruction condition, the path node that meets the reconstruction condition is marked as a reconstructed node. Call the new node construction operation with the set of all remaining primary keys covered in the subtree with the reconstructed node as the root as input to construct a second new node. Replace the child node pointer stored in the node entry pointing to the reconstructed node with the memory address pointer of the second new node, and delete the reconstructed node. Among them, each remaining primary key is composed of the common prefix of all nodes in the path from the reconstructed node to a data entry and the suffix of the data entry.

[0070] In a specific embodiment, the reconstruction condition is that the total number of nodes in the subtree where the path node is located is not less than a preset node number threshold (for example, 64), the number of entries in the entry list of the path node is not less than a preset multiple (for example, 2), and the proportion of data entries in all entries covered by the subtree where the path node is located is not less than a preset proportion (for example, 10%).

[0071] It is worth noting that the new primary key (or new field value) addition operation process proposed in this application requires that when a node in the new primary key (or new field value) addition path meets the preset reconstruction condition, a subtree with the node as the root is rebuilt, that is, a new node is constructed using the set of all primary key suffixes (or all field value suffixes) contained in the subtree as input, and the subtree with the new node as the root replaces the subtree with the node as the root. This application does not make specific restrictions on the preset reconstruction conditions, but only provides an example of a definition framework for the reconstruction conditions, clarifying that the specific assignment of parameters in the definition framework or changes to the definition framework should not be considered to exceed the technical protection scope of this application.

[0072] In one embodiment, when the operation instruction is a two-dimensional table record deletion operation, the specific execution process may include: Determine the primary key of the two-dimensional table record to be deleted in the instruction content of the two-dimensional table record deletion operation, and record the two-dimensional table record to be deleted as record old_1 , record old_1 The primary key is denoted as key old_1 .

[0073] Key old_1 Execute the primary key single point query to determine the primary key single point query positioning entry, called the second target entry. old_1 If the information is not found, the deletion operation is terminated; otherwise, the node where the second target entry is located is marked as the second failed node.

[0074] Taking the set of all remaining primary keys in the subtree rooted at the second failed node that do not involve the second target entry as input, a new node construction operation is invoked to construct a second new node. Each remaining primary key is formed by concatenating the common prefix of all nodes in the path from the second failed node to a data entry different from the second target entry and the suffix of the data entry. The child node pointer stored in the node entry pointing to the second failed node is replaced with the memory address pointer of the second new node, and the second failed node is deleted.

[0075] In one embodiment, when the operation instruction is a two-dimensional table record modification operation, the specific execution process may include: Determine the original primary key and the modified two-dimensional table record in the instruction content of the two-dimensional table record modification operation, and record the original primary key as key old_2 , key old_2 The corresponding two-dimensional table record is recorded as record old_2 , record the modified two-dimensional table record as record new_2 , record new_2 The primary key is denoted as key new_2 .

[0076] Determine key old_2 and key new_2 Are they the same? If key old_2 and key new_2 If they are not the same, first use record new_2 As the instruction content, perform the two-dimensional table record addition operation, and then use record old_2 The instruction content is to execute the two-dimensional table record deletion operation.

[0077] If key old_2 and key new_2 If they are consistent, then the key new_2 Execute a primary key single-point query to determine the positioning entry of the primary key single-point query, which is called the third target entry.

[0078] If the primary key single point query returns key new_2 If the information is not found, record new_2 As the instruction content, perform the two-dimensional table record addition operation, otherwise according to the record new_2 Construct a new value tuple and change the other column value tuples stored in the third target entry to the new value tuple.

[0079] In one embodiment, the above-mentioned record new Construct a new value tuple. The specific execution process may include: According to record newDetermine the other columns except the primary key and perform the following processing on the byte array type fields in the other columns: Get record new The field value of the byte array type field and the corresponding value array of the byte array type field are included. If the field value is not in the value array, the field value is marked as a new field value, the new field value is added to the value array, and the mapping relationship between the new field value and the number is added to the field value learning mapping tree diagram.

[0080] The process of adding a new field value in the field value learning mapping tree is basically the same as the process of adding a new primary key in a two-dimensional table record addition operation. The only difference is that the generated data entry stores the number of the new field value. Finally, the new value tuple is represented by record new It is composed of the field value number of the byte array type field and the original field value of the numeric type field in other column fields.

[0081] In the above-described learning-based mapping tree graph construction operation and two-dimensional table record addition, deletion, and modification operations, the new node construction operation is recursively called. In one embodiment, the new node construction operation is similar to the root node construction operation, and the specific execution process may include: The new node is recorded as node, and the input set required to construct node is obtained. The input set is the set of all input field values ​​required to construct the node, recorded as {v1[1:m1], ..., v n [1:m n ]}. Set the common prefix of node to the longest common prefix v1[1:p] of the input set; trim the common prefix of node from the input set and record it as {v1[p+1:m1], ..., v n [p+1:m n ]}.

[0082] Use {v1[p+1:m1], ..., v n [p+1:m n ]}Substitute into the conversion function F to determine the number of entries of node and construct the discretization model of node. The method for determining the number of entries and discretization model of node is consistent with the method for determining the number of entries and discretization model of field value learning mapping tree graph.

[0083] The entry list of the node is determined according to the input set and the discretization model of the node. The method for determining the entry type of the entries in the entry list is consistent with the method for determining the entry type of the entries in the root node.

[0084] When an entry of a node is determined to be a data entry, the corresponding input field value v i The byte array obtained after trimming the common prefix of node is v i [p+1:m iDetermine the suffix stored for this entry. When an entry of a node is determined to be a node entry, assume that the input field value of this node entry located through the discretization model of the node is v i [1:m i , …, v j [1:m j , where i < j, then use {v i [p + 1:m i , …, v j [p + 1:m j} as the input to recursively call the new node construction operation, and determine the memory address pointer of the newly constructed node as the child node pointer stored for this entry.

[0085] Compared with the prior art, there are three key differences in this application: (1) Monotonic conversion function from byte array to floating point number To build a simple and efficient prediction model for the primary key (or field value) entry number in the learning-based mapping tree graph node of the primary key (or field), it is necessary to convert the byte array into a floating point number so that the prediction model can use the floating point number as the input. In addition, to ensure the efficiency of range queries, that is, to ensure that all primary keys (or field values) between the target entry corresponding to the range upper limit and the target entry corresponding to the range lower limit are the exact result of the range query, it is necessary to ensure that the conversion function from byte array to floating point number is monotonic. A relatively simple and intuitive conversion function G is defined as G(a[1:m]) = a[1] / 256 + a[2] / 256 2 + … + a[m] / 256 m . It can be proved that this conversion function is also monotonic, that is, a1[1:m] < a2[1:n] means G(a1[1:m]) < G(a2[1:n]). This conversion function is equivalent to fixing the conditional probability P(b|H(a[1:k])) (or P K (or F)) used in the conversion function F K (b|H(a[1:k])) in this application to 1 / 256. Due to the finite precision truncation problem of floating point numbers in computers, using a fixed conditional probability will cause a large number of close primary keys (or field values) in the learning-based mapping tree graph to be located to the same entry number, resulting in a deeper subtree for the node entry corresponding to this entry number to distinguish different primary keys (or field values). In this application, the corresponding conditional probability is determined according to the distribution of primary keys (or field values) in the given two-dimensional table, so that close primary keys (or field values) can be distributed to different entry numbers as much as possible, thereby reducing the subtree depth of the node entries corresponding to different entry numbers and improving the query efficiency.

[0086] (2) Setting the entry type and prediction model in the learning mapping tree node In order to save storage space in the learning-based mapping tree graph, the entry types in the tree graph nodes are limited to data entries or node entries, unlike cutting-edge learning indexing methods such as SIndex, LIPP, and LITS, which include empty entries. Although the introduction of empty entries can reserve space for data changes and avoid generating new tree graph nodes every time a byte array is inserted or deleted, empty entries will reduce data compression and query efficiency, which outweighs the benefits in queryable compression applications. In addition, the positioning prediction model of the learning-based mapping tree graph nodes in this application adopts a discretization model, unlike the linear model used in learning indexing methods such as SIndex, LIPP, and LITS. Linear models make it difficult to ensure that the input floating-point numbers can be evenly distributed among different entry numbers, resulting in some node entries needing to distinguish a large number of input floating-point numbers, which leads to the subtree depth of the node entry being too large. The discretization model used in this application can ensure that the primary keys (or field values) in each discrete interval have a non-empty common prefix through the calculation of the partitioning points, and the number of different primary keys (or field values) in each discrete interval is as equal as possible. Requiring that all primary keys (or field values) within a discrete interval have a non-empty common prefix ensures that each non-root node in the tree graph stores a portion of the primary key (or field value), minimizing the depth of the tree graph. Requiring that each discrete interval cover a roughly equal number of distinguished objects ensures that the depths of the subtrees corresponding to each discrete interval do not vary significantly, thus ensuring that the overall depth of the tree graph is also kept within a certain range.

[0087] (3) Segmented storage of byte arrays in learning mapping tree graphs The learned mapping tree graph for primary keys (or fields) proposed in this application stores the byte array contents of all different primary keys (or all different field values ​​of a single field) in a given two-dimensional table. Each complete primary key (or field value) is divided into different parts and stored along a path from the root node to a data entry, where the nodes on the path store the different components of the byte array prefix, and the data entry at the end of the path stores the suffix of the byte array. This storage method saves storage space for the byte array of primary keys (or field values) because the same prefix of different byte arrays can share storage space. In addition, this storage method also reflects the advantages of the learned mapping tree graph over the traditional prefix tree, namely, each node stores multiple bytes in the byte array, while each node of the traditional prefix tree only stores a single byte in the byte array. Therefore, single-point query operations based on the learned mapping tree graph can locate the target data entry by accessing fewer nodes, compared to single-point query operations based on the traditional prefix tree, thereby achieving higher query efficiency.

[0088] Therefore, the two-dimensional table compression representation proposed in this application can directly perform single-point query and range query of the primary key without decompressing the data, so it belongs to a queryable compression method. In a single-point query operation, locating the primary key only requires accessing a certain data entry from top to bottom along a single path starting from the root node, and each node along the way only accesses one entry. Therefore, the queryable compression method proposed in this application has the ability to accurately locate and does not need to introduce a positioning correction mechanism similar to the misclassification table in DeepMapping. In addition, the construction, addition of records, deletion of records and modification of records in the above-mentioned compression representation are all carried out around the learning-type mapping tree graph. The specific technical effect can be explained by comparing it with the mapping compression method based on the prefix tree: both methods support the same prefix of different primary keys or field values ​​to share storage space, so both can greatly compress the storage space of primary keys and byte array type field values; in the learning-type mapping tree graph proposed in this application, each node covers a common prefix composed of multiple bytes, while each node in the traditional prefix tree only stores a single byte. Therefore, the depth of the learning-type mapping tree graph proposed in this application is smaller than that of the traditional prefix tree, and can more efficiently perform the addition, deletion, modification and query operations of two-dimensional table records.

[0089] Figure 4 FIG1 shows an internal structure diagram of a computer device in an embodiment. The computer device can be a terminal or a server. Figure 4 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor may implement a queryable compression method for a two-dimensional table of a database. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor may implement a queryable compression method for a two-dimensional table of a database. It will be understood by those skilled in the art that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0090] In one embodiment, the present application further proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the method described in any of the aforementioned embodiments.

[0091] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0092] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A queryable compression method for a two-dimensional table of a database, applied to a database, wherein the database is used to store a two-dimensional table, wherein the rows of the two-dimensional table are composed of records, and the columns of the two-dimensional table are composed of primary keys and other column fields, wherein the primary keys are represented by byte arrays, and the data types of the other column fields are one of byte array types and numerical types, and the value of one other column field in each record is called the field value of the field; It is characterized in that The database adds a data structure to compress and represent the two-dimensional table; the data structure includes a value array, a field value learning mapping tree diagram, and a primary key learning mapping tree diagram; The database includes a plurality of the value arrays, the value arrays have a one-to-one correspondence with the fields of the byte array type, and the value arrays store all different field values ​​corresponding to the fields of the byte array type in the two-dimensional table; The database includes a plurality of the field value learning mapping tree graphs, the field value learning mapping tree graphs have a one-to-one correspondence with the fields of the byte array type, the field value learning mapping tree graphs are used to store the mapping relationship between field values ​​and field value numbers, the field value numbers are the serial numbers of the field values ​​in the corresponding value array; the field value learning mapping tree graphs include a plurality of nodes; each node includes a discretization model, a common prefix and an entry list; the discretization model is used to map the field value after clipping the upper common prefix to the entry serial number in the entry list; the common prefix is ​​the common part of all field values ​​covered in the subtree corresponding to the node as the root; the entry list includes node entries and data entries; the node entry is used to store child node pointers; the data entry is used to store the suffix corresponding to the field value and the field value number; The database includes a primary key learning mapping tree diagram, which is used to store the mapping relationship between the given primary key and other column value tuples, the field values ​​of the byte array type in the other column value tuples are represented as the serial numbers in the corresponding value arrays, and the field values ​​of the numerical type in the other column value tuples are represented as the field values ​​themselves; the primary key learning mapping tree diagram includes multiple nodes; each node includes a discretization model, a common prefix and an entry list; the discretization model is used to map the primary key after clipping the upper-level common prefix to the entry serial number in the entry list; the common prefix is ​​the common part of all primary keys covered in the subtree corresponding to the node as the root; the entry list includes node entries and data entries; the node entry is used to save child node pointers; the data entry is used to save the suffix corresponding to the primary key and the other column value tuples; The method comprises the following steps: Perform a construction operation to construct the value arrays of all byte array type fields, the field value learning mapping tree graphs of all byte array type fields, and the unique primary key learning mapping tree graph; According to the received operation instructions, the primary key learning mapping tree diagram is added, deleted, modified and checked, and the addition, deletion, modification and query operations include primary key single point query, primary key range query, two-dimensional table record addition operation, two-dimensional table record deletion operation and two-dimensional table record modification operation.

2. The queryable compression method for a two-dimensional table of a database as claimed in claim 1, characterized in that: When the construction operation is used to construct the field value learning mapping tree diagram of a certain byte array type field, the construction operation includes: Obtain all field values ​​of the byte array type field stored in the two-dimensional table; Determine the first k-key number N according to all the field values H(a[1:k]) and the second k-bond number N H(a[1:k]),b ; The first k-bond number N H(a[1:k]) Defined as the number of field values ​​whose prefix H function value is equal to H(a[1:k]) among all field values; the second k-key number N H(a[1:k]),b The function value of H defined as a prefix is ​​equal to the number of field values ​​whose k+1th byte is equal to b among all field values ​​of H(a[1:k]); k is the number of bytes, b is the byte; H is a preset function that maps from a character array to a non-negative number; The byte conditional probability P(b|H(a[1:k])) is determined according to the first k-key number and the second k-key number, and the calculation method is: ； The byte array probability P(a[1:k]) is iteratively defined according to the byte conditional probability: P(a[1:0])=1; and for any k≥0, P(a[1:k+1])=P(a[1:k])·P(a[k+1]|H(a[1:k])); The conversion function F(a[1:k]) is iteratively defined according to the byte conditional probability P(b|H(a[1:k])) and the byte array probability P(a[1:k]): F(a[1:0])=0; And for any k ≥ 0, F(a[1:k+1])=F(a[1:k])+P(a[1:k])∑ b∈{0,1,…,a[k+1]-1} P(b|H(a[1:k])); Construct an input set {v1,…,v2,v3,v4,v5,v6,v7,v8,v9,v10,v11,v12,v13,v14,v15,v16,v17,v18,v29,v20,v21,v21,v22,v23,v24,v25,v26,v27,v28,v29,v30,v31, N }, N is the number of different field values ​​of the byte array type field, v i is the field value, i∈[1,N], v1,…,v N Sort in ascending order; The common prefix of the root node of the field value learning mapping tree diagram of the byte array type field is set to an empty byte array; the number of entries n of the root node is determined according to the number of field values ​​N and the preset expected number of entries E; the input set is used as the input of the conversion function F, and according to the first processing result F(v i ) construct the discretization model M; Using the first processing result corresponding to each field value in the input set as the input of the discretization model M, and substituting it into the discretization model M to obtain the corresponding second processing result; determining the entry list according to the second processing result; Determining the entry list according to the second processing result includes: assuming that M(F(v0))<M(F(v1)) and M(F(v N ))< M(F(v N+1 )) is always true, and iterates over all second processing results M(F(v i )); If there exists i that satisfies M(F(v i-1 )) < M(F(v i )) < M(F(v i+1 )), then the sequence number is M(F(v i )) is determined as a data entry, and the data entry is used to store v i The suffix and v i Serial number; If there exist \(i < j\) such that \(M(F(v i-1 )) < M(F(v i )) = M(F(v i+1 )) = \(\cdots\) = M(F(v j )) < M(F(v j+1 ))), then recursively call the new node construction operation with \(\{v i ,\cdots,v j \}\) as the input, and determine the entry with the serial number \(M(F(v i ))\) as the node entry, where the node entry is used to store the memory address pointer of the new node; All nodes that determine the discretization model M, the common prefix, and the entry list are aggregated to obtain the field value learning mapping tree diagram of the byte array type field.

3. The queryable compression method for a two-dimensional table of a database as claimed in claim 1, characterized in that: When the construction operation is used to construct the primary key learning mapping tree graph, the construction operation includes: Obtain all primary keys stored in the two-dimensional table; Determine the third k-key number N according to all the primary keys H(a[1:k]) and the fourth k-bond number N H(a[1:k]),b ; The third k-bond number N H(a[1:k]) Defined as the number of primary keys whose prefix H function value is equal to H(a[1:k]) in all primary keys; the fourth k-key number N H(a[1:k]),b The function value of H defined as a prefix is ​​equal to the number of primary keys whose k+1th byte is equal to b among all primary keys of a[1:k]; k is the number of bytes, b is the byte; H is the preset function that maps from character arrays to non-negative numbers; Determine the byte conditional probability P according to the third k-key number and the fourth k-key number K (b|H(a[1:k])), which is calculated as: ； Iteratively define the byte array probability P according to the byte conditional probability K (a[1:k]): P K (a[1:0])=1; and for any k≥0, P K (a[1:k+1])=P K (a[1:k])·P K (a[k+1]|H(a[1:k])); According to the byte conditional probability P K (b|H(a[1:k])) and the byte array probability P K (a[1:k]) iteratively defines the transformation function F K (a[1:k]): F K (a[1:0])=0; and for any k≥0, F K (a[1:k+1])=F K (a[1:k])+P K (a[1:k])∑ b∈{0,1,…,a[k+1]-1} P K (b|H(a[1:k])); Construct an input set {key1,…,key M }, M is the number of all primary keys, key i is the primary key, i∈[1,M], key1,…,key M Sort in ascending order; The process of determining the discretization model, the common prefix and the entry list of the primary key learning mapping tree graph, except replacing the input set, the monotonic conversion function F with the input set K Except for replacing the conversion function F and the data entry for storing the suffix corresponding to the primary key and the other column value tuple, the remaining steps are consistent with the process of determining the discretization model of the field value learning mapping tree graph, the common prefix and the entry list; All nodes that determine the discretization model, the common prefix, and the entry list are aggregated to obtain the primary key learning mapping tree graph.

4. The queryable compression method for a two-dimensional table of a database as claimed in claim 2, characterized in that: The number of entries n of the root node is determined according to the number of field values ​​N and the preset expected number of entries E; the input set is used as the input of the conversion function F, and the first processing result F(v i ) construct the discretization model M, comprising: The discretization interval size s is determined according to the number of field values ​​N and the preset expected number of entries E. The calculation process is: s = ceil(N / E); ceil(x) returns the smallest integer not less than x; Introducing v N1 Assume that v N1 is an empty byte array, where N1 is any integer greater than N; determine the partition points of the discretization model M one by one: the left neighbor v of the first partition point s1 is v1, …, v s In the equation "v1, ..., v s1 The longest common prefix of is not empty, and s1=s or v1, ..., v s1+1 The longest common prefix of "is empty" is the smallest field value, and the right neighbor of the first partition point is v s1+1 , the first dividing point is determined as (F(v s1 )+F(v s1+1 )) / 2; the left neighbor v of the k≥2 partition point sk Yes i ,…,v i+s In the satisfied "v i ,…,v sk The longest common prefix of is not empty, and s k =i+sorv i ,…,v sk+1 The longest common prefix of the "empty" field value with the smallest sequence number, where v i It is the right neighbor of the k-1th partition point; If k =N, then n=k and M(x) has k-1 partition points, otherwise the right neighbor of the kth partition point is v sk+1 , the kth partition point is determined as (F(v sk )+F(v sk+1 )) / 2; The discretization model M is obtained by summing up all the division points.

5. The queryable compression method for a two-dimensional table of a database as claimed in claim 1, characterized in that: The primary key single-point query operation includes: Determine the target key to be queried in the instruction content of the primary key single-point query operation, call the locate(node, key) function, the first parameter node of the locate(node, key) function is a node in the primary key learning mapping tree diagram, which is set as the root node of the primary key learning mapping tree diagram; the second parameter key is the byte array to be queried, which is set as the target key; the locate(node, key) function locates the node with the sequence number M(F K (key)) is marked as a query entry, where M is the discretization model of node, F K is a monotonic transformation function of the primary key learning mapping tree graph; Determining the item type of the query item; When the query entry is a data entry, the query entry is marked as a positioning entry, and the common prefixes of all nodes in the path from the root node to the positioning entry and the suffix stored in the positioning entry are concatenated into a comparison key; and it is determined whether the comparison key is the same as the target key; If the comparison key is not equal to the target key, information that the target key cannot be found is returned; if the comparison key is equal to the target key, the other column value tuple stored in the positioning entry is decoded, and the data tuple obtained after decoding is returned, and the decoding process for the other column value tuple is to replace the field value numbers of all byte array type fields in the other column value tuple with the byte array with the field value number as the serial number in the corresponding value array; When the entry type of the query entry is a node entry, accessing a child node through a child node pointer stored in the query entry; determining whether a common prefix stored in the child node is the same as a prefix of the target key; If the common prefix stored in the child node is the same as the prefix of the target key, the target key is trimmed of the common prefix of the child node to obtain the remaining target key, and locate (the child node, the remaining target key) is returned; if the common prefix stored in the child node is different from the prefix of the target key, the query entry is marked as a locate entry, and the information that the target key cannot be found is returned; The primary key range query includes: Determine a range lower limit query key and a range upper limit query key to be queried in the instruction content of the primary key range query; Execute the primary key point query according to the lower limit query key of the range, determine the location entry of the primary key point query, and mark it as the start entry; execute the primary key point query according to the upper limit query key of the range, determine the location entry of the primary key point query, and mark it as the end entry; Traverse the data entries in all nodes between the start entry and the end entry in a depth-first manner; obtain the primary key and other column value tuples stored in each of the data entries, and return the primary key and data tuple, wherein the data tuple is obtained by decoding the other column value tuple, and the decoding process for the other column value tuple is to replace the field value numbers of all byte array type fields in the other column value tuple with byte arrays in the corresponding value array with the field value number as the serial number.

6. The queryable compression method for a two-dimensional table of a database as claimed in claim 1, characterized in that: The two-dimensional table record adding operation includes: Determine the two-dimensional table record to be added in the instruction content of the two-dimensional table record adding operation, and record the two-dimensional table record to be added as record new_1 , record new_1 The primary key is denoted as key new_1 ; Key new_1 Execute the primary key single-point query to determine the location entry of the primary key single-point query, which is called the first target entry; determine each node in the path from the root node to the first target entry, and mark it as a path node; mark the result of concatenating the common prefixes of all the path nodes as prefix; and mark the key new_1 The result after trimming off the prefix is ​​marked as suffix new ; If the first target entry is a data entry, then the suffix originally stored in the first target entry is obtained, recorded as suffix old ; if suffix old with suffix new If they are consistent, then according to record new_1 Constructing a new value tuple, and updating other column value tuples originally stored in the first target entry with the new value tuple; If the first target entry is a node entry, the child node pointed to by the first target entry is marked as the first invalid node; all remaining primary keys and suffixes covered in the subtree with the first invalid node as the root are marked as the first invalid node. new as input, calling a new node construction operation to construct a first new node, wherein each of the remaining primary keys is formed by concatenating the common prefix of all nodes in the path from the first failed node to a data entry and the suffix of the data entry; replacing the child node pointer stored in the first target entry with the memory address pointer of the first new node, and deleting the first failed node; Determine whether each of the path nodes meets a preset reconstruction condition, where the reconstruction condition is that the total number of nodes in the subtree where the path node is located is not less than a preset node number threshold, the number of entries in the entry list of the path node is not less than a preset multiple, and the proportion of data entries in all entries covered by the subtree where the path node is located is not less than a preset proportion; When there is any node in the path node that meets the reconstruction condition, the path node that meets the reconstruction condition is marked as a reconstructed node; a new node construction operation is called with the set of all remaining primary keys covered in the subtree with the reconstructed node as input to construct a second new node, each of the remaining primary keys is composed of the common prefix of all nodes in the path from the reconstructed node to a data entry and the suffix of the data entry; the child node pointer stored in the node entry pointing to the reconstructed node is replaced with the memory address pointer of the second new node, and the reconstructed node is deleted.

7. The queryable compression method for a two-dimensional table of a database as claimed in claim 1, characterized in that: The two-dimensional table record deletion operation includes: Determine the primary key of the two-dimensional table record to be deleted in the instruction content of the two-dimensional table record deletion operation, and record the two-dimensional table record to be deleted as record old_1 , record old_1 The primary key is denoted as key old_1 ; Key old_1 Execute the primary key single-point query to determine the location entry of the primary key single-point query, which is called the second target entry; If the primary key single point query returns key old_1 If no information is found, the deletion operation is terminated; Otherwise, the node where the second target entry is located is marked as a second invalid node, and the set of all remaining primary keys that are not related to the second target entry and are covered in the subtree with the second invalid node as the root is taken as input, and a new node construction operation is called to construct a second new node, each of the remaining primary keys being composed of the common prefix of all nodes in the path from the second invalid node to a data entry different from the second target entry and the suffix of the data entry; the child node pointer stored in the node entry pointing to the second invalid node is replaced with the memory address pointer of the second new node, and the second invalid node is deleted.

8. The queryable compression method for a two-dimensional table of a database as claimed in claim 1, characterized in that: The two-dimensional table records the modification operation, including: Determine the original primary key and the modified two-dimensional table record in the instruction content of the two-dimensional table record modification operation, and record the original primary key as key old_2 , key old_2 The corresponding two-dimensional table record is recorded as record old_2 , the modified two-dimensional table record is recorded as record new_2 , record new_2 The primary key is denoted as key new_2 ; Determine key old_2 With key new_2 Are they the same? If key old_2 With key new_2 If they are not the same, first use record new_2 As the instruction content, the two-dimensional table record adding operation is performed, and then the record old_2 Execute the two-dimensional table record deletion operation as the instruction content; If key old_2 With key new_2 If they are consistent, then the key new_2 Execute the primary key single-point query to determine the location entry of the primary key single-point query, which is called the third target entry; if the primary key single-point query returns key new_2 If the information is not found, record new_2 As the instruction content, execute the two-dimensional table record adding operation, otherwise according to the record new_2 A new value tuple is constructed, and other column value tuples stored in the third target entry are changed to the new value tuple.

9. The queryable compression method for a two-dimensional table of a database according to any one of claims 6 and 8, characterized in that: According to the record new Constructs a new numeric tuple, consisting of: According to record new Determine other column fields except the primary key, and perform the following processing on the byte array type fields in the other column fields: Get record new The field value of the byte array type field in the byte array type field, and the value array corresponding to the byte array type field; if the field value is not in the value array, the field value is marked as a new field value, the new field value is added to the value array, and the mapping relationship between the new field value and the number is added in the field value learning mapping tree diagram; the process of adding the new field value in the field value learning mapping tree diagram is basically the same as the process of adding a new primary key in the two-dimensional table record adding operation, and the only difference is that the generated data entry stores the number of the new field value; Finally, the new value tuple is represented by record new It is composed of the field value number of the byte array type field and the original field value of the numeric type field in other column fields.

10. The queryable compression method for a two-dimensional table of a database according to any one of claims 2, 6 and 7, characterized in that: The new node construction operation includes: The new node is denoted as node, and the input set required to construct node is obtained. The input set is a set of all input field values ​​required to construct the node, denoted as {v1[1:m1], ..., v n [1:m n ]}; Set the common prefix of node to the longest common prefix v1[1:p] of the input set; The input set is pruned of the common prefix of node, and is recorded as {v1[p+1:m1], ..., v n [p+1:m n ]}; use {v1[p+1:m1], …, v n [p+1:m n ]}Substitute into the conversion function F to determine the number of entries of node and construct a discretization model of node, the method for determining the number of entries of node and the discretization model is consistent with the method for determining the number of entries of the field value learning mapping tree diagram and the discretization model; Determine an item list of a node according to the input set and the discretization model of the node, wherein the item type of the items in the item list is determined in the same manner as the item type of the items in the root node; When an entry of node is determined to be a data entry, the corresponding input field value v i The byte array obtained after trimming off the common prefix of node is v i [p+1:m i ] determines the suffix to be stored for the entry; When a certain entry of the node is determined to be a node entry, assume that the input field value of the node entry located through the discretization model of the node is v i [1:m i , …, v j [1:m j , where i < j, then use {v i [p+1:m i , …, v j [p+1:m j} as the input to recursively call the new node construction operation, and determine the memory address pointer of the newly constructed node as the child node pointer stored in this entry.

Images