Column data compression

Abstract

A column data compression method and apparatus and a storage medium. The method comprises: separately encoding column data according to a plurality of column-level encoding modes, so as to obtain at least one data flow corresponding to each encoding mode; among the plurality of encoding modes, determining a target encoding mode occupying the minimum storage space; determining at a flow level a target compression algorithm having the highest compression rate; and using the target compression algorithm to compress the at least one target data flow corresponding to the target encoding mode, and storing the compressed target data flow into a database. The method can perform encoding compression of column data on at least two levels, i.e. the column level and the flow level, thereby improving the compression rate of the column data and thus saving more storage space.

Description

Column data compression Technical Field

[0001] One or more embodiments of this specification relate to the field of data processing, and more particularly, to a column data compression method and apparatus, and a storage medium. Background Art

[0002] Currently, databases can use column storage for data storage. This means that data in the same column of a database table can be stored continuously together. This has a higher data compression rate than row-based storage and also offers higher read efficiency in column-based access scenarios.

[0003] To save storage space, column data can be compressed, but the compression efficiency may be low.

[0004] Summary of the Invention

[0005] One or more embodiments of this specification provide a column data compression method, apparatus, and storage medium.

[0006] To achieve the above objectives, one or more embodiments of this specification provide the following technical solutions.

[0007] According to a first aspect of one or more embodiments of the present specification, a column data compression method is proposed, comprising: encoding the column data according to a plurality of encoding methods at the column level to obtain at least one data stream corresponding to each encoding method; determining, among the plurality of encoding methods, a target encoding method that occupies the smallest storage space; determining a target compression algorithm with the highest compression rate at the stream level; using the target compression algorithm, compressing at least one target data stream corresponding to the target encoding method, and storing the compressed target data stream in a database.

[0008] According to the second aspect of one or more embodiments of the present specification, a column data compression device is proposed, including: an encoding module, used to encode the column data according to multiple encoding methods at the column level, to obtain at least one data stream corresponding to each encoding method; a first determination module, used to determine the target encoding method that occupies the smallest storage space among the multiple encoding methods; a second determination module, used to determine the target compression algorithm with the highest compression rate at the stream level; a compression module, used to use the target compression algorithm to compress at least one target data stream corresponding to the target encoding method, and store the compressed target data stream in a database.

[0009] According to a third aspect of one or more embodiments of this specification, an electronic device is proposed, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor implements the column data compression method as described in any one of the above items by running the executable instructions.

[0010] According to a fourth aspect of one or more embodiments of this specification, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, the steps of the column data compression method as described in any one of the above items are implemented.

[0011] The technical solutions provided by the embodiments of this specification may include the following beneficial effects.

[0012] In the present disclosure, the column data in the database table can be first converted into data streams according to multiple encoding methods at the column level, and then based on the target compression algorithm with the highest compression rate, at least one target data stream corresponding to the target encoding method that occupies the smallest storage space can be compressed. The present disclosure can encode and compress the column data from at least two levels, namely the column level and the stream level, to improve the compression rate of the column data and save more storage space. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] FIG1 is a flowchart of a column data compression method provided by an exemplary embodiment.

[0014] FIG2 is a block diagram of a column data compression apparatus provided by an exemplary embodiment.

[0015] FIG3 is a schematic structural diagram of an electronic device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0016] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0017] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0018] FIG1 is a flowchart of a column data compression method provided by an exemplary embodiment. Referring to FIG1 , the method may be executed by an electronic device, specifically, a server where a database is located, which is not limited in this disclosure. The method includes the following steps.

[0019] In step 101, the column data is encoded respectively according to multiple encoding modes at the column level to obtain at least one data stream corresponding to each encoding mode.

[0020] In the present disclosure, the column-level encoding method corresponding to the column data can be determined based on the characteristics of the column data, such as the storage type, so that the data belonging to the same column of the database table can be encoded according to different column-level encoding methods to obtain at least one data stream corresponding to the encoding method.

[0021] In an example, the storage type of the column data includes but is not limited to any one of the following: integer data; fixed-length character string; variable-length character string.

[0022] Integer data refers to data of fixed length and of integer type (e.g., uint type). Integer types can be further divided into uint8_t, uint16_t, uint32_t, and uint64_t based on the number of bits they occupy.

[0023] For example, uint8_t is an integer type that occupies 8 bits, uint16_t is an integer type that occupies 16 bits, and so on.

[0024] A fixed-length string refers to a string of constant length, for example, a string of 4 bytes.

[0025] A variable-length string is a string of variable length. For example, one string is 4 bytes long and the other is 2 bytes long.

[0026] In an example, the storage type of column data is integer data, and the column can be called an integer column (IntegerColumn).

[0027] Exemplarily, an integer column encoding method may be used to encode the integer column to obtain a first integer stream. The first integer stream may include each integer element in the current column of the database table.

[0028] Among them, the integer stream refers to a stream that continuously stores fixed-length integer data, that is, an integer array.

[0029] Optionally, based on the maximum value of the integer elements in the current column, at least one candidate integer type may be determined, where the number of bits occupied by the candidate integer types is greater than or equal to the number of bits occupied by the maximum value. The candidate integer type that occupies the least number of bits is determined as the integer type corresponding to the integer column encoding method.

[0030] For example, the integer type corresponding to the current column is uint16_t, but the maximum value of the integer element only occupies 7 bits. The candidate integer types are uint8_t, uint16_t, uint32_t, and uint64_t. Among these candidate integer types, uint8_t occupies the fewest bits. Therefore, the integer type corresponding to the encoding method of this integer column is uint8_t.

[0031] For example, the integer column may be encoded using a dictionary column encoding method to obtain a second integer stream and a first index stream. The second integer stream may include each integer element in the current column of the database table.

[0032] The second integer stream may include mutually different integer elements in the current column, that is, the second integer stream includes an array composed of unique values ​​of the current column.

[0033] For example, the current column includes the following integer elements in order from small to large row numbers: integer element #1; integer element #1; integer element #5, integer element #3, integer element #4, integer element #5,..., and the second integer stream can include integer element #1, integer element #5, integer element #3, and integer element #4.

[0034] It is understandable that the second integer stream may adopt the same integer column encoding method as the first integer stream, and the integer type corresponding to the integer column encoding method is the candidate integer type that occupies the least number of bits.

[0035] The first index stream may be a ref array, which is used to record the row number of each row of data in the second integer stream, that is, the first index stream may be used to query the second integer stream.

[0036] For example, the first index stream includes: {0, 1}, {2, 5}, {3}, {4}, ... .

[0037] In the present disclosure, when the number of unique values ​​included in the current column is less than or equal to a preset threshold, the current column data may be encoded using a dictionary column encoding method.

[0038] In one example, the storage type of the column data is a string, and the length of each string is constant, that is, a fixed-length string. This column can be called a string column (StringColumn).

[0039] Exemplarily, the string column may be encoded using a string column encoding method to obtain a first string stream, which may include each string in the current column of the database table.

[0040] Among them, a string stream refers to a continuously stored string array, which only includes the specific content of the string but does not include the string length information.

[0041] In the embodiment of the present disclosure, since the length of each character string remains unchanged, only the first character string stream can be obtained after encoding the character string sequence using the character string sequence encoding method.

[0042] Exemplarily, the string column may be encoded in a dictionary column encoding manner to obtain a second string stream and a second index stream.

[0043] The second character string stream may include mutually different character strings in the current column of the database table, that is, the second character string stream includes a unique value of the current column.

[0044] For example, the current column includes the following strings in ascending order of row numbers: string #1; string #3; string #2, string #2, string #4, string #5, ..., and the second string stream may include string #1, string #3, string #2, string #4, string #5, ...

[0045] It is understandable that the second string stream may adopt the same string column encoding method as the first string stream.

[0046] The second index stream may be a ref array, which is used to record the row number of each row of data in the second string stream, that is, the second index stream may be used to query the second string stream.

[0047] For example, the second index stream includes: {0}, {1}, {2, 3}, {4}, {5}...

[0048] In one example, the storage type of the column data is a string, and the length of each string is variable, that is, a variable-length string. This column can also be called a string column (StringColumn).

[0049] Exemplarily, the string column may be encoded using a string column encoding method to generate a third string stream and a third integer stream. The third string stream may include each string in the current column of the database table. The third integer stream is used to determine the length of each string in the third string stream.

[0050] Among them, a string stream refers to a continuously stored string array, which only includes the specific content of the string but does not include the string length information.

[0051] In the embodiment of the present disclosure, since the length of each character string is variable, the character string sequence is encoded using a character string sequence encoding method to obtain a third character string stream and a third integer stream.

[0052] Optionally, the third integer stream may store the offset of the end position of each line of character strings in the third character string stream, and the difference between two adjacent offsets is the length of the current line of character strings.

[0053] For example, the string column can be encoded using a dictionary column encoding method to obtain a fourth string stream, a fourth integer stream, and a third index stream. The fourth string stream includes unique strings in the current column, the fourth integer stream includes a value for determining the length of each string in the fourth string stream, and the third index stream is used to query the fourth string stream. The third index stream can also be a ref array.

[0054] Since the string length is variable, it is encoded using dictionary column encoding to obtain three streams: a fourth string stream for recording unique values ​​(different strings) in the current column, a fourth integer stream for recording the length of each string in the fourth string stream, and a third index stream for querying the fourth string stream.

[0055] The above description is merely an example, and the present disclosure does not limit the encoding method of column data based on the column level.

[0056] In step 102, a target encoding method that occupies the smallest storage space is determined among the multiple encoding methods.

[0057] In the present disclosure, the storage space occupied by the original stream data can be determined. For integer streams, the storage space occupied can be determined based on the size of the integer array, and for string streams, the storage space occupied can be determined based on the total length of the string. The storage space occupied by the data stream corresponding to the same encoding method can be obtained by summing the storage space occupied by at least one data stream corresponding to that encoding method.

[0058] For example, if the storage type corresponding to the column data is integer data, a first integer stream is obtained according to the integer column encoding method. A second integer stream and a first index stream are obtained according to the dictionary column encoding method. The storage space occupied by the first integer stream is a first number of bytes, and the storage space occupied by the second integer stream and the first index stream is a second number of bytes. If the first number of bytes is less than the second number of bytes, the target encoding method is the integer column encoding method. If the first number of bytes is greater than the second number of bytes, the target encoding method is the dictionary column encoding method.

[0059] For another example, the storage type corresponding to the column data is a string, and the length of each string remains unchanged. A first string stream is obtained according to the string column encoding method. A second string stream and a second index stream are obtained according to the dictionary column encoding method. The first string stream occupies a third number of bytes of storage space, and the second string stream and the second index stream occupy a fourth number of bytes of storage space. If the third number of bytes is less than the fourth number of bytes, the target encoding method is the string column encoding method; if the third number of bytes is greater than the fourth number of bytes, the target encoding method is the dictionary column encoding method.

[0060] For another example, the storage type corresponding to the column data is a string, and each string has a variable length. According to the string column encoding method, a third string stream and a third integer stream are obtained. According to the dictionary column encoding method, a fourth string stream, a fourth integer stream, and a third index stream are obtained. The storage space occupied by the third string stream and the third integer stream is a fifth number of bytes, and the storage space occupied by the fourth string stream, the fourth integer stream, and the third index stream is a sixth number of bytes. If the fifth number of bytes is less than the sixth number of bytes, the target encoding method is the string column encoding method; if the fifth number of bytes is greater than the sixth number of bytes, the target encoding method is the dictionary column encoding method.

[0061] In step 103, a target compression algorithm with the highest compression ratio in the stream level is determined.

[0062] In the present disclosure, the data stream obtained after column-level encoding may be an integer stream or a character string stream.

[0063] The electronic device may pre-set candidate compression algorithm sets corresponding to integer streams and character string streams respectively.

[0064] In one example, the set of candidate compression algorithms corresponding to the integer stream may include but is not limited to at least one of the following: inverted (Pfor) compression algorithm, run length encoding (RLE) compression algorithm, Huffman (ZigZag) compression algorithm, differential (Delta) compression algorithm, and double differential (Delta) compression algorithm.

[0065] In an example, the set of candidate compression algorithms corresponding to the string stream may include, but is not limited to, at least one of the following: a prefix compression algorithm and a general compression algorithm.

[0066] Among them, common compression algorithms include but are not limited to zip compression algorithm, LZW compression algorithm, LZ77 compression algorithm, etc.

[0067] In the present disclosure, the corresponding candidate compression algorithm can be selected according to the characteristics of the data stream, and according to each candidate compression algorithm, at least part of the data included in at least one of the target data streams is compressed to obtain compressed data, and then the candidate compression algorithm with the least number of bits occupied by the compressed data is determined as the target compression algorithm.

[0068] Exemplarily, all data included in the target data stream may be compressed according to different candidate compression algorithms, and then the candidate compression algorithm with the least number of bits occupied by the compressed data is determined as the target compression algorithm.

[0069] Considering that compressing all data included in a target data stream can take a significant amount of time, the present disclosure can sample data included in at least one target data stream at a preset sampling rate to obtain sampled data. Furthermore, the sampled data can be compressed separately according to each candidate compression algorithm to obtain compressed data. The candidate compression algorithm whose compressed data occupies the least number of bits is then determined as the target compression algorithm. This reduces the delay in determining the target compression algorithm.

[0070] The sampling rate may be any value within the interval (0, 1), that is, the sampling rate may be any value greater than 0 and less than 1. For example, the sampling rate may be 0.25 (25%).

[0071] In step 104, the target compression algorithm is used to compress at least one target data stream corresponding to the target encoding method, and the compressed target data stream is stored in a database.

[0072] In the present disclosure, the data stream corresponding to the target encoding mode is the target data stream, and the number of the target data stream can be one or more.

[0073] For example, the target encoding mode is an integer column encoding mode, the target data stream is the first integer stream, and the number thereof is 1.

[0074] For another example, the target encoding mode is a dictionary column encoding mode, and the target data stream includes a second integer stream and a first index stream, and the number of the second integer stream and the first index stream is 2.

[0075] For another example, the target encoding mode is a string column encoding mode, and the target data stream includes a first string stream, the number of which is 1.

[0076] For another example, the target encoding mode is a dictionary column encoding mode, and the target data stream includes a second character string stream and a second index stream, and the number of the second index streams is 2.

[0077] For another example, the target encoding mode is a string column encoding mode, and the target data stream includes a third string stream and a third integer stream, and the number of the third string stream and the third integer stream is 2.

[0078] For another example, the target encoding mode is the dictionary column encoding mode, and the target data stream includes a fourth character string stream, a fourth integer stream, and a third index stream, and the number of the streams is 3.

[0079] In the present disclosure, after determining the target compression algorithm, the target compression algorithm can be used to compress all data of the target data stream, and the compressed target data stream can be stored in a database.

[0080] Exemplarily, it may be stored in a storage medium of a database.

[0081] Exemplarily, it may also be stored in other hardware, software or associated storage media of the database, which is not limited in this disclosure.

[0082] When querying a database table, the database storage medium can be used to search for the compressed target data stream corresponding to the desired field (i.e., the field to which the column data belongs). Furthermore, the target compression algorithm can be used to decompress the data stream, resulting in the target data stream. The target data stream can then be searched for the desired integer element or string, and the search result is returned.

[0083] In the above embodiment, a layered encoding method is adopted to fully consider the characteristics of the data from at least two levels (column level and stream level), effectively encode and compress the column data, improve the compression rate, save more storage space, and have high availability.

[0084] In some optional embodiments, in order to better process Online Analytical Processing (OLAP) business in the database, column storage is generally used to store data in the same column of the database table continuously. Since the data in the same column have the same data type, they can be better encoded and compressed.

[0085] In order to save storage space, it is necessary to ensure high data encoding and decoding performance while performing column storage. Therefore, the present disclosure proposes the above-mentioned column data compression method.

[0086] First, the column data is encoded using different column-level encoding methods to generate at least one data stream corresponding to each encoding method. Based on the characteristics of each data stream, the candidate compression algorithm with the highest compression ratio is selected for compression. The process of encoding column data to generate a data stream is called column-level encoding, and the process of compressing a data stream is called stream-level encoding.

[0087] Compared to directly using a general algorithm to compress column data, the present disclosure repeatedly considers data features and encodes and compresses data at at least two levels, thereby achieving a higher compression rate.

[0088] In addition, the present disclosure can accurately calculate the data length obtained after using each candidate compression algorithm, and can select the target compression algorithm with the highest compression rate to compress the stream data, which also improves the compression rate and can reduce the size of the storage space occupied, with high usability.

[0089] Referring to FIG2 , a column data compression device can be applied to an electronic device, such as a server where a database is located, to implement the technical solution of this specification. The column data compression device may include: an encoding module 201 for encoding the column data according to multiple encoding methods at the column level to obtain at least one data stream corresponding to each encoding method; a first determination module 202 for determining, among the multiple encoding methods, a target encoding method that occupies the least storage space; a second determination module 203 for determining a target compression algorithm with the highest compression ratio at the stream level; and a compression module 204 for compressing at least one target data stream corresponding to the target encoding method using the target compression algorithm, and storing the compressed target data stream in the database.

[0090] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or physical devices, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0091] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0092] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0093] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0094] FIG3 is a schematic structural diagram of a server provided by an exemplary embodiment. Referring to FIG3 , at the hardware level, the device includes a processor 302, an internal bus 304, a network interface 306, a memory 308, and a non-volatile memory 310, and may also include hardware required for other services. One or more embodiments of this specification may be implemented based on software, such as the processor 302 reading the corresponding computer program from the non-volatile memory 310 into the memory 308 and then running it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but may also be hardware or logic devices.

[0095] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0096] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0097] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "an," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0098] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining."

[0099] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be included in the scope of protection of one or more embodiments of this specification.

Claims

1. A column data compression method, characterized in that: include: Encoding the column data respectively according to multiple encoding modes at the column level to obtain at least one data stream corresponding to each encoding mode; Determine a target encoding method that occupies the smallest storage space among the multiple encoding methods; Determine the target compression algorithm with the highest compression ratio in the stream level; The target compression algorithm is used to compress at least one target data stream corresponding to the target encoding method, and the compressed target data stream is stored in a database.

2. The method according to claim 1, characterized in that The storage type of the column data is integer data, and the column data is encoded respectively according to multiple encoding methods at the column level to obtain at least one data stream corresponding to each encoding method, including: Encode the column data by using an integer column encoding method to obtain a first integer stream, wherein the first integer stream includes each integer element in the current column; The column data is encoded using a dictionary column encoding method to obtain a second integer stream and a first index stream, wherein the second integer stream includes mutually different integer elements in the current column, and the first index stream is used to query the second integer stream.

3. The method according to claim 2, characterized in that The method further comprises: Determine at least one type of candidate integer type based on the maximum value of the integer elements in the current column, wherein the storage space occupied by the candidate integer type is greater than or equal to the storage space occupied by the maximum value; The candidate integer type that occupies the smallest storage space is determined as the integer type corresponding to the integer column encoding method.

4. The method according to claim 1, characterized in that: The storage type of the column data is a string, and the length of each string remains unchanged. The column data is encoded according to multiple encoding methods at the column level to obtain at least one data stream corresponding to each encoding method, including: Encoding the column data by using a string column encoding method to obtain a first string stream, wherein the first string stream includes each string in the current column; The column data is encoded by using a dictionary column encoding method to obtain a second string stream and a second index stream, wherein the second string stream includes mutually different strings in the current column, and the second index stream is used to query the second string stream.

5. The method according to claim 1, characterized in that The storage type of the column data is a string, and the length of each string is variable. The column data is encoded according to multiple encoding methods at the column level to obtain at least one data stream corresponding to each encoding method, including: Encode the column data by adopting a string column encoding method to obtain a third string stream and a third integer stream, wherein the third string stream includes each string in the current column, and the third integer stream is used to determine the length of each string in the third string stream; The column data is encoded by adopting a dictionary column encoding method to obtain a fourth string stream, a fourth integer stream and a third index stream, wherein the fourth string stream includes mutually different strings in the current column, the fourth integer stream includes a length for determining each string in the fourth string stream, and the third index stream is used to query the fourth string stream.

6. The method according to any one of claims 1 to 5, characterized in that: The determining of the target compression algorithm with the highest compression ratio in the stream level includes: According to each candidate compression algorithm, compress at least part of the data included in at least one of the target data streams to obtain compressed data; The candidate compression algorithm that occupies the smallest storage space for the compressed data is determined as the target compression algorithm.

7. The method according to claim 6, characterized in that The method further comprises: Sampling data included in at least one of the target data streams according to a preset sampling rate to obtain sampled data; The step of compressing at least part of the data included in at least one of the target data streams according to each of the candidate compression algorithms to obtain compressed data includes: According to each of the candidate compression algorithms, the sampled data are compressed respectively to obtain compressed data.

8. A column data compression device, characterized in that: include: An encoding module, used to encode the column data according to multiple encoding methods at the column level, to obtain at least one data stream corresponding to each encoding method; A first determining module, configured to determine, among the plurality of encoding modes, a target encoding mode that occupies the smallest storage space; A second determination module is used to determine a target compression algorithm with the highest compression rate in the stream level; The compression module is used to compress at least one target data stream corresponding to the target encoding method using the target compression algorithm, and store the compressed target data stream in a database.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor implements the column data compression method according to any one of claims 1 to 7 by running the executable instructions.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the column data compression method according to any one of claims 1 to 7 are implemented.

Images