Data compression method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a data compression method and device, computer equipment, a storage medium and a program product. The method comprises the steps of obtaining to-be-compressed data; dividing the to-be-compressed data into at least one data block according to data characteristics of the to-be-compressed data; for each data block, according to the data characteristics of the data block, compressing the data block by adopting a compression algorithm matched with the data block to obtain sub-compressed data corresponding to the data block; and splicing the sub-compressed data to obtain target compressed data of the to-be-compressed data. According to the scheme, the to-be-compressed data is divided into different data blocks according to the data characteristics of the to-be-compressed data, and the different data blocks are compressed by adopting the corresponding compression algorithms, so that a proper compression algorithm can be selected according to the data characteristics, and the data compression efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data compression, and in particular to a data compression method and device, computer equipment, a storage medium and a program product. BACKGROUND

[0002] One of the purposes of a data compression algorithm is to reduce the storage size and transmission time of data.

[0003] In the prior art, a single compression method is usually used, which is not suitable for various characteristic data and has the problem of low data compression efficiency. SUMMARY

[0004] Therefore, it is necessary to provide a data compression method, device, computer equipment, storage medium and program product to improve the compression efficiency of data.

[0005] In a first aspect, the present application provides a data compression method, comprising:

[0006] obtaining to-be-compressed data;

[0007] dividing the to-be-compressed data into at least one data block according to data characteristics of the to-be-compressed data, wherein the data characteristics include at least one of an entropy level, a repetition rate and context correlation;

[0008] for each data block, compressing the data block by using a compression algorithm matched with the data block according to data characteristics of the data block, to obtain sub-compression data corresponding to the data block;

[0009] splicing the sub-compression data to obtain target compression data of the to-be-compressed data.

[0010] In one embodiment, the step of compressing the data block by using a compression algorithm matched with the data block according to data characteristics of the data block, to obtain sub-compression data corresponding to the data block, comprises:

[0011] determining a basic compression algorithm matched with the data block according to the data characteristics of the data block;

[0012] performing basic compression on the data block by using the basic compression algorithm to obtain basic compression encoding;

[0013] determining a deep compression algorithm corresponding to the basic compression encoding;

[0014] performing deep compression on the basic compression encoding by using the deep compression algorithm to obtain the sub-compression data corresponding to the data block.

[0015] In one of the embodiments, the determining of the base compression algorithm matched with the data block according to the data characteristics of the data block comprises:

[0016] If the repetition rate of the data block is higher than a first preset threshold and the entropy level is lower than a second preset threshold, the base compression algorithm is determined as a sliding window algorithm.

[0017] If the entropy level of the data block is higher than a third preset threshold and the context correlation is higher than a fourth preset threshold, the base compression algorithm is determined as a context-aware modeling algorithm.

[0018] If the data block meets a preset condition, the base compression algorithm is determined as a combination algorithm of the sliding window algorithm and the adaptive Huffman coding, wherein the preset condition is other conditions except that the repetition rate of the data block is higher than the first preset threshold and the entropy level is lower than the second preset threshold, and that the entropy level of the data block is higher than the third preset threshold and the context correlation is higher than the fourth preset threshold.

[0019] In one of the embodiments, the determining of the deep compression algorithm corresponding to the base compression coding comprises:

[0020] determining an intermediate compression algorithm according to the data characteristics of the data block;

[0021] compressing the base compression coding by using the intermediate compression algorithm to obtain an intermediate compression coding;

[0022] determining a deep compression algorithm corresponding to the intermediate compression coding according to the data characteristics of the intermediate compression coding.

[0023] In one of the embodiments, the determining of the intermediate compression algorithm according to the data characteristics of the data block comprises:

[0024] If the repetition rate in the data characteristics is greater than a fifth preset threshold and the entropy level is less than or equal to a sixth preset threshold, the intermediate compression algorithm is determined as a dictionary construction algorithm.

[0025] If the repetition rate in the data characteristics is less than or equal to the fifth preset threshold and the entropy level is greater than the sixth preset threshold, the intermediate compression algorithm is determined as a bit plane coding algorithm.

[0026] If the repetition rate in the data characteristics is greater than the fifth preset threshold and the entropy level is less than or equal to the sixth preset threshold, or the repetition rate in the data characteristics is less than or equal to the fifth preset threshold and the entropy level is less than or equal to the sixth preset threshold, the intermediate compression algorithm is determined as a combination algorithm of the dictionary construction algorithm and the bit plane coding algorithm.

[0027] In one of the embodiments, if the intermediate compression algorithm is a dictionary construction algorithm, the compression of the basic compression encoding by using the intermediate compression algorithm to obtain an intermediate compression encoding comprises:

[0028] determining a dictionary index encoding corresponding to the basic compression encoding;

[0029] determining an intermediate compression encoding corresponding to the basic compression encoding according to the dictionary index encoding.

[0030] In one of the embodiments, if the intermediate compression algorithm is a bit plane encoding algorithm, the compression of the basic compression encoding by using the intermediate compression algorithm to obtain an intermediate compression encoding comprises:

[0031] determining a bit plane encoding strategy according to the bit level distribution feature of the basic compression encoding and the context information of the data block; wherein the bit plane encoding strategy comprises a multi-layer encoding strategy and a few-layer encoding strategy;

[0032] compressing the basic compression encoding according to the bit plane encoding strategy to obtain an intermediate compression encoding.

[0033] In one of the embodiments, the determination of the bit plane encoding strategy according to the bit level distribution feature of the basic compression encoding and the context information of the data block comprises:

[0034] if the bit level distribution feature of the basic compression encoding represents that the bit level data is continuous, and the context information of the data block represents that the data pattern in the data block is single, determining the bit plane encoding strategy as a few-layer encoding strategy;

[0035] if the bit level distribution feature of the basic compression encoding represents that the bit level data is disordered, and the context information of the data block represents that there is a dependent relationship among the data in the data block, determining the bit plane encoding strategy as a multi-layer encoding strategy.

[0036] In one of the embodiments, if the intermediate compression algorithm is a combination algorithm of the dictionary construction algorithm and the bit plane encoding algorithm, the compression of the basic compression encoding by using the intermediate compression algorithm to obtain an intermediate compression encoding comprises:

[0037] compressing the basic compression encoding according to the dictionary construction algorithm to obtain a dictionary compression encoding;

[0038] compressing the dictionary compression encoding according to the bit plane encoding algorithm to obtain an intermediate compression encoding.

[0039] In one of the embodiments, the determination of the deep compression algorithm corresponding to the intermediate compression encoding according to the data feature of the intermediate compression encoding comprises:

[0040] According to the data characteristics of the intermediate compression coded data, the intermediate compression coding is divided into at least one data segment;

[0041] The deep compression algorithm corresponding to each data segment is determined.

[0042] In one embodiment, the deep compression algorithm corresponding to each data segment is determined by:

[0043] If the repetition rate in the data characteristics of the data segment is greater than a seventh preset threshold, the deep compression algorithm is determined to be Huffman coding;

[0044] If the entropy level in the data characteristics of the data segment is greater than an eighth preset threshold, the deep compression algorithm is determined to be arithmetic coding;

[0045] If the repetition rate in the data characteristics of the data segment is less than or equal to the seventh preset threshold, or the entropy level in the data characteristics of the data segment is less than or equal to the eighth preset threshold, the deep compression algorithm is determined to be the Huffman coding and the arithmetic coding.

[0046] In a second aspect, the present application further provides a data compression device, comprising:

[0047] An acquisition module is configured to acquire to-be-compressed data;

[0048] A division module is configured to divide the to-be-compressed data into at least one data block according to data characteristics of the to-be-compressed data, wherein the data characteristics include at least one of an entropy level, a repetition rate, and context correlation;

[0049] A compression module is configured to, for each data block, compress the data block by using a compression algorithm matched with the data block according to data characteristics of the data block, to obtain sub-compression data corresponding to the data block;

[0050] A splicing module is configured to splice the sub-compression data to obtain target compression data of the to-be-compressed data.

[0051] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0052] Acquire to-be-compressed data;

[0053] Divide the to-be-compressed data into at least one data block according to data characteristics of the to-be-compressed data, wherein the data characteristics include at least one of an entropy level, a repetition rate, and context correlation;

[0054] According to the data characteristics of each data block, a compression algorithm matched with the data block is adopted to compress the data block, so as to obtain sub-compression data corresponding to the data block;

[0055] The sub-compression data are spliced to obtain target compression data of the to-be-compressed data.

[0056] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0057] Obtaining to-be-compressed data;

[0058] According to the data characteristics of the to-be-compressed data, the to-be-compressed data is divided into at least one data block; wherein the data characteristics include at least one of entropy level, repetition rate and context correlation;

[0059] According to the data characteristics of each data block, a compression algorithm matched with the data block is adopted to compress the data block, so as to obtain sub-compression data corresponding to the data block;

[0060] The sub-compression data are spliced to obtain target compression data of the to-be-compressed data.

[0061] In a fifth aspect, the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the following steps:

[0062] Obtaining to-be-compressed data;

[0063] According to the data characteristics of the to-be-compressed data, the to-be-compressed data is divided into at least one data block; wherein the data characteristics include at least one of entropy level, repetition rate and context correlation;

[0064] According to the data characteristics of each data block, a compression algorithm matched with the data block is adopted to compress the data block, so as to obtain sub-compression data corresponding to the data block;

[0065] The sub-compression data are spliced to obtain target compression data of the to-be-compressed data.

[0066] The data compression method, device, computer device, storage medium and program product obtain to-be-compressed data; according to the data characteristics of the to-be-compressed data, the to-be-compressed data is divided into at least one data block; for each data block, according to the data characteristics of the data block, a compression algorithm matched with the data block is used to compress the data block, to obtain sub-compressed data corresponding to the data block; and the sub-compressed data are spliced to obtain target compressed data of the to-be-compressed data. According to the data characteristics of the to-be-compressed data, the to-be-compressed data is divided into different data blocks, and different compression algorithms are used to compress different data blocks, so that a suitable compression algorithm can be selected according to the data characteristics, thereby improving the data compression efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other related drawings can be obtained by those skilled in the art without creative labor.

[0068] Figure 1 A flowchart of a data compression method in an embodiment;

[0069] Figure 2 A flowchart of obtaining sub-compressed data corresponding to a data block in an embodiment;

[0070] Figure 3 A flowchart of determining a deep compression algorithm corresponding to a basic compression encoding in an embodiment;

[0071] Figure 4 A flowchart of determining a deep compression algorithm corresponding to an intermediate compression encoding in an embodiment;

[0072] Figure 5 A block diagram of a data compression device in an embodiment;

[0073] Figure 6 An internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0075] The data compression method provided by the embodiments of the present application can be applied to an application scenario of compressing data. The method can be executed by a server or a terminal.

[0076] The server can be a physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, a projection device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc.

[0077] In an exemplary embodiment, as shown in Figure 1 A data compression method is provided. Taking the server as an example, the method comprises the following steps:

[0078] S101, obtaining to-be-compressed data.

[0079] The to-be-compressed data refers to original data that needs to be compressed by a compression algorithm to reduce storage size or transmission time consumption. The types of the to-be-compressed data can include text data, image data, audio data, mixed type data, etc.

[0080] For example, data uploaded through a network, original data stored in a local storage system, or data obtained from other servers through synchronization can be received. The data can be preliminarily checked during the obtaining process to ensure data integrity and avoid compression failure caused by data damage.

[0081] S102, dividing the to-be-compressed data into at least one data block according to a data characteristic of the to-be-compressed data.

[0082] The data characteristic refers to a parameter that can reflect the intrinsic characteristics of the data, including at least one of an entropy level, a repetition rate, and a context correlation. The entropy level is used to represent the degree of disorder of the data. The higher the entropy value, the more disordered the data and the less redundant the data. The repetition rate is used to represent the proportion of repeated patterns (such as repeated characters or repeated byte sequences) in the data. The higher the repetition rate, the more redundant the data. The context correlation is used to represent the dependency relationship between adjacent data, such as the correlation between adjacent characters in text or the correlation between adjacent pixels in an image.

[0083] For example, the data to be compressed is "AAABBBCCCCDDDEEEFFFGGG", and the data to be compressed can be divided into data block 1 "AAA", data block 2 "BBB", data block 3 "CCCC", data block 4 "DDD", data block 5 "EEE", data block 6 "FFF", and data block 7 "GGG".

[0084] S103, for each data block, according to the data characteristics of the data block, using a compression algorithm matched with the data block to compress the data block, to obtain sub-compressed data corresponding to the data block.

[0085] For example, the correspondence between the data characteristics and the compression algorithm can be set in advance, and then for each data block, the compression algorithm corresponding to the data block can be selected according to the data characteristics of the data block and the correspondence, and the compression algorithm matched with the data block is used to compress the data block, to obtain sub-compressed data corresponding to the data block.

[0086] For example, the compression algorithm matched with the data block can also be determined according to the prediction model, for example, the data block is input into the model for data characteristic identification, and then the compression algorithm matched with the data characteristics of the data block is output.

[0087] S104, splicing each sub-compressed data to obtain target compressed data of the data to be compressed.

[0088] For example, all sub-compressed data can be spliced in the order of each data block in the data to be compressed, and a block separator (such as a specific byte sequence "0xFFFF") and compression header information (such as the number of data blocks, the length of each data block, the type identification of the compression algorithm, etc.) can be added in the splicing process to form the target compressed data. The compression header information is used to quickly identify the boundaries of each data block and the corresponding decompression algorithm during decompression, to ensure that the order of the decompressed data is consistent with the original data.

[0089] The above data compression method obtains data to be compressed; divides the data to be compressed into at least one data block according to the data characteristics of the data to be compressed; for each data block, according to the data characteristics of the data block, using a compression algorithm matched with the data block to compress the data block, to obtain sub-compressed data corresponding to the data block; and splicing each sub-compressed data to obtain target compressed data of the data to be compressed. According to the data characteristics of the data to be compressed, the data to be compressed is divided into different data blocks, and different data blocks are compressed using corresponding compression algorithms, which can select a suitable compression algorithm according to the data characteristics, thereby providing data compression efficiency.

[0090] In some optional implementations, referring toFigure 2 , Figure 2 A flowchart for obtaining sub-compressed data corresponding to a data block is provided, and specifically includes the following steps:

[0091] S201, determine a basic compression algorithm matched with the data block according to data characteristics of the data block.

[0092] For example, the basic compression algorithm matched with the data block can be determined according to the repetition rate, entropy level and other characteristics of the data block. For example, a sliding window algorithm is used for a data block with high repetition rate, and a context-aware modeling algorithm is used for a data block with high entropy value and strong correlation.

[0093] In some optional implementations, the basic compression algorithm matched with the data block can be determined according to the data characteristics of the data block in the following manner.

[0094] For example, if the repetition rate of the data block is higher than a first preset threshold, and the entropy level is lower than a second preset threshold, the basic compression algorithm is determined to be a sliding window algorithm. The first preset threshold can be set according to the actual application scenario, and is used to distinguish between data blocks with high repetition rate and low repetition rate, for example, set to 80%. The second preset threshold can be set according to the actual application scenario, and is used to distinguish between data blocks with high entropy and low entropy, for example, set to 2.0. The sliding window algorithm can be a lossless data compression algorithm (Lempel-Ziv 77, LZ77).

[0095] For example, if the entropy level of the data block is higher than a third preset threshold, and the context correlation is higher than a fourth preset threshold, the basic compression algorithm is determined to be a context-aware modeling algorithm. The third preset threshold can be set according to the actual application scenario, and is used to distinguish between data blocks with high entropy and low entropy, for example, set to 3.5. The fourth preset threshold can be set according to the actual application scenario, and is used to distinguish between data blocks with strong correlation and weak correlation, for example, set to 85%.

[0096] For example, if the data block meets a preset condition, the basic compression algorithm is determined to be a combination algorithm of the sliding window algorithm and the adaptive Huffman coding. The preset condition is other than the case that the repetition rate of the data block is higher than the first preset threshold and the entropy level is lower than the second preset threshold, and the case that the entropy level of the data block is higher than the third preset threshold and the context correlation is higher than the fourth preset threshold. That is, the combination algorithm of the sliding window algorithm and the adaptive Huffman coding is used in the case other than the above two cases.

[0097] The sliding window algorithm is a dictionary-based compression algorithm, which stores the data sequence that has appeared in the window as a dictionary by maintaining a sliding window, and indicates the repeated sequence that appears subsequently by a pointer of "offset + length", and is suitable for high repetition rate data.

[0098] The context-aware modeling algorithm is a compression algorithm based on probability prediction. A probability model is constructed by analyzing the context information (such as the previous N data elements) of the data to predict the probability of occurrence of the next data element. High-probability data is assigned a short code. The algorithm is suitable for high-entropy, strongly correlated data.

[0099] The adaptive Huffman coding is an entropy coding algorithm that dynamically adjusts the coding table. The coding length can be adjusted according to the real-time frequency of data elements without the need for prior frequency distribution statistics.

[0100] In S202, a basic compression algorithm is used to perform basic compression on the data block to obtain a basic compression code.

[0101] For example, a basic compression algorithm can be used to perform basic compression on the data block to obtain a basic compression code to remove significant redundant data in the data block.

[0102] In S203, a deep compression algorithm corresponding to the basic compression code is determined.

[0103] For example, the deep compression algorithm corresponding to the basic compression code can be determined according to the coding characteristics of the basic compression code.

[0104] In S204, a deep compression algorithm is used to perform deep compression on the basic compression code to obtain sub-compressed data corresponding to the data block.

[0105] For example, the data characteristics (such as the coding length and the remaining redundancy distribution) of the basic compression code can be analyzed to select an appropriate deep compression algorithm (such as a combination of a dictionary construction algorithm, a bit plane coding algorithm, and an entropy coding algorithm) to perform secondary compression on the basic compression code, further reducing the data volume, and obtaining sub-compressed data corresponding to the data block.

[0106] For example, assuming the data block is "ABCABCABC" (repetition rate 90%, entropy level 1.5, context relevance 75%), in the basic compression stage, since the repetition rate is higher than the first preset threshold (80%) and the entropy level is lower than the second preset threshold (2.0), a sliding window algorithm (window size set to 6) can be selected. When the sliding window algorithm scans the data, it finds that "ABC" appears repeatedly, and replaces the subsequent repeated "ABC" with a pointer identifier of "offset 3, length 3", resulting in the basic compressed code "ABC[3,3][3,3]". In the deep compression stage, analyzing the characteristics of the basic compressed code (repeated pointer identifiers still exist), a combination of dictionary construction algorithm and Huffman coding can be selected as the deep compression algorithm. Specifically, "ABC" can be mapped to index "001" and "[3,3]" can be mapped to index "010" using the dictionary construction algorithm to obtain the intermediate code "001 010 010"; then, the index is compressed using Huffman coding to finally obtain the sub-compressed data "101 110 110".

[0107] In the above embodiments, the two-stage compression mode of "basic compression + deep compression" first quickly removes the main redundancy and then accurately optimizes the remaining encoding. Compared with single-stage compression, the compression ratio is improved, while avoiding the problem of high complexity of a single deep compression algorithm, thus balancing compression efficiency and computational overhead.

[0108] In some alternative implementations, see [link to relevant documentation]. Figure 3 , Figure 3 A flowchart illustrating the determination of a deep compression algorithm corresponding to a basic compression code is provided, specifically including the following steps:

[0109] S301, determine the intermediate compression algorithm based on the data characteristics of the data block.

[0110] Intermediate compression algorithms are used to handle specific types of redundancy in basic compression coding, such as pattern redundancy and bit-level redundancy, including dictionary construction algorithms, bit-plane coding algorithms and their combination algorithms.

[0111] For example, the correspondence between data characteristics and intermediate compression algorithms can be pre-set, so that for each data block, the intermediate compression algorithm corresponding to that data block can be selected according to the data characteristics and the correspondence.

[0112] For example, an intermediate compression algorithm matching the data block can also be determined based on the prediction model. For instance, the data block can be input into the model to identify data characteristics, and then an intermediate compression algorithm matching the data characteristics of the data block can be output.

[0113] In some alternative implementations, the intermediate compression algorithm that matches the data block can be determined in the following ways.

[0114] If the repetition rate in the data characteristics is greater than a fifth preset threshold, and the entropy level is less than or equal to a sixth preset threshold, it is determined that the intermediate compression algorithm is a dictionary construction algorithm. The fifth preset threshold can be set according to the actual application scenario, and is also used to distinguish high-repetition-rate and low-repetition-rate data blocks, for example, it can be set to 70%; the first preset threshold and the fifth preset threshold can be set to the same value, or can be set to different values. The sixth preset threshold can be set according to the actual application scenario, and is also used to distinguish high-entropy and low-entropy data blocks, for example, it can be set to 2.5; the second preset threshold and the sixth preset threshold can be set to the same value, or can be set to different values. The dictionary construction algorithm is an algorithm for realizing compression by establishing a mapping relationship between data sequences and indexes, and long sequences that appear frequently are represented by short indexes, and is suitable for encoding with mode repetition.

[0115] If the repetition rate in the data characteristics is less than or equal to the fifth preset threshold, and the entropy level is greater than the sixth preset threshold, it is determined that the intermediate compression algorithm is a bit plane coding algorithm. The bit plane coding algorithm is a compression algorithm based on the bit level distribution characteristics of data, which splits data into multiple bit planes (such as the highest bit plane, the second highest bit plane, etc.) according to bits, encodes each bit plane separately, and is suitable for data with obvious bit level redundancy (such as binary image data).

[0116] If the repetition rate in the data characteristics is greater than the fifth preset threshold, and the entropy level is less than or equal to the sixth preset threshold, or the repetition rate in the data characteristics is less than or equal to the fifth preset threshold, and the entropy level is less than or equal to the sixth preset threshold, it is determined that the intermediate compression algorithm is a combination algorithm of the dictionary construction algorithm and the bit plane coding algorithm.

[0117] S302, compressing the basic compression encoding by using the intermediate compression algorithm to obtain intermediate compression encoding.

[0118] Further, the intermediate compression algorithm can be used to compress the basic compression encoding to obtain intermediate compression encoding to further remove redundant data.

[0119] In some optional implementations, if the intermediate compression algorithm is a dictionary construction algorithm, the intermediate compression algorithm is used to compress the basic compression encoding to obtain intermediate compression encoding, which can be realized by the following way:

[0120] First, the dictionary index encoding corresponding to the basic compression encoding is determined.

[0121] The dictionary index encoding is obtained by mapping the high-frequency sequence in the basic compression encoding to a short-length index, and the index is usually a binary number with a length much smaller than the original sequence.

[0122] For example, assuming the base compression encoding is "[2, 3] ABC [2, 3] ABC [4, 2]", where [2, 3] appears 3 times and ABC appears 2 times. The dictionary index encoding is assigned to the base compression encoding, for example, the dictionary index encoding of [2, 3] is assigned as 01, the dictionary index encoding of [4, 2] is assigned as 10, and the constructed dictionary is "{[2, 3]: 01, ABC: 10}".

[0123] Further, according to the dictionary index encoding, the intermediate compression encoding corresponding to the base compression encoding is determined.

[0124] For example, the dictionary index encoding can be used to replace the corresponding encoding of the base compression encoding to obtain the intermediate compression encoding, for example, the intermediate compression encoding "{[2, 3]: 01, ABC: 10} | 01100110 [4, 2]" is obtained.

[0125] S303, according to the data characteristics of the intermediate compression encoding, determine the deep compression algorithm corresponding to the intermediate compression encoding.

[0126] Further, the deep compression algorithm corresponding to the intermediate compression encoding can be determined according to the data characteristics of the intermediate compression encoding.

[0127] For example, the deep compression algorithm corresponding to the intermediate compression encoding can be determined according to the data characteristics of the intermediate compression encoding, that is, the entropy encoding algorithm (such as Huffman encoding, arithmetic encoding) or its combination that is suitable for the intermediate compression encoding is selected.

[0128] In some optional implementations, the deep compression algorithm corresponding to the intermediate compression encoding can be determined in the following way:

[0129] First, the bit plane encoding strategy can be determined according to the bit level distribution characteristics of the base compression encoding and the context information of the data block.

[0130] The bit level distribution characteristics refer to the distribution law of the base compression encoding at the binary bit level, such as the proportion of consecutive 1s, the proportion of consecutive 0s, the bit flip frequency, etc. The bit plane encoding strategy includes a multi-layer encoding strategy and a few-layer encoding strategy. The multi-layer encoding strategy refers to the strategy of splitting the data into multiple bit planes, for example, 8-bit data is split into 8 independent bit planes, and each bit plane is separately encoded (such as combined with entropy encoding) strategy, which is suitable for data with disordered bit level distribution. The few-layer encoding strategy refers to the strategy of merging multiple bit planes of data into a few layers, for example, 8-bit data is split into 2 layers, i.e. high 4-bit layer and low 4-bit layer, and the merged layer is simplified (such as run-length encoding) strategy, which is suitable for data with continuous bit level distribution.

[0131] In some optional implementations, the bit plane encoding strategy can be determined in the following way:

[0132] If the bit-level distribution feature of the base compression encoding represents that the bit-level data is continuous, and the context information of the data block represents that the data pattern in the data block is single, the bit-plane encoding strategy is determined as a few-layered encoding strategy. Wherein, the bit-level data continuous refers to that the length of the continuous same bit (such as continuous 1 or continuous 0) in the binary bit stream is longer, and the proportion is higher than a preset continuous threshold (such as 70%); the data pattern single refers to that the data block only contains a few fixed data patterns (such as only two patterns of “1111” and “0000”), and the number of patterns is less than a preset pattern threshold (such as 3).

[0133] If the bit-level distribution feature of the base compression encoding represents that the bit-level data is disordered, and the context information of the data block represents that the data in the data block has a dependent relationship, the bit-plane encoding strategy is determined as a multi-layered encoding strategy. Wherein, the bit-level data disordered refers to that the length of the continuous same bit in the binary bit stream is shorter, the bit flips frequently, and the proportion of the continuous bit is lower than a preset continuous threshold (such as 50%); the data has a dependent relationship refers to that there is a logical association between adjacent data elements in the data block (such as the next element is “11” with a high probability when the previous element is “10”), and the dependent relationship strength is higher than a preset dependent threshold (such as 70%).

[0134] Further, the base compression encoding can be compressed according to the bit-plane encoding strategy to obtain intermediate compression encoding.

[0135] For example, the bit plane can be split according to the bit-plane encoding strategy, and each plane can be encoded (run-length encoding, entropy encoding, etc.), and finally the intermediate compression encoding is obtained by merging.

[0136] In some optional implementations, if the intermediate compression algorithm is a combined algorithm of the dictionary construction algorithm and the bit-plane encoding algorithm, the base compression encoding is compressed by using the intermediate compression algorithm to obtain the intermediate compression encoding, which can be implemented in the following way:

[0137] First, the base compression encoding is compressed according to the dictionary construction algorithm to obtain dictionary compression encoding.

[0138] For example, the base compression encoding can be compressed according to the dictionary construction algorithm to obtain the dictionary compression encoding, so as to remove the redundancy of repeated sequences.

[0139] Further, the dictionary compression encoding is compressed according to the bit-plane encoding algorithm to obtain the intermediate compression encoding.

[0140] For example, the dictionary compression encoding can be analyzed at the bit level, an appropriate bit-plane encoding strategy can be selected, and the dictionary compression encoding can be compressed according to the bit-plane encoding algorithm,

[0141] Bit plane coding stage: carry out bit-level analysis on the dictionary compression coding, select an adaptive bit plane coding strategy, and compress the dictionary compression coding according to the bit plane coding strategy to remove bit-level redundancy, and finally obtain the intermediate compression coding.

[0142] For example, assuming that the base compression coding is "XYZXYZ111000XYZ111000", the base compression coding is compressed according to the dictionary construction algorithm, that is, "XYZ" is mapped to "001", and "111000" is mapped to "010", and the dictionary compression coding obtained after replacement is "001 001 010 001 010" (binary "001001010001010"); Further analyze the bit-level distribution characteristics (continuous bit ratio is 60%) and the data block context information (the pattern is simple but there are bit-level differences) of the dictionary compression coding, select a few hierarchical coding strategy, split the binary coding into high 3-bit layer and low 2-bit layer, and carry out run-length coding on each layer, and finally obtain the intermediate compression coding "(001, 3), (010, 2), (001, 1), (010, 1)".

[0143] In some optional implementations, referring to Figure 4 , Figure 4 A flowchart for determining a deep compression algorithm corresponding to the intermediate compression coding is provided, which specifically includes the following steps:

[0144] S401, according to the data characteristics of the intermediate compression coding, the intermediate compression coding is divided into at least one data segment.

[0145] Among them, the data segment refers to a sub-coding unit obtained by dividing the intermediate compression coding according to the mutation point of the data characteristics, and each data segment has a relatively uniform repetition rate and entropy level.

[0146] For example, the intermediate compression coding can be scanned, the repetition rate and entropy level in the sliding window (such as a window size of 16 bytes) can be calculated, and when the change amount of the repetition rate or the entropy level is higher than a preset change threshold (such as 20%), the segmentation is carried out, and a plurality of data segments are obtained.

[0147] S402, determine the deep compression algorithm corresponding to each data segment.

[0148] For example, the deep compression algorithm corresponding to each data segment can be determined according to the repetition rate and entropy level of each data segment.

[0149] For example, the deep compression algorithm corresponding to each data segment can be determined in the following way:

[0150] If the repetition rate in the data characteristics of the data segment is greater than the seventh preset threshold, the deep compression algorithm is determined to be Huffman coding.

[0151] wherein the seventh preset threshold is a judgment threshold of the repetition rate of the data segment, and is used for selecting the final depth compression algorithm, and can be set according to an actual scene, for example, can be set to 40%. The Huffman coding is an entropy coding algorithm based on the frequency of data elements, and short coding is allocated to high-frequency elements, and long coding is allocated to low-frequency elements, and is suitable for data with high repetition rate and low entropy level.

[0152] If the entropy level in the data characteristics of the data segment is greater than the eighth preset threshold, it is determined that the depth compression algorithm is the arithmetic coding.

[0153] wherein the eighth preset threshold is a judgment threshold of the entropy level of the data segment, and is used for selecting the final depth compression algorithm, and can be set according to an actual scene, for example, can be set to 2.2. The arithmetic coding is an entropy coding algorithm based on a probability cumulative distribution, and maps the entire data sequence into a decimal in the interval [0, 1), and is suitable for data with high entropy level and complex data distribution, and the compression rate is usually higher than that of the Huffman coding, but the complexity is higher.

[0154] If the repetition rate in the data characteristics of the data segment is less than or equal to the seventh preset threshold, or the entropy level in the data characteristics of the data segment is less than or equal to the eighth preset threshold, it is determined that the depth compression algorithm is the Huffman coding and the arithmetic coding.

[0155] For example, it is assumed that the intermediate compression coding is “00100101011011001010”, and three data segments are obtained after division:

[0156] Data segment 1: “001001” (the repetition rate is 45%, which is greater than the seventh preset threshold 40%, and the entropy level is 1.8, which is less than the eighth preset threshold 2.2), and the Huffman coding is selected, and “101011” is obtained after coding;

[0157] Data segment 2: “010110” (the repetition rate is 25%, which is less than the seventh preset threshold 40%, and the entropy level is 2.5, which is greater than the eighth preset threshold 2.2), and the arithmetic coding is selected, and “01101001” is obtained after coding;

[0158] Data segment 3: “11001010” (the repetition rate is 30%, which is less than the seventh preset threshold 40%, and the entropy level is 1.9, which is less than the eighth preset threshold 2.2), and the combination of the Huffman coding and the arithmetic coding is selected, and “110100110” is obtained after coding;

[0159] The final sub-compression data is “101011|01101001|110100110” (adding a segment separator).

[0160] In the above embodiments, through data segment division and segment adaptive coding, fine final compression of intermediate compression coding is realized, compared with global single depth coding, the compression rate is improved, and arithmetic coding is used for high entropy data segment to ensure compression effect, and Huffman coding is used for low entropy data segment to control complexity, balancing compression rate and calculation overhead.

[0161] It should be understood that, although each step in the flowchart involved in each embodiment as described above is shown in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or steps or stages in other steps.

[0162] Based on the same inventive concept, the embodiments of the present application also provide a data compression device for implementing the above-mentioned data compression method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more data compression device embodiments provided below can refer to the limitations of the data compression method in the above text, which will not be repeated here.

[0163] In one exemplary embodiment, as shown in Figure 5 a data compression device is provided, comprising:

[0164] The acquisition module 10 is configured to acquire the to-be-compressed data.

[0165] The division module 20 is configured to divide the to-be-compressed data into at least one data block according to the data characteristics of the to-be-compressed data, wherein the data characteristics include at least one of an entropy level, a repetition rate, and a context correlation.

[0166] The compression module 30 is configured to, for each data block, according to the data characteristics of the data block, adopt a compression algorithm matched with the data block to compress the data block, and obtain sub-compression data corresponding to the data block.

[0167] The splicing module 40 is configured to splice the sub-compression data to obtain target compression data of the to-be-compressed data.

[0168] The data compression device obtains to-be-compressed data, divides the to-be-compressed data into at least one data block according to data characteristics of the to-be-compressed data, adopts a compression algorithm matched with each data block to compress the data block according to the data characteristics of the data block, obtains sub-compressed data corresponding to the data block, and splices the sub-compressed data to obtain target compressed data of the to-be-compressed data. According to the data characteristics of the to-be-compressed data, the to-be-compressed data is divided into different data blocks, and different compression algorithms are adopted to compress the data blocks, so that a suitable compression algorithm can be selected according to the data characteristics, thereby improving the data compression efficiency.

[0169] In one embodiment, the compression module 30 specifically includes:

[0170] A first determination unit is configured to determine a basic compression algorithm matched with the data block according to the data characteristics of the data block.

[0171] A first compression unit is configured to perform basic compression on the data block by using the basic compression algorithm to obtain a basic compression code.

[0172] A second determination unit is configured to determine a deep compression algorithm corresponding to the basic compression code.

[0173] A second compression unit is configured to perform deep compression on the basic compression code by using the deep compression algorithm to obtain sub-compressed data corresponding to the data block.

[0174] In one embodiment, the first determination unit is specifically configured to:

[0175] If the repetition rate of the data block is higher than a first preset threshold and the entropy level is lower than a second preset threshold, the basic compression algorithm is determined to be a sliding window algorithm; if the entropy level of the data block is higher than a third preset threshold and the context correlation is higher than a fourth preset threshold, the basic compression algorithm is determined to be a context-aware modeling algorithm; if the data block meets a preset condition, the basic compression algorithm is determined to be a combination algorithm of the sliding window algorithm and the adaptive Huffman coding; wherein the preset condition is other conditions except that the repetition rate of the data block is higher than the first preset threshold and the entropy level is lower than the second preset threshold, and that the entropy level of the data block is higher than the third preset threshold and the context correlation is higher than the fourth preset threshold.

[0176] In one embodiment, the second determination unit specifically includes:

[0177] A first determination sub-unit is configured to determine an intermediate compression algorithm according to the data characteristics of the data block.

[0178] A compression sub-unit is configured to compress the basic compression code by using the intermediate compression algorithm to obtain an intermediate compression code.

[0179] The second determining sub-unit is configured to determine the deep compression algorithm corresponding to the intermediate compression coding according to the data characteristics of the intermediate compression coding.

[0180] In one of the embodiments, the first determining sub-unit is specifically configured to:

[0181] If the repetition rate in the data characteristics is greater than the fifth preset threshold and the entropy level is less than or equal to the sixth preset threshold, the intermediate compression algorithm is determined to be the dictionary construction algorithm; if the repetition rate in the data characteristics is less than or equal to the fifth preset threshold and the entropy level is greater than the sixth preset threshold, the intermediate compression algorithm is determined to be the bit plane coding algorithm; if the repetition rate in the data characteristics is greater than the fifth preset threshold and the entropy level is less than or equal to the sixth preset threshold, or, the repetition rate in the data characteristics is less than or equal to the fifth preset threshold and the entropy level is less than or equal to the sixth preset threshold, the intermediate compression algorithm is determined to be the combined algorithm of the dictionary construction algorithm and the bit plane coding algorithm.

[0182] In one of the embodiments, if the intermediate compression algorithm is the dictionary construction algorithm, the compression sub-unit is specifically configured to:

[0183] determine the dictionary index coding corresponding to the basic compression coding; and determine the intermediate compression coding corresponding to the basic compression coding according to the dictionary index coding.

[0184] In one of the embodiments, if the intermediate compression algorithm is the bit plane coding algorithm, the compression sub-unit is specifically configured to:

[0185] determine the bit plane coding strategy according to the bit level distribution characteristics of the basic compression coding and the context information of the data block; wherein, the bit plane coding strategy includes the multi-layer coding strategy and the few-layer coding strategy; and compress the basic compression coding according to the bit plane coding strategy to obtain the intermediate compression coding.

[0186] In one of the embodiments, the compression sub-unit is specifically configured to:

[0187] If the bit level distribution characteristics of the basic compression coding represent that the bit level data is continuous, and the context information of the data block represents that the data mode in the data block is single, the bit plane coding strategy is determined to be the few-layer coding strategy; if the bit level distribution characteristics of the basic compression coding represent that the bit level data is disordered, and the context information of the data block represents that there is a dependent relationship among the data in the data block, the bit plane coding strategy is determined to be the multi-layer coding strategy.

[0188] In one of the embodiments, if the intermediate compression algorithm is the combined algorithm of the dictionary construction algorithm and the bit plane coding algorithm, the compression sub-unit is specifically configured to:

[0189] The base compression encoding is compressed according to a dictionary construction algorithm to obtain dictionary compression encoding; and the dictionary compression encoding is compressed according to a bit plane coding algorithm to obtain intermediate compression encoding.

[0190] In one of the embodiments, the second determining subunit is specifically configured to:

[0191] According to the data features of the intermediate compression encoding, the intermediate compression encoding is divided into at least one data segment; and a deep compression algorithm corresponding to each data segment is determined.

[0192] In one of the embodiments, the second determining subunit is specifically configured to:

[0193] If the repetition rate in the data features of the data segment is greater than a seventh preset threshold, the deep compression algorithm is determined to be Huffman coding; if the entropy level in the data features of the data segment is greater than an eighth preset threshold, the deep compression algorithm is determined to be arithmetic coding; if the repetition rate in the data features of the data segment is less than or equal to the seventh preset threshold, or the entropy level in the data features of the data segment is less than or equal to the eighth preset threshold, the deep compression algorithm is determined to be Huffman coding and arithmetic coding.

[0194] The modules in the data compression device described above can be all or partially implemented by software, hardware and combinations thereof. The modules described above can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so as to call and execute the operations corresponding to the modules by the processor.

[0195] In one exemplary embodiment, a computer device is provided, which can be a server, and an internal structure diagram thereof can be as shown in Figure 6 The computer device includes a processor, a memory, an input / output (I / O) interface and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store data to be compressed and target compression data. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement a data compression method.

[0196] Those skilled in the art can understand that, Figure 6The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0197] In an exemplary embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the data compression method described in any of the above embodiments when executing the computer program.

[0198] In an embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program implementing the steps of the data compression method described in any of the above embodiments when executed by a processor.

[0199] In an embodiment, a computer program product is provided, comprising a computer program, and the computer program implementing the steps of the data compression method described in any of the above embodiments when executed by a processor.

[0200] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0201] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0202] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a specific manner, but should not be construed as limiting the scope of the patent of the present application. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A data compression method, characterized in that, The method includes: Obtain the data to be compressed; Based on the data characteristics of the data to be compressed, the data to be compressed is divided into at least one data block; wherein, the data characteristics include at least one of entropy level, repetition rate and contextual relevance; For each data block, based on the data characteristics of the data block, a compression algorithm matching the data block is used to compress the data block to obtain the sub-compressed data corresponding to the data block; The sub-compressed data are concatenated to obtain the target compressed data of the data to be compressed.

2. The method according to claim 1, characterized in that, The step of compressing the data block according to its data characteristics using a compression algorithm matching the data block to obtain sub-compressed data corresponding to the data block includes: Based on the data characteristics of the data block, determine the basic compression algorithm that matches the data block; The data block is subjected to basic compression using the aforementioned basic compression algorithm to obtain basic compressed encoding; Determine the deep compression algorithm corresponding to the basic compression coding; The deep compression algorithm is used to perform deep compression on the basic compression code to obtain the sub-compressed data corresponding to the data block.

3. The method according to claim 2, characterized in that, The step of determining the basic compression algorithm matching the data block based on the data characteristics of the data block includes: If the repetition rate of the data block is higher than the first preset threshold and the entropy level is lower than the second preset threshold, then the basic compression algorithm is determined to be the sliding window algorithm. If the entropy level of the data block is higher than the third preset threshold and the context relevance is higher than the fourth preset threshold, then the basic compression algorithm is determined to be a context-aware modeling algorithm. If the data block meets the preset conditions, the basic compression algorithm is determined to be a combination of sliding window algorithm and adaptive Huffman coding; wherein, the preset conditions are other cases except for the case where the repetition rate of the data block is higher than the first preset threshold and the entropy level is lower than the second preset threshold, and the case where the entropy level of the data block is higher than the third preset threshold and the context relevance is higher than the fourth preset threshold.

4. The method according to claim 2, characterized in that, The determination of the deep compression algorithm corresponding to the basic compression coding includes: Based on the data characteristics of the data block, determine the intermediate compression algorithm; The intermediate compression algorithm is used to compress the basic compression code to obtain the intermediate compression code; Based on the data characteristics of the intermediate compression code, a deep compression algorithm corresponding to the intermediate compression code is determined.

5. The method according to claim 4, characterized in that, The step of determining the intermediate compression algorithm based on the data characteristics of the data block includes: If the repetition rate in the data characteristics is greater than the fifth preset threshold and the entropy level is less than or equal to the sixth preset threshold, then the intermediate compression algorithm is determined to be a dictionary construction algorithm. If the repetition rate in the data characteristics is less than or equal to the fifth preset threshold, and the entropy level is greater than the sixth preset threshold, then the intermediate compression algorithm is determined to be a bit-plane coding algorithm. If the repetition rate in the data characteristics is greater than the fifth preset threshold and the entropy level is less than or equal to the sixth preset threshold, or if the repetition rate in the data characteristics is less than or equal to the fifth preset threshold and the entropy level is less than or equal to the sixth preset threshold, then the intermediate compression algorithm is determined to be a combination of dictionary construction algorithm and bit-plane encoding algorithm.

6. The method according to claim 5, characterized in that, If the intermediate compression algorithm is a dictionary construction algorithm, then the step of using the intermediate compression algorithm to compress the basic compressed code to obtain the intermediate compressed code includes: Determine the dictionary index code corresponding to the basic compression code; Based on the dictionary index encoding, the intermediate compression encoding corresponding to the basic compression encoding is determined.

7. The method according to claim 5, characterized in that, If the intermediate compression algorithm is a bit-plane coding algorithm, then the step of using the intermediate compression algorithm to compress the basic compressed code to obtain the intermediate compressed code includes: Based on the bit-level distribution characteristics of the basic compression coding and the context information of the data block, a bit-plane coding strategy is determined; wherein, the bit-plane coding strategy includes a multi-layer coding strategy and a few-layer coding strategy; According to the bit-plane encoding strategy, the basic compressed code is compressed to obtain the intermediate compressed code.

8. The method according to claim 7, characterized in that, The step of determining the bit-plane coding strategy based on the bit-level distribution characteristics of the basic compression coding and the context information of the data block includes: If the bit-level distribution characteristics of the basic compression coding indicate that the bit-level data is continuous, and the context information of the data block indicates that the data pattern within the data block is singular, then the bit-plane coding strategy is determined to be a low-layer coding strategy. If the bit-level distribution characteristics of the basic compression coding indicate that the bit-level data is unordered, and the context information of the data block indicates that the data within the data block has a dependency relationship, then the bit-plane coding strategy is determined to be a multi-layer coding strategy.

9. The method according to claim 5, characterized in that, If the intermediate compression algorithm is a combination of a dictionary construction algorithm and a bit-plane encoding algorithm, then the step of using the intermediate compression algorithm to compress the basic compressed code to obtain the intermediate compressed code includes: The basic compression code is compressed according to the dictionary construction algorithm to obtain dictionary compression code; The dictionary compression code is compressed according to the bit-plane encoding algorithm to obtain the intermediate compression code.

10. The method according to claim 4, characterized in that, The step of determining the deep compression algorithm corresponding to the intermediate compression code based on the data characteristics of the intermediate compression code includes: Based on the data characteristics of the intermediate compressed code, the intermediate compressed code is divided into at least one data segment; Determine the deep compression algorithm corresponding to each data segment.

11. The method according to claim 10, characterized in that, The determination of the depth compression algorithm corresponding to each data segment includes: If the repetition rate in the data features of the data segment is greater than the seventh preset threshold, then the deep compression algorithm is determined to be Huffman coding; If the entropy level in the data features of the data segment is greater than the eighth preset threshold, then the deep compression algorithm is determined to be arithmetic coding; If the repetition rate in the data features of the data segment is less than or equal to the seventh preset threshold, or if the entropy level in the data features of the data segment is less than or equal to the eighth preset threshold, then the deep compression algorithm is determined to be the Huffman coding and the arithmetic coding.

12. A data compression device, characterized in that, The device includes: The acquisition module is used to acquire the data to be compressed; A partitioning module is used to partition the data to be compressed into at least one data block according to the data characteristics of the data to be compressed; wherein the data characteristics include at least one of entropy level, repetition rate and contextual relevance; The compression module is used to compress each data block according to its data characteristics using a compression algorithm that matches the data block, thereby obtaining sub-compressed data corresponding to the data block. The splicing module is used to splice the various sub-compressed data to obtain the target compressed data of the data to be compressed.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images