A data compression method, system, device and computer readable storage medium

Abstract

The application discloses a data compression method, system, device and computer readable storage medium, obtains target data to be compressed; statistics the total number of bytes in the target data and the occurrence probability of each type of character; determine the target linear function of the function x*log2(x) under the preset approximation condition; determine the information entropy operation formula without logarithm operation based on the target linear function; calculate the target information entropy of the target data based on the total number of bytes and the occurrence probability through the information entropy operation formula; judge whether the target information entropy is less than the preset value, if the target information entropy is less than the preset value, compress the target data, if the target information entropy is greater than or equal to the preset value, do not compress the target data. In the application, the function x*log2(x) is converted into the target linear function without logarithm operation, so that the information entropy operation formula without logarithm operation can be determined based on the target linear function, the calculation efficiency of the target information entropy is improved, and the data compression efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of data compression technology, and more specifically, to a data compression method, system, device, and computer-readable storage medium. Background Technology

[0002] Data compression can potentially reduce data storage space and increase the logical capacity of storage devices, thereby lowering data storage and transmission costs, making it a highly attractive technology. However, data compression is a computationally intensive operation, and its actual data reduction effect depends on the compressibility of the data being compressed. Compressing incompressible data will not reduce its size, wasting valuable computational resources. Therefore, accurately predicting data compressibility and avoiding compressing incompressible data can prevent the waste of expensive compression computations.

[0003] In the data compression process, information entropy is a key indicator for measuring data compressibility and predicting data compression rate. Calculating information entropy requires logarithmic operations, which involves computational complexity and complex hardware implementation logic, resulting in low computational efficiency and consequently affecting data compression efficiency.

[0004] In conclusion, improving the efficiency of data compression is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a data compression method that can, to some extent, solve the technical problem of how to improve data compression efficiency. This application also provides a data compression system, apparatus, and computer-readable storage medium.

[0006] To achieve the above objectives, this application provides the following technical solution:

[0007] A data compression method, comprising:

[0008] Obtain the target data to be compressed;

[0009] Calculate the total number of bytes in the target data and the probability of occurrence of each type of character;

[0010] Determine the target linear function x*log2(x) under the preset approximation conditions;

[0011] Based on the target linear function, determine the information entropy calculation formula without logarithmic operations;

[0012] The target information entropy of the target data is calculated using the information entropy calculation formula, based on the total number of bytes and the probability of occurrence.

[0013] Determine whether the target information entropy is less than a preset value. If the target information entropy is less than the preset value, then compress the target data. If the target information entropy is greater than or equal to the preset value, then do not compress the target data.

[0014] Preferably, the determination of the target linear function x*log2(x) under preset approximation conditions includes:

[0015] Determine the target linear function x*log2(x) under the preset approximation conditions;

[0016] The target linear function includes:

[0017] y = x * k(x) + 2(x - 2) k(x) );

[0018] Where k(x) represents a value such that x is between [2]. K ,2 K+1 The function of the K value.

[0019] Preferably, determining the information entropy calculation formula without logarithmic operations based on the target linear function includes:

[0020] The information entropy calculation formula without logarithmic operations is determined based on the target linear function;

[0021] The information entropy calculation formula includes:

[0022]

[0023] Wherein, H represents the target information entropy; N represents the total number of bytes; M represents the total number of character types; f(i) represents the occurrence probability of the i-th type of character, where the value of i ranges from [0, M].

[0024] Preferably, the determination of the target linear function x*log2(x) under preset approximation conditions includes:

[0025] Determine the target linear function x*log2(x) under the preset approximation conditions;

[0026] The target linear function includes:

[0027] y=x*k(x)+((2-a)x-2 k(x)+1 );

[0028] Where k(x) represents a value such that x is between [2]. K ,2 K+1 ) is a function of the K value; a represents the adjustment parameter.

[0029] Preferred options also include:

[0030] Based on 1.5*2 K The value of the adjustment parameter is determined by the value of the adjustment parameter.

[0031] Preferably, determining the information entropy calculation formula without logarithmic operations based on the target linear function includes:

[0032] The information entropy calculation formula without logarithmic operations is determined based on the target linear function;

[0033] The information entropy calculation formula includes:

[0034]

[0035] Wherein, H represents the target information entropy; N represents the total number of bytes; M represents the total number of character types; f(i) represents the occurrence probability of the i-th type of character, where the value of i ranges from [0, M].

[0036] Preferably, the calculation process for k(x) includes:

[0037] Convert the value of x to a 32-bit binary integer;

[0038] Count the number of prefix 0s in the given 32-bit integer;

[0039] The difference between 31 and the aforementioned values ​​is taken as the K value corresponding to k(x).

[0040] Preferably, the process of calculating the target information entropy of the target data based on the total number of bytes and the probability of occurrence using the information entropy calculation formula includes:

[0041] The result of x*k(x) is determined based on the shift-add operation;

[0042] The operation of taking the lower (k(x)-1) bits of x yields (x-2) k(x) The result of the operation.

[0043] A data compression system, comprising:

[0044] The first acquisition module is used to acquire the target data to be compressed;

[0045] The first statistics module is used to count the total number of bytes in the target data and the probability of occurrence of each type of character;

[0046] The first determining module is used to determine the target linear function x*log2(x) under preset approximation conditions;

[0047] The second determining module is used to determine the information entropy calculation formula without logarithmic operations based on the target linear function;

[0048] The first calculation module is used to calculate the target information entropy of the target data based on the total number of bytes and the probability of occurrence using the information entropy calculation formula.

[0049] The first judgment module is used to determine whether the target information entropy is less than a preset value. If the target information entropy is less than the preset value, the target data is compressed. If the target information entropy is greater than or equal to the preset value, the target data is not compressed.

[0050] A data compression device, comprising:

[0051] Memory, used to store computer programs;

[0052] A processor for implementing the steps of any of the above-described data compression methods when executing the computer program.

[0053] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the data compression methods described above.

[0054] This application provides a data compression method that involves: acquiring target data to be compressed; statistically analyzing the total number of bytes and the probability of occurrence of each type of character in the target data; determining a target linear function x*log2(x) under preset approximation conditions; determining an information entropy calculation formula without logarithmic operations based on the target linear function; calculating the target information entropy of the target data based on the total number of bytes and their occurrence probabilities using the information entropy calculation formula; and determining whether the target information entropy is less than a preset value. If the target information entropy is less than the preset value, the target data is compressed; if the target information entropy is greater than or equal to the preset value, the target data is not compressed. In this application, after statistically analyzing the total number of bytes and the probability of occurrence of each type of character in the target data, it is necessary to determine the target linear function x*log2(x) under preset approximation conditions, that is, to transform the function x*log2(x) into a target linear function without logarithmic operations and with simpler logic. In this way, an information entropy calculation formula without logarithmic operations can be determined based on the target linear function, and then the target information entropy of the target data can be calculated without performing complex logarithmic operations, thus improving the calculation efficiency of the target information entropy and consequently improving the efficiency of data compression based on the target information entropy. The data compression system, device, and computer-readable storage medium provided in this application also solve the corresponding technical problems. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0056] Figure 1 A flowchart illustrating a data compression method provided in this application embodiment;

[0057] Figure 2 The simulation graph of x*log2(x);

[0058] Figure 3 For x*log2(x) and x*k(x)+(2*x-2 k(x)+1 The simulation comparison results are shown in the figure.

[0059] Figure 4 This is a schematic diagram of the structure of a data compression system provided in an embodiment of this application;

[0060] Figure 5 This is a schematic diagram of the structure of a data compression device provided in an embodiment of this application;

[0061] Figure 6 This is another structural schematic diagram of a data compression device provided in an embodiment of this application. Detailed Implementation

[0062] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0063] Please see Figure 1 , Figure 1 This is a flowchart of a data compression method provided in an embodiment of this application.

[0064] This application provides a data compression method that may include the following steps:

[0065] Step S101: Obtain the target data to be compressed.

[0066] In practical applications, the target data to be compressed can be obtained first. The type and size of the target data can be determined according to the specific application scenario. This application does not make any specific restrictions here.

[0067] Step S102: Calculate the total number of bytes in the target data and the probability of occurrence of each type of character.

[0068] In practical applications, after obtaining the target data to be compressed, the total number of bytes in the target data and the probability of occurrence of each type of character can be counted. Taking a character as an example, computers usually use one byte (also known as 1 Byte), or 8 bits, to represent a character. Therefore, there are 2^8 = 256 types of characters, which means that the mathematical range of character values ​​is [0, 255]. The probability of occurrence of each type of character is the probability of each unique character, that is, each value in the range of 0 to 255, appearing separately, in order to prepare for the subsequent calculation of the information entropy of the target data.

[0069] Step S103: Determine the target linear function of the function x*log2(x) under the preset approximation conditions.

[0070] Step S104: Determine the information entropy calculation formula without logarithmic operations based on the objective linear function.

[0071] In practical applications, on the one hand, the existing information entropy calculation process involves logarithmic operations, which are time-consuming and lead to low efficiency in information entropy calculation; on the other hand, it is considered that using information entropy to measure data compressibility or predict data compression rate does not require an accurate information entropy value, and a high-precision information entropy approximation has a negligible impact on the accuracy of compression rate prediction. Therefore, this application determines the target linear function of the function x*log2(x) required in the information entropy calculation process under preset approximation conditions, and determines the information entropy calculation formula without logarithmic operations based on the target linear function, so as to eliminate the logarithmic operations in the information entropy calculation process and obtain an information entropy calculation formula that does not affect the compression rate prediction.

[0072] Step S105: Calculate the target information entropy of the target data based on the total number of bytes and the probability of occurrence using the information entropy calculation formula.

[0073] In practical applications, once the information entropy calculation formula without logarithmic operations is determined based on the target linear function, the target information entropy of the target data can be calculated based on the total number of bytes and the probability of occurrence using the information entropy calculation formula.

[0074] Step S106: Determine whether the target information entropy is less than the preset value. If the target information entropy is less than the preset value, then execute step S107: Compress the target data; if the target information entropy is greater than or equal to the preset value, then execute step S108: Do not compress the target data.

[0075] In practical applications, after calculating the target information entropy of the target data based on the total number of bytes and the probability of occurrence using the information entropy calculation formula, it can be determined whether the target information entropy is less than a preset value. If the target information entropy is less than the preset value, the target data is compressed; if the target information entropy is greater than or equal to the preset value, the target data is not compressed. It should be noted that the specific value of the preset value used to determine whether to compress the target data can be determined according to the application scenario, and this application does not impose a specific limitation here.

[0076] This application provides a data compression method that involves: acquiring target data to be compressed; statistically analyzing the total number of bytes and the probability of occurrence of each type of character in the target data; determining a target linear function x*log2(x) under preset approximation conditions; determining an information entropy calculation formula without logarithmic operations based on the target linear function; calculating the target information entropy of the target data based on the total number of bytes and their occurrence probabilities using the information entropy calculation formula; and determining whether the target information entropy is less than a preset value. If the target information entropy is less than the preset value, the target data is compressed; if the target information entropy is greater than or equal to the preset value, the target data is not compressed. In this application, after statistically analyzing the total number of bytes and the probability of occurrence of each type of character in the target data, it is necessary to determine the target linear function x*log2(x) under preset approximation conditions, that is, to transform the function x*log2(x) into a target linear function without logarithmic operations and with simpler logic. In this way, an information entropy calculation formula without logarithmic operations can be determined based on the target linear function, and then the target information entropy of the target data can be calculated without performing complex logarithmic operations, thus improving the calculation efficiency of the target information entropy and consequently improving the efficiency of data compression based on the target information entropy.

[0077] In an exemplary application scenario, the data compression method provided in this application embodiment takes into account that in an existing system S, there are multiple events S = {E1,...,E...}. n The probability distribution for each event is P = {p1,...,p}. n Then the information for each event itself is:

[0078] I e =-log2p i ;

[0079] The logarithm is base 2 and its unit is bits.

[0080] Information entropy is the average amount of information contained in each received message, that is:

[0081]

[0082] In the context of data compression, information entropy refers to byte entropy. The basic unit of data storage is a byte (also known as 1 Byte), which consists of 8 data bits, i.e., 8 bits. Therefore, the value of n in the above formula ranges from [0, 255], with a total of 256 possible values. Let the total number of bytes in the input message be N, and the probability (which can be understood as the number of times each character i appears) be f(i) (where i ∈ [0, 255]), then...

[0083] The probability of character x appearing is p(i) = f(i) / N. Then...

[0084] The formula for calculating information entropy is:

[0085]

[0086]

[0087] Where H represents the target information entropy, i.e. the information entropy of the target data; N represents the total number of bytes, i.e. the total number of bytes in the target data; M represents the total number of character types, i.e. the total number of unique characters in the target data, i.e., characters with different values; f(i) represents the probability of the i-th type of character appearing, i.e. the probability of the i-th type of character appearing in the target data, where the value of i ranges from [0, M].

[0088] Multiplying both sides of the above information entropy calculation formula by N yields:

[0089]

[0090] At this point, the forms of N·log2(N) and f(i)log2(f(i)) can be uniformly expressed as x*log2(x), that is, it is only necessary to determine the target linear function of the function x*log2(x) under the preset approximation conditions, without performing the complex logarithmic operations in the information entropy calculation formula; the simulation graph of x*log2(x) is as follows Figure 2 As shown, by Figure 2 It can be seen that x*log2(x) approximates a linear distribution y=kx;

[0091] Preset approximation conditions can be understood as calculating the target linear function by dividing it into intervals to improve accuracy. The interval to which the value of x belongs can be set [2]. K ,2 K+1 Then, based on this interval, x*log2(x) is transformed into an approximate objective linear function;

[0092] Specifically, x*log2(x) at point 2 K The value at that location is K·2 K At point 2 K+1 The value at point K is (K+1)·2K+1 By applying the slope formula to determine the objective linear function, the slope R is obtained as follows:

[0093]

[0094] The slope R = (y2 - y1) / (x2 - x1) is calculated using the formula for a linear distribution. The linear equation can then be expressed as y - y1 = R(x - x1). The objective linear function that approximates the function x*log2(x) can then be expressed as:

[0095] yK·2 K = (K+2)(x-2) K );

[0096] Suppose k(x) represents a value such that the value of x is between [2]. K ,2 K+1 Let k(x) be a function of the value of K, that is, a function that takes integer values ​​less than or equal to the logarithmic base of x. Then the objective linear function can be expressed as:

[0097] y = k(x) * 2 k(x) +(x-2 k(x) )*(2 k(x) +2)=x*k(x)+2(x-2 k(x) );

[0098] At this point, the formula for calculating information entropy can be expressed as:

[0099]

[0100] Furthermore, considering:

[0101] x*log2(x)≈k(x)*2 k(x) +(x-2 k(x) )*(k(x)+2)=x*k(x)+(2*x-2 k(x)+1 ), for x*log2(x)

[0102] and x*k(x)+(2*x-2 k(x)+1 By comparison, we can see that (2*x-2) k(x)+1 () is an estimate of the fractional part of log2(x). For ease of understanding, assume the value of x is 5, then k(x) = 2, log2(x) = 2.32. Therefore, x*k(x) corresponds to the integer part of x*log2(x), which is the operation of 5*2, (2*x-2) / (x*log2(x)). k(x)+1 This corresponds to the operation on the decimal part of x*log2(x), specifically the operation on 0.32*5. As mentioned earlier, x*log2(x) approximates a linear distribution, while x*k(x) is a linear distribution, (2*x-2...).k (x)+1 The approximation is a compensated approximation of the true x*log2(x) based on a linear distribution. Therefore, to further improve the similarity accuracy between the objective linear function and x*log2(x), we can... k(x)+1 Introducing the adjustment parameter 'a' further adjusts the objective linear function to:

[0103] y=x*k(x)+((2-a)x-2 k(x)+1 );

[0104] At this point, the formula for calculating information entropy can be:

[0105]

[0106] It should be noted that in specific application scenarios, a fixed value for the adjustment parameter 'a' can be preset based on the changes in x and log2(x). For example, different values ​​of 'a' can be given in the intervals [2,4], [4,8], [8,16], and [16,32]. Furthermore, comparing x*log2(x) and x*k(x)+(2*x-2... k(x)+1 After the simulation graphics are generated, the comparison results are as follows: Figure 3 As shown, the line at the top represents x*k(x)+(2*x-2). k(x)+1 As can be seen from the simulation graph, x*log2(x) and x*k(x)+(2*x-2) k(x)+1 ) in [2 K ,2 K +1 The maximum difference point in the interval is 1.5 * 2. K Therefore, in order to further improve the accuracy of adjusting parameter 'a', it can also be based on 1.5*2 K The value is used to determine the value of the adjustment parameter, etc.

[0107] In specific application scenarios, to find the integer K, the function k(x) is defined as finding the integer K for the value x, where the value of x is in the interval [2]. K ,2 K+1 The operation process of k(x) can be as follows: convert the value of x into a 32-bit binary integer; count the number of prefix 0s in the 32-bit integer; and take the difference between the 31 and the number of prefix 0s as the K value corresponding to k(x).

[0108] For ease of understanding, assume the value of x is 12, which is 12 = 1100b in binary. The highest bit is 1 in the 3rd bit, and the values ​​of bits 4 to 31 are all 0. Then K = 32 - (31 - 4 + 1) = 4.

[0109] Of course, the following formula can also be used to calculate k(x):

[0110]

[0111] In specific application scenarios, to improve computational efficiency, in the process of calculating the target information entropy of the target data based on the total number of bytes and the probability of occurrence using the information entropy calculation formula, k(x) is the logarithmic base of x, which is a very small number relative to x. Therefore, x*k(x) is a binary multiplication that can be transformed into a simple shift-add operation. That is, the result of the x*k(x) operation is determined based on the shift-add operation; the operation of taking the lower (k(x)-1) bits of x yields (x-2... k(x) The calculation result of the information entropy is used to reduce the computational complexity of the entire information entropy and facilitate the hardware implementation of information entropy calculation. In addition, experiments have shown that the information entropy calculation method of this application is about 15% faster than the standard information entropy calculation method, and the error is within 2%, resulting in fast data compression efficiency and small error.

[0112] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a data compression system provided in an embodiment of this application.

[0113] This application provides a data compression system that may include:

[0114] The first acquisition module 101 is used to acquire the target data to be compressed;

[0115] The first statistics module 102 is used to count the total number of bytes in the target data and the probability of occurrence of each type of character;

[0116] The first determining module 103 is used to determine the target linear function of the function x*log2(x) under preset approximation conditions;

[0117] The second determining module 104 is used to determine the information entropy calculation formula without logarithmic operations based on the target linear function;

[0118] The first calculation module 105 is used to calculate the target information entropy of the target data based on the total number of bytes and the probability of occurrence using the information entropy calculation formula.

[0119] The first judgment module 106 is used to determine whether the target information entropy is less than a preset value. If the target information entropy is less than the preset value, the target data is compressed. If the target information entropy is greater than or equal to the preset value, the target data is not compressed.

[0120] This application provides a data compression system, wherein the first determining module may include:

[0121] The first determining unit is used to determine the target linear function x*log2(x) under preset approximation conditions;

[0122] The objective linear functions include:

[0123] y = x * k(x) + 2(x - 2) k(x) );

[0124] Where k(x) represents a value such that x is between [2]. K ,2 K+1 The function of the K value.

[0125] This application provides a data compression system in which the second determining module may include:

[0126] The second determining unit is used to determine the information entropy calculation formula without logarithmic operations based on the target linear function;

[0127] The formulas for calculating information entropy include:

[0128]

[0129] Where H represents the target information entropy, i.e. the information entropy of the target data; N represents the total number of bytes, i.e. the total number of bytes in the target data; M represents the total number of character types, i.e. the total number of unique characters in the target data, i.e., characters with different values; f(i) represents the probability of the i-th type of character appearing, i.e. the probability of the i-th type of character appearing in the target data, where the value of i ranges from [0, M].

[0130] This application provides a data compression system, wherein the first determining module may include:

[0131] The third determining unit is used to determine the target linear function x*log2(x) under preset approximation conditions;

[0132] The objective linear functions include:

[0133] y=x*k(x)+((2-a)x-2 k(x)+1 );

[0134] Where k(x) represents a value such that x is between [2]. K ,2 K+1 ) is a function of the K value; a represents the adjustment parameter.

[0135] The data compression system provided in this application embodiment may further include:

[0136] The third determining module is used based on 1.5*2 K The value determines the value of the adjustment parameter.

[0137] This application provides a data compression system in which the second determining module may include:

[0138] The fourth determining unit is used to determine the information entropy calculation formula without logarithmic operations based on the target linear function;

[0139] The formulas for calculating information entropy include:

[0140]

[0141] Where H represents the target information entropy, i.e. the information entropy of the target data; N represents the total number of bytes, i.e. the total number of bytes in the target data; M represents the total number of character types, i.e. the total number of unique characters in the target data, i.e., characters with different values; f(i) represents the probability of the i-th type of character appearing, i.e. the probability of the i-th type of character appearing in the target data, where the value of i ranges from [0, M].

[0142] The data compression system provided in this application includes the following calculation process for k(x):

[0143] Convert the value of x to a 32-bit binary integer;

[0144] Count the number of prefix 0s in a 32-bit integer;

[0145] The difference between 31 and the numerical value is taken as the K value corresponding to k(x).

[0146] This application provides a data compression system in which the first calculation module calculates the target information entropy of the target data based on the total number of bytes and the probability of occurrence using the information entropy calculation formula. This calculation is used to: determine the result of the x*k(x) operation based on a shift-add operation; and perform an operation to take the lower (k(x)-1) bits of x to obtain (x-2... k(x) The result of the operation.

[0147] This application also provides a data compression device and a computer-readable storage medium, both of which have the corresponding effects of the data compression method provided in the embodiments of this application. Please refer to... Figure 5 , Figure 5 This is a schematic diagram of the structure of a data compression device provided in an embodiment of this application.

[0148] This application provides a data compression device, including a memory 201 and a processor 202. The memory 201 stores a computer program, and the processor 202 executes the computer program to implement the steps of the data compression method described in any of the above embodiments.

[0149] Please see Figure 6Another data compression device provided in this application embodiment may further include: an input port 203 connected to the processor 202 for transmitting commands input from the outside to the processor 202; a display unit 204 connected to the processor 202 for displaying the processing results of the processor 202 to the outside; and a communication module 205 connected to the processor 202 for enabling communication between the data compression device and the outside. The display unit 204 may be a display panel, a laser scanner, or the like; the communication method used by the communication module 205 includes, but is not limited to, Mobile High Definition Link (HML), Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), wireless connection: Wi-Fi, Bluetooth communication technology, Bluetooth Low Energy communication technology, and IEEE 802.11s-based communication technology.

[0150] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the data compression method described in any of the above embodiments.

[0151] The computer-readable storage media involved in this application include random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage media known in the art.

[0152] For descriptions of relevant parts of the data compression system, device, and computer-readable storage medium provided in the embodiments of this application, please refer to the detailed descriptions of the corresponding parts in the data compression method provided in the embodiments of this application, and they will not be repeated here. Furthermore, parts of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0153] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0154] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data compression method, characterized in that, include: Obtain the target data to be compressed; Calculate the total number of bytes in the target data and the probability of occurrence of each type of character; Determine function The target linear function under the preset approximation conditions; Based on the target linear function, determine the information entropy calculation formula without logarithmic operations; The target information entropy of the target data is calculated using the information entropy calculation formula, based on the total number of bytes and the probability of occurrence. Determine whether the target information entropy is less than a preset value. If the target information entropy is less than the preset value, then compress the target data. If the target information entropy is greater than or equal to the preset value, then do not compress the target data. Wherein, the determining function The target linear function under the pre-defined approximation conditions includes: Determine function The target linear function under the preset approximation conditions; The target linear function includes: ； in, Indicates that it makes certain The value is between of A function of value; The step of determining the information entropy calculation formula without logarithmic operations based on the target linear function includes: The information entropy calculation formula without logarithmic operations is determined based on the target linear function; The information entropy calculation formula includes: ； in, This represents the target information entropy; Indicates the total number of bytes; Indicates the total number of types of the character; Indicates the first The probability of occurrence of the class character, where The range of values ​​for is [0, ... ].

2. The method according to claim 1, characterized in that, The determining function The target linear function under the pre-defined approximation conditions includes: Determine function The target linear function under the preset approximation conditions; The target linear function includes: ； in, Indicates that it makes certain The value is between of A function of value; This indicates the adjustment parameter.

3. The method according to claim 2, characterized in that, Also includes: based on The value of the adjustment parameter is determined by the value of the adjustment parameter.

4. The method according to claim 2, characterized in that, The process of determining the information entropy calculation formula without logarithmic operations based on the target linear function includes: The information entropy calculation formula without logarithmic operations is determined based on the target linear function; The information entropy calculation formula includes: ； in, This represents the target information entropy; Indicates the total number of bytes; Indicates the total number of types of the character; Indicates the first The probability of occurrence of the class character, where The range of values ​​for is [0, ... ].

5. The method according to claim 1 or 4, characterized in that, The calculation process includes: Will The value is converted to a 32-bit binary integer; Count the number of prefix zeros in the given 32-bit integer; The difference between 31 and the aforementioned values ​​is taken as Corresponding value.

6. The method according to claim 1 or 4, characterized in that, The process of calculating the target information entropy of the target data based on the total number of bytes and the probability of occurrence using the information entropy calculation formula includes: Determined based on shift-add operation The result of the calculation; Pick Low Bitwise operations, to obtain The result of the calculation.

7. A data compression system, characterized in that, include: The first acquisition module is used to acquire the target data to be compressed; The first statistics module is used to count the total number of bytes in the target data and the probability of occurrence of each type of character; The first determining module is used to determine the function. The target linear function under the preset approximation conditions; The second determining module is used to determine the information entropy calculation formula without logarithmic operations based on the target linear function; The first calculation module is used to calculate the target information entropy of the target data based on the total number of bytes and the probability of occurrence using the information entropy calculation formula. The first judgment module is used to determine whether the target information entropy is less than a preset value. If the target information entropy is less than the preset value, the target data is compressed. If the target information entropy is greater than or equal to the preset value, the target data is not compressed. The first determining module includes: The first determining unit is used to determine the function. The target linear function under the preset approximation conditions; The target linear function includes: ； in, Indicates that it makes certain The value is between of A function of value; The second determining module includes: The second determining unit is used to determine the information entropy calculation formula without logarithmic operations based on the target linear function; The information entropy calculation formula includes: ； in, This represents the target information entropy; Indicates the total number of bytes; Indicates the total number of types of the character; Indicates the first The probability of occurrence of the class character, where The range of values ​​for is [0, ... ].

8. A data compression device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the data compression method as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data compression method as described in any one of claims 1 to 6.

Images