Data compression method and device, electronic equipment and computer readable storage medium

Abstract

The invention discloses a data compression method and device, electronic equipment and a computer readable storage medium. The method comprises the steps of obtaining to-be-compressed data in a to-be-compressed memory page; compressing the to-be-compressed data based on a first compression algorithm to obtain first compressed data; determining a target compression ratio based on the first compressed data and the to-be-compressed data; if the target compression ratio is smaller than or equal to a preset threshold value, the to-be-compressed data is compressed based on a second compression algorithm to obtain second compressed data, and the compression degree of the second compression algorithm on the data is larger than the compression degree of the first compression algorithm on the data, the data compression rate of the second compression algorithm is smaller than that of the first compression algorithm. Only when the target compression rate is smaller than or equal to the preset threshold value, the to-be-compressed data is compressed based on the second compression algorithm to obtain the second compressed data, and the compression rate can be reduced in a short time.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically, to a data compression method, device, electronic device and computer-readable storage medium. Background Art

[0002] At present, with the development of electronic information technology, the memory required by electronic devices during operation is getting larger and larger. Although, when the available memory of the electronic device is small, some data in the memory can be compressed. However, the compression rate of compressing the data in the memory is currently high or it takes a long time. Summary of the invention

[0003] The present application proposes a data compression method, device, electronic device and computer-readable storage medium.

[0004] In a first aspect, an embodiment of the present application provides a data compression method, which is applied to an electronic device, and the method includes: obtaining data to be compressed in a memory page to be compressed; compressing the data to be compressed based on a first compression algorithm to obtain first compressed data; determining a target compression rate based on the first compressed data and the data to be compressed; if the target compression rate is less than or equal to a preset threshold, compressing the data to be compressed based on a second compression algorithm to obtain second compressed data, wherein the degree of compression of data by the second compression algorithm is greater than the degree of compression of data by the first compression algorithm, and the rate at which data is compressed by the second compression algorithm is less than the rate at which data is compressed by the first compression algorithm.

[0005] In a second aspect, an embodiment of the present application further provides a data compression device, which is applied to an electronic device, and the device includes: a first acquisition unit, used to acquire the data to be compressed in the memory page to be compressed; a first compression unit, used to compress the data to be compressed based on a first compression algorithm to obtain first compressed data; a determination unit, used to determine a target compression rate based on the first compressed data and the data to be compressed; a second compression unit, used to compress the data to be compressed based on a second compression algorithm to obtain second compressed data if the target compression rate is less than or equal to a preset threshold, wherein the degree of compression of the data by the second compression algorithm is greater than the degree of compression of the data by the first compression algorithm, and the rate at which the data is compressed by the second compression algorithm is less than the rate at which the data is compressed by the first compression algorithm.

[0006] In a third aspect, an embodiment of the present application further provides an electronic device comprising: one or more processors; a memory; the one or more processors are used to execute instructions; the memory is used to provide physical addresses; the bus is used to provide data transmission; the one or more processors are used to execute the method described in the first aspect.

[0007] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program code executable by a processor, and when the program code is executed by the processor, the processor executes the method described in the first aspect.

[0008] The present application provides a data compression method, device, electronic device and computer-readable storage medium. The method first obtains the data to be compressed in the memory page to be compressed; then compresses the data to be compressed based on a first compression algorithm to obtain first compressed data; then determines a target compression rate based on the first compressed data and the data to be compressed; if the target compression rate is less than or equal to a preset threshold, compresses the data to be compressed based on a second compression algorithm to obtain second compressed data. Wherein, the compression degree of the data by the second compression algorithm is greater than the compression degree of the data by the first compression algorithm, and the rate at which the data is compressed by the second compression algorithm is less than the rate at which the data is compressed by the first compression algorithm. Although the second compression algorithm can provide a lower compression rate than the first compression algorithm, the rate at which the second compression algorithm compresses the data is lower than that of the first compression algorithm. If the second compression algorithm is directly used to compress all data, it may take more time for some data to obtain a small reduction in compression rate, which will cause the compression to take a long time. If the first compression algorithm is directly used to compress all data, some data may only have a higher compression rate. Therefore, in an embodiment of the present application, when the target compression rate is less than or equal to a preset threshold, it indicates that the data to be compressed is easier to compress. At this time, the data to be compressed is compressed based on a second compression algorithm to obtain second compressed data, which can exchange a smaller increase in compression time for a lower compression rate.

[0009] Other features and advantages of the embodiments of the present application will be described in the subsequent description, and partly become apparent from the description, or can be understood by practicing the embodiments of the present application. The purposes and other advantages of the embodiments of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 An application scenario diagram of the data compression method provided in an embodiment of the present application is shown;

[0012] Figure 2A method flow chart of a data compression method provided by an embodiment of the present application is shown;

[0013] Figure 3 A method flow chart of a data compression method provided by another embodiment of the present application is shown;

[0014] Figure 4 A method flow chart of a data compression method provided by another embodiment of the present application is shown;

[0015] Figure 5 A unit block diagram of a data compression device provided by an embodiment of the present application is shown;

[0016] Figure 6 A schematic diagram of an electronic device provided by an embodiment of the present application is shown;

[0017] Figure 7 A structural block diagram of a computer-readable storage medium provided in an embodiment of the present application is shown;

[0018] Figure 8 A structural block diagram of a computer program product provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] In order to make those skilled in the art better understand the present application scheme, the technical scheme in the present application embodiment will be clearly and completely described below in conjunction with the drawings in the present application embodiment. Obviously, the described embodiment is only a part of the present application embodiment, rather than all the embodiments. The components of the present application embodiment usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application for protection, but merely represents the selected embodiment of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0020] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0021] At present, with the development of electronic information technology, the memory required by electronic devices during operation is getting larger and larger. Although, when the available memory of the electronic device is small, some data in the memory can be compressed. However, the compression rate of compressing the data in the memory is currently high or it takes a long time.

[0022] When the available memory resources of an electronic device are small, some of the data in the memory can be written to the disk, and when this part of the data is needed, it can be read from the disk into the memory again. However, since the speed of the disk is generally slower than the speed of the memory, the process of reading data from the disk to the memory is slow. In addition, some data in the memory can be compressed by a compression algorithm, thereby reducing the memory space occupied. When this data is needed later, it can be decompressed by the decompression algorithm corresponding to the compression algorithm to obtain the uncompressed data again.

[0023] At present, a specific compression algorithm is selected so that the data in the memory can be compressed directly by the compression algorithm. For example, the LZ4 compression algorithm or the ZSTD compression algorithm, the LZO compression algorithm or the DEFLATE compression algorithm can be pre-set. Among them, the LZ4 compression algorithm is currently a more commonly used compression algorithm. The algorithm has the advantages of fast compression speed and decompression speed and simple compression format. However, compared with the LZO compression algorithm, the DEFLATE compression algorithm or the ZSTD compression algorithm, the compression rate of the LZ4 compression algorithm is relatively high. The ZSTD compression algorithm provides better compression speed, decompression speed and compression rate, and can provide a lower compression rate at a smaller loss of compression speed and decompression speed.

[0024] However, the inventors found in their research that the compressibility is different for different data. For some data, even if it takes a long time, the degree of compression rate reduction is small. At this time, if an algorithm with a high compression degree is used to compress the data, it may take a long time to compress, and the compression rate will not be reduced too much. If an algorithm with a low compression degree is used, although a faster compression rate can be achieved, the compression rate for some data will be higher.

[0025] Therefore, in order to overcome the above-mentioned defects, the present application provides a data compression method, device, electronic device and computer-readable storage medium.

[0026] See also Figure 1 , Figure 1 An application scenario diagram of a data compression method provided in an embodiment of the present application is shown, namely, a data compression scenario 100. The data compression scenario 100 includes an electronic device 110 and a user 120.

[0027] Among them, the electronic device 110 can run various applications to realize the functions corresponding to different applications. In the process of using the electronic device 110, the user 120 may open multiple different applications at the same time. Opening multiple applications at the same time will consume more memory of the electronic device 110, so that the available memory of the electronic device 110 will be reduced. At this time, the electronic device 110 can compress some data in the memory through a data compression algorithm, thereby reducing the space occupied by the compressed data, thereby increasing the available memory and improving the running smoothness of the electronic device 110. Among them, opening multiple different applications at the same time can mean keeping any one application running in the foreground, while having multiple applications running in the background. For an introduction to the specific data compression algorithm, please refer to the subsequent embodiments.

[0028] See also Figure 2 , Figure 2 A data compression method provided by an embodiment of the present application is shown, and the data compression method can be applied to Figure 1 In the electronic device 110, the processor of the electronic device 110 can be used as the execution subject. Specifically, the method includes steps S110 to S140.

[0029] Step S110: obtaining the data to be compressed in the memory page to be compressed.

[0030] When it is necessary to compress the data in the memory of the electronic device, different compression algorithms can be selected. For example, the data can be compressed by a first compression algorithm, wherein the first compression algorithm can have a faster compression rate and a not too high compression degree of the data, so that the data can be compressed by the first compression algorithm, and a not too low compression rate of the data can be achieved in a short time. For another example, the data can also be compressed by a second compression algorithm, wherein the second compression algorithm can have a lower compression rate and a lower rate of compressing the data, so that the data can be compressed by the second compression algorithm, and a lower compression rate of the data can be achieved in a longer time.

[0031] However, for some data, if the compression rate after the data is compressed by the first compression algorithm is small, it can be characterized that the data is relatively easy to compress. In this case, if the data is compressed by the second compression algorithm, the compression rate can be reduced to a greater extent at the cost of increasing the compression time. If the data is compressed by the first compression algorithm with a large compression rate, then compressing the data by the second compression algorithm will increase the compression time and reduce the compression rate to a lesser extent.

[0032] It should be noted that the compression rate obtained by compressing a certain data through a compression algorithm is used to characterize the degree of compression of the data after compression. The smaller the compression rate, the greater the degree of compression of the data by the compression algorithm; the larger the compression rate, the less the degree of compression of the data by the compression algorithm.

[0033] Therefore, in order to determine whether to use the first compression algorithm or the second compression algorithm for the data, the data to be compressed in the memory page to be compressed can be first obtained. The memory of the electronic device can include multiple memory pages, and each memory page stores data.

[0034] An exemplary method is to use a memory page in the memory of the electronic device that has not been used in the recent period as the memory page to be compressed, so that the data in the memory page to be compressed can be obtained as the data to be compressed.

[0035] Another exemplary method is to use the memory page corresponding to the application that occupies a larger memory space in the memory of the electronic device as the memory page to be compressed, so that the data in the memory page to be compressed can be obtained as the data to be compressed.

[0036] In another exemplary embodiment, the zRAM technology can be used to obtain a memory page as the memory page to be compressed, so that the data in the memory page to be compressed can be obtained as the data to be compressed.

[0037] It should be noted that there may be data in multiple memory pages in the memory of the electronic device that need to be compressed, so the data in multiple memory pages can be identified as data to be compressed in turn and the subsequent steps can be repeated until the data in each memory page is compressed. The subsequent steps of this application are specifically described with only the data to be compressed in one memory page.

[0038] Step S120: compressing the data to be compressed based on a first compression algorithm to obtain first compressed data.

[0039] From the above introduction, it can be known that the compression rate of the data to be compressed by the first compression algorithm can be used to determine whether the data to be compressed needs to be compressed by the second compression algorithm. Therefore, after obtaining the data to be compressed, the data to be compressed can be first compressed by the first compression algorithm to obtain the first compressed data.

[0040] In an exemplary embodiment, compressing the data to be compressed by the first compression algorithm may be based on a dictionary compression algorithm to achieve compression of the data to be compressed, thereby obtaining the first compressed data. For details, please refer to the introduction in the subsequent embodiments.

[0041] Step S130: determining a target compression ratio based on the first compressed data and the data to be compressed.

[0042] After the first compressed data is obtained, the target compression rate of the data to be compressed after the first compression algorithm compresses the data to be compressed can be determined. The target compression rate can be determined based on the first compressed data and the data to be compressed, and the target compression rate is used to characterize the degree to which the data to be compressed is compressed. In other words, the smaller the target compression rate, the greater the degree to which the first compressed data is compressed; the larger the target compression rate, the smaller the degree to which the first compressed data is compressed. For the specific method of calculating the target compression rate, please refer to the introduction in the subsequent embodiments.

[0043] Step S140: If the target compression rate is less than or equal to a preset threshold, the data to be compressed is compressed based on a second compression algorithm to obtain second compressed data, wherein the degree of data compression by the second compression algorithm is greater than the degree of data compression by the first compression algorithm, and the rate at which the second compression algorithm compresses data is less than the rate at which the first compression algorithm compresses data.

[0044] After obtaining the target compression rate, the relationship between the target compression rate and the preset threshold can be determined. Only when the target compression rate is less than or equal to the preset threshold, the data to be compressed is compressed based on the second compression algorithm to obtain the second compressed data. Among them, since the degree of data compression by the second compression algorithm is greater than the degree of data compression by the first compression algorithm, the rate at which the second compression algorithm compresses data is less than the rate at which the first compression algorithm compresses data. Since when the target compression rate is less than or equal to the preset threshold, it can be characterized that the data to be compressed is relatively easy to be compressed, then at this time, compressing the data to be compressed by the second compression algorithm can increase the compression time slightly in exchange for a greater degree of compression rate reduction.

[0045] If the target compression ratio is greater than the preset threshold, it can be indicated that the data to be compressed is not easy to be compressed. At this time, if the data to be compressed is still compressed by the second compression algorithm, the cost of increasing compression time will be exchanged for a smaller degree of compression ratio reduction. Therefore, in some embodiments, when the target compression ratio is greater than the preset threshold, the data to be compressed may not be compressed by the second compression algorithm.

[0046] The preset threshold value may be a pre-set value, which may be a critical value obtained through multiple tests. The critical value is used to indicate that when the target compression rate is greater than the critical value, using the second compression algorithm to compress the data to be compressed will cause a significant increase in compression time and a smaller effect on the target compression rate; and if the target compression rate is less than or equal to the critical value, using the second compression algorithm to compress the data to be compressed can increase the compression time in exchange for a larger reduction in the target compression rate.

[0047] The second compression algorithm is used to compress the data to be processed, which may be firstly compressed by a dictionary compression algorithm, and then the compressed data to be compressed by the dictionary compression algorithm is encoded by a specified encoding method, thereby obtaining the second compressed data. For a detailed description, please refer to the subsequent embodiments.

[0048] The present application provides a data compression method, which first obtains the data to be compressed in the memory page to be compressed; then compresses the data to be compressed based on a first compression algorithm to obtain first compressed data; then determines a target compression rate based on the first compressed data and the data to be compressed; if the target compression rate is less than or equal to a preset threshold, compresses the data to be compressed based on a second compression algorithm to obtain second compressed data. Wherein, the compression degree of the data by the second compression algorithm is greater than the compression degree of the data by the first compression algorithm, and the rate at which the data is compressed by the second compression algorithm is less than the rate at which the data is compressed by the first compression algorithm. Although the second compression algorithm can provide a lower compression rate than the first compression algorithm, the rate at which the second compression algorithm compresses the data is lower than that of the first compression algorithm. If the second compression algorithm is directly used to compress all data, it may take more time for some data to obtain a small reduction in compression rate, which will cause the compression to take a long time. If the first compression algorithm is directly used to compress all data, some data may only have a higher compression rate. Therefore, in an embodiment of the present application, when the target compression rate is less than or equal to a preset threshold, it indicates that the data to be compressed is easier to compress. At this time, the data to be compressed is compressed based on a second compression algorithm to obtain second compressed data, which can exchange a smaller increase in compression time for a lower compression rate.

[0049] See also Figure 3 , Figure 3 A data compression method provided by an embodiment of the present application is shown, and the data compression method can be applied to Figure 1 In the electronic device 110, the processor of the electronic device 110 can be used as the execution subject. Specifically, the method includes steps S210 to S290.

[0050] Step S210: Obtain the data to be compressed in the memory page to be compressed.

[0051] Among them, step S210 has been introduced in detail in the above embodiment and will not be repeated here.

[0052] Step S220: compress the data to be compressed based on a first compression algorithm to obtain first compressed data.

[0053] In some implementations, the first compression algorithm may be an LZ4 compression algorithm. Therefore, initialization may be performed first, for example, the initialization may include setting a scanning interval and a preset length during compression, and allocating a space in the memory of the electronic device for storing a hash table.

[0054] Sequentially obtain data of a preset length from the data to be compressed as fifth intermediate data, and then obtain a first Hash value corresponding to the fifth intermediate data; and search the hash table for a hash value that is the same as the first Hash value.

[0055] If it does not exist, the first hash value and the fifth intermediate data corresponding to the first hash value are stored in the hash table, and the first address corresponding to the fifth intermediate data is offset by a scanning interval to obtain a second address, and the data corresponding to the second address in the data to be compressed is determined as the new fifth intermediate data, and the process of obtaining the first hash value corresponding to the fifth intermediate data and subsequent steps is returned until a hash value identical to the first hash value exists.

[0056] If it exists, the fifth intermediate data is compressed by a dictionary compression algorithm to obtain the sixth intermediate data, and the first address corresponding to the fifth intermediate data is offset by a scanning interval to obtain a second address, and the data corresponding to the second address in the data to be compressed is determined as the new fifth intermediate data, and the process returns to execute the first hash value corresponding to the fifth intermediate data and subsequent steps until all the data in the data to be compressed are traversed.

[0057] The dictionary file required for the dictionary compression algorithm may be the hash table. Finally, the compressed data and the uncompressed data, ie, the sixth intermediate data and the uncompressed data, are used together to determine the first compressed data.

[0058] As an exemplary example, if the data to be compressed is a string "dfabcdefghijklmnabcdkkkkkk", the scanning interval is preset to 1, and the preset length is 4, then four bytes "dfab" are read as the fifth intermediate data, and the first hash value of "dfab" is calculated, and a hash value matching the first hash value is not found in the hash table, so the first hash value and the fifth intermediate data corresponding to the first hash value, namely "dfab", are placed in the hash table.

[0059] The first address corresponding to the fifth intermediate data is offset by a scanning interval, that is, offset 1, to obtain a second address and determine the data corresponding to the second address in the data to be compressed, that is, the new fifth intermediate data is "fabc", and the first hash value of "fabc" is calculated. A hash value matching the first hash value is not found in the hash table, and the first hash value and the fifth intermediate data corresponding to the first hash value, that is, "fabc", are placed in the hash table.

[0060] Repeat the above steps until the fifth intermediate data is confirmed to be "abcd" again. At this time, a hash value identical to the first hash value of the fifth intermediate data can be found in the hash table. The fifth intermediate data is then compressed using a dictionary compression algorithm until all data in the data to be compressed are traversed.

[0061] Step S230: taking a ratio of the data size of the first compressed data to the data size of the to-be-compressed data as first intermediate data.

[0062] Step S240: Obtain the target compression rate based on the first intermediate data.

[0063] As can be seen from the above introduction, the target compression ratio is used to characterize the degree to which the data to be compressed is compressed. Therefore, in some embodiments, the ratio of the data size of the first compressed data to the data size of the data to be compressed can be obtained as the first intermediate data. In an exemplary embodiment, if the data size of the first compressed data is A and the data size of the data to be compressed is B, then A / B can be used as the first intermediate data. For example, if A is half of B, then the first intermediate data can be determined to be 0.5.

[0064] Furthermore, the target compression rate can also be represented by a percentage. Therefore, based on the first intermediate data, the target compression rate can be obtained by multiplying the first intermediate data by 100%. Please continue to refer to the above example. If the first intermediate data is 0.5, the target compression rate can be 0.5x100%=50%.

[0065] Step S250: Whether the target compression ratio is less than or equal to a preset threshold.

[0066] Furthermore, it can be determined whether the target compression ratio is less than or equal to a preset threshold. If it is determined that the target compression ratio is less than or equal to the preset threshold, the process can jump to step S260; if it is determined that the target compression ratio is not less than or equal to the preset threshold, the process can jump to step S290.

[0067] Step S260: If the target compression rate is less than or equal to a preset threshold, the data to be compressed is compressed by a dictionary compression algorithm based on a pre-acquired dictionary file to obtain second intermediate data, wherein the dictionary file includes at least one standard data.

[0068] In some embodiments, the second compression algorithm may be a ZSTD compression algorithm. Therefore, if the target compression rate is less than or equal to a preset threshold, the data to be compressed may be first compressed by a dictionary compression algorithm to obtain second intermediate data. The dictionary compression algorithm may be to compress the data to be compressed by a dictionary file. In an exemplary manner, the dictionary file may be a pre-acquired general dictionary file, or may be obtained by training based on a portion of the data to be compressed, which is not specifically limited in the embodiments of the present application.

[0069] Specifically, when executing step S260, steps S261 to S266 may be included.

[0070] Step S261: If the target compression rate is less than or equal to a preset threshold, the data to be compressed and the data matching the standard data in the dictionary file are determined as compressible data.

[0071] Step S262: Obtain the length of the compressible data as a matching length.

[0072] Step S263: Obtain a numerical value of the offset of the first character of the compressible data relative to the first character of the data to be compressed as an offset.

[0073] Step S264: Determine the data to be compressed that does not match the standard data in the dictionary file as incompressible data.

[0074] Step S265: Determine the length of the incompressible data as the incompressible length.

[0075] Step S266: Use the matching length, offset, incompressible data and incompressible length as the second intermediate data.

[0076] In some embodiments, the dictionary file includes at least one standard data, so that the data to be compressed can be traversed to find data in the data to be compressed that can match the standard data in the dictionary file as compressible data. It is easy to understand that the data in the data to be compressed that can match the standard data in the dictionary file can be used to characterize the compressible data, thereby reducing the size of the space occupied by the compressible data, thereby achieving compression of the data to be compressed.

[0077] Furthermore, the length of the compressible data can be obtained as the match length (matchLength). In an exemplary embodiment, the number of characters occupied by the compressible data can be used as the match length. For example, if the compressible data is "abcd", the number of characters occupied by the compressible data is 4, and the match length can be determined to be 4.

[0078] Furthermore, the value of the offset of the first character of the compressible data relative to the first character of the data to be compressed can be obtained as an offset. For example, if the data to be compressed is "adedgdabcd", the pointer of the first character of the data to be compressed is 0, and if the compressible data is "abcd", the pointer of the first character of the compressible data is 6, so it can be determined that the offset is 6-0=6.

[0079] It is also possible to determine that the data to be compressed does not match the standard data in the dictionary file as incompressible data (literals). In an exemplary embodiment, the compressible data in the data to be processed can be excluded, and the remaining data is incompressible data. In another exemplary embodiment, the data in the data to be compressed that does not match the standard data can also be directly determined through the dictionary file as incompressible data.

[0080] Furthermore, the length of the incompressible data can be determined as the incompressible length (literalLength). Similar to the determination of the matching length, the number of characters occupied by the incompressible data can be used as the incompressible length. For example, if the incompressible data is "adedgd", the number of characters occupied by the incompressible data is 6, and the incompressible length can be determined to be 6.

[0081] Therefore, the matching length, offset, incompressible data and incompressible length can be used as the second intermediate data.

[0082] Step S270: Encode the second intermediate data to obtain the second compressed data.

[0083] Furthermore, in order to further reduce the compression rate of the second compressed data, the obtained second intermediate data may be further encoded to obtain the second compressed data.

[0084] In an exemplary embodiment, the second intermediate data may be encoded by a preset encoding method to obtain the second compressed data. For example, the preset encoding method may be Huffman encoding, and the second intermediate data may be encoded by Huffman encoding to obtain the second compressed data.

[0085] For another example, the preset encoding method may be finite state entropy coding, and the second intermediate data may be encoded by finite state entropy coding.

[0086] In some other implementations, the incompressible data may be Huffman encoded to obtain third intermediate data, and the matching length, the offset, and the incompressible length may be finite state entropy encoded to obtain fourth intermediate data. Thus, the second compressed data may be determined based on the third intermediate data and the fourth intermediate data.

[0087] Step S280: storing the second compressed data in the memory page to be compressed to update the memory page to be compressed.

[0088] After the second compressed data is acquired, the second compressed data can be stored in the memory page to be compressed to update the memory page to be compressed. When the second compressed data in the memory page to be compressed needs to be read later, the second compressed data can be decoded.

[0089] Step S290: If the target compression ratio is greater than a preset threshold, the first compressed data is stored in the memory page to be compressed to update the memory page to be compressed.

[0090] Among them, when it is determined that the target compression rate is not less than or equal to the preset threshold, the execution of step S290 can be jumped. At this time, since the target compression rate is not less than or equal to the preset threshold, it can be regarded as the target compression rate is greater than the preset threshold, indicating that the data to be compressed is not easy to be compressed. Therefore, even if the data to be compressed is compressed by the second compression algorithm, the compression time will be increased, and the reduction in the compression rate obtained in exchange is small. Therefore, at this time, the first compressed data obtained after compressing the data to be compressed by the first compression algorithm can be directly stored in the memory page to be compressed, so that a smaller degree of compression rate improvement can be exchanged for a greater degree of compression time reduction.

[0091] The present application provides a data compression method, which first obtains the data to be compressed in the memory page to be compressed; then compresses the data to be compressed based on a first compression algorithm to obtain first compressed data; uses the ratio of the data size of the first compressed data to the data size of the data to be compressed as the first intermediate data; obtains the target compression rate based on the first intermediate data; if the target compression rate is less than or equal to a preset threshold, compresses the data to be compressed based on a second compression algorithm to obtain second compressed data, and then stores the second compressed data in the memory page to be compressed to update the memory page to be compressed; if the target compression rate is greater than the preset threshold, stores the first compressed data in the memory page to be compressed to update the memory page to be compressed. Therefore, in the implementation mode provided by the present application, when it is determined that the target compression rate is not less than or equal to the preset threshold, the first compressed data obtained by compressing the data to be compressed by the first compression algorithm is directly stored in the memory page to be compressed, which can exchange a smaller degree of compression rate improvement for a larger degree of compression time reduction. When it is determined that the target compression rate is less than or equal to the preset threshold, the data to be compressed can be compressed by a second compression algorithm to obtain second compressed data, and the second compressed data is stored in the memory page to be compressed, thereby reducing the compression rate to a greater extent at the expense of a smaller increase in compression time.

[0092] See also Figure 4 , Figure 4 A data compression method provided by an embodiment of the present application is shown, and the data compression method can be applied to Figure 1 In the electronic device 110, the processor of the electronic device 110 can be used as the execution subject. Specifically, the method includes steps S310 to S3120.

[0093] Step S310: Obtain the data to be compressed in the memory page to be compressed.

[0094] Data in the memory page to be compressed may be obtained as the data to be compressed, wherein the memory page to be compressed may include at least one memory page, and the specific introduction may refer to the above embodiment, which will not be repeated here.

[0095] Step S320: Initialization.

[0096] For some embodiments, the first compression algorithm may be an LZ4 compression algorithm, so that before compressing the data to be compressed based on the first compression algorithm, initialization may be performed first. For example, the initialization may specifically include setting a scanning interval and a preset length during the compression process, and allocating a space in the memory of the electronic device for storing a hash table. Optionally, after obtaining the hash table, a clearing operation may be performed on the hash table to ensure that other data is not included in the hash table used subsequently.

[0097] Step S330: compressing the data to be compressed using a first compression algorithm.

[0098] The data to be compressed may be compressed by a first compression algorithm, and in an exemplary manner, the data to be compressed may be compressed by a dictionary compression algorithm. For a specific introduction, please refer to the above embodiment, which will not be described in detail here.

[0099] Step S340: Check whether the data to be compressed has been compressed.

[0100] From the above introduction, it can be known that the data to be compressed needs to be traversed through the dictionary compression algorithm. Therefore, it can be determined whether the data to be compressed has been compressed. If it is determined that the compression has not been completed, it can return to step S330 to continue compression; if it is determined that the compression has been completed, it can jump to step S350.

[0101] Step S350: determining a target compression ratio based on the first compressed data and the data to be compressed.

[0102] After the data to be compressed is compressed by the first compression algorithm, the compressed data can be used as the first compressed data, and further, the target compression rate can be determined based on the first compressed data and the data to be compressed. The specific method for determining the target compression rate can refer to the introduction of the above embodiment, which will not be repeated here.

[0103] Step S360: Whether the target compression ratio is less than or equal to a preset threshold.

[0104] As can be seen from the above introduction, the target compression rate can be used to characterize the degree to which the data to be compressed is compressed. When the target compression rate is less than or equal to the preset threshold, it can be characterized that the data to be compressed is relatively easy to be compressed. At this time, the second compression algorithm is used to compress the data to be compressed, which can increase the compression time slightly in exchange for a greater degree of compression rate reduction. When the target compression rate is greater than the preset threshold, it can be characterized that the data to be compressed is relatively difficult to be compressed. At this time, the second compression algorithm is used to compress the data to be compressed, which will increase the compression time and can only reduce the compression rate to a lesser extent. Therefore, if the target compression rate is less than or equal to the preset threshold, the execution can be jumped to step S380; if the target compression rate is greater than the preset threshold, the execution can be jumped to step S370.

[0105] Step S370: storing the first compressed data in the memory page to be compressed.

[0106] When the target compression rate is greater than the preset threshold, even if the data to be compressed is compressed by the second compression algorithm, the compression time will be increased, and the compression rate will decrease slightly. Therefore, at this time, the first compressed data obtained by compressing the data to be compressed by the first compression algorithm can be directly stored in the memory page to be compressed, so that a small degree of compression rate improvement can be exchanged for a large degree of compression time reduction.

[0107] Step S380: Obtain the data to be compressed in the memory page to be compressed.

[0108] When the target compression rate is less than or equal to the preset threshold, the data to be compressed can be further compressed by a second compression algorithm. The data to be compressed in the memory page to be compressed can be first obtained. It should be noted that the data to be compressed obtained at this time is essentially the same as the data to be compressed compressed by the first compression algorithm.

[0109] Step S390: Initialization.

[0110] For some implementations, the second compression algorithm may be a ZSTD compression algorithm, so that before compressing the data to be compressed based on the second compression algorithm, initialization may be performed first. For example, the initialization may specifically include obtaining a dictionary file for performing a dictionary compression algorithm.

[0111] Step S3100: compress the data to be compressed using a second compression algorithm.

[0112] The data to be compressed can be compressed by a second compression algorithm. For example, the data to be compressed can be compressed by a dictionary compression algorithm to obtain second intermediate data, and then the second intermediate data is encoded to obtain the second compressed data. For a specific introduction, please refer to the above embodiment, which will not be repeated here.

[0113] Step S3110: Check whether the data to be compressed has been compressed.

[0114] From the above introduction, it can be known that the data to be compressed needs to be traversed through the dictionary compression algorithm. Therefore, it can be determined whether the data to be compressed has been compressed. If it is determined that the compression has not been completed, it can return to step S3100 to continue compression; if it is determined that the compression has been completed, it can jump to step S3120.

[0115] Step S3120: storing the second compressed data in the memory page to be compressed.

[0116] At this time, the second compressed data obtained by compressing the data to be compressed by the second compression algorithm can be stored in the memory page to be compressed, so that the compression time can be reduced to a greater extent in exchange for a smaller degree of compression rate improvement.

[0117] See also Figure 5 , which shows a structural block diagram of a data compression device 500 provided in an embodiment of the present application, which is applied to electronic equipment. The device includes: a first acquisition unit 510, a first compression unit 520, a determination unit 530 and a second compression unit 540.

[0118] The first acquisition unit 510 is used to acquire the data to be compressed in the memory page to be compressed.

[0119] The first compression unit 520 is used to compress the data to be compressed based on a first compression algorithm to obtain first compressed data.

[0120] The determination unit 530 is configured to determine a target compression ratio based on the first compressed data and the data to be compressed.

[0121] Optionally, the determination unit 530 may also be configured to use a ratio of a data size of the first compressed data to a data size of the data to be compressed as first intermediate data; and obtain the target compression ratio based on the first intermediate data.

[0122] The second compression unit 540 is configured to compress the data to be compressed based on a second compression algorithm to obtain second compressed data if the target compression rate is less than or equal to a preset threshold, wherein the degree of compression of the data by the second compression algorithm is greater than the degree of compression of the data by the first compression algorithm, and the rate of compressing the data by the second compression algorithm is less than the rate of compressing the data by the first compression algorithm. The first compression algorithm includes an LZ4 compression algorithm, and the second compression algorithm includes a ZSTD algorithm.

[0123] Optionally, the second compression unit 540 can also be used to compress the data to be compressed by a dictionary compression algorithm based on a pre-acquired dictionary file to obtain second intermediate data if the target compression rate is less than or equal to a preset threshold, wherein the dictionary file includes at least one standard data; and encode the second intermediate data to obtain the second compressed data.

[0124] Optionally, the second compression unit 540 can also be used to, if the target compression rate is less than or equal to a preset threshold, determine the data to be compressed that matches the standard data in the dictionary file as compressible data; obtain the length of the compressible data as the matching length; obtain the numerical value of the offset of the first character of the compressible data relative to the first character of the data to be compressed as an offset; determine the data that does not match the standard data in the dictionary file as incompressible data; determine the length of the incompressible data as the incompressible length; and use the matching length, offset, incompressible data and incompressible length as the second intermediate data.

[0125] Optionally, the second compression unit 540 can also be used to perform Huffman encoding on the incompressible data to obtain third intermediate data; perform finite state entropy encoding on the matching length, the offset and the incompressible length respectively to obtain fourth intermediate data; and determine the second compressed data based on the third intermediate data and the fourth intermediate data.

[0126] The data compression device 500 may further include a first storage unit, and the first storage unit is used to store the second compressed data in the memory page to be compressed, so as to update the memory page to be compressed.

[0127] The data compression device 500 may further include a second storage unit, and the second storage unit is used to store the first compressed data in the memory page to be compressed if the target compression rate is greater than a preset threshold, so as to update the memory page to be compressed.

[0128] Please refer to Figure 6, which shows a structural block diagram of an electronic device 110 provided in an embodiment of the present application. The electronic device 110 may be an electronic device capable of running applications, such as a smart phone, a tablet computer, an e-book, etc. The electronic device 110 in the present application may include one or more of the following components: a processor 111, a memory 112, and one or more applications, wherein the one or more applications may be stored in the memory 112 and configured to be executed by one or more processors 111, and the one or more programs are configured to execute the method described in the aforementioned method embodiment.

[0129] The processor 111 may include one or more processing cores. The processor 111 uses various interfaces and lines to connect various parts of the entire electronic device 110, and executes various functions and processes data of the electronic device 110 by running or executing instructions, programs, code sets or instruction sets stored in the memory 112, and calling data stored in the memory 112. Optionally, the processor 111 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 111 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 111, but may be implemented separately through a communication chip.

[0130] The memory 112 may include a random access memory (RAM) or a read-only memory (ROM). The memory 112 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 112 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device 110 during use.

[0131] Please refer to Figure 7, which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 700 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0132] The computer readable storage medium 700 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 700 includes a non-transitory computer-readable storage medium. The computer readable storage medium 700 has storage space for program code 710 that performs any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 710 can be compressed, for example, in an appropriate form.

[0133] Please refer to Figure 8 , which shows a structural block diagram 800 of a computer program product provided by an embodiment of the present application. The computer program product 800 includes a computer program / instruction 810, and when the computer program / instruction 810 is executed by a processor, the steps of the above method are implemented.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data compression method, characterized in that: Applied to electronic equipment, the method comprises: Get the data to be compressed in the memory page to be compressed; Compressing the data to be compressed based on a first compression algorithm to obtain first compressed data; determining a target compression ratio based on the first compressed data and the data to be compressed; If the target compression rate is less than or equal to a preset threshold, the data to be compressed is compressed based on a second compression algorithm to obtain second compressed data, wherein the degree of data compression by the second compression algorithm is greater than that by the first compression algorithm, and the rate at which data is compressed by the second compression algorithm is less than the rate at which data is compressed by the first compression algorithm.

2. The method according to claim 1, characterized in that: The determining a target compression ratio based on the first compressed data and the data to be compressed includes: taking a ratio of a data size of the first compressed data to a data size of the data to be compressed as first intermediate data; The target compression ratio is obtained based on the first intermediate data.

3. The method according to claim 1, characterized in that If the target compression rate is less than or equal to a preset threshold, compressing the data to be compressed based on a second compression algorithm to obtain second compressed data includes: If the target compression rate is less than or equal to a preset threshold, compressing the data to be compressed by a dictionary compression algorithm based on a pre-acquired dictionary file to obtain second intermediate data, wherein the dictionary file includes at least one standard data; The second intermediate data is encoded to obtain the second compressed data.

4. The method according to claim 3, characterized in that The second intermediate data includes a matching length, an offset, incompressible data, and an incompressible length of the incompressible data. If the target compression rate is less than or equal to a preset threshold, the data to be compressed is compressed by a dictionary compression algorithm based on a pre-acquired dictionary file to obtain the second intermediate data, wherein the dictionary file includes at least one standard data, including: If the target compression rate is less than or equal to a preset threshold, determining the data to be compressed and the data matching the standard data in the dictionary file as compressible data; Obtaining the length of the compressible data as a matching length; Obtaining a numerical value of an offset of the first character of the compressible data relative to the first character of the data to be compressed as an offset; Determine the data to be compressed that does not match the standard data in the dictionary file as incompressible data; Determining the length of the incompressible data as the incompressible length; The matching length, offset, incompressible data and incompressible length are used as the second intermediate data.

5. The method according to claim 4, characterized in that The step of encoding the second intermediate data to obtain the second compressed data includes: Performing Huffman coding on the incompressible data to obtain third intermediate data; Performing finite state entropy coding on the matching length, the offset, and the incompressible length respectively to obtain fourth intermediate data; The second compressed data is determined based on the third intermediate data and the fourth intermediate data.

6. The method according to claim 1, characterized in that If the target compression rate is less than or equal to a preset threshold, the to-be-compressed data is compressed based on a second compression algorithm to obtain second compressed data, further comprising: The second compressed data is stored in the memory page to be compressed, so as to update the memory page to be compressed.

7. The method according to claim 1, characterized in that The method further comprises: If the target compression ratio is greater than a preset threshold, the first compressed data is stored in the memory page to be compressed to update the memory page to be compressed.

8. The method according to claim 1, characterized in that The first compression algorithm includes an LZ4 compression algorithm, and the second compression algorithm includes a ZSTD algorithm.

9. A data compression device, characterized in that: Applied to electronic equipment, the device comprises: A first acquisition unit, used for acquiring the data to be compressed in the memory page to be compressed; A first compression unit, used for compressing the data to be compressed based on a first compression algorithm to obtain first compressed data; a determining unit, configured to determine a target compression ratio based on the first compressed data and the data to be compressed; A second compression unit is used to compress the data to be compressed based on a second compression algorithm to obtain second compressed data if the target compression rate is less than or equal to a preset threshold, wherein the degree of compression of the data by the second compression algorithm is greater than the degree of compression of the data by the first compression algorithm, and the rate at which the second compression algorithm compresses the data is less than the rate at which the first compression algorithm compresses the data.

10. An electronic device, characterized in that: include: one or more processors; Memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 8.

Images