Data compression method, device, electronic device and computer-readable storage medium

Abstract

This application discloses a data compression method, apparatus, electronic device, and computer-readable storage medium. The method includes: acquiring data to be compressed from a memory page to be compressed; compressing the data to be compressed based on a first compression algorithm to obtain first compressed data; determining a target compression ratio based on the first compressed data and the data to be compressed; if the target compression ratio is less than or equal to a preset threshold, compressing the data to be compressed based on a second compression algorithm to obtain second compressed data. The second compression algorithm compresses the data to a greater degree than the first compression algorithm, and the compression rate of the second compression algorithm is less than the compression rate of the first compression algorithm. This application only compresses the data to be compressed based on the second compression algorithm to obtain second compressed data when the target compression ratio is less than or equal to the preset threshold, thus achieving a reduction in compression ratio with a smaller time investment.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a data compression method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Currently, with the development of electronic information technology, electronic devices require increasingly larger amounts of memory to operate. While some data in memory can be compressed when available memory is limited, current methods either achieve high compression rates or are time-consuming. Summary of the Invention

[0003] This application discloses a data compression method, apparatus, electronic device, and computer-readable storage medium.

[0004] In a first aspect, embodiments of this application provide a data compression method applied to an electronic device. The method includes: acquiring data to be compressed in a memory page to be compressed; compressing the data to be compressed based on a first compression algorithm to obtain first compressed data; determining a target compression ratio based on the first compressed data and the data to be compressed; if the target compression ratio is less than or equal to a preset threshold, compressing the data to be compressed based on a second compression algorithm to obtain second compressed data, wherein the compression degree of the second compression algorithm is greater than the compression degree of the first compression algorithm, and the data compression rate of the second compression algorithm is less than the data compression rate of the first compression algorithm.

[0005] Secondly, embodiments of this application also provide a data compression apparatus applied to an electronic device. The apparatus includes: a first acquisition unit for acquiring data to be compressed in a memory page to be compressed; a first compression unit for compressing the data to be compressed based on a first compression algorithm to obtain first compressed data; a determination unit for determining a target compression ratio based on the first compressed data and the data to be compressed; and a second compression unit for compressing the data to be compressed based on a second compression algorithm if the target compression ratio is less than or equal to a preset threshold to obtain second compressed data, wherein the compression degree of the data by the second compression algorithm is greater than the compression degree of the data by the first compression algorithm, and the data compression rate of the second compression algorithm is less than the data compression rate of the first compression algorithm.

[0006] Thirdly, embodiments of this application also provide an electronic device, including: one or more processors; a memory; the one or more processors for executing instructions; the memory for providing physical addresses; the bus for providing data transmission; and the one or more processors for performing the method described in the first aspect.

[0007] Fourthly, embodiments of this application also provide a computer-readable storage medium storing processor-executable program code, which, when executed by the processor, causes the processor to perform the method described in the first aspect.

[0008] This application provides a data compression method, apparatus, electronic device, and computer-readable storage medium. The method first acquires data to be compressed from a memory page to be compressed; then, it compresses the data to be compressed based on a first compression algorithm to obtain first compressed data; next, it determines a target compression ratio based on the first compressed data and the data to be compressed; if the target compression ratio is less than or equal to a preset threshold, it compresses the data to be compressed based on a second compression algorithm to obtain second compressed data. The second compression algorithm compresses the data to a greater degree than the first compression algorithm, and the compression rate of the second compression algorithm is less than the compression rate of the first compression algorithm. Although the second compression algorithm can provide a smaller compression ratio than the first compression algorithm, its compression rate is also lower. If the second compression algorithm is used directly to compress all data, some data may require a significant amount of time to achieve only a small reduction in compression ratio, resulting in a long compression time. Conversely, if the first compression algorithm is used directly to compress all data, some data may only achieve a high compression ratio. Therefore, in this embodiment of the application, when the target compression ratio is less than or equal to a preset threshold, it indicates that the data to be compressed is relatively easy to compress. At this time, the data to be compressed is compressed based on the second compression algorithm to obtain the second compressed data, which can achieve a lower compression ratio with a smaller compression time.

[0009] Other features and advantages of the embodiments of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the embodiments of this application. The objects and other advantages of the embodiments of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 The diagram illustrates an application scenario of the data compression method provided in this application embodiment;

[0012] Figure 2A flowchart of the data compression method provided in an embodiment of this application is shown;

[0013] Figure 3 A flowchart of a data compression method according to another embodiment of this application is shown;

[0014] Figure 4 A flowchart of a data compression method according to another embodiment of this application is shown;

[0015] Figure 5 A unit block diagram of the data compression apparatus provided in an embodiment of this application is shown;

[0016] Figure 6 A schematic diagram of an electronic device provided in an embodiment of this application is shown;

[0017] Figure 7 This paper shows a structural block diagram of a computer-readable storage medium provided in an embodiment of this application;

[0018] Figure 8 A structural block diagram of a computer program product provided in an embodiment of this application is shown. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of them. The components of the embodiments of the present application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without inventive effort are within the scope of protection of the present application.

[0020] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0021] Currently, with the development of electronic information technology, electronic devices require increasingly larger amounts of memory to operate. While some data in memory can be compressed when available memory is limited, current methods either achieve high compression rates or are time-consuming.

[0022] When an electronic device has limited available memory, some data in memory can be written to the disk. When this data is needed, it can be read back from the disk into memory. However, since disk speed is generally slower than memory speed, the process of reading data from the disk into memory is slow. Alternatively, some data in memory can be compressed using compression algorithms, thereby reducing memory usage. When this data is needed later, it can be decompressed using the corresponding decompression algorithm to retrieve the uncompressed data.

[0023] Currently, a specific compression algorithm is selected to directly compress the data in memory. For example, LZ4, ZSTD, LZO, or DEFLATE compression algorithms can be pre-defined. LZ4 compression is a commonly used algorithm, offering advantages such as fast compression and decompression speeds and a simple compression format. However, compared to LZO, DEFLATE, or ZSTD compression algorithms, LZ4 has a relatively high compression ratio. ZSTD compression, on the other hand, provides excellent compression and decompression speeds and compression ratios, offering a lower compression ratio with minimal loss in compression and decompression speed.

[0024] However, the inventors discovered during their research that compressibility varies depending on the type of data. For some data, even with significant time investment, the reduction in compression ratio is minimal. In such cases, consistently using a high-compression algorithm may result in prolonged compression time without a substantial reduction in the compression ratio. Conversely, using a low-compression algorithm can achieve a faster compression rate, but this can lead to higher compression ratios for certain types of data.

[0025] Therefore, in order to overcome the above-mentioned defects, this application provides a data compression method, apparatus, electronic device, and computer-readable storage medium.

[0026] Please see Figure 1 , Figure 1 The diagram illustrates an application scenario of a data compression method provided in this application, namely data compression scenario 100. This data compression scenario 100 includes an electronic device 110 and a user 120.

[0027] The electronic device 110 can run various applications to achieve the functions corresponding to different applications. When using the electronic device 110, the user 120 may simultaneously open multiple different applications. Opening multiple applications simultaneously consumes a significant amount of the electronic device 110's memory, thus reducing its available memory. In this case, the electronic device 110 can use a data compression algorithm to compress some data in memory, thereby reducing the space occupied by the compressed data, increasing available memory, and improving the smoothness of the electronic device 110's operation. Simultaneously opening multiple different applications can mean keeping one application running in the foreground while multiple other applications run in the background. For a detailed description of the data compression algorithm, please refer to the following embodiments.

[0028] Please see Figure 2 , Figure 2 This application illustrates a data compression method provided in an embodiment, which can be applied to... Figure 1 In the electronic device 110, the processor of the electronic device 110 can be used as the execution subject. Specifically, the method includes steps S110 to S140.

[0029] Step S110: Obtain the data to be compressed from the memory page to be compressed.

[0030] When compressing data in the memory of an electronic device, different compression algorithms can be selected. For example, a first compression algorithm can be used to compress the data. This first algorithm has a fast compression rate but a relatively low compression ratio, thus achieving a moderate compression rate in a short time. Alternatively, a second compression algorithm can be used. This second algorithm has a lower compression ratio and a slower compression rate, thus achieving a lower compression rate in a longer time.

[0031] However, for some data, if the compression ratio after using the first compression algorithm is relatively low, it indicates that the data is relatively easy to compress. In this case, using the second compression algorithm can achieve a significant reduction in compression ratio at the cost of a slight increase in compression time. Conversely, if the compression ratio after using the first compression algorithm is relatively high, using the second compression algorithm will incur a greater increase in compression time for a smaller reduction in compression ratio.

[0032] It should be noted that the compression ratio, obtained after compressing data using a compression algorithm, is used to characterize the degree of compression of that data. The smaller the compression ratio, the greater the degree of compression of the data by the compression algorithm; the larger the compression ratio, the less the degree of compression of the data by the compression algorithm.

[0033] Therefore, to determine whether to use the first or second compression algorithm for the data, the data to be compressed in the memory page to be compressed can be obtained first. The memory of an electronic device may include multiple memory pages, each storing data.

[0034] One example is to use the most recently unused memory page in the electronic device's memory as the memory page to be compressed, thereby obtaining the data in the memory page to be compressed as the data to be compressed.

[0035] Another example is to use the memory pages corresponding to applications that occupy a large amount of memory space in the electronic device's memory as memory pages to be compressed, so that the data in the memory pages to be compressed can be obtained as data to be compressed.

[0036] Another example is that zRAM technology can be used to obtain memory pages as memory pages to be compressed, thereby obtaining the data in the memory pages to be compressed as data to be compressed.

[0037] It should be noted that the electronic device's memory may contain data in multiple memory pages that need to be compressed. Therefore, the data in multiple memory pages can be identified as the data to be compressed sequentially, and the subsequent steps can be repeated until the data in each memory page is compressed. The subsequent steps in this application will only be specifically described using the data to be compressed in one memory page.

[0038] Step S120: Compress the data to be compressed based on the first compression algorithm to obtain the first compressed data.

[0039] As explained above, the compression ratio of the data to be compressed using the first compression algorithm can be used to determine whether a second compression algorithm is needed. Therefore, after obtaining the data to be compressed, the first compression algorithm can be used to compress the data first, resulting in the first compressed data.

[0040] One example is that the data to be compressed is compressed using a first compression algorithm, which can be based on a dictionary compression algorithm to obtain the first compressed data. For details, please refer to the description in the following embodiments.

[0041] Step S130: Determine the target compression ratio based on the first compressed data and the data to be compressed.

[0042] After obtaining the first compressed data, the target compression ratio of the data to be compressed by the first compression algorithm can be determined. Specifically, the target compression ratio can be determined based on the first compressed data and the data to be compressed, and this target compression ratio characterizes the degree to which the data to be compressed is compressed. That is, the smaller the target compression ratio, the greater the degree to which the first compressed data is compressed; the larger the target compression ratio, the less the degree to which the first compressed data is compressed. For a detailed description of the method for calculating the target compression ratio, please refer to the following embodiments.

[0043] Step S140: If the target compression ratio is less than or equal to a preset threshold, the data to be compressed is compressed based on the second compression algorithm to obtain second compressed data, wherein the compression degree of the data by the second compression algorithm is greater than the compression degree of the data by the first compression algorithm, and the data compression rate of the second compression algorithm is less than the data compression rate of the first compression algorithm.

[0044] After obtaining the target compression ratio, the relationship between the target compression ratio and a preset threshold can be determined. Only when the target compression ratio is less than or equal to the preset threshold is the data to be compressed using the second compression algorithm to obtain the second compressed data. Since the second compression algorithm compresses the data to a greater degree than the first compression algorithm, the compression rate of the second compression algorithm is less than that of the first compression algorithm. Because a target compression ratio less than or equal to the preset threshold indicates that the data to be compressed is relatively easy to compress, using the second compression algorithm in this case allows for a significant reduction in the compression ratio at the cost of a slight increase in compression time.

[0045] If the target compression ratio is greater than a preset threshold, it indicates that the data to be compressed is not easily compressed. In this case, if the data to be compressed is still compressed using the second compression algorithm, it will increase the compression time by a large margin in exchange for a small decrease in the compression ratio. Therefore, in some implementations, if the target compression ratio is greater than the preset threshold, the data to be compressed may not be compressed using the second compression algorithm.

[0046] The preset threshold can be a pre-defined value, which can be a critical value obtained through multiple tests. This critical value indicates that when the target compression ratio is greater than the critical value, using the second compression algorithm to compress the data will result in a significant increase in compression time and a smaller effect on achieving the target compression ratio; conversely, if the target compression ratio is less than or equal to the critical value, using the second compression algorithm to compress the data can achieve a significant reduction in the target compression ratio by increasing the compression time slightly.

[0047] The second compression algorithm compresses the data to be processed. This can be achieved by first compressing the data using a dictionary compression algorithm, and then encoding the compressed data using a specified encoding method to obtain the second compressed data. For a detailed explanation, please refer to the following embodiments.

[0048] This application provides a data compression method. The method first obtains the data to be compressed from a memory page to be compressed; then, it compresses the data to be compressed using a first compression algorithm to obtain first compressed data; next, it determines a target compression ratio based on the first compressed data and the data to be compressed; if the target compression ratio is less than or equal to a preset threshold, it compresses the data to be compressed using a second compression algorithm to obtain second compressed data. The second compression algorithm compresses the data to a greater degree than the first compression algorithm, and the compression rate of the second compression algorithm is less than the compression rate of the first compression algorithm. Although the second compression algorithm can provide a smaller compression ratio than the first compression algorithm, its compression rate is also lower. If the second compression algorithm is used directly to compress all data, some data may require a significant amount of time to achieve only a small reduction in compression ratio, resulting in a long compression time. Conversely, if the first compression algorithm is used directly to compress all data, some data may only achieve a high compression ratio. Therefore, in this embodiment of the application, when the target compression ratio is less than or equal to a preset threshold, it indicates that the data to be compressed is relatively easy to compress. At this time, the data to be compressed is compressed based on the second compression algorithm to obtain the second compressed data, which can achieve a lower compression ratio with a smaller compression time.

[0049] Please see Figure 3 , Figure 3 This application illustrates a data compression method provided in an embodiment, which can be applied to... Figure 1 In the electronic device 110, the processor of the electronic device 110 can be used as the execution subject. Specifically, the method includes steps S210 to S290.

[0050] Step S210: Obtain the data to be compressed from the memory page to be compressed.

[0051] Step S210 has been described in detail in the foregoing embodiments and will not be repeated here.

[0052] Step S220: Compress the data to be compressed based on the first compression algorithm to obtain the first compressed data.

[0053] In some implementations, the first compression algorithm can be the LZ4 compression algorithm. Therefore, initialization can be performed first, which may include setting the scan interval and preset length during the compression process, and allocating a space in the memory of the electronic device to store the hash table.

[0054] The data of a preset length in the data to be compressed is obtained sequentially as the fifth intermediate data, and then the first hash value corresponding to the fifth intermediate data is obtained; the hash table is checked to see if there is a hash value that is the same as the first hash value.

[0055] If it does not exist, the first hash value and the fifth intermediate data corresponding to the first hash value are stored in the hash table, and the first address offset corresponding to the fifth intermediate data is scanned to obtain the second address. The data corresponding to the second address in the data to be compressed is determined as the new fifth intermediate data, and the process of obtaining the first hash value corresponding to the fifth intermediate data and subsequent steps is returned until a hash value with the same first hash value exists.

[0056] If it exists, the fifth intermediate data is compressed using a dictionary compression algorithm to obtain the sixth intermediate data. The first address offset corresponding to the fifth intermediate data is scanned to obtain the second address. The data corresponding to the second address in the data to be compressed is determined as the new fifth intermediate data. The process then returns to obtain the first hash value corresponding to the fifth intermediate data and subsequent steps until all data in the data to be compressed is traversed.

[0057] The dictionary file required for the dictionary compression algorithm can be this hash table. Finally, the compressed data and the uncompressed data, i.e., the sixth intermediate data and the uncompressed data, are used together to determine the first compressed data.

[0058] For example, if the data to be compressed is the string "dfabcdefghijklmnabcdkkkkkk", and the preset scan interval is 1 and the preset length is 4, then four bytes "dfab" are read as the fifth intermediate data, and the first hash value of "dfab" is calculated. If no hash value matching the first hash value is found in the hash table, then the first hash value and the fifth intermediate data "dfab" corresponding to the first hash value are put into the hash table.

[0059] The first address offset scanning interval corresponding to the fifth intermediate data is 1, which is used to obtain the second address and determine the data corresponding to the second address in the data to be compressed, that is, the new fifth intermediate data is "fabc". The first hash value of "fabc" is calculated and searched in the hash table. No hash value matching the first hash value is found. Therefore, the first hash value and the fifth intermediate data "fabc" corresponding to the first hash value are put into the hash table.

[0060] Repeat the above steps until the fifth intermediate data is confirmed to be "abcd" again. At this point, a hash value with the same first hash value as the fifth intermediate data can be found in the hash table. Then, the fifth intermediate data is compressed using a dictionary compression algorithm until all data in the data to be compressed is traversed.

[0061] Step S230: The ratio of the size of the first compressed data to the size of the data to be compressed is used as the first intermediate data.

[0062] Step S240: Based on the first intermediate data, obtain the target compression ratio.

[0063] As described above, the target compression ratio characterizes the degree to which the data to be compressed is compressed. Therefore, in some implementations, the ratio of the size of the first compressed data to the size of the data to be compressed can be used as the first intermediate data. For example, if the size of the first compressed data is A and the size of the data to be compressed is B, then A / B can be used as the first intermediate data. For instance, if A is half of B, then the first intermediate data can be determined to be 0.5.

[0064] Furthermore, the target compression ratio can also be characterized as a percentage. Therefore, based on the first intermediate data, the target compression ratio can be obtained by multiplying the first intermediate data by 100%. Referring again to the example above, if the first intermediate data is 0.5, then the target compression ratio could be 0.5 x 100% = 50%.

[0065] Step S250: Is the target compression ratio less than or equal to a preset threshold?

[0066] Furthermore, it can be determined whether the target compression ratio is less than or equal to a preset threshold. If the target compression ratio is determined to be less than or equal to the preset threshold, the process can proceed to step S260; if the target compression ratio is determined not to be less than or equal to the preset threshold, the process can proceed to step S290.

[0067] Step S260: If the target compression ratio is less than or equal to a preset threshold, the data to be compressed is compressed using a dictionary compression algorithm based on a pre-acquired dictionary file to obtain second intermediate data, wherein the dictionary file includes at least one standard data.

[0068] In some implementations, the second compression algorithm can be the ZSTD compression algorithm. Therefore, if the target compression ratio is less than or equal to a preset threshold, the data to be compressed can first be compressed using a dictionary compression algorithm to obtain second intermediate data. The dictionary compression algorithm can compress the data to be compressed using a dictionary file. For example, the dictionary file can be a pre-acquired general dictionary file, or it can be trained based on a portion of the data to be compressed; this application does not impose specific limitations.

[0069] Specifically, step S260 may include steps S261 to S266.

[0070] Step S261: If the target compression ratio is less than or equal to a preset threshold, determine the data to be compressed that matches the standard data in the dictionary file as compressible data.

[0071] Step S262: Obtain the length of the compressible data as the matching length.

[0072] Step S263: Obtain the value of the offset of the first character of the compressible data relative to the first character of the data to be compressed, as the offset amount.

[0073] Step S264: Determine the data to be compressed that does not match the standard data in the dictionary file, and classify it as incompressible data.

[0074] Step S265: Determine the length of the incompressible data as the incompressible length.

[0075] Step S266: Use the matching length, offset, incompressible data, and incompressible length as the second intermediate data.

[0076] In some implementations, the dictionary file includes at least one set of standard data, allowing the data to be compressed to be traversed and data matching the standard data in the dictionary file to be identified as compressible data. It is readily understood that data matching the standard data in the dictionary file can be used to characterize compressible data, thereby reducing the space occupied by the compressible data and ultimately achieving compression of the data.

[0077] Furthermore, the length of the compressible data can be obtained as the match length. For example, the number of characters in the compressible data can be used as the match length. For instance, if the compressible data is "abcd", then the number of characters in the compressible data is 4, and the match length can be determined to be 4.

[0078] Furthermore, the offset of the first character of the compressible data relative to the first character of the data to be compressed can be obtained as the offset. For example, if the data to be compressed is "adedgdabcd", the pointer of the first character of the data to be compressed is 0, while if the compressible data is "abcd", the pointer of the first character of the compressible data is 6. Therefore, the offset can be determined to be 6-0=6.

[0079] It is also possible to identify data in the data to be compressed that does not match the standard data described in the dictionary file as incompressible data (literals). For example, compressible data in the data to be processed can be excluded, leaving incompressible data. Alternatively, data in the data to be compressed that does not match the standard data can be directly identified from the dictionary file and classified as incompressible data.

[0080] Furthermore, the length of the incompressible data can be determined as the literal length. Similar to determining the matching length, the number of characters occupied by the incompressible data can be used as the literal length. For example, if the incompressible data is "adedgd", then the number of characters occupied by the incompressible data is 6, and the literal length can be determined to be 6.

[0081] Therefore, the matching length, offset, incompressible data, and incompressible length can be used as the second intermediate data.

[0082] Step S270: Encode the second intermediate data to obtain the second compressed data.

[0083] Furthermore, in order to further reduce the compression ratio of the second compressed data, the obtained second intermediate data can be further encoded to obtain the second compressed data.

[0084] One example is that the second intermediate data can be encoded using a preset encoding method to obtain the second compressed data. For instance, the preset encoding method could be Huffman coding, in which case the second intermediate data can be encoded using Huffman coding to obtain the second compressed data.

[0085] For example, if the preset encoding method is finite state entropy encoding, then the second intermediate data can be encoded using finite state entropy encoding.

[0086] In other embodiments, the incompressible data can be Huffman encoded to obtain third intermediate data. Then, the matching length, the offset, and the incompressible length are respectively encoded using finite-state entropy encoding to obtain fourth intermediate data. Thus, the second compressed data can be determined based on the third and fourth intermediate data.

[0087] Step S280: Store the second compressed data into the memory page to be compressed to update the memory page to be compressed.

[0088] After obtaining the second compressed data, it can be stored in the memory page to be compressed to update the memory page. When the second compressed data in the memory page needs to be read subsequently, it can be decoded.

[0089] Step S290: If the target compression ratio is greater than a preset threshold, the first compressed data is stored in the memory page to be compressed to update the memory page to be compressed.

[0090] If the target compression ratio is not less than or equal to a preset threshold, the process can proceed to step S290. Since the target compression ratio is not less than or equal to the preset threshold, it can be considered that the target compression ratio is greater than the preset threshold, indicating that the data to be compressed is not easily compressed. Therefore, even if the data to be compressed is compressed using the second compression algorithm, it will increase the compression time significantly, and the resulting decrease in compression ratio will be small. Therefore, at this point, the first compressed data obtained after compressing the data to be compressed using the first compression algorithm can be directly stored in the memory page to be compressed, thereby achieving a significant reduction in compression time with a smaller increase in compression ratio.

[0091] This application provides a data compression method. The method first obtains the data to be compressed from a memory page to be compressed; then, it compresses the data to be compressed using a first compression algorithm to obtain first compressed data; the ratio of the size of the first compressed data to the size of the data to be compressed is used as first intermediate data; based on the first intermediate data, a target compression ratio is obtained; if the target compression ratio is less than or equal to a preset threshold, the data to be compressed is compressed using a second compression algorithm to obtain second compressed data, which is then stored in the memory page to be compressed to update the memory page; if the target compression ratio is greater than the preset threshold, the first compressed data is stored in the memory page to be compressed to update the memory page. Therefore, in the embodiment provided by this application, when the target compression ratio is determined to be not less than or equal to the preset threshold, directly storing the first compressed data obtained after compressing the data to be compressed using the first compression algorithm into the memory page to be compressed can achieve a significant reduction in compression time with a relatively small increase in compression ratio. If the target compression ratio is determined to be less than or equal to a preset threshold, the data to be compressed can be compressed using a second compression algorithm to obtain second compressed data, which is then stored in the memory page to be compressed. This allows for a significant reduction in compression ratio with a relatively small increase in compression time.

[0092] Please see Figure 4 , Figure 4 This application illustrates a data compression method provided in an embodiment, which can be applied to... Figure 1 In the electronic device 110, the processor of the electronic device 110 can be used as the execution subject. Specifically, the method includes steps S310 to S3120.

[0093] Step S310: Obtain the data to be compressed from the memory page to be compressed.

[0094] Data from the memory page to be compressed can be obtained and used as the data to be compressed. The memory page to be compressed may include at least one memory page. For a detailed description, please refer to the foregoing embodiments, which will not be repeated here.

[0095] Step S320: Initialization.

[0096] In some implementations, the first compression algorithm can be the LZ4 compression algorithm, allowing for initialization before compressing the data to be compressed based on the first compression algorithm. For example, this initialization may specifically include setting the scan interval and preset length during the compression process, and allocating a space in the electronic device's memory to store the hash table. Optionally, after obtaining the hash table, a clearing operation can be performed on the hash table to ensure that no other data is included in the hash table used subsequently.

[0097] Step S330: The first compression algorithm compresses the data to be compressed.

[0098] The data to be compressed can be compressed using a first compression algorithm. For example, a dictionary compression algorithm can be used. Detailed descriptions can be found in the foregoing embodiments, and will not be repeated here.

[0099] Step S340: Has the data to be compressed been completely compressed?

[0100] As described above, the data to be compressed needs to be traversed using a dictionary compression algorithm. Therefore, it can be determined whether the data to be compressed has been fully compressed. If it is determined that the data has not been fully compressed, the process can return to step S330 to continue compression. If it is determined that the data has been fully compressed, the process can jump to step S350.

[0101] Step S350: Determine the target compression ratio based on the first compressed data and the data to be compressed.

[0102] After compressing the data to be compressed using the first compression algorithm, the compressed data can be used as the first compressed data. Furthermore, a target compression ratio can be determined based on the first compressed data and the data to be compressed. The specific method for determining the target compression ratio can be found in the description of the foregoing embodiments, and will not be repeated here.

[0103] Step S360: Is the target compression ratio less than or equal to the preset threshold?

[0104] As described above, the target compression ratio can be used to characterize the degree to which the data to be compressed is compressed. When the target compression ratio is less than or equal to a preset threshold, it indicates that the data to be compressed is relatively easy to compress. In this case, compressing the data using the second compression algorithm can achieve a significant reduction in compression ratio at the cost of a slight increase in compression time. Conversely, when the target compression ratio is greater than the preset threshold, it indicates that the data to be compressed is relatively difficult to compress. In this case, compressing the data using the second compression algorithm will increase compression time significantly and only achieve a small reduction in compression ratio. Therefore, if the target compression ratio is less than or equal to the preset threshold, the process can proceed to step S380; if the target compression ratio is greater than the preset threshold, the process can proceed to step S370.

[0105] Step S370: Store the first compressed data into the memory page to be compressed.

[0106] If the target compression ratio exceeds a preset threshold, even if the data to be compressed is compressed using the second compression algorithm, it will increase the compression time significantly, and the resulting decrease in compression ratio will be relatively small. Therefore, in this case, the first compressed data obtained after compressing the data to be compressed using the first compression algorithm can be directly stored in the memory page to be compressed, thereby achieving a significant reduction in compression time with a smaller increase in compression ratio.

[0107] Step S380: Obtain the data to be compressed from the memory page to be compressed.

[0108] If the target compression ratio is less than or equal to a preset threshold, the data to be compressed can be further compressed using a second compression algorithm. This involves first obtaining the data to be compressed from the memory pages to be compressed. It should be noted that the data obtained at this stage is essentially the same as the data compressed using the first compression algorithm.

[0109] Step S390: Initialization.

[0110] In some implementations, the second compression algorithm can be the ZSTD compression algorithm, so that initialization can be performed before compressing the data to be compressed based on the second compression algorithm. For example, this initialization may specifically include obtaining a dictionary file for the dictionary compression algorithm.

[0111] Step S3100: The second compression algorithm compresses the data to be compressed.

[0112] The data to be compressed can be compressed using a second compression algorithm. For example, a dictionary compression algorithm can be used to compress the data to be compressed to obtain second intermediate data, which is then encoded to obtain the second compressed data. For a detailed description, please refer to the foregoing embodiments; further details will not be repeated here.

[0113] Step S3110: Has the data to be compressed been completely compressed?

[0114] As described above, the data to be compressed needs to be traversed using a dictionary compression algorithm. Therefore, it can be determined whether the data to be compressed has been fully compressed. If it is determined that the data has not been fully compressed, the process can return to step S3100 to continue compression. If it is determined that the data has been fully compressed, the process can jump to step S3120.

[0115] Step S3120: Store the second compressed data into the memory page to be compressed.

[0116] At this point, the second compressed data obtained by compressing the data to be compressed using the second compression algorithm can be stored in the memory page to be compressed, thereby achieving a significant reduction in compression time with a relatively small increase in compression ratio.

[0117] Please see Figure 5 The diagram illustrates a structural block diagram of a data compression device 500 provided in an embodiment of this application, which is applied to an electronic device. The device includes: a first acquisition unit 510, a first compression unit 520, a determination unit 530, and a second compression unit 540.

[0118] The first acquisition unit 510 is used to acquire the data to be compressed in the memory page to be compressed.

[0119] The first compression unit 520 is used to compress the data to be compressed based on the first compression algorithm to obtain the first compressed data.

[0120] The determining unit 530 is used to determine the target compression ratio based on the first compressed data and the data to be compressed.

[0121] Optionally, the determining unit 530 can also be used to take the ratio of the data size of the first compressed data to the data size of the data to be compressed as the first intermediate data; and obtain the target compression ratio based on the first intermediate data.

[0122] The second compression unit 540 is configured to compress the data to be compressed based on a second compression algorithm to obtain second compressed data if the target compression ratio is less than or equal to a preset threshold. The compression degree of the second compression algorithm is greater than that of the first compression algorithm, and the compression rate of the second compression algorithm is less than that of the first compression algorithm. The first compression algorithm includes the LZ4 compression algorithm, and the second compression algorithm includes the ZSTD algorithm.

[0123] Optionally, the second compression unit 540 can also be used to compress the data to be compressed based on a dictionary file and a dictionary compression algorithm if the target compression ratio is less than or equal to a preset threshold, to obtain second intermediate data, wherein the dictionary file includes at least one standard data; and to encode the second intermediate data to obtain the second compressed data.

[0124] Optionally, the second compression unit 540 can also be used to: determine, if the target compression ratio is less than or equal to a preset threshold, determine data that matches the standard data in the dictionary file as compressible data; obtain the length of the compressible data as the matching length; obtain the value of the offset of the first character of the compressible data relative to the first character of the data to be compressed as the offset; determine data that does not match the standard data in the dictionary file as incompressible data; determine the length of the incompressible data as the incompressible length; and use the matching length, offset, incompressible data, and incompressible length as the second intermediate data.

[0125] Optionally, the second compression unit 540 can also be used to perform Huffman coding on the incompressible data to obtain third intermediate data; perform finite state entropy coding on the matching length, the offset and the incompressible length respectively to obtain fourth intermediate data; and determine the second compressed data based on the third intermediate data and the fourth intermediate data.

[0126] The data compression device 500 may further include a first storage unit, which is used to store the second compressed data into the memory page to be compressed in order to update the memory page to be compressed.

[0127] The data compression device 500 may further include a second storage unit, which is used to store the first compressed data into the memory page to be compressed if the target compression ratio is greater than a preset threshold, so as to update the memory page to be compressed.

[0128] Please refer to Figure 6This diagram illustrates a structural block diagram of an electronic device 110 provided in an embodiment of this application. The electronic device 110 can be a smartphone, tablet computer, e-reader, or other electronic device capable of running applications. The electronic device 110 in this application may include one or more of the following components: a processor 111, a memory 112, and one or more applications, wherein the one or more applications can be stored in the memory 112 and configured to be executed by one or more processors 111, and the one or more applications are configured to perform the methods described in the foregoing method embodiments.

[0129] Processor 111 may include one or more processing cores. Processor 111 connects to various parts within the electronic device 110 using various interfaces and lines, and performs various functions and processes data of the electronic device 110 by running or executing instructions, programs, code sets, or instruction sets stored in memory 112, and by calling data stored in memory 112. Optionally, processor 111 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 111 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 111 and may be implemented separately using a communication chip.

[0130] The memory 112 may include random access memory (RAM) or read-only memory (ROM). The memory 112 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 112 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the electronic device 110 during use (such as phonebook data, audio and video data, chat log data, etc.).

[0131] Please refer to Figure 7This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable storage medium 700 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0132] The computer-readable storage medium 700 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 700 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 700 has storage space for program code 710 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 710 may be compressed, for example, in a suitable form.

[0133] Please refer to Figure 8 The diagram illustrates a structural block diagram 800 of a computer program product provided in an embodiment of this application. The computer program product 800 includes a computer program / instructions 810, which, when executed by a processor, implements the steps of the aforementioned method.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data compression method, characterized in that, Applied to electronic devices, the method includes: Retrieve the data to be compressed from the memory page to be compressed; The data to be compressed is compressed based on the first compression algorithm to obtain the first compressed data; The target compression ratio is determined based on the first compressed data and the data to be compressed. If the target compression ratio is less than or equal to a preset threshold, the data to be compressed is compressed based on the second compression algorithm to obtain second compressed data, wherein the compression degree of the data by the second compression algorithm is greater than the compression degree of the data by the first compression algorithm, and the data compression rate of the second compression algorithm is less than the data compression rate of the first compression algorithm.

2. The method according to claim 1, characterized in that, Determining the target compression ratio based on the first compressed data and the data to be compressed includes: The ratio of the size of the first compressed data to the size of the data to be compressed is used as the first intermediate data; The target compression ratio is obtained based on the first intermediate data.

3. The method according to claim 1, characterized in that, If the target compression ratio is less than or equal to a preset threshold, then the data to be compressed is compressed based on a second compression algorithm to obtain second compressed data, including: If the target compression ratio is less than or equal to a preset threshold, the data to be compressed is compressed using a dictionary compression algorithm based on a pre-acquired dictionary file to obtain second intermediate data, wherein the dictionary file includes at least one standard data. The second intermediate data is encoded to obtain the second compressed data.

4. The method according to claim 3, characterized in that, The second intermediate data includes a matching length, an offset, incompressible data, and the incompressible length of the incompressible data. If the target compression ratio is less than or equal to a preset threshold, the data to be compressed is compressed using a dictionary compression algorithm based on a pre-acquired dictionary file to obtain the second intermediate data. The dictionary file includes at least one set of standard data, including: If the target compression ratio is less than or equal to a preset threshold, the data to be compressed that matches the standard data in the dictionary file is determined as compressible data. Obtain the length of the compressible data and use it as the matching length; Obtain the numerical value of the offset of the first character of the compressible data relative to the first character of the data to be compressed, and use it as the offset; Data that does not match the standard data in the dictionary file is identified as incompressible data. Determine the length of the incompressible data as the incompressible length; The matching length, offset, incompressible data, and incompressible length are used as the second intermediate data.

5. The method according to claim 4, characterized in that, The process of encoding the second intermediate data to obtain the second compressed data includes: The incompressible data is Huffman encoded to obtain the third intermediate data; The matching length, the offset, and the incompressible length are respectively encoded using finite state entropy encoding to obtain the fourth intermediate data; The second compressed data is determined based on the third intermediate data and the fourth intermediate data.

6. The method according to claim 1, characterized in that, If the target compression ratio is less than or equal to a preset threshold, then after compressing the data to be compressed based on the second compression algorithm to obtain the second compressed data, the method further includes: The second compressed data is stored in the memory page to be compressed in order to update the memory page to be compressed.

7. The method according to claim 1, characterized in that, The method further includes: If the target compression ratio is greater than a preset threshold, the first compressed data is stored in the memory page to be compressed in order to update the memory page to be compressed.

8. The method according to claim 1, characterized in that, The first compression algorithm includes the LZ4 compression algorithm, and the second compression algorithm includes the ZSTD algorithm.

9. A data compression device, characterized in that, Applied to electronic devices, the device includes: The first acquisition unit is used to acquire the data to be compressed in the memory page to be compressed; The first compression unit is used to compress the data to be compressed based on the first compression algorithm to obtain the first compressed data; A determining unit is configured to determine a target compression ratio based on the first compressed data and the data to be compressed; The second compression unit is used to compress the data to be compressed based on the second compression algorithm if the target compression ratio is less than or equal to a preset threshold, to obtain the second compressed data, wherein the compression degree of the data by the second compression algorithm is greater than the compression degree of the data by the first compression algorithm, and the data compression rate of the second compression algorithm is less than the data compression rate of the first compression algorithm.

10. An electronic device, characterized in that, include: One or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to perform the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to execute the method as described in any one of claims 1-8.

Images