Data compression method, data decompression method, device and electronic equipment

Abstract

Embodiments of the present application disclose a data compression method, a data decompression method, a device, an electronic device and a storage medium. The method comprises: obtaining to-be-compressed data; obtaining a first hash table and a second hash table corresponding to a preset dictionary compression algorithm, the first hash table being a storage structure corresponding to the preset dictionary compression algorithm and comprising a first byte length, the second hash table being a storage structure corresponding to the preset dictionary compression algorithm and comprising a second byte length, the first byte length being greater than the second byte length; performing first matching on the to-be-compressed data according to a first preset byte length; if the first matching is unsuccessful, performing second matching on the to-be-compressed data according to a second preset byte length; and if the second matching is successful, performing compression processing on the to-be-compressed data based on the second hash table to obtain target compressed data. In this way, the compression rate of the preset compression algorithm is improved.

Description

Technical Field

[0001] This application belongs to the field of data compression technology, specifically relating to a data compression method, a data decompression method, an apparatus, an electronic device, and a storage medium. Background Technology

[0002] In an era of rapid development in network communication technology, lossless data compression technology has gradually demonstrated its unique importance, leading to its widespread application in cloud storage and internet communication, such as compressed file systems, data backup, and web front-end services. Lossless data compression technology can reduce data redundancy and decrease the resources occupied by information in storage without affecting the quality of data reconstruction, thus objectively improving data transmission efficiency. Lossless data compression technology is mainly divided into two categories: dictionary-based lossless data compression technology and statistical lossless data compression technology. However, the compression ratio of dictionary-based data compression still needs improvement. Summary of the Invention

[0003] In view of the above problems, this application proposes a data compression method, a data decompression method, an apparatus, an electronic device, and a storage medium to improve the above problems.

[0004] In a first aspect, embodiments of this application provide a data compression method, the method comprising: acquiring data to be compressed; acquiring a first hash table and a second hash table corresponding to a preset dictionary compression algorithm, wherein the first hash table is a storage structure including a first byte length corresponding to the preset dictionary compression algorithm, and the second hash table is a storage structure including a second byte length corresponding to the preset dictionary compression algorithm, wherein the first byte length is greater than the second byte length; performing a first matching on the data to be compressed according to the first preset byte length; if the first matching fails, performing a second matching on the data to be compressed according to the second preset byte length; if the second matching succeeds, compressing the data to be compressed based on the second hash table to obtain target compressed data.

[0005] Secondly, embodiments of this application provide a data decompression method, the method comprising: reading data stored in the first specified bits of target compressed data; if the hash table corresponding to the data is a first hash table corresponding to a preset dictionary compression algorithm, performing decompression processing on the target compressed data based on the first hash table to obtain decompressed data, wherein the first hash table is a storage structure including a first byte length corresponding to the preset dictionary compression algorithm; if the hash table corresponding to the data is a second hash table corresponding to the preset dictionary compression algorithm, performing decompression processing on the target compressed data based on the second hash table to obtain decompressed data, wherein the second hash table is a storage structure including a second byte length corresponding to the preset dictionary compression algorithm, and the first byte length is greater than the second byte length.

[0006] Thirdly, embodiments of this application provide a data compression apparatus, the apparatus comprising: a data acquisition unit for acquiring data to be compressed; a hash table acquisition unit for acquiring a first hash table and a second hash table corresponding to a preset dictionary compression algorithm, wherein the first hash table is a storage structure including a first byte length corresponding to the preset dictionary compression algorithm, and the second hash table is a storage structure including a second byte length corresponding to the preset dictionary compression algorithm, and the first byte length is greater than the second byte length; a first matching unit for performing a first matching on the data to be compressed according to the first preset byte length; a second matching unit for performing a second matching on the data to be compressed according to the second preset byte length if the first matching fails; and a compression unit for compressing the data to be compressed based on the second hash table if the second matching succeeds, to obtain target compressed data.

[0007] Fourthly, embodiments of this application provide a data decompression apparatus, the apparatus comprising: a data acquisition unit, configured to read data stored in the first specified bits of target compressed data; a first decompression unit, configured to, if the hash table corresponding to the data is a first hash table corresponding to a preset dictionary compression algorithm, perform decompression processing on the target compressed data based on the first hash table to obtain decompressed data, wherein the first hash table is a storage structure including a first byte length corresponding to the preset dictionary compression algorithm; and a second decompression unit, configured to, if the hash table corresponding to the data is a second hash table corresponding to the preset dictionary compression algorithm, perform decompression processing on the target compressed data based on the second hash table to obtain decompressed data, wherein the second hash table is a storage structure including a second byte length corresponding to the preset dictionary compression algorithm, and the first byte length is greater than the second byte length.

[0008] Fifthly, embodiments of this application provide an electronic device, including one or more processors and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the methods described above.

[0009] Sixthly, embodiments of this application provide a computer-readable storage medium storing program code, wherein the above-described method is executed when the program code is run.

[0010] This application provides a data compression method, apparatus, electronic device, and storage medium. The method involves acquiring data to be compressed, and obtaining a first hash table and a second hash table corresponding to a preset dictionary compression algorithm. The first hash table is a storage structure containing a first byte length corresponding to the preset dictionary compression algorithm, and the second hash table is a storage structure containing a second byte length corresponding to the preset dictionary compression algorithm. The first byte length is greater than the second byte length. The data to be compressed is then subjected to a first matching according to the first preset byte length. If the first matching fails, the data to be compressed is subjected to a second matching according to the second preset byte length. If the second matching succeeds, the data to be compressed is compressed based on the second hash table to obtain the target compressed data. This method introduces a second hash table to the preset dictionary compression algorithm. The two hash tables work together to compress the data to be compressed. When the second matching succeeds, the data to be compressed can be compressed using the second hash table, thus improving the compression ratio of the preset compression algorithm. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart of a data compression method according to an embodiment of this application is shown;

[0013] Figure 2 A schematic diagram of the structure of a first hash table in one embodiment of this application is shown;

[0014] Figure 3 A schematic diagram of the structure of a second hash table in one embodiment of this application is shown;

[0015] Figure 4 A flowchart of a data compression method according to another embodiment of this application is shown;

[0016] Figure 5 A schematic diagram of the structure of the second hash table in another embodiment of this application is shown;

[0017] Figure 6 A schematic diagram of the process described in steps S210-S280 in another embodiment of this application is shown;

[0018] Figure 7 A flowchart of a data decompression method according to an embodiment of this application is shown;

[0019] Figure 8 A flowchart of a data decompression method according to another embodiment of this application is shown;

[0020] Figure 9 A schematic diagram of the process described in steps S410-S430 in another embodiment of this application is shown;

[0021] Figure 10 A structural block diagram of a data compression apparatus according to an embodiment of this application is shown;

[0022] Figure 11 This paper shows a structural block diagram of a data decompression apparatus according to an embodiment of the present application;

[0023] Figure 12 This paper shows a structural block diagram of an electronic device for performing a data compression method or data decompression method according to an embodiment of this application.

[0024] Figure 13 This application illustrates a storage unit for storing or carrying program code that implements a data compression method or data decompression method according to an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0026] In an era of rapid development in network communication technology, lossless data compression technology has gradually demonstrated its unique importance, leading to its widespread application in cloud storage and internet communication, such as compressed file systems, data backup, and web front-end services. Lossless data compression technology can reduce data redundancy and decrease the resources occupied by information in storage without affecting the quality of data reconstruction, thus objectively improving data transmission efficiency. Lossless data compression technology is mainly divided into two categories: dictionary-based lossless data compression technology and statistical lossless data compression technology.

[0027] The inventors discovered in their research on dictionary-based lossless data compression technology that the principle behind this technology is to use a small amount of storage space to store previously existing information. Taking the LZ4 compression algorithm as an example, its minimum matching length for data compression is 4 bytes, so its minimum storage unit is 3 bytes; otherwise, compression will not be achieved. 4 bytes equals 32 bits. If data blocks smaller than 4 bytes (such as 2 or 3 bytes) are not considered for matching, many duplicate data blocks smaller than 4 bytes (such as 2 or 3 bytes) will not be compressed, thus failing to achieve a good compression ratio.

[0028] Therefore, the inventors have proposed the data compression method, data decompression method, apparatus, electronic device, and storage medium of this application. The method involves acquiring data to be compressed, and acquiring a first hash table and a second hash table corresponding to a preset dictionary compression algorithm. The first hash table is a storage structure containing a first byte length corresponding to the preset dictionary compression algorithm, and the second hash table is a storage structure containing a second byte length corresponding to the preset dictionary compression algorithm. The first byte length is greater than the second byte length. Then, the data to be compressed is first matched according to the first preset byte length. If the first match fails, the data to be compressed is second matched according to the second preset byte length. If the second match succeeds, the data to be compressed is compressed based on the second hash table to obtain the target compressed data. Through this method, a second hash table is introduced to the preset dictionary compression algorithm. The two hash tables work together to compress the data to be compressed. If the second match succeeds, the data to be compressed can be compressed using the second hash table, thus improving the compression ratio of the preset compression algorithm.

[0029] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0030] Please see Figure 1 This application provides a data compression method applied to an electronic device, the method comprising:

[0031] Step S110: Obtain the data to be compressed.

[0032] In this embodiment, the data to be compressed can be any type of data that needs to be compressed. For example, the data to be compressed can be data that needs to be written to memory, data generated by the processor when executing a program, data received when interacting with other devices or processes, image data or text data sent by other electronic devices, or data exchanged between multiple memories, etc., without specific limitations.

[0033] As one approach, the data to be compressed can be the data written to memory for the first time. Memory (also known as random access memory, RAM) refers to the memory used when a program is running (i.e., running memory). It can only temporarily store data and is used to exchange cached data with the processor, but memory itself cannot be used for long-term data storage. Applications need to occupy memory space to store data during runtime, and the amount of memory space occupied by different applications in different runtime states may vary.

[0034] The data written to memory for the first time can be hot data within time-series data, which refers to data generated in a time sequence. Time-series data is a data column recorded in chronological order according to a unified indicator. It can include time information, data type, and data information. Specifically, the time information can be a timestamp; the data type can be a known or potentially emerging data type, such as existing data types including Integer, Long, Float, Double, String, BinaryArray, and Image; the data information can be the data itself, i.e., the raw data without time information or data type descriptions.

[0035] Because time-series data is massive, it is impossible to store all of it in the memory of an electronic device. To address this issue, in this embodiment, data within the current time period is placed in memory as hot data. Then, when this hot data exceeds a specified time, the electronic device compresses it and finally persists it as cold data. Hot data can be frequently accessed data, typically stored in a cache storage device. Cold data refers to data with low access frequency and low performance requirements, such as data that will not be accessed in the short term or will no longer be needed in the future.

[0036] As another approach, the data to be compressed can be a defined amount of allocated memory. When compressing allocated memory, a portion of it can be selected from all allocated memory, and then compressed using a hardware-based compression algorithm. The size of this portion of allocated memory can be determined based on a preset compression amount. This preset compression amount is a pre-defined size of the allocated memory to be compressed.

[0037] Optionally, in this embodiment, when allocating memory to processes, the kernel knows how much memory each process needs, and can record how much memory is allocated to each process. Therefore, when compressing the allocated memory according to a preset compression amount, a portion of the allocated memory can be selected for compression based on the size of the memory corresponding to each process and the preset compression amount. For example, if the preset compression amount is 10MB, and the memory size corresponding to process 1 is 2MB, process 2 is 5MB, process 3 is 8MB, and process 4 is 3MB, then when selecting a portion of the allocated memory for compression based on the size of the memory corresponding to each process and the preset compression amount, the memory corresponding to processes 1, 2, and 4 can be compressed; alternatively, the memory corresponding to processes 1 and 3 can be compressed.

[0038] Optionally, in this embodiment, the data to be compressed in memory can be retrieved at set intervals. Alternatively, the operation of retrieving the currently stored data to be compressed in memory can be performed based on a user-input trigger command.

[0039] Step S120: Obtain the first hash table and the second hash table corresponding to the preset dictionary compression algorithm. The first hash table is a storage structure with a length of the first byte corresponding to the preset dictionary compression algorithm, and the second hash table is a storage structure with a length of the second byte corresponding to the preset dictionary compression algorithm. The length of the first byte is greater than the length of the second byte.

[0040] In this embodiment, the preset dictionary compression algorithm is a pre-set compression algorithm used to compress the data to be compressed. The specific working principle of the preset dictionary compression algorithm is to encode long strings (also called matches or phrases) into short tokens, replacing phrases in the dictionary with these small tokens. In other words, it achieves data compression by replacing long strings that appear repeatedly in the data with small tokens. The symbols it processes are not necessarily text characters; they can be symbols of any other size. The preset dictionary compression algorithm can be LZ4 compression, LZ77 compression, LZ78 compression, LZSS compression, or LZW compression, etc., and is not specifically limited here.

[0041] The first hash table and the second hash table can be hash tables pre-set for a preset dictionary compression algorithm. The first hash table is a modified storage structure based on the original storage structure of the preset dictionary compression algorithm, and the second hash table is a new storage structure introduced by the preset dictionary compression algorithm. The target compressed data obtained by compressing using the second hash table is smaller than the target compressed data obtained by compressing using the first hash table. The first hash table is a storage structure obtained by using the first specified bits as flag bits of the original storage structure of the preset dictionary compression algorithm, while keeping the rest of the structure unchanged.

[0042] For example, if the first hash table is a storage structure with a length of 3 bytes, then the second hash table can be a storage structure with a length of 2 bytes or 1 byte. The first hash table is used to store data compressed according to a first preset byte length, and the second hash table is used to store data compressed according to a second preset byte length. The first preset byte length can be understood as the minimum matching length corresponding to the first hash table, i.e., the original minimum matching length corresponding to the preset dictionary compression algorithm. The second preset byte length can be understood as the minimum matching length corresponding to the second hash table, i.e., the minimum matching length corresponding to the introduced second storage structure. For example, if the preset dictionary compression algorithm is LZ4 compression, then the first preset byte length can be 4 bytes, and the second preset byte length can be 3 bytes or 2 bytes. When the preset dictionary compression algorithm is LZ4 compression, the first hash table can be as follows: Figure 2 As shown, Figure 2 The storage structure shown is a modification of the original LZ4 compression algorithm's storage structure. The main idea is to take the first bit of the original LZ4 compression algorithm's storage structure as a flag bit F, with a value of 0 or 1, used to distinguish the current storage structure. Figure 2 The storage structure is still Figure 3 The storage structure of the second hash table; the second hash table can be like... Figure 3 As shown, Figure 3 The storage structure occupies 2 bytes, and its first bit is a flag bit, which serves the same purpose as... Figure 2 The flag bit in the middle has the same function. Its remaining 15 bits are used to store the offset D of the matched 3-byte character (representing the distance from the previous matched character). These 15 bits can represent a maximum matching length of 32767, which means that it can be used not only for ordinary 4096 memory page compression, but also for compression based on large memory pages of 16k and 32k, or for compression of general data.

[0043] As one approach, the first hash table and the second hash table can be pre-set and stored in a preset storage area. When the data to be compressed is obtained, the first hash table and the second hash table corresponding to the preset dictionary compression algorithm can be directly obtained from the preset storage area.

[0044] Step S130: Perform a first matching on the data to be compressed according to the first preset byte length.

[0045] In this embodiment of the application, the first preset byte length is the minimum matching length corresponding to the first hash table, that is, the minimum matching length originally corresponding to the preset dictionary compression algorithm.

[0046] After obtaining the data to be compressed, the data is first matched according to the minimum matching length of the preset dictionary compression algorithm. For example, if the preset dictionary compression algorithm is LZ4 compression algorithm, then the minimum matching length corresponding to the preset dictionary compression algorithm is 4 bytes. Therefore, the data to be compressed is matched according to 4 bytes. That is, in the process of compressing the data, at least 4 bytes are used as the scanning window to find the match, and 1 byte is moved each time to scan. If a duplicate is found, it is compressed.

[0047] Step S140: If the first matching fails, the data to be compressed is matched a second time according to the second preset byte length.

[0048] In this embodiment, the second preset byte length can be understood as the minimum matching length corresponding to the second hash table, that is, the minimum matching length corresponding to the introduced second storage structure. The first preset byte length is greater than the second preset byte length. For example, if the introduced second storage structure includes a byte length of 2 bytes, then the minimum matching length corresponding to this storage structure is 3 bytes.

[0049] As one approach, a failed first match indicates that the length of the data to be compressed is less than a first preset byte length. When it is determined that the first match has failed, the process can be switched to match according to a second preset byte length, that is, to match the data to be compressed with a smaller matching length. For example, if the first match is to match the data to be compressed as 4 bytes and the second match is to match the data to be compressed as 3 bytes, if the match to be compressed as 4 bytes fails, the process can be changed to match the data to be compressed as 3 bytes.

[0050] Step S150: If the second match is successful, the data to be compressed is compressed based on the second hash table to obtain the target compressed data.

[0051] In this embodiment of the application, if the second match is successful, the data to be compressed is stored according to the second hash table, and the offset of the matching character in the data to be compressed is stored in the second hash table.

[0052] This application provides a data compression method that involves obtaining data to be compressed, acquiring a first hash table and a second hash table corresponding to a preset dictionary compression algorithm, and then performing a first match on the data to be compressed according to a first byte length. If the first match fails, a second match is performed on the data to be compressed according to a second byte length. If the second match succeeds, the data to be compressed is compressed based on the second hash table to obtain the target compressed data. By introducing a second hash table into the preset dictionary compression algorithm, the two hash tables work together to compress the data. When the second match succeeds, the data can be compressed using the second hash table, thus improving the compression ratio of the preset compression algorithm.

[0053] Please see Figure 4 This application provides a data compression method applied to an electronic device, the method comprising:

[0054] Step S210: Obtain the data to be compressed.

[0055] Step S220: Obtain the first hash table and the second hash table corresponding to the preset dictionary compression algorithm.

[0056] Wherein, the data stored in the first specified bits of the first hash table is used as a first flag bit to identify whether the data to be compressed is compressed based on the first hash table; the data stored in the first specified bits of the second hash table is used as a second flag bit to identify whether the data to be compressed is compressed based on the second hash table.

[0057] In this embodiment, the pre-specified bits are pre-set bits used to store identifiers for distinguishing different storage structures. The number of pre-specified bits varies depending on the number of storage structures to be distinguished. For example, if the number of storage structures to be distinguished is 2, then the number of pre-specified bits is 1, meaning that 0 or 1 can distinguish between the two different storage structures; if the number of storage structures to be distinguished is 3 or 4, then the number of pre-specified bits can be 2, meaning that 00, 01, 10, and 11 can distinguish between 3 or 4 storage structures; if the number of storage structures to be distinguished is greater than 4, then the number of pre-specified bits will be greater than 2, and so on.

[0058] As a method, the number of second hash tables can be more than one. When the minimum matching length corresponding to the original storage structure of the preset dictionary compression algorithm is greater than 3 bytes, the number of second hash tables can include at least two. For example, if the preset dictionary compression algorithm is the LZ4 compression algorithm, and the minimum matching length corresponding to the original storage structure of the LZ4 compression algorithm is 4 bytes, then the number of second hash tables can include two, one of which is as follows: Figure 3 The storage structure shown is a storage structure with a length of 2 bytes. One type is as follows: Figure 5 The storage structure shown is a storage structure with a length of 1 Byte. This storage structure is adapted to match 2-Byte characters. Since we want to compress 2-Byte strings, we change D in the second hash table to 7 bits, that is, the maximum matching length is 127. In the corresponding compression process, when a 2-Byte compression block is matched, a judgment is also made. If the distance from the matching character in the dictionary is greater than 127, the compression is abandoned.

[0059] In this embodiment, when there may be more than one second hash table, one hash table can be selected from multiple second hash tables as the hash table in this embodiment. When selecting a hash table from multiple second hash tables, the selection can be based on the performance loss associated with each hash table, and the hash table with the smallest performance loss can be selected as the second hash table in this embodiment. Here, performance loss refers to the performance loss of the electronic device during the compression process of the data to be compressed.

[0060] Optionally, the second hash table can consist of two fields: a flag field and a matching character offset field. The flag field stores data representing different storage locations; the matching character offset stores the offset of the matching character in the data to be compressed.

[0061] Step S230: Perform a first match on the data to be compressed according to the first preset byte length, and determine whether the first match is successful; if the match is successful, proceed to step S240; if the match is unsuccessful, proceed to step S250.

[0062] In this embodiment of the application, when the first match is successful, the matching is expanded according to the original compression process of the preset dictionary compression algorithm, and the data to be compressed is stored according to the first hash table.

[0063] Step S240: Compress the data to be compressed based on the first hash table to obtain the target compressed data.

[0064] Step S250: Perform a second match on the data to be compressed according to the second preset byte length, and determine whether the second match is successful; if the match is successful, proceed to step S260; if the match is unsuccessful, proceed to steps S270 and S280.

[0065] Step S260: Compress the data to be compressed based on the second hash table to obtain the target compressed data.

[0066] Step S270: Update the first hash table and the second hash table.

[0067] In this embodiment, when the second match fails, the first hash table and the second hash table are updated. That is, if the second match fails, the original compression process of the preset dictionary compression algorithm is followed: the hash information of the current first preset byte length is stored in the first hash table, and the hash information of the current second preset byte length is stored in the second hash table. Then, the current position pointer continues to slide forward 1 byte, repeating the above process. Here, the current position pointer refers to the position of the starting character of the current match in the data to be compressed.

[0068] Step S280: Perform a first match on the data to be compressed according to the first preset byte length corresponding to the updated first hash table.

[0069] In this embodiment of the application, after updating the first hash table and the second hash table and sliding the current position pointer forward by 1 byte, the data to be compressed is re-matched according to the first preset byte length.

[0070] For example, the process described in steps S210-S280 can be as follows: Figure 6 As shown, the preset dictionary compression algorithm is LZ4 compression algorithm, and the first hash table is as follows: Figure 2 As shown, the second hash table is as follows Figure 3As shown. Compression begins by initializing two hash tables: LZ4_hash and LZ3_hash. Then, the data to be compressed is matched in 4-byte increments. If a match is found, the matching is expanded according to the original LZ4 compression algorithm, and the corresponding offset is stored in storage structure b1 (equivalent to the first hash table in this application). If no match is found, a 3-byte matching attempt is made. If a match is successful, the corresponding offset is stored in b2 (equivalent to the second hash table in this application). If no match is found, the current 4-byte hash information is stored in table lz4_hash, and the current 3-byte hash information is stored in table lz3_hash, following the original LZ4 method. Then, the pointer at the current position slides 1 byte, repeating the above process.

[0071] This application provides a data compression method. First, it obtains the data to be compressed and a first hash table and a second hash table corresponding to a preset dictionary compression algorithm. Then, it performs a first match on the data to be compressed according to a first byte length, determining whether the first match is successful. If the match is successful, the data to be compressed is compressed based on the first hash table to obtain the target compressed data. If the match is unsuccessful, it performs a second match on the data to be compressed according to a second byte length, determining whether the second match is successful. If the match is successful, the data to be compressed is compressed based on the second hash table. If the match is unsuccessful, the pointers of the first and second hash tables are moved forward by one byte to update the first and second hash tables. Finally, the data to be compressed is performed a first match according to the first byte length included in the updated first hash table. This method introduces a second hash table to the preset dictionary compression algorithm. The two hash tables work together to compress the data. When the second match is successful, the data can be compressed using the second hash table, improving the compression ratio of the preset compression algorithm.

[0072] Please see Figure 7 This application provides a data decompression method applied to an electronic device, the method comprising:

[0073] Step S310: Read the data stored in the first specified bits of the target compressed data.

[0074] In the embodiments of this application, the data stored in the first specified bits represents data that characterizes different storage structures.

[0075] One approach is to read the data stored in the first specified bits of the target compressed data to determine the corresponding storage structure. The number of the first specified bits can be determined based on a pre-defined dictionary compression algorithm. Different pre-defined dictionary compression algorithms may correspond to different numbers of the first specified bits. When compressing the data to be compressed using a pre-defined dictionary compression algorithm, the identifier of the corresponding pre-defined dictionary compression algorithm can be stored simultaneously with the compressed data. Therefore, when decompressing the target compressed data, the number of the first specified bits can be determined based on the identifier of the pre-defined dictionary compression algorithm.

[0076] Step S320: If the hash table corresponding to the data is the first hash table corresponding to the preset dictionary compression algorithm, the target compressed data is decompressed based on the first hash table to obtain decompressed data, wherein the first hash table is a storage structure with a length of the first byte corresponding to the preset dictionary compression algorithm.

[0077] In this embodiment of the application, if the hash table corresponding to the data stored in the specified bits is the first hash table corresponding to the preset dictionary compression algorithm, it means that the current target compressed data is stored according to the first hash table, and the target compressed data can be decompressed according to the decompression method corresponding to the preset dictionary compression algorithm.

[0078] Step S330: If the hash table corresponding to the data is the second hash table corresponding to the preset dictionary compression algorithm, the target compressed data is decompressed based on the second hash table to obtain decompressed data. The second hash table is a storage structure corresponding to the preset dictionary compression algorithm that includes a second byte length, and the first byte length is greater than the second byte length.

[0079] In this embodiment, if the hash table corresponding to the data stored in the specified bits is the second hash table corresponding to the preset dictionary compression algorithm, it means that the current target compressed data is stored according to the second hash table. Then, the data stored in other bytes of the second hash table can be read to obtain the offset information. After obtaining the offset information, the data of the second preset byte length can be copied from the corresponding place to the current position to complete the decompression of the target compressed data.

[0080] This application provides a data decompression method that reads data stored in the first specified bits of a target compressed data. If the hash table corresponding to the data is a first hash table corresponding to a preset dictionary compression algorithm, the target compressed data is decompressed based on the first hash table to obtain decompressed data. If the hash table corresponding to the data is a second hash table corresponding to a preset dictionary compression algorithm, the target compressed data is decompressed based on the second hash table to obtain decompressed data. Through this method, even when the data stored in the first specified bits of the target compressed data is different, decompression can be performed on the target compressed data based on different hash tables, improving the flexibility of target compressed data decompression.

[0081] Please see Figure 8 This application provides a data decompression method applied to an electronic device, the method comprising:

[0082] Step S410: Read the data in the first byte of the target compressed data.

[0083] In this embodiment of the application, when decompressing the target compressed data, the first byte of the target compressed data is read first, that is, the data stored in the first byte is read first.

[0084] Step S420: Read the data of the first specified bits from the data of the first byte.

[0085] In this embodiment of the application, after reading the data stored in the first byte, the first specified bits are separated from the data stored in the first byte to obtain the flag bit corresponding to the target compressed data.

[0086] Step S430: If the hash table corresponding to the data is the first hash table corresponding to the preset dictionary compression algorithm, the target compressed data is decompressed based on the first hash table to obtain decompressed data, wherein the first hash table is a storage structure with a length of the first byte corresponding to the preset dictionary compression algorithm.

[0087] Step S440: If the hash table corresponding to the data is the second hash table corresponding to the preset dictionary compression algorithm, the target compressed data is decompressed based on the second hash table to obtain decompressed data. The second hash table is a storage structure corresponding to the preset dictionary compression algorithm that includes a second byte length, and the first byte length is greater than the second byte length.

[0088] In this embodiment, if the hash table corresponding to the data stored in the first specified bit is the second hash table corresponding to the preset dictionary compression algorithm, it means that the current target compressed data is stored according to the second hash table. Then, continue to read the data stored in the remaining bytes, combine it with the data in the first byte, and obtain the offset information corresponding to the target compressed data. After obtaining the offset information corresponding to the target compressed data, the data of the second preset byte length can be copied from the corresponding place to the current position to complete the decompression of the target compressed data.

[0089] For example, the process described in steps S410-S430 can be as follows: Figure 9 As shown, the first hash table is as follows: Figure 2 As shown, the second hash table is as follows Figure 3 As shown. Decompression begins, the first byte is read, and the first bit F is obtained. If F is 1, it means that the current compressed block is compressed in b2 format (e.g., ...). Figure 2 If the data is stored in the manner shown, then continue reading 1 byte (a total of 2 bytes are read). After obtaining the offset information, copy 3 bytes from the corresponding location to the current position to complete the decompression of the current block (equivalent to the target compressed data in this application). If F is not 1, it means that the current compressed block is stored in the manner shown in b1. Then, decompress it according to the decompression steps of the LZ4 compression algorithm to complete the decompression of the current block (equivalent to the target compressed data in this application).

[0090] This application provides a data decompression method that, when the data stored in the first specified bits of the target compressed data are different, can decompress the target compressed data based on different hash tables, thereby improving the flexibility of decompressing the target compressed data.

[0091] Please see Figure 10 This application provides a data compression device 500, which includes:

[0092] The data acquisition unit 510 is used to acquire the data to be compressed.

[0093] The hash table acquisition unit 520 is used to acquire a first hash table and a second hash table corresponding to a preset dictionary compression algorithm. The first hash table is a storage structure with a length of a first byte corresponding to the preset dictionary compression algorithm, and the second hash table is a storage structure with a length of a second byte corresponding to the preset dictionary compression algorithm. The length of the first byte is greater than the length of the second byte.

[0094] Wherein, the data stored in the first specified bits of the first hash table is used as a first flag bit to identify whether the data to be compressed is compressed based on the first hash table; the data stored in the first specified bits of the second hash table is used as a second flag bit to identify whether the data to be compressed is compressed based on the second hash table.

[0095] The first matching unit 530 is used to perform a first matching on the data to be compressed according to a first preset byte length.

[0096] In one manner, the first matching unit 530 is specifically used to update the first hash table and the second hash table if the second matching fails; and to perform a first matching on the data to be compressed according to the first preset byte length corresponding to the updated first hash table.

[0097] The second matching unit 540 is used to perform a second matching on the data to be compressed according to a second preset byte length if the first matching fails.

[0098] Compression unit 550 is used to compress the data to be compressed based on the second hash table if the second match is successful, so as to obtain the target compressed data.

[0099] In one approach, the compression unit 550 is specifically used to compress the data to be compressed based on the first hash table if the first match is successful, thereby obtaining the target compressed data.

[0100] Please see Figure 11 This application provides a data decompression apparatus 600, which includes:

[0101] The data acquisition unit 610 is used to read the data stored in the first specified bits of the target compressed data.

[0102] In one manner, the data acquisition unit 610 is specifically used to read the data in the first byte of the target compressed data; and to read the data of the first specified bits from the data in the first byte.

[0103] The first decompression unit 620 is configured to decompress the target compressed data based on the first hash table if the hash table corresponding to the data is a first hash table corresponding to a preset dictionary compression algorithm, thereby obtaining decompressed data, wherein the first hash table is a storage structure including a first byte length corresponding to the preset dictionary compression algorithm.

[0104] The second decompression unit 630 is used to decompress the target compressed data based on the second hash table if the hash table corresponding to the data is the second hash table corresponding to the preset dictionary compression algorithm, so as to obtain decompressed data. The second hash table is a storage structure with a second byte length corresponding to the preset dictionary compression algorithm, and the first byte length is greater than the second byte length.

[0105] It should be noted that the device embodiments in this application correspond to the aforementioned method embodiments. The specific principles in the device embodiments can be found in the content of the aforementioned method embodiments, and will not be repeated here.

[0106] The following will combine Figure 12 This application describes an electronic device.

[0107] Please see Figure 12 Based on the aforementioned data compression or decompression method and apparatus, this application embodiment also provides another electronic device 800 capable of executing the aforementioned data compression or decompression method. The electronic device 800 includes one or more (only one shown in the figure) processors 802, a memory 804, and a network module 806 coupled together. The memory 804 stores programs capable of executing the contents of the aforementioned embodiments, and the processor 802 can execute the programs stored in the memory 804.

[0108] The processor 802 may include one or more processing cores. The processor 802 connects to various parts within the electronic device 800 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 804, and by calling data stored in the memory 804. Optionally, the processor 802 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 802 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 802 and may be implemented separately using a communication chip.

[0109] The memory 804 may include random access memory (RAM) or read-only memory (ROM). The memory 804 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 804 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the terminal 800 during use (such as phonebook data, audio and video data, chat log data, etc.).

[0110] The network module 806 is used to receive and transmit electromagnetic waves, realizing the mutual conversion between electromagnetic waves and electrical signals, thereby communicating with communication networks or other devices, such as electronic devices. The network module 806 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, SIM cards, memory, etc. The network module 806 can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices through wireless networks. The aforementioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. For example, the network module 806 can interact with base stations.

[0111] Please refer to Figure 13 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable storage medium 900 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0112] The computer-readable storage medium 900 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 900 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 900 has storage space for program code 910 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 910 may be compressed, for example, in a suitable form.

[0113] This application provides a data compression method or data decompression method, apparatus, electronic device, and storage medium. The method involves acquiring data to be compressed, obtaining a first hash table and a second hash table corresponding to a preset dictionary compression algorithm, where the first hash table is a storage structure containing a first byte length and the second hash table is a storage structure containing a second byte length, both corresponding to the preset dictionary compression algorithm. The first byte length is greater than the second byte length. The data to be compressed is then subjected to a first matching according to the first preset byte length. If the first matching fails, the data to be compressed is subjected to a second matching according to the second preset byte length. If the second matching succeeds, the data to be compressed is compressed based on the second hash table to obtain the target compressed data. This method introduces a second hash table into the preset dictionary compression algorithm. The two hash tables work together to compress the data. If the second matching succeeds, the data can be compressed using the second hash table, thus improving the compression ratio of the preset compression algorithm.

[0114] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A data compression method, characterized in that, The method includes: Obtain the data to be compressed; Obtain a first hash table and a second hash table corresponding to a preset dictionary compression algorithm. The first hash table is a storage structure with a length of a first byte corresponding to the preset dictionary compression algorithm. The data stored in the first specified bits of the first hash table is used as a first flag bit to identify whether the data to be compressed is compressed based on the first hash table. The second hash table is a storage structure with a length of a second byte corresponding to the preset dictionary compression algorithm. The length of the first byte is greater than the length of the second byte. The data stored in the first specified bits of the second hash table is used as a second flag bit to identify whether the data to be compressed is compressed based on the second hash table. The data to be compressed is matched according to a first preset byte length. If the first match fails, the data to be compressed will be matched a second time according to the second preset byte length; If the second match is successful, the data to be compressed is compressed based on the second hash table to obtain the target compressed data.

2. The method according to claim 1, characterized in that, After performing a first matching of the data to be compressed according to a first preset byte length, the process further includes: If the first match is successful, the data to be compressed is compressed based on the first hash table to obtain the target compressed data.

3. The method according to claim 1, characterized in that, If the first match fails, the process of performing a second match on the data to be compressed according to a second preset byte length further includes: If the second match fails, update the first hash table and the second hash table; The step of performing a first matching of the data to be compressed according to a first preset byte length includes: The data to be compressed is matched according to the first preset byte length corresponding to the updated first hash table.

4. A data decompression method, characterized in that, The method includes: Read the data stored in the first specified bits of the target compressed data; If the hash table corresponding to the data is the first hash table corresponding to the preset dictionary compression algorithm, the target compressed data is decompressed based on the first hash table to obtain decompressed data. The first hash table is a storage structure with a length of the first byte corresponding to the preset dictionary compression algorithm. The data stored in the first specified bits of the first hash table is used as the first flag bit to identify the data to be compressed based on the first hash table. If the hash table corresponding to the data is the second hash table corresponding to the preset dictionary compression algorithm, the target compressed data is decompressed based on the second hash table to obtain decompressed data. The second hash table is a storage structure with a second byte length corresponding to the preset dictionary compression algorithm. The first byte length is greater than the second byte length. The data stored in the first specified bits of the second hash table is used as a second flag bit to identify the compression processing of the data to be compressed based on the second hash table.

5. The method according to claim 4, characterized in that, The data stored in the first specified bits of the target compressed data includes: Read the data from the first byte of the target compressed data; Read the data from the first byte of data, starting with the specified number of bits.

6. A data compression device, characterized in that, The device includes: The data acquisition unit is used to acquire the data to be compressed; A hash table acquisition unit is used to acquire a first hash table and a second hash table corresponding to a preset dictionary compression algorithm. The first hash table is a storage structure with a length of a first byte corresponding to the preset dictionary compression algorithm. The data stored in the first specified bits of the first hash table is used as a first flag bit to identify whether the data to be compressed is compressed based on the first hash table. The second hash table is a storage structure with a length of a second byte corresponding to the preset dictionary compression algorithm. The length of the first byte is greater than the length of the second byte. The data stored in the first specified bits of the second hash table is used as a second flag bit to identify whether the data to be compressed is compressed based on the second hash table. The first matching unit is used to perform a first matching on the data to be compressed according to a first preset byte length; The second matching unit is used to perform a second matching on the data to be compressed according to a second preset byte length if the first matching fails. A compression unit is used to compress the data to be compressed based on the second hash table if the second match is successful, so as to obtain the target compressed data.

7. A data decompression apparatus, characterized in that, The device includes: The data acquisition unit is used to read the data stored in the first specified bits of the target compressed data; The first decompression unit is configured to decompress the target compressed data based on the first hash table if the hash table corresponding to the data is the first hash table corresponding to the preset dictionary compression algorithm, thereby obtaining decompressed data. The first hash table is a storage structure with a length of the first byte corresponding to the preset dictionary compression algorithm. The data stored in the first specified bits of the first hash table is used as a first flag bit to identify the data to be compressed based on the first hash table. The second decompression unit is configured to decompress the target compressed data based on the second hash table if the hash table corresponding to the data is the second hash table corresponding to the preset dictionary compression algorithm, thereby obtaining decompressed data. The second hash table is a storage structure with a second byte length corresponding to the preset dictionary compression algorithm, wherein the first byte length is greater than the second byte length, and the data stored in the first specified bits of the second hash table is used as a second flag bit to identify the compression processing of the data to be compressed based on the second hash table.

8. An electronic device, characterized in that, It includes one or more processors; one or more programs are stored in memory and configured to be executed by the one or more processors according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code, wherein the program code, when executed by a processor, performs the method according to any one of claims 1-6.

Images