A decoding method and decoding system based on the LZMA algorithm
By unifying the hardware decoding bit width and using a 32-bit sliding window for parallel decoding of length and distance, the low decoding efficiency of the existing LZMA algorithm is solved, achieving faster decoding speed and higher efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HEFEI JUNZHENG TECH CO LTD
- Filing Date
- 2025-01-21
- Publication Date
- 2026-07-21
AI Technical Summary
Existing hardware implementations of the LZMA algorithm suffer from inconsistent bitstream type identification and require multiple bit reads during decoding, resulting in low decoding efficiency. Furthermore, length and distance decoding are not performed in parallel, impacting the overall decoding speed.
By removing the probabilistic model, the bit width of hardware decoding is unified, a 32-bit sliding window is used for parallel decoding of length and distance, the number of bit stream type judgments is reduced, and multi-bit parallel decoding is achieved using a finite state machine and a multi-bit decoder.
It improves decoding speed and efficiency, reduces the number of reads required for bitstream type determination, enables parallel decoding of length and distance, and enhances overall decoding performance.
Smart Images
Figure CN122437556A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of encoding and decoding, and particularly to a decoding method and a decoding system based on the LZMA algorithm. Background Art
[0002] LZMA (Lempel-Ziv-Markov chain-Algorithm) is improved based on the famous LZ77 compression algorithm. It has a higher compression ratio than gzip and bzip2, the compressed file is smaller, and the compression of plain text files is more obvious. It is several times faster than bzip2 in decompression. This compression algorithm adds interval coding based on bit streams and Markov chain contexts and deep optimization based on dynamic programming to the LZ77 algorithm, and expands the 32K dictionary in DEFLATE to the default 8M or 16M, and theoretically can be up to 4G at most. This compression algorithm has the characteristics of high compression ratio, high decompression speed and low memory consumption. The most important thing is that this algorithm is convenient for hardware implementation.
[0003] Compression Principle of LZMA
[0004] Since there are many repeated characters in a file, and these repeated characters can be represented in the form of (distance, length), the file can be compressed by searching for repeated characters. Specific examples are as follows.
[0005] The encoding process involved in the LZMA algorithm is as Figure 1 shown and includes:
[0006] Step 1: Input the file to be compressed.
[0007] Step 2: Input the compression command and use the compression command for compression.
[0008] Step 3: Set a sliding window, and search for repeated characters of the current character within the range of the sliding window. If not found, mark it as literal, a non-repeated character; if a repeated character is found, mark it as match, and the match is encoded as (distance, length). As shown in Figure 2(a), the yellow part in the file is the repeated string part (i.e., the match encoding part). The second "easy." matches the previous repeated place. When compressed to this position, the "easy." here is three characters, while if represented by a repeated sequence pair, only two characters (distance, length) sequence pair are needed, where distance and length each occupy one character, thus playing a role in data compression. As shown in Figure 2(b), (7, 3) is the (distance, length) sequence pair of two characters, 7 is the value of distance, and 3 is the value of length.
[0009] Step 4: Perform range encoding on the distance and length values respectively. First, the distance will retain four most recently encoded distance values: distance0, distance1, distance2, and distance3. When encoding a new distance value, it will first be matched against these four values. For example, if the new distance is equal to distance0, only the length needs to be encoded, and 1101 will be added before the encoded bitstream to indicate that the distance value is rep0. If no match is found, both distance and length will be encoded. This type is called a "simple match," and 10 will be added before the bitstream to indicate that the distance value is updated.
[0010] Figure 3(a) shows the case where the distance value of the next encoded match data is equal to rep0. In this case, distance = 5, rep0 can be used, only length is encoded, and 1101 is added before the bitstream to indicate that the distance value is rep0, and then the distance value is updated. Because the distance values of the two most recently used encodings are both rep0 values, rep0 remains unchanged after the update, and rep1 is updated to rep0.
[0011] Figure 3(b) shows the case where the distance value of the next encoded match data is equal to rep1. In this case, distance = 7, rep1 can be used. Only the length is encoded, and 1110 is added before the bitstream to indicate that the distance value is rep1, and then the distance value is updated. Because the most recently used distance value is the value 7 of rep1, rep0 is updated to 7, and rep1 is updated to the value 5, representing the second to last used value.
[0012] In the above process, a total of four historical distance values need to be retained, that is, only the four closest distance values are retained. The bitstream, type and description of each code are shown in Table 1 below:
[0013] Table 1
[0014] bitstream type describe 0+Byte literal Literal stream 1+0+len+dis simplematch length-distance 1+1+0+0 shortmatch 1-byte matching character, nearest data 1+1+0+1+len rep0 len bytes match the character, the most recently used distance value 1+1+1+0+len rep1 The len byte matches the character; the second to last byte is the distance value used. 1+1+1+1+0+len rep2 The len byte matches the character; the third to last byte is the distance value used. 1+1+1+1+1+len rep3 The len byte matches the character; the fourth to last byte is the distance value used.
[0015] Step 5: Add header attribute information to form a compressed file. As shown in Figure 4(a), the header attribute information includes the dictionary size (i.e., the size of the sliding window) and the uncompressed data size, each of which is 32 bits. The compressed file format is shown in Figure 4(b).
[0016] The decoding process involved in the LZMA algorithm is as follows: Figure 5 As shown, it includes:
[0017] Step 1: When decompressing, input the compressed file into the LZMA decoder.
[0018] Step 2: Use the decompression command to decode the file. During decoding, parse the header attribute information, including Dictionary Size and Uncompressed Data Size.
[0019] Step 3: When parsing the bit stream, first parse the first bit to determine the stream type.
[0020] Step 4: If the value is 0, it indicates that the bitstream is of literal type. In this case, only the last 8 bits of the bitstream need to be decoded to obtain the corresponding character. If the value is 1, it indicates that the bitstream is of match type. In this case, the second bit needs to be decoded. If the value is 10, it indicates that it is simplematch, and the length and distance need to be decoded. If not, continue decoding the bits. If the value is 1100, it indicates that it is shortmatch. Shortmatch type is 1-byte matching character, and the distance is rep0. Continue decoding to decode rep0, rep1, rep2, and rep3.
[0021] Step 5: The decoding of match includes length and distance. When decoding, when the current compression position is reached, it is only necessary to search forward through the sliding window according to the distance and length of the match at the current position to restore the repeated string.
[0022] The decoder obtains the literal, distance, and length strings. The search for matching strings in the dictionary is then performed to combine them and output to the decompressed file.
[0023] In the existing technology, the hardware implementation scheme of the LZMA algorithm is to first read 1 bit of code stream in the state machine module to perform type judgment. If it is 0, it enters the literal decoding state. If it is 1, it enters the match type judgment and then continues to read 2 bits to perform detailed match type judgment.
[0024] This scheme requires a maximum of 4 cycles and a minimum of 2 cycles to decode a match, while the entire module takes a maximum of 10 cycles to decode a match. It can be seen that determining the match type takes up about half of the time.
[0025] In this scheme, after the bitstream type is determined, when decoding the simplematch type, the length and distance are decoded serially, because the encoded length of distance can only be known after the length is decoded. Furthermore, the existing distance decoding module decodes distance based on the encoded bits, requiring a maximum of 4 cycles for decoding, while the length decoding module operates on the same principle, requiring a maximum of 3 cycles for decoding.
[0026] In the existing hardware decoding scheme, a multi-bit decoder was designed to implement multi-bit parallel decoding. It receives 32-bit bitstream data and stores it in a FIFO. The parallel pointer is an 11-bit pointer. The module that needs to use the bitstream sends the required number of bits and a ready signal to the multi-bit decoder. Then, the multi-bit decoder transmits the bit data to the corresponding module according to the module request, pulls up the valid signal, and updates the pointer after the handshake is successful.
[0027] Therefore, the existing hardware implementation scheme of LZMA algorithm has the following drawbacks: the number of bits read by the state machine each time is inconsistent, and the bit stream needs to be read multiple times when determining the bit stream type. In addition, the length and distance need to be decoded sequentially, which reduces the decoding efficiency. At the same time, the hardware implementation scheme sets an 11-bit multi-bit decoding pointer, which cannot support the parallel decoding requirements of length and distance. Summary of the Invention
[0028] To address the aforementioned technical problems, this invention provides a decoding method and system based on the LZMA algorithm. In the decoding method of this invention, the LZMA algorithm eliminates the probabilistic model. For each original string, a data segment of a first preset length is read from the compressed bitstream, the first preset length data segment including the encoding type of the original string; the encoding type of the original string is determined based on the first preset length data segment; and the original string is parsed from the compressed bitstream according to the encoding type of the original string. Compared with existing technologies, the technical solution provided by this invention unifies the bit width of hardware decoding, reduces the number of reads required to determine the bitstream type (determination can be achieved with only one read), uses a 32-bit sliding window to implement multi-bit decoding, and decodes the length and distance of compressed characters in parallel, improving decoding speed and efficiency.
[0029] In one embodiment of the present invention, a decoding method based on the LZMA algorithm is provided, wherein the LZMA algorithm removes the probabilistic model, and the method includes:
[0030] For each original string, perform the following decoding steps:
[0031] S10, Read a data segment of a first preset length from the compressed bitstream, wherein the data segment of the first preset length includes the encoding type of the original string;
[0032] S20, determine the encoding type of the original string based on the data segment of the first preset length;
[0033] S30, the original string is parsed from the compressed bitstream according to the encoding type of the original string.
[0034] Preferably, in some embodiments, step S20 includes: determining the encoding type of the original string based on the first 7 bits of the data segment of the first preset length.
[0035] Preferably, in some embodiments, when the encoding type of the original string is a non-repeating string, the first preset length data segment is the encoding of the original string, the first preset length is 9 bits, wherein the first 1 bit is the encoding type of the original string, and the last 8 bits are the original string;
[0036] Step S30 includes:
[0037] S31, in response to determining that the encoding type of the original string is a non-repeating string, the last 8 bits of the first preset length data segment are determined as the original string.
[0038] Preferably, in some embodiments, when the encoding type of the original string is one of multiple repeating strings, the first preset length data segment further includes the encoding bit length of the original string;
[0039] The decoding step further includes:
[0040] S25, determine the number of bits used to encode the length of the original string based on the data segment of the first preset length;
[0041] Step S30 includes:
[0042] S321, in response to determining that the encoding type of the original string is a simple repeating string, the starting storage location of the distance information of the original string is determined according to the number of encoding bits of the length of the original string;
[0043] S322, the length and offset distance of the original string are parsed in parallel from the compressed bitstream according to the starting storage position of the distance information of the original string;
[0044] S323, Reconstruct the original string based on the length and offset distance of the original string;
[0045] S331, in response to determining that the encoding type of the original string is one of a variety of long repeating strings, the length of the original string is parsed from the compressed bitstream according to the number of encoding bits of the original string;
[0046] S332, Restore the original string according to the historical offset distance list and the length of the original string;
[0047] S341, in response to determining that the encoding type of the original string is a short repeating string, the original string is restored according to the historical offset distance list.
[0048] Preferably, in some embodiments, step S322 includes:
[0049] S322a, the length of the original string is parsed from the compressed bitstream based on the starting storage position of the distance information of the original string;
[0050] S322b, read a data segment of a second preset length from the compressed bitstream according to the starting storage position of the distance information of the original string, wherein the data segment of the second preset length includes the number of bits encoding the offset distance of the original string;
[0051] S322c, Determine the number of bits for the offset distance of the original string based on the data segment of the second preset length;
[0052] S322d, the offset distance of the original string is parsed from the compressed bitstream based on the number of encoded bits of the offset distance of the original string.
[0053] In another embodiment of the present invention, a decoding system based on the LZMA algorithm is provided, including a decoding module;
[0054] For each original string, the decoding module is used to:
[0055] Read a data segment of a first preset length from the compressed bitstream, wherein the data segment of the first preset length includes the encoding type of the original string;
[0056] The encoding type of the original string is determined based on the data segment of the first preset length;
[0057] The original string is parsed from the compressed bitstream according to the encoding type of the original string.
[0058] Preferably, in some embodiments, the decoding module includes a DMA module, a multi-bit decoder, a finite state machine, and a repeating string decoding unit;
[0059] The DMA module is used to: read the compressed bitstream from memory;
[0060] The multi-bit decoder is used to: receive the compressed bitstream from the DMA module, store it in the FIFO module, and transmit the compressed bitstream to the finite state machine and the repeating string decoding module respectively through the FIFO module;
[0061] For each original string, the finite state machine is used to: read a data segment of a first preset length from the compressed bitstream, the first preset length data segment including the encoding type of the original string; and determine the encoding type of the original string based on the first preset length data segment.
[0062] The repeating string decoding unit is used to parse the original string belonging to the repeating string from the compressed bitstream.
[0063] Preferably, in some embodiments, when the encoding type of the original string is a non-repeating string, the first preset length data segment is the encoding of the original string, the first preset length is 9 bits, wherein the first 1 bit is the encoding type of the original string, and the last 8 bits are the original string;
[0064] The finite state machine is further used to: determine the encoding type of the original string based on the first 7 bits of the data segment of the first preset length;
[0065] The finite state machine includes a non-repeating string decoding unit, which is used to: in response to determining that the encoding type of the original string is a non-repeating string, determine the last 8 bits of the first preset length data segment as the original string.
[0066] Preferably, in some embodiments, when the encoding type of the original string is one of multiple repeating strings, the first preset length data segment further includes the encoding bit length of the original string;
[0067] The finite state machine is also used to determine the number of bits in the encoding of the original string based on the data segment of the first preset length;
[0068] The repeating string decoding unit is further used for:
[0069] In response to determining that the encoding type of the original string is a simple repeating string, the starting storage location of the distance information of the original string is determined according to the number of encoding bits of the original string.
[0070] The length and offset distance of the original string are parsed in parallel from the compressed bitstream based on the starting storage position of the distance information of the original string;
[0071] The original string is reconstructed based on its length and offset distance.
[0072] In response to determining that the encoding type of the original string is one of a variety of long repeating strings, the length of the original string is parsed from the compressed bitstream based on the number of encoding bits of the original string.
[0073] The original string is reconstructed based on the historical offset distance list and the length of the original string;
[0074] In response to determining that the encoding type of the original string is a short repeating string, the original string is restored according to the historical offset distance list.
[0075] Preferably, in some embodiments, the FIFO module stores at most two compressed bitstreams of sliding window length simultaneously, and the multi-bit decoder is further used to: concatenate two compressed bitstreams of sliding window length; transmit one compressed bitstream of sliding window length to the module that needs to use the compressed bitstream according to the concatenated compressed bitstream and the position of the sliding window; receive the usage information of the compressed bitstream from the module that needs to use the compressed bitstream, and update the position of the sliding window according to the usage information; and read the next compressed bitstream of sliding window length in response to determining that the sliding distance of the sliding window has reached the sliding window length.
[0076] Compared with the prior art, the technical solution provided by the embodiments of the present invention unifies the bit width of hardware decoding and reading, reduces the number of readings required to determine the bit stream type, and can determine it with only one reading. It uses a 32-bit sliding window to realize multi-bit decoding, and decodes the length and distance of compressed characters in parallel, thereby improving the decoding speed and decoding efficiency. Attached Figure Description
[0077] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0078] Figure 1 A flowchart of the encoding process involved in the LZMA algorithm in the prior art;
[0079] Figure 2(a) shows an example of a repeated string portion in a file to be compressed in the prior art;
[0080] Figure 2(b) shows an example of repeating sequence pairs in a file to be compressed in the prior art;
[0081] Figure 3(a) shows an example of matching distance values and repeating characters in the next encoded match data in the prior art;
[0082] Figure 3(b) shows an example two of matching distance values and repeated characters in the next encoded match data in the prior art;
[0083] Figure 4(a) shows an example of the header attribute information format of compressed files in the prior art;
[0084] Figure 4(b) shows an example of a compressed file format in the prior art;
[0085] Figure 5 This is a flowchart of the decoding process involved in the LZMA algorithm in the prior art;
[0086] Figure 6 A flowchart example of the decoding method based on the LZMA algorithm provided by this invention;
[0087] Figure 7 A structural diagram example of the decoding system based on the LZMA algorithm provided by this invention;
[0088] Figure 8 This is an example of a hardware implementation scheme for a decoding system based on the LZMA algorithm provided by the present invention;
[0089] Figure 9 An example of a state diagram of a finite state machine in a decoding system based on the LZMA algorithm provided by this invention;
[0090] Figure 10(a) is an example of the decoding timing diagram of the length module in the decoding system based on the LZMA algorithm provided by the present invention;
[0091] Figure 10(b) is an example of the decoding timing diagram of the distance module in the decoding system based on the LZMA algorithm provided by the present invention;
[0092] Figures 11(a)-11(d) This is an example of the sliding window operation of the multi-bit decoding module in the LZMA-based decoding system provided by the present invention.
[0093] Figure 12 This is an example of the sliding window update process of the multi-bit decoding module in the LZMA-based decoding system provided by the present invention;
[0094] Figure 13(a) shows a comparison of the decoding system hardware implementation scheme provided by the present invention and the existing technology implementation scheme in terms of the cycle required for decoding the bit stream;
[0095] Figure 13(b) shows a comparison of the decoding time of the hardware implementation scheme of the decoding system provided by the present invention and the existing technology implementation scheme for decoding the same file;
[0096] Figure 13(c) shows a comparison of the time required to decode the same bitstream type between the hardware implementation scheme of the decoding system provided by the present invention and the existing technology implementation scheme. Detailed Implementation
[0097] To better understand the above-mentioned objectives, features, and advantages of the present invention, embodiments of the present invention will be further described below. It should be noted that, unless otherwise specified, embodiments of the present invention and features thereof can be combined with each other.
[0098] Numerous specific details are set forth in the following description to provide a thorough understanding of the invention, but the invention may also be practiced in other ways than those described herein. Clearly, the embodiments described in the specification are only a portion, and not all, of the embodiments of the invention.
[0099] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to the process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0100] The above description is merely an embodiment of the present invention, which enables those skilled in the art to understand and implement the invention. Various modifications to the embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
[0101] The terms and their interpretations in this invention are as follows:
[0102] LZMA: Lempel-Ziv-Markov chain-Algorithm, is a lossless compression algorithm that is an improvement and optimization of the deflate and lz77 algorithms;
[0103] LZ77: A general algorithm for sequential compression created by Lempel and Ziv in 1977;
[0104] literal: The type of non-repeating characters matched during decoding;
[0105] match: The type of repeating characters matched during decoding;
[0106] length: The length of the repeated string in the repeated character sequence pairs matched during decoding;
[0107] distance: The distance between repeated strings in the matched repeated character sequence pairs during decoding;
[0108] Dictionary Size: The size of the sliding window used during decoding, which is used to search for the length of duplicate strings during encoding and decoding;
[0109] Uncompressed Data Size: File size before compression;
[0110] shortmatch: When decoding, the matched repeating character is 1 byte long and the distance is the most recently used repeating character;
[0111] rep0: The repeated character matched during decoding is len bytes long and the distance is the most recently used distance value.
[0112] rep1: The repeated character matched during decoding is len bytes, and the distance is the second to last distance value used for repeated characters;
[0113] rep2: The repeated character matched during decoding is len bytes, and the distance is the third-to-last repeated character used in the distance value;
[0114] rep3: The repeated character matched during decoding is len bytes, and the distance is the fourth to last repeated character using the distance value.
[0115] valid: The data is valid;
[0116] ready: The receiving end is ready to receive data signals;
[0117] Bit_Stream: The bitstream data of the compressed file;
[0118] Lossless compression: Compression is achieved by utilizing the statistical redundancy of data, which can completely restore the original data without causing any distortion.
[0119] In one embodiment, the present invention provides a decoding method based on the LZMA algorithm, wherein the LZMA algorithm removes the probabilistic model, such as... Figure 6 As shown, the method includes:
[0120] For each original string, perform the following decoding steps:
[0121] S10, Read a data segment of a first preset length from the compressed bitstream, wherein the data segment of the first preset length includes the encoding type of the original string;
[0122] S20, determine the encoding type of the original string based on the data segment of the first preset length;
[0123] S30, the original string is parsed from the compressed bitstream according to the encoding type of the original string.
[0124] Preferably, in some embodiments, step S20 includes: determining the encoding type of the original string based on the first 7 bits of the data segment of the first preset length.
[0125] Preferably, in some embodiments, when the encoding type of the original string is a non-repeating string, the first preset length data segment is the encoding of the original string, the first preset length is 9 bits, wherein the first 1 bit is the encoding type of the original string, and the last 8 bits are the original string;
[0126] Step S30 includes:
[0127] S31, in response to determining that the encoding type of the original string is a non-repeating string, the last 8 bits of the first preset length data segment are determined as the original string.
[0128] Preferably, in some embodiments, when the encoding type of the original string is one of multiple repeating strings, the first preset length data segment further includes the encoding bit length of the original string;
[0129] The decoding step further includes:
[0130] S25, determine the number of bits used to encode the length of the original string based on the data segment of the first preset length;
[0131] Step S30 includes:
[0132] S321, in response to determining that the encoding type of the original string is a simple repeating string, the starting storage location of the distance information of the original string is determined according to the number of encoding bits of the length of the original string;
[0133] S322, the length and offset distance of the original string are parsed in parallel from the compressed bitstream according to the starting storage position of the distance information of the original string;
[0134] S323, Reconstruct the original string based on the length and offset distance of the original string;
[0135] S331, in response to determining that the encoding type of the original string is one of a variety of long repeating strings, the length of the original string is parsed from the compressed bitstream according to the number of encoding bits of the original string;
[0136] S332, Restore the original string according to the historical offset distance list and the length of the original string;
[0137] S341, in response to determining that the encoding type of the original string is a short repeating string, the original string is restored according to the historical offset distance list.
[0138] Preferably, in some embodiments, step S322 includes:
[0139] S322a, the length of the original string is parsed from the compressed bitstream based on the starting storage position of the distance information of the original string;
[0140] S322b, read a data segment of a second preset length from the compressed bitstream according to the starting storage position of the distance information of the original string, wherein the data segment of the second preset length includes the number of bits encoding the offset distance of the original string;
[0141] S322c, Determine the number of bits for the offset distance of the original string based on the data segment of the second preset length;
[0142] S322d, the offset distance of the original string is parsed from the compressed bitstream based on the number of encoded bits of the offset distance of the original string.
[0143] Preferably, in some embodiments, the second preset length is 21 bits.
[0144] In another embodiment, the present invention provides a decoding system based on the LZMA algorithm, such as... Figure 7 As shown, the decoding system includes a decoding module 100;
[0145] For each original string, the decoding module 100 is used to:
[0146] Read a data segment of a first preset length from the compressed bitstream, wherein the data segment of the first preset length includes the encoding type of the original string;
[0147] The encoding type of the original string is determined based on the data segment of the first preset length;
[0148] The original string is parsed from the compressed bitstream according to the encoding type of the original string.
[0149] Preferably, in some embodiments, the decoding module includes a DMA module 101, a multi-bit decoder 102, a finite state machine 103, and a repeating string decoding unit 104;
[0150] The DMA module 101 is used to: read the compressed bitstream from memory;
[0151] The multi-bit decoder 102 is used to: receive the compressed bitstream from the DMA module 101, store it in the FIFO module, and transmit the compressed bitstream to the finite state machine 103 and the repeating string decoding module 104 through the FIFO module respectively.
[0152] For each original string, the finite state machine 103 is used to: read a data segment of a first preset length from the compressed bitstream, the first preset length data segment including the encoding type of the original string; and determine the encoding type of the original string based on the first preset length data segment.
[0153] The repeating string decoding unit 104 is used to parse the original string belonging to the repeating string from the compressed bitstream.
[0154] Preferably, in some embodiments, when the encoding type of the original string is a non-repeating string, the first preset length data segment is the encoding of the original string, the first preset length is 9 bits, wherein the first 1 bit is the encoding type of the original string, and the last 8 bits are the original string;
[0155] The finite state machine 103 is further used to: determine the encoding type of the original string based on the first 7 bits of the data segment of the first preset length;
[0156] The finite state machine 103 includes a non-repeating string decoding unit, which is used to: in response to determining that the encoding type of the original string is a non-repeating string, determine the last 8 bits of the first preset length data segment as the original string.
[0157] Preferably, in some embodiments, when the encoding type of the original string is one of multiple repeating strings, the first preset length data segment further includes the encoding bit length of the original string;
[0158] The finite state machine 103 is also used to determine the number of bits in the encoding of the original string based on the data segment of the first preset length;
[0159] The repeating string decoding unit 104 is further used for:
[0160] In response to determining that the encoding type of the original string is a simple repeating string, the starting storage location of the distance information of the original string is determined according to the number of encoding bits of the original string.
[0161] The length and offset distance of the original string are parsed in parallel from the compressed bitstream based on the starting storage position of the distance information of the original string;
[0162] The original string is reconstructed based on its length and offset distance.
[0163] In response to determining that the encoding type of the original string is one of a variety of long repeating strings, the length of the original string is parsed from the compressed bitstream based on the number of encoding bits of the original string.
[0164] The original string is reconstructed based on the historical offset distance list and the length of the original string;
[0165] In response to determining that the encoding type of the original string is a short repeating string, the original string is restored according to the historical offset distance list.
[0166] Preferably, in some embodiments, the FIFO module stores at most two compressed bitstreams of sliding window length simultaneously, and the multi-bit decoder 102 is further configured to: concatenate two compressed bitstreams of sliding window length; transmit one compressed bitstream of sliding window length to the module that needs to use the compressed bitstream according to the concatenated compressed bitstream and the position of the sliding window; receive the usage information of the compressed bitstream from the module that needs to use the compressed bitstream, and update the position of the sliding window according to the usage information; and read the next compressed bitstream of sliding window length in response to determining that the sliding distance of the sliding window has reached the sliding window length.
[0167] Preferably, in some embodiments, the sliding window length is 32 bits.
[0168] Compared with the prior art, the technical solution provided by the embodiments of the present invention unifies the bit width of hardware decoding and reading, reduces the number of readings required to determine the bit stream type, and can determine it with only one reading. It uses a 32-bit sliding window to realize multi-bit decoding, and decodes the length and distance of compressed characters in parallel, thereby improving the decoding speed and decoding efficiency.
[0169] The hardware implementation scheme of the decoding system based on the LZMA algorithm proposed in this invention will be described in detail below, taking into account specific application scenarios.
[0170] like Figure 8 As shown, in the decoding scheme of this invention, Length and Distance decoding are merged into MATCH decoding, and parallel decoding of Length and Distance can be achieved, decoding data in one cycle. Furthermore, Literal decoding is directly merged into the LZMA_fsm module, determining the literal type and directly writing the literal to the subsequent module. The scheme of this invention unifies all internal interfaces to 32 bits, transmitting 32 bits of data to each module each time, and then updating the data according to the number of modules used.
[0171] According to the bitstream determination scheme in the LZMA_fsm module of this invention, the FSM state machine directly determines the bitstream type in the START state. All bitstream types can be determined in one cycle. Based on bitstream determination table 2, 9 bits of data are received in the START state, and the bitstream type is obtained by directly comparing the first 7 bits. Then, the machine jumps to the corresponding state to continue decoding. For example, for Literal, it directly jumps to the WR state to transmit the remaining 8 bits of data and returns it to the multi-bit decoding module, consuming 9 bits. The multi-bit decoding module then moves the sliding window forward by 9 bits to update it with new 32-bit data. This direct approach shortens the bitstream determination process, which requires a maximum of 4 cycles in the prior art, to 1 cycle, and allows for determination of any type in just 1 cycle.
[0172] Table 2
[0173]
[0174] In this invention, the states of the FSM state machine are as follows: Figure 9 As shown.
[0175] ① In START state:
[0176] i. Determine the bitstream type and perform a status transition.
[0177] ii. If it is a Literal or Short_match, jump directly to the WR state to write it out; for other types, jump to the state and continue decoding.
[0178] iii. Calculate the total number of bits consumed.
[0179] ② In the Simple_match state:
[0180] i. Send start signals to the Length module and Distance module, and pull the corresponding busy signals high.
[0181] ii. Wait for Length and Distance to finish decoding, receive the done signal, and jump to the WR state.
[0182] ③ In the Rep0_match||Rep1_match||Rep2_match||Rep3_match state:
[0183] i. Send a start signal to the Length module and pull the busy signal high.
[0184] ii. Wait for the done signal indicating that Length decoding is complete, then transition to the WR state.
[0185] The present invention combines the Length and Distance modules for parallel decoding. As shown in Bitstream Judgment Table 2, the first 7 bits reveal not only the bitstream type but also the number of bits used for length encoding, thus enabling parallel decoding of both length and distance. The parallel decoding process is as follows: After receiving 32 bits of data from the multi-bit decoder, the MATCH module decodes the length and distance based on the len_number data transmitted via the FSM. In parallel decoding, for the Length module, dec_bit directly sends the data required for decoding. dec_bit can calculate the start position of the Distance module's bitstream based on len_numbist. 21 bits are directly read from the start position and sent to the Distance decoding module. After distance decoding is complete, the module returns the number of bits used for decoding, and dec_bit updates its pointer accordingly for the next decoding transmission. The optimized distance and length can both be calculated in one cycle.
[0186] The decoding timing diagrams of the Length module and Distance module in this invention are as follows: Figure 10(a) and 10(b) As shown.
[0187] In this invention, the multi-bit decoding module replaces the multi-bit pointer with a sliding window for multi-bit decoding and updating. The multi-bit decoder first stores the received data in a FIFO, waiting for data requests from various modules. Then, the output concatenates two 32-bit numbers in the FIFO and uses 32 output pointers to transmit 32-bit data to each module. Once one 32-bit data is used up, the next 32-bit data is received, thus improving transmission efficiency. The window is updated only after other modules have used the data.
[0188] The implementation of the sliding window under the new scheme is as follows: Figures 11(a)-11(d) As shown.
[0189] See Figure 11(a) and 11(b) Initially, the FIFO is empty. Two data entries are read from the DMA to fill the FIFO. After the first data entry is read, the FIFO is no longer empty, and the decoding of the bitstream begins.
[0190] Referring to Figure 11(c), the interface for reading the code stream is 32-bit. The FIFO is full in the next clock cycle, and the code stream transmission continues.
[0191] Referring to Figure 11(d), when one of the 32 bits is consumed, the FIFO is not full, and data is read from the DMA for data processing.
[0192] The sliding window update in the present invention is as follows: Figure 12 As shown, the sliding window length is 32 bits. Each time the window is updated, the starting position of the window is moved to the right by the number of bits consumed in this decoding.
[0193] Figures 13(a)-13(c) The diagram illustrates a comparison between the hardware implementation of the decoding system of the present invention and existing implementations in terms of the number of cycles required to decode the bitstream, the decoding time for the same file, and the time required to decode the same bitstream type. In Figure 13(c), T represents the number of cycles required to decode this file. It is evident that, without increasing the hardware area, the hardware implementation of the decoding system of the present invention significantly improves the decoding speed and efficiency of the bitstream.
[0194] Compared with the prior art, the technical solution provided by the embodiments of the present invention unifies the bit width of hardware decoding and reading, reduces the number of readings required to determine the bit stream type, and can determine it with only one reading. It uses a 32-bit sliding window to realize multi-bit decoding, and decodes the length and distance of compressed characters in parallel, thereby improving the decoding speed and decoding efficiency.
Claims
1. A decoding method based on the LZMA algorithm, wherein the LZMA algorithm removes the probabilistic model, characterized in that, The method includes: For each original string, perform the following decoding steps: S10, Read a data segment of a first preset length from the compressed bitstream, wherein the data segment of the first preset length includes the encoding type of the original string; S20, determine the encoding type of the original string based on the data segment of the first preset length; S30, the original string is parsed from the compressed bitstream according to the encoding type of the original string.
2. The decoding method based on the LZMA algorithm according to claim 1, characterized in that, Step S20 includes: The encoding type of the original string is determined based on the first 7 bits of the data segment of the first preset length.
3. The decoding method based on the LZMA algorithm according to claim 2, characterized in that, When the encoding type of the original string is a non-repeating string, the first preset length data segment is the encoding of the original string. The first preset length is 9 bits, where the first bit is the encoding type of the original string and the last 8 bits are the original string. Step S30 includes: S31, in response to determining that the encoding type of the original string is a non-repeating string, the last 8 bits of the first preset length data segment are determined as the original string.
4. The decoding method based on the LZMA algorithm according to claim 1, characterized in that, When the encoding type of the original string is one of multiple repeating strings, the first preset length data segment also includes the encoding bit length of the original string; The decoding step further includes: S25, determine the number of bits used to encode the length of the original string based on the data segment of the first preset length; Step S30 includes: S321, in response to determining that the encoding type of the original string is a simple repeating string, the starting storage location of the distance information of the original string is determined according to the number of encoding bits of the length of the original string; S322, the length and offset distance of the original string are parsed in parallel from the compressed bitstream according to the starting storage position of the distance information of the original string; S323, Reconstruct the original string based on the length and offset distance of the original string; S331, in response to determining that the encoding type of the original string is one of a variety of long repeating strings, the length of the original string is parsed from the compressed bitstream according to the number of encoding bits of the original string; S332, Restore the original string according to the historical offset distance list and the length of the original string; S341, in response to determining that the encoding type of the original string is a short repeating string, the original string is restored according to the historical offset distance list.
5. The decoding method based on the LZMA algorithm according to claim 4, characterized in that, Step S322 includes: S322a, the length of the original string is parsed from the compressed bitstream based on the starting storage position of the distance information of the original string; S322b, read a data segment of a second preset length from the compressed bitstream according to the starting storage position of the distance information of the original string, wherein the data segment of the second preset length includes the number of bits encoding the offset distance of the original string; S322c, Determine the number of bits for the offset distance of the original string based on the data segment of the second preset length; S322d, the offset distance of the original string is parsed from the compressed bitstream based on the number of encoded bits of the offset distance of the original string.
6. A decoding system based on the LZMA algorithm, characterized in that, Includes a decoding module; For each original string, the decoding module is used to: Read a data segment of a first preset length from the compressed bitstream, wherein the data segment of the first preset length includes the encoding type of the original string; The encoding type of the original string is determined based on the data segment of the first preset length; The original string is parsed from the compressed bitstream according to the encoding type of the original string.
7. A decoding system based on the LZMA algorithm according to claim 6, characterized in that, The decoding module includes a DMA module, a multi-bit decoder, a finite state machine, and a repeating string decoding unit; The DMA module is used to: read the compressed bitstream from memory; The multi-bit decoder is used to: receive the compressed bitstream from the DMA module, store it in the FIFO module, and transmit the compressed bitstream to the finite state machine and the repeating string decoding module respectively through the FIFO module; For each original string, the finite state machine is used to: read a data segment of a first preset length from the compressed bitstream, the first preset length data segment including the encoding type of the original string; and determine the encoding type of the original string based on the first preset length data segment. The repeating string decoding unit is used to parse the original string belonging to the repeating string from the compressed bitstream.
8. A decoding system based on the LZMA algorithm according to claim 7, characterized in that, When the encoding type of the original string is a non-repeating string, the first preset length data segment is the encoding of the original string. The first preset length is 9 bits, where the first bit is the encoding type of the original string and the last 8 bits are the original string. The finite state machine is further used to: determine the encoding type of the original string based on the first 7 bits of the data segment of the first preset length; The finite state machine includes a non-repeating string decoding unit, which is used to: in response to determining that the encoding type of the original string is a non-repeating string, determine the last 8 bits of the first preset length data segment as the original string.
9. A decoding system based on the LZMA algorithm according to claim 7, characterized in that, When the encoding type of the original string is one of multiple repeating strings, the first preset length data segment also includes the encoding bit length of the original string; The finite state machine is also used to determine the number of bits in the encoding of the original string based on the data segment of the first preset length; The repeating string decoding unit is further used for: In response to determining that the encoding type of the original string is a simple repeating string, the starting storage location of the distance information of the original string is determined according to the number of encoding bits of the original string. The length and offset distance of the original string are parsed in parallel from the compressed bitstream based on the starting storage position of the distance information of the original string; The original string is reconstructed based on its length and offset distance. In response to determining that the encoding type of the original string is one of a variety of long repeating strings, the length of the original string is parsed from the compressed bitstream based on the number of encoding bits of the original string. The original string is reconstructed based on the historical offset distance list and the length of the original string; In response to determining that the encoding type of the original string is a short repeating string, the original string is restored according to the historical offset distance list.
10. A decoding system based on the LZMA algorithm according to claim 7, characterized in that, The FIFO module stores a maximum of two compressed bitstreams of sliding window length simultaneously. The multi-bit decoder is further used for: splicing two compressed bitstreams of sliding window length; transmitting one compressed bitstream of sliding window length to the module that needs to use the compressed bitstream based on the spliced compressed bitstream and the position of the sliding window; receiving the usage information of the compressed bitstream from the module that needs to use the compressed bitstream, and updating the position of the sliding window based on the usage information; and reading the next compressed bitstream of sliding window length in response to determining that the sliding distance of the sliding window has reached the sliding window length.