Device for processing received data

Abstract

The present invention relates to a device, a storage system and a method for speeding up the decoding of interval coding. A decompression circuit performs a process of calculating a plurality of candidate bit values ​​of a bit of an N-bit string in parallel based on a plurality of bit history records that can be obtained for bits before the K-th bit, and repeatedly performs a process of selecting a correct bit value of the K-th bit from a plurality of candidate bit values ​​based on the correct bit history records before the K-th bit, so as to decode the N-bit string.

Description

Technical Field

[0001] The present invention relates to a technique for decoding input codes compressed based on interval coding. Background Art

[0002] As a storage system of an information device for storing and managing a large amount of data, if more data is stored, the cost per unit capacity can be reduced. Therefore, some storage systems are equipped with a function of compressing the written data and storing it in the disk drive.

[0003] For example, in recent years, as a storage medium of a storage system, in addition to or instead of a HDD (Hard Disk Drive), an SSD (Solid State Drive) equipped with a NAND flash memory as a non-volatile semiconductor memory is used. An SSD does not have a physical head seek mechanism like an HDD in data access, so the initial delay (latency) is small and has excellent response performance in random data reading.

[0004] Therefore, in applications such as databases that require high-speed random reads, the storage system is replacing HDDs with SSDs as storage media. However, the bit cost of SSDs is decreasing year by year as the flash memory cells are highly integrated, but it is still about three times higher than the bit cost of HDDs.

[0005] Therefore, most storage systems that use SSD as storage medium have the function of introducing reversible compression technology to reduce the size of data stored in SSD. This can virtually make the system's storage capacity appear larger, reduce the cost per unit capacity, and approach storage systems that use HDD as storage medium.

[0006] When a read request for compressed data is issued from the host to the storage system, the compressed data is decompressed, restored to the original plaintext data, and then returned to the host. It is preferred to decompress the compressed data as quickly as possible so that the read response time at this time does not deteriorate significantly compared to the case of reading uncompressed data.

[0007] The compression algorithm used in the data compression function of the storage system has a higher compression rate, which can reduce the cost per unit capacity. The LZMA algorithm is a reversible data compression algorithm known to have a high compression rate. This algorithm combines arithmetic codes called interval coding with sliding word typical compression.

[0008] In the interval coding process, a multiplication operation is required every time 1 bit is input (when encoding) or output (when decoding). Therefore, the bit rate of the interval coding process is very slow. Although a table can be referenced for the values ​​used in the multiplication in the interval coding process, the index used for reference is determined by the bit history of the input (when encoding) or output (when decoding). If the multiplication operation in the interval coding process can be performed in parallel with multiple bits, its performance is improved, and the processing performance of the LZMA algorithm is improved.

[0009] Regarding the encoding process (during compression) of interval encoding, as described in Non-Patent Document 1, there is a known technique that prepares bit history records in advance, parallelizes table references, and performs multiplication operations on multiple bits in parallel to increase speed. If the degree of parallelism is set to N, the encoding performance is improved by N times. As a result, the compression process of LZMA can be accelerated.

[0010] Prior art literature

[0011] Non-patent literature

[0012] Non-patent document 1: "A Parallel Adaptive Range Coding Compressor: Algorithm, FPGA Prototype, Evaluation", Ivan Shcherbakov and Norbert Wehn, Data Compression Conference, 2012. Summary of the invention

[0013] Problems to be solved by the invention

[0014] Regarding decoding of interval codes (decompression), since the bit history is uncertain until the previous bits are decoded, it is difficult to perform table reference and multiplication using the reference values ​​on a plurality of bits in parallel.

[0015] Therefore, for example, when the interval coding is applied to the compression / decompression function of the storage system, the decompression performance is low, so the read response time for a read request from the host increases, and the convenience of the storage system may be deteriorated.

[0016] Technical solutions to solve problems

[0017] One embodiment of the present invention is a device for processing received data, comprising: a circuit for receiving an input code compressed based on interval coding; and a decompression circuit for decompressing a part or all of the input code to decode an N-bit string, wherein N represents an integer greater than 1, K represents integers from 1 to N, and the bit value of the Kth bit of the input code is decoded based on the bit history record of bits preceding the Kth bit. The decompression circuit, for candidate bit values ​​of bits of the N-bit string, performs processing in parallel on multiple bits to respectively calculate multiple candidate bit values ​​based on multiple bit history records that can be obtained for bits preceding the Kth bit, and repeatedly performs processing to select the correct bit value of the Kth bit from the multiple candidate bit values ​​based on the correct bit history record preceding the Kth bit to decode the N-bit string.

[0018] Effects of the Invention

[0019] According to one embodiment of the present invention, it is possible to speed up the decoding of interval codes. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This section shows an example of a storage system configuration.

[0021] Figure 2A This shows an overview of the LZMA algorithm.

[0022] Figure 2B A specific example of dictionary compression processing is shown.

[0023] Figure 3A Functional block diagram showing interval encoding.

[0024] Figure 3B Functional block diagram showing interval decoding.

[0025] Figure 4A An example for explaining the principle of interval coding is shown.

[0026] Figure 4B Another example for explaining the principle of interval coding is shown.

[0027] Figure 5A A flowchart showing the section encoding process.

[0028] Figure 5B A flowchart showing the section decoding process.

[0029] Figure 6 This is a functional block diagram showing a method for speeding up section encoding processing.

[0030] Figure 7 A flowchart showing a method for speeding up section encoding processing.

[0031] Figure 8 This is a functional block diagram showing a method for speeding up section decoding processing.

[0032] Fig. 9 A flowchart showing a method for speeding up section decoding processing. DETAILED DESCRIPTION

[0033] For the sake of convenience, the following description is divided into multiple parts or embodiments when necessary, but except for the cases that are specifically stated, they are not unrelated to each other, but one is a modification, detail, supplementary description, etc. of part or all of the other. In addition, below, when the number of elements (including number, value, amount, range, etc.) is mentioned, except for the cases that are specifically stated and the cases that are clearly limited to a specific number in principle, it is not limited to the specific number, and can be more than or less than the specific number.

[0034] (1) System structure

[0035] Next, as one embodiment of this specification, a storage system having a data compression function is described. The storage system reduces the amount of stored data by reversible compression. The decoding of the interval code described in this specification can be applied to a system different from the storage system, such as a communication system.

[0036] Figure 1 The storage system 101 includes a host I / F (Interface) 102, a storage controller 103, a plurality of SSDs (Solid State Drives) 105, and a cache 106 using a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0037] The storage controller 103 is connected to the host I / F 102, SSD 105, and cache 106, and includes a microprocessor for controlling them. The microprocessor performs content interpretation of read / write instructions from the host (not shown), data transmission and reception with the host, compression and decompression of data based on the LZMA compression and decompression circuit 104, and data transmission with the SSD 105 or cache 106.

[0038] The host I / F 102 is an interface mechanism for connecting to an external host, and responds to a read / write command to send or receive data to or from the host. The mechanism of the host I / F 102 and the protocol for sending and receiving commands and data comply with, for example, a standard interface standard.

[0039] The storage controller 103 includes an LZMA compression / decompression circuit 104 and a transmission circuit 108. The transmission circuit 108 receives and transmits data compressed or decompressed by the LZMA compression / decompression circuit 104. The transmission circuit 108 transmits data between components of the storage system 101, for example, between the LZMA compression / decompression circuit 104 and the cache 106. In order to reduce the amount of data stored in the SSD 105 as a storage drive, the LZMA compression / decompression circuit 104 reversibly compresses the received write data according to a write instruction to generate compressed data. In addition, in order to send the original plaintext data to the host according to a read instruction, the compressed data read from the SSD 105 is decompressed to generate plaintext data.

[0040] The write data from the host is first temporarily stored in the cache 106. At this time, the storage controller 103 returns the write completion to the host. After that, it is converted into compressed data by the LZMA compression and decompression circuit 104, and the compressed data is also temporarily stored in the cache 106. Then, the compressed data is written to the SSD 105.

[0041] On the other hand, the read data to the host is read from the SSD 105 in a compressed state and first temporarily stored in the buffer 106. Thereafter, it is converted into plain text data by the LZMA compression / decompression circuit 104, and the plain text data is also temporarily stored in the buffer 106. Then, the plain text data is sent to the host.

[0042] In this way, the data writing is compressed after the return writing is completed, so the write performance that the host can see is constant regardless of whether it is compressed or not, but the data reading is not completed before the return of data to the host is completed, so the read response performance that the host can see depends on the decompression time of the compressed data. That is, the LZMA compression and decompression circuit 104 requires high-performance decompression processing.

[0043] For example, the LZMA compression and decompression circuit 104 is implemented as hardware (logic circuit) designed based on a data decompression method of an embodiment of this specification. Since the LZMA compression and decompression circuit 104 has high-speed data decompression performance, the storage system 101 can not only apply the high-speed random read performance that is a characteristic of the SSD to uncompressed data, but also effectively utilize it for compressed data. The functions of the LZMA compression and decompression circuit 104 can also be implemented by multiple processing devices that execute programs. The processing device includes a processor, a processor core, a central processing unit, etc. A storage drive of a different type from the SSD, such as an HDD (Hard Disk Drive), can also be used.

[0044] (2) LZMA algorithm

[0045] As a prerequisite for explaining a data decompression method of one embodiment of this specification, use FIG. 2A to FIG. 5B The LZMA algorithm is described.

[0046] (2-1) Overview of the LZMA algorithm

[0047] Figure 2A 2 shows an overview of the LZMA algorithm. In the LZMA compression process, plain text data 201 before compression is first subjected to a dictionary compression process 202. Thereafter, the dictionary compression result is subjected to an interval encoding process 203. Thus, LZMA compressed data 204 is generated.

[0048] On the other hand, in the LZMA decompression process, the compressed data 204 is first applied to the interval decoding process 205. Thereafter, the decoding result is applied to the plaintext expansion process 206. Thus, the original plaintext data 201 is generated.

[0049] (2-2) Dictionary Compression Processing

[0050] Figure 2B A specific example of dictionary compression processing 202 constituting the LZMA algorithm is shown. In the character string stream of the plain text data 201, it is checked whether the same character string appears again. If a character string is continuous with L characters starting from the J character, the character string is converted into a duplicate symbol [L, J].

[0051] For example, a string 211 of four characters "b,c,d,e" has four characters that match each other starting from the first character "b" and six characters ago. In this case, the string 211 is converted into the duplicate symbol [4,6]. Similarly, a string 212 of four characters "a,b,a,b" has four characters that match each other starting from the first character "a" and two characters ago (including overlapping parts). In this case, the string 212 is converted into the duplicate symbol [4,2].

[0052] Similarly, the string 213 of 4 characters "c, d, e, f" starts with the initial character "c", and the 4 characters are consistent from the 14th character. In this case, the string 213 is converted into duplicate symbols [4, 14]. The amount of data of these duplicate symbols is less than the amount of data of the original string, so the amount of data can be reduced by this conversion.

[0053] The range of the string stream (hereinafter referred to as the dictionary) referenced in the match search is set to the range from one character before to the specified number of characters before. Each time the search is performed, the dictionary range slides backward, so this compression technology is also called sliding character typical compression. In addition, when there are multiple matching strings within the dictionary range, the continuous and longest matching strings are converted into duplicate symbols. This has the effect of reducing the amount of data even more.

[0054] In order to generate data to be input to the subsequent section encoding process 203, it is necessary to encode characters not converted into duplicate symbols (hereinafter referred to as literal characters) and duplicate symbols in a predetermined bit pattern and concatenate them into a bit stream.

[0055] Figure 2B A bit stream representing the result of encoding according to the rules of the LZMA specification. The bit stream is input to the interval encoding process 203. For example, the bit pattern 221 represents the copy symbol [4,6] with a length of 12 bits. The bit pattern 222 represents the copy symbol [4,2] with a length of 11 bits. The bit pattern 223 represents the copy symbol [4,14] with a length of 13 bits. In this way, the length of the bit pattern corresponding to the copy symbol is not fixed. On the other hand, the literal character is represented by a bit pattern with a length of 9 bits with 1 bit 0 added to the beginning of the 8-bit value of the character.

[0056] The interval decoding process 205 outputs such a bit stream in the LZMA decompression process. In the plain text expansion process 206, when such a bit stream is input, it is interpreted as a copy symbol and a literal character, and the character string stream of the plain text data 201 is restored.

[0057] (2-3) Section encoding / decoding processing

[0058] Figure 3A A functional block diagram showing interval coding, Figure 3B This is a functional block diagram showing the section decoding. Figure 3A The interval encoding function 300 is described. The encoder 301 is a computation block that receives input from an input bit string 302 in units of 1 bit and generates an output code 303. Figure 3A In the example, the input bit string is "1,1,0,1". The method for generating the output code 303 uses Figure 4A and 4B This will be explained in the next section. Output code 303 is equivalent to the compressed data of the LZMA algorithm ( Figure 2A LZMA204 compressed data).

[0059] The encoder 301 also uses as input a probability value referenced from a probability table 304. This probability value P(x) represents the probability that the next input bit from the input bit string 302 is "0" when the previously input bit history 305 is x.

[0060] The encoder 301 performs adaptation based on learning each time the probability value P(x) is used. For example, if the next input bit is actually "0", P(x) is increased, and if it is "1", P(x) is decreased. In addition, at the beginning of encoding, all the values ​​of P(x) in the unused state are 0.5 (the probability of "0" and "1" is equal).

[0061] Next, refer to Figure 3B The interval decoding function 310 is described. The decoder 311 is a computation block that receives an input code 312 and generates an output bit string 313 in units of 1 bit. The generation method uses Figure 4A and 4B This will be described in the next section. The output bit stream 313 corresponds to the input bit stream to the plaintext expansion process (plaintext expansion process 206 in FIG2 ) of the LZMA algorithm, that is, the output bit stream from the dictionary compression process 202 in FIG2 .

[0062] The decoder 311 also uses as input a probability value referenced from the probability table 314. This probability value P(x) represents the probability that the next output bit is "0" when the bit history 315 outputted previously is x.

[0063] As with the encoder 301, the decoder 311 performs adaptation based on learning each time the probability value P(x) is used. For example, if the next output bit is actually "0", P(x) is increased, and if it is "1", P(x) is decreased. In addition, at the beginning of decoding, the value of all P(x) in the unused state is 0.5 (the probability of "0" and "1" is equal).

[0064] When the output code 303 of the section encoding function 300 and the input code 312 of the section decoding function 310 are the same, the changes due to learning of all probability values ​​P(x) in the probability tables 304 and 314 are the same. Therefore, the changes during encoding are reproduced during decoding.

[0065] (2-4) Principles of interval encoding / decoding

[0066] Figure 4A and 4B An example is shown for explaining the principle of interval encoding / decoding. Figure 3AThe encoding process performed by the encoder 301 repeatedly performs the following processing: the numerical axis [0,1) is divided into lengths corresponding to the probability that each bit value of the input bit string is "0", and one of the sub-intervals is retained as the segmentation object for the next bit. The encoding process outputs the coordinate values ​​contained in the remaining sub-interval as a code. In addition, when the bit is "0", the sub-interval on the left after the division is retained, and when the bit is "1", the sub-interval on the right after the division is retained. In addition, the probability of each bit value being "0" is obtained from the probability table 304 using the input bit history record so far as an index.

[0067] According to the provisions of the LZMA algorithm, the bit history records used to refer to the probability value are cleared under the prescribed conditions. For example, in the 9 bits representing the literal character, the encoder 301 uses the empty (NULL) bit history records to encode the first bit of the 8 bits other than the first bit. The encoder 301 encodes the last 8 bits using the 1st to 7th bits as the bit history records, and then clears the bit history records.

[0068] Figure 3B The decoding process performed by the decoder 311 is as follows Figure 4A , 4B As shown, the value axis [0,1) is divided into lengths corresponding to the probability that each bit value of the output bit string is "0". Furthermore, the decoder 311 repeatedly performs the following processing: checking which subinterval the input code (coordinate value) is included in, determining each bit of the output bit string, and retaining the included subinterval as the segmentation object for the next bit. Then, all values ​​of the output bit string are finally determined.

[0069] Furthermore, when segmenting, if the left sub-interval contains a code, the decoder 311 determines the bit to be "0", and if the right sub-interval contains a code, the decoder 311 determines the bit to be "1". In addition, the probability of each bit value being "0" is obtained from the probability table 314 using the output bit history record to that point as an index.

[0070] The decoder 311 clears the bit history records used for the reference probability value under the same conditions as the encoding. For example, among the 9 bits representing the literal character, the decoder 311 decodes the first bit of the 8 bits except the first bit using the bit history records of NULL. For the last 8 bits, the decoder 311 decodes the first to seventh bits as the bit history records, and then clears the bit history records.

[0071] Figure 4A and Figure 4BBoth represent examples of transitions in the numerical axis division in the encoding / decoding process of the bit string "1,1,0,1". However, the values ​​of the probability that the bit value is "0" are different. Figure 4A In the example, the probability value of "0" is always 0.5. Figure 4B In the example, the probability values ​​of "0" are 0.25, 0.25, 0.75, and 0.25 in bit order. Figure 4A This shows a case where the probability values ​​referred to from the probability tables 304 and 314 are in an initial state. Figure 4B This shows how the probability values ​​referred to from the probability tables 304 and 314 change through learning.

[0072] In interval coding / decoding, if the bit string "1, 1, 0, 1" is frequently processed, the probability tables 304 and 314 learn that "1" is likely to appear first, "1" is likely to appear next if the history record is "11", "0" is likely to appear next if the history record is "110", and "1" is likely to appear next if the history record is "110". As the learning of such probabilities progresses, when the prediction of the bit appearance pattern comes true, the output code of the interval coding becomes shorter.

[0073] Figure 4A The encoding process in is as follows.

[0074] Step 1: According to the input "1", keep the right 1 / 2 subinterval [1 / 2 to 2 / 2).

[0075] Step 2: According to the input "1", keep the right 1 / 2 sub-interval [3 / 4 to 4 / 4).

[0076] Step 3: Input “0” to keep the left 1 / 2 sub-interval [6 / 8 to 7 / 8).

[0077] Step 4: According to the input “1”, keep the right 1 / 2 sub-interval [13 / 16~14 / 16).

[0078] ·Let 13 / 16 (binary number 1101) included in the last subinterval be output code 401.

[0079] on the other hand, Figure 4B The encoding process in is as follows:

[0080] Step 1: According to the input "1", keep the right 3 / 4 sub-interval [1 / 4 to 4 / 4).

[0081] Step 2: According to the input "1", keep the right 3 / 4 sub-interval [7 / 16~16 / 16).

[0082] Step 3: Enter "0" to keep the left 3 / 4 sub-interval [28 / 64 to 55 / 64).

[0083] Step 4: According to the input "1", the right 3 / 4 sub-interval [139 / 256 to 220 / 256) is retained.

[0084] ·The 3 / 4 (binary number 11) included in the last subinterval is set as the output code 411.

[0085] The more the bit string is input according to the prediction based on probability, the larger the size of the sub-interval remaining after the division. Therefore, the number of bits required to express the coordinate value of the output code included in the sub-interval remaining in the end is small. Figure 4A The example requires 4 bits, whereas Figure 4B In this example, it can be 2 bits. In this way, interval coding improves the compression rate by learning the bit occurrence probability corresponding to the bit history record.

[0086] (2-5) Flowchart of interval encoding / decoding

[0087] Figure 5A A flowchart showing an example of interval encoding. Figure 5B A flowchart showing an example of interval decoding. First, refer to Figure 5A The flow of interval coding is described.

[0088] The encoder 301 refers to the probability value of the next bit being "0" from the probability table 304 based on the input bit history (501). The encoder 301 divides the numerical axis interval (interval to be divided) into two sub-intervals based on the probability value (502). The multiplication of the interval size and the probability value is performed during the division. This is the most time-consuming part of the encoding process. Then, the encoder 301 selects one of the two sub-intervals based on whether the input bit value is "0" or "1" (503).

[0089] Next, in step 504 , encoder 301 determines whether the bit input is completed. If the bit input is completed ( 504 : Yes), the process moves to step 506 . If there is still input ( 504 : No), the process moves to step 505 .

[0090] In step 505, the encoder 301 updates the probability value used for the probability table 304 and updates the bit history record for the encoding of the next bit. The probability value is updated to increase if the input bit value is "0" and decrease if the input bit value is "1". For example, if the next input bit of "11" is "0", the update of the bit history record is changed to "110". Then, the encoder 301 returns to step 501 and continues the encoding process.

[0091] On the other hand, in step 506, the encoder 301 outputs the coordinate value used to determine the last remaining sub-interval, for example, the coordinate value represented by the least number of bits among the values ​​included in the sub-interval, as a code, and ends the encoding process.

[0092] refer to Figure 5B The process of an example of interval decoding is described. The decoder 311 refers to the probability table 314 based on the output bit history record and obtains the probability value of the next bit being "0" (511). The decoder 311 divides the numerical axis interval into two sub-intervals based on the probability value (512). The multiplication of the interval size and the probability value is performed during the division. This is the most time-consuming part of the decoding process. Then, the decoder 311 selects the sub-interval in which the value of the input code is included in the two sub-intervals (513). The decoder 311 outputs the bit value "0" or "1" represented by the selected sub-interval (514).

[0093] Next, in step 515 , the decoder 311 determines whether the output of bits has been completed. If it has been completed ( 515 : Yes), the decoding process is terminated. If there is still output ( 515 : No), the process moves to step 516 .

[0094] In step 516, decoder 311 updates the probability value used in probability table 314 and updates the bit history record for decoding of the next bit. The probability value is updated by increasing if the output bit value is "0" and decreasing if the output bit value is "1". For example, if the next output bit of "11" is "0", the bit history record is updated to "110". Then, decoder 311 returns to step 511 to continue the decoding process.

[0095] (3) Speeding up section coding processing

[0096] Figure 6 A functional block diagram showing an example of a method for speeding up section encoding processing. Figure 1 The LZMA compression and decompression circuit 104 performs the compression process shown in the block diagram. Figure 3A to Figure 5B In the example described above, the multiplication process is performed with reference to the probability every time 1 bit is input. Therefore, only 1 bit can be processed in one operation cycle, which may be the reason why the processing performance of the LZMA algorithm is slow.

[0097] Figure 6 The interval coding function 600 shown in the figure speeds up the interval coding process by operating multiple encoders simultaneously. That is, N intervals of segmentation objects (N>1) are prepared, and N types of bit history records are prepared in advance according to the input bit string. The interval coding function 600 simultaneously refers to N probability values ​​in the probability table, performs multiplication processing on N input bits in parallel, and improves the coding performance by N times.

[0098] Figure 6 The following is a coding example when N=4 according to the high-speed technology. Figure 3A The encoder 301 performs the same processing. Each encoder obtains one probability value from the probability table 604 for use. These four probability values ​​are referenced using the bit history records 605A to 605D as indices.

[0099] Bit history record 605A is used when encoder 601A processes the first bit "1" of input bit string 602, and its value is NULL. Bit history record 605B is used when encoder 601B processes the second bit "1" of input bit string 602, and its value is "1".

[0100] Bit history record 605C is used when encoder 601C processes the third bit "0" of input bit string 602, and its value is "11". Bit history record 605D is used when encoder 601D processes the fourth bit "1" of input bit string 602, and its value is "110".

[0101] Generally speaking, the bit history for encoding the Nth bit is composed of the concatenation of the 1st to (N-1)th bits. By preparing four types of bit history in this way, the four encoders 601A to 601D can simultaneously refer to four probability values ​​from the probability table 604 and perform multiplication operations simultaneously using these probability values.

[0102] The four subcodes 603A to 603D output from encoders 601A to 601D are finally connected to form output code 606. Output code 606 is equivalent to the compressed data of the LZMA algorithm. According to this method, a 4-bit input can be processed in one operation cycle, so the performance of interval coding in the compression processing of the LZMA algorithm is improved to 4 times that of the prior art.

[0103] Figure 7 Indicates reference Figure 6 This is a flowchart example of a method for increasing the speed of section coding. Figure 7 The flowchart of the high-speed interval coding method is shown in FIG. 1 . First, the LZMA compression and decompression circuit 104 generates N bit history records (701) for encoding the N bits in the input bit string. N is an integer greater than 2. The N encoders obtain the probability value of the next bit being "0" from the probability table based on their respective bit history records (702). The N encoders divide the N value axis intervals (division object intervals) into 2 sub-intervals (703) based on their probability values.

[0104] The multiplication of the interval size and the probability value by the N encoders is performed in parallel. In the first cycle, the value axis interval (interval size) is common to the N encoders. Figure 6 In the example shown, it is [0,1). After the second loop, the subinterval selected by the code in the previous loop becomes the numerical axis interval (interval size) of the object. Each encoder selects one of the two subintervals on the left and right according to whether the input bit value is "0" or "1" (704).

[0105] Next, in step 705, the LZMA compression / decompression circuit 104 determines whether the bit input is finished. If the bit input is finished (705: Yes), the process moves to step 707. If there is still input (705: No), the process moves to step 706. In step 706, the LZMA compression / decompression circuit 104 updates the N probability values ​​used in the probability table. The probability value is updated by increasing if the input bit value is "0" and decreasing if the input bit value is "1".

[0106] Then, the LZMA compression / decompression circuit 104 returns to step 701 and continues the encoding process. For example, when N is 4 and 8 bits of the literal character are encoded, the first cycle of this process encodes the first half 4 bits, and the second cycle encodes the second half 4 bits. In the second cycle, the bit history record for encoding the fifth bit is the bit string of the first half 4 bits.

[0107] For example, when the input bit string is 6 bits, the first loop may encode the first half 4 bits or 3 bits, and the second loop may encode the second half 2 bits or 3 bits. The maximum value of the input bit string to the LZMA compression / decompression circuit 104 is 4, and bit strings below this value can be encoded.

[0108] In step 707, each encoder generates a coordinate value for specifying the last remaining sub-interval, for example, a value having the fewest bits among the values ​​included in the sub-interval. The LZMA compression / decompression circuit 104 outputs a bit string obtained by concatenating N of these bits as a code, and ends the encoding process.

[0109] exist Figure 6 In the example of interval coding shown in the figure, the integer N>1, the number of input bits is N bits, that is, the length of the bit history record is a maximum of (N-1) bits. This interval coding can use the N intervals of the segmented object to speed up the coding to N times. Figure 7 In the description, as mentioned in the example of 8 bits of literal characters, when the integer M>N and the number of input bits is M bits, that is, the length of the bit history record is a maximum of (M-1) bits, the N intervals of the segmented object can be used to speed up the encoding of the input bits.

[0110] The following method is described by taking the case of M=8 and N=4 as an example. The LZMA compression and decompression circuit 104 includes a probability table of 255 entries with a maximum of 7 bits of bit history records as indexes. The LZMA compression and decompression circuit 104 prepares 4 types of bit history records (empty, 1 bit, 2 bits, and 3 bits) used in encoding the first half 4 bits of the 8-bit input bit string, and simultaneously refers to the 4 probability values ​​corresponding to them in the probability table. The LZMA compression and decompression circuit 104 uses these probability values ​​to encode the first half 4 bits in parallel in the first cycle.

[0111] Next, the LZMA compression / decompression circuit 104 prepares four types of bit history records (4 bits, 5 bits, 6 bits, and 7 bits including the first half 4 bits at the beginning, respectively) used in encoding the second half 4 bits of the input bits, and simultaneously refers to the corresponding four probability values ​​from the probability table. The LZMA compression / decompression circuit 104 uses these probability values ​​to encode the second half 4 bits in parallel in the second loop.

[0112] Thus, the LZMA compression / decompression circuit 104 processes 8-bit input in 2 cycles (ie, 4 times the performance), generates 4 sub-codes, and constructs an output code by concatenating them.

[0113] Generally speaking, the encoding process of the interval encoding of the input M bits uses a probability table of (2^M-1) entries with the bit history record of the maximum (M-1) bits as an index and N encoders to process M bits in [M / N] operation cycles, thereby improving its performance. In addition, the LZMA compression and decompression circuit 104 may generate subcodes without performing the above-mentioned parallel processing.

[0114] (4) Speeding up section decoding processing

[0115] The N-fold speedup of the section decoding process is achieved by using Figure 3B It is not possible to operate N decoders 311 in parallel. This is because the bit history used in the processing of a certain decoder X is not determined until the decoder Y that decoded the previous bit outputs the processing result. Therefore, decoder X cannot refer to the probability value from the probability table and perform multiplication using the value at the same time as decoder Y, and parallelization cannot be achieved.

[0116] Next, a method of speeding up the section decoding process according to one embodiment of the present specification will be described. Figure 8 This is a functional block diagram showing a method for speeding up section decoding processing. Figure 1 The LZMA compression and decompression circuit 104 performs decompression processing through the interval decoding function 800 shown in the block diagram.

[0117] Figure 8 15 decoders 8A (1), 8B0 and 8B1 (2), 8C00 to 8C11 (4), 8D000 to 8D111 (8) are all decoded with Figure 3B The decoder 311 performs the same processing. Figure 8 The four sub-codes 803A to 803D input to the 15 decoders are separated from the input code 802 (equivalent to the compressed data of the LZMA algorithm) and are Figure 6 The four subcodes 603A to 603D are the same. In addition, one subcode may be shared by multiple decoders.

[0118] Each decoder obtains one probability value from probability table 804 and uses it to output a candidate bit value. These 15 probability values ​​are values ​​that are referenced using all possible bit history records as indices.

[0119] The value of the bit history record used by one decoder 8A for decoding the first bit of the output bit string 806 is NULL. The values ​​of the bit history records used by two decoders 8B0 and 8B1 for decoding the second bit of the output bit string 806 are "0" and "1", respectively.

[0120] The values ​​of the bit history records used by the four decoders 8C00 to 8C11 for decoding the third bit of the output bit string 806 are "00", "01", "10", and "11", respectively. The values ​​of the bit history records used by the eight decoders 8D000 to 8D111 for decoding the fourth bit of the output bit string 806 are "000", "001", "010", "011", "100", "101", "110", and "111", respectively.

[0121] In general, the number of bit history records used in decoding the Kth bit is 2^(K-1). Each bit history record is a bit pattern (bit sequence) of (K-1) bits that may be the 1st to (K-1)th bits of the output bit sequence 806. By preparing 15 types of bit history records in this way, 15 probability values ​​are simultaneously referenced from the probability table 804, and 15 decoders simultaneously perform multiplication operations using these probability values.

[0122] When the first bit of the output bit string 806 output by the decoder 8A is "1", it is known that the second bit output by the decoder 8B1, which decoded by assuming that the first bit is "1" among the decoders 8B0 and 8B1, is the correct result. Therefore, the selector 805B selects "1" output by the decoder 8B1 from the two second bit candidates output by the decoders 8B0 and 8B1. That is, the first and second bits are determined to be "11".

[0123] Therefore, it can be known that the third bit output by decoder 8C11, which decoded assuming that the first and second bits are "11", is the correct result. Therefore, selector 805C selects "0" output by decoder 8C11 from the four third bit candidates output by decoders 8C00 to 8C11. That is, the first to third bits are determined to be "110".

[0124] Therefore, it can be seen that the 4th bit output by decoder 8D110, which decoded assuming that the 1st to 3rd bits were "110" among decoders 8D000 to 8D111, is the correct result. Therefore, selector 805D selects "1" output by decoder 8D110 from the 8 4th bit candidates output by decoders 8D000 to 8D111.

[0125] Through the above, it is determined that the 4 bits of the output bit string 806 are "1101". In general, the LZMA compression and decompression circuit 104 includes 2^(K-1) decoders to decode the Kth bit, and keeps the 2^(K-1) Kth bit candidates output by them. The LZMA compression and decompression circuit 104 selects the candidate output by one decoder that has been decoded assuming that the values ​​of the 1st to (K-1)th bits that have been determined are bit history records as the Kth bit.

[0126] The bit selection process performed by selectors 805B to 805D is performed in a sufficiently short time compared to the multiplication process of the decoder. Therefore, according to this method, a 4-bit output can be processed in one operation cycle. Therefore, the performance of the interval decoding process in the decompression process of the LZMA algorithm is improved to 4 times that of the prior art.

[0127] Below, refer to Fig. 9 Indicates reference Figure 8 The process of the high-speed interval decoding method described above. First, the LZMA compression and decompression circuit 104 generates (2^N-1) bit history records (901) that may be used to decode N bits in the output bit string. The bit history records used to decode the Kth bit are 2^(K-1).

[0128] The (2^N-1) decoders obtain the probability value (902) that the next bit is "0" from the probability table 804 according to the bit history records they are responsible for, and divide the value axis interval (division object interval) into two sub-intervals (903) according to the probability value. The value axis intervals divided by the 2^(K-1) decoders used in the decoding of the Kth bit are the same. Specifically, in the first cycle, the value axis intervals of all decoders are the same. Figure 8 In the example of [0,1], after the second cycle, the numerical axis interval of the decoder of the Kth bit is the interval of the segmentation result of the decoder that output the correct answer of the Kth bit in the previous cycle. When segmenting, the multiplication operation of the interval size and the probability value is performed.

[0129] The decoder selects a subinterval (904) in which the value of the input subcode is included in two subintervals, and generates a bit value "0" or "1" represented by the selected subinterval (905). In addition, the number of bit values ​​generated is (2^N-1), and the number of candidates for the Kth bit is 2^(K-1). Then, the selector selects one correct bit from each candidate in turn starting from the 1st bit, determines the N-bit pattern and outputs it (906). In addition, in the selection of the correct value of the Kth bit, the correct values ​​of the 1st to (K-1)th bits are used as bit history records.

[0130] Next, in step 907 , the LZMA compression / decompression circuit 104 determines whether the output of bits is completed. If it is completed ( 907 : Yes), the decoding process is terminated. If there is still output ( 907 : No), the process moves to step 908 .

[0131] In step 908, the LZMA compression and decompression circuit 104 updates the N probability values ​​used in the probability table 804. The probability value is updated by increasing if the output bit value is "0" and decreasing if the output bit value is "1". Furthermore, the LZMA compression and decompression circuit 104 adopts the subinterval selected in step 904 by the decoder that outputs the correct bit value as the value axis interval of the next cycle. Among the 2^(K-1) decoders for the K-th bit, the subinterval selected in step 904 by the 1 decoder that outputs the correct value of the K-th bit is used as the value axis interval divided in step 903 in the decoding of the K-th bit next time.

[0132] Then, the LZMA compression / decompression circuit 104 returns to step 901 to continue the decoding process. For example, for the 8 bits of the literal character, when the first half 4 bits and the second half 4 bits are encoded in two cycles, the bit history record for decoding the fifth bit in the second cycle of this process is the bit string of the first half 4 bits.

[0133] For example, when 6-bit input data is divided into the first half 4 bits and the remaining 2 bits and encoded in 2 cycles, the LZMA compression and decompression circuit 104 can decode 4 bits or 3 bits in the first cycle and then decode 2 bits or 3 bits in the second cycle. The maximum value of the input bit string to the LZMA compression and decompression circuit 104 is 4, and bit strings below this value can be decoded.

[0134] exist Figure 8 In the high-speed decoding method of the interval coding shown in the figure, when the integer N>1, the number of input bits is N bits (that is, the length of the bit history record is a maximum of (N-1) bits), and N intervals of segmentation objects are used, and the speed is increased to N times. Fig. 9 As mentioned in the example of the 8-bit literal character, when the integer M>N and the number of input bits is M bits, that is, the length of the bit history record is a maximum of (M-1) bits, the decoding of the output bit string can be accelerated by using N segmentation object intervals.

[0135] The following describes an example of decoding an 8-bit output bit string. Figure 8 Also includes 15 decoders, with Figure 8 Similarly, the 4 sub-codes separated from the input code (equivalent to the compressed data of the LZMA algorithm) are input to these 15 decoders. Each decoder obtains and uses one probability value from a probability table of 255 entries indexed by a bit history of 7 bits at most.

[0136] The 15 probability values ​​referenced in the first loop are values ​​referenced using all bit history records (empty, 1 bit, 2 bits, and 3 bits) that are possible as the first half of the 8-bit output bit string as indexes. The value of the bit history record used by a decoder that decodes the first bit of the output bit string is empty (NULL).

[0137] The values ​​of the bit history records used by the two decoders that decode the second bit of the output bit string are "0" and "1", respectively. The values ​​of the bit history records used by the four decoders that decode the third bit of the output bit string are "00", "01", "10", and "11", respectively. The values ​​of the bit history records used by the eight decoders that decode the fourth bit of the output bit string are "000", "001", "010", "011", "100", "101", "110", and "111", respectively.

[0138] The 15 decoders perform multiplications in parallel using the probability values ​​referenced in these bit histories. Figure 8Similarly, the bit selection process of the selector sequentially determines the values ​​of the first bit to the fourth bit. Here, for example, they are set to "1101".

[0139] Next, the 15 probability values ​​referenced in the second loop are values ​​referenced using all the bit history records that may be the second half of the 8-bit output bit string (the 4th bit, 5th bit, 6th bit, and 7th bit at the beginning of "1101" determined in the first loop, respectively) as indexes.

[0140] The bit history record value used by the one decoder that decodes the fifth bit of the output bit string is "1101". The bit history record values ​​used by the two decoders that decode the sixth bit of the output bit string are "11010" and "11011", respectively. The bit history record values ​​used by the four decoders that decode the seventh bit of the output bit string are "110100", "110101", "110110", and "110111", respectively.

[0141] The values ​​of the bit history records used by the eight decoders that decode the eighth bit of the output bit string are respectively "1101000", "1101001", "1101010", "1101011", "1101100", "1101101", "1101110", and "1101111".

[0142] The 15 decoders perform multiplications in parallel using the probability values ​​referenced in these bit histories. Figure 8 Likewise, the bit selection process of the selector sequentially determines the values ​​of the 5th to 8th bits.

[0143] and Figure 8 Similarly, the bit selection process of the selector is performed in a sufficiently short time compared to the multiplication process of the decoder. Therefore, according to this method, an 8-bit output can be processed in two operation cycles. As described above, in the probability table of 255 entries, 15 entries of the probability table are referenced in the first cycle, and 15 entries of the probability table are selected from the remaining 240 entries of the probability table for reference in the second cycle. In the second cycle, by indexing with the bit history record of the first half 4 bits determined in the first cycle, the number of entries referenced is reduced to 1 / 16.

[0144] Generally speaking, the decoding process of the interval code that outputs M bits uses a probability table of (2^M-1) entries indexed by the bit history record of the maximum (M-1) bits and (2^N-1) decoders, processing M bits in [M / N] operation cycles, thereby improving performance.

[0145] As described above, one embodiment of the present specification can decompress data compressed by interval coding at high speed, thereby improving the read response performance of compressed data in a device storage system having a data compression function based on an interval coding algorithm, for example.

[0146] In addition, the present invention is not limited to the above-mentioned embodiments, and includes various modified examples. For example, the above-mentioned embodiments are embodiments described in detail in order to explain the present invention in an easy-to-understand manner, and are not limited to all structures described. In addition, a part of the structure of a certain embodiment can be replaced with the structure of other embodiments, and in addition, the structure of other embodiments can be added to the structure of a certain embodiment. In addition, for a part of the structure of each embodiment, other structures can be added, deleted, or replaced.

[0147] In addition, part or all of the above-mentioned structures, functions, processing units, etc. may also be implemented by hardware, for example, by integrated circuit design, etc. In addition, the above-mentioned structures, functions, etc. may also be implemented by software by a processor interpreting and executing programs that implement each function. Information such as programs, tables, files, etc. that implement each function can be placed in a recording device such as a memory, a hard disk, an SSD (Solid State Drive), or a recording medium such as an IC card or an SD card.

[0148] In addition, the control lines and information lines are those considered necessary for explanation, and do not necessarily represent all the control lines and information lines on the product. In fact, it can be assumed that almost all structures are connected to each other.

[0149] Description of Reference Numerals

[0150] 101…storage system; 103…storage controller; 104…LZMA compression and decompression circuit; 301, 601A~D…encoder; 311, 8A, 8B0, 8B1, 8C00, 8C11, 8D000, 8D111…decoder; 304, 314, 604, 804…probability table; 302, 602…input bit string; 303, 606…output code; 312, 802…input code; 313, 806…output bit string.

Claims

1. A device for processing received data, characterized in that: include: A circuit for receiving an input code compressed based on the range coding; and a decompression circuit for decompressing a part or all of the input code to decode the N-bit string, The N represents an integer greater than 1, and K represents an integer from 1 to N. The bit value of the Kth bit of the input code is decoded based on the bit history record of the bits preceding the Kth bit, The decompression circuit, For the candidate bit values ​​of the bits of the N-bit string, a process is performed in parallel on a plurality of bits to respectively calculate a plurality of candidate bit values ​​based on a plurality of bit history records that can be obtained for the bits preceding the K-th bit, The N-bit string is decoded by repeatedly selecting the correct bit value of the K-th bit from the plurality of candidate bit values ​​based on the correct bit history records preceding the K-th bit.

2. The device according to claim 1, characterized in that: The decompression circuit, storing probability values ​​of bit values ​​occurring corresponding to the bit history records, Among the stored probability values, the probability value corresponding to the bit history record according to which the correct bits constituting the decoded N-bit string are followed is selected and updated.

3. The device according to claim 1, characterized in that: In the calculation of the candidate bit value of the Kth bit in the N-bit string, a multiplication operation is performed on the segmentation object interval based on the respective probability values ​​of the 2^(K-1) bit historical records to divide the segmentation object interval into two partitions, and 2^(K-1) respective candidate bit values ​​are determined from the partitions determined by the sub-codes in the two partitions of the Kth bit in the input code.

4. The device according to claim 1, characterized in that: When the number of bits to be decoded from the entire input code is greater than a preset maximum value, the decompression circuit performs a plurality of cycles of decoding a bit string from a portion of the input code.

5. The device according to claim 4, characterized in that: In the second and subsequent rounds of the plurality of rounds, the division target sections for respectively determining the candidate bit values ​​of the K-th bit are partitions of the correct bit value of the K-th bit in the previous round.

6. A storage system, characterized in that: include: An interface for receiving requests from a host; and a controller that writes data to and reads data from the storage drive in accordance with instructions from the host, The controller comprises the decompression circuit according to claim 1, The controller decodes the input code transmitted from the storage drive according to the read instruction from the host by using the decompression circuit to generate read data, and returns the read data to the host via the interface.

7. The storage system according to claim 6, wherein: The decompression circuit, storing probability values ​​of bit values ​​occurring corresponding to the bit history records, Among the stored probability values, the probability values ​​of the correct bit values ​​constituting the decoded N-bit string are selected and updated.

8. The storage system according to claim 6, wherein: In the calculation of the candidate bit value of the K-th bit in the N-bit string, a multiplication operation is performed on the segmentation object interval based on the probability values ​​of each of the 2^(K-1) bit historical records to divide the segmentation object interval into two partitions, and the candidate bit values ​​are respectively determined from the 2^(K-1) partitions of the sub-code of the K-th bit in the input code.

9. The storage system according to claim 6, wherein: When the number of bits to be decoded from the entire input code is greater than a preset maximum value, the decompression circuit performs a plurality of cycles of decoding a bit string from a portion of the input code.

10. The storage system according to claim 9, characterized in that: In the second and subsequent rounds of the plurality of rounds, the division target sections for respectively determining the candidate bit values ​​of the K-th bit are partitions of the correct bit value of the K-th bit in the previous round.

11. A method for decoding an N-bit string by decompressing part or all of an input code compressed based on interval coding, characterized in that: The N represents an integer greater than 1, and K represents an integer from 1 to N. In the method, The method comprises: calculating candidate bit values ​​of respective bits of the N-bit string in parallel, and in calculating the candidate bit value of the K-th bit in the N-bit string, performing a multiplication operation on the segmentation target interval based on the probability values ​​of the respective bit history records to segment the segmentation target interval into two partitions, and determining candidate bit values ​​from the partitions of the subcode for the K-th bit in the input code, respectively. The correct bit value of the K-th bit is selected from the candidate bit values ​​based on a correct bit history record preceding the K-th bit to decode the N-bit string.

12. The method according to claim 11, characterized in that: From the pre-set probability values, the probability values ​​of the correct bit values ​​constituting the decoded N-bit string are selected and updated.

13. The method according to claim 11, characterized in that: In the calculation of the candidate bit value of the Kth bit in the N-bit string, a multiplication operation is performed on the segmentation object interval based on the probability values ​​of each of the 2^(K-1) bit historical records to divide the segmentation object interval into two partitions, and the candidate bit values ​​are respectively determined from the 2^(K-1) partitions of the sub-code for the Kth bit in the input code.

14. The method according to claim 11, characterized in that: When the number of bits to be decoded from the entire input code is greater than a preset maximum value, a plurality of cycles of decoding of a bit string from a portion of the input code are performed.

15. The method according to claim 14, characterized in that: In the second and subsequent rounds of the plurality of rounds, the division target sections for respectively determining the candidate bit values ​​of the K-th bit are partitions of the correct bit value of the K-th bit in the previous round.

Images