Data processing method, device and electronic equipment
By using the Huffman decoding and data splicing operations of flash memory and access memory during data processing, the problem of low decoding efficiency in the prior art is solved, and more efficient data decoding and circuit resource saving is achieved.
Patent Information
- Application Number
- CN202411534212.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2025-05-13
- Estimated Expiration
- 2044-10-31
AI Technical Summary
The prior art has low decoding efficiency during the data decoding process, which affects the data transmission efficiency.
Flash memory FLASH and access memory DDR are used to improve decoding efficiency through Huffman decoding and data splicing operations.
Improve data decoding efficiency, reduce circuit area, facilitate sharing with other circuits, and save resources.
Smart Images
Figure CN119210466B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data transmission, and in particular to a data processing method, device and electronic equipment. Background Art
[0002] In recent years, with the continuous growth of data volume, various data decompression methods based on computer processing have seriously affected the decoding efficiency. In the implementation process of the existing method, in order to avoid the influence of computer processing in the communication process on the data decoding efficiency, the AXI (Advanced eXtensive Interface) bus is used for data transmission to improve the transmission efficiency. The disadvantage is that the decoding efficiency is low when processing data. Summary of the invention
[0003] Purpose of the invention: The purpose of the present invention is to provide a data processing method, device and electronic device in order to improve decoding efficiency.
[0004] Technical solution:
[0005] According to one aspect of the present disclosure, there is provided a data processing method, which is applied to a decompression unit of a circuit, the circuit further comprising: a flash memory FLASH and an access memory DDR;
[0006] The method comprises:
[0007] In response to a decompression request for compressed data, receiving and decompressing the compressed data in the flash memory FLASH to obtain decompressed data stored in partitions, and performing a splicing operation on the decompressed data and storing it in the access memory DDR;
[0008] The decompression operation includes:
[0009] Determine the corresponding first Huffman codeword value table and second Huffman codeword value table respectively according to two compressed data in the compressed data; perform Huffman decoding on the first Huffman codeword value table according to the first preset byte data in the compressed data, and perform Huffman decoding on the second Huffman codeword value table according to the second preset byte data in the compressed data to obtain decoding information, the compressed data obtains corresponding decompressed data according to the decoding information, and stores the decompressed data in N partitions; wherein the decoding information includes the decoded data and corresponding index information, the decoded data includes the original character and repeated character of the meta information, the index information includes the repeated character index length and the repeated character index position, and N is a positive integer greater than 1;
[0010] The splicing operation includes: splicing the decoded data stored in the N partitions to obtain a spliced decoded data, and storing the spliced decoded data in the access memory DDR.
[0011] According to another aspect of the present disclosure, there is provided a data processing device, which is applied to a decompression unit of a circuit, the circuit further comprising: a flash memory FLASH and an access memory DDR;
[0012] The device comprises:
[0013] A response module, responding to a decompression request for compressed data;
[0014] A decompression module receives and decompresses the compressed data in the flash memory FLASH to obtain decompressed data stored in partitions;
[0015] A splicing module, which performs a splicing operation on the decompressed data and stores the spliced data in a memory DDR;
[0016] Among them, the decompression module includes:
[0017] A Huffman code word value table acquisition unit determines the corresponding first Huffman code word value table and second Huffman code word value table according to two compressed data in the compressed data;
[0018] A Huffman decoding unit, performing Huffman decoding on a first Huffman codeword value table according to first preset byte data in the compressed data, and performing Huffman decoding on a second Huffman codeword value table according to second preset byte data in the compressed data to obtain decoding information, the compressed data obtains corresponding decompressed data according to the decoding information, and stores the decoded data in N partitions;
[0019] The decoded information includes decoded data and corresponding index information, the decoded data includes the original character and repeated character of the meta information, the index information includes the repeated character index length and repeated character index position, and N is a positive integer greater than 1;
[0020] The splicing module is used to splice the decoded data stored in the N partitions to obtain a spliced decoded data, and store the spliced decoded data in the access memory DDR.
[0021] According to yet another aspect of the present disclosure, there is provided an electronic device, comprising: at least one processor; and
[0022] a memory communicatively connected to the at least one processor; wherein,
[0023] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform any of the data processing methods described above.
[0024] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the above-mentioned data processing methods.
[0025] Beneficial effect: The circuit is used to perform decompression and data splicing operations on compressed data, and the decompression efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 is a schematic diagram of a circuit according to Embodiment 1 of the present invention;
[0027] Figure 2 It is a schematic diagram of the prior art using dual-port SRAM for data storage;
[0028] Figure 3 This is a schematic diagram of the data storage principle using a single-port SRAM in Embodiment 1 of the present invention;
[0029] Figure 4 It is a block diagram of the algorithm structure of Huffman decoding of embodiment 1 of the present invention;
[0030] Figure 5 Schematic diagram of the decompression principle of Embodiment 1 of the present invention;
[0031] Figure 6 It is a schematic diagram of the position information of repeated characters and the code stream length control decoding principle of embodiment 1 of the present invention;
[0032] Figure 7 is a schematic diagram of the working process of the decompression unit of Embodiment 1 of the present invention;
[0033] Figure 8 It is a splicing principle diagram of embodiment 1 of the present invention;
[0034] Fig. 9 It is a structural diagram of a data processing device according to embodiment 2 of the present invention. DETAILED DESCRIPTION
[0035] In order to make the technical solution of the present invention clearer, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0036] Embodiment 1: Figure 1 This is a circuit architecture diagram provided in Example 1 of the present disclosure, such as Figure 1 As shown, the circuit may include a decompression unit, a bus control unit APB, a flash memory FLASH and an access memory DDR;
[0037] Among them, the bus control unit APB is connected to the decompression unit, and the bus control unit APB configures the register to achieve the purpose of controlling the change of the decompression unit function. For example, the APB control unit configuration can realize the change of the matching position, matching length and data storage address, etc., so as to be more flexibly applied to various scenarios and improve the circuit compatibility;
[0038] The decompression unit is connected to the flash memory FLASH through an axi bus, and the decompression unit is connected to the access memory DDR through an axi bus.
[0039] During the process of decompressing the compressed data in the flash memory FLASH, the decompression unit can respond to the decompression request sent by the bus control unit APB, obtain the compressed file data from the flash memory FLASH through the axi bus, decompress the compressed file data according to the decompression request, and store the obtained decoded data in the storage unit of the decompression unit itself, and then splice the decoded data stored in the partitions and store them in the access memory DDR through the axi bus.
[0040] Thereby, it is realized that the compressed data can be decompressed and spliced through the circuit, thus freeing up the CPU, and in the decompression and data splicing operations on the compressed data in the flash memory FLASH, data can be transmitted through the bus, thereby improving the file decompression rate, and file decompression in various modes can be realized through bus control, and the decoded data can be stored in the access memory DDR through the data splicing operation, which can effectively save circuit area and is convenient for sharing with other circuits.
[0041] The access memory DDR is a single-port SRAM memory (Static Random-Access Memory), which further reduces the circuit area compared to the dual-port SRAM memory in the prior art.
[0042] Figure 2 This is the principle diagram of using dual-port SRAM for data storage in the existing technology. Figure 3 This is a schematic diagram of the principle of using a single-port SRAM for data storage in the present invention.
[0043] Figure 2 When using dual-port memory, Figure 2 It reflects the write enable signal (wen), write address signal (waddr), read enable signal (ren) and read address signal (raddr) of the dual-port memory. Figure 2 As shown, when using a dual-port SRAM to perform repeated character reading, a data needs to be read out first before a data write operation can be performed.
[0044] Figure 3 When using single-port memory, Figure 3 It reflects the chip select enable signal (cen) of the single-port memory, the address signal (waddr) when implementing the write function, the high bit of the address (waddr_high) when implementing the write data function, and the low bit of the address (waddr_low) when implementing the write data function, generally the lower 2 bits, address signal (addr). When cen=1&&wen=1, the write data function is executed, and when cen=1&&wen=0, the read data function is executed.
[0045] like Figure 3 As shown, when using a single-port SRAM, taking four data splicing as an example, when a single write operation is performed, the memory only needs to write data once in four cycles, instead of writing data in each cycle. When storing, the high bit address is used as the new memory address, and the low 2 bits are used as a count of how many times the data is written. When performing a read operation, only one read is required to read four data at a time, which greatly reduces the time to read data. Moreover, compared with single-port splicing SRAM that can store data of the same size, the area consumed is much smaller than that of dual-port SRAM (for example, the resource consumption of single-port SRAM1024*32 is much smaller than that of dual-port 4096*8), and the single-port SRAM is used more frequently in the chip, and can also be shared with other units, which can also reduce resource consumption.
[0046] The present disclosure provides a flowchart of a data processing method, which can be applied to Figure 1 The decompression unit in the method comprises: receiving and decompressing the compressed data in the flash memory FLASH in response to a decompression request for the compressed data to obtain decompressed data, and performing a splicing operation on the decompressed data and storing it in the access memory DDR;
[0047] The decompression operation includes:
[0048] Step S1, determining a first Huffman codeword value table and a second Huffman codeword value table corresponding to two compressed data in the compressed data respectively;
[0049] As a preferred embodiment, compressed data is received from the flash memory FLASH, and two Huffman code tables are first decoded according to the two compressed data, which are respectively recorded as the first Huffman code word value table huff1 (huff1 includes two parameters HUFFVAL and Huff_table) and the second Huffman code word value table huff2 (huff2 includes HUFFVAL 1and Huff_table 1 two parameters);
[0050] Step S2, respectively determining the corresponding first Huffman codeword value table and second Huffman codeword value table according to the two compressed data in the compressed data; performing Huffman decoding on the first Huffman codeword value table according to the first preset byte data in the compressed data, and performing Huffman decoding on the second Huffman codeword value table according to the second preset byte data in the compressed data to obtain decoding information, the compressed data obtains corresponding decompressed data according to the decoding information, and stores the decompressed data in N partitions; wherein the decoding information includes the decoded data and corresponding index information, the decoded data includes the original character and repeated character of the meta information, the index information includes the repeated character index length and the repeated character index position, and N is a positive integer greater than 1;
[0051] In the above embodiment, the index information is first obtained by decoding two Huffman code word value tables, the code stream to be decoded in the compressed data is matched with the index information, and the original characters corresponding to the code stream to be decoded are taken out.
[0052] As a preferred embodiment, step S2 includes:
[0053] Step S21, performing Huffman decoding on the first Huffman codeword value table according to the first preset byte data in the compressed data, and performing Huffman decoding on the second Huffman codeword value table according to the second preset byte data in the compressed data, so as to obtain decoding information;
[0054] Step S21 includes:
[0055] Step S211, performing Huffman decoding on the first Huffman codeword value table according to the first preset data in the compressed data to obtain first decoded data, the first decoded data including a first decoded character and corresponding first index information, the first decoded character including the original character of the meta information and a repeated character, the first index information including the first decoded character index length literal_L and the code stream length prefix org_length;
[0056] As a specific implementation method, Figure 4 This is the algorithm structure diagram of Huffman decoding. Figure 5 It is a schematic diagram of the decompression principle;
[0057] The first Huffman codeword value table is Huffman decoded using the first preset byte data, where the first preset byte data is 16-byte data (16 data, each data bit width is 1 byte), specifically including:
[0058] like Figure 5As shown, 16 bytes of the code stream to be decompressed are taken from the compressed data, and the values of the code length distribution array BITS corresponding to huff1 are BITS(1)~BITS(16); if the sum of the data in BITS is recorded as N, then N bytes of the code stream to be decompressed are taken from the compressed data, and their values correspond to the values of the character array HUFFVAL(1)~HUFFVAL(N), and each element of HUFFVAL is represented by two bytes (high bit first). According to BITS, the first Huffman code word value table huff1 is calculated, recorded as Huff_table1, and the length is huff1_num. It should be noted that the processing method of the second Huffman table (huff2 table) is the same as that of the first Huffman table (huff1 table) mentioned above, taking 16*2byte data, corresponding to HUFFVAL1(1)~HUFFVAL(N), recorded as Huff_table2, and the length is huff2_num.
[0059] It should be noted that the 16-byte data of the code stream to be decompressed is used to extract the huff1 table, and the sum of the 16-byte data huff1_num is the depth of the huff1 table data. Each of the 16 bytes of data can be decoded to obtain the corresponding control signal h1_byte_en0, h1_byte_en1, h1_byte_en2, h1_byte_en3, h1_byte_en4, h1_byte_en5, h1_byte_en6, h1_byte_en7, h1_byte_en8, h1_byte_en9, h1_byte_ena, h1_byte_enb, h1_byte_enc, h1_byte_end, h1_byte_ene, h1_byte_enf, etc. And solve the data volume parameters h1_byte_num0, h1_byte_num1, h1_byte_num2, h1_byte_num3, h1_byte_num4, h1_byte_num5, h1_byte_num6, h1_byte_num7, h1_byte_num8, h1_byte_num9, h1_byte_numa, h1_byte_numb, h1_byte_numc, h1_byte_numd, h1_byte_nume, h1_byte_numf;
[0060] For example, when h1_byte_en0=0, it means that 1-bit compressed data cannot be decrypted; when h1_byte_en0=1, it means that 1-bit compressed data can be decrypted, and the number of decrypted data is h1_byte_num0.
[0061] Specific as Figure 4 As shown, the process of solving Huff1_table and Huff2_table is the same, which is as follows:
[0062] Initialize the huff code subtable address idx=0, the decoding control code value val=0, the byte of compressed data j=1, the number of bits of data required for decoding bit=0;
[0063] First, take the first byte data, j=1, at this time bit=1, k is the comparison code value, BITS(1) determines whether it is greater than 0. If it is greater than 0, it means that the byte compressed data can be decoded to obtain the code value. If it is less than 0, the code value cannot be decoded (the same applies to BITS(2)~BITS(16)).
[0064] At the same time, the value of the parameter val is updated and stored in the address corresponding to Huff_table until the value of the byte data is less than k, then the decoding is terminated, and then the next byte data decoding value is performed until the 16 data code values are decoded.
[0065] Specifically, the Huffman decoding process is:
[0066] First take 1 bit from the bitstream, and then compare whether BITS(1) is greater than 0:
[0067] If it is greater than 0, check whether there is a value in Huff_table (1) ~ Huff_table (BITs (1)) that is equal to the obtained code stream value.
[0068] If so, remember the position of this value in Huff_table, and then take the corresponding value from the same position in HUFFVAL, which is the first solution character index length literal_L corresponding to the first solution character.
[0069] If there is no value in Huff_table (1) to Huff_table (BITs (1)) that is equal to the current value, then take another bit from the code stream and convert it into a new value using the principle of high bit first, and then compare it with BITS (2) in the same way as the previous process. And so on, until the first solution character index length literal_L corresponding to the first solution character can be obtained.
[0070] For example, according to the h1_byte_en0~h1_byte_enf control signals obtained above, we can know how many bits of data can be decoded. For example, h1_byte_en0~h1_byte_7 are all 0, h1_byte_en8=1, then when decompressing the data, directly take 8 bits of data for decoding to get the address huff1_addr of the huff1 table, and then find the corresponding literal_L.
[0071] It should be noted that, in order to improve the compression rate, the first solution character obtained by huff1 is a combination of the original character and the repeated character, and the first solution character index length literal_L = the original character length literal + the repeated character index length length.
[0072] Then, the huff1_num*byte data is taken, which is the information of the huff1 table, where huff1_num is the data length of the Huffman table (huff1 table) decompressed from the first 16-byte compressed data, as shown in Table 1 below.
[0073] Table 1:
[0074]
[0075] Step S212, performing Huffman decoding processing on the second Huffman codeword value table according to the second preset byte data in the compressed data to obtain second decoded data, the second decoded data including a second decoded character and corresponding second index information, the second decoded character including a repeated character, and the second index information including a repeated character index address distance;
[0076] As a specific implementation, the second data of the second preset byte is used to perform Huffman decoding on the second Huffman codeword value table. The second preset byte is 16*2byte, that is, the second Huffman codeword value table takes 16*2byte data for decoding, and the second data is 16 2byte data (16 data, each data bit width is 2byte), specifically including:
[0077] Take 16×2 bytes of the code stream to be decompressed from the compressed data, corresponding to the values BITS1(1)~BITS1(16) of the code length distribution array BITS1 of huff2; if the sum of the data in BITS1 is recorded as N1, then take N1 bytes of the code stream to be decompressed from the compressed data, and its value corresponds to the value HUFFVAL of the character array HUFFVAL1 1 (1) ~HUFFVAL 1(N), calculate the second Huffman codeword value table huff2 according to BITS1, recorded as Huff_table1, with a length of N1. The specific process is as follows Figure 4 As shown, no further elaboration is given here.
[0078] It should be noted that the 16×2byte data of the code stream to be decompressed is used to extract the huff2 table, and the sum of the 16 2byte data huff2_num is the depth of the huff2 table data. Each data can release the corresponding control signal h2_byte_en0, h2_byte_en1, h2_byte_en2, h2_byte_en3, h2_byte_en4, h2_byte_en5, h2_byte_en6, h2_byte_en7, h2_byte_en8, h2_byte_en9, h2_byte_ena, h2_byte_enb, h2_byte_enc, h2_byte_end, h2_byte_ene, h2_byte_enf, and the decoded data Data volume parameters h2_byte_num0, h2_byte_num1, h2_byte_num2, h2_byte_num3, h2_byte_num4, h2_byte_num5, h2_byte_num6, h2_byte_num 7. h2_byte_num8, h2_byte_num9, h2_byte_numa, h2_byte_numb, h2_byte_numc, h2_byte_numd, h2_byte_nume, h2_byte_numf.
[0079] For example, when h2_byte_en0=0, it means that 1-bit compressed data cannot be decrypted; when h2_byte_en0=1, it means that 1-bit compressed data can be decrypted, and the number of decrypted data is h2_byte_num0.
[0080] Specifically, the Huffman decoding process is:
[0081] First, take 1 bit from the bitstream, and then compare whether BITS1(1) is greater than 0:
[0082] If it is greater than 0, check whether there is a value in Huff_table1 (1) to Huff_table1 (BITS1 (1)) that is equal to the value of the code stream. If there is, remember the position of this value in Huff_table1, and then take the corresponding value from the same position in HUFFVAL1, which is the repeated character index address distance. If there is no value in Huff_table1 (1) to Huff_table1 (BITS1 (1)) that is equal to the current value, then take another bit from the code stream, use the principle of high bit first to convert it into a new value, and then compare it with BITS1 (2) in the same way as the previous process. And so on, until the repeated character index address distance can be obtained.
[0083] The data stored in the huff2 table is distance (the index address of repeated characters). Then, the huff2_num*byte data is taken, which is the information of the huff2 table, as shown in Table 2 below.
[0084] Table 2:
[0085]
[0086] Step S22, determining the type of the first solution character by comparing the first solution character index length with the first preset length:
[0087] When the index length of the first solution character is equal to the first preset length, the first solution character is the end character, and decoding ends;
[0088] When the first decoded character index length is less than the first preset length, the code stream to be decoded in the compressed data is the original character, and the original character is partitioned and stored in the first storage module and the second storage module as the decoded data;
[0089] If the first decrypted character index length is greater than the first preset length, the code stream to be decoded in the compressed data is used as repeated data, and a decrypted character operation is performed. The repeated characters are located and obtained from the first storage module, and the repeated characters are stored as decoded data partitions in the first storage module and the second storage module.
[0090] It should be noted that the compressed data obtained above can be used to decode repeated data, that is, distance and literal_L can be decoded. Literal_L can look up the corresponding parameter analysis table (Table 1) to get the repeated character index length length, and distance can look up the corresponding parameter analysis table (Table 2) to get the repeated character index distance pos. The repeated data can be taken out from the module storing the original characters as the result of this decoding.
[0091] As a preferred implementation, the first preset length is 256;
[0092] When the first decode character index length literal_L is equal to 256, the first decode character is the end character (literal_L=256), indicating that the decoding of the data to be compressed is completed;
[0093] Otherwise, continue decoding according to the first solution character index length literal_L for different situations.
[0094] If the first decoded character index length literal_L is less than 256, and the code stream to be decoded in the compressed data is the original character, the original character is directly stored in the first storage module and the second storage module as the decoded data partition. It should be noted that the first storage module here can be the to-be-matched repeated character storage module in the decompression unit, and the second storage module can be the decoded completed character storage module in the decompression unit;
[0095] If the first decoded character index length literal_L is greater than 256, the first decoded character is repeated data, and it is necessary to perform a de-repeated character operation on the code stream to be decoded in the compressed data to locate the repeated characters in the repeated data from the first storage module, obtain the repeated characters, and store the repeated characters as decoded data partitions in the second storage module.
[0096] Wherein, in step S23, the operation of removing repeated characters includes:
[0097] Step S231, calculating the length of the repeated character index: obtaining the corresponding code stream length prefix and code stream length suffix bit width according to the first solution character index length, and taking the sum of the code stream length prefix and code stream length suffix bit width as the repeated character index length;
[0098] As a preferred implementation, the code stream length of the repeated character is calculated based on the first decoded character, specifically: according to the above Table 1, the prefix value (that is, the code stream length prefix, recorded as org_length) and the suffix bit width of the code stream length literal_L of the first decoded character are obtained, and the repeated character index length length (that is, the code stream length of the repeated character) is calculated to realize the subsequent need to take out length repeated data from the memory. It should be noted that the memory here refers to the memory in the decompression unit, that is, the decoding completion character storage module.
[0099] If the suffix width is equal to 0, then the suffix width value is 0; if the suffix width is greater than 0, then read the suffix width bits (high bit first) from the code stream as the suffix width value (that is, the code stream length suffix width, recorded as ext_length), then the repeated character index length length = org_length + ext_length.
[0100] For example, if literal_L=274, refer to Table 1 above, at this time org_length=43, and then continue to take 3 bits of data. If the value of this data is 5, then ext_length=5, and length=org_length+ext_length=48.
[0101] Step S232, obtaining the repeated character index distance: obtaining the corresponding matching position base address and offset address bit width according to the second solution character index position, and taking the sum of the matching position base address and the offset address bit width as the repeated character index distance;
[0102] As a preferred implementation, according to the second solution character index position (i.e., the repeated character index address distance), the prefix value (i.e., the matching position base address, denoted as org_pos) and the suffix bit width (i.e., the offset address bit width, denoted as ext_pos) of the repeated character index distance pos are obtained from Table 2 above to calculate the repeated character index distance pos.
[0103] If the suffix width is equal to 0, then the suffix width value is 0, and the repeated character index distance pos = the matching position base address org_pos;
[0104] If the suffix width is greater than 0, then read the suffix width bits (high bit first) from the code stream as the suffix width value (that is, the offset address width, recorded as ext_pos), then the repeated character index distance pos = matching position base address org_pos + offset address width ext_pos.
[0105] For example, if distance=10, referring to Table 2 above, the matching position base address org_pos=33, and then continue to fetch 4 bits of data. Assuming that the suffix width of the data is 10, ext_pos=10, then pos=org_pos+ext_pos=43.
[0106] Step S233, locating repeated characters according to the repeated character index length and the repeated character index distance, and storing the repeated characters as decoded data partitions in the second storage module;
[0107] Figure 6 The repeated character index length and repeated character index distance control decoding principle diagram of repeated characters, such as Figure 6 As shown, since the length of the repeated character index length = the matching position base address org_pos + the offset address width ext_pos, the repeated character index distance pos data is pushed forward with the current point as the coordinate, and then length bytes of data are taken as the decoded data partition storage.
[0108] For example, when pos=1, length=n (n>=1), it means that a single character is repeated n times. When pos=n (n>1), length=n (n>1), it means that length characters are repeated.
[0109] As a preferred embodiment, Figure 7 Schematic diagram of the working process of the decompression unit of Embodiment 1 of the present invention; Figure 7 As shown, the decompression unit includes:
[0110] The to-be-decoded data cache module is used to store the compressed data obtained from the flash memory FLASH;
[0111] A huff1 decoding module is connected to the to-be-decoded data cache module, and is used to receive a compressed data (i.e., a Huff_table code stream) in the compressed data in the to-be-decoded data cache module, and the Huff_table code stream is the compressed data used to decode the Huff_table, and performs a decoding operation on the first Huffman code word value table huff1 to obtain control information and first decoded data;
[0112] The huff2 decoding module is connected to the to-be-decoded data buffer module and is used to receive a compressed data (i.e., Huff_table1 code stream) in the compressed data in the to-be-decoded data buffer module and perform a decoding operation on the second Huffman codeword value table huff2 to obtain control information and second decoded data;
[0113] The original character decoding module is connected with the data buffer module to be decoded, the huff1 decoding module, the literal_L buffer module and the repeated character decoding module, and is used to determine whether the code stream to be decoded in the compressed data is the original character according to the control information output by the huff1 decoding module (the control information corresponds to the above-mentioned h1_byte_en0~h1_byte_enf):
[0114] If yes, the information of the data to be decoded is sent to the cache literal_L module, so that the cache literal_L module stores the original character partitions into the decoded character storage module;
[0115] If not, output control information to the de-repeating character module so that the de-repeating character module performs a de-repeating character operation;
[0116] The cache literal_L module is connected with the huff1 decompression module, the original character decompression module, the to-be-matched repeated character storage module, the repeated character decompression module and the decoded character storage module, receives the control information output by the huff1 decompression module (the control information is the first decoded data, including the first decoded character and the corresponding first index information), receives the control information output by the original character decompression module, and outputs the repeated character index length length to the repeated character decompression module;
[0117] Among them, the specific steps for decompressing data are: in the first step, the original character module is decoded to obtain the address. If the literal_L taken out from the cache literal_L module is smaller than the preset character, it is the original character and will be sent to the to-be-matched repeated character storage module and the decoded character storage module. The to-be-matched repeated character storage module is to decode the length and pos at the end, and take out length data from it, which is the repeated data.
[0118] A cache distance module is connected to the huff2 decryption module and the repeated character decryption module, receives the control information (the control information is the second decoded data) output by the huff2 decryption module, and outputs the repeated character index address distance to the repeated character decryption module;
[0119] The repeated character decoding module is connected with the to-be-decoded data cache module, the huff1 decoding module, the huff2 decoding module, the original character decoding module, the cache literal_L module, the cache distance module, the to-be-matched repeated character storage module and the decoded character storage module, obtains the repeated character index length length according to the first decoded character index length literal_L, obtains the repeated character index distance pos according to the repeated character index address distance, and performs the repeated character decoding operation according to the repeated character index length length and the repeated character index distance pos, that is, taking out the repeated characters from the to-be-matched repeated character storage module, and storing the repeated characters in the decoded character storage module by partition.
[0120] As a specific implementation method, the 100th characters from the decompressed data are all original characters, and these 100 original characters are stored in the to-be-matched repeated character storage module and the decoded character storage module. When the 101st compressed data is added, its literal_L value is greater than 256, indicating that some characters of the compressed data are repeated with the characters stored in the to-be-matched repeated character storage module and the decoded character storage module. Therefore, the repeated character index distance pos (position information) and the repeated character index length length (length information) of the compressed data can be solved, so as to lock in the repeated characters solved this time. For example, if pos=50 and length=50 are solved, then the 50th to 99th original characters are taken out from the to-be-matched repeated character storage module. This section of character data is the repeated characters solved from the compressed data. Then this section of repeated characters is stored in the decoded character storage module, and finally stored in the DDR through the bus.
[0121] in, Figure 8 It is the splicing principle diagram of the present invention, such as Figure 8 As shown, step S4, the splicing operation includes: splicing the decoded data stored in N partitions to obtain a spliced decoded data, and storing the spliced decoded data in the access memory DDR, wherein N is a positive integer greater than 1.
[0122] In the above embodiment, the decoded data is spliced in the sub-storage unit and then sent to the access memory DDR through the AXI bus. When the decoded data is needed, it can be used at any time. The above splicing method can effectively save circuit area and is convenient for sharing with other circuits. The core idea is to splice multiple 1-byte data into one multi-byte data.
[0123] Example 2: Combined with Figure 8 , the data processing device provided by the present invention is explained.
[0124] Figure 8 is a schematic diagram of the structure of a data processing device provided according to the second embodiment of the present disclosure, such as Figure 8 As shown, the data processing device is applied to a decompression unit of a circuit, and the circuit further comprises: a flash memory FLASH and an access memory DDR;
[0125] The device comprises:
[0126] A response module, responding to a decompression request for compressed data;
[0127] A decompression module receives and decompresses the compressed data in the flash memory FLASH to obtain decompressed data stored in partitions;
[0128] A splicing module, which performs a splicing operation on the decompressed data and stores the spliced data in a memory DDR;
[0129] Among them, the decompression module includes:
[0130] A Huffman code word value table acquisition unit determines the corresponding first Huffman code word value table and second Huffman code word value table respectively according to two compressed data in the compressed data;
[0131] A Huffman decoding unit, performing Huffman decoding on a first Huffman codeword value table according to first preset byte data in the compressed data, and performing Huffman decoding on a second Huffman codeword value table according to second preset byte data in the compressed data to obtain decoding information, the compressed data obtains corresponding decompressed data according to the decoding information, and stores the decoded data in N partitions;
[0132] The decoded information includes decoded data and corresponding index information, the decoded data includes the original characters and repeated characters of the meta information, the index information includes the decoded data index length, the decoded data index position, the basic repeated character index length and the basic repeated character index distance length, and N is a positive integer greater than 1;
[0133] The splicing module is used to splice the decoded data stored in the N partitions to obtain a spliced decoded data, and store the spliced decoded data in the access memory DDR.
[0134] The specific implementation of the data processing device provided in the embodiment of the present disclosure is consistent with the data processing method provided in Example 1, and will not be described in detail here.
[0135] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0136] Among them, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data processing method described in the aforementioned aspect.
[0137] Among them, a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the data processing method described in the above aspect.
[0138] Among them, a computer program product includes a computer program, and the computer program implements the data processing method described in the above aspect when executed by a processor.
[0139] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0140] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0141] Any process or method description in a flowchart or otherwise described herein may be understood to represent a unit, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
[0142] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.
[0143] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0144] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.
Claims
1. A data processing method, characterized in that: In a decompression unit applied to a circuit, the circuit further comprises: a flash memory, an access memory and a bus control unit; the decompression unit is connected to the flash memory via an axi bus, and the decompression unit is connected to the access memory via an axi bus; The bus control unit is connected to the decompression unit, and the bus control unit configures the register to achieve the purpose of controlling the function change of the decompression unit; The method comprises: In response to a decompression request for compressed data, receiving and decompressing the compressed data in the flash memory to obtain decompressed data stored in partitions, and performing a splicing operation on the decompressed data and storing it in the access memory; The decompression operation includes: Determine the corresponding first Huffman codeword value table and second Huffman codeword value table respectively according to two compressed data in the compressed data; perform Huffman decoding on the first Huffman codeword value table according to the first preset byte data in the compressed data, and perform Huffman decoding on the second Huffman codeword value table according to the second preset byte data in the compressed data to obtain decoding information, the compressed data obtains corresponding decompressed data according to the decoding information, and stores the decompressed data in N partitions; wherein the decoding information includes the decoded data and corresponding index information, the decoded data includes the original character and repeated character of the meta information, the index information includes the repeated character index length and the repeated character index position, and N is a positive integer greater than 1; The splicing operation includes: splicing the decoded data stored in the N partitions to obtain a spliced decoded data, and storing the spliced decoded data in the access memory; The access memory is a single-port SRAM memory.
2. The data processing method according to claim 1, characterized in that: The method of performing Huffman decoding on the first Huffman codeword value table according to the first preset byte data in the compressed data, and performing Huffman decoding on the second Huffman codeword value table according to the second preset byte data in the compressed data to obtain decoding information specifically includes: Perform Huffman decoding on the first Huffman codeword value table according to the first preset byte data in the compressed data to obtain first decoded data, the first decoded data including a first decoded character and corresponding first index information, the first decoded character including an original character and a repeated character, and the first index information including a first decoded character index length and a basic repeated character index length; The second Huffman code word value table is Huffman decoded according to the second preset byte data in the compressed data to obtain second decoded data, the second decoded data includes a second decoded character and corresponding second index information, the second decoded character includes a repeated character, and the second index information includes a repeated character index distance.
3. The data processing method according to claim 1, characterized in that: The decompression unit includes a first storage module and a second storage module, and the compressed data obtains corresponding decompressed data according to the decoding information, specifically including: Determine the type of the first solution character by comparing the first solution character index length with the first preset length: if the first solution character index length is equal to the first preset length, the first solution character is the end character, and decoding ends; If the first decoded character index length is less than the first preset length, the to-be-decoded code stream in the compressed data is used as the original character, and the original character is stored as the decoded data partitions in the first storage module and the second storage module; If the first decrypted character index length is greater than the first preset length, the to-be-decoded code stream in the compressed data is used as repeated data, a decrypted character operation is performed, the repeated characters are located and obtained from the first storage module, and the repeated characters are stored as decoded data partitions in the second storage module.
4. The data processing method according to claim 3, characterized in that: The de-repeating character operation comprises: Calculate the length of the repeated character index: obtain the corresponding code stream length prefix and code stream length suffix bit width according to the first solution character index length, and use the sum of the code stream length prefix and code stream length suffix bit width as the repeated character index length; Obtaining the repeated character index distance: Obtaining the corresponding matching position base address and offset address width according to the second solution character index position, and taking the sum of the matching position base address and the offset address width as the repeated character index distance; The repeated characters are located according to the repeated character index length and the repeated character index distance, and the repeated characters are stored as decoded data partitions in the second storage module.
5. The data processing method according to claim 1, characterized in that: The decompression unit includes a decoding completed character storage module, and the decoding completed character storage module includes a plurality of partitions; The decompression operation includes storing the decoded data partitions into N partitions of the decoded character storage module.
6. A data processing device, characterized in that: In a decompression unit applied to a circuit, the circuit further comprises: a flash memory, an access memory and a bus control unit; the decompression unit is connected to the flash memory via an axi bus, and the decompression unit is connected to the access memory via an axi bus; The bus control unit is connected to the decompression unit, and the bus control unit configures the register to achieve the purpose of controlling the function change of the decompression unit; The device comprises: A response module, responding to a decompression request for compressed data; A decompression module receives and decompresses the compressed data in the flash memory to obtain decompressed data stored in partitions; a splicing module splices the decompressed data and stores it in the access memory; Among them, the decompression module includes: A Huffman code word value table acquisition unit determines the corresponding first Huffman code word value table and second Huffman code word value table according to two compressed data in the compressed data; A Huffman decoding unit, performing Huffman decoding on a first Huffman codeword value table according to first preset byte data in the compressed data, and performing Huffman decoding on a second Huffman codeword value table according to second preset byte data in the compressed data to obtain decoding information, the compressed data obtains corresponding decompressed data according to the decoding information, and stores the decompressed data in N partitions; The decoded information includes decoded data and corresponding index information, the decoded data includes the original character and repeated character of the meta information, the index information includes the repeated character index length and repeated character index position, and N is a positive integer greater than 1; The splicing module is used to splice the decoded data stored in the N partitions to obtain a spliced decoded data, and store the spliced decoded data in the access memory; The access memory is a single-port SRAM memory.
7. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.
8. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.
Citation Information
Patent Citations
Data compression method and flash memory device
CN114968837A