Method, device and equipment for decompressing data and method, device and equipment for compressing data

Abstract

The invention provides a data decompression method, a data compression method, devices and equipment, and the method comprises the steps: obtaining a compressed data packet, and analyzing the compressed data packet, thereby obtaining a packet header and compressed data; the packet header comprises a mapping table and a decoding table; the mapping table comprises a plurality of indication code words and a plurality of base addresses; the indication code words are in one-to-one correspondence with the base addresses; aiming at the primary query, querying the mapping table based on the index information to obtain an indication code word and a base address; for a base address, querying a decoding table based on the base address to obtain at least one decoded character; each base address indicates one storage area; the base address indicates at least one offset address code word in the compressed data; for one matched character parameter, querying a preset dictionary based on the matched character parameter to obtain an original character; and splicing the plurality of original characters and the plurality of unmatched characters to obtain target decompressed data. According to the technical scheme, implementation logic is simplified, and hardware implementation is easy.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method for decompressing data, a method, device and equipment for compressing data. Background Art

[0002] gzip is a lossless file compression format based on the DEFLATE algorithm. During the encoding process, the DEFLATE algorithm performs two operations on the data: preprocessing (prefix encoding) and Huffman coding. First, the preprocessing operation uses the LZ77 (Lempel-Ziv 1977) algorithm, which scans the input data and replaces repeated data blocks with pointers. In this way, repeated data in the file only needs to be stored once, thereby reducing the file size. Secondly, Huffman coding continues to compress the preprocessed data blocks.

[0003] In the decoding process, Huffman decoding includes static decoding and dynamic decoding. Among them, static decoding requires checking multiple tables. Dynamic decoding not only requires checking multiple tables, but also requires building multiple tables and performing run-length coding. Therefore, the Huffman decoding method in the related art has complex implementation logic and is not easy to implement in hardware. Summary of the invention

[0004] The embodiments of the present application provide a method for decompressing data, a method, device and apparatus for compressing data, which reduces the number of table lookups during decoding, and does not require rebuilding tables or performing run-length encoding, thereby simplifying the implementation logic and facilitating hardware implementation. The technical solution is as follows:

[0005] According to a first aspect of an embodiment of the present application, a method for decompressing data is provided, the method comprising:

[0006] Obtain a compressed data packet, and parse the compressed data packet to obtain a packet header and compressed data; the packet header includes a mapping table and a decoding table; the mapping table includes a plurality of indicator code words and a plurality of base addresses; the indicator code words correspond to the base addresses one by one; the decoding table includes a plurality of decoding characters; the compressed data includes index information;

[0007] For a query, query the mapping table based on the index information to obtain the indication codeword and the base address;

[0008] For one of the base addresses, query the decoding table based on the base address to obtain at least one of the decoded characters; the decoded characters include matching character parameters and unmatched characters;

[0009] For one of the matching character parameters, query a preset dictionary based on the matching character parameter to obtain the original character;

[0010] The plurality of original characters are concatenated with the plurality of unmatched characters to obtain target decompressed data.

[0011] In one possible implementation, each of the base addresses indicates an initial storage area; the base address indicates at least one offset address codeword in the compressed data; each of the offset address codeword indicates an offset address; each of the offset addresses indicates an offset storage area; the initial storage area and the offset storage area are part of the decoding table; the initial storage area is used to store one of the decoded characters; and the offset storage area is used to store one of the decoded characters.

[0012] In one possible implementation, the matching character parameters include a length value and an offset value; the index information includes multiple first code lengths, multiple second code lengths, and multiple third code lengths; the first code length indicates multiple unmatched character codewords; the second code length indicates multiple length value codewords; and the third code length indicates multiple offset value codewords.

[0013] In a possible implementation, the mapping table includes a first mapping table and a second mapping table; the first mapping table includes multiple unmatched character indication codewords, multiple first base addresses, multiple length value indication codewords and multiple second base addresses; the unmatched character indication codeword corresponds one-to-one to the first base address; the unmatched character indication codeword is less than or equal to the unmatched character codeword; the length value indication codeword corresponds one-to-one to the second base address; the length value indication codeword is less than or equal to the length value codeword; the second mapping table includes multiple offset value indication codewords and multiple third base addresses; the offset value indication codeword corresponds one-to-one to the third base address; the offset value indication codeword is less than or equal to the offset value codeword.

[0014] In a possible implementation, querying the mapping table based on the index information for a query to obtain the indication codeword and the base address includes:

[0015] querying the first mapping table based on the first code length to obtain the unmatched character indication codeword; querying the first mapping table based on the unmatched character indication codeword to obtain the first base address; or

[0016] querying the first mapping table based on the second code length to obtain the length value character indication codeword; querying the first mapping table based on the length value indication codeword to obtain the second base address; or

[0017] The second mapping table is queried based on the third code length to obtain the offset value indicating codeword; and the second mapping table is queried based on the offset value indicating codeword to obtain the third base address.

[0018] In a possible implementation, the decoding table includes a first decoding table and a second decoding table; the first decoding table includes a plurality of the unmatched characters and the length values; the second decoding table includes a plurality of the offset values;

[0019] For one of the base addresses, querying the decoding table based on the base address to obtain at least one of the decoded characters includes:

[0020] Based on the first base address, at least one first offset address is obtained; based on the first base address and one of the first offset addresses, one of the unmatched characters is obtained; the first base address indicates a first initial storage area; the first initial storage area is a part of the first decoding table; the first initial storage area is used to store one of the unmatched characters; the first base address and one of the first offset addresses indicate a first offset storage area; the first offset storage area is a part of the first decoding table; the first offset storage area is used to store one of the unmatched characters; or

[0021] Based on the second base address, at least one second offset address is obtained; based on the second base address and one second offset address, one length value is obtained; the second base address indicates a second initial storage area; the second initial storage area is a part of the first decoding table; the second initial storage area is used to store one length value; the second base address and one second offset address indicate a second offset storage area; the second offset storage area is a part of the first decoding table; the second offset storage area is used to store one length value; or

[0022] At least one third offset address is obtained based on the third base address; based on the third base address and one third offset address, one offset value is obtained; the third base address indicates a third initial storage area; the third initial storage area is a part of the second decoding table; the third initial storage area is used to store an interval code; the third base address and one third offset address indicate a third offset storage area; the third offset storage area is a part of the second decoding table; the third offset storage area is used to store one interval code; the interval code indicates one offset value.

[0023] In a possible implementation, the method further includes:

[0024] For a set of corresponding length values ​​and offset values, the length values ​​and the offset values ​​are combined to obtain a matching combination.

[0025] In a possible implementation, the method further includes:

[0026] Obtaining a data to be processed; the data to be processed is the matching combination or the unmatched character;

[0027] If the type flag of the data to be processed is the first threshold, the data to be processed is the matching combination; the type flag indicates the matching combination or the unmatched combination;

[0028] If the type flag of the data to be processed is the second threshold, the data to be processed is the unmatched character.

[0029] In a possible implementation, for one of the matching character parameters, querying a preset dictionary based on the matching character parameter to obtain the original character includes:

[0030] For one of the matching combinations, when the offset value is less than the length value, the original character is determined based on the preset dictionary and the offset value, and the original character is copied based on the length value to obtain matching data; the length of the matching data is equal to the length value; or

[0031] For one of the matching combinations, when the offset value is greater than the length value, the matching data is determined based on the preset dictionary, the offset value and the difference value; the difference value is the difference between the offset value and the length value.

[0032] In a possible implementation, the step of querying a preset dictionary based on the matching character parameter to obtain the original character for one of the matching character parameters further includes:

[0033] The original character is obtained by searching a memory based on the preset dictionary and the offset value, and searching a cache based on a preset sliding window and the offset value; the memory is used to store the target decompressed data; and the cache is used to store the target decompressed data that has not been written back to the memory.

[0034] In a possible implementation, the method further includes:

[0035] For one of the matching combinations, when the matching combination is being decoded, a plurality of the unmatched characters are concatenated;

[0036] When the matching combination has been decoded, the splicing of the unmatched characters is stopped to obtain spliced ​​unmatched characters.

[0037] In a possible implementation manner, the compressed data includes a plurality of compressed data blocks, and the packet header includes the size of each compressed data block and the number of the compressed data blocks.

[0038] According to a second aspect of an embodiment of the present application, a method for compressing data is provided, the method comprising:

[0039] Get the data to be compressed;

[0040] Preprocessing the data to be compressed based on a preset dictionary to obtain a preprocessing result; the preprocessing result includes multiple combinations; the combination includes unmatched characters and matching character parameters; the matching character parameters include length values ​​and offset values;

[0041] Segmenting a plurality of the preprocessing combinations to obtain a plurality of data segments;

[0042] For one of the data segments, the unmatched character is encoded based on a first code table to obtain an unmatched character codeword; and the length value is encoded based on the first code table to obtain the length value codeword; the first code table includes a first numerical interval, a first coding bit, a second numerical interval and a second coding bit; the first numerical interval indicates the unmatched character; the first coding bit indicates the code length range of the unmatched character; the second numerical interval indicates the length value; the second coding bit indicates the code length range of the length value;

[0043] For one of the data segments, the offset value is encoded based on a second code table to obtain an offset value codeword; the second code table includes a third numerical interval, an interval code and an additional bit; the additional bit indicates the code length range of the third numerical interval; the interval code is encoded based on the initial bit;

[0044] For one of the data segments, the unmatched character codewords, the length value codewords and the offset value codewords are taken as a compressed data block.

[0045] In a possible implementation manner, segmenting the plurality of preprocessing combinations to obtain a plurality of data segments includes:

[0046] Obtaining t+1 consecutive preset segments from the uncoded portion of the preprocessing combination, where t is an integer greater than or equal to 0;

[0047] Obtaining a preset encoding result; the preset encoding result is obtained by encoding t+1 of the preset segments based on the first code table and the second code table;

[0048] Obtaining a first compression rate; the first compression rate is obtained based on the preset encoding result and t+1 of the preset segments;

[0049] If the first compression rate is greater than the second compression rate, t preset fragments are taken as one data fragment; the second compression rate is the compression rate of t preset fragments; or if the first compression rate is less than the second compression rate, t+2 preset fragments are encoded based on the first code table and the second code table until the nth compression rate is greater than the n-1th compression rate, and t+n-1 preset fragments are taken as one data fragment; the nth compression rate is the compression rate of t+n preset fragments; the n-1th compression rate is the compression rate of t+n-1 preset fragments; n is an integer greater than or equal to 1.

[0050] In a possible implementation, the method further includes:

[0051] Constructing a first Huffman tree and a second Huffman tree; the first Huffman tree is constructed based on a plurality of the unmatched characters and the length values; the second Huffman tree is constructed based on a plurality of the offset values;

[0052] Based on the first Huffman tree, a plurality of first code lengths and a plurality of second code lengths are obtained; the first code length indicates a plurality of the unmatched character code words; the second code length indicates a plurality of the length value code words;

[0053] Based on the second Huffman tree, a plurality of third code lengths are obtained; the third code lengths indicate a plurality of the offset value code words.

[0054] In a possible implementation, the method further includes: generating a mapping table and a decoding table based on the offset value codeword, the length value codeword and the unmatched character codeword; the mapping table includes multiple indication codewords and multiple base addresses; the indication codeword corresponds one-to-one to the base address; the decoding table includes multiple decoded characters; each base address indicates an initial storage area; each base address indicates at least one offset address; the offset address indicates an offset storage area; the initial storage area and the offset storage area are part of the decoding table; the initial storage area is used to store one of the decoded characters; the offset storage area is used to store one of the decoded characters; the indication codeword is determined based on any one of the offset value codeword, the length value codeword and the unmatched character codeword.

[0055] According to a third aspect of an embodiment of the present application, a device for decompressing data is provided, the device comprising:

[0056] A parsing module, used to obtain a compressed data packet, and parse the compressed data packet to obtain a packet header and compressed data; the packet header includes a mapping table and a decoding table; the mapping table includes a plurality of indicator code words and a plurality of base addresses; the indicator code words correspond to the base addresses one by one; the decoding table includes a plurality of decoding characters; the compressed data includes index information;

[0057] A first decoding module, configured to query the mapping table based on the index information for a query, and obtain the indication codeword and the base address;

[0058] A second decoding module is used for querying the decoding table based on the base address for one of the base addresses to obtain at least one of the decoded characters; the decoded characters include matching character parameters and unmatched characters;

[0059] A third decoding module is used for querying a preset dictionary based on one of the matching character parameters to obtain the original character;

[0060] The splicing module is used to splice the multiple original characters with the multiple unmatched characters to obtain target decompressed data.

[0061] According to a fourth aspect of an embodiment of the present application, a device for compressing data is provided, including:

[0062] An acquisition module, used for acquiring data to be compressed;

[0063] A preprocessing module, used for preprocessing the data to be compressed based on a preset dictionary to obtain a preprocessing result; the preprocessing result includes a plurality of combinations; the combination includes unmatched characters and matching character parameters; the matching character parameters include a length value and an offset value;

[0064] A segmentation module, used for segmenting the plurality of preprocessing combinations to obtain a plurality of data segments;

[0065] A first encoding module is configured to encode the unmatched character based on a first code table for one of the data segments to obtain an unmatched character codeword; and encode the length value based on the first code table to obtain the length value codeword; the first code table includes a first numerical interval, a first coding bit, a second numerical interval, and a second coding bit; the first numerical interval indicates the unmatched character; the first coding bit indicates a code length range of the unmatched character; the second numerical interval indicates the length value; the second coding bit indicates the code length range of the length value;

[0066] A second encoding module is used for encoding the offset value based on a second code table for one of the data segments to obtain an offset value codeword; the second code table includes a third numerical interval, an interval code and an additional bit; the additional bit indicates the code length range of the third numerical interval; the interval code is encoded based on the initial bit;

[0067] The result module is used to treat the unmatched character codeword, the length value codeword and the offset value codeword as a compressed data block for one of the data fragments.

[0068] According to a fifth aspect of an embodiment of the present application, a computer device is provided, characterized in that the computer device includes a processor and a memory, the memory is used to store at least one program, and the at least one program is loaded by the processor and executes a method for decompressing data or a method for compressing data.

[0069] According to the sixth aspect of an embodiment of the present application, a computer-readable storage medium is provided, characterized in that at least one program is stored in the computer-readable storage medium, and the at least one program is loaded and executed by a processor to implement a method for decompressing data or a method for compressing data.

[0070] In an embodiment of the present application, an embodiment of the present application provides a method for decompressing data. In the embodiment of the present application, when decompressing data, a mapping table, a decoding table and index information are directly obtained. The mapping table is searched through the index information to obtain a base address. The decoding table is searched through the base address to obtain an unmatched character, a length value or an offset value. The technical solution of the present application does not require the reconstruction of the table and the run-length encoding during the decoding process. The decoding table can be quickly searched through the base address, which simplifies the decompression logic, thereby effectively improving the throughput rate and being easy to implement in hardware. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0072] Figure 1 is a schematic diagram of an implementation environment provided according to an embodiment of the present application;

[0073] Figure 2 Provides a storage Huffman coding mode;

[0074] Figure 3 A first table to be looked up for a static Huffman coding pattern is provided;

[0075] Figure 4 A second table is provided that needs to be looked up for static Huffman coding patterns;

[0076] Figure 5 A third table is provided that needs to be looked up for static Huffman coding patterns;

[0077] Figure 6 A data structure for a dynamic Huffman coding pattern is provided;

[0078] Figure 7 A coding process of a dynamic Huffman coding mode is provided;

[0079] Figure 8 It is a flowchart of a method for decompressing data provided according to an embodiment of the present application;

[0080] Fig. 9 is a schematic diagram of the structure of a first mapping table provided according to an embodiment of the present application;

[0081] Fig.10 is a schematic diagram of the structure of a second mapping table provided according to an embodiment of the present application;

[0082] Fig.11 is a schematic diagram of the structure of a first decoding table provided according to an embodiment of the present application;

[0083] Fig.12 is a schematic diagram of the structure of a second decoding table provided according to an embodiment of the present application;

[0084] Fig.13 is a schematic diagram of a hardware structure of a method for executing decompressed data provided according to an embodiment of the present application;

[0085] Fig.14 is a hardware logic schematic diagram of a Huffman module provided according to an embodiment of the present application;

[0086] Fig.15 It is a hardware logic schematic diagram of an LZ77 module provided according to an embodiment of the present application;

[0087] Fig.16 is a flowchart of a method for compressing data provided according to an embodiment of the present application;

[0088] Fig.17 is a compression ratio comparison schematic diagram provided according to an embodiment of the present application;

[0089] Fig.18 is a flowchart of step 1603 provided according to an embodiment of the present application;

[0090] Fig.19is a schematic diagram of the structure of a first code table provided according to an embodiment of the present application;

[0091] Fig. 20 is a schematic diagram of the structure of a second code table provided according to an embodiment of the present application;

[0092] Fig.21 is a structural schematic diagram of a device for decompressing data provided according to an embodiment of the present application;

[0093] Fig. 22 is a structural schematic diagram of a device for compressing data provided according to an embodiment of the present application;

[0094] Fig.23 is a schematic diagram of the structure of a terminal provided according to an embodiment of the present application;

[0095] Fig.24 It is a structural diagram of a server provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0096] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0097] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0098] In this application, the terms "first", "second", etc. are used to distinguish between identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there a limitation on the quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms.

[0099] These terms are only used to distinguish one element from another element. For example, without departing from the scope of various examples, a first action can be referred to as a second action, and similarly, a second action can also be referred to as a first action. Both the first action and the second action can be actions, and in some cases, can be separate and different actions.

[0100] Here, at least one means one or more than one, for example, at least one action can be one action, two actions, three actions, or any other action that is an integer greater than or equal to one. And multiple means two or more than two, for example, multiple actions can be two actions, three actions, or any other action that is an integer greater than or equal to two.

[0101] Figure 1 It is a schematic diagram of an implementation environment provided according to an embodiment of the present application, and the implementation environment may include a terminal 101 and a server 102.

[0102] The terminal 101 is provided with a chip, on which a device for compressing data and a device for decompressing data are provided, so as to compress and decompress the data.

[0103] The terminal 101 may be a smart phone having the above chip, a wearable device, a personal computer, a laptop, a tablet computer, a smart TV, a vehicle-mounted terminal, etc.

[0104] The server 102 may be a single server, a server cluster consisting of multiple servers, or a cloud processing center.

[0105] The terminal 101 is connected to the server 102 via a wired or wireless network.

[0106] In some embodiments, a wireless network or a wired network uses standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or a virtual private network. In some embodiments, technologies and / or formats including HyperText Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged through the network. Additionally, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. can be used to encrypt all or some of the links. In other embodiments, customized and / or proprietary data communication technologies can also be used to replace or supplement the above data communication technologies.

[0107] In the related art, the LZ77 compression algorithm includes a dictionary cache and a sliding window match; that is, a string sequence at the current position has also appeared in the previous historical data. Then, representing it in a special format or a special small sequence can achieve the compression effect because the special format or special small sequence is usually smaller than the original string sequence. The historical data mentioned here is the dictionary cache, the window of the dictionary is fixed, and data matching is completed by continuously sliding the window forward. The LZ77 algorithm performs lossless compression based on the data matching situation and outputs (d, l, c). Among them, d represents the offset value (distance); l represents the length value (length); c represents the unmatch character (char). During decompression, (d, l, c) will be parsed to obtain the original data, that is, the data to be compressed. dict represents the dictionary.

[0108] In an example, when d = 0; l = 0, it represents one byte of data before the pointer, and the decoded output is c. When d ≥ l, it means there is matching data, and the decoded output is the dictionary dict[d - l - 1:d]; that is, this section of data in the dictionary cache includes all the data corresponding to l. When d < l, it means cyclic coding occurs, and multiple reads are required, and then cyclic splicing is performed from left to right.

[0109] The Huffman compression algorithm encodes the gzip data part into several compressed data blocks; the compressed data block can also be represented as a block. The data structure of each compressed data block is independent of each other. There are three encoding modes for compressed data blocks, which are distinguished by three bits of the compressed data block header (not shown in the figure). Among them, the Huffman encoding mode includes the storage Huffman encoding mode, the static Huffman encoding mode and the dynamic Huffman encoding mode.

[0110] Figure 2 A storage Huffman coding mode is provided.

[0111] Combine the following Figure 2 The storage of the Huffman coding mode is exemplified.

[0112] In the Huffman coding mode, LEN represents the length of the code stream, which occupies two bytes. NLEN represents the complement of the length of the code stream, which occupies two bytes.

[0113] Figure 3 A first table to look up for a static Huffman coding pattern is provided.

[0114] Combine the following Figure 3 The static Huffman coding mode is exemplified.

[0115] In the static Huffman coding mode, it is necessary to search the fixed first table. The first table includes three columns. The first column indicates the unmatched character or length value. The second column indicates the code length of the codeword, that is, the number of bits. For example: if the value falls between 0-143, it means an unmatched character, and one of "00110000-10111111" is used to represent the codeword after the unmatched character is encoded. Among them, the unmatched character is encoded together with the length value in the first table. 0-255 represents an unmatched character; 257-287 represents the length value; 256 indicates the end of the unmatched character encoding.

[0116] Figure 4 A second table is provided that is looked up for static Huffman coding patterns.

[0117] Combine the following Figure 4 The static Huffman coding mode is exemplified.

[0118] The second table is the length value interval encoding table, that is, the length value is interval encoded, and 257 to 287 represents the length value. The actual length value range is 3 to 258. A total of 29 intervals are included, and each interval is encoded. However, since some intervals include multiple length values, additional bits are set to distinguish the length values ​​of each interval.

[0119] Figure 5A third table is provided that is looked up for static Huffman coding patterns.

[0120] Combine the following Figure 5 The static Huffman coding mode is exemplified.

[0121] The third table is the offset value interval coding table. That is, the offset value is also interval coded. The offset value is independently coded, ranging from 0 to 32768. The offset value is divided into 30 intervals. Each interval of the offset value is coded with 5 bits of Huffman coding. However, since some intervals include multiple offset values, additional bits are set to distinguish the offset values ​​of each interval.

[0122] In addition, when performing dynamic Huffman coding, it is also necessary to look up the second table and the third table mentioned above.

[0123] Figure 6 A data structure for a dynamic Huffman coding pattern is provided.

[0124] Below Figure 6 Explain the various symbols involved.

[0125] In the figure, the length of header (block header) is 3 bits, which is used to record the dynamic Huffman coding mode and compression status. The length of HLIT (Huffman Literal counts) is 5 bits, which is used to record the number of unmatched character / length value codeword length sequences (cl1, code length value 1). The number of cl1 is HLIT+257 (0-255 unmatched characters plus a 256 end, the length value is variable), so the number of cl1 is [257, 286]. The length of HDIST is 5 bits, which is used to record the number of offset value codeword length sequences (cl2). The number of cl2 is HDIST (Huffman offset value counts) + 1, and the number of cl2 is [1, 30]. The length of HCLEN (Huffman code length value s) is 5 bits, which is used to record the number of codeword length sequences (ccl, run-length encoding) in the SQ (Sequence) code table. The number of ccl is HCLEN+4, with a maximum of 19. The length of ccl is (HCLEN+4)*3 bits, which is used to construct the SQ code table. After cl1 is shortened by run-length coding, SQ1 is obtained, and then SQ1 is encoded by Huffman. When decoding, it can be decoded to HLIT+257. It is used to construct the unmatched character / length value code table. After cl2 is shortened by run-length coding, SQ2 is obtained, and then SQ2 is encoded by Huffman. When decoding, it can be decoded to HDIST+1. It is used to construct the offset value code table. When decoding the coded stream of unmatched character code words, length value code words and offset value code words, first use the unmatched character / length value code table for decoding. If the decoded integer is between 0-255, it indicates an unmatched character, and the unmatched character / length value code table is used for decoding; if it is between 257-285, it indicates a length value. Decode using the offset value code table to obtain the offset value; if it is equal to 256, it indicates that the decoding of the compressed data block is completed.

[0126] Figure 7 A dynamic Huffman coding mode encoding process is provided.

[0127] Combine the following Figure 7 The encoding process of the dynamic Huffman coding mode is exemplified.

[0128] The encoding process of the dynamic Huffman coding mode includes: counting unmatched characters and length values ​​to generate a Huffman tree, called an unmatched character / length value code table; counting offset values ​​to generate a Huffman tree, called an offset value code table; according to the properties of the paradigm Huffman table, the Huffman tree information is saved using the code word length, called cl (code length value), that is, the lengths of the unmatched character / length value code table and the offset value code table are cl1 and cl2 respectively; run-length encoding is performed on cl1 / cl2 to obtain SQ1 / SQ2; counting SQ1+SQ2 to generate a Huffman tree, called an SQ code table. Since SQ1+SQ2 has only 19 symbols in total, it is believed that the depth of the Huffman tree does not exceed 7, so each element can be represented by 3 bits, ranging from 0 to 7.

[0129] The decoding process of the dynamic Huffman coding mode includes: reading ccl and constructing the SQ code table. Reading the SQ1 bit / SQ2 bit, and constructing the SQ1 / SQ2 sequence according to the SQ code table. Performing run-length decoding on SQ1 / SQ2 to obtain cl1 / cl2. According to cl1 / cl3, constructing the unmatched character / length value code table and the offset value code table. Reading the unmatched character codeword / length value codeword encoding stream, and obtaining the unmatched character / length value according to the unmatched character / length value code table. Reading the offset value codeword encoding stream, and obtaining the offset value according to the offset value code table, and obtaining the (unmatched character, length value, offset value) sequence for use by the LZ77 decoding module.

[0130] Through the above analysis, it can be seen that dynamic Huffman requires 3 sets of Huffman tables. According to the algorithm description, the specifications of the 3 sets of Huffman tables are as follows: the depth of the unmatched character / length value code table is 286, and the longest code word bit length is Nmax=15; the depth of the offset value code table is 30, and the longest code word bit length is Nmax=15; the depth of the SQ code table is 19, and the longest code word bit length is Nmax=7. Taking the smallest SQ code table as an example, it can be implemented in the following two ways: The first implementation method: add a counter to each group of code length (cl) statistics to record each search operation, then 7 counters and 19 clock cycles are required to complete the construction of the SQ code table. The second implementation method: use the addition matrix, 19 1-bit additions, mask the repeated values, that is, the required offset value, 19 1-bit adders. In order to meet the timing, an additional 2 clock cycles need to be stored; therefore, 3 clock cycles are required to complete the operation of building the SQ code table. Although the second implementation method can quickly build a table, it is difficult to use a set of addition matrices to implement such a large table as the unmatched character / length value code table / offset value code table, and multiple sets of implementation are required. Therefore, both the area and timing are difficult to meet. In addition, the dynamic Huffman table lookup has a long latency. This can be seen from the dynamic decompression process: starting to read ccl to obtain the final sequence of unmatched characters, length values, and offset values ​​is a serial process, so there will be a large delay. In addition, the compression rate is low if only a static Huffman code table is used. From the static Huffman code table, it can be seen that the maximum is to compress the decompression code from 9 bits to 7 bits. Therefore, most of the compressed files need to fall in the range of 256 to 279 to achieve a better compression result. Therefore, the table lookup efficiency is low. In short, the algorithm for decompressing data and the algorithm for compressing data in the related art are complex in logic and not easy to implement in hardware.

[0131] In order to solve the technical problems existing in the related art, the embodiments of the present application provide a method for decompressing data and a method for compressing data.

[0132] Figure 8 is a flowchart of a method for decompressing data provided in accordance with an embodiment of the present application, such as Figure 8 As shown, in the embodiment of the present application, the application is described by taking the application on a terminal with a chip as an example. The method comprises the following steps:

[0133] In step 801, the terminal obtains a compressed data packet, and parses the compressed data packet to obtain a packet header and compressed data.

[0134] In some embodiments, the terminal configures registers, parses registers, and obtains multiple read instructions, control instructions, and write instructions. The terminal obtains a compressed data packet in response to the read instruction. The terminal parses the compressed data packet in response to the control instruction to obtain a header and compressed data. The terminal decompresses the compressed data to obtain target decompressed data. The terminal writes the target decompressed data into a memory in response to the write instruction. In the process of decompressing the compressed data, the compressed data is decompressed according to the compression order of the compressed data to ensure the correct order of the target decompressed data.

[0135] In some embodiments, the packet header includes a mapping table and a decoding table; the mapping table includes multiple indicator codewords and multiple base addresses; the indicator codewords correspond to the base addresses one by one; the decoding table includes multiple decoding characters; the compressed data includes index information, which can be understood as compressed data including a coding stream; the coding stream includes multiple unmatched character codewords, length value codewords and offset value codewords. When an unmatched character codeword is read, a first code length is obtained. Or when a length value codeword is read, a second code length is obtained. Or when an offset value code length is read, a third code length is obtained. Optionally, the compressed data includes multiple compressed data blocks. Optionally, the packet header includes the size of each compressed data block and the number of compressed data blocks. It should be noted that the mapping table and the decoding table are generated based on software and are constructed before decoding, so there is no need to build a table based on hardware during the decoding process, and only hardware-based table lookup is required. In the embodiment of the present application, the information included in the packet header is greatly reduced compared to the packet header in the related art, which reduces the hardware storage area, shortens the table lookup delay, and is more suitable for hardware decompression scenarios.

[0136] In one example, the matching character parameter includes a length value and an offset value; the index information includes multiple first code lengths, multiple second code lengths, and multiple third code lengths; the first code length indicates multiple unmatched character code words; the second code length indicates multiple length value code words; the third code length indicates multiple offset value code words. Optionally, the terminal reads the compressed data to obtain the index information. When any one of the first code length, the second code length, or the third code length is read, the mapping table is searched.

[0137] In one example, the first code length is determined based on a first Huffman tree; the second code length is determined based on a second Huffman tree; the first Huffman tree is constructed based on a first statistic and a second statistic; the second Huffman tree is constructed based on a third statistic; the first statistic indicates the frequency of occurrence of the same unmatched characters; the second statistic indicates the frequency of occurrence of the same length value; the third statistic indicates the frequency of occurrence of the same offset value.

[0138] Fig. 9 It is a structural schematic diagram of a first mapping table provided according to an embodiment of the present application.

[0139] Combine the following Fig. 9 The first mapping table is exemplarily described.

[0140] In some embodiments, the mapping table includes a first mapping table and a second mapping table; the first mapping table includes a plurality of unmatched character indication codewords, a plurality of first base addresses, a plurality of length value indication codewords, and a plurality of second base addresses; the unmatched character indication codeword corresponds to the first base address one-to-one; the unmatched character indication codeword is less than or equal to the unmatched character codeword; the length value indication codeword corresponds to the second base address one-to-one; the length value indication codeword is less than or equal to the length value codeword. It should be understood that the unmatched character indication codeword is less than or equal to the unmatched character codeword means that the numerical value indicated by the unmatched character indication codeword is less than or equal to the numerical value indicated by the unmatched character codeword. Similarly, the numerical value indicated by the length value indication codeword is less than or equal to the numerical value indicated by the length value codeword. That is, the first mapping table only stores the unmatched character codeword with the smallest numerical value corresponding to each first code length, and the length value codeword with the smallest numerical value corresponding to each second code length.

[0141] In one example, the first mapping table includes two columns, the first column is used to store an unmatched character indication codeword and a length value indication codeword; the second column is used to store multiple first base addresses and multiple second base addresses. The first mapping table includes multiple rows, each row is used to store an unmatched character indication codeword and a first base address; or is used to store a length value indication codeword and a second base address. For example: the maximum depth of the first column is 22 bits. The maximum depth of the second column is 9 bits. When the unmatched character indication codeword is "0x3", the first base address is "10010". When the unmatched character indication codeword is "0x0", the first base address is "0100".

[0142] In one example, each unmatched character indication codeword is provided with an invalid code length identifier. When the hardware recognizes the invalid code length identifier, it will no longer continue to recognize the invalid bits behind the invalid code length identifier, thereby facilitating hardware implementation. Optionally, when the code length of the unmatched character indication codeword is less than the maximum depth, a bit is added to the end of the unmatched character indication codeword as an invalid code length identifier. When the code length of the unmatched character indication codeword is equal to the maximum depth, there is no need to add an invalid code length identifier. For example: based on next_code[Q+1]>>1, an invalid code length identifier is added to each unmatched character indication codeword. Q is the code length of the unmatched character indication codeword. Q is an integer greater than or equal to 0. Assuming that the code length of an unmatched character indication codeword is 15 bits, the 16th bit is added and the 16th bit is used as an invalid code length identifier, that is, the valid code length of the unmatched character indication codeword is only 15 bits. Similarly, each offset value indication codeword and each length value codeword are also provided with an invalid code length identifier. It should be noted that there are free positions in the first mapping table because some values ​​may not have corresponding first code lengths and second code lengths, and thus no corresponding unmatched character indication codewords and length value indication codewords. In addition, there may be redundant information in the coded stream, which may also result in free positions in the first mapping table.

[0143] Fig.10 It is a structural schematic diagram of a second mapping table provided according to an embodiment of the present application.

[0144] Combine the following Fig.10 The second mapping table is exemplarily described.

[0145] In some embodiments, the second mapping table includes multiple offset value indicating codewords and multiple third base addresses; the offset value indicating codeword corresponds to the third base address one by one; the offset value indicating codeword is smaller than the offset value codeword. It should be understood that the value indicated by the offset value indicating codeword is smaller than or equal to the value indicated by the offset value codeword. That is, the second mapping table only stores the offset value codeword with the smallest value corresponding to each third code length.

[0146] In one example, the second mapping table includes two columns, the first column is used to store the offset value indicating codeword, and the second column is used to store the third base address; the second mapping table includes multiple rows, each row includes the offset value indicating codeword and the third base address. That is, the second mapping table only stores the smallest codeword corresponding to each third code length, that is, the offset value indicating codeword. For example: the maximum depth of the offset value indicating codeword is 15 bits. The maximum depth of the third base address is 5 bits. When the offset value indicating codeword is "0x3", the third base address is "10010". When the offset value indicating codeword is "0x0", the third base address is "0100".

[0147] Through the above analysis, it can be seen that the embodiment of the present application only stores the unmatched character indication codewords and the length value indication codewords in the first mapping table; and stores the offset value indication codewords in the second mapping table, thereby improving the rate of querying the mapping table, thereby improving the rate of decompressing the data.

[0148] In step 802, the terminal searches a mapping table based on index information for a query and obtains an indication codeword and a base address.

[0149] In some embodiments, step 802 may be implemented as follows:

[0150] The first implementation method: the terminal reads the compressed data, obtains the first code length, queries the first mapping table based on the first code length, and obtains the unmatched character indication codeword; queries the first mapping table based on the unmatched character indication codeword to obtain the first base address.

[0151] The second implementation method: the terminal reads the compressed data, obtains the second code length, queries the first mapping table based on the second code length, and obtains the length value character indication codeword; queries the first mapping table based on the length value indication codeword to obtain the second base address.

[0152] The third implementation method: the terminal reads the compressed data, obtains the third code length, queries the second mapping table based on the third code length, and obtains the offset value indicating codeword; queries the second mapping table based on the offset value indicating codeword to obtain the third base address.

[0153] In step 803, the terminal searches a decoding table for a base address based on the base address to obtain at least one decoded character.

[0154] In some embodiments, the encoding stream also includes multiple offset address codewords. Each base address indicates an initial storage area; the base address indicates at least one offset address codeword in the compressed data; each offset address codeword indicates an offset address; each offset address indicates an offset storage area; the initial storage area and the offset storage area are part of the decoding table; the initial storage area is used to store a decoded character; the offset storage area is used to store a decoded character; the decoded character includes a matching character parameter and an unmatched character. Optionally, the terminal parses an offset address codeword in the encoding stream to obtain an offset address. Optionally, the decoding table includes a first decoding table and a second decoding table; the first decoding table includes multiple unmatched characters and length values; the second decoding table includes multiple offset values.

[0155] In some embodiments, the above step 803 may be implemented as follows:

[0156] Fig.11 It is a structural schematic diagram of a first decoding table provided according to an embodiment of the present application.

[0157] Combine the following Fig.11 The first implementation manner and the second implementation manner are exemplarily described.

[0158] The first implementation method: the terminal obtains at least one first offset address based on the first base address; based on the first base address and a first offset address, obtains an unmatched character; the first base address indicates a first initial storage area; the first initial storage area is part of the first decoding table; the first initial storage area is used to store an unmatched character; the first base address and a first offset address indicate a first offset storage area; the first offset storage area is part of the first decoding table; the first offset storage area is used to store an unmatched character. Optionally, each first code length corresponds to a first base address and (first statistical number-1) first offset addresses. For example: the first first base address "0100" corresponds to two first offset addresses, that is, the first first offset address and the second first offset address. That is, the first statistical number is 3. The first code length corresponds to three unmatched characters. "0x0" is the smallest codeword corresponding to the first code length, that is, the unmatched character indication codeword.

[0159] It should be understood that, for the first base address, although the first base address corresponds to at least one first offset address; in the process of parsing the coding stream, only one offset address codeword is parsed each time to obtain any one of a first offset address, a second offset address, and a third offset address. In addition, in the process of parsing the coding stream, the previous parsing of the offset address codeword may obtain a first offset address, and the next parsing of the offset address codeword may obtain a second offset address. That is, the multiple first offset addresses corresponding to the first base address may not be parsed continuously, but are parsed according to the actual arrangement order of the offset address codewords in the coding stream. Similarly, the first offset address corresponding to each first base address is parsed.

[0160] The second implementation method: obtaining at least one second offset address based on the second base address; obtaining a length value based on the second base address and a second offset address; the second base address indicates a second initial storage area; the second initial storage area is part of the first decoding table; the second initial storage area is used to store a length value; the second base address and a second offset address indicate a second offset storage area; the second offset storage area is part of the first decoding table; the second offset storage area is used to store a length value. Optionally, each second code length corresponds to a second base address and (second statistical number - 1) second offset addresses. Optionally, the second offset address corresponding to each second base address obtained by parsing is the same as the first offset address corresponding to each first base address obtained above, and the embodiments of the present application will not be described in detail.

[0161] Fig.12 It is a structural schematic diagram of a second decoding table provided according to an embodiment of the present application.

[0162] Combine the following Fig.12 The second implementation method is described by way of example.

[0163] A third implementation method: obtaining at least one third offset address based on a third base address; obtaining an offset value based on the third base address and a third offset address; the third base address indicates a third initial storage area; the third initial storage area is part of the second decoding table; the third initial storage area is used to store an interval code; the third base address and a third offset address indicate a third offset storage area; the third offset storage area is part of the second decoding table; the third offset storage area is used to store an interval code. The interval code indicates an offset value. Optionally, an interval code is obtained based on a third base address and a third offset address; an additional bit is determined according to the interval code, a section of an additional bit length of the encoding stream is intercepted from the encoding stream, and the encoding stream of the additional bit is parsed to obtain an offset value. Optionally, the third offset address corresponding to each third base address obtained by parsing is the same as the first offset address corresponding to each first base address obtained above, and the embodiments of the present application will not be described in detail. For example: the first third base address "0100" corresponds to two third offset addresses, namely the first third offset address and the second third offset address. That is, the third statistic is 3. The third code length corresponds to three unmatched characters. "0x0" is the smallest codeword corresponding to the third code length, that is, the offset value indicating codeword. For another example: if the interval code is 19, the extra bit is 8. An 8-bit code stream is intercepted from the code stream and parsed to obtain a number between 769 and 1024. It should be noted that, corresponding to the method of compressing data, after decompressing to obtain an offset value, the offset "3" needs to be added to obtain the actual offset value.

[0164] Fig.13 It is a hardware structure diagram of a method for executing decompressed data provided according to an embodiment of the present application.

[0165] Combine the following Fig.13 The hardware structure of the method for decompressing data is exemplified.

[0166] In some embodiments, the terminal decompresses data online based on hardware. The terminal configures a read module and a write module based on an interface. For example, the interface is an Advanced Peripheral Bus (APB) interface. Optionally, the central processing unit (CPU) configures registers and then waits for an interrupt signal. Optionally, based on the parser parsing the registers, multiple read instructions, control instructions and write instructions are obtained, and the read instructions are sent to the read module. The read module obtains a compressed data packet of 64 bits (b) size from the bus each time. The parser obtains the control instruction, and in response to the control instruction, the parser obtains the compressed data packet, and parses the compressed data packet to obtain a mapping table, a decoding table, compressed data and an initial check code. Among them, the parser obtains compressed data and an update check code in the process of parsing the compressed data packet, compares the initial check code and the update check code, and if the initial check code is consistent with the update check code, it indicates that the transmission and parsing process of the compressed data are correct. Or if the initial check code is inconsistent with the update check code, it indicates that an error has occurred in the transmission and / or parsing process of the compressed data. The Huffman module obtains information such as compressed data, mapping table and decoding table, decodes the compressed data, and obtains multiple groups (offset value, length value, unmatched characters). The LZ77 module decodes multiple groups (offset value, length value, unmatched characters) to obtain the target decompressed data. Based on the check code, it is determined whether the target decompressed data is correct. If the target decompressed data is correct, the write module obtains the write instruction, and in response to the write instruction, writes the target decompressed data into the memory. For example: the memory is a double data rate synchronous dynamic random access memory (Double Data Rate Synchronous Dynamic Random Access Memory, DDR). The bus is implemented based on the AXI (Advanced eXtensible Interface) bus. The check code is a cyclic redundancy check (Cyclic Redundancy Check, CRC) code.

[0167] Fig.14 It is a hardware logic schematic diagram of a Huffman module provided according to an embodiment of the present application.

[0168] Combine the following Fig.14 The hardware logic of the Huffman module is illustrated below.

[0169] In some embodiments, in the first clock cycle: obtain an unmatched character indication codeword of the compressed data, and parse it to obtain a first code length; query the first mapping table based on the first code length to obtain a first base address. Based on an offset address codeword, obtain a first offset address. In the second clock cycle: query the first decoding table based on a first base address and a first offset address to obtain a decoded character. When the value of a decoded character obtained is greater than or equal to 256, it indicates that the decoded character is a length value. Or when the value of the decoded character obtained is less than 256, it indicates that the decoded character is an unmatched character. That is, query the first mapping table and the first decoding table respectively in different clock cycles. Based on the first offset address, determine the position where decoding starts in the next clock cycle.

[0170] K+1th clock cycle: obtain an unmatched character indication codeword of the compressed data, parse it to obtain the first code length; query the first mapping table based on the first code length to obtain a first base address. Based on an offset address codeword, obtain a first offset address. K+2th clock cycle: query the first decoding table based on a first base address and a first offset address to obtain an unmatched character. At the same time, obtain an offset value indication codeword of the compressed data, parse it to obtain a third code length; query the second mapping table based on the third code length to obtain a third base address. Based on a third base address and a third offset address, obtain an interval code; determine the extra bits according to the interval code, intercept a section of the encoding stream with an extra bit length from the encoding stream, parse the encoding stream of the extra bits, and obtain the offset value. K is an integer greater than or equal to 2. Optionally, based on the first offset address and the third offset address, determine the position where decoding starts in the next clock cycle. It should be noted that since the starting position of the offset value indication codeword in the compressed data is uncertain, the first base address of an unmatched character is first determined, and the second mapping table is queried while querying the first decoding table.

[0171] The P-1th clock cycle: obtain an unmatched character indication codeword of the compressed data, and parse it to obtain the first code length; query the first mapping table based on the first code length to obtain a first base address. Based on an offset address codeword, obtain a first offset address. The Pth clock cycle: query the first decoding table based on a first base address and a first offset address to obtain a decoded character. Among them, when the value of a decoded character obtained is greater than or equal to 256, it means that the decoded character is a length value. Or when the value of the decoded character obtained is less than 256, it means that the decoded character is an unmatched character. That is, query the first mapping table and the first decoding table respectively in different clock cycles. Based on the first offset address, determine the position where decoding starts in the next clock cycle. P is an integer greater than or equal to 2.

[0172] The following is a detailed description of the process of LZ77 module decoding multiple groups (offset value, length value, unmatched characters) to obtain the target decompressed data.

[0173] In step 804, the terminal searches a preset dictionary for a matching character parameter based on the matching character parameter to obtain the original character. The matching character parameter includes a length value and an offset value.

[0174] In some embodiments, the above step 804 may be implemented as follows:

[0175] The memory is searched based on a preset dictionary and an offset value, and the cache is searched based on a preset sliding window and an offset value to obtain the original character; the memory is used to store decompressed data; and the cache is used to store decompressed data that has not yet been written back to the memory. Since the original character just decoded will first be cached in the first pipeline, it is necessary to wait for the original character of the previous clock cycle to be written into the memory, that is, after waiting for one clock cycle, before the original character corresponding to the matching combination can be found in the memory, which obviously wastes decoding time. In order to solve this problem, when decoding the matching combination, the embodiment of the present application uses a preset dictionary to search the memory, and uses a preset sliding window to search the cache of the first pipeline, thereby not only expanding the search range, but also saving search time.

[0176] In some embodiments, in order to decode the original character according to the matching character parameter during the decoding process, in an embodiment of the present application, for a set of corresponding length values ​​and offset values, the length value and the offset value are combined to obtain a matching combination. For example, the high 13 bits in the matching combination represent the length value, and the low 13 bits in the matching combination represent the offset value, that is, the matching combination is encapsulated with 26 bits.

[0177] In one example, in order to distinguish the type of the acquired data to be processed, an embodiment of the present application acquires a data to be processed; the data to be processed is a matching combination or an unmatched character; if the type flag of the data to be processed is a first threshold, the data to be processed is a matching combination; the type flag indicates a matching combination or an unmatched combination; if the type flag of the data to be processed is a second threshold, the data to be processed is an unmatched character. For example: the type flag is set at the highest bit of the data to be processed. When the type flag is "1", it indicates an unmatched character, and when the type flag is "0", it indicates a matching combination.

[0178] In some embodiments, for a matching combination, when the offset value is less than the length value, the original character is determined based on the preset dictionary and the offset value, and the original character is copied based on the length value to obtain matching data; the length of the matching data is equal to the length value. That is, in the embodiments of the present application, there is no need to access the same address multiple times, and the original characters can be directly copied multiple times and spliced ​​from left to right, thereby improving the decoding efficiency. For example: when the preset dictionary = abed, the current (d, l, unmatched characters) = (2, 9, e). That is, the offset value (2) is less than the length value (9). First, it can be determined that the original character is ab, and the length value is 9, then ab is copied until the length is 9, and the decompression result of (2, 9, e) is: abcdcdcdcdce. In the related art, each time the preset dictionary is accessed, only one "cd" can be obtained from the preset dictionary, that is, it is necessary to write the "cd" and then access the preset dictionary again to obtain a "cd". Therefore, it is necessary to repeatedly access the preset dictionary 5 times to complete the decoding of this matching combination. Compared with the related art, the embodiment of the present application only needs to access the preset dictionary once, and then copy "cd" according to the length value, which greatly reduces the number of repeated accesses to the preset dictionary, thereby improving the decoding efficiency of the matching combination.

[0179] In some embodiments, for a matching combination, when the offset value is greater than the length value, it is determined based on the preset dictionary, the offset value and the difference value; the difference value is the difference between the offset value and the length value. For example: when d is greater than or equal to l, indicating that there is an original character, the preset dictionary [dl-1:d] is output.

[0180] In some embodiments, for a matching combination, when the matching combination is being decoded, multiple unmatched characters are spliced; when the matching combination has completed decoding, the splicing of unmatched characters is stopped to obtain a spliced ​​unmatched character. In the embodiment of the present application, in the process of decoding the matching combination, the unmatched characters are spliced, which not only does not occupy the decoding time of the matching combination, but also can process multiple unmatched characters in one clock cycle. Compared with the related art, only one unmatched character can be output in one clock cycle, which greatly improves the output efficiency of unmatched characters.

[0181] In step 805, the terminal concatenates the multiple original characters with the multiple unmatched characters to obtain target decompressed data.

[0182] In some embodiments, the unmatched characters obtained after decoding and the original characters are concatenated in the order before compression to obtain target decompressed data.

[0183] Fig.15 It is a hardware logic schematic diagram of an LZ77 module provided according to an embodiment of the present application.

[0184] Combine the following Fig.15The hardware logic of the LZ77 module is illustrated as an example.

[0185] In some embodiments, the splicing module obtains a matching combination or an unmatched character. Optionally, the splicing module may also obtain multiple matching combinations in succession.

[0186] In one example, the splicing module obtains a matching combination, and after recognizing that the first identification mark is correct, the splicing module transmits it to the first pipeline, decodes the matching combination, and obtains the corresponding original character. The first pipeline is used to decode the matching combination. Optionally, in the process of decoding the matching combination by the first pipeline, the splicing module splices at least one unmatched character obtained continuously, and when receiving the signal of the decoding end of the matching combination fed back by the first pipeline, the splicing is stopped to obtain the spliced ​​unmatched character. The spliced ​​unmatched character is transmitted to the second pipeline. Optionally, the second pipeline outputs the spliced ​​unmatched character according to the arrangement order of the data to be compressed. After the second pipeline obtains the spliced ​​unmatched character, the invalid length is first identified, so that the second pipeline only outputs the unmatched character of the valid length. It can be understood that the unmatched character does not need to be decoded. Therefore, the second pipeline only needs to output the unmatched character according to the arrangement order of the data to be compressed. Then, the splicing module splices the unmatched character with the original character according to the arrangement order of the data to be compressed. For example: the bus width is 64 bits, and the invalid length can be understood as the length of the spliced ​​unmatched characters being less than 64 bits. When the terminal recognizes the invalid length based on hardware, the invalid length will not be output, which facilitates hardware implementation. Another example: when the splicing module completes the splicing of 8 unmatched characters, that is, the size of the spliced ​​unmatched characters is 64 bits, the splicing is stopped. Optionally, when the size of the target decompressed data meets 64 bits, the output module outputs the target decompressed data and writes the target decompressed data to DDR; or when the size of the target decompressed data does not meet 64 bits, the output module does not output the target decompressed data.

[0187] In one example, the original characters just decoded will first be cached in the first pipeline, and then written into the DDR in order. Therefore, when the first pipeline decodes the matching combination of the next clock cycle, if the DDR is searched based on the preset dictionary only, the original characters matching the matching combination cannot be found within one clock cycle. That is, it is necessary to wait for the original characters of the previous clock cycle to be written into the DDR before the original characters corresponding to the matching combination can be found, which obviously wastes decoding time. In order to solve this problem, the embodiment of the present application uses a preset dictionary of 4k size to search the DDR when decoding the matching combination, and uses a preset sliding window of 32B size to search the cache of the first pipeline, thereby not only expanding the search range, but also saving search time, and the original characters can be found quickly.

[0188] In the embodiment of the present application, when decompressing data, the mapping table, the decoding table and the index information are directly obtained. The mapping table is searched through the index information to obtain the base address. The decoding table is searched through the base address to obtain the unmatched character, length value or offset value. In the technical solution of the present application, the decoding process does not require the reconstruction of the table and the run-length encoding. The decoding table can be quickly searched through the base address, which simplifies the decompression logic, thereby effectively improving the throughput rate and being easy to implement in hardware.

[0189] Fig.16 1 is a flow chart of a method for compressing data provided in accordance with an embodiment of the present application. In the embodiment of the present application, the method is described by taking application on a terminal with a chip as an example. The method comprises the following steps:

[0190] In step 1601, the terminal obtains data to be compressed.

[0191] In some embodiments, the data to be compressed is image data. The terminal compresses the data in an offline manner based on software.

[0192] In step 1602, the terminal preprocesses the data to be compressed based on a preset dictionary to obtain a preprocessing result.

[0193] In some embodiments, the data to be compressed is preprocessed based on the LZ77 algorithm. The size of the preset dictionary is 4k. That is, the maximum addressing range is 4096. In the related art, the size of the preset dictionary is 64k. Therefore, in the embodiment of the present application, the size of the preset dictionary is greatly reduced, which not only effectively controls the search depth, but also can quickly find the original character, thereby saving the search time.

[0194] In some embodiments, the preprocessing result includes multiple combinations; the combination includes unmatched characters and matching character parameters; the matching character parameters include length values ​​and offset values. Each combination can be represented by (d, l, unmatched characters).

[0195] Fig.17 It is a compression ratio comparison diagram provided according to an embodiment of the present application.

[0196] Combine the following Fig.17 The compression rate of the present application is exemplified.

[0197] In step 1603, the terminal segments the multiple preprocessing combinations to obtain multiple data segments.

[0198] The embodiment of the present application, after dynamic segmentation compression, obtains a compression rate that is improved by within 6% compared with the compression rate in the related art, which is within the allowable range. Therefore, the embodiment of the present application simplifies the logic of decompressing data while ensuring the compression rate, and is easy to implement in hardware.

[0199] Fig.18 It is a flowchart of step 1603 provided according to an embodiment of the present application.

[0200] Combine the following Fig.18 Step 1603 is exemplarily described.

[0201] In some embodiments, in order to improve the compression rate, the above step 1603 includes the following steps 16031 to 16034:

[0202] In step 16031, the terminal obtains t+1 consecutive preset segments from the uncoded part in the preprocessing combination; t is an integer greater than or equal to 0.

[0203] In step 16032, the terminal obtains a preset coding result; the preset coding result is obtained by encoding t+1 preset segments based on the first code table and the second code table.

[0204] In step 16033, the terminal obtains a first compression rate; the first compression rate is obtained based on a preset encoding result and t+1 preset segments.

[0205] In step 16034, if the first compression ratio is greater than the second compression ratio, t preset segments are taken as one data segment; the second compression ratio is the compression ratio of t preset segments. Wherein, compression ratio = size of compressed data / size of data to be compressed.

[0206] The above step 16034 is only one implementation method. In addition, if the first compression rate is less than the second compression rate, t+2 preset segments are encoded based on the first code table and the second code table until the nth compression rate is greater than the n-1th compression rate, and the t+n-1 preset segments are used as a data segment; the nth compression rate is the compression rate of the t+n preset segments; the n-1th compression rate is the compression rate of the t+n-1 preset segments; n is an integer greater than or equal to 1.

[0207] For example, each preset segment is a 4k coded stream, which is 4,000 (d, l, unmatched characters). Each data segment can be a 2M coded stream at most, which is 2 million (d, l, unmatched characters).

[0208] The embodiment of the present application adopts a combination of static code table and dynamic segment compression to compress data, eliminating the more complicated three-level Huffman code table in the related art, and adopts a one-level code table to facilitate rapid search of d, l and unmatched characters. In addition, the dynamic segment compression method is adopted to alleviate the compression rate problem caused by the reduction of the number of table levels.

[0209] Fig.19 It is a schematic diagram of the structure of a first code table provided according to an embodiment of the present application.

[0210] Combine the following Fig.19 The first code table is exemplified.

[0211] In step 1604, the terminal encodes the unmatched character for a data segment based on the first code table to obtain an unmatched character codeword; and encodes the length value based on the first code table to obtain a length value codeword.

[0212] In some embodiments, the first code table includes a first numerical interval, a first coding bit, a second numerical interval and a second coding bit; the first numerical interval indicates an unmatched character; the first coding bit indicates a code length range of the unmatched character; the second numerical interval indicates a length value; and the second coding bit indicates a code length range of the length value.

[0213] In some embodiments, the first numerical range is 0 to 255; the code length range indicated by the first encoding bit is [7:0], that is, the maximum is 8 bits and the minimum is 1 bit. Each unmatched character is encoded with a maximum of 8 bits of binary to obtain multiple unmatched character codewords. Optionally, when the value of the data to be compressed falls between 0 and 255, it indicates that the data to be compressed is an unmatched character, and then it is encoded with a maximum of 8 bits. In addition, the second numerical range is 256 to 512, and the code length range indicated by the second encoding bit is [7:0], that is, the maximum is 8 bits and the minimum is 1 bit. There are 24 interval codes, each interval code represents an interval. That is, the second numerical range is divided into 24 intervals. Each length value is also encoded with a maximum of 8 bits of binary to obtain multiple length value codewords. Optionally, an additional bit is also provided, and when the additional bit is "0", it indicates an unmatched character. When the additional bit is "1", it indicates a length value. That is, the additional bit is used to distinguish between unmatched characters and length values. Optionally, when the value of the data to be compressed falls between 256 and 512, it indicates that the data to be compressed is a length value, which is encoded with a maximum of 8 bits. In addition, since the actual range of the length value is 3 to 258, when the value represented by the length value codeword is "0", it has no practical meaning. In order to solve this problem, the length value codeword adds an offset "3" so that the value represented by the length value codeword starts from "3", so that each length value codeword has practical meaning. Among them, the offset "3" is a decimal number. Compared with the 29 intervals of the related art, the embodiment of the present application only involves two static code tables, the first code table and the second code table, and the number of tables is reduced. Therefore, the table lookup logic is simple, the calculation efficiency is high, and a codeword can be decoded within one clock cycle, which is easy to implement in hardware. And the required storage space is reduced, which reduces the hardware area. In addition, the number of intervals in the second code table is reduced compared with the number of intervals in the related art, which optimizes the table lookup method and is easier to implement in hardware.

[0214] Fig. 20 It is a schematic diagram of the structure of a second code table provided according to an embodiment of the present application.

[0215] Combine the following Fig. 20 The second code table is exemplified.

[0216] In step 1605, the terminal encodes the offset value for a data segment based on the second code table to obtain an offset value codeword.

[0217] In some embodiments, the second code table includes a third numerical interval, an interval code, and an additional bit; the additional bit indicates the code length range of the third numerical interval; the interval code is encoded based on the initial bit. For example, the initial bit is at most 5 bits and at least 1 bit. The interval code is 0 to 23. That is, the third numerical interval is divided into 24 intervals.

[0218] In step 1606, the terminal regards the unmatched character codeword, the length value codeword, and the offset value codeword for a data segment as a compressed data block.

[0219] In some embodiments, a first Huffman tree and a second Huffman tree are constructed; the first Huffman tree is constructed based on multiple unmatched characters and length values; the second Huffman tree is constructed based on multiple offset values; based on the first Huffman tree, multiple first code lengths and multiple second code lengths are obtained; the first code length indicates multiple unmatched character code words; the second code length indicates multiple length value code words; based on the second Huffman tree, multiple third code lengths are obtained; the third code length indicates multiple offset value code words. It should be noted that the method for constructing Huffman can be directly obtained from the relevant technology, and the embodiments of this application will not be repeated.

[0220] In one example, for a first code length, based on the first Huffman tree, the statistical number of unmatched character codewords corresponding to the first code length can be determined. Therefore, after determining the minimum codeword corresponding to the first code length, that is, the unmatched character indication codeword, the offset address codewords corresponding to the remaining unmatched character codewords are determined, that is, the offset address codewords corresponding to each unmatched character codeword in the (statistical number - 1) unmatched character codewords. Similarly, the offset value indication codeword and the length value indication codeword can be obtained.

[0221] In some embodiments, a mapping table and a decoding table are generated based on an offset value codeword, a length value codeword, and an unmatched character codeword; the mapping table includes multiple indication codewords and multiple base addresses; the indication codewords correspond one-to-one to the base addresses; the decoding table includes multiple decoded characters; each base address indicates an initial storage area; each base address indicates at least one offset address; the offset address indicates an offset storage area; the initial storage area and the offset storage area are part of the decoding table; the initial storage area is used to store a decoded character; the offset storage area is used to store a decoded character; the indication codeword is determined based on any one of the offset value codeword, the length value codeword, and the unmatched character codeword.

[0222] In the embodiment of the present application, a mapping table and a decoding table are constructed based on software, and the mapping table and the decoding table are stored in the header of the compressed data packet. Therefore, when hardware decoding is performed, the mapping table and the decoding table can be directly called for decoding without rebuilding the table, which simplifies the logic of decompressing data and facilitates hardware implementation.

[0223] Fig.21 21 is a schematic diagram of a structure of a device 2100 for decompressing data provided according to an embodiment of the present application, the device comprising:

[0224] The parsing module 2101 is used to obtain a compressed data packet and parse the compressed data packet to obtain a packet header and compressed data; the packet header includes a mapping table and a decoding table; the mapping table includes a plurality of indicator code words and a plurality of base addresses; the indicator code words correspond to the base addresses one by one; the decoding table includes a plurality of decoding characters; the compressed data includes index information;

[0225] The first decoding module 2102 is used to query the mapping table based on the index information for a query, and obtain the indication codeword and the base address;

[0226] The second decoding module 2103 is used for querying a decoding table based on a base address to obtain at least one decoded character; the decoded character includes a matching character parameter and an unmatched character;

[0227] The third decoding module 2104 is used for querying a preset dictionary based on a matching character parameter to obtain an original character;

[0228] The splicing module 2105 is used to splice multiple original characters with multiple unmatched characters to obtain target decompressed data.

[0229] In one possible implementation, each base address indicates an initial storage area; the base address indicates at least one offset address codeword in the compressed data; each offset address codeword indicates an offset address; each offset address indicates an offset storage area; the initial storage area and the offset storage area are part of a decoding table; the initial storage area is used to store a decoded character; and the offset storage area is used to store a decoded character.

[0230] In one possible implementation, the matching character parameters include a length value and an offset value; the index information includes multiple first code lengths, multiple second code lengths, and multiple third code lengths; the first code length indicates multiple unmatched character code words; the second code length indicates multiple length value code words; and the third code length indicates multiple offset value code words.

[0231] In some embodiments, the mapping table includes a first mapping table and a second mapping table; the first mapping table includes multiple unmatched character indication codewords, multiple first base addresses, multiple length value indication codewords and multiple second base addresses; the unmatched character indication codewords correspond one-to-one to the first base addresses; the unmatched character indication codewords are less than or equal to the unmatched character codewords; the length value indication codewords correspond one-to-one to the second base addresses; the length value indication codewords are less than or equal to the length value codewords; the second mapping table includes multiple offset value indication codewords and multiple third base addresses; the offset value indication codewords correspond one-to-one to the third base addresses; the offset value indication codewords are less than or equal to the offset value codewords.

[0232] In some embodiments, the device is further configured to:

[0233] querying the first mapping table based on the first code length to obtain an unmatched character indication codeword; querying the first mapping table based on the unmatched character indication codeword to obtain a first base address; or

[0234] querying the first mapping table based on the second code length to obtain a length value character indicating a codeword; querying the first mapping table based on the length value indicating the codeword to obtain a second base address; or

[0235] The second mapping table is queried based on the third code length to obtain an offset value indicating codeword; and the second mapping table is queried based on the offset value indicating codeword to obtain a third base address.

[0236] In some embodiments, the decoding table includes a first decoding table and a second decoding table; the first decoding table includes a plurality of unmatched characters and a length value; the second decoding table includes a plurality of offset values;

[0237] For a base address, query the decoding table based on the base address to obtain at least one decoded character, including:

[0238] Based on the first base address, at least one first offset address is obtained; based on the first base address and a first offset address, an unmatched character is obtained; the first base address indicates a first initial storage area; the first initial storage area is part of the first decoding table; the first initial storage area is used to store an unmatched character; the first base address and a first offset address indicate a first offset storage area; the first offset storage area is part of the first decoding table; the first offset storage area is used to store an unmatched character; or

[0239] Based on the second base address, at least one second offset address is obtained; based on the second base address and a second offset address, a length value is obtained; the second base address indicates a second initial storage area; the second initial storage area is part of the first decoding table; the second initial storage area is used to store a length value; the second base address and a second offset address indicate a second offset storage area; the second offset storage area is part of the first decoding table; the second offset storage area is used to store a length value; or

[0240] At least one third offset address is obtained based on a third base address; an offset value is obtained based on the third base address and a third offset address; the third base address indicates a third initial storage area; the third initial storage area is part of the second decoding table; the third initial storage area is used to store an interval code; the third base address and a third offset address indicate a third offset storage area; the third offset storage area is part of the second decoding table; the third offset storage area is used to store an interval code; the interval code indicates an offset value.

[0241] In some embodiments, the device is further configured to:

[0242] For a set of corresponding length values ​​and offset values, the length values ​​and the offset values ​​are combined to obtain a matching combination.

[0243] In some embodiments, the device is further configured to:

[0244] Get a data to be processed; the data to be processed is a matching combination or an unmatched character;

[0245] If the type flag of the data to be processed is the first threshold, the data to be processed is a matching combination; the type flag indicates a matching combination or an unmatched combination;

[0246] If the type flag of the data to be processed is the second threshold, the data to be processed is an unmatched character.

[0247] In some embodiments, the device is further configured to:

[0248] For a matching combination, when the offset value is less than the length value, the original character is determined based on the preset dictionary and the offset value, and the original character is copied based on the length value to obtain matching data; the length of the matching data is equal to the length value; or

[0249] For a matching combination, when the offset value is greater than the length value, the matching data is determined based on the preset dictionary, the offset value and the difference value; the difference value is the difference between the offset value and the length value.

[0250] In some embodiments, the device is further configured to:

[0251] The memory is searched based on a preset dictionary and an offset value, and the cache is searched based on a preset sliding window and an offset value to obtain the original character; the memory is used to store the target decompressed data; the cache is used to store the target decompressed data that has not been written back to the memory.

[0252] In some embodiments, the device is further configured to:

[0253] For a matching combination, when the matching combination is being decoded, multiple unmatched characters are concatenated;

[0254] When the matching combination has been decoded, the concatenation of unmatched characters is stopped to obtain concatenated unmatched characters.

[0255] In some embodiments, the device is further configured to:

[0256] The compressed data includes a plurality of compressed data blocks, and the packet header includes the size of each compressed data block and the number of compressed data blocks.

[0257] It should be noted that: the device for decompressing data provided in the above embodiment only uses the division of the above functional modules as an example when executing the corresponding steps. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device for decompressing data provided in the above embodiment and the method embodiment for decompressing data belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0258] In the embodiment of the present application, when decompressing data, the mapping table, the decoding table and the index information are directly obtained. The mapping table is searched through the index information to obtain the base address. The decoding table is searched through the base address to obtain the unmatched character, length value or offset value. In the technical solution of the present application, the decoding process does not require the reconstruction of the table and the run-length encoding. The decoding table can be quickly searched through the base address, which simplifies the decompression logic, thereby effectively improving the throughput rate and being easy to implement in hardware.

[0259] Fig. 22 2200 is a schematic diagram of a data compression device according to an embodiment of the present application, the device comprising:

[0260] An acquisition module 2201 is used to acquire data to be compressed;

[0261] The preprocessing module 2202 is used to preprocess the data to be compressed based on a preset dictionary to obtain a preprocessing result; the preprocessing result includes multiple combinations; the combination includes unmatched characters and matching character parameters; the matching character parameters include length values ​​and offset values;

[0262] Segmentation module 2203, used for segmenting multiple preprocessing combinations to obtain multiple data segments;

[0263] The first encoding module 2204 is used for encoding the unmatched character based on the first code table for a data segment to obtain the unmatched character codeword; and encoding the length value based on the first code table to obtain the length value codeword; the first code table includes a first numerical interval, a first encoding bit, a second numerical interval and a second encoding bit; the first numerical interval indicates the unmatched character; the first encoding bit indicates the code length range of the unmatched character; the second numerical interval indicates the length value; the second encoding bit indicates the code length range of the length value;

[0264] A second encoding module 2205 is used to encode the offset value for a data segment based on a second code table to obtain an offset value codeword; the second code table includes a third numerical interval, an interval code and an extra bit; the extra bit indicates a code length range of the third numerical interval; the interval code is encoded based on the initial bit;

[0265] The result module 2206 is used to treat the unmatched character codewords, length value codewords and offset value codewords as a compressed data block for a data segment.

[0266] In some embodiments, the device is further configured to:

[0267] Obtaining t+1 consecutive preset segments from the uncoded portion of the preprocessing combination; t is an integer greater than or equal to 0;

[0268] Obtain a preset encoding result; the preset encoding result is obtained by encoding t+1 preset segments based on the first code table and the second code table;

[0269] Obtaining a first compression rate; the first compression rate is obtained based on a preset encoding result and t+1 preset segments;

[0270] If the first compression rate is greater than the second compression rate, t preset fragments are regarded as a data fragment; the second compression rate is the compression rate of t preset fragments; or if the first compression rate is less than the second compression rate, t+2 preset fragments are encoded based on the first code table and the second code table until the nth compression rate is greater than the n-1th compression rate, and t+n-1 preset fragments are regarded as a data fragment; the nth compression rate is the compression rate of t+n preset fragments; the n-1th compression rate is the compression rate of t+n-1 preset fragments; n is an integer greater than or equal to 1.

[0271] In some embodiments, the device is further configured to:

[0272] Constructing a first Huffman tree and a second Huffman tree; the first Huffman tree is constructed based on a plurality of unmatched characters and a length value; and the second Huffman tree is constructed based on a plurality of offset values;

[0273] Based on the first Huffman tree, a plurality of first code lengths and a plurality of second code lengths are obtained; the first code length indicates a plurality of unmatched character code words; the second code length indicates a plurality of length value code words;

[0274] Based on the second Huffman tree, a plurality of third code lengths are obtained; the third code lengths indicate a plurality of offset value code words.

[0275] In some embodiments, the device is further configured to:

[0276] A mapping table and a decoding table are generated based on an offset value codeword, a length value codeword and an unmatched character codeword; the mapping table includes a plurality of indication codewords and a plurality of base addresses; the indication codewords correspond one to one with the base addresses; the decoding table includes a plurality of decoded characters; each base address indicates an initial storage area; each base address indicates at least one offset address; the offset address indicates an offset storage area; the initial storage area and the offset storage area are part of the decoding table; the initial storage area is used to store a decoded character; the offset storage area is used to store a decoded character; the indication codeword is determined based on any one of the offset value codeword, the length value codeword and the unmatched character codeword.

[0277] It should be noted that: when the device for compressing data provided in the above embodiment performs the corresponding steps, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device for compressing data provided in the above embodiment and the method embodiment for compressing data belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0278] In the embodiment of the present application, a mapping table and a decoding table are constructed based on software, and the mapping table and the decoding table are stored in the header of the compressed data packet. Therefore, when hardware decoding is performed, the mapping table and the decoding table can be directly called for decoding without rebuilding the table, which simplifies the logic of decompressing data and facilitates hardware implementation.

[0279] An embodiment of the present application further provides a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the above method when executing the computer program.

[0280] Taking computer equipment as the terminal as an example, Fig.23 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application, see Fig.23 The terminal 230 may be a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, or a desktop computer. The terminal 230 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal, or other names.

[0281] Typically, the terminal 230 includes a processor 231 and a memory 232 .

[0282] The processor 231 may include one or more processing cores, such as a 4-core processor, a 5-core processor, etc. The processor 231 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 231 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 231 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 231 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0283] The memory 232 may include one or more computer-readable storage media, which may be non-transitory. The memory 232 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 232 is used to store at least one program code, which is used to be executed by the processor 231 to implement the process provided by the method embodiment of the present application for the terminal execution in the above method.

[0284] In some embodiments, the terminal 230 may also optionally include: a peripheral device interface 233 and at least one peripheral device. The processor 231, the memory 232 and the peripheral device interface 233 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 233 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a display screen 234, a camera assembly 235, an audio circuit 236 and a power supply 237.

[0285] The peripheral device interface 233 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 231 and the memory 232. In some embodiments, the processor 231, the memory 232 and the peripheral device interface 233 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 231, the memory 232 and the peripheral device interface 233 can be implemented on a separate chip or circuit board, which is not limited in the embodiments of the present application.

[0286] The display screen 234 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 234 is a touch display screen, the display screen 234 also has the ability to collect touch signals on the surface or above the surface of the display screen 234. The touch signal can be input to the processor 231 as a control signal for processing. At this time, the display screen 234 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 234 can be one, set on the front panel of the terminal 230; in other embodiments, the display screen 234 can be at least two, respectively set on different surfaces of the terminal 230 or in a folding design; in other embodiments, the display screen 234 can be a flexible display screen, set on the curved surface or folding surface of the terminal 230. Even, the display screen 234 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 234 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0287] The camera assembly 235 is used to capture images or videos. In some embodiments, the camera assembly 235 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 235 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0288] The audio circuit 236 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 231 for processing. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 230. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 231 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 236 may also include a headphone jack.

[0289] The power supply 237 is used to power various components in the terminal 230. The power supply 237 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 237 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0290] Those skilled in the art will understand that Fig.23 The structure shown in the figure does not constitute a limitation on the terminal 230, and the terminal 230 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0291] Take the computer device as a server as an example. Fig.24 2400 is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server 2400 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 2401 and one or more memories 2402, wherein the one or more memories 2402 store at least one computer program, and the at least one computer program is loaded and executed by the one or more processors 2401 to implement the above-mentioned background removal method. Of course, the server 2400 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server 2400 may also include other components for implementing device functions, which will not be described in detail here.

[0292] The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned method for generating an image processing model. Optionally, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0293] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0294] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for decompressing data, characterized in that: include: Obtain a compressed data packet, and parse the compressed data packet to obtain a packet header and compressed data; The packet header includes a mapping table and a decoding table; the mapping table includes a plurality of indication code words and a plurality of base addresses; the indication code words correspond to the base addresses one by one; the decoding table includes a plurality of decoding characters; the compressed data includes index information; For a query, query the mapping table based on the index information to obtain the indication codeword and the base address; For one of the base addresses, query the decoding table based on the base address to obtain at least one of the decoded characters; the decoded characters include matching character parameters and unmatched characters; For one of the matching character parameters, query a preset dictionary based on the matching character parameter to obtain the original character; The plurality of original characters are concatenated with the plurality of unmatched characters to obtain target decompressed data.

2. The method according to claim 1, characterized in that Each of the base addresses indicates an initial storage area; the base address indicates at least one offset address codeword in the compressed data; each of the offset address codeword indicates an offset address; each of the offset addresses indicates an offset storage area; the initial storage area and the offset storage area are part of the decoding table; the initial storage area is used to store one of the decoded characters; the offset storage area is used to store one of the decoded characters.

3. The method according to claim 1, characterized in that The matching character parameters include a length value and an offset value; the index information includes multiple first code lengths, multiple second code lengths and multiple third code lengths; the first code length indicates multiple unmatched character code words; the second code length indicates multiple length value code words; the third code length indicates multiple offset value code words.

4. The method according to claim 3, characterized in that The mapping table includes a first mapping table and a second mapping table; the first mapping table includes multiple unmatched character indication codewords, multiple first base addresses, multiple length value indication codewords and multiple second base addresses; the unmatched character indication codeword corresponds one-to-one to the first base address; the unmatched character indication codeword is less than or equal to the unmatched character codeword; the length value indication codeword corresponds one-to-one to the second base address; the length value indication codeword is less than or equal to the length value codeword; the second mapping table includes multiple offset value indication codewords and multiple third base addresses; the offset value indication codeword corresponds one-to-one to the third base address; the offset value indication codeword is less than or equal to the offset value codeword.

5. The method according to claim 4, characterized in that The querying the mapping table based on the index information for one query to obtain the indication codeword and the base address includes: querying the first mapping table based on the first code length to obtain the unmatched character indication codeword; querying the first mapping table based on the unmatched character indication codeword to obtain the first base address; or querying the first mapping table based on the second code length to obtain the length value character indication codeword; querying the first mapping table based on the length value indication codeword to obtain the second base address; or The second mapping table is queried based on the third code length to obtain the offset value indicating codeword; and the second mapping table is queried based on the offset value indicating codeword to obtain the third base address.

6. The method according to claim 4, characterized in that The decoding table includes a first decoding table and a second decoding table; the first decoding table includes a plurality of the unmatched characters and the length value; The second decoding table includes a plurality of the offset values; For one of the base addresses, querying the decoding table based on the base address to obtain at least one of the decoded characters includes: Based on the first base address, at least one first offset address is obtained; based on the first base address and one of the first offset addresses, one of the unmatched characters is obtained; the first base address indicates a first initial storage area; the first initial storage area is a part of the first decoding table; the first initial storage area is used to store one of the unmatched characters; the first base address and one of the first offset addresses indicate a first offset storage area; the first offset storage area is a part of the first decoding table; the first offset storage area is used to store one of the unmatched characters; or Based on the second base address, at least one second offset address is obtained; based on the second base address and one second offset address, one length value is obtained; the second base address indicates a second initial storage area; the second initial storage area is a part of the first decoding table; the second initial storage area is used to store one length value; the second base address and one second offset address indicate a second offset storage area; the second offset storage area is a part of the first decoding table; the second offset storage area is used to store one length value; or At least one third offset address is obtained based on the third base address; based on the third base address and one third offset address, one offset value is obtained; the third base address indicates a third initial storage area; the third initial storage area is a part of the second decoding table; the third initial storage area is used to store an interval code; the third base address and one third offset address indicate a third offset storage area; the third offset storage area is a part of the second decoding table; the third offset storage area is used to store one interval code; the interval code indicates one offset value.

7. The method according to claim 3, characterized in that The method further comprises: For a set of corresponding length values ​​and offset values, the length values ​​and the offset values ​​are combined to obtain a matching combination.

8. The method according to claim 7, characterized in that The method further comprises: Obtaining a data to be processed; the data to be processed is the matching combination or the unmatched character; If the type flag of the data to be processed is the first threshold, the data to be processed is the matching combination; the type flag indicates the matching combination or the unmatched combination; If the type flag of the data to be processed is the second threshold, the data to be processed is the unmatched character.

9. The method according to claim 7, characterized in that: The step of querying a preset dictionary based on the matching character parameter to obtain the original character for one of the matching character parameters includes: For one of the matching combinations, when the offset value is less than the length value, the original character is determined based on the preset dictionary and the offset value, and the original character is copied based on the length value to obtain matching data; the length of the matching data is equal to the length value; or For one of the matching combinations, when the offset value is greater than the length value, the matching data is determined based on the preset dictionary, the offset value and the difference value; the difference value is the difference between the offset value and the length value.

10. The method according to claim 9, characterized in that The step of querying a preset dictionary based on the matching character parameter to obtain the original character for one of the matching character parameters further includes: The original character is obtained by searching a memory based on the preset dictionary and the offset value, and searching a cache based on a preset sliding window and the offset value; the memory is used to store the target decompressed data; and the cache is used to store the target decompressed data that has not been written back to the memory.

11. The method according to claim 7, characterized in that The method further comprises: For one of the matching combinations, when the matching combination is being decoded, a plurality of the unmatched characters are concatenated; When the matching combination has been decoded, the splicing of the unmatched characters is stopped to obtain spliced ​​unmatched characters.

12. The method according to claim 1, characterized in that The compressed data includes a plurality of compressed data blocks, and the packet header includes the size of each of the compressed data blocks and the number of the compressed data blocks.

13. A method for compressing data, characterized in that: include: Get the data to be compressed; preprocessing the data to be compressed based on a preset dictionary to obtain a preprocessing result; The preprocessing result includes a plurality of combinations; The combination includes unmatched characters and matching character parameters; the matching character parameters include a length value and an offset value; Segmenting a plurality of the preprocessing combinations to obtain a plurality of data segments; For one of the data segments, the unmatched character is encoded based on a first code table to obtain an unmatched character codeword; and the length value is encoded based on the first code table to obtain the length value codeword; the first code table includes a first numerical interval, a first coding bit, a second numerical interval and a second coding bit; the first numerical interval indicates the unmatched character; the first coding bit indicates the code length range of the unmatched character; the second numerical interval indicates the length value; the second coding bit indicates the code length range of the length value; For one of the data segments, the offset value is encoded based on a second code table to obtain an offset value codeword; the second code table includes a third numerical interval, an interval code and an additional bit; the additional bit indicates the code length range of the third numerical interval; the interval code is encoded based on the initial bit; For one of the data segments, the unmatched character codewords, the length value codewords and the offset value codewords are taken as a compressed data block.

14. The method according to claim 13, characterized in that The step of segmenting the plurality of preprocessing combinations to obtain a plurality of data segments includes: Obtaining t+1 consecutive preset segments from the uncoded portion of the preprocessing combination, where t is an integer greater than or equal to 0; Obtaining a preset encoding result; the preset encoding result is obtained by encoding t+1 of the preset segments based on the first code table and the second code table; Obtaining a first compression rate; the first compression rate is obtained based on the preset encoding result and t+1 of the preset segments; If the first compression rate is greater than the second compression rate, t preset fragments are taken as one data fragment; the second compression rate is the compression rate of t preset fragments; or if the first compression rate is less than the second compression rate, t+2 preset fragments are encoded based on the first code table and the second code table until the nth compression rate is greater than the n-1th compression rate, and t+n-1 preset fragments are taken as one data fragment; the nth compression rate is the compression rate of t+n preset fragments; the n-1th compression rate is the compression rate of t+n-1 preset fragments; n is an integer greater than or equal to 1.

15. The method according to claim 13, characterized in that The method further comprises: Constructing a first Huffman tree and a second Huffman tree; the first Huffman tree is constructed based on a plurality of the unmatched characters and the length values; the second Huffman tree is constructed based on a plurality of the offset values; Based on the first Huffman tree, a plurality of first code lengths and a plurality of second code lengths are obtained; the first code length indicates a plurality of the unmatched character code words; the second code length indicates a plurality of the length value code words; Based on the second Huffman tree, a plurality of third code lengths are obtained; the third code lengths indicate a plurality of the offset value code words.

16. The method according to claim 13, characterized in that The method also includes: generating a mapping table and a decoding table based on the offset value codeword, the length value codeword and the unmatched character codeword; the mapping table includes multiple indication codewords and multiple base addresses; the indication codeword corresponds to the base address one by one; the decoding table includes multiple decoded characters; each base address indicates an initial storage area; each base address indicates at least one offset address; the offset address indicates an offset storage area; the initial storage area and the offset storage area are part of the decoding table; the initial storage area is used to store one decoded character; the offset storage area is used to store one decoded character; the indication codeword is determined based on any one of the offset value codeword, the length value codeword and the unmatched character codeword.

17. A device for decompressing data, characterized in that: include: A parsing module, used to obtain a compressed data packet and parse the compressed data packet to obtain a packet header and compressed data; The packet header includes a mapping table and a decoding table; the mapping table includes a plurality of indication code words and a plurality of base addresses; the indication code words correspond to the base addresses one by one; the decoding table includes a plurality of decoding characters; the compressed data includes index information; A first decoding module, configured to query the mapping table based on the index information for a query, and obtain the indication codeword and the base address; A second decoding module is used for querying the decoding table based on the base address for one of the base addresses to obtain at least one of the decoded characters; the decoded characters include matching character parameters and unmatched characters; A third decoding module is used for querying a preset dictionary based on one of the matching character parameters to obtain the original character; The splicing module is used to splice the multiple original characters with the multiple unmatched characters to obtain target decompressed data.

18. A device for compressing data, characterized in that: include: An acquisition module, used for acquiring data to be compressed; A preprocessing module, used for preprocessing the data to be compressed based on a preset dictionary to obtain a preprocessing result; The preprocessing result includes multiple combinations; the combination includes unmatched characters and matching character parameters; the matching character parameters include length values ​​and offset values; A segmentation module, used for segmenting the plurality of preprocessing combinations to obtain a plurality of data segments; A first encoding module is configured to encode the unmatched character based on a first code table for one of the data segments to obtain an unmatched character codeword; and encode the length value based on the first code table to obtain the length value codeword; the first code table includes a first numerical interval, a first coding bit, a second numerical interval, and a second coding bit; the first numerical interval indicates the unmatched character; the first coding bit indicates a code length range of the unmatched character; the second numerical interval indicates the length value; the second coding bit indicates the code length range of the length value; A second encoding module is used for encoding the offset value based on a second code table for one of the data segments to obtain an offset value codeword; the second code table includes a third numerical interval, an interval code and an additional bit; the additional bit indicates the code length range of the third numerical interval; the interval code is encoded based on the initial bit; The result module is used to treat the unmatched character codeword, the length value codeword and the offset value codeword as a compressed data block for one of the data fragments.

19. A computer device, characterized in that: The computer device includes a processor and a memory, the memory is used to store at least one program, and the at least one program is loaded by the processor and executes the method for decompressing data as described in any one of claims 1 to 12 or the method for compressing data as described in any one of claims 13 to 16.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one program, which is loaded and executed by a processor to implement the method for decompressing data according to any one of claims 1 to 12 or the method for compressing data according to any one of claims 13 to 16.

Images