Compression and decompression chip, method, equipment and medium

Abstract

The invention discloses a compression and decompression chip, method, equipment and medium, and relates to the technical field of compression and decompression. Comprising a matching pair searching unit which is stripped from an original compression coding and compression decoding unit and is used for compression preparation, a matching pair recovery unit which is used for decompression preparation, a stripped target compression coding and compression decoding unit which is used for hardware compression and decompression and comprises various protocols, and a host interface management unit, the software definition engine interface unit and the multi-core central processing unit computing power reserve unit are used for carrying out software compression and decompression. In the application, the target compression coding and compression decoding unit comprises various protocols, so that various protocol coding and decoding are supported; only the only matching pair recovery unit and the only matching pair search unit are installed in the chip, so that chip resources are saved; the existence of software compression and decompression supports the use of new compression coding and compression decoding protocols.

Description

Technical Field

[0001] The present invention relates to the technical field of compression and decompression, and particularly relates to a compression and decompression chip, method, device and medium. Background Art

[0002] Since the appearance of the dictionary-based lossless data compression algorithm LZ77 (Lempel-Ziv 77), the industrial community has gradually developed a large family of LZ77 compression algorithms. Common ones include 842, Brotli, Deflate (zlib / gzip), LZ4, LZMA, Snappy, and Zstandard, etc. compression algorithms. They each have their application characteristics. For example, Deflate has a high compression ratio but a low data throughput rate; another example is that LZ4 has a significant improvement in the data throughput rate of compression and decompression compared to Deflate, but it makes a compromise in the compression ratio. In the specific application process, it is necessary to reasonably select a suitable algorithm according to the characteristics of the application; among them, Deflate is a lossless data compression algorithm that simultaneously uses the LZ77 algorithm and Huffman Coding.

[0003] However, the above compression algorithms were initially implemented by software, and due to their respective algorithm characteristics, they all consume a lot of CPU (Central Processing Unit) computing resources and bus bandwidth. Therefore, the industrial community has been exploring their hardware acceleration offloading solutions. Typical examples are IBM's NX series (842) and Intel's QAT (Deflate). However, these hardware acceleration offloading solutions are dedicated acceleration offloading hardware solutions for single algorithms, which makes it impossible to flexibly select different compression and decompression algorithms for hardware offloading when designing the system architecture. If multiple compression and decompression algorithms are supported in a hardware product, it usually requires separate independent hardware sub-modules inside to support them respectively, which also causes an increase in the hardware chip area (cost) and power consumption; and if a new compression coding protocol needs to be supported later, mass-produced products cannot be supported through firmware iteration and upgrade, and new hardware chips need to be redefined, developed and produced. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a compression and decompression chip, method, device and medium, which can save costs and support the use of new compression coding and compression decoding protocols. The specific solutions are as follows:

[0005] In a first aspect, the present application discloses a compression and decompression chip, which includes a matching pair search unit for compression preparation and a matching pair recovery unit for decompression preparation separated from the original compression encoding and compression decoding units, a target compression encoding and compression decoding unit for hardware compression and decompression after separation, which includes various protocols, a host interface management unit, a task finalization unit, a software-defined engine interface unit for software compression and decompression, and a multi-core central processing unit computing power reserve unit. Among them,

[0006] The host interface management unit is used to obtain a target command sent by the host; the target command includes a source address; the target command is a decompression command or a compression command;

[0007] The matching pair search unit is used to perform matching pair search processing on the first source data to obtain a first target processing result; the first source data is the data in the source address corresponding to the compression command;

[0008] The target compression encoding and compression decoding unit is used to perform compression encoding on the first target processing result corresponding to the compression command including the first target protocol through the target compression encoding unit corresponding to the first target protocol in the target compression encoding and compression decoding unit to obtain a first encoding result; the first target protocol is one of several protocols corresponding to the target compression encoding and compression decoding unit;

[0009] The software-defined engine interface unit and the multi-core central processing unit computing power reserve unit are used to perform compression encoding on the first target processing result corresponding to the compression command including the second target protocol to obtain a second encoding result; the second target protocol is a protocol that does not exist among all the protocols corresponding to all the target compression encoding units;

[0010] The target compression encoding and compression decoding unit is further used to perform compression decoding on the second source data in the source address corresponding to the decompression command including the first target protocol through the target compression decoding unit corresponding to the first target protocol in the target compression encoding and compression decoding unit to obtain a first decoding result;

[0011] The software-defined engine interface unit and the multi-core central processing unit computing power reserve unit are further used to perform compression decoding on the second source data corresponding to the decompression command including the second target protocol to obtain a second decoding result;

[0012] The matching pair recovery unit is used to perform matching pair recovery processing on the first decoding result or the second decoding result to obtain a second target processing result;

[0013] The task finalization unit is used to transmit the first encoding result, the second encoding result, or the second target processing result to the host.

[0014] Optionally, the compression command includes a first target address; the target chip further includes a dynamic memory management unit, a host data direct access unit, and a data cache; the host data direct access unit includes a first access unit and a second access unit, where,

[0015] The dynamic memory management unit is used to reserve a first storage space, a second storage space, and a third storage space in the data cache for the compression command;

[0016] The first access unit is used to move the first source data to the first storage space;

[0017] The matching pair search unit is used to perform matching pair search processing on the first source data in the first storage space to obtain the first target processing result, and store the first target processing result in the second storage space;

[0018] The target compression encoding and decompression decoding unit is used to perform compression encoding on the first target processing result in the second storage space to obtain the first encoding result when the compression command includes the first target protocol, and store the first encoding result in the third storage unit;

[0019] The software-defined engine interface unit and the multi-core central processor computing power reserve unit are used to perform compression encoding on the first target processing result in the second storage space to obtain the second encoding result when the compression command includes the second target protocol, and store the second encoding result in the third storage unit;

[0020] The second access unit is used to move the first encoding result or the second encoding result in the third storage space to the first target address;

[0021] The dynamic memory management unit is further used to recycle the first storage space, the second storage space, and the third storage space;

[0022] The task finalization unit is used to transmit the first encoding result or the second encoding result in the first target address to the host.

[0023] Optionally, the decompression command includes a second target address; the target chip further includes a dynamic memory management unit, a host data direct access unit, and a data cache, and the host data direct access unit includes a first access unit and a second access unit, where,

[0024] The dynamic memory management unit is used to reserve a fourth storage space, a fifth storage space, and a sixth storage space in the data cache for the decompression command;

[0025] The first access unit is used to move the second source data to the fourth storage space;

[0026] The target compression encoding and decompression decoding unit is used to perform decompression decoding on the second source data in the fourth storage space to obtain a first decoding result when the decompression command includes the first target protocol, and store the first decoding result in the fifth storage space;

[0027] The software-defined engine interface unit and the multi-core central processor computing power reserve unit are used to perform decompression decoding on the second source data in the fourth storage space to obtain a second decoding result when the decompression command includes the second target protocol, and store the second decoding result in the fifth storage space;

[0028] The matching pair recovery unit is used to perform matching pair recovery processing on the first decoding result or the second decoding result in the fifth storage space to obtain a second target processing result, and store the second target processing result in the sixth storage space;

[0029] The second access unit is used to move the second target processing result in the sixth storage space to the second target address;

[0030] The dynamic memory management unit is further used to reclaim the fourth storage space, the fifth storage space, and the sixth storage space;

[0031] The task finishing unit is used to transmit the second target processing result in the second target address to the host.

[0032] Optionally, the target chip further includes a task processing sequence creation unit and a work queue management unit, where

[0033] The task processing sequence creation unit is used to construct a target control block corresponding to the target command, store the target control block in a pre-allocated control page table, and send a control block handle corresponding to the dynamic memory management unit to the work queue management unit; where the control block handle is used to mark the next unit other than the work queue management unit and the entry address of the corresponding control page table of the target control block of the part of the control block required by the next unit; the part of the control block is part of the target control block;

[0034] The working queue management unit is configured to send the control block gripper obtained from the previous unit to the next unit, so that the next unit can complete the local operation task based on the obtained control block gripper and generate a new control block gripper locally and then send it to the working queue management unit until the task termination unit releases the pre-allocated control page table and the working queue management unit no longer obtains a new control block gripper corresponding to the target command, so as to realize the sequential operation of each target unit in the target unit group;

[0035] Wherein, the target units arranged in the running order in the target unit group include the dynamic memory management unit, the first access unit, the matching pair search unit, the target compression encoding and decompression decoding unit or the combination unit, the second access unit, the dynamic memory management unit and the task termination unit; the combination unit is the software-defined engine interface unit and the multi-core central processing unit computing power reserve unit;

[0036] Or, the target units arranged in the running order in the target unit group include the dynamic memory management unit, the first access unit, the target compression encoding and decompression decoding unit or the combination unit, the matching pair recovery unit, the second access unit, the dynamic memory management unit and the task termination unit.

[0037] Optionally, the working queue management unit is further configured to configure a target working queue for storing control block grippers for each target unit in the target unit group to sequentially store control block grippers of different target commands; the target working queue stores a control block gripper marking the next unit as this target unit.

[0038] Optionally, different target units in the target unit group can concurrently complete their own tasks based on control block grippers of different target commands.

[0039] Optionally, the matching pair search unit is specifically configured to perform matching pair search processing on the first source data to obtain a first initial processing result, and encapsulate the first initial processing result based on the target byte format to obtain a first target processing result;

[0040] The matching pair recovery unit is specifically configured to perform matching pair recovery processing on the first decoding result or the second decoding result to obtain a second initial processing result, and encapsulate the second initial processing result based on the target byte format to obtain a second target processing result;

[0041] Correspondingly, the target byte format includes an original text length byte, a matching length byte, an original text length reserve byte, a matching offset byte, a matching length reserve byte, and original text bytes; the sum of the contents in the original text length byte and the original text length reserve byte is the original text length; the sum of the contents in the matching length byte and the matching length reserve byte is the matching length; the original text length byte includes an original text base length byte and a matching base length byte.

[0042] In a second aspect, the present application discloses a compression and decompression method applied to a compression and decompression chip. The compression and decompression chip includes a matching pair search unit for compression preparation and a matching pair recovery unit for decompression preparation stripped from the original compression encoding and compression decoding units, a target compression encoding and compression decoding unit for hardware compression and decompression after stripping, including various protocols, a host interface management unit, a task closing unit, a software-defined engine interface unit for software compression and decompression, and a multi-core central processing unit computing power reserve unit. The method includes:

[0043] Obtaining a target command sent by a host through the host interface management unit; the target command includes a source address; the target command is a decompression command or a compression command;

[0044] If the target command is the compression command, then through the matching pair search unit, perform matching pair search processing on the first source data to obtain a first target processing result, and through the target compression encoding and compression decoding unit, perform compression encoding on the first target processing result corresponding to the compression command including the first target protocol to obtain a first encoding result, or, and through the software-defined engine interface unit and the multi-core central processing unit computing power reserve unit, perform compression encoding on the first target processing result corresponding to the compression command including the second target protocol to obtain a second encoding result; the first source data is the data in the source address corresponding to the compression command; the first target protocol is one of several protocols corresponding to the target compression encoding and compression decoding unit; the second target protocol is a protocol that does not exist in all the protocols corresponding to all the target compression encoding units.

[0045] If the target command is the compression command, the target compression encoding and compression decoding unit is used to perform compression decoding on the second source data in the source address corresponding to the decompression command including the first target protocol to obtain a first decoding result, or the software-defined engine interface unit and the multi-core central processing unit computing power reserve unit are used to perform compression decoding on the second source data corresponding to the decompression command including the second target protocol to obtain a second decoding result, and the matching pair restoration unit is used to perform matching pair restoration processing on the first decoding result or the second decoding result to obtain a second target processing result;

[0046] The first encoding result, the second encoding result, or the second target processing result is transmitted to the host through the task closing unit.

[0047] In a third aspect, the present application discloses an electronic device, including:

[0048] A memory for storing a computer program;

[0049] A processor for executing the computer program to implement the compression and decompression methods disclosed above.

[0050] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the compression and decompression methods disclosed above are implemented.

[0051] It can be seen that in the present application, units of different protocols are all installed in the chip, enabling a single chip to support different encoding and decoding protocols; only a single matching pair restoration unit and a matching pair search unit are installed in the chip, saving chip resources, reducing costs and energy consumption; the software-defined engine interface unit and the multi-core central processing unit computing power reserve unit are added to achieve the integration of hardware compression and decompression and software compression and decompression, so that when there are no units corresponding to the new compression and decompression protocols in the chip, software compression and decompression can be performed using the new compression and decompression protocols through software call, thereby supporting the use of new compression encoding and compression decoding protocols. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings according to the provided drawings without creative efforts.

[0053] Figure 1 Schematic diagram of a compression and decompression chip disclosed in the present application;

[0054] Figure 2 A schematic diagram of a simple compression and decompression process disclosed in this application;

[0055] Figure 3 A schematic diagram of a simple compression and decompression process disclosed in this application;

[0056] Figure 4 A schematic diagram of a target byte format disclosed in this application;

[0057] Figure 5 A schematic diagram of a specific compression and decompression chip disclosed in this application;

[0058] Figure 6 A schematic diagram of a specific compression encoding process disclosed in this application;

[0059] Figure 7 A schematic diagram of a specific compression and decompression chip disclosed in this application;

[0060] Figure 8 Another schematic diagram of a specific compression encoding process disclosed in this application;

[0061] Figure 9 A schematic diagram of a compression and decompression chip architecture disclosed in this application;

[0062] Figure 10 A schematic diagram of a compression and decompression method process disclosed in this application;

[0063] Figure 11 A structural diagram of an electronic device disclosed in this application. Detailed implementation manners

[0064] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0065] Compression algorithms were initially implemented in software. Due to their respective algorithm characteristics, they consume a great deal of CPU computing resources and bus bandwidth. Therefore, the industrial community has been exploring their hardware acceleration offloading solutions. Typical examples include IBM's NX series (842) and Intel's QAT (Deflate). However, these hardware acceleration offloading solutions are dedicated acceleration offloading hardware solutions for single algorithms. This makes it impossible to flexibly select different compression and decompression algorithms for hardware offloading during system architecture design. If multiple compression and decompression algorithms are to be supported in a hardware product, it usually requires independent hardware sub-modules to support them respectively inside, which in turn leads to an increase in the area (cost) and power consumption of the hardware chip. Moreover, if a new compression coding protocol needs to be supported later, mass-produced products cannot be supported through firmware iteration and upgrade, and new hardware chips need to be redefined, developed, and produced.

[0066] Therefore, an embodiment of this application proposes a compression and decompression chip that can save costs and support the use of new compression coding and compression decoding protocols.

[0067] An embodiment of this application discloses a compression and decompression chip. As shown in Figure 1 the figure, the chip includes a matching pair search unit 11 for compression preparation and a matching pair recovery unit 12 for decompression preparation separated from the original compression coding and compression decoding units, a target compression coding and compression decoding unit 13 containing various protocols for hardware compression and decompression after separation, a host interface management unit 14, a task finalization unit 15, a software-defined engine interface unit for software compression and decompression, and a multi-core central processing unit computing power reserve unit 16. Among them,

[0068] the host interface management unit 14 is used to obtain a target command sent by the host; the target command includes a source address; the target command is a decompression command or a compression command;

[0069] the matching pair search unit 11 is used to perform matching pair search processing on the first source data to obtain a first target processing result; the first source data is the data in the source address corresponding to the compression command;

[0070] the target compression coding and compression decoding unit 13 is used to perform compression coding on the first target processing result corresponding to the compression command containing the first target protocol through the target compression coding unit corresponding to the first target protocol in the target compression coding and compression decoding unit to obtain a first coding result; the first target protocol is one of several protocols corresponding to the target compression coding and compression decoding unit;

[0071] The software-defined engine interface unit and the multi-core central processing unit computing power reserve unit 16 are used to perform compression encoding on the first target processing result corresponding to the compression command including the second target protocol to obtain a second encoding result; the second target protocol is a protocol that does not exist in all the protocols corresponding to all the target compression encoding units;

[0072] The target compression encoding and decompression unit 13 is further used to perform decompression decoding on the second source data in the source address corresponding to the decompression command including the first target protocol through the target decompression decoding unit corresponding to the first target protocol in the target compression encoding and decompression unit to obtain a first decoding result;

[0073] The software-defined engine interface unit and the multi-core central processing unit computing power reserve unit 16 are further used to perform decompression decoding on the second source data corresponding to the decompression command including the second target protocol to obtain a second decoding result;

[0074] The matching pair recovery unit 12 is used to perform matching pair recovery processing on the first decoding result or the second decoding result to obtain a second target processing result;

[0075] The task closing unit 15 is used to transmit the first encoding result or the second encoding result or the second target processing result to the host.

[0076] It should be noted that the original matching pair recovery unit and matching pair search unit existed in each of the compression encoding unit and decompression decoding unit in the original compression encoding and decompression unit. Now, the matching pair search unit for compression preparation and the matching pair recovery unit for decompression preparation separated out only retain one matching pair recovery unit and one matching pair search unit in the entire chip, saving hardware resources, reducing costs and energy consumption.

[0077] It should be noted that a software-defined engine interface unit and a multi-core central processing unit computing power reserve unit are added to the chip. When there are no units corresponding to the new compression and decompression protocols in the chip, software calls are used to perform software compression and decompression using the new compression and decompression protocols; that is, the present application realizes the integration of hardware compression and decompression and software compression and decompression.

[0078] In this embodiment, the matching pair search unit is specifically used to perform matching pair search processing on the first source data to obtain a first initial processing result, and encapsulate the first initial processing result based on the target byte format to obtain a first target processing result;

[0079] The matching pair recovery unit is specifically configured to perform matching pair recovery processing on the first decoding result or the second decoding result to obtain a second initial processing result, and encapsulate the second initial processing result based on the target byte format to obtain a second target processing result;

[0080] Correspondingly, the target byte format includes an original text length byte, a matching length byte, an original text length reserve byte, a matching offset byte, a matching length reserve byte, and an original text byte; the sum of the contents in the original text length byte and the original text length reserve byte is the original text length; the sum of the contents in the matching length byte and the matching length reserve byte is the matching length; the original text length byte includes an original text base length byte and a matching base length byte.

[0081] It should be noted that after the matching pair recovery unit and the matching pair search unit are extracted in this application, an encapsulation format is specified for the processing results of the matching pair recovery unit and the matching pair search unit. A schematic diagram of a simple compression and decompression process is shown in Figure 2 and Figure 3 As shown, specifically, for the compression scenario, the data original text is queried for matching pairs through an MSU (Match Search Unit), that is, a longest continuous matching string from the current position backward is searched in a sliding window (a range of several KB) forward from the current position, and the repeated string is replaced with (matching offset, matching length). This is the basis of various dictionary-based compression algorithms. The intermediate result is encapsulated and output, and then different compression protocols are selected according to the protocol requirements to perform secondary encapsulation and output on the intermediate result; for the decompression scenario, first, it is decoded according to the compression encoding protocol followed by the current compressed package and output in the intermediate result encapsulation format, and then the original text is recovered and output through an MRU (Match Recover Unit).

[0082] It should be noted that the target byte format also becomes the intermediate result format, which is the output of the MSU and the input of the MRU, and is composed of one or several LMPC (Literal Match Pair Cell, original text matching pair unit). See Figure 4As shown in the figure, it is a schematic diagram of a target byte format. The format definition of LMPC is as follows: 1. The high four bits (LL) of the "start" byte represent the length of the original text. The effective representation range is 0 to 14 bytes. When the required length is greater than or equal to 15, then LL = 15, and the original text length reserve is enabled. Each byte can represent a maximum length of 0 to 255. The range ends at the first byte whose value is not equal to 255. The final original text length is the sum L of LL and the original text length reserve added byte by byte. 2. The low four bits (ML) of the "start" byte represent the matching length. The effective representation range is 0 to 14 bytes. When the required length is greater than or equal to 15, then ML = 15, and the matching length reserve is enabled. Each byte can represent a maximum length of 0 to 255. The range ends at the first byte whose value is not equal to 255. The final matching length is the sum of ML and the matching length reserve added byte by byte. 3. The matching offset is fixed at 2 bytes, representing the offset of 0 to 65535 bytes forward from the current position. 4. The original text is output in sequence with a length of L.

[0083] In this embodiment, when processing the same target command, each unit processes it sequentially. However, when processing different target commands, they do not affect each other. Therefore, different units can process different commands simultaneously, that is, different units process concurrently.

[0084] It can be seen that in this application, units of different protocols are all installed in the chip, enabling the same chip to support different encoding and decoding protocols; only a single matching pair recovery unit and a matching pair search unit are installed in the chip, saving chip resources, reducing costs and energy consumption; a software-defined engine interface unit and a multi-core central processor computing power reserve unit are added, realizing the integration of hardware compression and decompression and software compression and decompression. So that when there is no unit corresponding to the new compression and decompression protocol in the chip, software compression and decompression can be performed using the new compression and decompression protocol through software call, thereby supporting the use of the new compression encoding and compression decoding protocol; different units process different commands concurrently, improving the rate.

[0085] An embodiment of the present application discloses a specific compression and decompression chip. Compared with the previous embodiment, the technical solution in this embodiment is further described and optimized. The chip includes a matching pair search unit 11 for compression preparation and a matching pair recovery unit 12 for decompression preparation separated from the original compression encoding and compression decoding units, a target compression encoding and compression decoding unit 13 for hardware compression and decompression after separation, including various protocols, a host interface management unit 14, a task closing unit 15, a software-defined engine interface unit for software compression and decompression, and a multi-core central processing unit computing power reserve unit 16. The chip also includes a dynamic memory management unit 17, a host data direct access unit 18, and a data cache 19. The host data direct access unit 18 includes a first access unit 181 and a second access unit 182, and the target command is the compression command, and the compression command includes a first target address. See Figure 5 as shown, specifically including:

[0086] The dynamic memory management unit 17 is used to reserve a first storage space, a second storage space, and a third storage space for the compression command in the data cache;

[0087] The first access unit 181 is used to move the first source data to the first storage space;

[0088] The matching pair search unit 11 is used to perform matching pair search processing on the first source data in the first storage space to obtain the first target processing result, and store the first target processing result in the second storage space;

[0089] When the compression command includes the first target protocol, the target compression encoding and compression decoding unit 13 is used to perform compression encoding on the first target processing result in the second storage space to obtain the first encoding result, and store the first encoding result in the third storage unit;

[0090] When the compression command includes the second target protocol, the software-defined engine interface unit and the multi-core central processing unit computing power reserve unit 16 are used to perform compression encoding on the first target processing result in the second storage space to obtain the second encoding result, and store the second encoding result in the third storage unit;

[0091] The second access unit 182 is used to move the first encoding result or the second encoding result in the third storage space to the first target address;

[0092] The dynamic memory management unit 17 is further used to recycle the first storage space, the second storage space, and the third storage space;

[0093] The task closing unit 15 is used to transmit the first encoding result or the second encoding result in the first target address to the host.

[0094] In this embodiment, the sequential processing of commands between different units is achieved through the task processing sequence creation unit and the work queue management unit. The task processing sequence creation unit constructs a control block, and the work queue management unit controls the running order of different units based on the control block handle. Correspondingly, the target chip further includes a task processing sequence creation unit and a work queue management unit. Among them, the task processing sequence creation unit is used to construct a target control block corresponding to the target command, store the target control block in a pre-allocated control page table, and send the control block handle corresponding to the dynamic memory management unit to the work queue management unit; wherein, the control block handle is used to mark the next unit other than the work queue management unit and the entry address of the corresponding control page table of the partial control block required by the next unit in the target control block; the partial control block is a partial control block in the target control block; the work queue management unit is used to send the obtained control block handle sent by the previous unit to the next unit, so that the next unit can complete the local operation task based on the obtained control block handle and generate a new control block handle locally and then send it to the work queue management unit until the task closing unit releases the pre-allocated control page table and the work queue management unit no longer obtains a new control block handle corresponding to the target command, so as to realize the sequential operation of each target unit in the target unit group; wherein, the target units arranged in the running order in the target unit group include the dynamic memory management unit, the first access unit, the matching pair search unit, the target compression encoding and compression decoding unit or the combination unit, the second access unit, the dynamic memory management unit and the task closing unit; the combination unit is the software-defined engine interface unit and the multi-core central processing unit computing power reserve unit.

[0095] Specifically, reference can be made to Figure 6 shown in a specific compression encoding process schematic diagram constructed according to the above unit order.

[0096] In this embodiment, the work queue management unit stores the control block handle and ensures the running order of the control block handles of different commands, and constructs a target work queue for each processed unit. Correspondingly, the work queue management unit is further used to configure a target work queue for storing the control block handle for each target unit in the target unit group to sequentially store the control block handles of different target commands; the target work queue stores the control block handle marking the next unit as this target unit.

[0097] In this embodiment, different target units in the target unit group can concurrently complete their own tasks based on the control block grippers of different target commands, so as to improve efficiency.

[0098] It can be seen that the present application proposes a more perfect structural composition of the compression and decompression chip. The existence of the work queue management unit stipulates the processing order of each unit, preventing the phenomenon of incorrect unit processing order. The existence of the target work queue prevents the disorder of the processing order between different target commands.

[0099] The embodiment of the present application discloses a specific compression and decompression chip. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. The chip includes a matching pair search unit 11 for compression preparation and a matching pair recovery unit 12 for decompression preparation separated from the original compression encoding and compression decoding units, a target compression encoding and compression decoding unit 13 for hardware compression and decompression containing various protocols after separation, a host interface management unit 14, a task finishing unit 15, a software-defined engine interface unit for software compression and decompression, and a multi-core central processor computing power reserve unit 16. The chip further includes a dynamic memory management unit 17, a host data direct access unit 18, and a data cache 19. The host data direct access unit 18 includes a first access unit 181 and a second access unit 182, and the target command is the decompression command, and the decompression command includes a second target address. See Figure 7 As shown, specifically including:

[0100] The dynamic memory management unit 17 is used to reserve a fourth storage space, a fifth storage space, and a sixth storage space for the decompression command in the data cache;

[0101] The first access unit 181 is used to move the second source data to the fourth storage space;

[0102] When the decompression command includes the first target protocol, the target compression encoding and compression decoding unit 13 is used to perform compression decoding on the second source data in the fourth storage space to obtain a first decoding result, and store the first decoding result in the fifth storage space;

[0103] When the decompression command includes the second target protocol, the software-defined engine interface unit and the multi-core central processor computing power reserve unit 16 are used to perform compression decoding on the second source data in the fourth storage space to obtain a second decoding result, and store the second decoding result in the fifth storage space;

[0104] The matching pair restoration unit 12 is configured to perform matching pair restoration processing on the first decoding result or the second decoding result in the fifth storage space to obtain a second target processing result, and store the second target processing result in the sixth storage space;

[0105] The second access unit 182 is configured to move the second target processing result in the sixth storage space to the second target address;

[0106] The dynamic memory management unit 17 is further configured to recycle the fourth storage space, the fifth storage space, and the sixth storage space;

[0107] The task closing unit 15 is configured to transmit the second target processing result in the second target address to the host.

[0108] In this embodiment, the sequential processing of commands between different units is implemented by a task processing sequence creation unit and a work queue management unit. The task processing sequence creation unit constructs a control block, and the work queue management unit controls the running order of different units based on the control block handle. Correspondingly, the task processing sequence creation unit is configured to construct a target control block corresponding to the target command, store the target control block in a pre-allocated control page table, and send the control block handle corresponding to the dynamic memory management unit to the work queue management unit; wherein, the control block handle is used to mark the next unit other than the work queue management unit and the entry address of a part of the control block required by the next unit in the corresponding control page table of the target control block; the part of the control block is a part of the target control block; the work queue management unit is configured to send the obtained control block handle sent by the previous unit to the next unit, so that the next unit can complete the local operation task based on the obtained control block handle and generate a new control block handle locally and then send it to the work queue management unit until the task closing unit releases the pre-allocated control page table and the work queue management unit no longer obtains a new control block handle corresponding to the target command, so as to realize the sequential operation of each target unit in the target unit group; wherein the target units arranged in the running order in the target unit group include the dynamic memory management unit, the first access unit, the target compression encoding and compression decoding unit or the combination unit, the matching pair restoration unit, the second access unit, the dynamic memory management unit, and the task closing unit.

[0109] Specifically, reference can be made to Figure 8 shown, which is a schematic diagram of a compression decoding process constructed according to the above unit order.

[0110] In this embodiment, the work queue management unit is a saved control block gripper, and ensures the running order of the control block grippers of different commands. A target work queue is constructed for each processing unit. Correspondingly, the work queue management unit is further configured to configure a target work queue for storing the control block gripper for each target unit in the target unit group, so as to sequentially store the control block grippers of different target commands; the target work queue stores the control block gripper that marks the next unit as the target unit.

[0111] In this embodiment, different target units in the target unit group can concurrently complete their own tasks based on the control block grippers of different target commands, so as to improve efficiency.

[0112] It should be noted that the first target address and the second target address may be the same or different, and the first storage space, the second storage space, the third storage space and the corresponding fourth storage space, the fifth storage space, and the sixth storage space may be the same or different.

[0113] It can be seen that the present application proposes a more perfect structural composition of the compression and decompression chip, and the existence of the work queue management unit stipulates the processing order of each unit, preventing the phenomenon of incorrect unit processing order, and the existence of the target work queue prevents the disorder of the processing order between different target commands.

[0114] Specifically, refer to Figure 9As shown in the figure, it is a schematic diagram of a compression and decompression chip architecture; the compression and decompression chip architecture is a multi-protocol extensible lossless compression architecture (chip) based on the UAA (Unified Acceleration Architecture, general acceleration architecture) general acceleration architecture, and internally integrates basic engines related to UAA including AEM (Acceleration Engine Manager, host interface management engine), cpBuild (Control Page Build, control page (task processing sequence) creation engine), WQS (Work Queue Scheduler, work queue scheduling (management) engine), HDMA (C2H / H2C) (HDMAC2H, Host Direct Memory Access: Client to Host, host direct memory access engine: host to client direction) (HDMA H2C, Host Direct Memory Access: Host to Client, host direct memory access engine: host to client direction), deAlloc (Deallocate&Allocate, memory dynamic management engine), SEWB (Software Engine Bank, software-defined engine interface), and Closure (task end finishing engine), etc., as well as on-chip storage (OCBc, On-Chip-Buffer-Control, on-chip control information storage) for storing control information; for the compression service scenario, one or more of the following are integrated according to the throughput requirement: MSU, MRU, Deflate / Inflate, LZ4, ZSTD (Zstandard) protocol encoding and decoding engines, and other protocol encoding and decoding engines may also be included. These encoding and decoding protocol engines are collectively referred to as the target compression encoding and compression decoding units, as well as multi-core CPU resources reserved for later support of other compression protocols and on-chip storage (OCBd, On-Chip-Buffer-Data, on-chip data information storage) or off-chip DDR (Double Data Rate, double data rate synchronous dynamic random access memory) storage for storing data. In the figure, H2C in the host data direct storage access unit is the first access unit, and C2H is the second access unit.

[0115] Correspondingly, the embodiment of the present application also discloses a compression and decompression method. See Figure 10As shown, it is applied to a compression and decompression chip. The compression and decompression chip includes a matching pair search unit for compression preparation and a matching pair recovery unit for decompression preparation that are stripped from the original compression encoding and compression decoding units, a target compression encoding and compression decoding unit for hardware compression and decompression after stripping, which includes various protocols, a host interface management unit, a task closing unit, a software-defined engine interface unit for software compression and decompression, and a multi-core central processing unit computing power reserve unit. The method includes:

[0116] Step S11: Obtain a target command sent by the host through the host interface management unit; the target command includes a source address; the target command is a decompression command or a compression command;

[0117] It should be noted that the original matching pair recovery unit and matching pair search unit exist in each compression encoding unit and compression decoding unit of the original compression encoding and compression decoding units. Now, the stripped matching pair search unit for compression preparation and the matching pair recovery unit for decompression preparation only retain one matching pair recovery unit and one matching pair search unit in the entire chip, saving hardware resources, reducing costs and energy consumption.

[0118] It should be noted that a software-defined engine interface unit and a multi-core central processing unit computing power reserve unit are added to the chip. When there is no unit corresponding to the new compression and decompression protocol in the chip, software compression and decompression are performed using the new compression and decompression protocol through software call; that is, the present application realizes the integration of hardware compression and decompression and software compression and decompression.

[0119] Step S12: If the target command is the compression command, perform matching pair search processing on the first source data through the matching pair search unit to obtain a first target processing result, and perform compression encoding on the first target processing result corresponding to the compression command including the first target protocol through the target compression encoding unit corresponding to the first target protocol in the target compression encoding and compression decoding unit to obtain a first encoding result, or, perform compression encoding on the first target processing result corresponding to the compression command including the second target protocol through the software-defined engine interface unit and the multi-core central processing unit computing power reserve unit to obtain a second encoding result; the first source data is the data in the source address corresponding to the compression command; the first target protocol is one of several protocols corresponding to the target compression encoding and compression decoding unit; the second target protocol is a protocol that does not exist in all the protocols corresponding to all the target compression encoding units.

[0120] Step S13: If the target command is the compression command, then through the target decompression decoding unit corresponding to the first target protocol in the target compression encoding and decompression decoding unit, decompress and decode the second source data in the source address corresponding to the decompression command including the first target protocol to obtain a first decoding result, or, through the software-defined engine interface unit and the multi-core central processing unit computing power reserve unit, decompress and decode the second source data in the decompression command including the second target protocol to obtain a second decoding result, and through the matching pair restoration unit, perform matching pair restoration processing on the first decoding result or the second decoding result to obtain a second target processing result.

[0121] In this embodiment, the performing matching pair search processing on the first source data by the matching pair search unit to obtain a first target processing result includes: performing matching pair search processing on the first source data by the matching pair search unit to obtain a first initial processing result, and encapsulating the first initial processing result based on the target byte format to obtain a first target processing result.

[0122] In this embodiment, the performing matching pair restoration processing on the first decoding result or the second decoding result by the matching pair restoration unit to obtain a second target processing result includes: the matching pair restoration unit is specifically configured to perform matching pair restoration processing on the first decoding result or the second decoding result to obtain a second initial processing result, and encapsulate the second initial processing result based on the target byte format to obtain a second target processing result.

[0123] It should be noted that the target byte format includes an original text length byte, a matching length byte, an original text length reserve byte, a matching offset byte, a matching length reserve byte, and an original text byte; the sum of the contents in the original text length byte and the original text length reserve byte is the original text length; the sum of the contents in the matching length byte and the matching length reserve byte is the matching length; the original text length byte includes an original text base length byte and a matching base length byte.

[0124] It should be noted that after extracting the matching pair restoration unit and the matching pair search unit in this application, an encapsulation format is specified for the processing results of the matching pair restoration unit and the matching pair search unit. A simple schematic diagram of the compression and decompression process is as Figure 2 and Figure 3 shown. The target byte format also becomes the intermediate result format, which is the output of the MSU and the input of the MRU, and is composed of one or several LMPCs (original text matching pair units). For specific reference, see Figure 4 shown.

[0125] In this embodiment, the chip further includes a dynamic memory management unit, a host data direct access unit, and a data cache; the host data direct access unit includes a first access unit and a second access unit; the chip further includes a task processing sequence creation unit and a work queue management unit. The specific usage method is not introduced in detail here, and reference can be made to the content in the foregoing embodiment.

[0126] It should be noted that the sequential processing of commands between different units is achieved through the task processing sequence creation unit and the work queue management unit. The task processing sequence creation unit constructs a control block, and the work queue management unit controls the running order of different units based on the control block handle.

[0127] It should be noted that the work queue management unit stores the control block handle and ensures the running order of the control block handles of different commands, and constructs a target work queue for each processed unit.

[0128] Step S14: Transmit the first encoding result, the second encoding result, or the second target processing result to the host through the task termination unit.

[0129] It can be seen that in this application, units of different protocols are all installed in the chip to achieve the same chip supporting different encoding and decoding protocols; only the unique match pair recovery unit and the match pair search unit are installed in the chip, saving chip resources, reducing costs and energy consumption; the software-defined engine interface unit and the multi-core central processing unit computing power reserve unit are added to achieve the integration of hardware compression and decompression and software compression and decompression, so that when there is no unit corresponding to the new compression and decompression protocol in the chip, software compression and decompression can be performed by software call using the new compression and decompression protocol, thereby supporting the use of the new compression encoding and compression decoding protocol; different units concurrently process different commands to improve the rate; the existence of the work queue management unit stipulates the processing order of each unit, preventing the phenomenon of incorrect unit processing order, and the existence of the target work queue prevents the processing order confusion between different target commands.

[0130] In a specific embodiment, refer to Figure 9 As shown, a specific compression operation process is as follows (taking Deflate as an example):

[0131] First, the AEM engine is responsible for the host IO (Input / Output) task interface, retrieves the IO commands issued by the host to the local through the PCIe EP module, and passes the IO instructions to the cpBuild module in sequence.

[0132] Second, the cpBuild module parses the IO commands. According to the required operation type (compression / decompression), algorithm selection, source data address and size, and destination data address and size of the result, etc., it constructs a step-by-step processing CB (Control Block) chain in accordance with the UAA protocol and stores the CB chain in one or more pre-allocated CP pages on the OCBc. After completing the above tasks, it submits the CB_Handle (grip, including the entry address of the CB and the corresponding engine type) of the first CB to the WQS to start the step-by-step sequential scheduling of the task.

[0133] Third, inside the WQS, a work queue is matched for each engine type, and CB_Handles of the same type are stored in the corresponding work queue in the order of entry. According to the concurrent processing ability of each engine for CBs, at most the corresponding number of CB_Handles (CB grips, engine type information, and corresponding control block storage address) are dispatched to the engine. After each CB task is completed on the engine side, it needs to prepare the CB_Handle for the next task sequential operation and submit it to the WQS to enter the corresponding work queue, and at the same time reply to the WQS with a "completed" signal. After receiving this "completed" signal each time, the WQS issues a CB task to the engine.

[0134] Fourth, after receiving the task dispatched by the WQS, the Alloc engine dynamically allocates storage spaces A (for storing source data), B (for storing intermediate result data), and C (for storing final result data) of the corresponding size in the OCBd or DDR according to the task description. After completion, it replies to the WQS with "completed" and submits the CB_Handle of the next HDMA H2C to the WQS.

[0135] Fifth, after receiving the task dispatched by the WQS, the HDMA H2C engine is responsible for moving the source data (original data) in the IO instruction to the local data cache A through the DMA of the PCIe EP according to the task description. After completion, it replies to the WQS with "completed" and submits the CB_Handle of the next MSU to the WQS.

[0136] Sixth, after receiving the task dispatched by the WQS, the MSU engine takes the data in A as input, searches for and marks the matching pairs, and finally outputs the intermediate result to the storage space B. After completion, it replies to the WQS with "completed" and submits the CB_Handle of the next Deflate to the WQS.

[0137] Seventh, after receiving the task dispatched by WQS, the Deflate engine takes the data in B as input according to the task description, performs dynamic / static Huffman coding on it, and outputs the result to storage space C. After completion, it replies "completed" to WQS and submits the CB_Handle of the next HDMAC2H to WQS.

[0138] Eighth, after receiving the task dispatched by WQS, the HDMAC2H engine is responsible for moving the result data (compressed data packet) in the local data cache C to the target data address specified by the IO instruction through the DMA of the PCIe EP. After completion, it replies "completed" to WQS and submits the CB_Handle of the next Dealloc to WQS.

[0139] Ninth, after receiving the task dispatched by WQS, the Dealloc engine reclaims storage spaces A, B, and C according to the task description. After completion, it replies "completed" to WQS and submits the CB_Handle of the next Closure to WQS.

[0140] Tenth, after receiving the task dispatched by WQS, the Closure engine, after completion, replies "completed" to WQS, responds to the host through AEM with the execution result, and reclaims and releases the occupied CP (Control Page) resources.

[0141] A specific decompression operation process is as follows (taking Inflate as an example):

[0142] First, the AEM engine is responsible for the host IO task interface, retrieves the IO command sent by the host to the local through the PCIe EP module, and passes the IO instruction to the cpBuild module in sequence.

[0143] Second, the cpBuild module parses the IO command. According to the required operation type (compression / decompression), algorithm selection, source data address and size, and destination data address and size of the result, etc., it constructs a step-by-step processing CB chain in accordance with the UAA protocol and stores the CB chain in one or more pre-allocated CP pages on the OCBc. After completing the above tasks, it submits the CB_Handle (grabber, including the entry address of the CB and the corresponding engine type) of the first CB to WQS to start the step-by-step sequential scheduling of the task.

[0144] Thirdly, inside WQS, a work queue is matched according to each engine type, and CB_Handles of the same type are stored in the corresponding work queue in the order of entry. According to the concurrent processing ability of each engine for CBs, at most the corresponding number of CB_Handles are dispatched to the engine. Each time an engine completes a CB task, it needs to prepare the CB_Handle for the next task sequence operation and submit it to WQS to enter the corresponding work queue, and at the same time send a "completed" signal to WQS. After receiving this "completed" signal each time, WQS issues a CB task to the engine again.

[0145] Fourthly, after receiving the task dispatched by WQS, the Alloc engine dynamically allocates storage spaces A (for storing source data), B (for storing intermediate result data), and C (for storing final result data) of corresponding sizes in OCBd or DDR according to the task description. After completion, it sends a "completed" to WQS and submits the CB_Handle of the next HDMA H2C to WQS.

[0146] Fifthly, after receiving the task dispatched by WQS, the HDMA H2C engine is responsible for moving the source data (compressed data packet) in the IO instruction to the local data cache A through the DMA of PCIe EP according to the task description. After completion, it sends a "completed" to WQS and submits the CB_Handle of the next Inflate to WQS.

[0147] Sixthly, after receiving the task dispatched by WQS, the Inflate engine takes the data in A as input, performs dynamic / static Huffman decoding on it, and outputs the result to the storage space B according to the task description. After completion, it sends a "completed" to WQS and submits the CB_Handle of the next MRU to WQS.

[0148] Seventhly, after receiving the task dispatched by WQS, the MRU engine takes the data in B as input, restores the matching pairs in it, and outputs the original text result to the storage space C according to the task description. After completion, it sends a "completed" to WQS and submits the CB_Handle of the next HDMAC2H to WQS.

[0149] Eighthly, after receiving the task dispatched by WQS, the HDMAC2H engine is responsible for moving the result data (original text data) in the local data cache C to the target data address specified by the IO instruction through the DMA of PCIe EP according to the task description. After completion, it sends a "completed" to WQS and submits the CB_Handle of the next Dealloc to WQS.

[0150] Ninth, after receiving the task dispatched by the WQS, the Dealloc engine reclaims storage spaces A, B, and C according to the task description. After completion, it replies "completed" to the WQS and submits the CB_Handle of the next Closure to the WQS.

[0151] Tenth, after receiving the task dispatched by the WQS, the Closure engine completes the task according to the task description. After completion, it replies "completed" to the WQS, responds to the host through the AEM with the execution result, and reclaims and releases the occupied CP resources.

[0152] It should be noted that for the support of multiple protocols, only Deflate and Inflate in the above compression and decompression processing flows need to be replaced with the encoding and decoding engines corresponding to the compression standards, and other engines and process steps remain unchanged.

[0153] It should be noted that for the support of extensibility, it needs to be completed in cooperation with the reserved multi-core CPU computing power reserve and the SWEB module. As a hardware module, the SWEB has multiple parallel channels built-in to receive different types of CB_Handles dispatched by the WQS. After receiving the CB_Handle, it notifies a certain processing core of the CPU in the form of a hardware interrupt or a software event. The CPU core reads the CB_Handle from the SWEB and completes the processing of the corresponding task. After completion, it replies "completed" to the WQS. After receiving the completion signal, the WQS continues to send a similar CB_Handle to the corresponding SWEB channel. After the CPU core completes the task, it is also responsible for calculating and preparing the CB_handle of the next sequential task and submitting it to the WQS. Subsequently, after the chip is mass-produced, if there is a new requirement for the support of a compression protocol, it can be supported in this way by software embedding to implement the encoding and decoding functions of the corresponding compression protocol, thus achieving the extensibility of protocol support in the later stage.

[0154] In summary, under the premise of limited cost and complexity, this application supports the hardware offloading of multiple compression standard protocols, facilitating the selection of different compression algorithms according to different application characteristics in the server system design; based on the UAA architecture, it can also provide extended support for subsequent new compression standard formats after the chip is mass-produced.

[0155] Furthermore, the embodiment of this application also provides an electronic device. Figure 11 It is the structural diagram of the electronic device 20 shown according to an exemplary embodiment, and the content in the figure cannot be regarded as any limitation on the scope of use of this application.

[0156] Figure 11Schematic structural diagram of an electronic device 20 provided by an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the compression and decompression methods disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0157] In this embodiment, the power supply 26 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and no specific limitation is imposed on it here; the input / output interface 24 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application needs, and no specific limitation is made here.

[0158] In addition, as a carrier for resource storage, the memory 22 may be a read-only memory, a random access memory, a disk, or an optical disc, etc., and the resources stored thereon may include a computer program 221, and the storage method may be temporary storage or permanent storage. Among them, in addition to the computer program capable of implementing the compression and decompression methods executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 221 may further include a computer program capable of completing other specific tasks.

[0159] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the compression and decompression methods disclosed above are implemented.

[0160] For the specific steps of this method, reference may be made to the corresponding content disclosed in the foregoing embodiments, and no further elaboration will be made here.

[0161] The various embodiments in this application are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts between the various embodiments, reference may be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and reference may be made to the description in the method part for the relevant parts.

[0162] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0163] The steps of the methods or algorithms described in combination with the embodiments disclosed herein can be directly implemented by hardware, software units executed by a processor, or a combination of the two. The software units can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0164] Finally, it should also be noted that in this text, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "comprising a..." does not exclude the existence of additional identical elements in the process, method, article, or device comprising the element.

[0165] The above has introduced in detail a compression and decompression chip, method, device, and storage medium provided by this application. Specific examples are used in this text to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, there will be changes in the specific implementation manner and application scope according to the idea of this application. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A compression and decompression chip, characterized in that, it includes a matching pair search unit for compression preparation and a matching pair recovery unit for decompression preparation separated from the original compression encoding and decompression decoding units, the separated target compression encoding and decompression decoding units for hardware compression and decompression including various protocols, a host interface management unit, a task closing unit, a software-defined engine interface unit for software compression and decompression, and a multi-core central processing unit computing power reserve unit. Among them, the host interface management unit is used to obtain the target command sent by the host; the target command includes a source address; the target command is a decompression command or a compression command; the matching pair search unit is used to perform matching pair search processing on the first source data to obtain a first target processing result; the first source data is the data in the source address corresponding to the compression command; the target compression encoding and decompression decoding unit is used to perform compression encoding on the first target processing result corresponding to the compression command including the first target protocol through the target compression encoding unit corresponding to the first target protocol in the target compression encoding and decompression decoding unit to obtain a first encoding result; the first target protocol is one of several protocols corresponding to the target compression encoding and decompression decoding unit; the software-defined engine interface unit and the multi-core central processing unit computing power reserve unit are used to perform compression encoding on the first target processing result corresponding to the compression command including the second target protocol to obtain a second encoding result; the second target protocol is a protocol that does not exist among all the protocols corresponding to all the target compression encoding units; the target compression encoding and decompression decoding unit is further used to perform compression decoding on the second source data in the source address corresponding to the decompression command including the first target protocol through the target decompression decoding unit corresponding to the first target protocol in the target compression encoding and decompression decoding unit to obtain a first decoding result; the software-defined engine interface unit and the multi-core central processing unit computing power reserve unit are further used to perform compression decoding on the second source data corresponding to the decompression command including the second target protocol to obtain a second decoding result; the matching pair recovery unit is used to perform matching pair recovery processing on the first decoding result or the second decoding result to obtain a second target processing result; the task closing unit is used to transmit the first encoding result or the second encoding result or the second target processing result to the host.

2. The compression and decompression chip according to claim 1, characterized in that, the compression command includes a first target address; the target chip further includes a dynamic memory management unit, a host data direct access unit, and a data cache; the host data direct access unit includes a first access unit and a second access unit. Among them, the dynamic memory management unit is used to reserve a first storage space, a second storage space, and a third storage space in the data cache for the compression command; the first access unit is used to move the first source data to the first storage space; The matching pair search unit is configured to perform a matching pair search process on the first source data in the first storage space to obtain the first target processing result, and store the first target processing result in the second storage space; The target compression encoding and decompression decoding unit is configured to, when the compression command includes the first target protocol, perform compression encoding on the first target processing result in the second storage space to obtain the first encoding result, and store the first encoding result in the third storage unit; The software-defined engine interface unit and the multi-core central processing unit computing power reserve unit are configured to, when the compression command includes the second target protocol, perform compression encoding on the first target processing result in the second storage space to obtain the second encoding result, and store the second encoding result in the third storage unit; The second access unit is configured to move the first encoding result or the second encoding result in the third storage space to the first target address; The dynamic memory management unit is further configured to reclaim the first storage space, the second storage space, and the third storage space; The task termination unit is configured to transmit the first encoding result or the second encoding result in the first target address to the host.

3. The compression and decompression chip according to claim 1, wherein, the decompression command includes a second target address; the target chip further includes a dynamic memory management unit, a host data direct access unit, and a data cache, and the host data direct access unit includes a first access unit and a second access unit, wherein the dynamic memory management unit is configured to reserve a fourth storage space, a fifth storage space, and a sixth storage space in the data cache for the decompression command; the first access unit is configured to move the second source data to the fourth storage space; the target compression encoding and decompression decoding unit is configured to, when the decompression command includes the first target protocol, perform decompression decoding on the second source data in the fourth storage space to obtain a first decoding result, and store the first decoding result in the fifth storage space; the software-defined engine interface unit and the multi-core central processing unit computing power reserve unit are configured to, when the decompression command includes the second target protocol, perform decompression decoding on the second source data in the fourth storage space to obtain a second decoding result, and store the second decoding result in the fifth storage space; The matching pair restoration unit is configured to perform a matching pair restoration process on the first decoding result or the second decoding result in the fifth storage space to obtain a second target processing result, and store the second target processing result in the sixth storage space; The second access unit is configured to move the second target processing result in the sixth storage space to the second target address; The dynamic memory management unit is further configured to reclaim the fourth storage space, the fifth storage space, and the sixth storage space; The task closing unit is used to transmit the second target processing result in the second target address to the host.

4. The compression and decompression chip according to claim 2 or 3, wherein, the target chip further includes a task processing sequence creation unit and a work queue management unit, wherein, the task processing sequence creation unit is used to construct a target control block corresponding to the target command, store the target control block in a pre-allocated control page table, and send a control block handle corresponding to the dynamic memory management unit to the work queue management unit; wherein, the control block handle is used to mark the next unit other than the work queue management unit and the entry address of a part of the control block required by the next unit in the corresponding control page table of the target control block; the part of the control block is a part of the target control block; the work queue management unit is used to send the obtained control block handle sent by the previous unit to the next unit, so that the next unit can complete the local operation task based on the obtained control block handle and generate a new control block handle locally and then send it to the work queue management unit until the task closing unit releases the pre-allocated control page table and the work queue management unit no longer obtains a new control block handle corresponding to the target command, so as to realize the sequential operation of each target unit in the target unit group; wherein, the target units arranged in the target unit group according to the running order include the dynamic memory management unit, the first access unit, the matching pair search unit, the target compression encoding and decompression unit or the combination unit, the second access unit, the dynamic memory management unit and the task closing unit; the combination unit is the software-defined engine interface unit and the multi-core central processing unit computing power reserve unit; or, the target units arranged in the target unit group according to the running order include the dynamic memory management unit, the first access unit, the target compression encoding and decompression unit or the combination unit, the matching pair recovery unit, the second access unit, the dynamic memory management unit and the task closing unit.

5. The compression and decompression chip according to claim 4, wherein, the work queue management unit is further used to configure a target work queue for storing control block handles for each target unit in the target unit group to sequentially store the control block handles of different target commands; the target work queue stores a control block handle marking the next unit as this target unit.

6. The compression and decompression chip according to claim 4, wherein, different target units in the target unit group can concurrently complete their own tasks based on the control block handles of different target commands.

7. The compression and decompression chip according to claim 1, wherein, The matching pair searching unit is specifically configured to perform matching pair searching processing on the first source data to obtain a first initial processing result, and encapsulate the first initial processing result based on the target byte format to obtain a first target processing result; The matching pair restoring unit is specifically configured to perform matching pair restoring processing on the first decoding result or the second decoding result to obtain a second initial processing result, and encapsulate the second initial processing result based on the target byte format to obtain a second target processing result; Correspondingly, the target byte format includes an original text length byte, a matching length byte, an original text length reserve byte, a matching offset byte, a matching length reserve byte, and an original text byte; the sum of the contents in the original text length byte and the original text length reserve byte is the original text length; the sum of the contents in the matching length byte and the matching length reserve byte is the matching length; the original text length byte includes an original text base length byte and a matching base length byte.

8. A compression and decompression method characterized in that it is applied to a compression and decompression chip, the compression and decompression chip includes a matching pair searching unit for compression preparation and a matching pair restoring unit for decompression preparation separated from the original compression encoding and compression decoding units, a target compression encoding and compression decoding unit for hardware compression and decompression containing various protocols after separation, a host interface management unit, a task finalization unit, a software-defined engine interface unit for software compression and decompression, and a multi-core central processor computing power reserve unit, and the method includes: obtaining a target command sent by the host through the host interface management unit; the target command includes a source address; the target command is a decompression command or a compression command; if the target command is the compression command, then through the matching pair searching unit, perform matching pair searching processing on the first source data to obtain a first target processing result, and through the target compression encoding unit corresponding to the first target protocol in the target compression encoding and compression decoding unit, perform compression encoding on the first target processing result corresponding to the compression command containing the first target protocol to obtain a first encoding result, or, and through the software-defined engine interface unit and the multi-core central processor computing power reserve unit, perform compression encoding on the first target processing result corresponding to the compression command containing the second target protocol to obtain a second encoding result; the first source data is the data in the source address corresponding to the compression command; the first target protocol is one of several protocols corresponding to the target compression encoding and compression decoding unit; the second target protocol is a protocol that does not exist in all the protocols corresponding to all the target compression encoding units If the target command is the compression command, the second source data in the source address corresponding to the decompression command including the first target protocol is decompressed and decoded by the target decompression decoding unit corresponding to the first target protocol in the target compression encoding and decompression decoding unit to obtain a first decoding result, or the second source data corresponding to the decompression command including the second target protocol is decompressed and decoded by the software-defined engine interface unit and the multi-core central processing unit computing power reserve unit to obtain a second decoding result, and the matching pair restoration unit performs matching pair restoration processing on the first decoding result or the second decoding result to obtain a second target processing result; The first encoding result or the second encoding result or the second target processing result is transmitted to the host through the task ending unit.

9. An electronic device, characterized in that, comprising: a memory for storing a computer program; a processor for executing the computer program to implement the compression and decompression method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, for storing a computer program; wherein, when the computer program is executed by a processor, the compression and decompression method according to any one of claims 1 to 6 is implemented.

Images