A high-performance memory compression system based on QAT hardware accelerator

Abstract

This invention discloses a high-performance memory compression system based on the QAT (QuickAssist Technology) hardware accelerator, relating to the field of memory compression. A new page-based parallel read / write framework is added to the ZRAM memory compression module, enabling ZRAM to better utilize the multi-core characteristics of the CPU and avoiding the waste of significant computer resources due to prolonged read / write blocking. The compression and decompression functions of ZRAM are offloaded to the QAT hardware, reducing CPU resource usage during compression algorithm execution, freeing up substantial computer resources, and minimizing the performance impact of the compression process on upper-layer applications.

Description

Technical Field

[0001] This invention relates to the field of memory compression, and more particularly to a high-performance memory compression system based on the QAT hardware accelerator. Background Technology

[0002] With the rapid development of high-performance computing, the demand for memory has grown exponentially, especially in scientific and technological applications, database applications, and artificial intelligence applications. The growth in application memory resource requirements is unlimited; each application wants to consume as much memory as possible to improve its performance, making memory resources perpetually insufficient. In this context, memory overflow has become a common type of program error. For example, in a Cassandra cluster consisting of 10 machines, each machine might experience a memory overflow approximately once every 10 days. Relying on Cassandra data replication and scheduled tasks to start guard scripts, a brief failure of one machine's process does not lead to business losses. Another example is iOS, which uses a management mechanism called Jetsam to control excessive memory usage. This is an independently running process, and each process has a memory threshold. Once this threshold is exceeded, Jetsam immediately kills the process, causing a crash. Even in the field of artificial intelligence, OOM (Out of Memory) can occur. Engineers have found that adding the Lamb optimizer to BERT training can greatly shorten the training time by increasing the batch size, but this will lead to a sharp increase in memory consumption, which can easily cause the system to OOM. In actual tests, when the batch size was increased to 1600 on an x86 server, 240G of memory was completely exhausted, and the training system was interrupted due to OOM.

[0003] ZRAM memory compression technology is a crucial feature in the Linux kernel, primarily used to provide virtual memory compression. ZRAM improves performance by paging compressed memory blocks within RAM until swap space on the hard drive is used, thus avoiding on-disk paging.

[0004] In the existing framework, the Linux kernel's ZRAM high-performance memory module uses a single-threaded blocking read / write framework and a purely software-implemented compression algorithm. The existing ZRAM high-performance memory module has three problems:

[0005] 1. ZRAM commonly uses compression algorithms with relatively low compression efficiency. ZRAM defaults to using LZx-based compression algorithms such as LZO and LZ4 during compression and decompression. While LZx-based compression algorithms offer relatively fast compression speeds, they are weak in terms of compression ratio, only reaching 60%.

[0006] 2. The compression module consumes a significant amount of CPU resources. The compression algorithm used by conventional ZRAM is implemented entirely in software, and when the compression module is performing compression, the CPU utilization of a single CPU exceeds 60%.

[0007] 3. It cannot take advantage of the multi-physical core feature of current servers. ZRAM driver read and write is single-threaded blocking read and write, which causes (1) frequent long-term blocking in I / O, resulting in a large number of CPU cycles being used for waiting and (2) low utilization of parallel computing units in the accelerator, because neither CPU nor accelerator resources are fully utilized.

[0008] Therefore, those skilled in the art are dedicated to developing a high-performance memory compression system based on the QAT hardware accelerator. This involves adding a new page-based parallel read / write framework to ZRAM memory compression technology, enabling ZRAM to better utilize the multi-core characteristics of the CPU and avoiding the waste of significant computer resources due to prolonged read / write blocking. Furthermore, offloading the compression / decompression functions of ZRAM to the QAT hardware reduces CPU resource consumption during compression algorithm execution, freeing up substantial computer resources and minimizing the impact on upper-layer applications.

[0009] QAT hardware-assisted compression technology is a modern compression acceleration solution based on PCIe 3.0, proposed by Intel. It boasts excellent compression performance and has wide applications in cloud, networking, big data, and storage applications. The QAT compression algorithm provided by QAT hardware-assisted technology is a hardware compression algorithm that, compared to software compression algorithms such as Lempel-Ziv and Zip, combines high compression ratios with high compression / decompression speeds. Furthermore, because QAT hardware-assisted technology is essentially a hardware accelerator using specialized integrated circuits, its CPU usage is also very low. Summary of the Invention

[0010] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is that commonly used compression algorithms have low compression efficiency, the data compression process consumes a lot of CPU computer resources, and cannot make use of the multi-core characteristics of current servers.

[0011] To achieve the above objectives, the present invention provides a high-performance memory compression system QZRAM based on the QAT hardware accelerator, characterized in that it includes a ZRAM page-based parallel read / write module.

[0012] Furthermore, it includes a compression module based on QAT hardware coprocessing.

[0013] Furthermore, the parallel read / write module replaces the original single-threaded blocking read / write of ZRAM.

[0014] Furthermore, the parallel read / write module divides the ZRAM read / write execution process into two stages: a preprocessing stage and a parallel read / write stage.

[0015] Furthermore, the preprocessing stage accepts bio blocks issued by the upper-layer application, then parses the bio blocks page by page and encapsulates them in page_list, and finally adds page_list to the first-in-first-out queue req_queue.

[0016] Furthermore, in the parallel read / write phase, the worker thread retrieves the requests from the req_queue, calls the read / write API to continue the read / write process, and returns to the top layer by calling bio_endio after completion.

[0017] Furthermore, when the ZRAM kernel module is loaded into the kernel, the compression module generates the same number of compressed streams as the CPU physical cores, and calls the QAT driver API to create as many sessions and instances as possible.

[0018] Furthermore, the parallel read / write module calls the compression module. The compression module first obtains the compressed stream on the currently running CPU, then organizes the original data into an SGL format acceptable to the QAT device, adds a ZLIB header and footer to the original data, and finally sends the uncompressed data to the QAT hardware via DMA and enters sleep mode.

[0019] Furthermore, the QAT device compresses the data and writes it to the destination address via DMA. Then, it notifies the compression module via an interrupt, and the compression module returns the compressed data to the parallel read / write layer.

[0020] Furthermore, the compression module employs the GZIP algorithm.

[0021] In a preferred embodiment of the present invention, this project is based on ZRAM. ZRAM memory compression technology is a very important functional module in the Linux kernel, and ZRAM is mainly used to provide virtual memory compression. ZRAM improves performance by paging on compressed memory block devices (RAM DISK) within RAM until swap space on the hard disk must be used, thus avoiding paging on the disk.

[0022] However, conventional ZRAM high-performance memory compression technology has three problems:

[0023] Problem 1: ZRAM commonly uses compression algorithms with relatively low compression efficiency. ZRAM defaults to using LZx-based compression algorithms such as LZO and LZ4 for compression and decompression. While LZx-based compression algorithms have fast compression speeds, they are weak in compression ratio, only reaching 60%.

[0024] Question 2: The compression module consumes a large amount of CPU resources. The compression algorithm used by conventional ZRAM is implemented purely in software, and when the compression module performs compression, the CPU utilization of a single CPU exceeds 60%.

[0025] The third problem is that it cannot take advantage of the multi-physical-core characteristics of current servers. ZRAM driver read and write operations are single-threaded blocking read and write operations, causing frequent long-term blocking in I / O, resulting in a large number of CPU cycles being spent waiting and low utilization of parallel computing units within the accelerator. As a result, neither CPU nor accelerator resources are fully utilized.

[0026] First, to address issues one and two mentioned above, QAT hardware-assisted compression technology was introduced. QAT hardware-assisted compression technology is a modern ASIC-based compression acceleration solution with excellent compression performance and wide applications in cloud, networking, big data, and storage applications. The QAT compression algorithm provided by QAT hardware-assisted compression technology is a hardware compression algorithm. Compared with software compression algorithms such as Lempel-Ziv and Zip, it combines high compression ratio with high compression / decompression speed. Furthermore, since QAT hardware-assisted compression technology is essentially a hardware accelerator based on a special integrated circuit, its CPU usage is also very low. Therefore, replacing the original purely software-implemented LZx-based compression algorithm with the QAT hardware-assisted GZIP algorithm increases the compression ratio while reducing single-core CPU usage.

[0027] Second, to address issue three, a page-based parallel read / write framework was added to ZRAM. After upper-layer I / O scheduling, the application layer passes the request to the ZRAM request processing layer. Here, the request is broken down into page-level granularity and then passed to the read / write page layer for specific read / write operations. The request processing layer first determines whether the I / O request is within the range of a ZRAM logical block, and then selects to perform a write or read operation based on the request's transmission direction. The read / write page layer performs the specific read / write operations on the physical page, during which it calls the compression library layer to compress the original data. Then, based on the actual size of the compressed data, it requests and allocates a block of memory and writes the compressed data into this memory.

[0028] Compared with the prior art, the present invention has the following obvious substantive features and significant advantages:

[0029] 1. Increased compression ratio reduces single-core CPU physical resource usage. Computer-intensive projects like ZRAM high-performance memory compression should allocate more processor performance to user-space application execution. Purely software-implemented algorithms like LZ4 and LZO consume significant processor resources, leading to a substantial performance degradation in upper-layer applications. This invention offloads the compression module to the QAT coprocessor, reducing required processor resources and improving overall system performance. More directly, ZRAM can replace the compression algorithm with the GZIP algorithm without significantly impacting performance. ZRAM improves system performance while increasing the memory compression ratio by 30%.

[0030] 2. Improve scalability. When computer physical memory is very limited, using a single processor to run a task distribution module can lead to excessive load and waste of other CPU resources. Using multiple processors for parallel read and write operations significantly improves ZRAM throughput, frees up memory resources more quickly, and creates new memory space for upper-layer applications, preventing them from experiencing OutOfMemory errors (OOM) or significant performance degradation.

[0031] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0032] Figure 1 This is a ZRAM overall architecture diagram of a preferred embodiment of the present invention;

[0033] Figure 2 This is a flowchart of the ZRAM parallel read / write execution process according to a preferred embodiment of the present invention;

[0034] Figure 3 This is a flowchart illustrating the ZRAM compression operator offloading to the QAT coprocessor according to a preferred embodiment of the present invention. Detailed Implementation

[0035] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0036] In the accompanying drawings, components with the same structure are indicated by the same numerical designation, and components with similar structures or functions are indicated by similar numerical designations. The dimensions and thicknesses of each component shown in the drawings are arbitrary, and the present invention does not limit the dimensions and thicknesses of each component. To make the illustrations clearer, the thickness of some components has been appropriately exaggerated in the drawings.

[0037] This invention provides a high-performance memory compression system based on the QAT hardware accelerator. Figure 2 This is a flowchart of the ZRAM parallel read / write execution process; Figure 3 It is a data shaping module for ZRAM compression.

[0038] First, a new page-based parallel read / write framework is added to ZRAM memory compression technology, enabling ZRAM to better utilize the characteristics of multiple physical cores of the CPU and avoid wasting a lot of computer resources due to long-term read / write blocking.

[0039] Second, by offloading the compression / decompression functions of ZRAM to the QAT hardware, the CPU resource usage during compression algorithm execution is reduced, freeing up a large amount of computer resources and reducing the impact on upper-layer applications.

[0040] The invention consists of two parts: a ZRAM-based page-based parallel read / write module and a compression module based on QAT hardware coprocessing.

[0041] The first part is the ZRAM page-based parallel read / write module. The parallel read / write module is designed to replace ZRAM's original single-threaded blocking read / write scheme.

[0042] The parallel read / write module divides the ZRAM read / write execution process into two stages: a preprocessing stage and a parallel read / write stage. The preprocessing stage receives BIO blocks from the upper-layer application, parses each BIO block page by page and encapsulates them in a `page_list`, and finally adds the `page_list` to the first-in-first-out queue `req_queue`. In the parallel read / write stage, numerous worker threads retrieve requests from `req_queue`, call the read / write API to continue executing the latter half of the read / write process, and return to the uppermost layer by calling `bio_endio` after completion.

[0043] The second part is a compression module based on the QAT hardware coprocessor. When the ZRAM kernel module is loaded into the kernel, the compression module generates a compressed stream equal to the number of physical CPU cores and calls the QAT driver API to create as many sessions and instances as possible. When the parallel read / write module calls the compression module, the compression module first obtains the compressed stream from the currently running CPU, then reorganizes the original data into an SGL format acceptable to the QAT device. To ensure compatibility with the purely software-implemented GZIP algorithm, it adds ZLIB headers and trailers to the original data. Finally, it sends the uncompressed data to the QAT hardware via DMA and enters sleep mode. Once the QAT device has finished compressing the data and written it to the destination address via DMA, it notifies the compression module via an interrupt. The compression module then returns the compressed data to the parallel read / write layer.

[0044] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A high performance memory compression system based on QAT hardware accelerator, characterized in that, The parallel read-write module based on physical pages of ZRAM; The compression module based on QAT hardware coprocessor; The parallel read-write module replaces the original single-thread blocking read-write of ZRAM; The parallel read-write module divides the ZRAM read-write execution flow into two stages: the preprocessing stage and the parallel read-write stage; The preprocessing stage accepts the bio block issued by the upper layer application, then parses the bio block page by page and encapsulates it in page_list, and finally adds page_list to the first-in-first-out queue req_queue; The parallel read-write stage, the worker thread gets the request in req_queue to call the read-write API to continue the read-write flow and returns to the top layer by calling bio_endio after completion; The compression module generates the same number of compression streams as the CPU physical cores when the ZRAM kernel module is loaded into the kernel, and calls the QAT driver API to create a session and an instance; The parallel read-write module calls the compression module, which first gets the compression stream on the current running CPU, then arranges the original data into the SGL format that the QAT device can accept, adds ZLIB headers and trailers to the original data, and finally sends the uncompressed data to the QAT hardware through DMA and enters sleep; The QAT device writes the compressed data to the destination address through DMA, and then notifies the compression module through interruption, which returns the compressed data to the parallel read-write layer.

2. The high performance memory compression system based on QAT hardware accelerator of claim 1, wherein, The compression module uses the GZIP algorithm.

Images