Compression accelerator card, method for accessing compression accelerator card in analog hard disk mode
By accessing the compression acceleration card through a simulated hard drive based on data intelligent processor technology, the problem of high CPU resource consumption is solved, achieving a low-cost, high-efficiency data transmission and storage solution.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INNOGRIT TECH CO LTD
- Filing Date
- 2023-02-01
- Publication Date
- 2026-04-21
AI Technical Summary
In existing technologies, CPU compression algorithms consume a large amount of CPU resources, resulting in high costs for enterprises, and directly using PCIe compression accelerator cards requires the development of a large amount of supporting software.
The compression accelerator card is accessed using a simulated hard drive based on data intelligent processor technology. By setting compression and decompression identifiers, data compression and decompression are performed using a virtual compressed hard drive. The host system and applications only need limited software modifications to access the compression accelerator card.
It reduces CPU utilization, lowers enterprise equipment costs, and provides superior data transfer performance and storage utilization while reducing network load.
Smart Images
Figure CN116089386B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data compression technology, and in particular to a method for accessing a compression accelerator card in a manner that simulates a hard disk based on data intelligent processor technology, as well as the compression accelerator card itself. Background Technology
[0002] Currently, in the market, most applications involving big data compression and transmission, such as data centers, enterprise private clouds, gaming platforms, and storage servers, primarily rely on CPU compression algorithms. CPU compression consumes considerable CPU resources, which are relatively expensive, and excessive CPU usage hinders rapid market growth for businesses. Therefore, hardware-accelerated PCIe compression accelerator cards have emerged to address this need, significantly reducing CPU usage while maintaining compression performance. However, directly using PCIe compression accelerator cards requires users to develop substantial amounts of accompanying software to access and compress the PCIe accelerator card. Summary of the Invention
[0003] The purpose of this application is to provide a method for accessing a compression acceleration card using a simulated hard drive based on data intelligent processor technology, thereby reducing CPU utilization and minimizing the software modifications required for customers migrating from CPU compression to hardware compression, thus reducing enterprise equipment costs.
[0004] This application discloses a method for accessing a compression acceleration card in a manner similar to a hard disk based on data intelligent processor technology, including:
[0005] The application sets a compression identifier and sends a write command to the virtual compressed hard disk, which is then sent to the compression accelerator card via the virtual compressed hard disk;
[0006] The compression acceleration card compresses the data according to the compression identifier, caches the compressed data in the compression acceleration card, and returns a message to the application.
[0007] The application sets a decompression identifier and sends a read command to the virtual compressed hard disk. The read command is sent to the compression accelerator card via the virtual compressed hard disk. The logical address corresponding to the read command is the same as that corresponding to the write command.
[0008] The compression acceleration card does not decompress the compressed data according to the decompression identifier, and the virtual compressed hard disk returns the compressed data to the application.
[0009] In a preferred embodiment, the compression identifier is added to the write command and sent to the virtual compressed hard disk, and the decompression identifier is added to the read command and sent to the virtual compressed hard disk.
[0010] In a preferred embodiment, the compression identifier and the decompression identifier are set in the register of the compression accelerator card.
[0011] In a preferred embodiment, the hard drive driver is an NVMe driver.
[0012] In a preferred embodiment, before the step of the application setting a compression identifier and sending a write command to the virtual compressed hard disk, the method further includes:
[0013] Insert the compression accelerator card into the host;
[0014] The host computer loads the hard drive driver and application, and successfully detects the virtual compressed hard drive.
[0015] In a preferred embodiment, the logical addresses of all input / output interfaces of the compression accelerator card are based on 32K alignment.
[0016] This application also discloses a method for accessing a compression accelerator card based on data intelligent processor technology in a simulated hard disk manner, including:
[0017] The application sets a compression identifier and sends a write command to the virtual compressed hard disk, which is then sent to the compression accelerator card via the virtual compressed hard disk.
[0018] The compression acceleration card does not compress data according to the compression identifier, caches previously compressed data in the compression acceleration card, and returns a message to the application;
[0019] The application sets a decompression identifier and sends a read command to the virtual compressed hard disk. The read command is then sent to the compression accelerator card via the virtual compressed hard disk. The logical address of the read command is the same as that of the write command.
[0020] The compression accelerator card decompresses the previously compressed data according to the decompression identifier and returns the decompressed data to the application.
[0021] In a preferred embodiment, the compression identifier is added to the write command and sent to the virtual compressed hard disk, and the decompression identifier is added to the read command and sent to the virtual compressed hard disk.
[0022] In a preferred embodiment, the compression identifier and the decompression identifier are set in the register of the compression accelerator card.
[0023] In a preferred embodiment, the hard drive driver is an NVMe driver.
[0024] In a preferred embodiment, before the step of the application setting a compression identifier and sending a write command to the virtual compressed hard disk, the method further includes:
[0025] Insert the compression accelerator card into the host;
[0026] The host computer loads the hard drive driver and application, and successfully detects the virtual compressed hard drive.
[0027] In a preferred embodiment, the logical addresses of all input / output interfaces of the compression accelerator card are based on 32K alignment.
[0028] This application also discloses a compression accelerator card, including:
[0029] When a compression acceleration card is inserted into the host and queried by the host, the compression acceleration card is detected as a virtual compressed hard disk, and the hard disk protocol interface module reports the hard disk attributes.
[0030] A storage module is used to store compressed or decompressed data, and the hard disk protocol interface module reports the capacity of the storage module as the hard disk capacity; and
[0031] A compression engine, which stores the mapping relationship between logical block addresses on the host side and physical block addresses on the virtual compressed hard disk side.
[0032] In a preferred embodiment, the hard disk attributes include hard disk block size, the method of accessing the virtual hard disk, and the logical block size supported by the input / output interface.
[0033] In this application, the PCIe compression acceleration card solution based on the DIPU simulated hard disk compression method aims to provide a hardware-accelerated compression card solution that reduces CPU utilization and enterprise equipment costs for various scenarios requiring compressed data transmission, while minimizing the software modifications required for customers migrating from CPU compression to hardware compression.
[0034] The specification of this application contains numerous technical features distributed across various technical solutions. Listing all possible combinations of these technical features (i.e., technical solutions) would make the specification excessively lengthy. To avoid this problem, the various technical features disclosed in the above-described invention, the various technical features disclosed in the following embodiments and examples, and the various technical features disclosed in the accompanying drawings can be freely combined to form various new technical solutions (all of which should be considered as described in this specification), unless such a combination of technical features is technically infeasible. For example, one example discloses feature A+B+C, and another example discloses feature A+B+D+E. Features C and D are equivalent technical means that serve the same function, and technically only one needs to be used; they cannot be used simultaneously. Feature E can technically be combined with feature C. Therefore, the solution A+B+C+D should not be considered as described because it is technically infeasible, while the solution A+B+C+E should be considered as described. Attached Figure Description
[0035] Figure 1 This is a schematic flowchart of a method for accessing a compression accelerator card based on data intelligent processor technology in a simulated hard disk manner according to one embodiment of this application.
[0036] Figure 2 This is a schematic flowchart of a method for accessing a compression accelerator card based on data intelligent processor technology in a simulated hard disk manner, according to another embodiment of this application.
[0037] Figure 3 This is a diagram illustrating the process of accessing a compression accelerator card in a simulated hard disk manner to compress data according to one embodiment of this application.
[0038] Figure 4 This is a diagram illustrating the process of accessing a compression accelerator card in a simulated hard disk manner to decompress data according to one embodiment of this application.
[0039] Figure 5 and Figure 6 This is a mapping relationship between logical addresses and physical addresses at different times according to one embodiment of this application. Detailed Implementation
[0040] In the following description, many technical details are presented to help the reader better understand this application. However, those skilled in the art will understand that the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments.
[0041] Explanation of some concepts:
[0042] DIPU (Data Intelligently-Processing Unit): This technology primarily builds upon traditional hard drive controller design by introducing on-disk intelligent data processing, such as encryption, authorized access, compression, and hot / cold data analysis. When a user stores or accesses hard drive data, the data is intelligently processed according to the application before being used by the user.
[0043] PCIe (Peripheral Component Interconnect Express): A high-speed serial computer expansion bus standard.
[0044] Virtual compressed hard disk: The host simulates the compression accelerator card as a compressed hard disk and accesses the compressed hard disk in the same way as accessing a hard disk.
[0045] The following is a brief summary of some of the innovative aspects of the embodiments of this application:
[0046] This application provides a hardware-accelerated compression card solution that reduces CPU utilization and enterprise equipment costs—a PCIe compression accelerator card solution based on a DIPU-simulated compressed hard drive design. This solution simulates a DIPU compressed hard drive with the PCIe compression accelerator card. When the host detects this PCIe compression accelerator card, the host system and software automatically recognize it as a DIPU compressed hard drive. The host system and applications only need to make very limited software modifications to access this DIPU compressed hard drive to access the compression accelerator card as if it were a normal hard drive. Leveraging the superior performance provided by the accelerator card, it helps customers compress data before data transmission, reducing CPU load. If compressed data needs to be transmitted between network and terminal storage devices, this solution has excellent compatibility and adaptability, closely resembling a storage hard drive, and can also significantly reduce network load and increase storage device utilization.
[0047] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0048] The first embodiment of this application relates to a method for accessing a compression accelerator card in a manner that simulates a hard disk based on data intelligent processor technology. The method includes the following steps:
[0049] Step 1: Insert the compression accelerator card into the host computer.
[0050] Step 2: The host loads the hard disk driver and application program. It should be noted that in traditional CPU compression algorithms, input and output buffers (on the host side) are set in the PCIe commands. The accelerator card obtains data from the input buffer, compresses it, and then places it in the output buffer, which requires significant modifications to the upper-layer software. However, the driver loaded in this application is a hard disk driver, not a PCIe adapter driver. For example, in one embodiment, the hard disk driver can be an NVMe driver.
[0051] Step 3: The host successfully detected the virtual compressed hard drive. The host system and applications require only very limited software modifications to access this virtual compressed hard drive, allowing them to access the compression accelerator card as if it were a normal hard drive. Leveraging the superior performance of the accelerator card, customers can compress data before data transfer, thus reducing CPU load.
[0052] Step 4: The host-side application can access this virtual compressed hard disk to compress host-side data. The compression operation process is as follows: Figure 1 As shown, the specific steps include the following:
[0053] Step 101: The application sets a compression identifier and sends a write command to the virtual compressed hard disk. The write command is sent to the compression accelerator card via the virtual compressed hard disk. The compression identifier indicates whether the data is compressed or uncompressed. In this embodiment, the compression identifier is set to compressed data.
[0054] In one embodiment, the compression identifier is added to the write command and sent to the virtual compressed hard disk. In another embodiment, the compression identifier and the decompression identifier are set in the registers of the compression accelerator card.
[0055] Step 102: The compression acceleration card compresses the data according to the compression identifier and caches the compressed data in the compression acceleration card, and returns a message to the hard disk driver and the application.
[0056] Step 103: The application sets a decompression identifier and sends a read command to the virtual compressed hard disk. The read command is sent to the compression acceleration card via the virtual compressed hard disk. The logical address corresponding to the read command is the same as that corresponding to the write command. The decompression identifier indicates whether the data is decompressed or not. In this embodiment, the decompression identifier is for data that is not decompressed.
[0057] In one embodiment, the decompression identifier is added to the read command and sent to the virtual compressed hard disk. In another embodiment, the decompression identifier is set in a register of the compression accelerator card.
[0058] In step 104, the compression acceleration card does not decompress the compressed data according to the decompression identifier, and the virtual compressed hard disk returns the compressed data to the application.
[0059] Step 5: The host-side application can access this virtual compressed hard disk to decompress the host-side data. The decompression process is as follows: Figure 2 As shown, the specific steps include the following:
[0060] Step 201: The application sets a compression identifier and sends a write command to the virtual compressed hard disk. The write command is then sent to the compression acceleration card via the virtual compressed hard disk. In this embodiment, the compression identifier is set to uncompressed data.
[0061] Step 202: The compression acceleration card does not compress the data according to the compression identifier and caches the previously compressed data in the compression acceleration card, and returns a message to the application.
[0062] Step 203: The application sets a decompression identifier and sends a read command to the virtual compressed hard disk. The read command is sent to the compression acceleration card via the virtual compressed hard disk. The logical address of the read command is the same as that of the write command. In this embodiment, the decompression identifier is the decompressed data.
[0063] Step 204: The compression acceleration card decompresses the previously compressed data according to the decompression identifier and returns the decompressed data to the application.
[0064] In one embodiment, the logical addresses of all input / output interfaces of the compression accelerator card are aligned to 32K, which can achieve a better compression ratio.
[0065] To better understand the technical solution of this application, a specific example is provided below. The details listed in this example are mainly for ease of understanding and are not intended to limit the scope of protection of this application.
[0066] The PCIe compression accelerator card, based on the DIPU simulated compressed hard drive design, can be selected from PCIe Gen4 and Gen5 interfaces. The PCIe compression accelerator card is connected to the host through the PCIe slot. The host loads the device driver and application, and successfully detects a virtual DIPU compressed hard drive. The host application can access this hard drive to compress the data on the host. Figure 3 The diagram illustrates the specific process of accessing a compression accelerator card and performing compression operations based on a data intelligent processor technology that simulates a hard drive in one embodiment.
[0067] Step 1: The application sets a compression identifier, which is for data compression, and sends a write command to the DIPU hard drive.
[0068] Step 2: The command is sent to the hardware acceleration card via the DIPU hard drive driver.
[0069] Step 3: After receiving the data, the compression accelerator card compresses the data according to the identifier and temporarily stores the data on the compression accelerator card.
[0070] Step 4: After the compression and acceleration card finishes processing the data, it sends a successful return value back to the hard drive driver and application.
[0071] Step 5: The application receives the return value, indicating successful data compression. The application configures a decompression identifier, which means the data will not be decompressed. It then sends a read command with the LBA set to the LBA of the corresponding write command in step 1, and reads the data.
[0072] Step 6: The command is sent to the hardware acceleration card via the DIPU hard drive driver.
[0073] Step 7: The compression accelerator card receives the command and does not decompress the data according to the identifier.
[0074] Step 8: Send the compressed data back to the application.
[0075] Step 9: The application receives the compressed data from the hardware acceleration card and decides how to use the compressed data.
[0076] Figure 4 The diagram illustrates the specific process of accessing a compression accelerator card for decompression operations based on a data intelligent processor technology that simulates a hard drive in one embodiment.
[0077] Step 1: The application sets a compression identifier to uncompressed data, indicating that the input data is already compressed, and sends a write command to the DIPU hard drive.
[0078] Step 2: The command is sent to the hardware acceleration card via the DIPU hard drive driver.
[0079] Step 3: After receiving the data, the compression accelerator card does not compress the data according to the identifier, but directly stores the data on the compression accelerator card.
[0080] Step 4: After the data is temporarily stored, send a successful return value of the command back to the hard drive and application.
[0081] Step 5: The application receives the return value, configures the decompression identifier (which represents the decompressed data), sends a read command with the LBA set to the LBA of the corresponding write command in step 1, and reads the data.
[0082] Step 6: The command is sent to the hardware acceleration card via the DIPU hard drive driver.
[0083] Step 7: The compression accelerator card receives the command and decompresses the data according to the identifier.
[0084] Step 8: Send the decompressed data back to the application.
[0085] Step 9: The application receives the decompressed data from the hardware acceleration card and decides how to use the decompressed data.
[0086] The second embodiment of this application relates to a compression accelerator card, which includes a hard disk protocol interface module, a storage module, and a compression engine. When the compression accelerator card is inserted into a host and queried by the host, the compression accelerator card is detected as a virtual compressed hard disk, and the hard disk protocol interface module reports hard disk attributes. In one embodiment, the hard disk attributes include the hard disk block size, the method of accessing the virtual hard disk, and the logical block size supported by the input / output interface. The storage module is used to store compressed or decompressed data. The storage module can be DDR, SRAM, or FLASH. The hard disk protocol interface module reports the capacity of the storage module as the hard disk capacity. The compression engine stores the mapping relationship between the logical block addresses on the host side and the physical block addresses on the virtual compressed hard disk side. Figure 5 This application illustrates a mapping relationship between logical addresses and physical addresses in one embodiment. Figure 6 This shows the mapping relationship between logical addresses and physical addresses after a period of time.
[0087] The first embodiment is a method embodiment corresponding to this embodiment. The technical details in the first embodiment can be applied to this embodiment, and the technical details in this embodiment can also be applied to the first embodiment.
[0088] It should be noted that in this patent application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this patent application, if it refers to performing an action according to an element, it means performing the action at least according to that element, including two cases: performing the action only according to that element, and performing the action according to that element and other elements. Expressions such as "multiple," "repeatedly," and "various" include two, two times, two kinds, and more than two, more than two times, and more than two kinds.
[0089] All references to this specification are considered to be incorporated integrally into the disclosure of this application so that they can serve as the basis for modifications if necessary. Furthermore, it should be understood that the above descriptions are merely preferred embodiments of this specification and are not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the scope of protection of one or more embodiments of this specification.
[0090] In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
Claims
1. A method for accessing a compression accelerator card in a manner analogous to a hard disk based on data intelligent processor technology, characterized by, include: Insert the compression accelerator card into the host; The host loads the hard disk driver and application instead of the PCIe adapter driver, and the host successfully recognizes the compression accelerator card as a virtual compressed hard disk, allowing the host to access the compression accelerator card as a normal hard disk. The application sets a compression identifier and adds it to the write command, and sends the write command to the virtual compressed hard disk, which then sends the write command to the compression acceleration card. The compression acceleration card compresses the data according to the compression identifier, caches the compressed data in the compression acceleration card, and returns a message to the application. The application sets a decompression identifier and adds it to the read command, then sends the read command to the virtual compressed hard disk. The read command is sent to the compression accelerator card via the virtual compressed hard disk. The logical address corresponding to the read command is the same as that corresponding to the write command. The compression identifier and the decompression identifier are set in the registers of the compression accelerator card. The compression acceleration card does not decompress the compressed data according to the decompression identifier, and the virtual compressed hard disk returns the compressed data to the application.
2. The method for accessing a compression accelerator card in a simulated hard disk mode as recited in claim 1, wherein, The hard drive driver is an NVMe driver.
3. The method for accessing a compression accelerator card in a simulated hard disk mode as recited in claim 1, wherein, The logical addresses of all input / output interfaces of the compression accelerator card are based on 32K alignment.
4. A method for accessing a compression accelerator card in a manner analogous to a hard disk based on data intelligent processor technology, characterized by, include: Insert the compression accelerator card into the host; The host loads the hard disk driver and application instead of the PCIe adapter driver, and the host successfully recognizes the compression accelerator card as a virtual compressed hard disk, allowing the host to access the compression accelerator card as a normal hard disk. The application sets a compression identifier and adds it to the write command, and sends the write command to the virtual compressed hard disk, which then sends the write command to the compression acceleration card. The compression acceleration card does not compress data according to the compression identifier, caches previously compressed data in the compression acceleration card, and returns a message to the application; The application sets a decompression identifier, adds it to the read command, and sends the read command to the virtual compressed hard disk. The read command is then sent to the compression accelerator card via the virtual compressed hard disk. The read command has the same logical address as the write command, and the compression identifier and the decompression identifier are set in the registers of the compression accelerator card. The compression accelerator card decompresses the previously compressed data according to the decompression identifier and returns the decompressed data to the application.
5. The method for accessing a compression accelerator card in a simulated hard disk mode as recited in claim 4, wherein, The hard drive driver is an NVMe driver.
6. The method for accessing a compression accelerator card in a simulated hard disk manner as described in claim 4, characterized in that, The logical addresses of all input / output interfaces of the compression accelerator card are based on 32K alignment.
7. A compression accelerator card, characterized by, include: When a compression acceleration card is inserted into a host and queried by the host, the compression acceleration card is detected as a virtual compressed hard disk. The host accesses the compression acceleration card using the method described in any one of claims 1 to 6. The hard disk protocol interface module reports the hard disk attributes. The storage module is used to store compressed or decompressed data, and the hard disk protocol interface module reports the capacity of the storage module as the hard disk capacity. and A compression engine, which stores the mapping relationship between logical block addresses on the host side and physical block addresses on the virtual compressed hard disk side.
8. The compression accelerator card of claim 7, wherein, The hard disk attributes include the hard disk block size, the method of accessing the virtual hard disk, and the logical block size supported by the input / output interface.