Data processing method and device based on hardware accelerator zero copy, virtual electronic equipment and storage medium

Abstract

The invention discloses a data processing method and device based on zero copy of a hardware accelerator, virtual electronic equipment and a storage medium, and the method comprises the steps: building a compression or decompression session, and distributing a buffer region which can be accessed by a user space for the session based on a buffer region distribution strategy perceived by hardware characteristics; mapping the allocated buffer area to a user space, and writing to-be-compressed or to-be-decompressed data into a mapped user space address by the user mode application; submitting the compression or decompression request to a kernel driver through a calling interface; based on the compression or decompression request, triggering a kernel driver to configure a DMA descriptor of a hardware accelerator according to metadata, and driving the hardware accelerator to perform compression or decompression calculation on data in a buffer area; and after calculation is completed, informing the user state application of a result state through an asynchronous informing mechanism. According to the zero-copy and on-demand allocation dynamic buffer mechanism, invalid consumption of CPU and memory resources is reduced, and the method is more suitable for various actual calculation tasks.

Description

Technical Field

[0001] This application relates to compression and decompression testing technology, and more particularly to a data processing method and apparatus based on hardware accelerator zero-copy, a virtual electronic device, and a storage medium. Background Technology

[0002] With the explosive growth of data volume, data compression technology has become crucial in storage, transmission, and other fields. To improve compression efficiency, hardware accelerators such as Intel QAT and dedicated compression cards are widely used. In traditional hardware-accelerated compression architectures, data processing typically follows a layered copy pattern. Specifically, the application layer prepares the data to be compressed in user space and stores it in a user buffer. Subsequently, system calls such as read / write or ioctl are used to copy the data from the user space buffer to the kernel space buffer. The kernel driver then configures the direct memory access (DMA) descriptor of the hardware accelerator to point to the physical address of the kernel buffer. After the hardware accelerator performs compression calculations, it writes the results to the kernel space output buffer. The kernel driver again uses system calls to copy the result data from the kernel output buffer back to the target buffer in user space.

[0003] At least two data copies are required between user mode and kernel mode, consuming significant CPU cycles and memory bandwidth, becoming a major bottleneck for system performance, especially when processing large amounts of data. Insufficient output buffer capacity leads to operation failures, requiring the application to retry or allocate a larger fixed buffer; the former increases latency, while the latter wastes memory. Hardware accelerators have varying memory access characteristics. For example, some older or dedicated hardware requires DMA buffers to be physically contiguous, while buffers allocated by general-purpose operating systems may be non-contiguous. Traditional drivers lack intelligent adaptation to these hardware characteristics, resulting in malfunction or poor performance on some hardware platforms. When hardware fails or incompressible data is encountered, traditional methods often return failure directly, lacking a mechanism for graceful degradation while maintaining functionality, thus leading to insufficient system robustness. Summary of the Invention

[0004] This application provides a data processing method and apparatus, a virtual electronic device, and a storage medium based on zero-copy hardware accelerators, in order to at least solve the above-mentioned technical problems existing in the prior art.

[0005] According to a first aspect of this application, a zero-copy data processing method based on a hardware accelerator is provided, comprising: Establish a compression or decompression session, and allocate a user-space accessible buffer for the session based on a hardware-aware buffer allocation strategy. The allocated buffer is mapped to user space, and user-space applications write the data to be compressed or decompressed to the mapped user space address. By calling the interface, a compression or decompression request containing metadata pointing to the buffer is submitted to the kernel driver; Based on a compression or decompression request, the kernel driver is triggered to configure the DMA descriptor of the hardware accelerator according to the metadata, and the hardware accelerator is driven to perform compression or decompression calculations on the data in the buffer; the DMA descriptor points to the physical address of the buffer. After the calculation is completed, the result status is notified to the user-space application through an asynchronous notification mechanism.

[0006] In some executable embodiments, the method further includes: A pre-allocated first-level main buffer is used to receive regular compressed or decompressed output; When the hardware accelerator returns a state of insufficient output space, a second-level extended buffer is dynamically allocated based on the actual output length information returned by the hardware; the capacity of the second-level extended buffer matches the actual output length information.

[0007] In some executable embodiments, the method further includes: User-space applications write data to be compressed into a pre-allocated buffer via memory mapping, establishing a zero-copy data transmission path between user space and the hardware accelerator, and eliminating redundant data copying between user space and kernel space.

[0008] In some executable embodiments, the hardware-aware buffer allocation strategy includes: querying the capability parameters of the target hardware accelerator, selecting a contiguous memory allocator or a general DMA-BUF heap allocator for buffer allocation based on the physical address continuity requirements in the capability parameters, and following the memory alignment constraints in the capability parameters.

[0009] In some executable embodiments, the query target hardware accelerator's capability parameters include: Obtain at least one of the following from the virtual file system: a list of compression algorithms supported by the hardware accelerator, maximum and minimum processing block sizes, physical address continuity requirements, maximum number of supported Scatter Gather table entries, and memory alignment constraints.

[0010] In some executable embodiments, before triggering the kernel driver to configure the DMA descriptor of the hardware accelerator based on metadata, the following steps are also included: Obtain the original SG table corresponding to the DMA-BUF buffer; Based on the capability parameters of the hardware accelerator, the original SG table is optimized: If the hardware accelerator natively supports distributed-aggregated DMA and the number of entries in the original SG table does not exceed the hardware limit, then the original SG table structure is maintained. If the hardware accelerator requires physical continuity, then the non-contiguous physical pages in the original SG table are integrated into a contiguous DMA buffer through a memory remapping mechanism. If the number of entries in the original SG table exceeds the limit of the hardware accelerator, a merging operation of adjacent segments is performed to generate an optimized SG table that meets the hardware entry limit.

[0011] In some executable embodiments, the method further includes: Monitor the error status register of the hardware accelerator; Based on the detection results of the error status register, when the error status is determined to be a correctable error, the error correction mechanism is triggered and the compression or decompression task is resubmitted. Based on the detection results of the error status register, if the error status is determined to be an uncorrectable hardware error and the error frequency exceeds a preset threshold, the hardware accelerator is marked as degraded, and the hardware accelerator is switched from dynamic compression mode to static compression mode. If the switch fails, it is switched to software compression algorithm.

[0012] According to a second aspect of this application, a hardware accelerator-based zero-copy data compression apparatus is provided, comprising: The allocation unit is used to establish compression or decompression sessions and allocates user-space accessible buffers to the sessions based on a hardware-aware buffer allocation strategy. The mapping unit is used to map the allocated buffer to user space, so that user-space applications can write the data to be compressed or decompressed to the mapped user space address. The submission unit is used to submit a compression or decompression request containing metadata pointing to the buffer to the kernel driver by calling the interface; A triggering unit is used to trigger the kernel driver to configure the DMA descriptor of the hardware accelerator according to the metadata based on a compression or decompression request, and drive the hardware accelerator to perform compression or decompression calculations on the data in the buffer; the DMA descriptor points to the physical address of the buffer; The notification unit is used to notify the user-space application of the result status through an asynchronous notification mechanism after the calculation is completed.

[0013] In some executable embodiments, the apparatus further includes: A pre-allocation unit is used to pre-allocate the first-level main buffer for receiving regular compressed or decompressed output; When the hardware accelerator returns a state of insufficient output space, a second-level extended buffer is dynamically allocated based on the actual output length information returned by the hardware; the capacity of the second-level extended buffer matches the actual output length information.

[0014] In some executable embodiments, the apparatus further includes: The building unit is used to write data to be compressed into a pre-allocated buffer through memory mapping, thereby establishing a zero-copy data transmission path between user space and hardware accelerator and eliminating redundant data copying between user space and kernel space.

[0015] In some executable embodiments, the hardware-aware buffer allocation strategy includes: querying the capability parameters of the target hardware accelerator, selecting a contiguous memory allocator or a general DMA-BUF heap allocator for buffer allocation based on the physical address continuity requirements in the capability parameters, and following the memory alignment constraints in the capability parameters.

[0016] In some executable embodiments, the query target hardware accelerator's capability parameters include: Obtain at least one of the following from the virtual file system: a list of compression algorithms supported by the hardware accelerator, maximum and minimum processing block sizes, physical address continuity requirements, maximum number of supported scatter-aggregate SG table entries, and memory alignment constraints.

[0017] In some executable embodiments, the apparatus further includes: The optimization unit is configured to obtain the original SG table corresponding to the DMA-BUF buffer before the triggering unit triggers the kernel driver to configure the DMA descriptor of the hardware accelerator according to the metadata; and optimize the original SG table according to the capability parameters of the hardware accelerator. If the hardware accelerator natively supports distributed-aggregated DMA and the number of entries in the original SG table does not exceed the hardware limit, then the original SG table structure is maintained. If the hardware accelerator requires physical continuity, then the non-contiguous physical pages in the original SG table are integrated into a contiguous DMA buffer through a memory remapping mechanism. If the number of entries in the original SG table exceeds the limit of the hardware accelerator, a merging operation of adjacent segments is performed to generate an optimized SG table that meets the hardware entry limit.

[0018] In some executable embodiments, the apparatus further includes: The adjustment unit is used to monitor the error status register of the hardware accelerator. Based on the detection result of the error status register, when the error status is determined to be a correctable error, the error correction mechanism is triggered and the compression or decompression task is resubmitted. Based on the detection results of the error status register, if the error status is determined to be an uncorrectable hardware error and the error frequency exceeds a preset threshold, the hardware accelerator is marked as degraded, and the hardware accelerator is switched from dynamic compression mode to static compression mode. If the switch fails, it is switched to software compression algorithm.

[0019] According to a third aspect of this application, a virtual electronic device is provided, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps of the hardware accelerator-based zero-copy data processing method.

[0020] According to a fourth aspect of this application, a non-transitory computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the steps of the hardware accelerator-based zero-copy data processing method.

[0021] This application presents a hardware accelerator-based zero-copy data processing method and apparatus, virtual electronic device, and storage medium. By constructing a complete zero-copy path based on DMA-BUF, user-space data can be directly accessed by the hardware, completely eliminating the data copy overhead between user space and kernel space. This significantly improves the throughput of data compression / decompression and reduces CPU usage and latency. Through hardware-aware buffer allocation and SG table optimization strategies, the system can automatically identify the special requirements of different hardware, such as continuity, alignment, and SG entry limitations, and allocate appropriate buffers or perform optimization transformations accordingly, thereby achieving good compatibility and performance optimization for diverse hardware platforms. A two-level dynamic buffer management system effectively solves the problem of uncertain compressed output length. This application uses a main buffer to handle normal situations and expands the buffer for precise allocation as needed, avoiding failures caused by insufficient fixed buffer capacity and preventing memory waste caused by blindly allocating excessively large buffers. Accurate fault detection is achieved by monitoring the hardware error status register. The progressive degradation mechanism of this application ensures that the core compression function remains available even when the hardware partially or completely fails, thereby improving the overall reliability and service continuity of the system. Meanwhile, the differentiated handling of decompression errors ensures that data integrity is not compromised. The zero-copy and on-demand dynamic buffer allocation mechanism in this application jointly reduce the unnecessary consumption of CPU and memory resources, allowing system resources to be used more for actual computing tasks.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0023] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, in which: In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0024] Figure 1 This paper illustrates a schematic diagram of a hardware accelerator zero-copy data compression architecture based on the DMA-BUF mechanism according to an embodiment of this application. Figure 2 A flowchart illustrating a zero-copy data processing method based on a hardware accelerator according to an embodiment of this application is shown. Figure 3 This paper shows a schematic diagram of the composition structure of a data processing device based on a hardware accelerator zero-copy according to an embodiment of this application; Figure 4 A schematic diagram of the composition structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0025] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] This application presents a zero-copy data compression architecture for a hardware accelerator based on the DMA-BUF mechanism. By constructing a complete zero-copy transmission path, redundant data copying between user space and kernel space is eliminated. Through a hardware-aware buffer management strategy and a dynamic multi-level buffer system, efficient utilization and reliable management of the hardware compression accelerator are achieved. The DMA-BUF mechanism is the core mechanism for implementing zero-copy in the Linux kernel, allowing user programs to directly access shared buffers via mmap. Its hardware-aware DMA-BUF optimization mechanism enables intelligent adaptation to different hardware platforms. This application employs a two-level dynamic buffer management system, effectively solving the technical problem of uncertain compression and decompression output. It adopts a fault detection and progressive degradation recovery mechanism based on hardware error reporting, and eliminates redundant copying between user space and kernel space through a complete zero-copy data transmission path.

[0027] The following specific examples further illustrate the essence of the technical solutions in the embodiments of this application.

[0028] Figure 1 This illustration shows a schematic diagram of a hardware accelerator zero-copy data compression architecture based on the DMA-BUF mechanism according to an embodiment of this application. The hardware accelerator zero-copy data compression architecture based on the DMA-BUF mechanism according to this embodiment of the application is the hardware foundation for realizing zero-copy data compression according to this embodiment of the application. It adopts a hierarchical architecture design, with clear functions at each level, and works together to complete the zero-copy compression processing task.

[0029] like Figure 1 As shown, the zero-copy data compression architecture of the hardware accelerator based on the DMA-BUF mechanism in this application includes: User-space application layer: Provides a standardized hardware-accelerated compression programming interface, supports synchronous and asynchronous operation modes, and encapsulates DMA-BUF buffer management functions and completion event handling mechanisms. The user-space interface unit provides a unified hardware-accelerated compression API, supporting asynchronous operations and batch processing.

[0030] Kernel-mode DMA-BUF management layer: As a core innovative component of this application embodiment, it implements a hardware-aware buffer allocation strategy, provides a physical continuity guarantee mechanism, and performs hardware-specific memory alignment optimization operations. The kernel-mode DMA-BUF management layer implements hardware characteristic identification and buffer optimization allocation, including physical address continuity detection, an SG table analysis engine, and hardware alignment preprocessing.

[0031] Kernel-mode engine driver layer: integrates multiple functional modules, including a hardware-specific DMA-BUF optimization module, namely the descriptor parsing command issuance engine, responsible for adapting to the characteristics of different hardware platforms; a processing response module, which realizes dynamic buffer allocation and switching; and a software algorithm degradation calculation module that coordinates the entire processing flow to ensure system reliability.

[0032] Hardware Abstraction Layer: Provides a unified interface for devices, supporting multiple hardware accelerator platforms and software degradation implementations.

[0033] Physical hardware layer: This layer contains the actual compression acceleration devices, which are connected to the host via the system bus to perform specific compression calculation tasks.

[0034] As an implementation example, the hardware physical layer adopts Phytium Compress Decompress (PCD), a hardware compression acceleration engine integrated on the Phytium platform, which supports two compression algorithms: deflate and lz4. The memory alignment precision is 64 bytes, and the maximum number of SG table entries is 128. No physical contiguous memory is required. The user-space application is a real-time data compression module for an enterprise-level storage system, which needs to perform high-speed compression processing on the stored data. The input data volume is 10MB-100MB, and the estimated compression ratio is 2:1-5:1.

[0035] The user-space application layer uses a standardized compression API interface, including compression session creation (create_compress_session), task submission (submit_compress_task), result retrieval (get_compress_result), and session destruction (destroy_compress_session). The application layer creates a compression session by passing in parameters such as the input data volume, estimated compression ratio, and target compression algorithm (lz4) through the create_compress_session interface. It then establishes a mapping between the user buffer and the DMA-BUF buffer through the mmap interface, directly writing the stored data to be compressed into the DMA-BUF buffer.

[0036] The DMA-BUF management layer in this application specifically includes the following processing modules: Hardware feature awareness module: Reads the feature parameters of the hardware compression acceleration engine integrated in the Phytium platform through the sysfs file system, including supported algorithms (deflate / lz4), alignment precision (64 bytes), maximum number of SG table entries (128), physical continuity requirements (none), and establishes a parameter mapping table; Buffer allocation module: Based on hardware characteristic parameters, selects a general system DMA-BUF heap allocator to allocate buffers, ensuring that buffer addresses meet the 64-byte alignment requirement; SG table optimization module: Parse the original SG table (assuming it contains 200 segments, each segment is 512KB in length), and evaluate that the number of SG table entries (200) exceeds the hardware limit (128). Perform adjacent segment merging optimization, merge adjacent segments with consecutive addresses into a single segment, and finally generate an optimized SG table containing 98 entries, which meets the hardware requirements.

[0037] The kernel-mode engine driver layer in this embodiment of the application specifically includes the following processing modules: Compression task parsing module: Receives task parameters submitted by the application layer, parses them to obtain the compression algorithm (lz4), input data length (50MB), and output requirements (real-time feedback). DMA descriptor generation module: Based on the optimized SG table, it generates DMA descriptors containing information such as input data address (physical address of DMA-BUF buffer), output buffer address (physical address of main buffer), compression algorithm type, and data length; Two-level dynamic buffer management module: The main buffer capacity is preset to 25MB based on the estimated compression ratio (2:1-5:1) and the input data length (50MB); the capacity adaptation unit monitors the length of the compression result in real time; The soft algorithm degradation calculation module collects status register information from hardware compression acceleration engines integrated into the Phytium platform through an interrupt mechanism, and monitors DMA transfer and compression engine working status.

[0038] Hardware Abstraction Layer: Encapsulates the communication protocol of the PCIe bus and the underlying instruction set of the hardware compression acceleration engine integrated with the Phytium platform, providing a unified hardware operation interface (such as starting compression, reading status, and obtaining results) to achieve the adaptation of kernel drivers and hardware accelerators.

[0039] Physical hardware layer: The hardware compression acceleration engine integrated in the Phytium platform is connected to the host via the PCIe 4.0 bus, receives the DMA descriptor issued by the kernel driver, reads 50MB of data to be compressed from the DMA-BUF buffer, and performs compression calculations using the lz4 algorithm.

[0040] The zero-copy data compression architecture of this application follows a strict timing control process: the initialization phase establishes a compression session and allocates a hardware-optimized buffer; the data preparation phase enables direct data writing to user space through memory mapping; the task submission phase transmits compression request metadata through the IOCTL interface; the kernel processing phase executes the complete hardware-accelerated compression pipeline; the result return phase transmits the processing result through an asynchronous notification mechanism; and the resource reclamation phase safely releases all allocated resources.

[0041] The zero-copy data compression architecture of this application embodiment completes the hardware capability discovery and registration process through the standard kernel-mode DMA BUF management layer during the initialization phase. Regardless of their physical connection method (e.g., via PCIe or Platform bus), the hardware accelerators in this application embodiment are all abstracted into unified device objects by the corresponding kernel driver. After detecting the hardware, the driver exposes its capability parameters to user space and other kernel components through the sysfs system file, including but not limited to: a list of supported compression algorithms (such as deflate and lz4), maximum and minimum processing block sizes, physical address contiguity requirements, the maximum number of supported SG table entries, and memory alignment constraints.

[0042] The buffer allocation process employs a hardware-aware intelligent decision-making mechanism. Upon receiving an allocation request, the decision module first obtains the target hardware's characteristic data by querying the sysfs interface. For hardware requiring physically contiguous memory, it selects the contiguous memory allocator (CMA) or allocates from a reserved memory region specific to the hardware. For hardware supporting SG tables and without contiguity requirements, a general-purpose system DMA-BUF heap allocator is used to improve flexibility. Throughout the allocation process, the system strictly adheres to the alignment constraints specified by the hardware via sysfs, ensuring the efficiency and correctness of DMA transfers.

[0043] The SG table processing employs a hierarchical optimization strategy. After obtaining the original SG table, the DMA mapping module first analyzes its segment distribution characteristics and continuity indicators. The hardware compatibility assessment module then determines whether the current SG table can be used directly based on device capability data read from sysfs. For devices that natively support scatter-cluster DMA and whose number of scatter-cluster (SG) entries is within the hardware limit, the original SG table structure remains unchanged. For hardware requiring physical continuity, a memory remapping mechanism is used to consolidate non-contiguous physical pages into a contiguous DMA buffer. When the number of SG entries exceeds the hardware limit, the system performs a merging optimization operation on adjacent segments to generate an optimized SG table that meets the hardware requirements.

[0044] The zero-copy data compression architecture of this application constructs a two-level dynamic buffer management system to cope with the uncertainty of the output length of compression or decompression. The first-level main buffer is pre-allocated based on the expected data characteristics provided by the application when the compression or decompression session is created. Its capacity is optimized according to the hardware processing capabilities and the typical compression ratio of the target algorithm, serving as the preferred output target for regular compression and decompression tasks.

[0045] When the hardware returns an insufficient buffer space status code, the zero-copy data compression architecture of this application automatically parses the actual output length information contained in the hardware response and precisely triggers the allocation process of the second-level extended buffer accordingly. The second-level buffer is dynamically sized based on the precise length requirement returned by the hardware, and adopts the same hardware optimization allocation strategy as the main buffer to ensure that the allocated buffer capacity perfectly matches the actual requirements of the compressed output.

[0046] The zero-copy data compression architecture of this application establishes a fault detection mechanism based on hardware error reporting. The hardware interface layer monitors the DMA transfer error status register to detect data transmission integrity; the compression engine error status register provides algorithm execution exception information; and the device health status register reflects the overall hardware operating status. All error statuses are collected in real time through interrupt mechanisms or polling.

[0047] In this embodiment, the fault classification and processing adopts a hierarchical decision-making strategy: for the detected correctable errors (such as single-bit ECC errors), the zero-copy data compression architecture of this embodiment automatically triggers the error correction mechanism and resubmits the compression task; for uncorrectable hardware errors, the system records the error mode and evaluates the hardware reliability. When the error frequency exceeds the set threshold, the hardware is marked as degraded.

[0048] The compression failure handling implements a progressive degradation scheme: when hardware compression fails consecutively, the system first attempts to switch from dynamic compression mode to static compression mode to improve the success rate by reducing the amount of data processed at one time; if static compression still fails, the system automatically switches to software compression algorithm to ensure the continuous availability of compression function.

[0049] The decompression error handling adopts a differentiated treatment principle: for hardware-correctable errors encountered during the decompression process, the system automatically performs error repair and continues processing; for unrecoverable decompression errors, the system determines that there is an integrity problem with the input data source, directly returns a data corruption error code to the upper layer, and does not perform downgrade processing.

[0050] The zero-copy data compression architecture of the hardware accelerator in this application embodiment has a profound impact on multiple cutting-edge fields. This architecture fully leverages the high-performance characteristics of the hardware compression engine while significantly reducing system overhead through zero-copy design, providing innovative solutions for the data processing needs of various industries. In enterprise-level storage systems, it can significantly optimize data storage efficiency. Through real-time hardware compression, the storage system can significantly increase effective storage capacity while maintaining high performance, providing technical support for cost control in large-scale data centers. Data deduplication and archiving systems can achieve line-rate compression processing using this application embodiment, meeting the performance requirements of data lifecycle management. In virtualization environments, real-time compression of virtual machine images can reduce storage space usage, while hardware acceleration ensures that virtualization performance is not affected. In edge computing scenarios, real-time data preprocessing of edge nodes in this application embodiment can reduce network transmission load through hardware compression, meeting the operational needs of bandwidth-constrained environments. Massive amounts of sensor data generated by IoT devices can be efficiently compressed at the edge, extending data storage cycles and optimizing cloud transmission efficiency. Storage optimization in video surveillance systems can also benefit from the technical solutions of this application embodiment, enabling real-time compression and archiving of surveillance video streams.

[0051] The technical solutions in this application are also applicable to the field of data security and privacy protection. Compression processing of encrypted data before storage can optimize ciphertext storage efficiency, while hardware acceleration maintains system performance. Data desensitization and compression in privacy-preserving computation can be performed simultaneously, achieving a balance between data processing efficiency and security.

[0052] Based on the above system architecture, the data processing method based on hardware accelerator zero copy in the embodiments of this application is described in detail.

[0053] Figure 2 The following is a flowchart illustrating a zero-copy data processing method based on a hardware accelerator according to an embodiment of this application. Figure 2 As shown, the data processing method based on hardware accelerator zero-copy in this application embodiment includes the following processing steps: Step 201: Establish a compression or decompression session and allocate a user-space accessible buffer for the session based on a hardware-aware buffer allocation strategy.

[0054] The kernel device model is used to discover and register the capabilities of hardware accelerators. After the driver detects the hardware, it exposes the hardware capability parameters to user space and other kernel components through the sysfs file system. The hardware capability parameters include at least a list of supported compression algorithms (such as deflate, lz4, etc.), maximum and minimum processing block size, physical address continuity requirements, maximum number of supported SG table entries, and memory alignment constraints.

[0055] User applications initiate requests via user-space APIs. The APIs interact with kernel drivers to create compression / decompression sessions and initialize hardware-aware DMA-BUF management modules and two-level dynamic buffer management modules based on hardware characteristic parameters. The driver calls the hardware-aware DMA-BUF allocator, allocating one or more DMA-BUF buffers of the required size, alignment, and contiguousness according to the target hardware's registration capabilities (read from sysfs), and returns their file descriptors (fds) to user space. The user-space library then maps these buffers to the application process's virtual address space using the mmap system call.

[0056] The core of DMA-BUF is representing the buffer as a scatter-gather list (SGlist) to efficiently handle DMA operations on non-contiguous memory pages. The SGlist's key role in DMA-BUF is to provide a physical memory description; it's a data structure of type `struct sg_table` containing a set of `struct sg_entry` structures, each describing a contiguous block of physical memory pages (identified by its physical address and length). This design allows device drivers to directly access buffer data scattered throughout physical memory via the DMA engine, without the kernel needing to copy the data to contiguous memory, thus reducing memory copy overhead and improving the efficiency of large data transfers.

[0057] DMA-BUF passes the SG lis via a file descriptor (fd). The Exporter driver calls dma_buf_export() to create a dma_buf object and associate it with an anonymous file, then uses dma_buf_fd() to convert it into a file descriptor. The file descriptor can be passed to user space or other kernel modules. The Importer driver uses dma_buf_get() to obtain a DMA-BUF reference from the file descriptor and uses dma_buf_map_attachment() to bind the DMA-BUF to the local device. Auxiliary functions: dma_buf_vmap() and dma_buf_vunmap() provide virtual address mapping, suitable for scenarios requiring contiguous virtual addresses.

[0058] Specifically, regardless of whether the physical connection method of the hardware accelerator is PCIe bus or Platform bus, it is abstracted into a unified device object by the corresponding kernel driver, ensuring that upper-layer modules can access the hardware without discrimination. During the hardware detection phase, the driver obtains the inherent characteristics of the hardware by reading hardware registers and other methods, and writes them to a specified path in the sysfs file system after standardization for other modules to query and call.

[0059] Step 202: Map the allocated buffer to user space, and the user-space application writes the data to be compressed or decompressed to the mapped user space address.

[0060] The application directly writes the data to be processed to the user-space virtual address obtained in the previous step. The data actually resides in the DMA-BUF buffer and does not require additional copying.

[0061] Step 203: Submit a compression or decompression request containing metadata pointing to the buffer to the kernel driver by calling the interface.

[0062] The application submits a task request to the kernel driver via the ioctl system call. This request mainly contains task metadata, such as: the file descriptor of the input / output DMA-BUF, the offset of the data in the buffer, the data length, and the compression algorithm type.

[0063] Step 204: Based on the compression or decompression request, the kernel driver is triggered to configure the DMA descriptor of the hardware accelerator according to the metadata, and drive the hardware accelerator to perform compression or decompression calculations on the data in the buffer.

[0064] In this embodiment, the DMA descriptor points to the physical address of the buffer.

[0065] SG Table Processing: The driver retrieves the SG table of the corresponding physical memory page based on the input DMA-BUF file descriptor. A hardware-specific optimization module is invoked to review and optimize the SG table according to hardware capabilities. DMA and Hardware Configuration: The driver fills the optimized SG table information (or contiguous physical addresses) into the hardware accelerator's DMA descriptor register, informing the hardware of the exact locations of input and output data. The driver writes to the control register, starting the hardware accelerator to perform compression or decompression calculations. The hardware directly reads data from the input buffer via DMA, processes it, and writes it directly to the output buffer. During compression operations, if the hardware returns an error code indicating insufficient output buffer length (usually containing the actual required length), the two-level buffer management module is triggered. It dynamically allocates a second-level extended buffer based on the required length, updates the hardware DMA descriptor, and resubmits this part of the task.

[0066] In this embodiment, before the kernel driver configures the DMA descriptor of the hardware accelerator based on metadata, the method further includes: obtaining the original SG table corresponding to the DMA-BUF buffer; optimizing the original SG table according to the capability parameters of the hardware accelerator: if the hardware accelerator natively supports scatter-cluster DMA and the number of entries in the original SG table does not exceed the hardware limit, then the original SG table structure is maintained; if the hardware accelerator requires physical continuity, then non-contiguous physical pages in the original SG table are integrated into a contiguous DMA buffer through a memory remapping mechanism; if the number of entries in the original SG table exceeds the limit of the hardware accelerator, then adjacent segments are merged to generate an optimized SG table that meets the hardware entry limit.

[0067] The following details the hardware-aware DMA-BUF management and SG table optimization process: During system startup or hardware loading, the kernel driver detects the hardware accelerator and exposes its detailed capability parameters through the sysfs interface (e.g., / sys / class / accel / accel0 / ), such as: requires_contiguous=1, max_sg_entries=8, dma_alignment=4096, etc. It receives buffer allocation requests. When a buffer needs to be allocated for a session, the request reaches the improved DMA-BUF management layer. The allocator queries the target hardware's registered capability parameters in sysfs. Based on the parameters, it makes a decision: if requires_contiguous=1, a contiguous memory allocator (e.g., CMA) is selected for allocation. If requires_contiguous=0, a general DMA-BUF heap allocator (e.g., system_heap) is selected for better flexibility. During allocation, it ensures that the address and size meet the dma_alignment requirements. For allocated non-contiguous DMA-BUFs, when they are used for DMA, the kernel can obtain their original SG table describing the physical page distribution. The hardware-specific optimization module analyzes the number of entries and contiguousness of the original SG table, comparing them with hardware capabilities (max_sg_entries, requires_contiguous). If the hardware supports non-contiguous entries and the number of entries is within the limit (path 371), the SG table can be used directly. If the hardware requires contiguous entries but the SG table is non-contiguous (path 372), memory remapping is triggered, consolidating multiple physical pages into a contiguous DMA address region. This may incur a copy overhead, but it only occurs during driver initialization for specific hardware, far superior to the traditional approach of copying for every transfer. If the number of SG entries exceeds the limit (path 373), an attempt is made to merge segments with adjacent physical addresses to generate an optimized SG table with fewer entries. The final hardware-recognizable physical address information (contiguous addresses or the optimized SG table) is then configured into the hardware DMA engine.

[0068] Step 205: After the calculation is completed, the result status is notified to the user-space application through an asynchronous notification mechanism.

[0069] In this embodiment of the application, the aforementioned method further includes: pre-allocating a first-level main buffer for receiving regular compressed or decompressed output; when the hardware accelerator returns an insufficient output space state, dynamically allocating a second-level extended buffer according to the actual output length information returned by the hardware; the capacity of the second-level extended buffer is matched with the actual output length information.

[0070] The following details the two-level buffer allocation method of the embodiments of this application.

[0071] During session initialization, a moderately sized DMA-BUF is pre-allocated as the main output buffer based on the typical data block size and average compression ratio indicated by the application. During processing, the hardware calculates the output data volume in real time and checks if the buffer is sufficient. The hardware compares the output data volume with the remaining space in the current output buffer. If the buffer is sufficient, the hardware completes the calculation normally and writes the result. If the buffer is insufficient, the hardware aborts the current block processing and returns a specific error status code, which carries the actual output data length L required for the calculation. The kernel driver's two-level buffer management module catches this error and parses the length L. Subsequently, it immediately allocates an extended DMA-BUF buffer of size L (or with appropriate margin) precisely according to the hardware-aware allocation strategy of this embodiment. The driver updates the output address in the hardware DMA descriptor to the new extended buffer address and resubmits the previously unfinished data block processing request. The hardware continues the calculation from the breakpoint and writes the result to the extended buffer. The driver records the association between the main buffer and the extended buffer. After the task is completed, the user space is notified. The user-space API may need to collect the complete output data from both buffers (logical concatenation, not physical copying). For decompression, since the output length is fixed (equal to the original data length), it can usually be precisely allocated; this mechanism primarily serves compression operations. The hardware-aware buffer allocation strategy includes: querying the capability parameters of the target hardware accelerator, selecting a contiguous memory allocator or a general-purpose DMA-BUF heap allocator for buffer allocation based on the physical address continuity requirements in the capability parameters, and adhering to the memory alignment constraints in the capability parameters. In this application embodiment, querying the capability parameters of the target hardware accelerator includes: obtaining at least one of the following through a virtual file system: a list of compression algorithms supported by the hardware accelerator, maximum and minimum processing block sizes, physical address continuity requirements, the maximum number of supported scatter-aggregate SG table entries, and memory alignment constraints.

[0072] The method in this application embodiment further includes: monitoring the error status register of the hardware accelerator; based on the detection result of the error status register, when it is determined that the error status is a correctable error, triggering an error correction mechanism and resubmitting the compression or decompression task; based on the detection result of the error status register, when it is determined that the error status is an uncorrectable hardware error and the error frequency exceeds a preset threshold, marking the hardware accelerator in a degraded state; and switching the hardware accelerator from dynamic compression mode to static compression mode, and if the switching fails, switching to a software compression algorithm.

[0073] The following details the fault detection and progressive degradation process: The hardware's DMA error status register and engine error status register are read via polling or interrupts. The error type is determined: Correctable errors (e.g., single-bit ECC errors). Hardware or driver error correction mechanisms are triggered, such as rereading or using checksums for repair. The process ends upon success. Uncorrectable / persistent hardware errors are identified. Error logs are recorded, and the error occurrence rate is assessed. If errors occur frequently within a short period, exceeding a preset threshold, the hardware unit is deemed unstable. The degradation management module guides the compression pipeline execution module to attempt the following modes in sequence: Switching to static compression mode. The hardware may support dynamic (optimal compression) and static (fast compression) modes. Static mode typically has lower computational load and is more stable. The failed task is resubmitted using static mode. The success of static mode is determined. If successful, subsequent sessions can default to or recommend using static mode. If static mode also fails, or the hardware is deemed completely faulty, the compression task is routed to a pure software algorithm library (e.g., zlib, LZ4) for execution via the Hardware Abstraction Layer (HAL). For decompression tasks, if an unrecoverable error such as data corruption (S584) is detected, the degradation process will not be initiated; instead, a data corruption error will be returned to ensure data integrity is not compromised by spurious data. The system continuously monitors the degraded hardware, and if no further errors occur within a long period, it may attempt to restore it to a usable state and reinstate it to the workload.

[0074] User-space applications write data to be compressed into a pre-allocated buffer via memory mapping, establishing a zero-copy data transmission path between user space and the hardware accelerator, and eliminating redundant data copying between user space and kernel space.

[0075] Figure 3 This paper illustrates a schematic diagram of the composition structure of a hardware accelerator-based zero-copy data processing device according to an embodiment of this application. Figure 3 As shown in the figure, the hardware accelerator-based zero-copy data compression apparatus of this application includes: Allocation unit 30 is used to establish compression or decompression sessions and allocates user-space accessible buffers for the sessions based on a hardware-aware buffer allocation strategy. Mapping unit 31 is used to map the allocated buffer to user space, so that user-mode applications can write the data to be compressed or decompressed to the mapped user space address. Submission unit 32 is used to submit a compression or decompression request containing metadata pointing to the buffer to the kernel driver by calling the interface; Triggering unit 33 is used to trigger the kernel driver to configure the DMA descriptor of the hardware accelerator according to the metadata based on the compression or decompression request, and drive the hardware accelerator to perform compression or decompression calculations on the data in the buffer; the DMA descriptor points to the physical address of the buffer; Notification unit 34 is used to notify the user-space application of the result status through an asynchronous notification mechanism after the calculation is completed.

[0076] exist Figure 3 Based on the hardware accelerator-based zero-copy data compression device shown, the hardware accelerator-based zero-copy data compression device in this application embodiment further includes: Pre-allocated units ( Figure 3 (not shown in the image), used to pre-allocate the first-level main buffer for receiving regular compressed or decompressed output; When the hardware accelerator returns a state of insufficient output space, a second-level extended buffer is dynamically allocated based on the actual output length information returned by the hardware; the capacity of the second-level extended buffer matches the actual output length information.

[0077] exist Figure 3 Based on the hardware accelerator-based zero-copy data compression device shown, the hardware accelerator-based zero-copy data compression device in this application embodiment further includes: Building blocks ( Figure 3 (Not shown in the image) is used to write data to be compressed into a pre-allocated buffer via memory mapping, thereby establishing a zero-copy data transmission path between user space and hardware accelerator and eliminating redundant data copying between user space and kernel space.

[0078] In some executable embodiments, the hardware-aware buffer allocation strategy includes: querying the capability parameters of the target hardware accelerator, selecting a contiguous memory allocator or a general DMA-BUF heap allocator for buffer allocation based on the physical address continuity requirements in the capability parameters, and following the memory alignment constraints in the capability parameters.

[0079] In some executable embodiments, the query target hardware accelerator's capability parameters include: Obtain at least one of the following from the virtual file system: a list of compression algorithms supported by the hardware accelerator, maximum and minimum processing block sizes, physical address continuity requirements, maximum number of supported scatter-aggregate SG table entries, and memory alignment constraints.

[0080] exist Figure 3 Based on the hardware accelerator-based zero-copy data compression device shown, the hardware accelerator-based zero-copy data compression device in this application embodiment further includes: Optimization unit ( Figure 3 (Not shown in the image), used to obtain the original SG table corresponding to the DMA-BUF buffer before the triggering unit triggers the kernel driver to configure the DMA descriptor of the hardware accelerator according to the metadata; and to optimize the original SG table according to the capability parameters of the hardware accelerator: If the hardware accelerator natively supports distributed-aggregated DMA and the number of entries in the original SG table does not exceed the hardware limit, then the original SG table structure is maintained. If the hardware accelerator requires physical continuity, then the non-contiguous physical pages in the original SG table are integrated into a contiguous DMA buffer through a memory remapping mechanism. If the number of entries in the original SG table exceeds the limit of the hardware accelerator, a merging operation of adjacent segments is performed to generate an optimized SG table that meets the hardware entry limit.

[0081] exist Figure 3 Based on the hardware accelerator-based zero-copy data compression device shown, the hardware accelerator-based zero-copy data compression device in this application embodiment further includes: Adjustment unit ( Figure 3 (not shown in the image) is used to monitor the error status register of the hardware accelerator. Based on the detection result of the error status register, when the error status is determined to be a correctable error, the error correction mechanism is triggered and the compression or decompression task is resubmitted. Based on the detection results of the error status register, if the error status is determined to be an uncorrectable hardware error and the error frequency exceeds a preset threshold, the hardware accelerator is marked as degraded, and the hardware accelerator is switched from dynamic compression mode to static compression mode. If the switch fails, it is switched to software compression algorithm.

[0082] In the hardware accelerator-based zero-copy data compression device of this application embodiment, each processing unit can be implemented by one or more central processing units (CPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components.

[0083] Regarding the apparatus in the above embodiments, the specific manner in which each module and unit performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0084] Figure 4 A schematic block diagram of example 800, which can be used to implement embodiments of this application, is shown. Figure 4 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 may also store various programs and data required for the operation of the electronic device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0085] Multiple components in electronic device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, data transceiver, etc. Communication unit 809 allows electronic device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0086] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as hardware accelerator-based zero-copy data processing methods. For example, in some embodiments, the hardware accelerator-based zero-copy data processing method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the hardware accelerator-based zero-copy data processing method described above can be performed. Alternatively, in other embodiments, computing unit 801 may be configured by any other suitable means (e.g., by means of firmware) to perform a hardware accelerator-based zero-copy data processing method.

[0087] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0088] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0089] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0090] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0091] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0092] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0093] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0094] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0095] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A zero-copy data processing method based on a hardware accelerator, characterized in that, The method includes: Establish a compression or decompression session, and allocate a user-space accessible buffer for the session based on a hardware-aware buffer allocation strategy. The allocated buffer is mapped to user space, and user-space applications write the data to be compressed or decompressed to the mapped user space address. By calling the interface, a compression or decompression request containing metadata pointing to the buffer is submitted to the kernel driver; Based on a compression or decompression request, the kernel driver is triggered to configure the DMA descriptor of the hardware accelerator according to the metadata, and the hardware accelerator is driven to perform compression or decompression calculations on the data in the buffer; the DMA descriptor points to the physical address of the buffer. After the calculation is completed, the result status is notified to the user-space application through an asynchronous notification mechanism.

2. The method according to claim 1, characterized in that, The method further includes: A pre-allocated first-level main buffer is used to receive regular compressed or decompressed output; When the hardware accelerator returns a state of insufficient output space, a second-level extended buffer is dynamically allocated based on the actual output length information returned by the hardware; the capacity of the second-level extended buffer matches the actual output length information.

3. The method according to claim 1, characterized in that, The method further includes: User-space applications write data to be compressed into a pre-allocated buffer via memory mapping, establishing a zero-copy data transmission path between user space and the hardware accelerator, and eliminating redundant data copying between user space and kernel space.

4. The method according to claim 1, characterized in that, The hardware-aware buffer allocation strategy includes: querying the capability parameters of the target hardware accelerator, selecting a contiguous memory allocator or a general DMA-BUF heap allocator for buffer allocation based on the physical address continuity requirements in the capability parameters, and following the memory alignment constraints in the capability parameters.

5. The method according to claim 4, characterized in that, The capability parameters of the target hardware accelerator being queried include: Obtain at least one of the following from the virtual file system: a list of compression algorithms supported by the hardware accelerator, maximum and minimum processing block sizes, physical address continuity requirements, maximum number of supported scatter-aggregate SG table entries, and memory alignment constraints.

6. The method according to claim 5, characterized in that, Before triggering the kernel driver to configure the DMA descriptor of the hardware accelerator based on metadata, the following steps are also included: Obtain the original SG table corresponding to the DMA-BUF buffer; Based on the capability parameters of the hardware accelerator, the original SG table is optimized: If the hardware accelerator natively supports distributed-aggregated DMA and the number of entries in the original SG table does not exceed the hardware limit, then the original SG table structure is maintained. If the hardware accelerator requires physical continuity, then the non-contiguous physical pages in the original SG table are integrated into a contiguous DMA buffer through a memory remapping mechanism. If the number of entries in the original SG table exceeds the limit of the hardware accelerator, a merging operation of adjacent segments is performed to generate an optimized SG table that meets the hardware entry limit.

7. The method according to claim 1, characterized in that, The method further includes: Monitor the error status register of the hardware accelerator; Based on the detection results of the error status register, when the error status is determined to be a correctable error, the error correction mechanism is triggered and the compression or decompression task is resubmitted. Based on the detection results of the error status register, if the error status is determined to be an uncorrectable hardware error and the error frequency exceeds a preset threshold, the hardware accelerator is marked as degraded, and the hardware accelerator is switched from dynamic compression mode to static compression mode. If the switch fails, it is switched to software compression algorithm.

8. A zero-copy data compression device based on a hardware accelerator, characterized in that, The device includes: The allocation unit is used to establish compression or decompression sessions and allocates user-space accessible buffers to the sessions based on a hardware-aware buffer allocation strategy. The mapping unit is used to map the allocated buffer to user space, so that user-space applications can write the data to be compressed or decompressed to the mapped user space address. The submission unit is used to submit a compression or decompression request containing metadata pointing to the buffer to the kernel driver by calling the interface; A triggering unit is used to trigger the kernel driver to configure the DMA descriptor of the hardware accelerator according to the metadata based on a compression or decompression request, and drive the hardware accelerator to perform compression or decompression calculations on the data in the buffer; the DMA descriptor points to the physical address of the buffer; The notification unit is used to notify the user-space application of the result status through an asynchronous notification mechanism after the calculation is completed.

9. The apparatus according to claim 8, characterized in that, The device further includes: A pre-allocation unit is used to pre-allocate the first-level main buffer for receiving regular compressed or decompressed output; When the hardware accelerator returns a state of insufficient output space, a second-level extended buffer is dynamically allocated based on the actual output length information returned by the hardware; the capacity of the second-level extended buffer matches the actual output length information.

10. The apparatus according to claim 8, characterized in that, The device further includes: The building unit is used to write data to be compressed into a pre-allocated buffer through memory mapping, thereby establishing a zero-copy data transmission path between user space and hardware accelerator and eliminating redundant data copying between user space and kernel space.

11. The apparatus according to claim 8, characterized in that, The hardware-aware buffer allocation strategy includes: querying the capability parameters of the target hardware accelerator, selecting a contiguous memory allocator or a general DMA-BUF heap allocator for buffer allocation based on the physical address continuity requirements in the capability parameters, and following the memory alignment constraints in the capability parameters.

12. The apparatus according to claim 11, characterized in that, The capability parameters of the target hardware accelerator being queried include: Obtain at least one of the following from the virtual file system: a list of compression algorithms supported by the hardware accelerator, maximum and minimum processing block sizes, physical address continuity requirements, maximum number of supported scatter-aggregate SG table entries, and memory alignment constraints.

13. The apparatus according to claim 12, characterized in that, The device further includes: The optimization unit is configured to obtain the original SG table corresponding to the DMA-BUF buffer before the triggering unit triggers the kernel driver to configure the DMA descriptor of the hardware accelerator according to the metadata; and optimize the original SG table according to the capability parameters of the hardware accelerator. If the hardware accelerator natively supports distributed-aggregated DMA and the number of entries in the original SG table does not exceed the hardware limit, then the original SG table structure is maintained. If the hardware accelerator requires physical continuity, then the non-contiguous physical pages in the original SG table are integrated into a contiguous DMA buffer through a memory remapping mechanism. If the number of entries in the original SG table exceeds the limit of the hardware accelerator, a merging operation of adjacent segments is performed to generate an optimized SG table that meets the hardware entry limit.

14. The apparatus according to claim 8, characterized in that, The device further includes: The adjustment unit is used to monitor the error status register of the hardware accelerator. Based on the detection result of the error status register, when the error status is determined to be a correctable error, the error correction mechanism is triggered and the compression or decompression task is resubmitted. Based on the detection results of the error status register, if the error status is determined to be an uncorrectable hardware error and the error frequency exceeds a preset threshold, the hardware accelerator is marked as degraded, and the hardware accelerator is switched from dynamic compression mode to static compression mode. If the switch fails, it is switched to software compression algorithm.

15. A virtual electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the steps of the hardware accelerator-based zero-copy data processing method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the steps of the hardware accelerator-based zero-copy data processing method as described in any one of claims 1 to 7.

Images