Data processing method and communication apparatus

Abstract

Disclosed in the embodiments of the present application are a data processing method and a communication apparatus, which are used for reducing the access delay. The data processing method is applied to a processor, and the method comprises: acquiring first data from a host; acquiring metadata for describing the first data, the metadata being configured to be shared between the processor and an accelerator; and sending a first instruction to the accelerator, the first instruction comprising the metadata.

Description

A data processing method and a communication device

[0001] This application claims priority to Chinese Patent Application No. 2025102146275, filed on February 25, 2025, entitled "A Data Processing Method and Communication Device", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of computer technology, specifically to a data processing method and a communication device. Background Technology

[0003] The number of parameters in artificial intelligence models is growing exponentially, and the demand for memory is increasing far faster than the growth in the physical capacity of current accelerators. To meet this ever-expanding memory demand, current memory-based solutions mainly include two approaches: one is to expand memory resources by integrating numerous accelerators; the other is to utilize host memory as a cache for the accelerators. However, given the high price of memory, both of these solutions incur significant costs.

[0004] Currently, a storage device-based accelerator memory expansion solution has emerged as a cost-effective option, and in many scenarios, its performance is almost unaffected compared to memory-based solutions. Most storage device-based accelerator memory expansion solutions are currently CPU-centric, relying on the CPU to initiate access requests to storage devices. However, this approach struggles to meet the throughput demands of accelerators. On one hand, redundant storage software stacks increase memory access latency; on the other hand, when thousands or even tens of thousands of accelerator threads simultaneously initiate data access, the limited parallelism of the CPU and the synchronization overhead between the CPU and accelerators become bottlenecks to system performance improvement.

[0005] Therefore, current memory expansion solutions involve lengthy storage software stacks, increasing access latency. Furthermore, the limited parallelism of the CPU further contributes to the increase in access latency. Summary of the Invention

[0006] This application provides a data processing method and a communication device for reducing access latency.

[0007] To address the aforementioned technical problems, this application provides the following technical solutions:

[0008] In a first aspect, embodiments of this application provide a data processing method applied to a processor, the method comprising: acquiring first data from a host; acquiring metadata describing the first data, the metadata being shared between the processor and an accelerator; and sending a first instruction to the accelerator, the first instruction including the metadata.

[0009] In the above scheme, the processor can interact with the host to obtain first data. The processor can also generate metadata based on this first data, which can be shared between the processor and the accelerator. The processor can also interact with the accelerator to read this metadata into the accelerator. The accelerator can obtain the metadata from the processor and, based on this metadata, initiate an access request to the storage device, thereby storing the file data generated by the accelerator in the storage device. In this embodiment, data acceleration is accelerator-centric, allowing the accelerator to directly submit access requests to the storage device. The interaction process between the accelerator and the storage device completely bypasses the processor, thus reducing access latency.

[0010] In one possible implementation of the first aspect, obtaining the metadata describing the first data includes: obtaining the metadata from the first data according to the data format of the accelerator virtual disk. In the above scheme, the accelerator virtual disk is a custom format that achieves efficient sharing of metadata between the processor and the accelerator through metadata embedding. The accelerator obtains the address mapping between file data and storage devices by parsing the metadata.

[0011] In one possible implementation of the first aspect, the method further includes: converting the first data and the metadata according to the data format of the accelerator virtual disk to obtain second data, wherein the metadata is embedded in the second file data. In the above scheme, the accelerator virtual disk achieves metadata sharing between the processor and the accelerator by embedding the second data with metadata. The accelerator obtains the second data and address mapping by parsing the metadata, thereby obtaining usable storage space. This allows for the construction of a lightweight data management system within the accelerator, where the interface on the accelerator does not need to access data managed by the processor, achieving efficient metadata sharing.

[0012] In one possible implementation of the first aspect, the method further includes: sending a first access request to a storage device, the storage device being used to process the first access request and a second access request, the second access request being sent by the accelerator. In the above scheme, both the processor and the accelerator can interact with the storage device, enabling the storage device to process multiple access requests simultaneously. For example, the processor sends a first access request to the storage device, the accelerator sends a second access request to the storage device, and the storage device is used to process the first and second access requests, thus enabling multiple devices (processor and accelerator) to share the storage device.

[0013] In one possible implementation of the first aspect, sending the first access request to the storage device includes: sending the first access request to the storage device via a parallel storage queue between the processor and the storage device, wherein the parallel storage queue is also used by the accelerator to send a second access request to the storage device. In the above scheme, the processor can establish a parallel storage queue, which can be used by the processor to send the first access request and by the accelerator to send the second access request. The parallel storage queue enables multiple devices, including the processor and accelerator, to share the storage device.

[0014] In one possible implementation of the first aspect, obtaining the first data from the host includes: obtaining the first data from the host according to the checkpoint running frequency. In the above scheme, the processor can send metadata to the accelerator, the accelerator can generate checkpoint data, and the accelerator can perform address mapping between the metadata allocated by the processor and the storage device. Since the processor can obtain the first data from the host according to the checkpoint running frequency, the access latency of storing model checkpoint data to the storage device is reduced.

[0015] In one possible implementation of the first aspect, the metadata includes at least one of the following: the size of the first data, access permissions, modification time, access time, and the mapping relationship between the virtual file address of the first data and the address of the storage device.

[0016] Secondly, embodiments of this application provide a data processing method, characterized in that the method is applied to an accelerator, and the method includes: acquiring metadata according to a first instruction from a processor, the metadata being shared between the processor and the accelerator; determining the address of a storage device based on the metadata; and sending a second access request to the storage device, the second access request including the address of the storage device. In the above scheme, after the accelerator performs address mapping through metadata and determines the address of the storage device, the accelerator sends a second access request to the storage device, the second access request including the address of the storage device, enabling the storage device to store and modify data in response to the accelerator's second access request.

[0017] In one possible implementation of the second aspect, sending the second access request to the storage device includes: allocating a parallel storage queue in the accelerator's memory, the parallel storage queue also being used by the processor to send a first access request to the storage device; and writing the second access request into the parallel storage queue. In the above scheme, the accelerator can allocate a parallel storage queue in its memory, which can be used by the processor to send the first access request, and also by the accelerator to send the second access request. The parallel storage queue enables multiple devices, including the processor and accelerator, to share the storage device.

[0018] In one possible implementation of the second aspect, the parallel storage queue is further used by the storage device to read the first checkpoint data generated by the accelerator. In the above scheme, the processor can send metadata to the accelerator, the accelerator can generate checkpoint data, and the accelerator can perform address mapping between the processor-allocated metadata and the storage device. Since the processor can obtain the first data from the host according to the checkpoint execution frequency, the access latency for storing model checkpoint data to the storage device is reduced.

[0019] In one possible implementation of the second aspect, sending the second access request to the storage device includes: sending the second access request to the storage device through the computing cores of the accelerator. In the above scheme, the accelerator may include multiple computing cores, and sending the second access request to the storage device through these cores refines the granularity of the access request submission. In this embodiment, the second access request can be submitted directly through the accelerator computing cores, achieving high bandwidth utilization even with a small access granularity by leveraging the high parallelism within the accelerator.

[0020] In one possible implementation of the second aspect, the accelerator's memory stores intermediate parameters for training the model; sending a second access request to the storage device includes: sending the second access request to the storage device through the accelerator's read / write interface to move the intermediate parameters of the training model to the storage device. In the above scheme, the accelerator can interact with the storage device through the read / write interface, so that the storage device can move the intermediate parameters of the training model to the storage device in response to the second access request. In this embodiment, the accelerator's read / write interface replaces the memory access interface for forward and backward propagation in the training model, realizing fast movement of the intermediate parameters of the training model and reducing the memory usage of the accelerator.

[0021] In one possible implementation of the second aspect, sending the second access request to the storage device includes: sending the second access request to the storage device via a first thread in the parallel thread combination of the accelerator, wherein multiple threads in the parallel thread combination use the same lock. In the above scheme, the accelerator performs lock waiting at the granularity of parallel thread combinations, and multiple threads in the parallel thread combination use the same lock. There is no waiting between threads within the parallel thread combination, thereby achieving a higher level of access efficiency than a single thread of the processor and reducing access latency to the storage device.

[0022] Thirdly, embodiments of this application also provide a communication device, the communication device including a processor, the communication device comprising:

[0023] The data acquisition module is used to acquire the first data from the host.

[0024] A metadata acquisition module is used to acquire metadata describing the first data, the metadata being shared between the processor and the accelerator;

[0025] A sending module is used to send a first instruction to the accelerator, the first instruction including the metadata.

[0026] In a third aspect of this application, the constituent modules of the communication device may also perform the steps described in the first aspect and various possible implementations, as detailed in the foregoing description of the first aspect and various possible implementations.

[0027] Fourthly, embodiments of this application also provide a communication device, the communication device including an accelerometer, the communication device comprising:

[0028] A metadata acquisition module is used to acquire metadata based on a first instruction from the processor, the metadata being shared between the processor and the accelerator;

[0029] The address mapping module is used to determine the address of the storage device based on the metadata.

[0030] The sending module is configured to send a second access request to the storage device, the second access request including the address of the storage device.

[0031] In the fourth aspect of this application, the constituent modules of the communication device may also perform the steps described in the second aspect and various possible implementations, as detailed in the foregoing description of the second aspect and various possible implementations.

[0032] Fifthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described in the first or second aspect above.

[0033] In a sixth aspect, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the methods described in the first or second aspect above.

[0034] In a seventh aspect, embodiments of this application provide a communication device that may include entities such as a processor, an accelerator, or a chip. The communication device includes a processor and a memory, the memory being coupled to the processor. The processor executes instructions stored in the memory, causing the communication device to perform the method as described in any one of the preceding first aspects. Alternatively, the communication device includes an accelerator and a memory, the memory being coupled to the accelerator. The accelerator executes instructions stored in the memory, causing the communication device to perform the method as described in any one of the preceding second aspects.

[0035] Eighthly, embodiments of this application provide a chip including one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from a memory of a communication device and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the communication device to perform the method described in any one of the first aspects.

[0036] In a ninth aspect, embodiments of this application provide a chip including one or more interface circuits and one or more accelerators; the interface circuits are configured to receive signals from a memory of a communication device and send the signals to the accelerators, the signals including computer instructions stored in the memory; when the accelerators execute the computer instructions, the communication device performs the method described in any one of the second aspects.

[0037] In a tenth aspect, embodiments of this application provide a communication device, the communication device including a processor and an accelerator;

[0038] The processor is configured to perform the method described in any one of the first aspects;

[0039] The accelerator is used to perform the method described in any one of the second aspects. Attached Figure Description

[0040] Figure 1a is a schematic diagram of an application scenario of a CPU-centric accelerator storage software stack;

[0041] Figure 1b is a schematic diagram of another application scenario of CPU-centric accelerator storage software stack;

[0042] Figure 1c is a schematic diagram of another application scenario of CPU-centric accelerator storage software stack;

[0043] Figure 2 is a schematic diagram of the average latency and tail latency of 4KB read I / O of GDS under different batch sizes;

[0044] Figure 3 is a schematic diagram of the average latency and tail latency of 4KB read I / O for GPUfs with different numbers of threads;

[0045] Figure 4 is a schematic diagram of a GPU-centric storage software stack solution;

[0046] Figure 5 is a schematic diagram of the composition structure of a data processing system provided in an embodiment of this application;

[0047] Figure 6 is a schematic diagram of the interaction process between a processor and an accelerator provided in an embodiment of this application;

[0048] Figure 7 is a schematic diagram of a CPU and GPU connected to an NVMe device via a PCIe interface according to an embodiment of this application;

[0049] Figure 8 is a schematic diagram of writing model checkpoint data based on Geminifs to an NVMe device according to an embodiment of this application;

[0050] Figure 9 is a schematic diagram of writing intermediate parameters of model training based on Geminifs into an NVMe device according to an embodiment of this application;

[0051] Figure 10 is a schematic diagram of the training performance test results of a large language model using GPT-2 provided in an embodiment of this application;

[0052] Figure 11 is a schematic diagram of the composition structure of a communication device provided in an embodiment of this application;

[0053] Figure 12 is a schematic diagram of the composition structure of another communication device provided in an embodiment of this application. Detailed Implementation

[0054] This application provides a data processing method and a communication device for reducing access latency.

[0055] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0056] As will be known to those skilled in the art, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0057] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of singular or plural items. The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not expressly listed or inherent to those processes, methods, products, or apparatus.

[0058] CPU-centric accelerator storage solutions rely on the CPU initiating storage access requests. The data processing system can include a central processing unit (CPU), a graphics processing unit (GPU), host memory, and storage devices, such as non-volatile memory express (NVMe) devices. The CPU, GPU, host memory, and NVMe devices are connected via a peripheral component interconnect express (PCIe) bus. Currently, there are three main solutions using accelerator storage software stacks: As shown in Figure 1a, one solution uses PyTorch's DataLoader, which relies on host jumper memory for data transfer. This storage software stack uses a file interface to read data into host memory, then uses a general-purpose accelerator driver interface to transfer data between host memory and device memory, and uses a file interface to transfer data between the host and storage devices. The data copying between the accelerator and host memory increases data transfer latency.

[0059] To address this issue, as shown in Figure 1b, a graphics processing unit direct storage (GDS) scheme exists. GDS utilizes PCIe's pear-to-pear direct memory access (P2P DMA) to enable direct data transfer between the GPU and storage devices without involving CPU memory usage, thus improving bandwidth utilization and reducing transmission latency. However, this scheme requires a fixed segment of host memory, still wasting host memory resources. Furthermore, it requires a non-portable operating system interface (POSIX) application programming interface (API), resulting in complex programming and high software overhead. In addition, GDS still relies on the CPU initiating input / output (I / O) requests to the GPU, thus becoming a bottleneck for GPU memory access. For example, the cuFileBatchI / OSubmit operation in GDS is limited to processing a maximum of 128 operations.

[0060] As shown in Figure 1c, there exists a CPU-centric storage architecture, such as the GPU File System (GPUfs), which can provide a Posix-like file system on the accelerator side. This allows the accelerator to transmit file system requests to the host, where a daemon thread initiates the actual file system request and data transfer between host memory and storage device memory. However, the complex control logic between the CPU and the accelerator lengthens the storage access path and reduces access efficiency.

[0061] Next, we compare the read performance of two representative CPU-centric methods, GPUfs and GDS, at different levels of parallelism. Test experiments were run on a server equipped with a GPU featuring 64 CPU cores and 80GB of memory, with the GPU's L1 cache disabled. Two NVMe storage devices were used: NVMe storage device 1 had a read / write latency of 15 microseconds, and NVMe storage device 2 had a read / write latency of 4 microseconds. Both achieved read bandwidths up to 7GB / s. The experimental results are shown in Figures 2 and 3 below.

[0062] Specifically, Figure 2 illustrates the average latency and tail latency of GDS's 4KB read I / O under different batch sizes. Batch size refers to the number of samples used in each iteration during deep learning training. The average (avg) of the sample size, percentile P95, percentile P99, and percentile P99.9 describe the latency corresponding to data points at different distribution locations in the dataset. Figure 2 shows that the average latency and tail latency of GDS decrease with increasing batch size, but remain relatively high (approximately 160 microseconds). The software stack overhead of GDS remains non-negligible.

[0063] Figure 3 illustrates the average and tail latency of 4KB read I / O for GPUfs under different thread counts. The average latency (avg), percentile (P95), percentile (P99), and percentile (P99.9) represent the latency of data points at different distribution locations within the dataset, categorized by the number of GPU threads. For GPUfs, when the number of GPU threads is small, the I / O latency on both storage devices exceeds 190 microseconds, meaning that software stack overhead accounts for over 90% of the overall I / O latency. When the number of GPU threads exceeds the number of CPU cores, both the average latency and tail latency increase dramatically. When the number of GPU threads reaches 1024, the increase is approximately 250%.

[0064] The above analysis reveals that CPU-centric accelerator card storage solutions face the following common problems: a lengthy storage software stack significantly increases I / O latency. Furthermore, when handling high-concurrency GPU I / O requests, the limited number of CPU cores will compete for resources, leading to increased I / O latency.

[0065] Figure 4 illustrates a GPU-centric storage software stack scheme. To avoid the inefficiency of CPU-centric methods, a GPU-centric storage software stack scheme is proposed. One approach involves a big accelerator memory (BaM) interconnected with GPU memory and storage. GPU memory includes queue metadata and cache metadata, as well as serial queues and I / O buffers. Storage includes multiple controllers (CTRLs) and media, with each controller containing a database register (DB Reg). High-speed data transfer is achieved using direct memory access (DMA). This scheme allocates NVMe queues in GPU memory and maps these queues through GPU drivers, making them visible to other devices on the PCIe bus. Furthermore, the NVMe driver can be integrated into the GPU process, allowing the GPU process to directly send NVMe I / O commands, which are then executed by the solid-state drive (SSD) controller. Similarly, it can be seen that BaM adopts a similar approach to the Storage Performance Development Kit (SPDK): both provide a complete block stack as a user-level library to enable direct storage access, thus avoiding the inefficiencies of CPU-centric approaches. BaM enables GPUs to orchestrate high-throughput and fine-grained access requests without CPU orchestration overhead.

[0066] However, in a GPU-centric storage software stack, while each GPU process can directly manage and access NVMe devices, it loses the file management functionality provided by the host memory file system. Therefore, this approach shares the same problems as SPDK: exclusive use of NVMe devices, preventing sharing between CPUs and GPUs, and between GPUs. Furthermore, the lack of file system management capabilities makes it impossible to guarantee system integrity and achieve file data persistence. Additionally, when accessing data managed by the host file system, memory copying is still unavoidable.

[0067] In accelerator-centric solutions, the accelerator directly manages the storage devices. However, storage devices in this approach lack file abstraction and management capabilities from the host file system. For example, they cannot achieve fine-grained isolation and access control, which prevents accelerator processes from efficiently utilizing NVMe storage devices and makes it difficult to meet the data sharing needs of training or inference scenarios.

[0068] To address the problems existing in the above solutions, this application provides a data processing method. This method can be applied to a data processing system, which can be used to process file data or control data. As shown in Figure 5, the data processing system may include a processor and an accelerator. The processor can interact with the host to obtain file data or operation data from the host. For example, the processor can obtain file data from the host file system, or it can obtain operation data from an operation request initiated by the host. Specifically, taking the processor's interaction with the host file system as an example, it can call the host file system's interface to allocate file data of a specified length; for example, the processor may be a CPU. The processor can also generate metadata based on the file data, which can be shared between the processor and the accelerator. The processor can also interact with the accelerator to read the metadata into the accelerator. The accelerator may include memory. It can obtain metadata from the processor and, based on this metadata, initiate access requests to the storage device, thereby storing the file data generated by the accelerator in the storage device. For example, the accelerator can also be called an accelerator card, specifically a GPU. The storage device can be a storage device for storing file data, such as an NVMe device, or a storage device for storing operational data, such as the host's dynamic random access memory (DRAM). Operation instructions sent by the host can be stored in this storage device, for example, in the virtual address of the storage device, which can be mapped to the physical address of the storage device. In this embodiment, data acceleration is centered on the accelerator, allowing the accelerator to directly submit access requests to the storage device. The interaction between the accelerator and the storage device completely bypasses the processor, thus reducing access latency. Furthermore, the accelerator maps the storage device address according to the metadata allocated by the processor, enabling the processor and accelerator to share the storage device and enabling the processor to manage file data. In addition, the accelerator maps the addresses of storage devices according to metadata, which enables the accelerator's file management function and allows file data to be persistently stored.

[0069] In this embodiment of the application, the data processing method executed by the processor includes: obtaining first data from a host; obtaining metadata describing the first data, the metadata being shared between the processor and the accelerator; and sending a first instruction to the accelerator, the first instruction including the metadata.

[0070] In this embodiment of the application, the data processing method executed by the accelerator includes: obtaining metadata according to a first instruction from a processor, the metadata being shared between the processor and the accelerator; determining the address of a storage device according to the metadata; and sending a second access request to the storage device, the second access request including the address of the storage device.

[0071] In the above data processing method, the accelerator can access and interact with the storage device based on the metadata sent by the processor, thereby completing the processing of the first data. This first data can be either first file data or first operation data; there is no specific limitation here. The choice between the processor-generated metadata describing the first file data or the first operation data can be determined based on the application scenario.

[0072] Please refer to Figure 6, which is a schematic diagram of the interaction flow between a processor and an accelerator provided in an embodiment of this application. The processor executes steps 601 to 603, and the accelerator executes steps 603 to 606. A detailed explanation is given using the example of first file data as the first data and metadata used to describe the first file data. Scenarios where metadata describes first operation data will not be described in detail. The processor can interact with the host file system, the accelerator, and the storage device. The accelerator can interact with the processor and the storage device. Specifically, the storage device can be a device with file data storage capabilities, such as an NVMe device. The main processes include the following:

[0073] 601. The processor retrieves the first file data from the host file system.

[0074] The processor can call the host file system interface to allocate file data of a specified length from the host file system. This specified length of file data can be referred to as the first file data.

[0075] For example, the accelerator can call the interface provided by the library file to create the specified file if it does not exist, and the processor can obtain the first file data from the host file system according to this interface. The type and content of the first file data need to be determined based on the specific application scenario.

[0076] 602. The processor acquires metadata describing the first file data, which is used for sharing between the processor and the accelerator.

[0077] After determining the first file data, the processor retrieves metadata describing the first file data. This metadata is generated by the processor based on the first file data, and therefore can be used to describe the first file data. Moreover, this metadata is shared by the processor and the accelerator; not only the processor but also the accelerator can use it. Metadata can also be called file system metadata. This metadata can be managed on the processor and shared with the accelerator, allowing the accelerator to directly commit storage.

[0078] In this embodiment, the metadata obtained by the processor can be shared between the processor and the accelerator. The processor can determine the content included in the metadata according to the specific scenario. In some embodiments of this application, the metadata includes at least one of the following: the size of the first file data, access permissions, modification time, access time, and the mapping relationship between the virtual file address of the first file data and the physical address of the storage device.

[0079] The metadata may include data describing the first file data. For example, metadata may describe the size, access permissions, modification time, access time, and address mapping relationship of the first file data. This address mapping relationship may be a mapping between the virtual file address of the first file data and the physical address of the storage device. This metadata accurately describes the first file data, enabling the accelerator to access and interact with the storage device based on this metadata shared by the processor. In this embodiment, the metadata is not limited to including at least one of the following: file size, access permissions, modification time, access time, and address mapping relationship. The specific content of the metadata can be determined based on the application scenario, and is not limited here.

[0080] 603. The processor sends a first instruction to the accelerator, the first instruction including metadata.

[0081] After the processor obtains the metadata for sharing between the processor and the accelerator, the processor can interact with the accelerator and send metadata to the accelerator. For example, the processor sends a first instruction, which includes metadata. The implementation of the first instruction sent by the processor is not limited here.

[0082] 604. The accelerator obtains metadata based on the first instruction from the processor, and the metadata is used for sharing between the processor and the accelerator.

[0083] In this embodiment, the accelerator can receive a first instruction from the processor and obtain metadata. For example, the accelerator can read the metadata into its memory through a high-performance companion file system (GeminiFS) for accelerator cards. This metadata is allocated to the accelerator by the processor and is used for sharing between the processor and the accelerator.

[0084] 605. The accelerator determines the physical address of the storage device based on the metadata.

[0085] After obtaining the metadata, the accelerator performs address mapping between the metadata and the storage device to determine the physical address of the storage device. This physical address is the address where the accelerator accesses the storage device. For example, the accelerator can store cached file data in the physical address of the storage device.

[0086] 606. The accelerator sends a second access request to the storage device, the second access request including the physical address of the storage device.

[0087] In this process, the accelerator performs address mapping using metadata to determine the physical address of the storage device. Then, the accelerator sends a second access request to the storage device, which includes the physical address of the storage device. This allows the storage device to respond to the accelerator's second access request by storing and modifying file data. For example, this second access request could be an input / output request.

[0088] In some embodiments of this application, step 602 involves the processor acquiring metadata describing the first file data, including:

[0089] A1. The processor retrieves metadata from the first file data according to the file format of the accelerator virtual disk.

[0090] Among them, the accelerator virtual disk (AVDK) is a custom format that uses file system metadata embedding to enable efficient sharing of metadata between the processor and the accelerator. The accelerator parses the metadata to obtain the address mapping between file data and storage devices, thereby building a lightweight file system.

[0091] For example, the processor can load the AVDK kernel helper module into the kernel, and provide an interface to the application through the AVDK kernel helper module for reading file system metadata.

[0092] In other embodiments of this application, metadata can also be obtained from the first data according to the data format of the accelerator virtual disk. The data format of the accelerator virtual disk may include the aforementioned file format of the accelerator virtual disk.

[0093] In some embodiments of this application, in addition to executing the aforementioned data processing method, the processor may also execute the following method steps:

[0094] B1. The processor performs file format conversion on the first file data and metadata according to the file format of the accelerator virtual disk to obtain the second file data, in which metadata is embedded.

[0095] In this embodiment, the processor can call the host file system interface to allocate a first file of a specified length and extend the file length as needed to accommodate the metadata to be embedded. The processor can selectively embed metadata, embedding only file-related metadata such as file size, access permissions, and address mapping relationships in the first file data. Directory-related metadata is still managed by the host, reducing the complexity of the file system design. In this embodiment, the accelerator virtual disk achieves metadata sharing between the processor and the accelerator by embedding second file data with metadata. The accelerator obtains the file data and address mapping by parsing the file, thereby obtaining usable disk space and building a lightweight file system within the accelerator. The file interface on the accelerator does not need to access the data managed by the processor, achieving efficient metadata sharing.

[0096] For example, the processor can load the AVDK kernel help module in the kernel to convert the first file data and metadata according to the file format of the accelerator virtual disk to obtain the second file data, in which metadata is embedded.

[0097] In other embodiments of this application, the first file data and metadata may be converted to a file format according to the data format of the accelerator virtual disk. The data format of the accelerator virtual disk may include the aforementioned file format of the accelerator virtual disk.

[0098] In some embodiments of this application, in addition to executing the aforementioned data processing method, the processor may also execute the following method steps:

[0099] C1. The processor sends a first access request to the storage device, which processes the first access request and a second access request. The second access request is sent by the accelerator.

[0100] Both the processor and the accelerator can access and interact with the storage device, enabling the storage device to handle multiple access requests simultaneously. For example, the processor sends a first access request to the storage device, and the accelerator sends a second access request to the storage device. The storage device then processes the first and second access requests, enabling multiple devices, including the processor and accelerator, to share the storage device.

[0101] For example, current user-mode memory expansion methods cannot achieve host sharing. In this embodiment, the processor and accelerator can share metadata, so the processor and accelerator can send their respective access requests to the storage device in parallel, thereby realizing multi-device sharing of the storage device by the processor and accelerator.

[0102] In some embodiments of this application, step 603, in which the processor sends a first access request to the storage device, includes:

[0103] D1. The processor sends a first access request to the storage device through a parallel storage queue between the processor and the storage device. The parallel storage queue is also used by the accelerator to send a second access request to the storage device.

[0104] The processor can establish a parallel memory queue, which can be used by the processor to send a first access request and by the accelerator to send a second access request. This parallel memory queue enables multiple devices, including the processor and accelerator, to share the same memory.

[0105] For example, a parallel storage queue can be a queue pair (QP) in the NVMe protocol, meaning a parallel storage queue can be an NVMe QP. NVMe QPs can be built on the device memory of multiple accelerators, breaking the isolation of NVMe devices and enabling storage device sharing.

[0106] In some embodiments of this application, step 601, in which the processor obtains the first file data from the host file system, includes:

[0107] E1. The processor retrieves the first file data from the host file system based on the checkpoint's operating frequency.

[0108] Model checkpoints are a crucial step in the deep learning model training process. The host file system stores the frequency of checkpoint execution. Based on the triggering frequency of checkpoints, the processor retrieves the first file data from the host file system. This allows the processor to send metadata to the accelerator, which then generates checkpoint data. The accelerator can then map the physical addresses of the data to the storage device according to the metadata allocated by the processor. Because the processor can retrieve the first file data from the host file system based on the checkpoint execution frequency, the access latency for storing model checkpoint data to the storage device is reduced.

[0109] In some embodiments of this application, step 606, in which the accelerator sends a second access request to the storage device, includes:

[0110] F1. The accelerator allocates a parallel storage queue in the accelerator's memory, and the parallel storage queue is also used by the processor to send the first access request to the storage device.

[0111] F2. The accelerator writes the second access request into the parallel storage queue.

[0112] Specifically, the accelerator can allocate a parallel storage queue in its memory. This parallel storage queue can be used by the processor to send a first access request, and it can also be used by the accelerator to send a second access request. The parallel storage queue enables multiple devices, including the processor and accelerator, to share storage.

[0113] In some embodiments of this application, the parallel storage queue is also used to store the first checkpoint data read by the device from the accelerator.

[0114] The processor retrieves the first file data from the host file system based on the frequency of checkpoint execution. This allows the processor to send metadata to the accelerator, which in turn generates checkpoint data and maps the physical addresses of the data to the storage device according to the metadata allocated by the processor. Because the processor can retrieve the first file data from the host file system based on the checkpoint execution frequency, the access latency for storing model checkpoint data to the storage device is reduced.

[0115] In some embodiments of this application, step 606, in which the accelerator sends a second access request to the storage device, includes:

[0116] G1. The accelerator sends a second access request to the storage device through the accelerator's computing core.

[0117] The accelerator may include multiple computing cores, which send a second access request to the storage device, thereby refining the granularity of the access request submission. In this embodiment, the second access request can be submitted directly through the accelerator's computing cores, and high bandwidth utilization can be achieved even with a small access granularity by leveraging the high parallelism within the accelerator.

[0118] In some embodiments of this application, the accelerator can fully utilize the device's internal bandwidth to improve memory access performance, using an accelerator-friendly page cache, whose theoretical bandwidth can reach over 600GB / s. Currently, the accelerator's PCIe bandwidth does not exceed 128GB / s; therefore, the page cache will not become a system bottleneck.

[0119] In some embodiments of this application, intermediate parameters for training the model are stored in the accelerator's memory;

[0120] Step 606: The accelerator sends a second access request to the storage device, including:

[0121] H1. The accelerator sends a second access request to the storage device through the accelerator's read / write interface in order to move the intermediate parameters of the training model to the storage device.

[0122] In scenarios where intermediate parameters for model training need to be unloaded, the intermediate parameters used for training the model are stored in the accelerator's memory. The accelerator can access and interact with the storage device through a read / write interface, so that the storage device can move the intermediate parameters of the training model to the storage device in response to the second access request. In this embodiment, the accelerator's read / write interface is used to replace the memory access interface for forward and backward propagation in the training model, so as to realize the rapid movement of the intermediate parameters of the training model and reduce the memory usage of the accelerator.

[0123] In some embodiments of this application, step 606, in which the accelerator sends a second access request to the storage device, includes:

[0124] J1. The accelerator sends a second access request to the storage device through the first thread in the accelerator's parallel thread combination, wherein multiple threads in the parallel thread combination use the same lock.

[0125] The accelerator can include multiple threads, which can form a parallel thread combination. The accelerator sends a second access request to the storage device through the first thread in the parallel thread combination. The multiple threads in the parallel thread combination use the same lock. That is, the accelerator performs lock waiting at the granularity of the parallel thread combination. The multiple threads in the parallel thread combination use the same lock, and there is no waiting between threads within the parallel thread combination, thereby achieving a higher level of access efficiency than the single thread of the processor and reducing the access latency to the storage device.

[0126] As illustrated by the foregoing embodiments, in this application embodiment, the processor can interact with the host file system to allocate first file data. The processor can also generate metadata based on this first file data, which can be shared between the processor and the accelerator. The processor can also interact with the accelerator to read this metadata into the accelerator. The accelerator can obtain the metadata from the processor and, based on this metadata, initiate an access request to the storage device, thereby storing the file data generated by the accelerator in the storage device. In this application embodiment, data acceleration is accelerator-centric, allowing the accelerator to directly submit access requests to the storage device. The interaction process between the accelerator and the storage device completely bypasses the processor, thus reducing access latency.

[0127] To facilitate a better understanding and implementation of the above-described solutions in the embodiments of this application, specific examples of corresponding application scenarios are provided below.

[0128] This embodiment uses a lightweight accelerator card file system that coexists with the host file system. Therefore, file system metadata can be managed on the CPU and shared with the accelerator card. This allows the accelerator card to directly submit data to the storage device.

[0129] In this embodiment, the accelerator's storage software stack has the following characteristics: It is accelerator-centric, allowing I / O requests to be submitted directly within the accelerator's processes, completely bypassing the CPU and minimizing software overhead. It provides file system management, enabling disk space management capabilities. It provides a unified namespace for both the CPU and the accelerator, allowing both to open and directly read / write files. All files accessed by the accelerator can be managed by the CPU, such as creation, moving, and deletion, which aligns with the accelerator's programming paradigm, such as the hybrid C++ and CUDA programming used by GPUs.

[0130] Figure 7 illustrates a schematic diagram of a CPU and GPU connected to an NVMe device via a PCIe interface, according to an embodiment of this application. The CPU, GPU, and NVMe device are connected via a PCIe interface. The CPU may include CPU memory, which can be divided into user mode and kernel mode. The user mode records CPU runtime. The kernel mode includes an AVDK module, a file system, and a block layer. The file system includes metadata and AVDK files. The CPU memory uses shared NVMe and includes an NVMe control block, which includes an Admin queue (QP) and input / output queues (IO QPs).

[0131] GPU memory can include GPU application layers, LibGemini library files, GPU storage volume layer, page cache, and NVMe control block. The GPU storage volume layer can include a header, multiple data clusters, and NVMe offsets. The header includes: an information header (Info), mapping relationships, and NVMe offsets. The NVMe control block includes an I / O queue driver.

[0132] As shown in Figure 7, this embodiment of the application provides a high-performance companion file system (GeminiFS) for accelerators, offering a set of file system interfaces to support direct file-based access to storage devices hosted in the host file system. GeminiFS achieves file system metadata synchronization between the CPU and the accelerator by directly embedding metadata into the file data. This embodiment also extends the NVMe driver to allow the CPU and accelerator to set control paths for storage devices in parallel. Furthermore, GeminiFS provides a hardware-friendly software-defined page cache to fully utilize the accelerator's internal bandwidth. Finally, GeminiFS provides the LibGemini library to accelerator applications, which abstracts various low-level complexities, thereby reducing high programming complexity.

[0133] The accelerator provided in this application embodiment can also be an NPU chip or a DPU chip, which can serve as a high-speed storage software stack for the NPU or DPU.

[0134] This application provides an accelerator-based data processing system that offers five POSIX-like interfaces through the LibGemini library to enable fast access to data managed by the host file system. Here, POSIX-like refers to a Unix-like operating system conforming to the POSIX standard. The five interfaces are described below:

[0135] For example, the first interface provided by the LibGemini library file is int Geminifs_init(char*dev_path,char*GPU_ids,int Q_num), which initializes the system, including shared NVMe driver settings, by calling Geminifs_init.

[0136] The second interface provided by the LibGemini library is `dev_fd G_open(char* path, uint16 flag, uint64_tcache_capacity, int page_size)`. The accelerator process calls the `Gopen` interface provided by LibGemini. If the interface contains `O_CREATE`, it calls the host file system interface to allocate a file of the specified length and extends the file length as needed to accommodate the embedded metadata. The `Gopen` interface calls the AVDK module to obtain the mapping table of virtual file offsets to NVMe device physical addresses from the host file system, converts it into a flat address mapping defined by the AVDK file format, and then writes it into the file data. Here, the AVDK module refers to the kernel module that implements AVDK file format conversion; for example, this AVDK module can also be called the AVDK helper kernel module.

[0137] Additionally, the Gopen interface allows you to set the O_DIRECT flag to indicate whether a page cache should be used. If O_DIRECT is not set, a page cache will be established, with the file cache size and page size determined by the `cache_capacity` and `page_size` parameters in Gopen, respectively. After completing this process, GeminiFS reads the file's metadata into the GPU's memory and provides the GPU memory address, abstracted as `dev_fd`. The read / write interface can access the page cache by resolving the `dev_fd`.

[0138] The third interface provided by the LibGemini library is `int G_read / G_write(dev_fd fd, void *buf, uint64_t offset, size_t nbyte)`. GeminiFS provides two read / write interfaces, `G_read` and `G_write`, both of which contain four parameters: `dev_fd`, `buf`, `offset`, and `nbyte`, identical to the standard `read / write`. Initially, the interface retrieves the metadata based on `dev_fd` and determines the access's validity, including the access mode, and checks if the offset and `nbyte` fall within the file boundaries. If the request is valid, the file offset is translated into the NVMe device's physical address using an embedded address mapping. Subsequently, the NVMe I / O queue driver is invoked to generate an NVMe I / O request on the GPU side, which is then written to the NVMe I / O queue. The queue doorbell then notifies the NVMe device to process the I / O request.

[0139] The fourth interface provided by the LibGemini library is `int G_sync(dev_fd fd)`. GeminiFs provides an accelerator-side interface function `G_sync` to ensure that modified file system metadata and cached file data are written to the file data, guaranteeing data persistence. The accelerator explicitly calls `G_sync` to ensure file consistency.

[0140] The fifth interface provided by the LibGemini library is `int G_close(dev_fd fd)`. GeminiFS provides an accelerator-side `G_close` interface function. This function takes `dev_fd` as its input parameter. Similar to the Posix `close` interface, this interface locates the file's metadata and page cache in the GPU. After ensuring that all modified data is written to the NVMe device, it releases the memory resources allocated on the accelerator. Additionally, this interface calls the host file system's `close` interface to synchronize file metadata such as last access time and last modification time on the host side.

[0141] The following describes the model checkpointing based on Geminifs provided in this application. Model checkpointing is a crucial step in the training process of deep learning models. When training large models, due to the large number of model parameters, the training cycle is often long. To prevent the loss of training progress due to unexpected situations (such as power outages, system crashes, etc.), the current state of the model is saved periodically during training; this process is called checkpointing. The checkpoint file contains the model's weight parameters, the optimizer's state (such as learning rate, momentum, etc.), and the current training progress (such as the number of training epochs completed). These files are important archive points in the model training process, allowing training to resume from the most recent checkpoint when training is interrupted, thereby avoiding starting from scratch and greatly saving time and computing resources.

[0142] The current checkpointing process is as follows: During training, the system performs checkpoints at a predetermined frequency, such as after a certain number of training rounds. Each checkpoint first calls the host file interface to allocate a file, then copies the model data from the GPU high bandwidth memory (HBM) to the CPU dynamic random access memory (DRAM) using memory copying, and finally calls the system to write the data to disk. The path for reading the checkpoint is the reverse. In contrast, the GDS solution does not require data copying between the CPU DRAM and GPU HBM during checkpointing. Instead, it calls the interface provided by the accelerator software stack to register the GPU HBM, establish a PCIe passthrough path, and then calls the I / O interface provided by GDS to achieve passthrough access between the NVMe device and the GPU HBM.

[0143] Figure 8 illustrates an example of a model checkpointing process based on GeminiFS. Since the deep learning model design is fixed before the system allows it, and the checkpoint size is fixed, GeminiFS can be used to accelerate this process. In this embodiment, the accelerator first needs to call GeminiFS_init to initialize the system, allocate an NVMe queue in GPU memory, and then set up a shared NVMe (SNVMe) driver to enable the CPU and accelerator to access data on the same NVMe device. Then, the accelerator calls Gopen to create the required checkpoint file based on the system checkpoint frequency and embeds metadata. Finally, the accelerator executes checkpoints by simply calling G_read / G_write with the GPU Buffer as the cache parameter, enabling fast data write-to-disk.

[0144] In some embodiments of this application, the following describes a scenario of offloading intermediate parameters during model training based on GeminiFS. As the number of model parameters increases, large language models such as DeepseekV3 and Llama-405B require hundreds of gigabytes (GB) of memory, and during training, the memory requirement reaches petabytes (PB). Furthermore, model sizes are expected to continue growing rapidly in the foreseeable future. This necessitates the aggregation of numerous accelerators to meet the model's memory demands. To reduce system costs, model parameters are offloaded to storage devices, reducing the number of accelerators required. However, compared to the internal bandwidth of the accelerator (TB / s), the bandwidth between the accelerator and the storage device is limited by PCIe bandwidth, for example, 4.064 GB / s. Therefore, it is necessary to reduce the data transfer latency between the storage device and the accelerator computing cores, and to fully utilize bandwidth to prevent memory access from becoming a computational bottleneck. GPU model computation typically employs parallel models such as data parallelism and operator (tensor) parallelism, which greatly increases the concurrency of memory I / O. However, current CPU-centric accelerator memory software stacks are limited by the number of CPUs and I / O contention during processing, resulting in a decline in system performance.

[0145] The GeminiFS provided in this application embodiment can submit I / O requests via accelerator computation cores. By utilizing the accelerator's internal bandwidth, I / O parallelism is improved, and shorter I / O control and data paths minimize I / O latency. The offloading of intermediate parameters during model training based on GeminiFS is shown in Figure 9. For example, intermediate parameters during model training may include: output parameters for each layer, model weights, errors, gradients, and activation parameters for each layer's output. The read / write interface of GeminiFS replaces the memory access interfaces for forward propagation (FP) and back propagation (BP) during training. Offloading occurs when calculating the parameters of each intermediate activation layer, reducing the required memory capacity. Furthermore, by inheriting GeminiFS's Pagecache, the high bandwidth of HBM can be fully utilized, and I / O access can be reduced, further improving I / O performance.

[0146] The solution provided in this application can be applied to a many-core architecture data processing unit (DPU) to improve the availability of the DPU under resource constraints, and to achieve efficient local and remote load offloading and invocation methods.

[0147] Figure 10 illustrates the GPT2-124M training performance test results. (a) shows a comparison of checkpoint performance, and (b) shows a comparison of intermediate parameter unloading performance. Under real GPT2 load, the performance of Geminifs versus native (copy-write) and GDS in checkpoint disk write-to-disk (124M) and intermediate parameter unloading (e.g., 50GB) was compared. In (a), the checkpoint disk write latency was reduced by 85% and 50% compared to native and GDS, respectively. In (b), with intermediate parameters unloaded, the total training time was reduced by 94.5% and 91% compared to native and GDS, respectively, and only increased by 4 times compared to keeping intermediate parameters in HBM. The gap can be further narrowed by increasing the number of NVMe disks. The performance improvement mainly comes from the reduction in I / O bandwidth, the reduction in I / O software stack overhead, and the elimination of communication overhead between GPU kernel functions and CPU processes.

[0148] In this embodiment, the GPU memory access interface does not need to access data managed by the CPU, achieving efficient metadata sharing. Comparing the read bandwidth and latency of GPUfs, GDS, and the Bam solution at small I / O granularity (4K), this embodiment shows that GeminiFS has higher bandwidth across different thread counts compared to GPUfs, with an average bandwidth 7.33 times that of GPUfs. When the number of threads reaches 1024, GeminiFS reaches the peak bandwidth of the NVMe device. As the number of GPU threads increases to 1024, the bandwidth of the GeminiFS system reaches 6.2 times that of GDS. Compared to Bam, GeminiFS performs similarly at different levels of parallelism. However, GeminiFS accesses the storage device through a file interface, resulting in higher availability. Regarding latency, compared to GPUfs, GeminiFS achieves a latency reduction of 79.6%-90.9% across different thread counts. Compared to GDS, when the number of threads is 1-8, GeminiFS latency is 57.2% higher than GDS. However, the average latency begins to decrease as parallelism increases.

[0149] Currently, when CPU-centric storage software performs checkpoints, data transfer between the accelerator and storage device relies on the CPU to organize data transfer. This lengthens the I / O control path and increases I / O latency. The GeminiFS provided in this application reduces the frequent communication overhead between the GPU and CPU. On the other hand, data copying between the accelerator HBM and CPU DRAM further increases computational latency. Although GDS can avoid memory copying, the overhead of the complex and redundant storage software stack is still significant. The GeminiFS provided in this application achieves pass-through between the accelerator and storage device through a shared NVMe device, with minimal software overhead.

[0150] Furthermore, current CPU-centric storage software struggles to fully utilize PCIe bandwidth during intermediate parameter unloading for training due to limited CPU parallelism. In contrast, the GeminiFS embodiment provided in this application can directly submit I / O requests via accelerator cores, leveraging the accelerator's internal capabilities to improve parallelism. It achieves high bandwidth utilization even with smaller I / O granularity. The ultra-high bandwidth of the page cache used in this embodiment also enhances system throughput.

[0151] First, it should be explained that the accelerator virtual disk provided in this application embodiment is mainly used to solve the problem of inefficient sharing of CPU and accelerator metadata. The CPU and accelerator are isolated from each other, and file system metadata, such as address mapping file size, access permissions, and directory-related metadata, is still managed by the host kernel. Accelerator access to host metadata involves complex software overhead (context switching, locking) and communication overhead. Metadata needs secure synchronization; every I / O operation involves metadata access and modification. If two systems use the same file system, locking overhead and synchronization issues need to be considered. Furthermore, data structures cannot fully leverage the parallelism of the accelerator. Taking the ext4 file system as an example, address mapping is organized in the form of an extended tree (red-black tree), and the software code involves complex control logic and branch jumps. This is unsuitable for implementation on hardware architectures like NPUs or GPUs, which are designed for simple logic and high parallel computing, reducing I / O efficiency and limiting parallelism.

[0152] To address the aforementioned issues, this application's embodiments, based on the workloads of large language models (LLM) and deep neural networks (DNN), observe the following fundamental characteristics: 1. Short-term data (intermediate training data) does not require persistence; 2. Long-term data (Checkpoint, KV Cache) is append-only; 3. The memory access patterns of most applications are predictable; 4. Data needs to be shared among multiple GPU processes (Checkpoint, KV Cache). Therefore, it can be concluded that the metadata allocation and management for most accelerator workloads are predictable.

[0153] Therefore, this application proposes an Accelerator Virtual Disk (AVDK). An AVDK kernel helper module is loaded into the kernel to provide an interface for applications to read file system metadata. Through selective metadata embedding, only file data-related metadata, such as file size, access permissions, and address mapping, is embedded in the file, while directory-related metadata remains managed by the host, reducing the complexity of the file system design. Furthermore, flat address mapping embedding is performed. GeminiFS embeds a two-level mapping table (L1, L2) in the file, similar to a memory management unit (MMU), converting the tree-like data structure in the file system into a flat address mapping. The accelerator only needs to perform a simple table lookup to achieve address translation, without needing to parse complex tree structures, such as the extent tree used by EXT4. AVDK achieves metadata sharing between the CPU and accelerator through file system metadata embedding. The accelerator parses the file to obtain file data and address mappings, thereby obtaining usable disk space and building a lightweight file system within the accelerator. The file interface on the accelerator does not need to access data managed by the CPU, achieving efficient metadata sharing.

[0154] In this embodiment, by examining kernel usage, it can be seen that when the accelerator directly accesses the storage device, a dedicated memory access interface is required to open local files. Opening the file increases its size due to metadata embedding. The accelerator storage software stack can use accelerator-friendly file formats.

[0155] The following describes the multi-device shared NVMe device driver provided in the embodiments of this application. Addressing the difficulty of simultaneously establishing independent control planes on the CPU and accelerator, the embodiments of this application employ a multi-device shared NVMe device driver. This driver supports the simultaneous establishment of NVMe device I / O queues on both the host and accelerator sides, allowing the host and multiple accelerators to submit NVMe requests simultaneously, thus enabling NVMe devices to be shared between the CPU and accelerator.

[0156] First, the current NVMe driver queue establishment process is explained. It begins by creating a pair of management queues: a Submission Queue (SQ) and its associated Completion Queue (CQ). These are used to manage the NVMe controller, such as creating and deleting I / O queues, and aborting commands. Next, the driver submits management commands to obtain the controller's data structure capabilities and settings, as well as to obtain the capabilities and settings of specific namespaces. Finally, the driver allocates an appropriate number of I / O SQs and CQs in CPU memory and uses management commands to register the queues' direct memory access (DMA) addresses with the NVMe device.

[0157] In the shared NVMe driver queue establishment provided in this embodiment, the shared NVMe driver first allocates memory for the I / O queue on the accelerator memory. Then, it calls the device driver accelerator virtual address to be converted to a DMA address, making it visible to the NVMe device. Additionally, the shared NVMe driver also registers the I / O queues already allocated in the accelerator memory. These queues do not require host interrupts to complete I / O operations; instead, they are polled by the accelerator thread.

[0158] In this embodiment, unlike the user-mode implementation, the detection method checks whether the NVMe is replaced during accelerator storage software stack initialization. Additionally, it analyzes NVMe management commands to check whether the DMA address used in the I / O queue registration command is within the accelerator address space. Through these methods, it can be determined that the accelerator storage software stack requires a customized NVMe driver to implement its functions.

[0159] The following describes the accelerator-friendly Pagecache provided in this embodiment. Currently, the Pagecache designed for CPU architecture in the kernel is difficult to share among multiple processes; the page cache can only be allocated within the process's memory space. When multiple processes open the same file, this leads to redundancy in the page cache and wasted memory. Furthermore, the current Pagecache design struggles to leverage the parallelism of accelerators. The Pagecache contains complex control logic, requiring locking of critical data structures to ensure data consistency. Accelerators offer higher parallelism than host threads, resulting in higher lock contention; implementing the same logic leads to reduced memory access parallelism. Therefore, this embodiment employs an accelerator-friendly Pagecache. To alleviate lock contention under high concurrency loads, this embodiment acquires pages at a higher parallel level rather than at the thread level like the CPU, reducing lock contention. Secondly, a combination of hash tables and doubly linked lists is used to manage the page cache. The hash table quickly checks if a page cache hit has occurred, while the doubly linked list collaboratively manages hot and cold pages, minimizing access time to critical sections. Specifically, the hash table stores the disk page cache; the hash algorithm takes the disk logical offset as input and outputs the hash table offset. The doubly linked list is the actual page cache, which is sorted according to the number of page references to determine hot pages and cold pages. Cold pages are then stored in batches from the page cache to the NVMe device.

[0160] In this embodiment, the number of I / O commands between the accelerator and the storage device is reduced compared to a scenario without page cache. Additionally, by analyzing the frequency of lock contention waiting in the profiling pipeline, it can be determined that lock waiting is observed at the warp granularity, with no waiting between threads within a warp.

[0161] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0162] To facilitate better implementation of the above-described solutions in the embodiments of this application, related apparatus for implementing the above-described solutions is also provided below.

[0163] As shown in Figure 11, this application embodiment provides a communication device, the communication device including a processor, the communication device comprising:

[0164] Data acquisition module 1101 is used to acquire first data from the host;

[0165] Metadata acquisition module 1102 is used to acquire metadata describing the first data, the metadata being shared between the processor and the accelerator;

[0166] The sending module 1103 is used to send a first instruction to the accelerator, the first instruction including the metadata.

[0167] As shown in Figure 12, this application embodiment provides a communication device, the communication device including an accelerometer, the communication device comprising:

[0168] Metadata acquisition module 1201 is used to acquire metadata according to a first instruction from the processor, the metadata being shared between the processor and the accelerator;

[0169] Address mapping module 1202 is used to determine the address of the storage device based on the metadata;

[0170] The sending module 1203 is used to send a second access request to the storage device, the second access request including the address of the storage device.

[0171] In this embodiment of the application, a communication device is also provided, including a processor and a memory, wherein the memory and the processor are coupled, and the processor is used to execute the method described by the processor in FIG6 above.

[0172] In this embodiment of the application, a communication device is also provided, including an accelerator and a memory, wherein the memory and the accelerator are coupled, and the accelerator is used to execute the method described by the accelerator in FIG6 above.

[0173] In this embodiment of the application, a computer-readable storage medium is also provided, including instructions that, when executed on a computer, cause the computer to perform the method described in FIG6.

[0174] In this application embodiment, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the method described in FIG6.

[0175] In this embodiment of the application, a chip is also provided, including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of the communication device and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the communication device performs the method described by the processor in FIG6 above.

[0176] In this embodiment of the application, a chip is also provided, including one or more interface circuits and one or more accelerators; the interface circuits are used to receive signals from the memory of a communication device and send the signals to the accelerators, the signals including computer instructions stored in the memory; when the accelerators execute the computer instructions, the communication device performs the method described by the accelerator in FIG6 above.

[0177] In this application embodiment, a communication device is also provided, the communication device including a processor and an accelerator;

[0178] The processor is used to execute the method described by the processor in Figure 6 above;

[0179] The accelerator is used to perform the method described in Figure 6 above.

[0180] It should be noted that the information interaction and execution process between the modules / units of the above-mentioned device are based on the same concept as the method embodiments of this application, and the resulting technical effects are the same as those of the method embodiments of this application. For details, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.

[0181] In this embodiment, the module is an example of a hardware functional unit, and the communication device may include at least one computer device, such as a server. Alternatively, the communication device may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex PLD (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0182] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0183] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0184] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0185] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0186] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0187] Finally, it should be noted that the above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method is applied to a processor, and the method includes: Retrieve the first data from the host; Obtain metadata describing the first data, the metadata being shared between the processor and the accelerator; Send a first instruction to the accelerator, the first instruction including the metadata.

2. The method according to claim 1, characterized in that, The step of obtaining metadata describing the first data includes: The metadata is obtained from the first data according to the data format of the accelerator virtual disk.

3. The method according to claim 2, characterized in that, The method further includes: The first data and the metadata are converted according to the data format of the accelerator virtual disk to obtain the second data, in which the metadata is embedded.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: A first access request is sent to a storage device, the storage device being used to process the first access request and a second access request, the second access request being sent by the accelerator.

5. The method according to claim 4, characterized in that, Sending the first access request to the storage device includes: The first access request is sent to the storage device via a parallel storage queue between the processor and the storage device, and the parallel storage queue is also used by the accelerator to send the second access request to the storage device.

6. The method according to any one of claims 1 to 5, characterized in that, The step of obtaining the first data from the host includes: The first data is obtained from the host based on the operating frequency of the checkpoint.

7. The method according to any one of claims 1 to 6, characterized in that, The metadata includes at least one of the following: the size of the first data, access permissions, modification time, access time, and the mapping relationship between the virtual file address of the first data and the address of the storage device.

8. A data processing method, characterized in that, The method is applied to an accelerator, and the method includes: Metadata is obtained according to a first instruction from the processor, and the metadata is used for sharing between the processor and the accelerator; The address of the storage device is determined based on the metadata; A second access request is sent to the storage device, the second access request including the address of the storage device.

9. The method according to claim 8, characterized in that, Sending the second access request to the storage device includes: A parallel storage queue is allocated in the memory of the accelerator, and the parallel storage queue is also used by the processor to send a first access request to the storage device. Write the second access request into the parallel storage queue.

10. The method according to claim 9, characterized in that, The parallel storage queue is also used by the storage device to read the first checkpoint data generated by the accelerator.

11. The method according to any one of claims 8 to 10, characterized in that, Sending the second access request to the storage device includes: The second access request is sent to the storage device via the computing core of the accelerator.

12. The method according to any one of claims 8 to 11, characterized in that, The accelerator's memory stores intermediate parameters used for training the model; Sending the second access request to the storage device includes: The second access request is sent to the storage device through the read / write interface of the accelerator to move the intermediate parameters of the training model to the storage device.

13. The method according to any one of claims 8 to 11, characterized in that, Sending the second access request to the storage device includes: A second access request is sent to the storage device by a first thread in the parallel thread combination of the accelerator, wherein multiple threads in the parallel thread combination use the same lock.

14. A communication device, characterized in that, The communication device includes a processor, the communication device comprising: The data acquisition module is used to acquire the first data from the host. A metadata acquisition module is used to acquire metadata describing the first data, the metadata being shared between the processor and the accelerator; A sending module is used to send a first instruction to the accelerator, the first instruction including the metadata.

15. A communication device, characterized in that, The communication device includes an accelerator, the communication device comprising: A metadata acquisition module is used to acquire metadata based on a first instruction from the processor, the metadata being shared between the processor and the accelerator; The address mapping module is used to determine the address of the storage device based on the metadata. The sending module is configured to send a second access request to the storage device, the second access request including the address of the storage device.

16. A communication device comprising a processor and a memory coupled to the processor, the processor being configured to perform the method according to any one of claims 1 to 7.

17. A communication device comprising an accelerator and a memory, the memory being coupled to the accelerator, the accelerator being configured to perform the method of any one of claims 8 to 13.

18. A computer-readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 13.

19. A computer program product comprising instructions that, when run on a computer, cause the computer to perform the method as described in any one of claims 1 to 13.

20. A chip, characterized in that, The device includes one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from the memory of the communication device and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the communication device performs the method of any one of claims 1 to 7.

21. A chip, characterized in that, The device includes one or more interface circuits and one or more accelerators; the interface circuits are configured to receive signals from the memory of the communication device and send the signals to the accelerators, the signals including computer instructions stored in the memory; when the accelerators execute the computer instructions, the communication device performs the method of any one of claims 8 to 13.

22. A communication device, characterized in that, The communication device includes a processor and an accelerator; The processor is configured to perform the method according to any one of claims 1 to 7; The accelerator is used to perform the method according to any one of claims 8 to 13.

Images