A data caching method, system, product, device and storage medium

Abstract

The present invention provides a data caching method, system, product, device and storage medium, which relates to the field of data processing technology and is applied to a graphics processor, including: obtaining a target computing task that needs to be executed by a graphics processor in the current to-be-executed task sent by a host; the graphics processor pre-establishes a connection with a memory pool, and the memory pool contains a shared first memory partition and a second memory partition exclusively used by the graphics processor; determining the data type of each first to-be-cached data when executing the target computing task; determining the corresponding target memory partition based on the data type; caching the first to-be-cached data to the corresponding target memory partition, so as to complete the subsequent computing process using the cached data. In this way, the graphics processor of the present application can implement data caching through a mounted memory pool, and can flexibly cache data to different memory partitions in combination with the data type and memory partition set, which can improve data exchange performance and reduce data latency.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data caching method, system, product, device and storage medium. Background Art

[0002] The current video memory capacity is completely limited by the design of the video memory controller of the GPU (Graphics Processing Unit). A large amount of cache data is required for inference calculations. Due to the GPU's configuration and hardware limitations, customers cannot flexibly expand the video memory capacity, and increasing the number of GPUs will increase costs and reduce economic benefits. In addition, during GPU computing, when you want to obtain cache data from another GPU, you can use the GPUDirect P2P (Peer-to-Peer) technology (a technology used for high-speed communication between stand-alone GPUs) to use the characteristics of the PCIe (Peripheral Component Interconnect express, a high-speed serial computer expansion bus standard) bus to achieve direct data transmission between GPUs. In a system that supports GPUDirect P2P, GPUs can communicate directly through the PCIe bus without passing through the host memory. For example, video memory data is uploaded to PCIE_Switch (switch) through GPU0, and forwarded to GPU1's video memory through PCIE_Switch, which can significantly reduce the delay of data transmission and increase the transmission bandwidth. However, in actual cross-domain transmission, it will be affected by PCIE link loss, and the delay will reach several milliseconds or even more than ten milliseconds.

[0003] It can be seen that how to expand the video memory capacity and improve the data exchange performance is a problem that those skilled in the art need to solve. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a data caching method, system, product, device and storage medium, which can solve the problem of being unable to effectively expand video memory and having low data exchange performance.

[0005] To solve the above technical problems, in a first aspect, the present application provides a data caching method, which is applied to a graphics processor, comprising:

[0006] Acquire a target computing task that needs to be executed by a graphics processor from among the currently executed tasks sent by the host; the graphics processor preliminarily establishes a connection with a memory pool, wherein the memory pool includes a first memory partition shared by the host and the graphics processor and a second memory partition exclusively used by the graphics processor;

[0007] Determine the data type of each first to-be-cached data when executing the target computing task;

[0008] Determine, based on the data type, a target memory partition corresponding to each of the first to-be-cached data from a preset memory partition set; wherein the preset memory partition set includes the first memory partition, the second memory partition, and a third memory partition located in the local memory of the graphics processor, and different data types correspond to different memory partitions;

[0009] The first data to be cached is cached in the corresponding target memory partition, so as to complete the subsequent calculation process of the current task to be executed by using the cached data in the target memory partition.

[0010] Optionally, the determining of the data type of each first to-be-cached data when executing the target computing task includes:

[0011] Determine at least one target parameter corresponding to each first data to be cached in the process of executing the target computing task, and perform weighted calculation based on each target parameter corresponding to each first data to be cached and a corresponding weight to obtain a weighted result corresponding to each first data to be cached;

[0012] Each of the first data to be cached is classified based on the weighted result to determine the data type of each of the first data to be cached.

[0013] Optionally, the determining at least one target parameter corresponding to each first to-be-cached data in the process of executing the target computing task includes:

[0014] In the process of executing the target computing task, the task level, parameter model size and matrix operation method corresponding to each first cached data are determined as the target parameters.

[0015] Optionally, the data type of the first data to be cached includes:

[0016] first type of data to be cached that requires data transfer between the host and the graphics processor during the execution of the target computing task, second type of data to be cached whose data dependency level with the target computing task satisfies a first preset low dependency condition during the execution of the target computing task, and third type of data to be cached whose data dependency level with the target computing task satisfies a first preset high dependency condition during the execution of the target computing task;

[0017] Among them, the target memory partitions corresponding to the first type of data to be cached, the second type of data to be cached, and the third type of data to be cached are the first memory partition, the second memory partition, and the third memory partition, respectively.

[0018] Optionally, the number of the graphics processors is one or more, the host establishes a connection with each of the graphics processors through a first switch, and each of the graphics processors establishes a connection with the memory pool through a second switch;

[0019] Accordingly, the data caching method further includes:

[0020] A corresponding number of memory partitions are created in the memory pool according to the number of the graphics processors to obtain second memory partitions corresponding to each of the graphics processors, so that the graphics processor can send the second type of cached data generated when executing the target computing task to the corresponding second memory partition for caching.

[0021] Optionally, the first switch is a switch based on a high-speed serial computer expansion bus standard, and the second switch is a switch based on a computing fast link.

[0022] Optionally, the method further includes:

[0023] Mounting the memory pool to the second switch through a preset expander;

[0024] Allocate a corresponding memory address space for the memory pool through the second switch, so as to divide the memory pool into a first memory partition shared by the host and the graphics processor, a second memory partition exclusively used by the graphics processor, and a fourth memory partition exclusively used by the host, and mount the first memory partition and the fourth memory partition to the host;

[0025] Wherein, the preset memory partition set also includes the fourth memory partition.

[0026] Optionally, the method further includes:

[0027] When the host sends the current task to be executed to the graphics processor, the host determines the data type of the second data to be cached during the processing of the current task to be executed, and determines the target memory partition corresponding to each second data to be cached from the preset memory partition set according to the data type of the second data to be cached, and then caches the second data to be cached to the corresponding target memory partition;

[0028] The second to-be-cached data of different data types correspond to different memory partitions.

[0029] Optionally, the data type of the second data to be cached includes:

[0030] a fourth type of data to be cached that requires data transfer between the host and the graphics processor during the processing of the current task to be executed, a fifth type of data to be cached that has a data dependency level with the current task to be executed that satisfies a second preset low dependency condition during the processing of the current task to be executed, and a sixth type of data to be cached that has a data dependency level with the current task to be executed that satisfies a second preset high dependency condition during the processing of the current task to be executed;

[0031] Among them, the target memory partitions corresponding to the fourth category of data to be cached, the fifth category of data to be cached, and the sixth category of data to be cached are the first memory partition, the fourth memory partition, and the fifth memory partition, respectively, and the fifth memory partition is the local memory of the host.

[0032] Optionally, the number of the hosts is one or more;

[0033] Wherein, the data caching method further includes:

[0034] Creating a corresponding number of memory partitions in the memory pool according to the number of the hosts, so as to obtain the fourth memory partitions corresponding to each of the hosts, so that the hosts can send the fifth type of data to be cached in the processing of the current task to be executed to the corresponding fourth memory partition for caching;

[0035] Correspondingly, mounting the first memory partition and the fourth memory partition to the host includes:

[0036] The first memory partition is mounted to each of the hosts through the second switch, and each of the fourth memory partitions is mounted to a corresponding host respectively.

[0037] Optionally, the method further includes:

[0038] Using the first memory partition to respond to a data access request initiated by a target data access party, so as to transfer data in the first memory partition corresponding to the data access request to the target data access party;

[0039] The target data access party is any of the host or any of the graphics processors, and the data corresponding to the data access request includes data cached in the first memory partition by any of the host and / or any of the graphics processors.

[0040] In a second aspect, the present application provides a data cache system, including a host, a memory pool, and a graphics processor pre-connected to the host and the memory pool;

[0041] The graphics processor implements the following data caching method by executing a computer program:

[0042] Obtain a target computing task that needs to be executed by the graphics processor in the current task to be executed sent by the host; the memory pool includes a first memory partition shared by the host and the graphics processor and a second memory partition exclusively used by the graphics processor; determine the data type of each first cached data when executing the target computing task; determine the target memory partition corresponding to each first cached data from a preset memory partition set based on the data type; wherein the preset memory partition set includes the first memory partition, the second memory partition and a third memory partition located in the local memory of the graphics processor, and different data types correspond to different memory partitions; cache the first cached data to the corresponding target memory partition, so as to use the cached data in the target memory partition to complete the subsequent computing process of the current task to be executed.

[0043] Optionally, there are one or more graphics processors, and there are one or more hosts. The hosts are connected to each of the graphics processors via a first switch, and each of the graphics processors and each of the hosts are connected to the memory pool via a second switch.

[0044] The first switch is a switch based on a high-speed serial computer expansion bus standard, and the second switch is a switch based on a computing fast link.

[0045] In a third aspect, the present application provides an electronic device, including:

[0046] Memory for storing computer programs;

[0047] A processor is used to execute the computer program to implement the steps of the above data caching method.

[0048] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned data caching method are implemented.

[0049] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned data caching method when executed by a processor.

[0050] It can be seen that the present application is applied to a graphics processor, firstly obtaining the target computing task that needs to be executed by the graphics processor in the current task to be executed sent by the host; the graphics processor pre-establishes a connection with a memory pool, the memory pool contains a first memory partition shared by the host and the graphics processor and a second memory partition exclusively used by the graphics processor; then determines the data type of each first cached data when executing the target computing task; then determines the target memory partition corresponding to each first cached data from the preset memory partition set based on the data type; wherein the preset memory partition set includes the first memory partition, the second memory partition and the third memory partition located in the local memory of the graphics processor, and different data types correspond to different memory partitions; then caches the first cached data to the corresponding target memory partition, so as to use the cached data in the target memory partition to complete the subsequent computing process of the current task to be executed. In this way, in the present application, the graphics processor can realize data caching through the mounted memory pool, and can flexibly cache data to different memory partitions in combination with the data type and the memory partition set, which can improve data exchange performance and reduce data latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 A flow chart of a data caching method disclosed in this application;

[0053] Figure 2 A schematic diagram of the structure of a graphics processor disclosed in this application;

[0054] Figure 3 A schematic diagram of a specific graphics processor structure disclosed in this application;

[0055] Figure 4 A schematic diagram of the structure of a data cache system disclosed in this application;

[0056] Figure 5 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] The terms "including" and "having" in the specification of the present invention and the above-mentioned drawings, as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.

[0059] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0060] like Figure 1 As shown, the embodiment of the present application discloses a data caching method, which is applied to a graphics processor, including:

[0061] Step S11, obtaining a target computing task that needs to be executed by a graphics processor from among the current tasks to be executed sent by the host; the graphics processor preliminarily establishes a connection with a memory pool, wherein the memory pool includes a first memory partition shared by the host and the graphics processor and a second memory partition exclusively used by the graphics processor.

[0062] In the embodiment of the present application, the graphics processor establishes a connection with the memory pool in advance, and the memory pool is pre-divided into a first shared memory partition for the host and the graphics processor and a second memory partition exclusively used by the graphics processor; it can be understood that the exclusive second memory partition can only be read and written by the graphics processor, while the shared first memory partition is a memory partition that can be read and written by both the host and the graphics processor. Furthermore, the current task to be executed is first obtained by the host, and the target computing task that needs to be executed by the graphics processor is sent to the graphics processor by the host, so that the corresponding target computing task is executed by the graphics processor.

[0063] Step S12: Determine the data type of each first data to be cached when executing the target computing task.

[0064] In the embodiment of the present application, in the process of executing the target computing task through the graphics processor, a number of computing results, namely, first data to be cached, are generated. These data need to be temporarily cached, and some of the data may be used for subsequent computing processes. In order to cache the first data to be cached generated by these computing processes, the graphics processor here first needs to determine the data type of each first data to be cached.

[0065] In a specific embodiment, the determination of the data type of each first data to be cached when executing the target computing task may include: determining at least one target parameter corresponding to each first data to be cached in the process of executing the target computing task, and performing weighted calculation based on each target parameter corresponding to each first data to be cached and the corresponding weight to obtain a weighted result corresponding to each first data to be cached; classifying each first data to be cached based on the weighted result to determine the data type of each first data to be cached. Further, in another specific embodiment, determining at least one target parameter corresponding to each first data to be cached in the process of executing the target computing task may include: determining the task level, parameter model size and matrix operation method corresponding to each first data to be cached as the target parameter in the process of executing the target computing task. Specifically, in the process of determining the data type of each first data to be cached generated in the process of executing the target computing task, the graphics processor first determines at least one target parameter corresponding to each first data to be cached; it can be understood that the target parameter mentioned here can be one or any combination of the task level, parameter model size and matrix operation method corresponding to each first data to be cached; further, after determining the target parameter corresponding to each first data to be cached, weighted calculation is performed based on the target parameters corresponding to the first data to be cached and the corresponding weights, so that the weighted result corresponding to each first data to be cached can be obtained; it can be understood that the first data to be cached can be divided into different categories according to different weighted results, that is, the data type of each first data to be cached is finally obtained.

[0066] Step S13, determining the target memory partitions corresponding to each of the first to-be-cached data from a preset memory partition set based on the data type; wherein the preset memory partition set includes the first memory partition, the second memory partition, and a third memory partition located in the local memory of the graphics processor, and different data types correspond to different memory partitions.

[0067] In an embodiment of the present application, after obtaining the data types corresponding to the first data to be cached through the above steps, the target memory partitions corresponding to each first data to be cached can be determined from the preset memory partition set based on the data type; it should be pointed out that the preset memory partition set can include a first memory partition in the memory pool shared by the host and the graphics processor, a second memory partition in the memory pool exclusively occupied by the graphics processor, and a third memory partition in the local memory of the graphics processor; each data type corresponds to a different memory partition.

[0068] In a specific embodiment, the data type of the first data to be cached may include: a first type of data to be cached that requires data transfer between the host and the graphics processor during the execution of the target computing task, a second type of data to be cached whose data dependency level with the target computing task satisfies a first preset low dependency condition during the execution of the target computing task, and a third type of data to be cached whose data dependency level with the target computing task satisfies a first preset high dependency condition during the execution of the target computing task; wherein the target memory partitions corresponding to the first type of data to be cached, the second type of data to be cached, and the third type of data to be cached are the first memory partition, the second memory partition, and the third memory partition, respectively. Specifically, through the above steps, the first data to be cached can be divided into three data types; among them, data that needs to be transferred during the execution of the computing task is determined to be the first type of data to be cached, and this type of data may require the host or graphics processor to read and call in the subsequent computing process; and the second type of data to be cached is data whose data dependency level with the current computing task meets the first preset low dependency condition. This type of data is unlikely to be read and called during the execution of subsequent computing tasks, and has a low level of association with the computing task; further, the third type of data to be cached is data whose data dependency level with the computing task meets the first preset high dependency condition. This type of data is necessary and frequently used data during the execution of the computing task. It should be pointed out that the first memory partition in the memory pool shared by the host and the graphics processor is the target memory partition corresponding to the first type of data to be cached, and the first memory partition can be used to efficiently transfer the first type of data to be cached during the execution of the computing task; accordingly, the second memory partition in the memory pool exclusively occupied by the graphics processor is the target memory partition corresponding to the second type of data to be cached, and the third memory partition in the local memory of the graphics processor is the target partition corresponding to the third type of data to be cached; it can be understood that data with a high level of data dependency on the computing task is directly cached to the local memory to facilitate direct reading by subsequent computing tasks, while data with a low level of data dependency on the computing task can be cached in the second memory partition in the memory pool exclusively occupied by the graphics processor to reduce the impact on subsequent computing processes.

[0069] In a specific embodiment, the number of the graphics processors is one or more, the host establishes a connection with each of the graphics processors through a first switch, and each of the graphics processors establishes a connection with the memory pool through a second switch; accordingly, the data caching method may also include: creating a corresponding number of memory partitions in the memory pool according to the number of the graphics processors to obtain each of the second memory partitions corresponding to each of the graphics processors, so that the graphics processor sends the second type of cached data generated by itself when executing the target computing task to the corresponding second memory partition for caching. Specifically, the host can be connected to a plurality of graphics processors through the first switch, and each graphics processor is connected to the mounted memory pool through the second switch; in this case, the mounted memory pool needs to divide the memory for each graphics processor, specifically, creating a corresponding number of memory partitions in the memory pool according to the number of graphics processors, so that a plurality of second memory partitions corresponding to each graphics processor and exclusively occupied by the graphics processor can be obtained; in this way, the subsequent graphics processor can send the second type of cached data generated by itself when executing the target computing task to the corresponding second memory partition for caching.

[0070] Step S14: caching the first data to be cached into the corresponding target memory partition, so as to complete the subsequent calculation process of the current task to be executed by using the cached data in the target memory partition.

[0071] In an embodiment of the present application, through the above steps, the graphics processor can determine the target memory partition corresponding to the first data to be cached in the process of executing the target computing task, and then cache the first data to be cached to the corresponding target memory partitions respectively, and then continue to execute related computing processes using the data cached in the target memory partitions.

[0072] In a specific embodiment, the method may further include: mounting the memory pool to the second switch through a preset expander; allocating a corresponding memory address space to the memory pool through the second switch to divide the memory pool into a first memory partition shared by the host and the graphics processor, a second memory partition exclusively occupied by the graphics processor, and a fourth memory partition exclusively occupied by the host, and mounting the first memory partition and the fourth memory partition to the host; wherein the preset memory partition set also includes the fourth memory partition. In addition, the first switch is a switch based on a high-speed serial computer expansion bus standard, and the second switch is a switch based on a computing express link. It is understandable that the first switch is connected to the host and the graphics processor respectively through a high-speed serial computer expansion bus standard (PCIE), and the second switch is connected to the host and the graphics processor respectively through a computing express link (Compute Express Link, CXL); and in the process of mounting the memory pool, the memory pool is mounted to the second switch through an expander; it is understandable that the memory pool is also connected to the second switch through CXL, and the host and the graphics processor can directly access the memory pool through CXL. Furthermore, the memory pool can be divided into a first memory partition shared with the host and the graphics processor, a second memory partition exclusively used by the graphics processor, and a fourth memory partition exclusively used by the host through the second switch; and the first memory partition and the fourth memory partition need to be mounted to the host so that the host can cache the data generated during the execution of the computing task to the corresponding memory partition. It should be pointed out that the preset memory partition set can also include a fourth memory partition, that is, the data that needs to be cached generated by the host during the execution of the computing task can be cached in the first memory partition shared by the host and the graphics processor and the fourth memory partition exclusively used by the host in the memory pool. In a specific embodiment, a CXL Type 3 device (a memory expansion device) can be mounted to realize memory pooling, which is divided into several memory partitions corresponding to the host and the graphics processor respectively.

[0073] In a specific embodiment, it may also include: when the host sends the current task to be executed to the graphics processor, the host determines the data type of the second data to be cached during the processing of the current task to be executed, and determines the target memory partition corresponding to each second data to be cached from the preset memory partition set according to the data type of the second data to be cached, and then caches the second data to be cached to the corresponding target memory partition; wherein the second data to be cached of different data types correspond to different memory partitions. Specifically, the host can send the acquired current task to the graphics processor, and at the same time determine the second data to be cached that needs to be cached during the processing of the current task to be executed, and determine the data type of the second data to be cached; accordingly, the host can determine the target memory partition corresponding to each second data to be cached from the preset memory partition set according to the data type corresponding to these data to be cached; it can be understood that the target memory partition here can be the first memory partition shared by the host and the graphics processor in the memory pool, the fourth memory partition exclusively occupied by the host, and the fifth memory partition in the local memory of the host; and the second data to be cached of different data types correspond to different target memory partitions respectively. Furthermore, the data type of the second data to be cached may include: a fourth type of data to be cached that requires data transfer between the host and the graphics processor during the processing of the current task to be executed, a fifth type of data to be cached that has a data dependency level with the current task to be executed that satisfies a second preset low dependency condition during the processing of the current task to be executed, and a sixth type of data to be cached that has a data dependency level with the current task to be executed that satisfies a second preset high dependency condition during the processing of the current task to be executed; wherein the target memory partitions corresponding to the fourth type of data to be cached, the fifth type of data to be cached, and the sixth type of data to be cached are the first memory partition, the fourth memory partition, and the fifth memory partition, respectively, and the fifth memory partition is the local memory of the host. Specifically, the data type of the second cached data generated by the host in the process of executing the computing task can be the fourth type of cached data that needs to be transferred between hosts or between the host and the graphics processor during the processing of the computing task, that is, computing parameters, computing results and other data that need to be exchanged; it can also be the fifth type of cached data whose data dependency level with the current computing task meets the second preset low dependency condition, that is, data with a low level of association with the computing task; it can also be the sixth type of cached data whose data dependency level with the current computing task meets the second preset high dependency condition, that is, some of the most necessary and frequently used data to implement the computing task.Accordingly, it is necessary to cache the sixth type of data to be cached to the fifth memory partition of the host's local memory to facilitate efficient execution of computing tasks; cache the fifth type of data to be cached to the fourth memory partition of the memory pool to reduce the impact on the execution process of computing tasks; cache the fourth type of data to be cached to the first memory partition in the memory pool, which can be directly read and called by the host or graphics processor, thereby improving the execution efficiency of computing tasks.

[0074] In a specific embodiment, the number of the hosts is one or more; wherein the data caching method may further include: creating a corresponding number of memory partitions in the memory pool according to the number of the hosts to obtain the fourth memory partitions corresponding to each of the hosts, so that the host sends the fifth type of cached data in the processing of the current task to be executed to the corresponding fourth memory partition for caching; accordingly, mounting the first memory partition and the fourth memory partition to the host includes: mounting the first memory partition to each of the hosts through the second switch, and mounting each of the fourth memory partitions to the corresponding hosts. Specifically, it can be understood that there may be several hosts, which are respectively connected to the memory pool mounted to the second switch, in which memory partitions of corresponding data volume are created according to the number of hosts, that is, the fourth memory partitions corresponding to each host, and several fourth memory partitions in the memory pool and the first memory partition shared by the host and the graphics processor are all mounted to each host through the second switch, so that the host can cache the fifth type of cached data in the processing of the computing task to the fourth memory partition of the memory pool, and cache the fourth type of cached data to the first memory partition in the memory pool.

[0075] In another specific embodiment, it may also include: using the first memory partition to respond to a data access request initiated by a target data access party, so as to pass the data corresponding to the data access request in the first memory partition to the target data access party; wherein the target data access party is any of the host or any of the graphics processor, and the data corresponding to the data access request includes data cached in the first memory partition by any of the host and / or any of the graphics processor. Specifically, after the process of executing the computing task through the steps of the above embodiment, the data related to the computing task is also cached in the memory pool, and subsequently the data access request initiated by the target data access party can be responded to by the data in the first memory partition, and the cached data corresponding to the data access request can be passed to the target data access party; it can be understood that the target data access party here can be either a host or a graphics processor, and both can obtain data from the first memory partition of the memory pool according to computing requirements to perform computing processes, and data exchange between the host and the graphics processor can be achieved through the memory pool.

[0076] In another specific embodiment, if the current computing task to be executed obtained by the current host can be performed without the aid of a graphics processor, the current host can directly execute the computing task, and at the same time cache the data generated during the computing process in a hierarchical manner to the fifth memory partition of the local memory and the first memory partition and the fourth memory partition in the memory pool; in this way, other hosts can directly access the data cached by the current host from the first memory partition of the memory pool according to the computing requirements of the relevant computing tasks, thereby facilitating the execution of the relevant computing tasks, thereby realizing the memory expansion of the host and improving the data transmission efficiency between the hosts.

[0077] It should be noted that the graphics processor (graphics processing accelerator card) in this application is responsible for data processing and high-performance computing, such as Figure 2 As shown and Figure 3As shown in the figure, the accelerator card mainly includes HBM (High Bandwidth Memory), RJ45 (an information socket connector in a wiring system), DPU (Data Processing Unit) chip, GPU chip, CPLD (Complex Programming logic device) logic controller, QSFP (Quad Small Form-factor Pluggable, four-channel interface) optical port connector, Gold Finger connector (Gold Finger), CDFP (a connector) and M.2 (a hard disk protocol) hard disk, etc. The DPU chip can expand the serial data interface to connect to the external system hard disk, and interconnect with the GPU chip (graphics processor) through PCIE (high-speed serial bus). The board can be managed and controlled through the ARM (a processor) core to achieve task allocation; and it can connect to the RJ45 (such as optical fiber interface) management network port, and route information with the Host and each Switch Box through the switch; in other words, the graphics processor in this application can carry the ARM kernel and operating system, and has functions such as system management and memory management, which can be equivalent to the HOST processing computing tasks by itself; it can be interconnected with the Host through the PCIE protocol and can be used as a downlink device; and it can be interconnected with the CXLSwitch (second switch) through the CXL protocol and can be used as an uplink device. The DPU chip can have a QSFP interface externally, connected to the data plane switch, and GPU Direct RDMA (Remote Direct Memory Access, a technology that allows remote devices or nodes to directly access the memory of another device) technology can be used to forward GPU computing data and improve GPU computing performance. CPLD (programmable logic device) is responsible for the management of the accelerator card, including heat dissipation control, power-on and power-off timing, status indication, etc. The GPU chip is responsible for data processing and high-performance computing tasks. Its PCIE link is connected to the PCIE Switch (first switch) through a gold finger connector, and the CXL link is interconnected with the CXL Switch through the CDFP (connector). The GPU accelerator card comes with HBM (high-bandwidth memory) memory, which can increase data transmission bandwidth and reduce latency.

[0078] In a specific embodiment, Figure 3The GPU in the system can obtain the target computing task to be executed from the host through the Gold Finger, or it can obtain the computing task to be executed through its own operating system. Then, in the process of executing the target computing task, the GPU determines the data type of the cached data obtained in each calculation to determine the importance of these data to the current target computing task, or whether data needs to be transferred with other hosts or other GPUs; the process of determining the data type can divide the cached data generated by the calculation into three types of data, which can be the most frequently used data L1 in the computing task, the data L2 that needs to be exchanged, and the data L3 with a low level of association with the computing task. Furthermore, Figure 3 The GPU in the system can use CXL (Compute Express Link) to store the cached data of data type L2 that needs to be exchanged in the first memory partition in the mounted memory pool, store the cached data of data type L3 with a low association level with the computing task in the second memory partition in the memory pool, and store the cached data of data type L1 that is most frequently used in computing tasks in the third memory partition (i.e., the high-bandwidth memory in the GPU); in this way, the memory pool pre-connected through the CXL link can be used to cache data related to the computing task, thereby realizing the memory expansion of the GPU.

[0079] Furthermore, Figure 3 The GPU (graphics processing unit) in the system can expand the system hard disk through the DPU (data processing unit), and manage the acquisition and execution of computing tasks through the operating system. In other words, the GPU in this application can be used as a downlink device of the host to execute computing tasks issued by the host with the help of a pre-mounted memory pool; and the GPU can be used as an uplink device to obtain computing tasks through its own operating system and execute related computing tasks with the help of a mounted memory pool.

[0080] It can be seen that the graphics processor in this application can realize data caching through the mounted memory pool. By combining the data type and the memory partition set, the data can be flexibly cached to different memory partitions, realizing the functions of memory capacity expansion, allocation and control of the host and the graphics processor. The graphics processor can have its own ARM kernel and operating system, and can realize data partition storage like the host. In this way, each memory can store data with each other, thereby improving data exchange efficiency.

[0081] The embodiment of the present application discloses a data cache system, including a host, a memory pool, and a graphics processor pre-connected with the host and the memory pool;

[0082] The graphics processor implements the following data caching method by executing a computer program:

[0083] Obtain a target computing task that needs to be executed by the graphics processor in the current task to be executed sent by the host; the memory pool includes a first memory partition shared by the host and the graphics processor and a second memory partition exclusively used by the graphics processor; determine the data type of each first cached data when executing the target computing task; determine the target memory partition corresponding to each first cached data from a preset memory partition set based on the data type; wherein the preset memory partition set includes the first memory partition, the second memory partition and a third memory partition located in the local memory of the graphics processor, and different data types correspond to different memory partitions; cache the first cached data to the corresponding target memory partition, so as to use the cached data in the target memory partition to complete the subsequent computing process of the current task to be executed.

[0084] In a specific embodiment, the number of the graphics processors is one or more, the number of the hosts is one or more, the hosts are connected to each of the graphics processors through a first switch, and each of the graphics processors and each of the hosts are connected to the memory pool through a second switch;

[0085] The first switch is a switch based on a high-speed serial computer expansion bus standard, and the second switch is a switch based on a computing fast link.

[0086] Further, such as Figure 4 The figure shows a schematic diagram of a data cache system structure, which specifically includes:

[0087] In this application, several hosts (HOST) are connected to the first switch (PCIE Switch) through the PCIE bus, and the first switch is connected to several graphics processors (GPUs) through the PCIE bus; wherein, the local memory area corresponding to HOST0 and HOST1 is recorded as DIMM (Dual-Inline-Memory-Modules, dual inline memory module), and the local memory area corresponding to GPU0 and GPU1 is recorded as HBM; and the host and the graphics processor are connected to the second switch (CXLSwitch) through the CXL link respectively, and the second switch is pre-mounted with a memory pool through the CXL protocol, which includes the first memory pool (Sharing memory area, i.e., C area memory pool) shared by the host and the graphics processor, the fourth memory pool (Host0 memory area and Host1 memory area, i.e., B area and D area memory pool) exclusively occupied by the host, and the second memory partition (GPU0 memory area and GPU1 memory area, i.e., A area and E area memory pool) exclusively occupied by the graphics processor, so that memory capacity expansion can be achieved. Furthermore, the host and the graphics processor both have their own local memory, which are the fifth memory partition and the third memory partition respectively. When users perform computing tasks under the host's operating system, they can weight the various parameter data corresponding to the task according to the task level, parameter model size, matrix operation method, etc., and classify the corresponding data into three levels, such as L1, L2, and L3; among them, L1 is the most necessary and frequently used parameter data to achieve the computing task, L2 is the computing parameters, calculation results and other data that need to be exchanged in the subsequent computing process, and L3 is the data with a low correlation level with the computing task.

[0088] Specifically, when a computing task to be executed reaches the host, the host first sends the task to the GPU accelerator card through the first switch (PCIE switch), instructing it to start executing parallel computing. At the same time, the host's operating system automatically classifies the task data (data to be cached) of the current computing task to be executed. At this time, the L1 level data can be stored in the fifth memory partition (DIMM) of the host's local memory, the L2 level data can be stored in the Sharing memory area of ​​the memory pool, and the L3 level data can be stored in the fourth memory partition exclusively used by the host.

[0089] Correspondingly, after obtaining the computing task sent by the host, the graphics processor can execute related computing services, and at the same time can verify and distinguish the data generated in the computing process, that is, partition the data generated in the computing process (data to be cached), and obtain L1, L2, and L3 level data. The L1 level data can be stored in the third memory partition of the graphics processor's local memory, the L2 level data can be stored in the first memory partition of the memory pool, and the L3 level data can be stored in the second memory partition in the memory pool exclusively used by the graphics processor.

[0090] After the above steps, during the execution of the computing task, the data that needs to be transmitted is cached by the host or the graphics processor to the first memory partition of the memory pool. If the computing task executed by a certain host requires the computing data of another host as support, it can be obtained immediately through the first memory partition of the memory pool; similarly, if the computing task executed by a certain graphics processor requires the computing data of another graphics processor as support, it can also be obtained immediately through the first memory partition of the memory pool, which can improve the efficiency of data exchange.

[0091] It can be seen that in this application, when the computing task starts to execute, the host and the graphics processor can cache the latest computing results in the first memory partition shared by the host and the graphics processor and the fourth memory partition and the second memory partition exclusively occupied by the host and the graphics processor in the memory pool; the graphics processor can obtain the latest computing results in different cache spaces according to computing needs to continue the computing process. From the system level, the data exchange between hosts, between graphics processors, and between hosts and graphics processors can be achieved only through the memory pool mounted by the computing fast link under the second switch, which can reduce the delay of data transmission, thereby achieving the effect of improving data exchange performance.

[0092] Furthermore, the present application also discloses an electronic device. Figure 5 It is a structural diagram of an electronic device according to an exemplary embodiment, and the content in the figure cannot be regarded as any limitation on the scope of use of this application. The electronic device may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input and output interface 25 and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the data caching method disclosed in any of the aforementioned embodiments. In addition, the electronic device in this embodiment may specifically be an electronic computer.

[0093] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0094] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0095] The operating system 221 is used to manage and control various hardware devices on the electronic device and the computer program 222, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the data caching method performed by the electronic device disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0096] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the aforementioned disclosed data caching method. The specific steps of the method can refer to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.

[0097] An embodiment of the present application discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the data caching method disclosed in any of the aforementioned embodiments.

[0098] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0099] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0100] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0101] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0102] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data caching method, characterized in that: Applicable to graphics processors, including: Acquire a target computing task that needs to be executed by a graphics processor from among the currently executed tasks sent by the host; the graphics processor preliminarily establishes a connection with a memory pool, wherein the memory pool includes a first memory partition shared by the host and the graphics processor and a second memory partition exclusively used by the graphics processor; Determine the data type of each first to-be-cached data when executing the target computing task; Determine, based on the data type, a target memory partition corresponding to each of the first to-be-cached data from a preset memory partition set; wherein the preset memory partition set includes the first memory partition, the second memory partition, and a third memory partition located in the local memory of the graphics processor, and different data types correspond to different memory partitions; Cache the first to-be-cached data into the corresponding target memory partition, so as to complete the subsequent calculation process of the current to-be-executed task by using the cached data in the target memory partition; Among them, the data type of the first data to be cached includes: a first type of data to be cached that requires data transmission between the host and the graphics processor during the execution of the target computing task, a second type of data to be cached whose data dependency level with the target computing task during the execution of the target computing task meets a first preset low dependency condition, and a third type of data to be cached whose data dependency level with the target computing task during the execution of the target computing task meets a first preset high dependency condition.

2. The data caching method according to claim 1, characterized in that: The determining of the data type of each first to-be-cached data when executing the target computing task includes: Determine at least one target parameter corresponding to each first data to be cached in the process of executing the target computing task, and perform weighted calculation based on each target parameter corresponding to each first data to be cached and a corresponding weight to obtain a weighted result corresponding to each first data to be cached; Each of the first data to be cached is classified based on the weighted result to determine the data type of each of the first data to be cached.

3. The data caching method according to claim 2, characterized in that: The determining at least one target parameter corresponding to each first to-be-cached data in the process of executing the target computing task includes: In the process of executing the target computing task, the task level, parameter model size and matrix operation method corresponding to each first cached data are determined as the target parameters.

4. The data caching method according to claim 1, characterized in that: The target memory partitions corresponding to the first type of data to be cached, the second type of data to be cached, and the third type of data to be cached are the first memory partition, the second memory partition, and the third memory partition, respectively.

5. The data caching method according to claim 4, characterized in that: The number of the graphics processors is one or more, the host establishes a connection with each of the graphics processors through a first switch, and each of the graphics processors establishes a connection with the memory pool through a second switch; Accordingly, the data caching method further includes: A corresponding number of memory partitions are created in the memory pool according to the number of the graphics processors to obtain second memory partitions corresponding to each of the graphics processors, so that the graphics processor can send the second type of cached data generated when executing the target computing task to the corresponding second memory partition for caching.

6. The data caching method according to claim 5, characterized in that: The first switch is a switch based on a high-speed serial computer expansion bus standard, and the second switch is a switch based on a computing fast link.

7. The data caching method according to claim 5, characterized in that: Also includes: Mounting the memory pool to the second switch through a preset expander; Allocate a corresponding memory address space for the memory pool through the second switch, so as to divide the memory pool into a first memory partition shared by the host and the graphics processor, a second memory partition exclusively used by the graphics processor, and a fourth memory partition exclusively used by the host, and mount the first memory partition and the fourth memory partition to the host; Wherein, the preset memory partition set also includes the fourth memory partition.

8. The data caching method according to claim 7, characterized in that: Also includes: When the host sends the current task to be executed to the graphics processor, the host determines the data type of the second data to be cached during the processing of the current task to be executed, and determines the target memory partition corresponding to each second data to be cached from the preset memory partition set according to the data type of the second data to be cached, and then caches the second data to be cached to the corresponding target memory partition; The second to-be-cached data of different data types correspond to different memory partitions.

9. The data caching method according to claim 8, characterized in that: The data type of the second data to be cached includes: a fourth type of data to be cached that requires data transfer between the host and the graphics processor during the processing of the current task to be executed, a fifth type of data to be cached that has a data dependency level with the current task to be executed that satisfies a second preset low dependency condition during the processing of the current task to be executed, and a sixth type of data to be cached that has a data dependency level with the current task to be executed that satisfies a second preset high dependency condition during the processing of the current task to be executed; Among them, the target memory partitions corresponding to the fourth category of data to be cached, the fifth category of data to be cached, and the sixth category of data to be cached are the first memory partition, the fourth memory partition, and the fifth memory partition, respectively, and the fifth memory partition is the local memory of the host.

10. The data caching method according to claim 9, characterized in that: The number of the hosts is one or more; Wherein, the data caching method further includes: Creating a corresponding number of memory partitions in the memory pool according to the number of the hosts, so as to obtain the fourth memory partitions corresponding to each of the hosts, so that the hosts can send the fifth type of data to be cached in the processing of the current task to be executed to the corresponding fourth memory partition for caching; Correspondingly, mounting the first memory partition and the fourth memory partition to the host includes: The first memory partition is mounted to each of the hosts through the second switch, and each of the fourth memory partitions is mounted to a corresponding host respectively.

11. The data caching method according to any one of claims 1 to 10, characterized in that: Also includes: Using the first memory partition to respond to a data access request initiated by a target data access party, so as to transfer data in the first memory partition corresponding to the data access request to the target data access party; The target data access party is any of the host or any of the graphics processors, and the data corresponding to the data access request includes data cached in the first memory partition by any of the host and / or any of the graphics processors.

12. A data cache system, characterized in that: It includes a host, a memory pool, and a graphics processor that is pre-connected with the host and the memory pool; The graphics processor implements the following data caching method by executing a computer program: Obtain a target computing task that needs to be executed by the graphics processor in the current task to be executed sent by the host; the memory pool includes a first memory partition shared by the host and the graphics processor and a second memory partition exclusively used by the graphics processor; determine the data type of each first cached data when executing the target computing task; determine the target memory partition corresponding to each first cached data from a preset memory partition set based on the data type; wherein the preset memory partition set includes the first memory partition, the second memory partition and a third memory partition located in the local memory of the graphics processor, and different data types correspond to different memory partitions; cache the first cached data to the corresponding target memory partition, so as to use the cached data in the target memory partition to complete the subsequent computing process of the current task to be executed; Among them, the data type of the first data to be cached includes: a first type of data to be cached that requires data transmission between the host and the graphics processor during the execution of the target computing task, a second type of data to be cached whose data dependency level with the target computing task during the execution of the target computing task meets a first preset low dependency condition, and a third type of data to be cached whose data dependency level with the target computing task during the execution of the target computing task meets a first preset high dependency condition.

13. The data cache system according to claim 12, characterized in that: The number of the graphics processors is one or more, the number of the hosts is one or more, the hosts are connected to each of the graphics processors via a first switch, and each of the graphics processors and each of the hosts are connected to the memory pool via a second switch; The first switch is a switch based on a high-speed serial computer expansion bus standard, and the second switch is a switch based on a computing fast link.

14. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the data caching method according to any one of claims 1 to 11.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data caching method according to any one of claims 1 to 11 are implemented.

16. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the data caching method according to any one of claims 1 to 11 are implemented.

Images