Hybrid accelerator card management method and device, electronic device and storage medium

Abstract

The invention relates to a hybrid accelerator card management method and device, an electronic device and a storage medium, and is applied to the field of computer hardware acceleration and data storage and transmission, and the hybrid accelerator card management method comprises the following steps: determining a runtime library interface of a hybrid accelerator card in response to a received server data transmission request; registering a memory address of the hybrid acceleration card to a preset hardware abstraction layer based on a remote direct memory access technology; a unified hardware abstraction interface is preset in the hardware abstraction layer; the runtime library interface is bound with the unified hardware abstract interface; and calling the runtime library interface to execute data transmission operation between the hybrid acceleration cards through a preset hardware abstraction layer. According to the method and the device, the GPU acceleration cards of different manufacturers are compatible, and meanwhile, the effect of improving the data processing and transmission efficiency is achieved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a hybrid accelerator card management method, device, electronic device and storage medium. Background Art

[0002] In the fields of high performance computing (HPC), artificial intelligence (AI), scientific computing and large-scale data processing, the demand for computing device performance is increasing. To meet these needs, GPU (Graphics Processing Unit) has gradually become a key acceleration device in high-performance computing systems due to its excellent parallel computing capabilities and high throughput. Compared with traditional CPU (Central Processing Unit), GPU can process a large amount of data streams at the same time, which is particularly suitable for tasks such as deep learning training, image processing and complex scientific computing. However, existing GPU accelerator cards are usually provided by multiple manufacturers. There are significant differences between different manufacturers in hardware architecture, software development interface (Application Programming Interface, API) and data transmission methods, resulting in many challenges in the development and optimization of cross-platform applications.

[0003] Some existing solutions attempt to simplify the compatibility issues of hardware from different manufacturers through virtualization technology or containerization technology. However, these methods can only achieve hardware abstraction and resource management to a certain extent, and cannot completely solve the problem of unified processing of data storage and transmission between GPU accelerator cards from different manufacturers. In addition, virtualization technology usually has a certain performance overhead, and existing solutions will further limit the efficiency of high-performance computing applications. Therefore, there is an urgent need for a unified processing method that is compatible with GPUs from different manufacturers and can efficiently handle data storage and transmission.

[0004] There is currently no effective solution to the problem of how to improve data processing and transmission efficiency while being compatible with GPU accelerator cards from different manufacturers in related technologies. Summary of the invention

[0005] In this embodiment, a hybrid acceleration card management method, device, electronic device and storage medium are provided to solve the problem in the related art of how to improve data processing and transmission efficiency while being compatible with GPU acceleration cards from different manufacturers.

[0006] In a first aspect, a hybrid accelerator card management method is provided in this embodiment, which is applied to a server system, wherein the server system includes a server; the method includes: In response to the received server data transmission request, determining a runtime library interface of the hybrid accelerator card; Based on remote direct memory access technology, the memory address of the hybrid acceleration card is registered to a preset hardware abstraction layer; a unified hardware abstraction interface is preset in the hardware abstraction layer; the runtime library interface is bound to the unified hardware abstraction interface; The preset hardware abstraction layer is used to call the runtime library interface to perform data transmission operations between hybrid acceleration cards.

[0007] In some embodiments, the hybrid acceleration card includes a plurality of acceleration cards; before determining the runtime library interface of the hybrid acceleration card in response to the received server data transmission request, the method includes: Determine a plurality of accelerator cards corresponding to the runtime library linked to the server; An adapter management component is established, and a hardware adapter of the acceleration card is managed by the adapter management component; wherein the hardware adapter is used to link the acceleration card and the server.

[0008] In some embodiments, determining the runtime library interface of the hybrid accelerator card in response to the received server data transmission request includes: In response to the received data transmission request from the server, determining a runtime library type of the hybrid accelerator card according to environment information of the server system environment; Based on the runtime library type of the hybrid acceleration card, a runtime library interface of the hybrid acceleration card is determined.

[0009] In some embodiments, after determining the runtime library type of the hybrid acceleration card according to the environment information of the server system environment in response to the received server data transmission request, the method further includes: Determining whether the server system environment of the hybrid accelerator card includes a single server or multiple servers; When it is determined that the server system environment of the hybrid accelerator card includes a single server, numbering the runtime library type of the hybrid accelerator card to obtain a library type number; The library type number and the current thread identifier corresponding to the server data transmission request are matched in a key-value pair manner.

[0010] In some embodiments, the method further comprises: Based on a preset calling method, loading the dynamic link library of the runtime library of the accelerator card in the server system; Obtaining the runtime library handle of the accelerator card; Determining a runtime library interface function of the accelerator card from the dynamic link library according to the runtime library handle; The runtime library interface function is bound to the preset unified hardware abstract interface.

[0011] In some embodiments, registering the memory address of the hybrid accelerator card to a preset hardware abstraction layer based on remote direct memory access technology includes: Acquire data memory addresses of multiple acceleration cards; the multiple acceleration cards include memory addresses for receiving data and / or sending data; Determine whether there is a memory direct access module corresponding to the acceleration card in the server system; When a memory direct access module corresponding to the acceleration card exists in the server system, memory registration is performed in the hardware abstraction layer according to the data memory address of the acceleration card.

[0012] In some embodiments, the determining whether there is a memory direct access module corresponding to the accelerator card in the server system further includes: When the memory direct access module corresponding to the acceleration card does not exist in the server system, the data memory address of the acceleration card is converted into a virtual address according to a preset mapping method; Memory registration is performed at the hardware abstraction layer according to the virtual address.

[0013] In a second aspect, a hybrid accelerator card management device is provided in this embodiment, the device comprising: a response module, a registration module and a transmission module; The response module is used to determine the runtime library interface of the hybrid acceleration card in response to the server data transmission request; The registration module is used to register the memory address of the hybrid acceleration card to a preset hardware abstraction layer for remote direct memory access based on remote direct memory access technology; a unified hardware abstraction interface is preset in the hardware abstraction layer; the runtime library interface is bound to the unified hardware abstraction interface; The transmission module is used to call the runtime library interface to perform data transmission operations between hybrid acceleration cards according to the preset hardware abstraction layer.

[0014] In a third aspect, an electronic device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the hybrid acceleration card management method described in the first aspect when executing the computer program.

[0015] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored, and when the program is executed by a processor, the hybrid acceleration card management method described in the first aspect is implemented.

[0016] Compared with the related art, a hybrid acceleration card management method, device, electronic device and storage medium provided in this embodiment bind the runtime library interfaces of multiple acceleration cards to a preset unified hardware abstract interface, and manage different hardware acceleration cards through a unified interface to be compatible with GPU acceleration cards from different manufacturers; at the same time, based on the method of remote direct access to memory registration, the data transmission of the hybrid acceleration card device memory is optimized to improve data processing and transmission efficiency.

[0017] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 It is a hardware structure block diagram of a terminal of the hybrid acceleration card management method provided in an embodiment of the present application; Figure 2 is a flow chart of a hybrid accelerator card management method provided in an embodiment of the present application; Figure 3 This is a schematic diagram of a unified abstract device operation access interface for a hybrid acceleration card provided in this specific embodiment; Figure 4 It is a schematic diagram of a method for unified processing of hybrid acceleration card hardware abstraction and data transmission in this specific embodiment; Figure 5 is a flowchart of a method for unifying hardware abstraction layer interfaces provided by this specific embodiment; Figure 6 It is a schematic diagram of performing data transmission operation on memory data of an accelerator card device provided in this specific embodiment; Figure 7 is a schematic diagram of data transmission of a cross-server hybrid acceleration card device provided in this specific embodiment; Figure 8 It is a structural block diagram of a hybrid acceleration card management device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0020] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0021] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 1 is a hardware structure diagram of a terminal of a hybrid acceleration card management method provided in an embodiment of the present application. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0022] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the hybrid acceleration card management method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0023] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0024] At present, in the field of computer hardware acceleration, data storage and transmission, GPU accelerator cards with excellent parallel computing capabilities and high throughput are usually used. However, the hardware architecture and programming interface of GPUs from different manufacturers have their own characteristics, and the underlying implementation mechanism of the platform varies greatly. When developers use different GPUs in applications, they need to write specific code to adapt to each manufacturer's API, which increases the complexity of development. At the same time, the hardware abstraction layers of different manufacturers lack universality, making it more difficult to achieve unified hardware abstraction and data transmission processing in a multi-vendor mixed accelerator card environment, resulting in poor application portability and high maintenance costs.

[0025] In order to deal with the compatibility issues of hybrid GPUs from multiple vendors, the industry has proposed a variety of cross-platform hardware abstraction solutions and programming interfaces. These technologies have achieved abstract encapsulation of different GPU accelerator cards to a certain extent, but they often require a lot of adaptation work in actual use, and there are still performance bottlenecks, making it difficult to meet the needs of efficient parallel computing. In addition, some cross-platform hardware abstraction solutions and programming interfaces have unstable performance when performing complex deep learning tasks, which limits their application in some high-performance computing scenarios.

[0026] In addition, in terms of data transmission, the data transmission efficiency in the cross-platform GPU system has a significant impact on the overall computing performance. Especially when multiple accelerator cards are used in combination, data transmission between different GPUs is often restricted by factors such as interface differences, inconsistent data formats, and different memory management mechanisms, resulting in low data transmission efficiency and low resource utilization of the system. Taking the training process of deep learning models as an example, data needs to be frequently transmitted between different accelerator cards. If the data transmission efficiency is not high, it will directly affect the training speed and overall performance of the model, and may even cause data transmission to become a bottleneck for the system. Therefore, realizing a unified processing mechanism for data transmission and resource management between different accelerator cards is crucial to improving the resource utilization and overall efficiency of multi-accelerator card systems.

[0027] Some existing solutions attempt to simplify the compatibility issues of hardware from different manufacturers through virtualization technology or containerization technology. However, these methods can only achieve hardware abstraction and resource management to a certain extent, and cannot completely solve the problem of unified processing of data storage and transmission between GPU accelerator cards from different manufacturers. In addition, virtualization technology usually has a certain performance overhead, which may further limit the efficiency of high-performance computing applications. Therefore, there is an urgent need for a unified processing method and device that is compatible with GPUs from different manufacturers and can efficiently handle data storage and transmission, so as to reduce the adaptation workload of developers and improve the computing performance and resource utilization of heterogeneous computing systems.

[0028] Therefore, in the prior art, when using multiple hybrid accelerator cards, it is necessary to call the drivers and runtime libraries of each accelerator card, which is quite inconvenient. For example, the memory of each accelerator card cannot achieve unified memory management. When performing operations such as memory application and release, it is necessary to link and call the runtime library interface of each accelerator card separately, which leads to very low efficiency in the use of hybrid accelerator cards, and the data transmission rate between hybrid cards has a large bottleneck.

[0029] In order to solve the above problems, this paper proposes a method and device for unified processing of hybrid accelerator card hardware abstraction and data transmission. By building a unified runtime API, this method not only supports GPU accelerator cards from multiple manufacturers, but also provides an efficient and unified processing mechanism for data storage and transmission. In this way, developers can simplify the development process, reduce the complexity of data transmission and resource management between different accelerator cards, and improve the overall performance and computing efficiency of multi-accelerator card heterogeneous systems.

[0030] Based on the above problems, a hybrid acceleration card management method is provided in this embodiment, which is applied in a server system including a server. Figure 2 is a flow chart of a hybrid accelerator card management method provided in an embodiment of the present application. Figure 2As shown, the process includes the following steps: Step S210: determining a runtime library interface of the hybrid acceleration card in response to a received server data transmission request.

[0031] In this step, after the server system receives the server data transmission request, it responds to the server data transmission request and determines the hybrid acceleration card corresponding to the server for data transmission according to the data transmission request. The hybrid acceleration card includes acceleration cards on multiple different servers. For different types of acceleration cards on each server, the corresponding SDK (Software Development Kit) development kit, such as the acceleration card driver and runtime library, is installed on the server. At the same time, the runtime library interface of the hybrid acceleration card is determined according to the data type and transmission type in the data transmission request, so as to realize the data transmission between servers by calling the runtime library interface later.

[0032] Before determining the runtime library interface of the hybrid acceleration card, the method includes: determining multiple acceleration cards corresponding to the runtime library linked to the server; establishing an adapter management component to manage the hardware adapter of the acceleration card through the adapter management component; wherein the hardware adapter is used to link the acceleration card and the server.

[0033] Specifically, firstly, the API interfaces of different types of accelerator card runtime libraries on the server in the server system are sorted out, and classified and summarized according to the types of different accelerator cards, so as to facilitate the subsequent binding with the unified hardware abstract interface. Afterwards, check whether the dynamic link library of the runtime library corresponding to each accelerator card can be correctly linked by the server system, and establish an adapter management component. The hardware adapter corresponding to the accelerator card correctly linked to the server system is managed by the adapter management component name. Based on the adapter management component, the accelerator card runtime library interface is managed to realize the initialization of the unified hardware abstract interface. If more accelerator card runtime libraries need to be supported in the future, it is only necessary to add the hardware adapter of the new accelerator card runtime library in the adapter management component without modifying the existing interface function, so as to realize the seamless switching and compatible extension of different types of accelerator card interface calls.

[0034] Step S220, based on remote direct memory access technology, registering the memory address of the hybrid acceleration card to a preset hardware abstraction layer; a unified hardware abstraction interface is preset in the hardware abstraction layer; and the runtime library interface is bound to the unified hardware abstraction interface.

[0035] In this step, after the corresponding hybrid acceleration card is determined based on the server data transmission request, the memory address of the hybrid acceleration card is registered to the preset hardware abstraction layer based on the remote direct memory access technology; a remote direct memory access connection between the hybrid acceleration cards in the server is established through the hardware abstraction layer, and according to the data source and target address after the remote direct memory access memory registration, the remote direct memory access data, i.e., the writing or reading operation of the data to be transmitted in the server data transmission request is performed. By encapsulating the writing and reading related operations of the remote direct memory access data into the hardware abstraction layer, and specifically encapsulating them in the unified hardware abstraction interface preset in the hardware abstraction layer, the remote direct memory access related interface corresponding to the hybrid acceleration card is called through the hardware abstraction layer, thereby optimizing the data transmission of the memory of the hybrid acceleration card device, which is conducive to improving the efficiency of subsequent data transmission between hybrid acceleration cards.

[0036] Among them, Remote Direct Memory Access (RDMA) is a technology that allows one computer to directly access the memory of another computer without involving the host processor. This technology is mainly used for high-speed network communications, especially in high-performance computing (HPC) and data center environments to achieve efficient data transmission. The Runtime Library is a set of program libraries that provide various services and functions required by programs at runtime. These services typically include memory management, input / output operations, system call interfaces, exception handling, string processing, mathematical calculations, etc. The runtime library is an indispensable part of the program runtime. They are usually provided by the compiler and are closely related to a specific programming language or compiler.

[0037] Step S230: calling the runtime library interface through the preset hardware abstraction layer to perform data transmission operations between the hybrid acceleration cards.

[0038] In this step, the runtime library interface of the hybrid acceleration card is called through the hardware abstraction layer interface in the hardware abstraction layer to realize the connection between the remote direct memory access network cards corresponding to the hybrid acceleration card, and perform data transmission operations between the hybrid acceleration cards to achieve unified processing of data transmission between different hybrid acceleration cards.

[0039] Through the above steps, by binding the runtime library interfaces of multiple accelerator cards to the preset unified hardware abstract interface, different hardware accelerator cards are managed through a unified interface to be compatible with GPU accelerator cards from different manufacturers; at the same time, based on the remote direct access memory registration method, the data transmission of the hybrid accelerator card device memory is optimized to improve data processing and transmission efficiency.

[0040] In some of the embodiments, determining the runtime library interface of the hybrid acceleration card in response to a received server data transmission request in step S210 includes: determining the runtime library type of the hybrid acceleration card according to environmental information of the server system environment in response to the received server data transmission request; and determining the runtime library interface of the hybrid acceleration card based on the runtime library type of the hybrid acceleration card.

[0041] In this embodiment, it is determined whether the server system environment of the hybrid acceleration card includes a single server or multiple servers; when it is determined that the server system environment of the hybrid acceleration card includes multiple servers, that is, the server system includes multiple servers with multiple types of hybrid acceleration cards, and only one type of acceleration card exists for a single server among the multiple servers, the type of the hybrid acceleration card runtime library is automatically selected based on the environment information of the server system environment. Exemplarily, the environment information of the server system environment can be PCIe (Peripheral Component Interconnect Express, a high-speed serial computer expansion bus standard) device information, or it can be the driver information of the system kernel.

[0042] Furthermore, in some of the embodiments, in response to a received server data transmission request, after determining the runtime library type of the hybrid acceleration card according to the environmental information of the server system environment, it also includes: judging whether the server system environment of the hybrid acceleration card includes a single server or multiple servers; when judging that the server system environment of the hybrid acceleration card includes a single server, numbering the runtime library type of the hybrid acceleration card to obtain a library type number; matching the library type number and the current thread identifier corresponding to the server data transmission request in the form of a key-value pair.

[0043] Among them, when it is determined that the server system environment of the hybrid accelerator card includes a single server, that is, when the usage scenario is that there are multiple hybrid types of accelerator cards in a single server, it is necessary to manually select the type of the corresponding accelerator card runtime library. Specifically, after selecting the accelerator card runtime library corresponding to the hybrid accelerator card, a hash table is established to match the runtime library type of the current hybrid accelerator card and the current thread identification number corresponding to the data transmission request by key-value pairs; when calling the runtime library related interface, the unified hardware abstract interface of the corresponding hardware abstraction layer will search for the currently selected hybrid accelerator card runtime library type according to the hash table.

[0044] In some of the embodiments, a hybrid acceleration card management method also includes: based on a preset calling method, loading a dynamic link library of the runtime library of the acceleration card in the server system; obtaining the runtime library handle of the acceleration card; determining the runtime library interface function of the acceleration card from the dynamic link library according to the runtime library handle; and binding the runtime library interface function to a preset unified hardware abstract interface.

[0045] In this embodiment, the method for binding the runtime interface of the unified hardware abstract interface of the preset hardware abstraction layer with the selected acceleration card runtime library interface includes loading the dynamic link library of the acceleration card runtime library in the server system path by means of explicit dynamic library calling, and obtaining the runtime library handle of the acceleration card; searching and obtaining the runtime library interface function of the corresponding acceleration card from the runtime library according to the runtime library handle; and then binding the obtained runtime interface of the corresponding acceleration card with the pre-defined unified hardware abstract runtime interface, so as to facilitate the subsequent calling of the corresponding acceleration card runtime library interface through the unified hardware abstraction layer runtime interface to achieve the effect of high-speed data transmission between different types of acceleration cards.

[0046] In some of the embodiments, in step S220, the memory address of the hybrid acceleration card is registered to a preset hardware abstraction layer based on remote direct memory access technology, including: obtaining data memory addresses of multiple acceleration cards; multiple acceleration cards include memory addresses for receiving data and / or sending data; determining whether there is a memory direct access module corresponding to the acceleration card in the server system; when there is a memory direct access module corresponding to the acceleration card in the server system, performing memory registration in the hardware abstraction layer according to the data memory address of the acceleration card.

[0047] Among them, judging whether there is a memory direct access module corresponding to the acceleration card in the server system also includes: when there is no memory direct access module corresponding to the acceleration card in the server system, converting the data memory address of the acceleration card into a virtual address according to a preset mapping method; and performing memory registration in the hardware abstraction layer according to the virtual address.

[0048] In this embodiment, when data is transmitted between hybrid accelerator card device memories, if the selected accelerator card has a kernel driver module for direct access to the accelerator card device memory in the server system, the virtual address corresponding to the memory allocated by the accelerator card is registered for RDMA (Remote Direct Memory Access) memory through the hardware abstraction layer interface; if the selected accelerator card does not have a kernel driver module for direct access to the accelerator card device memory in the system, the memory allocated by the accelerator card is mapped to the CPU of the server through the hardware abstraction layer interface to obtain its virtual address, and then RDMA memory registration is performed according to the virtual address. Afterwards, an RDMA connection is established through the hardware abstraction layer interface, and RDMA data write or read operations are performed according to the data source address and target address after RDMA memory registration.

[0049] In one possible embodiment, when there is a single server in the server system and there is only one type of acceleration card in the single server, the type of the acceleration card runtime library is automatically selected by the system environment information of the server system. The system environment can be PCIe device information or system kernel driver information.

[0050] The present embodiment is described and illustrated by means of specific examples below.

[0051] This specific embodiment proposes a method for unified processing of hybrid acceleration hardware abstraction and data transmission, which can enable developers to improve the use efficiency and data transmission speed of hybrid acceleration cards to a certain extent, and reduce the burden on developers to deal with hardware differences. First, for different types of acceleration cards on each server in the server system, install the corresponding SDK (Software Development Kit) development kit on the server, such as the acceleration card driver and runtime library. Secondly, sort out and classify the runtime library function interface of each acceleration card, which can be roughly divided into device interface, error interface, memory interface, stream and event interface, graph interface, driver interface, context driver interface, memory and device driver interface, etc. After sorting out the interface categories of various hybrid acceleration cards, define a unified hardware abstract interface. Figure 3 Schematic diagram of the unified abstract device operation access interface of the hybrid acceleration card provided in this specific embodiment. Figure 3 The unified hardware abstract device runtime interface includes memory management interface, device query interface, image processing interface, exception handling interface, and stream and event interface; at the same time, when calling the hardware abstract initialization interface, an adapter management component will be established to provide a unified interface to manage different hardware acceleration card adapters. If more acceleration card runtime libraries need to be supported in the future, you only need to add new adapters to the adapter management component without modifying the existing interface functions.

[0052] Figure 4 This is a schematic diagram of a method for unified processing of hybrid acceleration card hardware abstraction and data transmission in this specific embodiment, refer to Figure 4 The method includes steps S11 to S15.

[0053] Step S11, combing through the interfaces of the runtime libraries of various accelerator cards to facilitate subsequent binding with the unified hardware abstract runtime interface.

[0054] Step S12, calling the hardware abstract initialization interface.

[0055] Specifically, calling the hardware abstract initialization interface specifically includes: checking whether the dynamic link library of the runtime library corresponding to each accelerator card can be correctly linked to by the system; establishing an adapter management component to provide a unified interface to manage different hardware accelerator card adapters. Figure 3 , through the adapter management component, the A-type computing accelerator card, the B-type computing accelerator card, and the X-type computing accelerator card, the A-type accelerator card software tool development kit SDK, the B-type accelerator card software tool development kit SDK, and the X-type computing accelerator card software tool development kit SDK are managed respectively.

[0056] Step S13, selecting the type of the required accelerator card runtime library according to the data processing request.

[0057] Specifically, according to the server data transmission request, determine the type of hybrid acceleration card runtime library to be used. Further, for the hybrid acceleration card in the server, that is, a single server simultaneously exists and is compatible with multiple acceleration cards, after calling the hardware abstract initialization interface, it is necessary to select the corresponding runtime library type according to the type of acceleration card that the current thread needs to use, and the acceleration card initialization interface is effective for the entire process after calling, and each platform only needs to be executed once, and the acceleration card type that does not need to be used can be initialized. The acceleration card runtime library type selection function interface is for the current thread. If each thread needs it after creation, it calls this function, and the type of runtime library can be changed at any time. The implementation principle is that when a runtime library type is selected, a hash table will be established, and then the currently selected library type number and the thread id number are matched by key-value pairs. When calling the runtime library related interface, such as the memory allocation interface, the corresponding abstract layer memory allocation interface will search the currently selected runtime library type according to the hash table, so as to achieve seamless switching of the hybrid acceleration card runtime library.

[0058] In addition, for the situation where there are multiple servers with hybrid accelerator cards but only one accelerator card in one server, the hardware abstraction layer and its corresponding runtime library can be bound directly by querying the system environment information on the server. This environment information can be PCIe device information or be judged by system driver information.

[0059] Step S14, binding the defined hardware abstraction layer runtime interface with the selected accelerator card runtime library interface.

[0060] Specifically, after matching the accelerator card runtime library, the interface of the hardware abstraction layer is bound to the matched accelerator card runtime library interface. Through the explicit call of the dynamic library, the dynamic link library corresponding to the accelerator card runtime library is loaded in the server system path, the handle of the runtime library is obtained, and the runtime library interface function of the corresponding accelerator card is searched and obtained from the runtime library according to the handle, and then this function is bound to the pre-defined unified hardware abstract runtime interface.

[0061] Step S15, calling the corresponding accelerator card runtime library interface through the unified hardware abstraction layer runtime interface, and performing high-speed data transmission for data between different types of accelerator cards.

[0062] Among them, in the prior art, data transmission between hybrid accelerator card device memories usually has a certain speed bottleneck. For scenarios with high computing real-time and efficiency requirements, ordinary Ethernet data transmission often cannot meet the requirements. For example, for data transmission between accelerator card device memories across server machines, the traditional approach, whether it is the sending end or the receiving end, generally copies the data in the accelerator card device memory to the CPU memory first, and then performs Ethernet data transmission. In scenarios with large amounts of data, this approach has a relatively low transmission rate due to the two additional accelerator card device memory to CPU memory copy processes.

[0063] This application optimizes the data transmission of hybrid accelerator card device memory based on RDMA (Remote Direct Memory Access). Among them, RDMA-related operations, such as the establishment of RDMA connections, RDMA accelerator card device memory write operations and read operations are directly implemented in the hardware abstraction layer, and RDMA-related interfaces can be directly called through the hardware abstraction layer. The implementation of these interfaces also depends on RDMA-related core libraries and tool sets.

[0064] In one of the specific embodiments, the specific implementation steps of memory data transmission between hybrid accelerator card devices in a server system are as follows: for the data sending end, first obtain the data address and size on the accelerator card that needs to be transmitted across servers, and then call the hardware abstraction layer runtime interface to perform RDMA memory registration. If the currently selected accelerator card is equipped with a kernel driver module for direct access to the memory of the accelerator card device on the server where it is located, when the hardware abstraction layer runtime interface is used, the dynamically allocated memory address will be directly registered for RDMA memory; if there is no kernel driver module for direct access to the memory of the accelerator card device, when the hardware abstraction layer runtime interface is called, the memory address of the data to be transmitted on the accelerator card device will be converted into a virtual address of the user space by memory mapping, and then RDMA memory registration will be performed; similarly, for the data receiving end, it is necessary to call the hardware abstraction layer runtime interface to dynamically allocate a piece of accelerator card device memory for receiving, and at the same time, this memory address must also perform the corresponding RDMA memory registration operation. After the data source address and target address after RDMA memory registration are obtained, the hardware abstraction layer interface is called to perform the establishment of RDMA connection and RDMA write operation, so as to realize the high-speed transmission of data between hybrid accelerator cards across servers.

[0065] Furthermore, under the premise of the above steps, after each server node completes data copying using the hardware abstraction layer, it can execute the corresponding kernel function to complete the parallel computing task. At the same time, the hybrid accelerator cards on each server node can also exchange data with each other through RDMA when executing parallel computing tasks to complete the synchronization of calculation results.

[0066] Through the method of unified processing of hybrid acceleration card hardware abstraction and data transmission provided by this specific embodiment, in the current development environment, there are multiple hybrid acceleration cards inside a single server machine. When initializing through the runtime library of the hardware abstraction layer, you only need to select the type of acceleration card currently required. After the selection, you can directly call the memory copy function interface of the hardware abstraction layer runtime library to copy the data that needs to be calculated in parallel to the acceleration card device memory. Optionally, you can also switch the type of acceleration card according to your own needs, so as to complete the memory copy operation of the hybrid acceleration card at the same time. Through the data processing method of this application, it is possible to be compatible with a variety of hardware accelerators, and improve the data transmission speed between hybrid acceleration cards, reduce the burden on developers to deal with hardware differences, and thus improve the overall system performance and efficiency.

[0067] Figure 5 is a flowchart of the hardware abstraction layer interface unified method provided by this specific embodiment, such as Figure 5As shown, the hardware abstraction layer interface unified method includes: firstly, initializing the hardware abstraction interface, and judging whether there is an in-machine hybrid acceleration card in the server system, that is, whether there is a hybrid acceleration card in a single server; if so, selecting the runtime library type currently required to be used, and matching the current runtime library type and the current thread number, that is, the thread id number, through a hash table, where the current thread id number is the current thread identifier in the aforementioned embodiment. Thereafter, the hardware abstraction layer runtime interface is bound to the selected acceleration card runtime library interface, wherein each hardware abstraction layer runtime interface will match the corresponding acceleration card runtime library according to the thread id, and will check the initialization state of the runtime library at the same time. If not, there is no need to select the acceleration card runtime library type, and the acceleration card runtime library type is automatically selected directly according to the system environment information of the server system, and then the hardware abstraction layer runtime interface is bound to the selected acceleration card runtime library interface. Finally, regardless of whether there is an in-machine hybrid acceleration card, the runtime interface of the corresponding acceleration card needs to be called through the hardware abstraction layer interface to realize the hybrid acceleration card data transmission; when the data transmission is completed, that is, after the use of multiple related interfaces is completed, the related resources will be released to avoid resource occupation.

[0068] Figure 6 is a schematic diagram of the data transmission operation performed by the memory data of the acceleration card device provided in this specific embodiment, with reference to Figure 6 , when performing data transmission operations through a hybrid accelerator card device, first obtain the memory addresses of the data sent by the A-type accelerator card and the B-type accelerator card respectively, and determine whether there is a memory direct access kernel driver module for the A-type accelerator card and / or the B-type accelerator card in the server system; wherein the A-type accelerator card is used to send data, and the B-type accelerator card is used to receive data. If there is an accelerator card memory direct access kernel driver module, then the obtained memory address is directly used to perform remote direct memory access RDMA memory registration, and the hardware abstraction layer is called to perform remote direct memory access RDMA data transmission operations. If there is no accelerator card memory direct access kernel driver module, then it is necessary to use the memory mapping method to convert both the memory address of the sent data and the memory address of the received data into virtual addresses, and then perform remote direct memory access RDMA memory registration, and call the hardware abstraction layer to perform remote direct memory access RDMA data transmission operations.

[0069] Figure 7 Schematic diagram of cross-server hybrid acceleration card device data transmission provided by this specific embodiment. Figure 7, wherein different types of class A accelerator cards and class B accelerator cards are connected to the RDMA network card through the expansion card pcie switch respectively; the RDMA network card obtains the system memory of the server system through the processor CPU; the processor stores executable code, and when the executable code is executed, it is used to implement the hybrid accelerator card hardware abstraction and data transmission unified processing method in the above embodiment.

[0070] In this embodiment, a hybrid accelerator card management device is also provided, which is used to implement the above embodiments and preferred implementations, and the descriptions that have been made will not be repeated. The terms "module", "unit", "sub-unit", etc. used below can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0071] Figure 8 is a structural block diagram of a hybrid acceleration card management device provided in an embodiment of the present application, such as Figure 8 As shown, the device includes: a response module 10, a registration module 20 and a transmission module 30.

[0072] The response module 10 is used to determine the runtime library interface of the hybrid acceleration card in response to the server data transmission request.

[0073] The registration module 20 is used to register the memory address of the hybrid acceleration card to a preset hardware abstraction layer for remote direct memory access based on remote direct memory access technology; a unified hardware abstraction interface is preset in the hardware abstraction layer; and the runtime library interface is bound to the unified hardware abstraction interface.

[0074] The transmission module 30 is used to call the runtime library interface to perform data transmission operations between hybrid acceleration cards according to a preset hardware abstraction layer.

[0075] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0076] In this embodiment, an electronic device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0077] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0078] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program: S1, in response to a received server data transmission request, determining a runtime library interface of the hybrid accelerator card.

[0079] S2, based on remote direct memory access technology, registers the memory address of the hybrid accelerator card to a preset hardware abstraction layer; a unified hardware abstraction interface is preset in the hardware abstraction layer; and the runtime library interface is bound to the unified hardware abstraction interface.

[0080] S3, through the preset hardware abstraction layer, calls the runtime library interface to perform data transmission operations between hybrid accelerator cards.

[0081] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0082] In addition, in combination with the hybrid accelerator card management method provided in the above embodiments, a storage medium may be provided in this embodiment to implement the hybrid accelerator card management method. The storage medium stores a computer program, and when the computer program is executed by a processor, any one of the hybrid accelerator card management methods in the above embodiments is implemented.

[0083] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0084] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0085] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0086] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A hybrid accelerator card management method, characterized in that: Applied in a server system, the server system includes a server; the method includes: In response to the received server data transmission request, determining a runtime library interface of the hybrid accelerator card; Based on remote direct memory access technology, the memory address of the hybrid acceleration card is registered to a preset hardware abstraction layer; a unified hardware abstraction interface is preset in the hardware abstraction layer; the runtime library interface is bound to the unified hardware abstraction interface; The preset hardware abstraction layer is used to call the runtime library interface to perform data transmission operations between hybrid acceleration cards.

2. The hybrid accelerator card management method according to claim 1, characterized in that: The hybrid acceleration card includes a plurality of acceleration cards; before determining the runtime library interface of the hybrid acceleration card in response to the received server data transmission request, the method includes: Determine a plurality of accelerator cards corresponding to the runtime library linked to the server; An adapter management component is established, and a hardware adapter of the acceleration card is managed by the adapter management component; wherein the hardware adapter is used to link the acceleration card and the server.

3. The hybrid accelerator card management method according to claim 1, characterized in that: The step of determining the runtime library interface of the hybrid accelerator card in response to the received server data transmission request includes: In response to the received data transmission request from the server, determining a runtime library type of the hybrid accelerator card according to environment information of the server system environment; Based on the runtime library type of the hybrid acceleration card, a runtime library interface of the hybrid acceleration card is determined.

4. The hybrid accelerator card management method according to claim 3, characterized in that: After the method determines the runtime library type of the hybrid acceleration card according to the environment information of the server system environment in response to the received server data transmission request, the method further includes: Determining whether the server system environment of the hybrid accelerator card includes a single server or multiple servers; When it is determined that the server system environment of the hybrid accelerator card includes a single server, numbering the runtime library type of the hybrid accelerator card to obtain a library type number; The library type number and the current thread identifier corresponding to the server data transmission request are matched in a key-value pair manner.

5. The hybrid accelerator card management method according to any one of claims 1 to 4, characterized in that: The method further comprises: Based on a preset calling method, loading the dynamic link library of the runtime library of the accelerator card in the server system; Obtaining the runtime library handle of the accelerator card; Determining a runtime library interface function of the accelerator card from the dynamic link library according to the runtime library handle; The runtime library interface function is bound to the preset unified hardware abstract interface.

6. The hybrid accelerator card management method according to claim 2, characterized in that: The method of registering the memory address of the hybrid accelerator card to a preset hardware abstraction layer based on remote direct memory access technology includes: Acquire data memory addresses of multiple acceleration cards; the multiple acceleration cards include memory addresses for receiving data and / or sending data; Determine whether there is a memory direct access module corresponding to the acceleration card in the server system; When a memory direct access module corresponding to the acceleration card exists in the server system, memory registration is performed in the hardware abstraction layer according to the data memory address of the acceleration card.

7. The hybrid accelerator card management method according to claim 6, characterized in that: The determining whether there is a memory direct access module corresponding to the acceleration card in the server system also includes: When the memory direct access module corresponding to the acceleration card does not exist in the server system, the data memory address of the acceleration card is converted into a virtual address according to a preset mapping method; Memory registration is performed at the hardware abstraction layer according to the virtual address.

8. A hybrid accelerator card management device, characterized in that: The device comprises: a response module, a registration module and a transmission module; The response module is used to determine the runtime library interface of the hybrid acceleration card in response to the server data transmission request; The registration module is used to register the memory address of the hybrid acceleration card to a preset hardware abstraction layer for remote direct memory access based on remote direct memory access technology; a unified hardware abstraction interface is preset in the hardware abstraction layer; the runtime library interface is bound to the unified hardware abstraction interface; The transmission module is used to call the runtime library interface to perform data transmission operations between hybrid acceleration cards according to the preset hardware abstraction layer.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the hybrid accelerator card management method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the hybrid accelerator card management method according to any one of claims 1 to 7 are implemented.

Images