Deep learning training method based on Tensorflow framework

Abstract

The present application provides a deep learning training method based on the Tensorflow framework, including: receiving a deep learning training request; virtualizing the FPGA board corresponding to the deep learning training request as a local FPGA node; registering the local FPGA node as a VFPGA device corresponding to the Tensorflow framework; configuring the forward operator and reverse operator of the VFPGA device, and compiling the forward operator and reverse operator to obtain the bit file of the FPGA; burning the bit file to the local FPGA node to generate the FPGA device corresponding to the local FPGA node; and performing deep learning training using the FPGA device. The present application reduces communication latency, thereby improving the efficiency of deep learning training. The present application also provides a deep learning training system, FPGA board, computer-readable storage medium, and electronic device based on the Tensorflow framework, which have the above-mentioned beneficial effects.

Description

Technical Field

[0001] The present application relates to the field of big data, and in particular to a deep learning training method, system, FPGA board, computer-readable storage medium and electronic device based on the Tensorflow framework. Background Art

[0002] Tensorflow is one of the most widely used deep learning training frameworks, and most companies use tensorflow as their preferred training framework. Therefore, tensorflow has been supported by many chip manufacturers, such as Intel's CPU, AMD's CPU and APU, Nvidia's GPU, and leading FPGA chip manufacturers such as Intel and Silinx have implemented FPGA support for tensorflow reasoning. However, when using a PCIE mode host for model training, it is limited by the number of FGPA card slots in the server, which will become a bottleneck during large-scale deployment. At the same time, since the communication between multiple FPGA boards based on the PCIE mode needs to be carried out with the help of the CPU, the communication time is relatively long, that is, the communication is limited. Summary of the invention

[0003] The purpose of this application is to provide a deep learning training method, system, FPGA board, computer-readable storage medium and electronic device based on the Tensorflow framework, which can reduce the communication delay in the deep learning training process and improve the efficiency of deep learning training.

[0004] To solve the above technical problems, this application provides a deep learning training method based on the Tensorflow framework, including:

[0005] Receive deep learning training requests;

[0006] Virtualize the FPGA board corresponding to the deep learning training request as a local FPGA node;

[0007] Register the local FPGA node as a VFPGA device corresponding to the Tensorflow framework;

[0008] Configuring the forward operator and the reverse operator of the VFPGA device, and compiling the forward operator and the reverse operator to obtain a bit file of the FPGA;

[0009] Burn the bit file to the local FPGA node to generate an FPGA device corresponding to the local FPGA node; wherein the FPGA device includes an RDMA protocol module, and the RDMA protocol module is used to implement full-duplex reading and writing of memory between FPGA devices;

[0010] The deep learning training is performed using the FPGA device.

[0011] Optionally, using the FPGA device to perform the deep learning training includes:

[0012] Get the data to be trained from the dataset;

[0013] Using the FPGA device to perform forward calculation, back propagation and gradient calculation on the data to be trained, respectively, to obtain a gradient value corresponding to the data to be trained;

[0014] The gradient value is shared in the aggregate communication module through the RDMA protocol module, and an average gradient is calculated according to the gradient values ​​of each FPGA device; the average gradient determines the minimum value of the loss function.

[0015] Optionally, before virtualizing the FPGA board corresponding to the deep learning training request as a local FPGA node, the method further includes:

[0016] Determine the FPGA board corresponding to the deep learning training request from the FPGA board resource pool; the FPGA board in the FPGA board resource pool includes an independent power supply unit.

[0017] Optionally, the network transmission layer of the FPGA board includes an RDMA protocol module, a network physical layer protocol module and a network application layer protocol.

[0018] The present application also provides an FPGA board card, wherein the network connection layer of the FPGA board card is provided with an RDMA protocol module, and the RDMA protocol module is used to realize full-duplex reading and writing of memory between FPGA devices.

[0019] Optionally, the FPGA board supports L3 layer network strategy and L4 layer network strategy.

[0020] Optionally, the FPGA board also includes an independent power supply unit.

[0021] The present application also provides a deep learning training system based on the Tensorflow framework, including:

[0022] A request receiving module, used to receive deep learning training requests;

[0023] A board virtualization module, used to virtualize the FPGA board corresponding to the deep learning training request into a local FPGA node;

[0024] A node registration module, used to register the local FPGA node as a VFPGA device corresponding to the Tensorflow framework;

[0025] A node compilation module, used for configuring the forward operator and the reverse operator of the VFPGA device, and compiling the forward operator and the reverse operator to obtain a bit file of the FPGA;

[0026] A device configuration module, used to burn the bit file to the local FPGA node to generate an FPGA device corresponding to the local FPGA node; wherein the FPGA device includes an RDMA protocol module, and the RDMA protocol module is used to implement full-duplex reading and writing of memory between FPGA devices;

[0027] A deep learning training module is used to perform the deep learning training using the FPGA device.

[0028] The present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above-mentioned method when executed by a processor.

[0029] The present application also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned method when calling the computer program in the memory.

[0030] The present application provides a deep learning training method based on the Tensorflow framework, comprising: receiving a deep learning training request; virtualizing an FPGA board corresponding to the deep learning training request as a local FPGA node; registering the local FPGA node as a VFPGA device corresponding to the Tensorflow framework; configuring a forward operator and a reverse operator of the VFPGA device, and compiling the forward operator and the reverse operator to obtain a bit file of the FPGA; burning the bit file to the local FPGA node to generate an FPGA device corresponding to the local FPGA node; wherein the FPGA device includes an RDMA protocol module, and the RDMA protocol module is used to implement full-duplex reading and writing of memory between FPGA devices; and using the FPGA device to perform the deep learning training.

[0031] This application virtualizes the FPGA board as an FPGA node and sets an RDMA protocol module in the FPGA board, so that full-duplex memory reading and writing can be achieved between FPGA boards, and data interaction can be carried out in a timely manner, thereby eliminating the need for CPU transfer, reducing communication latency, and thus improving deep learning training efficiency.

[0032] The present application also provides a deep learning training system based on the Tensorflow framework, an FPGA board, a computer-readable storage medium and an electronic device, which have the above-mentioned beneficial effects and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0034] Figure 1 A flowchart of a deep learning training method based on the Tensorflow framework provided in an embodiment of the present application;

[0035] Figure 2 A schematic diagram of the structure of an FPGA board provided in an embodiment of the present application;

[0036] Figure 3 Schematic diagram of data flow for traditional FPGA training;

[0037] Figure 4 A schematic diagram of data flow for FPGA training provided in an embodiment of the present application;

[0038] Figure 5 A flowchart of a deep learning training system based on the Tensorflow framework provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] Please refer to Figure 1 , this application provides a deep learning training method based on the Tensorflow framework, including:

[0041] S101: receiving a deep learning training request;

[0042] S102: Virtualize the FPGA board corresponding to the deep learning training request as a local FPGA node;

[0043] S103: Register the local FPGA node as a VFPGA device corresponding to the Tensorflow framework;

[0044] S104: configuring the forward operator and the reverse operator of the VFPGA device, and compiling the forward operator and the reverse operator to obtain a bit file of the FPGA;

[0045] S105: Burn the bit file to the local FPGA node to generate an FPGA device corresponding to the local FPGA node;

[0046] S106: Utilize the FPGA device to perform the deep learning training.

[0047] In order to better describe this embodiment, the following first describes an FPGA board provided by the present application:

[0048] See also Figure 2 The present application provides an FPGA board, and the network connection layer of the FPGA board is provided with an RDMA protocol module, and the RDMA protocol module is used to realize full-duplex reading and writing of memory between FPGA devices.

[0049] FPGA boards can also support L3 and L4 network policies, and can also include independent power supply units, so that FPGA boards do not rely on the CPU, can perform independent network computing, and be separated from the host as an independent computing unit in the cluster. The network transport layer of the FPGA board includes the RDMA protocol module, the network physical layer protocol module, and the network application layer protocol. Figure 2 In the above, Kernel refers to the functional module implemented inside the FPGA, which needs to be burned and solidified into the internal storage of the FPGA. PR is the remote loading module of the FPGA. Flash is the storage used to solidify the FPGA program, and the program will not be lost when the power is off. DMA is a direct data access technology without the participation of the CPU. DDR is a memory controller. PCIE is a bus protocol connecting the FPGA and the Host. RDMA is a high-speed and low-latency network communication protocol. MAC is a network physical layer protocol module. UDP is a network application layer protocol. Connet is the network transmission layer inside the FPGA. Match engine is a general term for functional modules such as flow control, data matching and distribution, and data verification. ETH is a network hardware module.

[0050] The FPGA device includes an RDMA protocol module, and the RDMA protocol module is used to implement full-duplex reading and writing of memory between FPGA devices.

[0051] Based on the FPGA board described above, the deep learning training method based on the Tensorflow framework provided in this embodiment is described below:

[0052] After receiving the deep learning training request, the corresponding FPGA board can be determined from the FPGA resource pool composed of several FPGABOXs. Of course, the number of FPGA boards required is not specifically limited here. That is, before the FPGA board corresponding to the deep learning training request is virtualized as a local FPGA node, the FPGA board corresponding to the deep learning training request can be determined from the FPGA board resource pool.

[0053] In addition, before using FPGA resources, you need to virtualize one or more FPGAs in the network as local FPGA nodes. The number of supported FPGA nodes can be expanded to multiple according to the host performance, and then you can use the network FPGA resources just like accessing the FPGA resources of the PCIE interface.

[0054] After that, register the local FPGA node as the VFPGA device corresponding to the Tensorflow framework, configure the forward operator and reverse operator of the VFPGA device, and compile the forward operator and reverse operator to obtain the FPGA bit file. This process is the necessary configuration for implementing deep learning training based on the Tensorflow framework.

[0055] After burning the bit file to the local FPGA node, the local FPGA node can participate in the deep learning training based on the Tensorflow framework. Since the local FPGA node, i.e., the FPGA board, contains the RDMA protocol module, the PGAs can directly read and write memory in full duplex with each other. Using this high-speed communication mechanism, the reduce kernel inside the FPGA is implemented for data interaction between FPGA nodes during training.

[0056] Utilizing the FPGA device to perform the deep learning training may include the following steps:

[0057] The first step is to obtain the data to be trained from the data set;

[0058] The second step is to use the FPGA device to perform forward calculation, back propagation and gradient calculation on the data to be trained, so as to obtain the gradient value corresponding to the data to be trained;

[0059] The third step is to share the gradient value in the aggregate communication module through the RDMA protocol module, and calculate the average gradient according to the gradient value of each FPGA device; the average gradient determines the minimum value of the loss function.

[0060] For details, please see Figure 3 and Figure 4 , Figure 3 This is a diagram of the data flow for traditional FPGA training. Figure 4A schematic diagram of the data flow of FPGA training provided in the embodiment of the present application, Figure 3 and Figure 4 The dashed boxes in the middle represent FPGA boards, Enqueuebuffer represents the operation of transferring data from the CPU to the FPGA, MINI-batch: small batch data used in each step during training, Reduce is an aggregate communication operation, and PCIE is a bus protocol connecting the FPGA and the host device.

[0061] Depend on Figure 3 It can be clearly seen that the gradient output by the FPGA board needs to be transferred to the CPU via PCIE and returned to the parameter server through the GRPC framework. Figure 4 ,This application can directly implement gradient data sharing between FPGA boards based on the RDMA protocol. FPGAs can directly communicate through RDMA and read the gradient of other FPGAs from the agreed address. Only one communication is required, which greatly saves the communication delay.

[0062] This application virtualizes the FPGA board as an FPGA node and sets an RDMA protocol module in the FPGA board, so that full-duplex memory reading and writing can be achieved between FPGA boards, and data interaction can be carried out in a timely manner, thereby eliminating the need for CPU transfer, reducing communication latency, and thus improving deep learning training efficiency.

[0063] The following is an introduction to the deep learning training system based on the Tensorflow framework provided in an embodiment of the present application. The deep learning training system based on the Tensorflow framework described below and the deep learning training method based on the Tensorflow framework described above can refer to each other.

[0064] See also Figure 5 , Figure 5 A flowchart of a deep learning training system based on the Tensorflow framework provided in an embodiment of the present application. The present application also provides a deep learning training system based on the Tensorflow framework, including:

[0065] A request receiving module, used to receive deep learning training requests;

[0066] A board virtualization module, used to virtualize the FPGA board corresponding to the deep learning training request into a local FPGA node;

[0067] A node registration module, used to register the local FPGA node as a VFPGA device corresponding to the Tensorflow framework;

[0068] A node compilation module, used for configuring the forward operator and the reverse operator of the VFPGA device, and compiling the forward operator and the reverse operator to obtain a bit file of the FPGA;

[0069] A device configuration module, used to burn the bit file to the local FPGA node to generate an FPGA device corresponding to the local FPGA node; wherein the FPGA device includes an RDMA protocol module, and the RDMA protocol module is used to implement full-duplex reading and writing of memory between FPGA devices;

[0070] A deep learning training module is used to perform the deep learning training using the FPGA device.

[0071] Based on the above embodiment, as a preferred embodiment, the board virtualization module includes:

[0072] Get the data to be trained from the dataset;

[0073] Using the FPGA device to perform forward calculation, back propagation and gradient calculation on the data to be trained, respectively, to obtain a gradient value corresponding to the data to be trained;

[0074] The gradient value is shared in the aggregate communication module through the RDMA protocol module, and an average gradient is calculated according to the gradient values ​​of each FPGA device; the average gradient determines the minimum value of the loss function.

[0075] Based on the above embodiments, as a preferred embodiment, it also includes:

[0076] A board determination module is used to determine the FPGA board corresponding to the deep learning training request from an FPGA board resource pool; the FPGA board in the FPGA board resource pool includes an independent power supply unit.

[0077] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0078] The present application also provides an electronic device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the electronic device may also include various network interfaces, power supplies and other components.

[0079] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the system provided in the embodiment, since it corresponds to the method provided in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0080] Specific examples are used herein to illustrate the principles and implementation methods of the present application, and the description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

[0081] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A deep learning training method based on the Tensorflow framework, characterized in that: include: Receive deep learning training requests; Virtualize the FPGA board corresponding to the deep learning training request as a local FPGA node; Register the local FPGA node as a VFPGA device corresponding to the Tensorflow framework; Configuring the forward operator and the reverse operator of the VFPGA device, and compiling the forward operator and the reverse operator to obtain a bit file of the FPGA; Burn the bit file to the local FPGA node to generate an FPGA device corresponding to the local FPGA node; wherein the FPGA device includes an RDMA protocol module, and the RDMA protocol module is used to implement full-duplex reading and writing of memory between FPGA devices; Utilizing the FPGA device to perform the deep learning training; Wherein, using the FPGA device to perform the deep learning training includes: Get the data to be trained from the dataset; Using the FPGA device to perform forward calculation, back propagation and gradient calculation on the data to be trained, respectively, to obtain a gradient value corresponding to the data to be trained; The gradient value is shared in the aggregate communication module through the RDMA protocol module, and an average gradient is calculated according to the gradient values ​​of each FPGA device; the average gradient determines the minimum value of the loss function.

2. The deep learning training method according to claim 1, characterized in that: Before virtualizing the FPGA board corresponding to the deep learning training request as a local FPGA node, the method further includes: Determine the FPGA board corresponding to the deep learning training request from the FPGA board resource pool; the FPGA board in the FPGA board resource pool includes an independent power supply unit.

3. The deep learning training method according to claim 2, characterized in that: The network transmission layer of the FPGA board includes an RDMA protocol module, a network physical layer protocol module and a network application layer protocol.

4. An FPGA board, characterized in that: The network connection layer of the FPGA board is provided with an RDMA protocol module, and the RDMA protocol module is used to implement full-duplex reading and writing of memory between FPGA devices; The FPGA board corresponds to the deep learning training request; the FPGA board corresponding to the deep learning training request is virtualized as a local FPGA node; the local FPGA node is registered as a VFPGA device corresponding to the Tensorflow framework; the forward operator and the reverse operator of the VFPGA device are configured, and the forward operator and the reverse operator are compiled to obtain the bit file of the FPGA; the bit file is burned to the local FPGA node to generate the FPGA device corresponding to the local FPGA node; the FPGA device includes an RDMA protocol module, and the RDMA protocol module is used to realize full-duplex reading and writing of memory between FPGA devices; the deep learning training is performed using the FPGA device; Wherein, using the FPGA device to perform the deep learning training includes: Get the data to be trained from the dataset; Using the FPGA device to perform forward calculation, back propagation and gradient calculation on the data to be trained, respectively, to obtain a gradient value corresponding to the data to be trained; The gradient value is shared in the aggregate communication module through the RDMA protocol module, and an average gradient is calculated according to the gradient values ​​of each FPGA device; the average gradient determines the minimum value of the loss function.

5. The FPGA board according to claim 4, characterized in that: The FPGA board supports L3 layer network strategy and L4 layer network strategy.

6. The FPGA board according to claim 4, characterized in that: The FPGA board also includes an independent power supply unit.

7. A deep learning training system based on the Tensorflow framework, characterized in that: include: A request receiving module, used to receive deep learning training requests; A board virtualization module, used to virtualize the FPGA board corresponding to the deep learning training request into a local FPGA node; A node registration module, used to register the local FPGA node as a VFPGA device corresponding to the Tensorflow framework; A node compilation module, used for configuring the forward operator and the reverse operator of the VFPGA device, and compiling the forward operator and the reverse operator to obtain a bit file of the FPGA; A device configuration module, used to burn the bit file to the local FPGA node to generate an FPGA device corresponding to the local FPGA node; wherein the FPGA device includes an RDMA protocol module, and the RDMA protocol module is used to implement full-duplex reading and writing of memory between FPGA devices; A deep learning training module, used to perform the deep learning training using the FPGA device; Wherein, using the FPGA device to perform the deep learning training includes: Get the data to be trained from the dataset; Using the FPGA device to perform forward calculation, back propagation and gradient calculation on the data to be trained, respectively, to obtain a gradient value corresponding to the data to be trained; The gradient value is shared in the aggregate communication module through the RDMA protocol module, and an average gradient is calculated according to the gradient values ​​of each FPGA device; the average gradient determines the minimum value of the loss function.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the deep learning training method based on the Tensorflow framework as described in any one of claims 1 to 3 are implemented.

9. An electronic device, characterized in that: It includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps of the deep learning training method based on the Tensorflow framework as described in any one of claims 1 to 3 are implemented.

Images