GPU and DSA heterogeneous fusion system
By using a heterogeneous fusion system of GPU, DSA unit and FPGA chip, the problem of low verification efficiency of mixed precision operation in GPU system is solved, and efficient verification and performance improvement of low precision operation are achieved. The low precision computing power of DSA unit is used to improve the system's operating performance.
Patent Information
- Application Number
- CN202410914956.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-09
- Publication Date
- 2026-01-09
AI Technical Summary
Existing technologies have low verification efficiency for mixed-precision computation in GPU systems, making it difficult to effectively utilize the low-precision computing power of DSA to improve performance. In particular, when faced with the low-precision computation requirements of specific models, existing methods are unable to achieve efficient verification and performance improvement.
A heterogeneous fusion system employing GPU, DSA unit, and FPGA chip converts GPU computing tasks into tasks that can be executed by the DSA unit through the interface and controller in the FPGA chip. It supports computing tasks of different number systems, realizes rapid integration of DSA unit and GPU, and improves verification efficiency and running performance.
It enables efficient verification of low-precision calculations in GPU systems, improves the flexibility and performance of mixed-precision tasks, and can improve running performance by utilizing the low-precision computing power of the DSA unit when low-precision requirements are needed.
Smart Images
Figure CN121301265A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of hardware architecture design, in particular to a GPU and DSA heterogeneous fusion system. BACKGROUND
[0002] A graphics processing unit (GPU) is widely used to perform training and inference tasks of large models due to its characteristics of large-scale parallel computing and high data throughput. The computing power of a GPU chip is usually represented by the number of floating-point operations performed per second. One effective way to improve the computing power of a GPU chip is to reduce the precision of the number system used when the GPU processes computing tasks, thereby improving the computing power of the GPU chip. However, this method can cause the computing precision of the training and inference tasks of large models to decrease, and also increase the possibility of numerical overflow beyond the range of the number system.
[0003] A domain specific architecture (DSA) is a programmable processor customized for a specific domain. For example, a DSA can be customized for low-precision computing, which usually refers to computing with a precision of 8-bit or less. The typical architecture of a DSA chip can be an ASIC, a DRAM in-memory computing chip, a Flash in-memory computing chip, etc.
[0004] Currently, to improve the efficiency of large model training and inference, existing methods introduce low-precision computing in GPU chips. To avoid the computing precision of large model training and inference tasks from decreasing too much, existing technologies further propose a mixed precision computing method, i.e., only part of the computing is performed with low-precision computing during the execution of training or inference tasks. To ensure the computing precision of large model training and inference tasks, a complex mixed precision computing strategy needs to be set. This kind of mixed precision computing strategy needs to be fully verified under a verification system and flexibly adjusted according to the precision requirements of the model during runtime to ensure its effectiveness and feasibility in actual application. In existing methods, verification is usually performed through software simulation during design. However, software simulation is inefficient and difficult to cover large model verification tasks. Verification during runtime only relies on the computing precision on the GPU chip. In the case where a specific model has a demand for low-precision computing, the performance cannot be improved using the low-precision computing power of the DSA.
[0005] Therefore, how to improve the verification efficiency and running performance of mixed precision computing in GPU systems has become a problem to be solved. SUMMARY
[0006] To solve the above technical problems, the technical solution adopted by the present application is:
[0007] A GPU and DSA heterogeneous fusion system, the system comprising: M GPU chips, N DSA units and an FPGA chip, M and N are both integers greater than zero.
[0008] For any GPU chip, the GPU chip comprises a first interface, the first interface is used to support information transmission between the GPU chip and the FPGA chip, and the GPU chip is used to send a first computing task executed in a reference number system to the FPGA chip through the first interface.
[0009] For any DSA unit, the DSA unit comprises a second interface, the second interface is used to support information transmission between the DSA unit and the FPGA chip, the DSA unit corresponds to a preset number system, and the DSA unit is used to execute a second computing task received from the second interface in its corresponding preset number system.
[0010] The FPGA chip comprises the first interface, the second interface and a DSA controller, the DSA controller is used to convert the first computing task into the second computing task, and then send the second computing task to the corresponding DSA unit through the second interface.
[0011] Compared with the prior art, the GPU and DSA heterogeneous fusion system provided by the application has obvious beneficial effects, and can achieve considerable technical progress and practicality, and has wide industrial utilization value, and at least has the following beneficial effects:
[0012] The application provides a GPU and DSA heterogeneous fusion system, the system comprising: M GPU chips, N DSA units and an FPGA chip, M and N are both integers greater than zero, for any GPU chip, the GPU chip comprises a first interface, the first interface is used to support information transmission between the GPU chip and the FPGA chip, and the GPU chip is used to send a first computing task executed in a reference number system to the FPGA chip through the first interface, for any DSA unit, the DSA unit comprises a second interface, the second interface is used to support information transmission between the DSA unit and the FPGA chip, the DSA unit corresponds to a preset number system, and the DSA unit is used to execute a second computing task received from the second interface in its corresponding preset number system, and the FPGA chip comprises the first interface, the second interface and a DSA controller, the DSA controller is used to convert the first computing task into the second computing task, and then send the second computing task to the corresponding DSA unit through the second interface.
[0013] It can be known that the heterogeneous fusion of the DSA unit supporting the preset number system calculation and the GPU chip through the first interface, the second interface and the DSA controller provided by the FPGA chip enables the first computing task of the GPU chip to be converted into the second computing task for the DSA unit to perform calculation, which can effectively support the rapid fusion of the DSA unit in various number systems into the GPU architecture, thereby realizing the verification of the large model training or inference task, improving the verification efficiency of the low-precision operation applied to the GPU system, and improving the flexibility of the heterogeneous fusion system when performing the mixed-precision training or inference task, so that when encountering a low-precision task requirement, the low-precision computing power of the DSA unit can be used to improve the running performance. BRIEF DESCRIPTION OF DRAWINGS
[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.
[0015] Figure 1 A first hardware architecture schematic diagram of a GPU and DSA heterogeneous fusion system provided by the embodiment of the present application;
[0016] Figure 2 A second hardware architecture schematic diagram of a GPU and DSA heterogeneous fusion system provided by the embodiment of the present application. DETAILED DESCRIPTION
[0017] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.
[0018] The embodiment provides a GPU and DSA heterogeneous fusion system, referring to Figure 1 A first hardware architecture schematic diagram of a GPU and DSA heterogeneous fusion system provided by the embodiment of the present application, the system comprising: M GPU chips, N DSA units and an FPGA chip, M and N are both integers greater than zero;
[0019] For any GPU chip, the first interface is included in the GPU chip, the first interface is used to support the information transmission between the GPU chip and the FPGA chip, and the GPU chip is used to send a first computing task executed in a reference number system to the FPGA chip through the first interface;
[0020] The second interface in the DSA unit corresponds to a preset number system, and the DSA unit is configured to perform a second computing task received from the second interface in the preset number system corresponding to the DSA unit.
[0021] The first interface, the second interface, and a DSA controller are included in the FPGA chip, and the DSA controller is configured to convert the first computing task into the second computing task, and send the second computing task to the corresponding DSA unit through the second interface.
[0022] In this embodiment, the DSA unit is configured to support a computing task in a preset number system, and the preset number system can be a low-precision number system.
[0023] Specifically, each GPU chip includes a first interface, and the first interface can also be configured to support high-speed data transmission between GPU chips, realize high-speed interconnection between GPU chips, and improve the overall computing power of the GPU system in a multi-GPU chip interconnection manner to support the computing power requirement of a large model training and inference task.
[0024] The preset number system corresponding to different DSA units can be different, thereby fusing different low-precision operation modes in the GPU system, and the FPGA chip can include at least one DSA controller, and a single GPU chip can call only a fixed DSA unit or call any DSA unit from N DSA units.
[0025] In a specific implementation, for any DSA unit, the second interface in the DSA unit corresponds to a communication protocol defined by the DSA unit.
[0026] Each DSA unit can define its own communication protocol, and only the second interface included in the DSA unit and the corresponding second interface included in the FPGA chip conform to the defined communication protocol.
[0027] In a specific implementation, the first computing task corresponds to an access address, and each DSA unit corresponds to an address range.
[0028] The conversion of the first computing task into the second computing task and the sending of the second computing task to the corresponding DSA unit through the second interface include:
[0029] According to the access address corresponding to the first computing task and the N address ranges, it is determined that the address range to which the access address belongs is a target range;
[0030] It is determined that the DSA unit corresponding to the target range is a target unit, and a communication protocol corresponding to the target unit is obtained;
[0031] The first computing task is converted into the second computing task, and the second computing task is executed by the target unit according to the preset number system corresponding thereto;
[0032] The second computing task is sent to the target unit through a second interface conforming to the communication protocol corresponding to the target unit.
[0033] Wherein, the address ranges corresponding to different DSA units are different, then the GPU chip can select the corresponding DSA unit according to the required preset number system, and determine the access address in the address range corresponding to the DSA unit.
[0034] In a specific embodiment, the DSA unit corresponds to a DSA chip.
[0035] Wherein, the DSA unit corresponds to a DSA chip, which means that the DSA unit has been formed into a DSA chip through chip fabrication, and the DSA unit is fused with the GPU system in the form of a DSA chip.
[0036] In a specific embodiment, the DSA unit and the FPGA chip are integrated into a verification board.
[0037] Wherein, when the size of the DSA unit meets the preset size limit, the DSA unit can also be integrated as a module with the FPGA chip to obtain a verification board, at this time, the DSA unit and the FPGA chip are located in the same verification board.
[0038] Further, the size of the verification board can be the same as the size of the GPU chip, so as to more conveniently replace several GPU chips in a multi-GPU chip server with a verification board, and realize the heterogeneous fusion of GPU and DSA.
[0039] On the premise that the DSA unit and the FPGA chip are integrated into a verification board, taking a single GPU chip and a single verification board architecture as an example, referring to Figure 2 , a second hardware architecture schematic diagram of a GPU and DSA heterogeneous fusion system provided by an embodiment of the application.
[0040] In a specific embodiment, the reference number system is a single-precision floating-point number.
[0041] The reference number system can also be a half-precision floating-point number, BF16, INT8, and other number system types commonly used in large model training and inference architecture. The implementer can set the reference number system according to the actual situation.
[0042] In a specific embodiment, the preset number system at least includes an 8-bit floating-point number.
[0043] The preset number system generally refers to a low-precision number system less than or equal to 8-bit, and can also include a 4-bit floating-point number (FP4) and the like.
[0044] In a specific embodiment, the system further includes a CPU configured to send the first computing task to the corresponding GPU chip.
[0045] The GPU chip can include a PCIE interface for receiving information transmitted by the CPU. In an embodiment, the FPGA chip can also include a PCIE interface for power supply.
[0046] In this embodiment, the DSA unit supporting preset number system calculation and the GPU chip are heterogeneously fused through the first interface, the second interface, and the DSA controller provided by the FPGA chip, so that the first computing task of the GPU chip can be converted into a second computing task for execution by the DSA unit. This can effectively support the rapid fusion of DSA units in various number systems into the GPU architecture, thereby realizing the verification of large model training or inference tasks, improving the verification efficiency of low-precision operation applied to the GPU system, and improving the flexibility of the heterogeneous fusion system when executing mixed-precision training or inference tasks. When encountering low-precision task requirements, the low-precision computing power of the DSA unit can be used to improve the running performance.
[0047] Although some specific embodiments of the present application have been described in detail through examples, those skilled in the art should understand that the above examples are only for illustration, not for limiting the scope of the present application. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the present application. The scope of the present application is defined by the appended claims.
Claims
1. A GPU and DSA heterogeneous fusion system, characterized in that, The system comprises M GPU chips, N DSA units and an FPGA chip, M and N are integers greater than zero; For any GPU chip, the GPU chip comprises a first interface, the first interface is used to support information transmission between the GPU chip and the FPGA chip, and the GPU chip is used to send a first computing task executed in a reference number system to the FPGA chip through the first interface; For any DSA unit, the DSA unit comprises a second interface, the second interface is used to support information transmission between the DSA unit and the FPGA chip, the DSA unit corresponds to a preset number system, and the DSA unit is used to execute a second computing task received from the second interface in the preset number system corresponding to the DSA unit; The FPGA chip comprises the first interface, the second interface and a DSA controller, and the DSA controller is used to convert the first computing task into the second computing task and then send the second computing task to the corresponding DSA unit through the second interface.
2. The GPU and DSA heterogeneous fusion system according to claim 1, characterized in that, For any DSA unit, the second interface in the DSA unit corresponds to a communication protocol defined by the DSA unit.
3. The GPU and DSA heterogeneous fusion system according to claim 2, characterized in that, The first computing task corresponds to an access address, and each DSA unit corresponds to an address range; The conversion of the first computing task into the second computing task and the sending of the second computing task to the corresponding DSA unit through the second interface comprise: According to the access address corresponding to the first computing task and N address ranges, determining that the address range to which the access address belongs is a target range; Determining that the DSA unit corresponding to the target range is a target unit, and obtaining the communication protocol corresponding to the target unit; Converting the first computing task into the second computing task, and the second computing task is executed by the target unit according to the preset number system corresponding to the target unit; Sending the second computing task to the target unit through the second interface conforming to the communication protocol corresponding to the target unit.
4. The GPU and DSA heterogeneous fusion system according to claim 1, characterized in that, The DSA unit corresponds to a DSA chip.
5. The GPU and DSA heterogeneous fusion system according to claim 1, characterized in that, The DSA unit and the FPGA chip are integrated into a verification board.
6. The GPU and DSA heterogeneous fusion system according to claim 1, characterized in that, The reference number system is a single-precision floating-point number.
7. The GPU and DSA heterogeneous fusion system according to claim 1, characterized in that, The preset number system at least comprises an 8-bit floating-point number.
8. The GPU and DSA heterogeneous fusion system according to claim 1, characterized in that, The system further comprises a CPU, and the CPU is used to send the first computing task to the corresponding GPU chip.