Device and method for model and reconfigurable hardware

Abstract

An apparatus and method for a model and reconfigurable hardware are provided. The method includes the following steps: setting the number of pipeline stages and the pipeline split points as software parameters, and setting the block size, the number of processing elements, and the size of the processing elements as hardware parameters, wherein the segmented model includes pipeline stages and split points, and the processing elements correspond to reconfigurable hardware; compiling the segmented model using a machine learning compiler to obtain host code; synthesizing a bitstream of the reconfigurable hardware using a high-order synthesis tool and the hardware parameters; and obtaining the execution time corresponding to the host code and the bitstream.

Description

[Technical Field]

[0001] This invention relates to an apparatus and method for models and reconfigurable hardware. [Previous Technology]

[0002] Currently, for reconfigurable hardware, machine learning compilers cannot integrate model (software) segmentation with resource (hardware) configuration. This will lead to model deployment failure. [Summary of the Invention]

[0003] The present invention provides an apparatus and method for a model and reconfigurable hardware, which can integrate software and hardware to optimize model performance.

[0004] The apparatus for a model and reconfigurable hardware of the present invention includes a processor, wherein the processor is configured to perform the following steps: S1: The processor sets the number of pipeline stages and the pipeline split points as software parameters, and the processor sets the pinning size, the number of processing elements (PEs), and the size of the processing elements as hardware parameters, wherein the segmented model includes pipeline stages and split points, and wherein the processing elements correspond to reconfigurable hardware; S2: The processor uses a machine learning compiler to compile the segmented model to obtain host code; S3: The processor uses a high-level synthesis tool and the hardware parameters to synthesize a bitstream of the reconfigurable hardware; and S4: The processor obtains the execution time corresponding to the host code and the bitstream.

[0005] The method for model and reconfigurable hardware of the present invention is suitable for an apparatus including a processor, wherein the method includes the following steps: S1: the processor sets the number of pipeline stages and the pipeline segmentation points as software parameters, and the processor sets the tiling size, the number of processing elements (PEs), and the size of the processing elements as hardware parameters, wherein the segmented model includes pipeline stages and segmentation points, and the processing elements correspond to reconfigurable hardware; S2: the processor compiles the segmented model using a machine learning compiler to obtain host code; S3: the processor synthesizes a bitstream of the reconfigurable hardware using a high-level synthesis tool and the hardware parameters; and S4: the processor obtains the execution time corresponding to the host code and the bitstream.

[0006] Based on the above, the apparatus and method for models and reconfigurable hardware of the present invention can set the number of pipeline stages and the pipeline split points as software parameters, and set the block size, the number of processing elements, and the size of the processing elements as hardware parameters. Then, the execution time corresponding to the host program code and the bit stream can be obtained. Furthermore, the apparatus and method for models and reconfigurable hardware of the present invention can obtain optimized software parameters and optimized hardware parameters. Therefore, the apparatus and method for models and reconfigurable hardware of the present invention can pipeline the model and simultaneously consider the resources of the reconfigurable hardware, thereby integrating software and hardware to optimize model performance.

Implementation Method

[0008] FIG1 is a schematic diagram illustrating an apparatus 100 for a model and reconfigurable hardware according to an embodiment of the present invention. The apparatus 100 may include a processor 150. In other embodiments, the apparatus 100 may include a storage medium (not shown) coupled to the processor 150 and a transceiver (not shown).

[0009] The processor 150 is, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose microcontroller (MCU), microprocessor, digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), graphics processing unit (GPU), image signal processor (ISP), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar elements or combinations thereof. The processor 150 can access and execute multiple modules and various applications stored in a storage medium.

[0010] In one embodiment, the reconfigurable hardware may include an FPGA (Field Programmable Gate Array). However, the invention is not limited thereto.

[0011] FIG2 is a flowchart illustrating a method for a model and reconfigurable hardware according to an embodiment of the present invention, wherein the method may be implemented by the apparatus 100 shown in FIG1. ​​Please refer to FIG1 and FIG2 simultaneously.

[0012] In step S1, the processor 150 can set the number of pipeline stages and the pipeline split points as software parameters, and the processor 150 can set the pinning size, the number of processing elements (PEs) and the size of the processing elements as hardware parameters. The segmented model may include pipeline stages and split points, and the processing elements may correspond to reconfigurable hardware.

[0013] In one embodiment, the trained model may include operators. Before executing step S1, the processor 150 may obtain the operator execution time corresponding to the operator. Then, the processor 150 may segment the trained model into the segmented model described above.

[0014] In one embodiment, the processing element may include an adder tree, wherein the number of adder trees may be Tm. Further, the adder tree may include a multiplier, wherein the number of multipliers may be Tn. More specifically, the size of the processing element may be Tm. On the other hand, the number of processing elements may be Tn.

[0015] In one embodiment, the tile size may correspond to Tre and Tc. Specifically, the processor 150 may input a tensor to the reconfigurable hardware. The processor 150 may then tile the tensor into Tre columns and Tc columns.

[0016] Please refer to Figure 2. In step S2, the processor 150 may use a machine learning compiler to compile the segmented model to obtain host code. In one embodiment, the processor 150 may use intermediate representations of the machine learning compiler to annotate pipeline stages and segmentation points in the segmented model. Based on this, the machine learning compiler may optimize the segmented model to obtain host code. In one embodiment, the processor 150 may enable a reconfigurable hardware accelerator driver in the machine learning compiler.

[0017] In step S3, the processor 150 can use a high-level synthesis tool and hardware parameters to synthesize a bitstream of reconfigurable hardware.

[0018] In step S4, the processor 150 can obtain the execution time corresponding to the host program code and the bit stream. In one embodiment, the processor 150 can use the above-mentioned operand execution time to obtain the execution time corresponding to the host program code and the bit stream.

[0019] In one embodiment, the execution time may include an initial execution time and a current execution time. When the current execution time is greater than or equal to the initial execution time, the processor 150 may re-execute steps S1 to S4. Specifically, the processor 150 may use initial software parameters and initial hardware parameters to execute steps S1 to S4 to obtain an initial execution time. Then, the processor 150 may use updated software parameters and updated hardware parameters to execute steps S1 to S4 again to obtain a current execution time. If the processor 150 determines that the current execution time is greater than or equal to the initial execution time, the processor 150 may re-execute steps S1 to S4. On the other hand, if the processor 150 determines that the current execution time is less than the initial execution time, the processor 150 may accept / adopt updated software parameters and updated hardware parameters. In other words, the processor 150 may obtain optimized software parameters and optimized hardware parameters.

[0020] In one embodiment, if the processor 150 determines that the current execution time is less than a preset execution time threshold, the processor 150 may not re-execute steps S1 to S4.

[0021] Table 1 is an example of the software parameters, hardware parameters and the utilization rate of reconfigurable hardware of the present invention.

[0022] Table 1 Software parameters (convolutional layer) GFLOPS Hardware parameters (T) r T c T n T m ) LUT usage rate DSP utilization BRAM utilization Phase 1 1-3 5.72 (14, 28, 2, 64) 19% 14% 10% Phase Two 4-6 9.25 (28, 14, 8, 48) 26% 36% 18% Phase Three 7-9 9.25 (28, 14, 8, 28) 17% twenty three% 27% Phase 4 10-13 6.46 (14, 14, 16, 16) 19% twenty four% 12% sum - 30.68 - 81% 97% 67%

[0023] Table 2 shows the evaluation results of the present invention using the VGG16 model. As shown in Table 2, the time to process one piece of data can be compressed to 258ms. Compared with using only the CPU, the present invention can achieve a 23.65-fold speedup in execution time.

[0024] Table 2 CPU CPU and reconfigurable hardware Execution time (ms) 6101 258 Compilation time (s) 121 145 Execution time acceleration factor 1 23.65

[0025] In summary, the apparatus and method for models and reconfigurable hardware of the present invention can set the number of pipeline stages and the pipeline split points as software parameters, and set the block size, the number of processing elements, and the size of the processing elements as hardware parameters. Then, the execution time corresponding to the host program code and the bit stream can be obtained. Furthermore, the apparatus and method for models and reconfigurable hardware of the present invention can obtain optimized software parameters and optimized hardware parameters. Based on this, the apparatus and method for models and reconfigurable hardware of the present invention can pipeline the model and simultaneously consider the resources of the reconfigurable hardware, thereby integrating software and hardware to optimize model performance. [Simplified Explanation of the Diagram]

[0007] FIG1 is a schematic diagram of an apparatus for a model and reconfigurable hardware according to an embodiment of the present invention. FIG2 is a flowchart of a method for a model and reconfigurable hardware according to an embodiment of the present invention.

Claims

1. An apparatus for models and reconfigurable hardware, comprising: A processor configured to perform the following steps: S1: The processor sets the number of pipeline stages and the pipeline split points as software parameters, and the processor sets the tile size, the number of processing elements (PEs), and the size of the processing elements as hardware parameters, wherein the segmented model includes the pipeline stages and the split points, wherein the processing elements correspond to the reconfigurable hardware; S2: The processor uses a machine learning compiler to compile the segmented model to obtain host code; S3: The processor uses a high-level synthesis tool and the hardware parameters to synthesize a bitstream of the reconfigurable hardware; and S4: The processor obtains the execution time corresponding to the host code and the bitstream, wherein the tile size corresponds to Tr and Tc, wherein the processor tiles a tensor into Tr columns and Tc columns, wherein the processor inputs the tensor to the reconfigurable hardware.

2. The apparatus of claim 1, wherein the processor segments the trained model into the segmented model.

3. The apparatus of claim 2, wherein the processor obtains an operator execution time corresponding to an operator, wherein the trained model includes the operator.

4. The apparatus of claim 3, wherein the processor uses the arithmetic sub-execution time to obtain the execution time corresponding to the host program code and the bit stream.

5. The apparatus of claim 1, wherein the reconfigurable hardware includes an FPGA (Field Programmable Gate Array).

6. The apparatus of claim 1, wherein the processing element comprises an adder tree, wherein the number of adder trees is Tm, and wherein the adder tree comprises a multiplier, wherein the number of multipliers is Tn.

7. The apparatus of claim 1, wherein the execution time includes an initial execution time and a current execution time, wherein when the current execution time is greater than or equal to the initial execution time, the processor re-executes steps S1 to S4.

8. A method for a model and reconfigurable hardware, suitable for an apparatus including a processor, wherein the method includes the following steps: S1: The processor sets the number of pipeline stages and the pipeline segmentation points as software parameters, and the processor sets the titling size, the number of processing elements (PEs), and the size of the processing elements as hardware parameters, wherein the segmented model includes the pipeline stages and the segmentation points, wherein the processing elements correspond to the reconfigurable hardware; S2: The processor compiles the segmented model using a machine learning compiler to obtain host code; S3: The processor synthesizes a bitstream of the reconfigurable hardware using a high-level synthesis tool and the hardware parameters; and S4: The processor obtains the execution time corresponding to the host code and the bitstream, wherein the titling size corresponds to Tr and Tc, wherein the method further includes the following steps: The processor divides the tensor into Tr columns and Tc rows, wherein the processor inputs the tensor to the reconfigurable hardware.

9. The method as described in claim 8 further includes the step of: the processor segmenting the trained model into the segmented model.

10. The method of claim 9 further includes the step of: obtaining, by the processor, an operator execution time corresponding to an operator, wherein the trained model includes the operator.

11. The method of claim 10, wherein the step of obtaining the execution time corresponding to the host program code and the bit stream by the processor includes: The processor uses the execution time of the arithmetic sub-processor to obtain the execution time corresponding to the host program code and the bit stream.

12. The method of claim 8, wherein the reconfigurable hardware includes an FPGA (Field Programmable Gate Array).

13. The method of claim 8, wherein the processing element comprises an adder tree, wherein the number of adder trees is Tm, and wherein the adder tree comprises a multiplier, wherein the number of multipliers is Tn.

14. The method as described in claim 8, wherein the execution time includes an initial execution time and a current execution time, and wherein the method further includes the following steps: when the current execution time is greater than or equal to the initial execution time, the processor re-executes steps S1 to S4.

Images