A fast code generation device supporting the generation of fusion operators

Abstract

A fast code generation device supporting the generation of fusion operators, belonging to the technical field of deep learning. The present invention includes: an LDM area division module for functionally partitioning the local storage space according to the network size parameters input by the upper-layer framework; a fusion operator address configuration module for defining the addresses of the input, output, and intermediate result data in the operator in the functional partition according to the type of fusion operator input by the upper-layer framework; a fusion operator data interaction module providing function interfaces for asynchronous memory access between the local and the main memory, and between the local and the local; a SIMD fusion operator calculation module for fusing the operator according to the addresses generated by the fusion operator address configuration module; and a systolic array instruction configuration module for configuring the instructions for driving the systolic array to perform calculations. The present invention can effectively reduce the code error rate, improve the code generation efficiency, and simplify the debugging process.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a fast code generation device that supports fusion operator generation. Background Art

[0002] Due to the universality of some fusion operators and the universality of the calculation modules within the operators, using Python to implement the assembly code generator can reduce some of the workload. For example, in ResNet50, some layer parameters are the same, and most layers have the same calculation modules such as Norm_b, ReLU, and Conv, so general functions can be directly constructed to implement these calculation modules. In addition, handwritten code will have register reference errors, instruction usage errors, loop nesting errors, etc., and using library functions can reduce these errors with a high probability. In addition, the calculation order or method of different networks is different. If it is manually written, it will be time-consuming, labor-intensive, and prone to errors. Summary of the invention

[0003] The purpose of the present invention is to solve the problems existing in the above-mentioned prior art and to provide a fast code generation device supporting fusion operator generation, which can effectively reduce the code error rate, improve the code generation efficiency and simplify the debugging process.

[0004] The objective of the present invention is achieved through the following technical solutions:

[0005] A fast code generation device supporting fusion operator generation, comprising:

[0006] LDM area partition module is used to functionally partition the local storage space according to the network size parameters input by the upper framework;

[0007] The fusion operator address configuration module is used to define the addresses of the input, output, and intermediate result data in the operator in the functional partition according to the fusion operator type input by the upper-level framework;

[0008] The fusion operator data interaction module provides a function interface for asynchronous memory access between local and main memory, and between local and local memory.

[0009] A SIMD fusion operator calculation module, used for fusing operators according to addresses generated by the fusion operator address configuration module;

[0010] The systolic array instruction configuration module is used to configure the instructions for driving the systolic array to perform calculations.

[0011] The device encapsulates different instructions into corresponding functional modules according to their functions. When writing operators, various functions in the corresponding functional modules can be directly called, which reduces the workload of manual writing and reduces the error rate.

[0012] Preferably, the LDM area division module specifically includes:

[0013] A storage partition unit that partitions the local storage space according to the network size parameter types. The network size parameters include picture size, convolution kernel size, and result size; that is, partitioning by function for easy management and memory access.

[0014] A storage allocation unit that allocates corresponding storage space according to the size of each network size parameter; that is, allocating storage space according to size to maximize the utilization of local storage space while ensuring memory access efficiency.

[0015] Preferably, the fusion operator types include pooling calculation operator, de-pooling calculation operator, backward update residual calculation operator, and forward training parameter calculation operator.

[0016] Preferably, the fusion operator data interaction module realizes asynchronous memory access according to the local data address and data volume, as well as the external broadcast and main memory address.

[0017] Preferably, the systolic array instruction configuration module includes: a systolic array initial configuration unit, a systolic array data loading descriptor configuration unit, and an accumulator buffer result write-back descriptor configuration unit, thereby driving the entire process of systolic array calculation.

[0018] Preferably, it includes a file management module, and the file management module includes:

[0019] A define file unit that defines a file and enables code generation in this file;

[0020] A clear file unit for clearing the entire file;

[0021] An add comment unit for adding comments to the file;

[0022] A save and close unit for saving and closing the file.

[0023] Preferably, it includes a load data instruction module for generating the address of an immediate number in immediate addressing; since the addresses of some immediate numbers are very large, this module is used to improve the generation efficiency.

[0024] Preferably, it further includes an instruction pipeline arrangement module for evaluating the performance of the generated code to ensure that the code can run effectively.

[0025] The advantages of the present invention are: By utilizing the generality of some fusion operators and the generality of the internal calculation modules of the operators, the instructions are encapsulated, and various functions are directly called when writing operators, reducing the manual writing workload and errors, and simplifying the debugging process. Brief Description of the Drawings

[0026] Figure 1 This is a functional block diagram of a fast code generation device for supporting the generation of fusion operators according to the present invention. Detailed Description of the Preferred Embodiments

[0027] The present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0028] As Figure 1 shown, the present invention provides a fast code generation device for supporting the generation of fusion operators, including:

[0029] The LDM area partitioning module is used to perform functional partitioning on the local storage space according to the network size parameters input by the upper-layer framework; the LDM area partitioning module specifically includes:

[0030] The storage partitioning unit partitions the local storage space according to the type of network size parameters, and the network size parameters include picture size, convolution kernel size, and result size; that is, partitioning according to functions for easy management and memory access;

[0031] The storage allocation unit allocates storage space of corresponding size according to the size of each network size parameter; that is, allocating storage space according to size to maximize the utilization of local storage space and ensure memory access efficiency at the same time;

[0032] The fusion operator address configuration module is used to define the addresses of the input, output, and intermediate result data in the operator in the functional partition according to the type of fusion operator input by the upper-layer framework, including pooling calculation operator, de-pooling calculation operator, backward update residual calculation operator, and forward training parameter calculation operator;

[0033] The fusion operator data interaction module provides function interfaces for asynchronous memory access between the local and the main memory, and between the local and the local according to the local data address and data volume, as well as the external broadcast and main memory address;

[0034] The SIMD fusion operator calculation module is used to fuse the operators according to the addresses generated by the fusion operator address configuration module;

[0035] The systolic array instruction configuration module is used to configure the instructions for driving the systolic array to perform calculations, including a systolic array initial configuration unit, a systolic array data loading descriptor configuration unit, and an accumulator buffer result write-back descriptor configuration unit, so as to drive the entire process of systolic array calculation;

[0036] The file management module, the file management module includes:

[0037] The definition file unit defines a file for code generation to occur in this file;

[0038] A file clearing unit for clearing the entire file;

[0039] A comment adding unit for adding comments to the file;

[0040] A save and close unit for saving and closing the file;

[0041] A load data instruction module for generating the address of an immediate number in immediate addressing; since the addresses of some immediate numbers are very large, this module is used to improve the generation efficiency;

[0042] An instruction pipeline arrangement module for evaluating the performance of the generated code and ensuring that the code can run effectively.

[0043] Specifically, this device mainly divides assembly instructions into swap type, SIMD type, and ACE type; then further subdivides the functions in each type: the swap type is mainly responsible for the descriptor configuration of DMA and RMA; the SIMD type is mainly responsible for simple EBOX instructions, simple EBOX calculation modules such as Norm_b / ReLU / Conv, and fusion operators; the ACE type is mainly responsible for the descriptor configuration of SA north-west data loading instructions. On this basis, common operations are encapsulated into individual modules, and at the same time, the corresponding file management, parameter management, and instruction pipeline arrangement are unified. The following are the main functions of this device:

[0044]

[0045]

[0046]

[0047] As mentioned above, it is only a preferred specific implementation manner of the present invention. This specific implementation manner is an implementation based on the overall concept of the present invention, and the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A fast code generation device supporting the generation of fusion operators, characterized in that including: an LDM area division module, configured to perform functional partitioning on local storage space according to network size parameters input by an upper-layer framework; a fusion operator address configuration module, configured to define addresses of input, output, and intermediate result data in the operator in the functional partition according to the type of fusion operator input by the upper-layer framework; a fusion operator data interaction module, providing function interfaces for asynchronous memory access between local and main memory and between local and local; a SIMD fusion operator calculation module, configured to fuse operators according to the addresses generated by the fusion operator address configuration module; a systolic array instruction configuration module, configured to configure instructions for driving the systolic array to perform calculations.

2. The fast code generation device supporting the generation of fusion operators according to claim 1, characterized in that The LDM area division module specifically includes: a storage partition unit, configured to partition local storage space according to the type of network size parameters, where the network size parameters include picture size, convolution kernel size, and result size; a storage allocation unit, configured to allocate storage space of corresponding size according to the size of each network size parameter.

3. The fast code generation device supporting the generation of fusion operators according to claim 1, wherein The types of fusion operators include pooling calculation operators, de-pooling calculation operators, backward update residual calculation operators, and forward training parameter calculation operators.

4. The fast code generation device supporting the generation of a fusion operator according to claim 1, wherein The fusion operator data interaction module realizes asynchronous memory access of data according to local data addresses and data volumes, as well as external broadcasts and main memory addresses.

5. The fast code generation device supporting the generation of a fusion operator according to claim 1, characterized in that The systolic array instruction configuration module includes: a systolic array initial configuration unit, a systolic array data loading descriptor configuration unit, and an accumulator buffer result write-back descriptor configuration unit.

6. The fast code generation device supporting the generation of a fusion operator according to claim 1, characterized in that including a file management module, and the file management module includes: a define file unit, defining a file for code generation to occur in this file; a clear file unit, configured to clear the entire file; an add comment unit, configured to add comments to the file; a save and close unit, configured to save and close the file.

7. The fast code generation device supporting the generation of a fusion operator according to claim 1, wherein including a load data instruction module, configured to generate an address of an immediate number in immediate addressing.

8. The fast code generation device supporting the generation of a fusion operator according to claim 1, characterized in that further including an instruction pipeline arrangement module, configured to evaluate the performance of the generated code.

Images