System and method for simulating micro accelerator

Abstract

The present invention relates to a system for simulating a micro accelerator, comprising: a system emulator that emulates the process from when embedded software is executed in the core of a micro controller unit (MCU) to when it drives an on-chip bus, and transmits parameters for a micro accelerator connected to the on-chip bus; and an RTL simulator that simulates the micro accelerator by dynamically generating an RTL (Registor Transfer Level) structure according to the parameters for the micro accelerator.

Description

Technology Field

[0001] The present invention relates to a system and method for simulating a micro-accelerator. Background Technology

[0003] Recently, active research is underway on on-chip accelerators to accelerate and directly perform Artificial Intelligence (AI) computations on edge devices designed as Micro Controller Unit (MCU)-based embedded systems.

[0004] To design an accelerator connected to the on-chip bus of an MCU, consideration must be given to the AI ​​embedded software running on the MCU host simultaneously with the hardware design. In order to overcome the heterogeneity between the embedded software and the hardware accelerator design and to design an optimal hardware accelerator architecture, it must be possible to quickly profile the target embedded software and accelerator usage at the emulator and simulator levels.

[0005] However, these accelerators have the disadvantage that establishing a verification environment is very complex and time-consuming; while functional verification can be achieved by abstracting at the software level, verification at the actual hardware level is difficult, and designing the hardware for the entire structure incurs excessive time and cost.

[0006] Therefore, research on accelerator simulation methods is necessary to perform such accelerator verification more efficiently. Prior art literature

[0008] Korean Registered Patent Publication No. 10-2426416 The problem to be solved

[0009] The present invention has been devised to solve the above-mentioned problems, and the objective of the present invention is to provide a system and method for simulating a micro-accelerator. means of solving the problem

[0011] A system for simulating a micro accelerator according to an embodiment of the present invention for achieving the above objective includes: a system emulator that emulates the process of embedded software running on the core of a micro controller unit (MCU) to drive an on-chip bus and transmits parameters for a micro accelerator connected to the on-chip bus; and an RTL simulator that simulates the micro accelerator by dynamically generating an RTL (Registor Transfer Level) structure according to the parameters for the micro accelerator.

[0012] A simulation method for a micro accelerator simulation system according to an embodiment of the present invention for achieving the above objective comprises: a micro accelerator simulation system including a system emulator and a Register Transfer Level (RTL) simulator, wherein the system emulator emulates the process up to the point where embedded software is executed in the core of a micro controller unit (MCU) and drives an on-chip bus; a step in which the system emulator transmits parameters for a micro accelerator connected to the on-chip bus; a step in which the RTL simulator dynamically generates an RTL structure according to the parameters for the micro accelerator; and a step in which the RTL simulator simulates the micro accelerator based on the RTL structure. Effects of the invention

[0014] According to one aspect of the present invention described above, by providing a system and method for simulating a micro-accelerator, it is possible to easily explore accelerator specifications that can achieve maximum effect using minimum resources for AI computation acceleration performed on an MCU where available resources are constrained.

[0015] Furthermore, profiling of the MCU's on-chip accelerator is possible at minimal cost, which reduces the overhead of iteratively searching for the optimal accelerator structure for the target software. Brief explanation of the drawing

[0017] FIG. 1 is a diagram showing the overall framework of a micro-accelerator simulation system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating the micro-accelerator simulation system of FIG. 1. FIG. 3 is a diagram abstracting the RTL structure generation operation in a micro-accelerator simulation system according to an embodiment of the present invention, FIG. 4 is a diagram illustrating the operation of generating an RTL structure of FIG. 3. FIG. 5 is a diagram abstracting the DMA modeling operation in a micro-accelerator simulation system according to an embodiment of the present invention, FIG. 6 is a diagram illustrating the DMA modeling operation of FIG. 5. And, FIG. 7 is a flowchart illustrating a micro-accelerator simulation method of a micro-accelerator simulation system according to an embodiment of the present invention. Specific details for implementing the invention

[0018] The following detailed description of the invention refers to the accompanying drawings, which illustrate specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that various embodiments of the invention are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in relation to one embodiment. It should also be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not intended to be limiting, and the scope of the invention is limited only by the appended claims, including all equivalents to those claimed therein, provided appropriately described. Similar reference numerals in the drawings refer to the same or similar functions across various aspects.

[0019] The components according to the present invention are defined by functional distinction rather than physical distinction, and can be defined by the functions each performs. Each component may be implemented as hardware or as program code and processing units that perform each function, and the functions of two or more components may be included and implemented in a single component. Therefore, it should be noted that the names assigned to the components in the following embodiments are not intended to physically distinguish each component but are assigned to imply the representative function performed by each component, and that the technical concept of the present invention is not limited by the names of the components.

[0020] Preferred embodiments of the present invention will be described in more detail below with reference to the drawings.

[0021] FIG. 1 is a diagram showing the overall framework of a micro-accelerator simulation system according to an embodiment of the present invention, and FIG. 2 is a diagram showing the micro-accelerator simulation system of FIG. 1 in detail.

[0022] The illustrated micro-accelerator (MA) simulation system (100) includes a system emulator (102) and a register transfer level (RTL) simulator (104), and is mapped to a micro controller unit (MCU).

[0023] The above MCU includes an on-chip bus (116), which is a bus system for communication between various hardware blocks and components within the hardware design, an on-chip flash memory (110), a core (112), an on-chip static random access memory (114), a direct memory access (DMA) controller (120), and an on-chip accelerator (122) connected to the on-chip bus (116), and a peripheral device (120) connected to the DMA controller (120).

[0024] Here, the on-chip bus (116) may be an Advanced High-performance Bus (AHB) bus (116-1) used in ARM architecture and an Advanced Peripheral Bus (APB) bus (116-2) for communication with low-performance peripheral devices, and the on-chip accelerator (122) may be a micro accelerator (122-1).

[0025] The system emulator (102) functionally emulates the process from when the embedded software is executed on the core (112) to when the on-chip bus (116) is driven, and transmits parameters for the on-chip accelerator (122) connected to the on-chip bus (116) to the RTL simulator (104). Here, the parameters for the on-chip accelerator (122) include information regarding the neural network model of the embedded software and information related to the detailed operation settings of the DMA controller (118) connected to the on-chip bus (116).

[0026] The RTL simulator (104) performs a clock-cycle level simulation to verify whether it operates correctly according to the period of the clock signal when access to the on-chip accelerator (122) occurs through the on-chip bus (116). That is, the RTL simulator (104) simulates the on-chip accelerator (122) by dynamically generating an RTL structure according to parameters for the on-chip accelerator (122).

[0027] More specifically, the micro accelerator (122-1) is connected to the APB bus (116-2) to serve as a peripheral device, and the micro accelerator (122-1) is used to accelerate embedded software running on the MCU, for example, a Tiny Machine Learning (TinyML) application.

[0028] Peripheral devices (120) are configured and triggered by software embedded in the flash memory (110) of the MCU, and likewise, micro-accelerators (122-1) access data required for computation by setting registers in the instruction execution of the flash memory (110). However, integrating micro-accelerators (122-1) designed in RTL into the underlying MCU for performance evaluation is not an easy task, and most cores currently targeted by TinyML applications keep their internal RTL code confidential for commercial purposes (i.e., ARM).

[0029] In addition, even if the entire MCU chip is implemented in RTL and the micro accelerator (122-1) is connected, there is overhead involved in simulating the entire RTL and reflecting the interaction between the software. Therefore, a custom micro accelerator designed in RTL requires simulation based on an accurate period, that is, the clock signal period, and access or initialization of the remaining peripherals is sufficient with high-level abstraction functional emulation.

[0030] Accordingly, the present invention proposes a micro-accelerator simulation system (100) that modifies an emulator to emulate the process of initializing hardware in a system emulator and accessing memory-mapped registers in TinyML firmware to custom micro-accelerator peripherals.

[0031] The system emulator (102) of the micro-accelerator simulation system (100) emulates peripherals connected to the core peripheral bus for an ARM core-based MCU, namely the AHB bus (116-1). The manufacturer of the MCU using the ARM core provides memory-mapped register address regions and register bit field information for all peripherals in the MCU in the form of a System View Description (SVD).

[0032] In addition, the system emulator (102) adopted a QEMU structure that parses the SVD and automatically generates a template for each peripheral device that converts firmware-level register access into an emulator program, adds micro accelerator (122-1) peripheral device register information to the SVD file, and enables micro accelerator access using TinyML firmware instructions.

[0033] If the micro-accelerator simulation system (100) can access the peripheral registers of the micro-accelerator (122-1), the behavior of the system emulator (102) can be defined at the firmware level when read or write access to each register occurs. Like other peripherals, the micro-accelerator (122-1) also triggers calculations by writing activation values ​​to memory-mapped registers. At this time, the micro-accelerator simulation system (100) remotely runs the RTL simulator (104) to periodically and accurately evaluate the work performed within the generated micro-accelerator (122-1).

[0034] Additionally, the micro-accelerator simulation system (100) includes an RTL generation engine (130) for evaluating the performance of a flexible and scalable micro-accelerator (122-1) when running the RTL simulator (104) remotely.

[0035] Meanwhile, the RTL simulator (104) simulates the micro accelerator (122-1) by dynamically generating an RTL structure according to parameters for the micro accelerator (122-1) received from the system emulator (102). Here, the parameters for the micro accelerator (122-1) include information about the neural network model of the TinyML application and information related to the detailed operation settings of the DMA controller (118) connected to the AHB bus (116-1) and the APB bus (116-2).

[0036] First, information regarding the neural network model includes parameters used in the AI ​​operation of the neural network model, and the RTL simulator (104) generates an RTL structure according to the parameters used in the AI ​​operation of the neural network model. At this time, assuming that the neural network model is a Convolutional Neural Network (CNN) model, the parameters used in the AI ​​operation of the neural network model may be image size, filter size, filter stride, and number of channels.

[0037] Additionally, parameters determining this RTL structure must define registers that connect the firmware layer containing the remote execution code of the RTL simulator (104) and the emulator layer. When the TinyML firmware writes the above parameters to the Special Function Register (SFR) of the micro-accelerator (122-1), the system emulator (102) transmits the written register value to the RTL simulator (104). The micro-accelerator simulation system (100) evaluates the clock latency when using the micro-accelerator (122-1) in a 32-bit processor architecture.

[0038] Next, information related to the detailed operation settings of the DMA controller (118) includes at least one parameter among the clock required by the DMA controller (118), the memory read clock, and the memory write clock, and the RTL simulator (104) simulates the micro accelerator (122-1) according to at least one parameter among the clock required by the DMA controller (118), the memory read clock, and the memory write clock.

[0039] Finally, the RTL simulator (104) transmits the performance profile result of the micro accelerator (122-1) as the simulation result to the system emulator (102).

[0040] Below, the operation of generating an RTL structure for evaluating the performance of a micro accelerator (122-1) will be described in more detail through FIGS. 3 and FIGS. 4.

[0041] FIG. 3 is a diagram abstracting the RTL structure generation operation in a micro-accelerator simulation system according to an embodiment of the present invention, and FIG. 4 is a diagram specifying the RTL structure generation operation of FIG. 3.

[0042] The illustrated RTL generation engine (130) dynamically generates an RTL structure based on parameters used in AI computations, such as neural network models of embedded software, for example, CNN models, and fully connected (FC) layer models. The RTL simulator (104) simulates a micro-accelerator based on the generated RTL structure and returns the performance profile result of the micro-accelerator to the system emulator as a result of the simulation.

[0043] Figure 4 illustrates the process of generating an RTL structure for a microaccelerator based on neural network model information recorded in the four register fields of the microaccelerator, where it is assumed that the four 32-bit regions for the microaccelerator are reserved in the empty spaces of the existing MCU registers.

[0044] The DMA model, which models the RTL structure and the operation of the DMA controller, is generated by the configured values ​​of the registers. Since the micro-accelerator simulation system proposed in this invention evaluates performance when using a micro-accelerator in TinyML firmware, actual data is not transmitted to the micro-accelerator's input, output, and weight registers; instead, only the row and column sizes of the matrix are transmitted to the RTL simulator to reconstruct matrix operations using dummy data. Here, the micro-accelerator's input register and weight register are used to determine the matrix size when generating the RTL structure, and the most significant bit (MSB) and least significant bit (LSB) of the 32-bit registers represent the row and column sizes, respectively.

[0045] In Figure 4, a Convolutional Neural Network (CNN) and a Fully Connected (FC) layer are described as examples of neural network models supported by a micro accelerator according to an embodiment of the present invention.

[0046] At this time, neural network model information, such as CNN models and FC layer models, is also recorded in the control register (CR register), because the structure of the RTL generated differs depending on the type of artificial intelligence operation used in the neural network model. The remaining fields of the CR register record values ​​that model the clock timing required by the DMA controller and the clock timing resulting from memory access between the micro-accelerator and the DMA controller. That is, the clock required by the DMA controller, the memory read clock, and the memory write clock are recorded.

[0047] A micro-accelerator according to an embodiment of the present invention is defined as a peripheral device connected to the on-chip bus of an MCU, and access to such a peripheral device is achieved through a special function register mapped to a memory address. Therefore, for a micro-accelerator to be connected to the core of an MCU as a peripheral device, the memory address space must be mapped to a register that allows TinyML software to configure the hardware of the micro-accelerator. This register is used to trigger the start of the micro-accelerator's operation or to exchange data required for computation. However, to accelerate the computation of a neural network model, a large amount of data transmission is required due to the large volume of weights, activation values, and output values.

[0048] However, MCUs have limitations in memory capacity compared to PCs (Personal Computers) or servers, and if all data used by the micro-accelerator is mapped to memory register addresses, it exceeds the physical memory address range of the MCU. Therefore, it is inevitable for the micro-accelerator to use a DMA controller to access the data required for AI computation of neural network models.

[0049] Meanwhile, the RTL generation engine (130) generates an RTL structure for the micro-accelerator using a standard systolic array. Inside the Process Element (PE), there are three registers, and it operates by passing weights from the left to the right from the weight register, passing inputs from the top to the bottom from the input register, and calculating multiplication and accumulation (MAC).

[0050] The RTL structure generated for the FC layer model has PEs equal to the number of output nodes and calculates the MAC of the weights and inputs for each cycle. During this process, a one-clock-cycle delay occurs depending on the column index of the weights, and the MAC result for each PE represents the output when all inputs and weights have been consumed in the PE.

[0051] The PE matrix of the RTL structure generated for the CNN model is produced by multiplying the output column and row created by the filter by the filter column. The RTL structure of the CNN model sums each PE of the row into a single output value, and register delays are used for the weights and inputs flowing into the PE matrix.

[0052] Below, the DMA modeling operation in which the RTL simulator (104) models the operation of the DMA controller will be explained in more detail through FIGS. 5 and 6.

[0053] FIG. 5 is a diagram abstracting the DMA modeling operation in a micro-accelerator simulation system according to an embodiment of the present invention, and FIG. 6 is a diagram specifying the DMA modeling operation of FIG. 5.

[0054] The RTL simulator (104), having received information regarding the detailed operation settings of the DMA controller from the system emulator (102), models the operation of the DMA controller to simulate the data transmission and reception overhead between the core of the MCU and the micro-accelerator. That is, the RTL simulator (104) simulates the operation of the DMA controller according to the clock cycles that occur during access through the DMA controller in the process of reading and writing data to the internal buffer of the micro-accelerator. Here, the detailed operation setting information of the DMA controller includes clock, memory read clock, and memory write clock parameters required by the DMA controller.

[0055] Figure 6 illustrates the process of modeling data transfer of a micro-accelerator connected to the on-chip bus of an MCU. When transferring a large amount of data without the intervention of a Central Processing Unit (CPU), the DMA controller requires repeated transfers because the size of the data that can be processed at once is fixed. At the same time, the source or destination address of the data can be fixed or automatically incremented.

[0056] In addition, as previously mentioned, because the microaccelerator requires a limited register sector space, the DMA controller accesses a single peripheral address, and when on-chip bus access to a peripheral occurs in the DMA controller, the internal buffer address within the microaccelerator is incremented. When the DMA controller stores data in the microaccelerator's input and weight buffers and loads data from the output buffer, the microaccelerator can transfer sequential data through the same peripheral address access.

[0057] DMA modeling information that models the operation of this DMA controller, that is, detailed operation setting information of the DMA controller, is defined in the CR register of the microaccelerator. When the microaccelerator is enabled by the SFR configuration, the DMA request is set to populate the input register and the weight register.

[0058] The clock required by the DMA controller is the clock parameter until the DMA controller responds with data to the peripheral bus in response to a DMA request. The memory read clock and memory write clock are clock parameters required to access memory in successive DMA operations.

[0059] The detailed operation configuration information of this DMA controller is used to create a test bench that models the data transfer process of the micro-accelerator and peripheral bus protocols. When a data transfer request is generated from the micro-accelerator, the data used for AI computation of the neural network model is provided to the micro-accelerator or transferred to the system emulator according to the operation of the configured DMA controller.

[0060] FIG. 7 is a flowchart illustrating a micro-accelerator simulation method of a micro-accelerator simulation system according to an embodiment of the present invention.

[0061] The system emulator emulates the process from the execution of embedded software to driving the on-chip bus, that is, the process from the execution of the embedded software in the core of the MCU to driving the on-chip bus (S702), and transmits parameters for a micro accelerator connected to the on-chip bus (S704).

[0062] The RTL simulator dynamically generates an RTL structure based on parameters for the micro-accelerator received in S704 (S706) and simulates the micro-accelerator based on the generated RTL structure (S708).

[0063] Then, the RTL simulator returns the performance profile result of the micro-accelerator to the system emulator as the simulation result of S708. (S710)

[0064] The method for simulating a micro-accelerator according to the present invention as described above can be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination.

[0065] The program instructions recorded on the above-mentioned computer-readable recording medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software.

[0066] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, and flash memory.

[0067] Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0068] Although various embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the invention as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present invention.

Claims

Claim 1 A system emulator for a system simulating a micro accelerator, which emulates the process from when embedded software runs on the core of a micro controller unit (MCU) to when it drives an on-chip bus, and transmits parameters for a micro accelerator connected to the on-chip bus; A micro-accelerator simulation system comprising: an RTL simulator that simulates the micro-accelerator by dynamically generating an RTL (Registor Transfer Level) structure according to parameters for the micro-accelerator; wherein the parameters for the micro-accelerator include parameters used in artificial intelligence (AI) operations of a neural network model of the embedded software; and wherein the RTL simulator dynamically generates the RTL structure including a structure of a processing element (PE) array including a systolic array according to the parameters used in the AI ​​operations, wherein the number or matrix configuration of the processing elements (PE) is determined based on the parameters used in the AI ​​operations, and wherein the RTL simulator simulates the performance of the micro-accelerator by reconstructing operations by generating dummy data corresponding to the matrix size included in the parameters without receiving actual operation data. Claim 2 A micro-accelerator simulation system according to claim 1, characterized in that the parameters for the micro-accelerator include information regarding the neural network model of the embedded software and information related to the detailed operation settings of a Direct Memory Access (DMA) controller connected to the on-chip bus. Claim 3 delete Claim 4 A microaccelerator simulation system according to claim 2, wherein the information related to the detailed operation setting of the DMA controller includes at least one parameter among the clock required by the DMA controller, the memory read clock, and the memory write clock, and the RTL simulator simulates the microaccelerator according to at least one parameter among the clock required by the DMA controller, the memory read clock, and the memory write clock. Claim 5 A micro-accelerator simulation system according to claim 1, characterized in that the RTL simulator returns the performance profile result of the micro-accelerator as the simulation result to the system emulator. Claim 6 A micro-accelerator simulation system according to paragraph 2, wherein the RTL simulator models the operation of the DMA controller, and modeling the operation of the DMA controller models the data transfer between the core and the micro-accelerator. Claim 7 A simulation method for a micro accelerator simulation system comprises a system emulator and a Register Transfer Level (RTL) simulator, wherein the system emulator emulates the process from the execution of embedded software in the core of a micro controller unit (MCU) to the operation of an on-chip bus; the system emulator transmits parameters for a micro accelerator connected to the on-chip bus; and the RTL simulator dynamically generates an RTL structure according to the parameters for the micro accelerator. A method for simulating a micro-accelerator, comprising the step of simulating the micro-accelerator based on the RTL structure using the RTL simulator; wherein the parameters for the micro-accelerator include parameters used in artificial intelligence (AI) operations of a neural network model of the embedded software; and the step of dynamically generating the RTL structure includes generating the RTL structure including the structure of a processing element (PE) array including a systolic array according to the parameters used in the AI ​​operation, wherein the RTL structure is generated such that the number or matrix configuration of the processing elements (PE) is determined based on the parameters used in the AI ​​operation, and wherein the RTL simulator simulates the performance of the micro-accelerator by reconstructing the operation by generating dummy data corresponding to the matrix size included in the parameters without receiving actual operation data. Claim 8 A micro-accelerator simulation method according to claim 7, characterized in that the parameters for the micro-accelerator include information regarding the neural network model of the embedded software and information related to the detailed operation settings of a Direct Memory Access (DMA) controller connected to the on-chip bus. Claim 9 delete Claim 10 A micro-accelerator simulation method according to claim 8, wherein the information related to the detailed operation setting of the DMA controller includes at least one parameter among the clock required by the DMA controller, the memory read clock, and the memory write clock, and the step of simulating the micro-accelerator is to simulate the micro-accelerator according to at least one parameter among the clock required by the DMA controller, the memory read clock, and the memory write clock. Claim 11 A microaccelerator simulation method according to claim 7, further comprising the step of the RTL simulator returning the performance profile result of the microaccelerator as the simulation result to the system emulator. Claim 12 A micro-accelerator simulation method according to claim 8, wherein the step of simulating the micro-accelerator includes the step of modeling the operation of the DMA controller, and the step of modeling the operation of the DMA controller models the data transmission between the core and the micro-accelerator.

Images