Deep neural network accelerator for optimizing data processing and control method thereof

Abstract

A control method for optimizing a deep neural network (DNN) accelerator for data processing is provided. The control method includes calculating a plurality of offsets representing starting components of a plurality of data tiles in neural network data assigned to a first level memory based on a data flow and a hardware mapping value; identifying a data type of an update data tile in the plurality of data tiles corresponding to a received update request for the neural network data based on receiving the update request from a second level memory; identifying one or more components of the update data tile based on the data type of the update data tile and an offset of the update data tile in the calculated plurality of offsets; and updating neural network data of the identified one or more components between the first level memory and the second level memory.

Description

[0001] Cross-reference to related applications

[0002] This application is based on Korean Patent Application No. 10-2022-0038795 filed with the Korean Intellectual Property Office on March 29, 2022 and Korean Patent Application No. 10-2023-0016353 filed on February 7, 2023, and claims priority to it under 35 USC §119, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] One or more embodiments relate to deep neural network accelerators, and more specifically, to deep neural network accelerators that support flexible hardware mapping and data flow for optimizing data processing, and to control methods for deep neural network accelerators. Background Technology

[0004] To optimally execute various configurations and layers of deep neural networks (DNNs), accelerators are needed to perform DNN workloads utilizing various data streams. To this end, various efforts have been made in this disclosure to optimize computation by providing flexibility in data streams to the accelerators.

[0005] Traditionally, attempts have been made to find an execution method for a small number of target workloads at the hardware design level by heuristically defining the order in which multidimensional data should be processed and the methods by which the data is tiled and spatially distributed in processing elements (PEs). However, this heuristic design strategy is unsustainable because the optimal data flow and optimal hardware mapping solution vary greatly depending on various workload configurations and hardware specifications such as buffer size and the number of PEs.

[0006] To provide flexible scheduling options for DNNs, recent accelerators have tended to incorporate reconfigurable but cumbersome interconnect networks between data peers (PEs). These accelerators are hardware-intensive, and the complexity of the network-on-chip (NoC) must be controlled to activate reconfiguration during hardware design. However, the NoC of the accelerator consumes a significant amount of hardware overhead, and the relative contribution of NoC overhead increases with the accelerator size (e.g., the number of PEs). Therefore, the entire configuration of traditional accelerators is implemented in a biased manner to support specific types of data flows. However, by only providing flexible hardware mapping choices and invariant data flows, a significant opportunity to further improve the energy efficiency and performance of the accelerator is missed. Summary of the Invention

[0007] Additional aspects will be set forth in part in the description which follows, and will be apparent in part from the description, or may be learned by practicing the embodiments presented in this disclosure.

[0008] A control method for a deep neural network (DNN) accelerator is provided. The control method may include calculating multiple offsets representing start components of multiple data tiles of neural network data based on a data stream of neural network data allocated to a first-level memory and hardware mapping values. The control method may include identifying the data type of an updated data tile corresponding to a received update request for neural network data in the multiple data tiles based on receiving an update request for neural network data from a second-level memory. The control method may include identifying one or more components in the updated data tiles based on the data type of the updated data tiles and the offsets of the updated data tiles among the calculated multiple offsets. The control method may include updating the identified neural network data of the one or more components between the first-level memory and the second-level memory. The data stream may include information about the data type of the neural network data reused in the second-level memory among the multiple data types of neural network data. The hardware mapping values ​​may include information about the shape of the data tiles of the neural network data allocated to the first-level memory and the second-level memory.

[0009] A DNN accelerator for optimizing data processing is provided. The DNN accelerator may include a first-level memory, a second-level memory, and at least one processor configured to control the operation of the first-level memory and the second-level memory. Based on a data stream of neural network data allocated to the first-level memory and hardware mapping values, the at least one processor may calculate multiple offsets representing the start components of multiple data tiles of the neural network data. Based on receiving an update request for neural network data from the second-level memory, the at least one processor may identify the data type of the updated data tile corresponding to the received update request among the multiple data tiles. The at least one processor may identify the storage address of the neural network data of the updated data tile allocated to the first-level memory based on the data type of the updated data tile and the offset of the updated data tile among the multiple offsets. The at least one processor may update the neural network data stored at the identified storage address between the first-level memory and the second-level memory. The data stream may include information about the data type of the neural network data reused by the second-level memory among the multiple data types of the neural network data. The hardware mapping values ​​may include information about the shape of the data tiles of the neural network data allocated to the first-level memory and the second-level memory.

[0010] A non-transient computer-readable recording medium is provided, on which a program is recorded for performing at least one control method for a DNN accelerator described above and later. Attached Figure Description

[0011] The above and other aspects, features, and advantages of certain embodiments of the present invention will become more apparent from the following description taken in conjunction with the accompanying drawings, wherein:

[0012] Figure 1 This is a schematic diagram of the structure of a deep neural network (DNN) accelerator according to an embodiment;

[0013] Figure 2 This is a block diagram showing the detailed structure of the DNN accelerator according to an embodiment;

[0014] Figure 3 This is a view used to interpret hardware mapping values ​​according to an embodiment;

[0015] Figure 4 This is a flowchart of the operation method of the DNN accelerator according to an embodiment;

[0016] Figure 5 It is a graph used to explain the neural network data updated between the global buffer and the local buffer according to an embodiment;

[0017] Figure 6 This is a view used to explain the detailed structure of the memory controller according to an embodiment;

[0018] Figure 7 This is pseudocode describing the operation of the offset calculator according to an embodiment;

[0019] Figure 8 This is pseudocode describing the operation of the sorting manager according to an embodiment;

[0020] Figure 9 This is pseudocode describing the operation of the address calculator according to an embodiment;

[0021] Figure 10 This is a table used to compare the specifications of the DNN accelerator according to the embodiment with other DNN accelerators;

[0022] Figure 11A and 11B It is a graph showing a comparison between the respective performance of the DNN accelerator according to the embodiment and other DNN accelerators;

[0023] Figure 12A and 12B This is a graph showing a comparison of performance and design area with or without the interconnect network of the DNN accelerator according to an embodiment; and

[0024] Figure 13 This is a block diagram showing the detailed configuration of the DNN accelerator according to an embodiment. Detailed Implementation

[0025] Reference will now be made in detail to embodiments, examples of which are shown in the accompanying drawings, wherein the same reference numerals refer to the same elements throughout. In this respect, the embodiments may take different forms and should not be construed as limited to the description set forth herein. Therefore, the embodiments are described below only by reference to the accompanying drawings to explain various aspects of this specification. As used herein, the term “and / or” includes any and all combinations of one or more of the listed items. Expressions preceding the list of elements (such as “at least one of”) modify the entire list of elements without modifying any individual element within the list.

[0026] Embodiments of the present disclosure will now be described more fully with reference to the accompanying drawings. However, in the following description, detailed descriptions of well-known functions or configurations will be omitted where there is a risk of unnecessarily obscuring the essential points of the disclosure.

[0027] In the accompanying drawings, the same or corresponding elements are given the same reference numerals. Furthermore, in the following description of the embodiments, overlapping descriptions of the same or corresponding components may be omitted. However, omitting a description of a component does not mean that such a component is not included in a particular embodiment.

[0028] The advantages and features of the disclosed embodiments, as well as the methods for achieving these advantages and features, will become apparent from the embodiments of this disclosure described below with reference to the accompanying drawings. However, this disclosure may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of this disclosure to those skilled in the art.

[0029] The terminology used herein will be briefly described, and embodiments will be described in detail. Although commonly used terms have been chosen to describe this disclosure in consideration of its functionality, these commonly used terms may vary depending on the intent of those skilled in the art, precedents, the emergence of new technologies, etc. Terms arbitrarily chosen by the applicant of this disclosure may also be used in specific circumstances. In such cases, their meanings need to be given in the detailed description. Therefore, terms must be defined based on their meanings and the entirety of the specification, rather than simply stating the terms.

[0030] Unless explicitly stated in the context as singular, a statement used in the singular includes multiple statements. Multiple statements include singular statements unless explicitly stated in the context as plural. The terms “comprising” and / or “including” or “including” and / or “including”, when used in this specification, specify the presence of the stated element but do not exclude the presence or addition of one or more other elements.

[0031] "Neural network" is a representative example of an artificial neural network model that simulates brain nerves, but is not limited to artificial neural network models that use a specific algorithm. According to embodiments, a neural network may also be referred to as a deep neural network.

[0032] As used herein, "neural network accelerator" can refer to a processor specifically optimized for handling deep neural network workloads or an electronic device including such a processor. According to embodiments, a neural network accelerator may also be referred to as a deep neural network accelerator.

[0033] The term "workload" as used in this article can refer to data processing performed on one or more layers that make up a neural network.

[0034] As used herein, "memory" can be broadly interpreted to include any electronic component capable of storing electronic information. "Memory" can refer to various types of processor-readable media, such as random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, and registers. However, embodiments are not necessarily limited to these. A memory buffer can be declared to be in electronic communication with the processor (or memory controller) when the processor (or memory controller) is capable of reading information from and / or writing information to the memory.

[0035] According to embodiments, "memory" can refer to a memory hierarchy or a level of memory among multiple levels of memory forming a memory hierarchy. For example, the memory hierarchy of a deep neural network accelerator may include DRAM as off-chip memory to global buffers (GLBs), local buffers (LBs), and registers as on-chip memory, and "memory" may refer to all of them or to a specific level of memory included in a memory hierarchy corresponding to one of them. Data processing commands sent to or received from the "memory" may refer to sending and receiving by a "memory controller" that controls the operation of the "memory".

[0036] The term "neural network data" as used in this article can refer to data used or created when a deep neural network accelerator performs a workload. Neural network data, as tensors with one or more dimensions, can be stored in at least one memory included in the deep neural network accelerator.

[0037] The "neural network data" used in this document can have one of several data types. These multiple data types may include input data types, weight data types, and output data types. According to an embodiment, a "data tile" may refer to a tensor corresponding to a portion of a tensor that includes the entire neural network data. Because the entire neural network data is a large-scale multidimensional tensor, the data of the entire neural network can be distributed to multiple levels of memory at different memory levels in units of data tiles, or it can be updated between different levels of memory.

[0038] As used herein, “updating neural network data” may refer to a lower-level memory (or a lower-level memory controller) responding to an update request by sending neural network data of the update data tile corresponding to the update request from the neural network data stored in the lower-level memory to a higher-level memory, or storing neural network data received from the higher-level memory in the update data tile.

[0039] Figure 1 This is a schematic diagram of the structure of a deep neural network (DNN) accelerator 100 according to an embodiment. Figure 1 As shown, the DNN accelerator 100 may include a GLB 110, a global controller 120, and a processing element (PE) array 130.

[0040] DNN accelerator 100 can refer to a processor optimized to handle DNN workloads or an electronic device that includes such a processor.

[0041] GLB 110 can store neural network data necessary for executing the workload of DNN accelerator 100. According to an embodiment, GLB 110 can be one of multiple levels of memory in the memory hierarchy of DNN accelerator 100. GLB 110 can be connected to other levels of memory in the memory hierarchy and can send and receive neural network data.

[0042] Global controller 120 can be a memory controller that controls the operation of GLB 110 to perform DNN workloads. Global controller 120 can control the transmission / reception of neural network data stored in GLB 110 to perform DNN workloads.

[0043] According to an embodiment, the global controller 120 can receive update requests for neural network data from at least one PE included in the PE array 130. In response to the update request, the global controller 120 can update the neural network data corresponding to the update request stored in the neural network data in the GLB 110 between the GLB 110 and a higher-level memory (e.g., a LB) included in the at least one PE. According to an embodiment, the global controller 120 can send the update request for neural network data to a memory at a lower level than the GLB 110 in the memory hierarchy, or to a memory controller for controlling the operation of the lower-level memory, and can store the neural network data sent in response to the update request in the GLB 110. For example, when all neural network data allocated to the GLB 110 is updated, the global controller 120 can send an update request for new neural network data to a memory at a lower level than the GLB 110.

[0044] The PE array 130 may include at least one PE. Each PE may perform a DNN workload based on neural network data received from the GLB 110. The multiple PEs include an LB (not shown) storing neural network data, an LB controller (not shown) for controlling the transmission / reception of the neural network data stored in the LB, and at least one arithmetic unit (not shown). Reference will be made below. Figure 2 Describe its details.

[0045] According to an embodiment, because the size of the multiple levels of memory included in the DNN accelerator 100 is finite, neural network data can be allocated to the multiple levels of memory as data tiles, which are part of the overall neural network data, during workload execution. The determination of the shape of the data tiles allocated to each of the multiple levels of memory can be referred to as the hardware mapping of the neural network data. A hardware mapping value, which includes information about the shape of the data tiles allocated to each of the multiple levels of memory, can be determined based on the hardware mapping of the neural network data. Reference will be made below. Figure 3 Describe its details.

[0046] According to an embodiment, when the DNN accelerator 100 performs a workload, updates to the neural network data may occur multiple times between adjacent levels of memory in the memory hierarchy. During this process, some neural network data updated in the higher-level memory can be reused, and the order among the multiple updated data tiles can be determined based on the data type of the reused neural network data. In this case, the movement of neural network data determined by which of the multiple data types of the reused neural network data is referred to as a data stream. For example, the data stream may include at least one of an input static (IS) data stream formed when input data is reused, a weight static (WS) data stream formed when weight data is reused, and an output static (OS) data stream formed when output data is reused. Thus, the data stream of the neural network data may include information about the data type of the neural network data reused in the higher-level memory in the adjacent level of memory in the memory hierarchy where the neural network data update is performed.

[0047] According to an embodiment, the energy efficiency and performance of the DNN accelerator 100 are determined based on how the hardware mapping and data flow of neural network data allocated to a specific level of memory are selected. The energy efficiency of the DNN accelerator relates to the energy required for the DNN accelerator to perform operations on DNN layers; good or high energy efficiency may mean lower energy required to perform all operations on a specific layer. The performance of the DNN accelerator relates to the data processing cycles (or time) required for the DNN accelerator to perform operations on layers, and good or high performance may mean lower data processing cycles required to perform all operations on a specific layer. In other words, for the DNN accelerator 100, it is important to select the optimal hardware mapping and optimal data flow to achieve at least one of high energy efficiency and high performance. The hardware mapping and data flow that enable the optimal energy efficiency and / or performance of the DNN accelerator can vary depending on the hardware specifications of the DNN accelerator 100 and / or the structure of the DNN layers. In other words, the DNN accelerator needs to be able to support flexible data flow and flexible hardware mapping to ensure versatility across various DNN layers and achieve optimal energy efficiency and performance across various hardware specifications.

[0048] According to an embodiment, based on the data stream of neural network data allocated to the first-level memory and hardware mapping values, the DNN accelerator 100 can calculate multiple offsets representing the start components of multiple data tiles of the neural network data. According to an embodiment, based on receiving an update request for neural network data from the second-level memory, the DNN accelerator 100 can identify the data type of the updated data tile corresponding to the received update request among the multiple data tiles. According to an embodiment, the DNN accelerator 100 can identify one or more components in the updated data tiles based on the data type of the updated data tiles and the offsets of the updated data tiles among the calculated multiple offsets. According to an embodiment, the DNN accelerator 100 can update the identified neural network data of one or more components between the first-level memory and the second-level memory. Details will be described below with reference to the following figures.

[0049] Figure 2 This is a block diagram showing the detailed structure of a DNN accelerator according to an embodiment. Figure 2 As shown, the DNN accelerator 200 may include a GLB 210, a global controller 220, and a PE array 230. The DNN accelerator 200, GLB 210, global controller 220, and PE array 230 may respectively correspond to... Figure 1 The DNN accelerator 100, GLB 110, global controller 120, and PE array 130.

[0050] The PE array 230 may include multiple PEs. PE 232 of the multiple PEs may perform a DNN workload based on neural network data received from GLB 210. PE may include an LB 234 storing neural network data therein, a local controller 236 for controlling the transmission / reception and storage of the neural network data stored in LB 234, and at least one arithmetic unit 238.

[0051] LB 234 can store neural network data necessary for executing DNN workloads. According to an embodiment, LB 234 is one of multiple levels of memory in the memory hierarchy of the DNN accelerator 200, and can be a higher level of memory than GLB 210. According to an embodiment, LB 234 can be connected to other levels of memory in the memory hierarchy of the DNN accelerator 200, and can send and receive neural network data.

[0052] Local controller 236 can control the operation of LB 234 to perform the DNN workload. Local controller 236 can control the transmission / reception of neural network data stored in LB 234 to perform the DNN workload.

[0053] According to an embodiment, local controller 236 can receive update requests for neural network data from upper-level memory. In response to the update request, local controller 236 can update the neural network data corresponding to the update request between LB 234 and a register (not shown), which is upper-level memory included in arithmetic unit 238. According to an embodiment, when all the workload of neural network data allocated to the current LB 234 has been executed, local controller 236 can send update requests for neural network data to global controller 220, which is a lower-level memory, to execute the next workload. Arithmetic unit 238 can receive neural network data from LB 234 and execute DNN workloads. According to an embodiment, arithmetic unit 238 may include a multiply-accumulate (MAC) arithmetic unit (not shown), a register (not shown) storing neural network data, and a register controller (not shown) for controlling the transmission and reception of neural network data stored in the register.

[0054] The register can store neural network data necessary for executing the DNN workload. According to an embodiment, the register is one of multiple levels of memory in the memory hierarchy of the DNN accelerator 200, and can be a higher level of memory than LB 234. According to an embodiment, the register can be connected to other levels of memory in the memory hierarchy of the DNN accelerator 200, and can send and receive neural network data.

[0055] The register controller controls the operation of registers to execute the workload of the DNN accelerator 200. The register controller can control the sending / receiving of neural network data stored in registers to execute the DNN workload.

[0056] According to an embodiment, when all the workloads of neural network data allocated to the current register are executed, the register controller may send an update request for the neural network data to the local controller 236, which is a lower-level memory, to execute the next workload.

[0057] According to an embodiment, the PE array 230 may include a plurality of PEs. For example, the PE array 230 may include a structure in which PEs are arranged in a 16*14 pattern. However, the embodiment is not necessarily limited to this, and the number and arrangement of PEs included in the PE array 230 may be chosen differently.

[0058] According to an embodiment, multiple PEs can execute DNN workloads in parallel. Multiple PEs (or multiple local controllers included in multiple PEs) can send update requests for neural network data from a global controller. According to an embodiment, in response to an update request, the global controller 220 can update the neural network data of the updated data tile between the GLB 210 and the LBs included in the multiple PEs.

[0059] According to an embodiment, an interconnect network (hereinafter referred to as the interconnect network) for sending and receiving neural network data between PEs including different LBs may not be present. An interconnect network is a configuration that can help reduce access to lower-level memory or increase the flexibility of hardware mapping and data flow by sharing neural network data among multiple PEs; however, it can reduce design efficiency due to increased hardware complexity and design costs. However, even without an interconnect network, the DNN accelerator 200 according to the embodiment can flexibly control hardware mapping and data flow. Thus, because the DNN accelerator 200 according to the embodiment can perform neural network computations based on hardware mapping and data flow that satisfy at least one of optimal energy efficiency and optimal performance, eliminating the interconnect network ensures both energy efficiency and performance, while also improving design efficiency and reducing design costs. Due to the reduction in hardware size caused by the removal of the interconnect network, the application areas of the DNN accelerator 200 can be further expanded.

[0060] According to an embodiment, LB 234 may include individual buffers of the same size, to which neural network data of multiple data types are respectively allocated. In other words, LB 234 may include an input data buffer allocated with input data, a weight data buffer allocated with weight data, and an output data buffer allocated with output data, and each buffer may have the same size.

[0061] According to an embodiment, LB 234 may include an input data buffer, a weight data buffer, and an output data buffer, each buffer being 32 bytes in size. However, this is an example chosen empirically based on experimental results considering optimal energy efficiency and / or performance for implementing the DNN accelerator, and the size of the individual buffers for the various data types included in LB 234 is not limited to 32 bytes. Of course, other sizes may be chosen based on at least one of the hardware design cost, energy efficiency, and performance of the DNN accelerator 200.

[0062] According to an embodiment, GLB 210 may include a unified buffer that shares regions where neural network data of multiple data types are allocated. In other words, the sum of the sizes of the regions where neural network data of multiple data types are allocated can be flexibly set within the size of GLB 210. Generally, the larger the region where neural network data reused according to a data stream is allocated, the higher the reuse efficiency of the neural network data. When GLB 210 includes individual buffers of finite size where only neural network data of a specific data type is allocated, the size of the region where neural network data reused according to a specific data stream is also limited to the size of the individual buffer, and therefore the efficiency of neural network data reuse may decrease. On the other hand, when GLB 210 consists of a unified buffer, the size of the region where neural network data reused according to a specific data stream is allocated can be flexibly set within the size of GLB 210, thereby improving the reuse efficiency of neural network data. Thus, because GLB 210 includes a unified buffer, the DNN accelerator 200 according to the embodiment can improve the reuse efficiency of neural network data, regardless of the type of data stream.

[0063] According to an embodiment, the GLB 210 may consist of a uniform buffer of size 128 KB. However, this is an example selected empirically based on experimental results to achieve optimal energy efficiency and / or performance for the DNN accelerator, and the size of the GLB 210 is not limited to 128 KB. Of course, other sizes may be selected based on at least one of the hardware design costs, energy efficiency, and performance of the DNN accelerator 200.

[0064] According to an embodiment, the DNN accelerator 200 may be connected to off-chip memory 250 and host 260 to perform DNN workloads. However, the embodiment is not limited thereto, and the DNN accelerator 200 may include at least one of off-chip memory 250, off-chip memory controller 250 that controls the off-chip memory, and host 260.

[0065] Off-chip memory 250 can store neural network data necessary for executing DNN workloads. According to an embodiment, off-chip memory 250 is one of multiple levels of memory in the memory hierarchy of the DNN accelerator 200, and can be a memory at a lower level than GLB 210. Based on update requests for neural network data to the global controller 220, the neural network data stored in off-chip memory 250 can be updated between off-chip memory 250 and GLB 210. According to an embodiment, the transmission and reception of neural network data stored in off-chip memory 250 can be controlled by an off-chip memory controller (not shown), and update requests for neural network data to the global controller 220 can be sent to the off-chip memory controller.

[0066] Host 260 may include a device driver or compiler for identifying at least one of a data stream and a hardware mapping value of a layer on which operations are performed. According to an embodiment, host 260 may send data processing commands for controlling the operation of DNN accelerator 200 to a memory controller that controls each of multiple levels of memory in the memory hierarchy of DNN accelerator 200. According to an embodiment, the data processing commands may include information about at least one of a data stream and a hardware mapping value of neural network data allocated to the corresponding level of memory. The information about the hardware mapping value may refer to information about the hardware mapping value itself, or it may refer to information about the shape of data tiles of neural network data allocated to the memory necessary to determine the hardware mapping value. According to an embodiment, the information about at least one of the data stream and hardware mapping value included in the data processing command may be determined and sent differently for each of the multiple levels of memory to which the data processing command is sent.

[0067] According to an embodiment, at least one of the data stream and hardware mapping value can be determined based on at least one of the energy and data processing cycles required for the operation of the DNN accelerator 200. According to an embodiment, the host 260 can calculate at least one of the energy and data processing cycles required for the DNN accelerator 200 to perform operation on a specific layer for multiple combinations created by various hardware mapping options and various data stream options that can be implemented in the DNN accelerator.

[0068] According to an embodiment, host 260 can identify at least one of the hardware mapping value and data stream for optimal data processing based on at least one of the calculated energy and data processing cycles. For example, host 260 can identify the combination with the lowest or smallest energy or data processing cycles required for the DNN accelerator 200 to perform operations among multiple combinations created by various hardware mapping options and various data stream options, and can send a data processing command to a memory controller that controls the operation of multiple levels of memory, the data processing command including information about the hardware mapping value and data stream of the identified combination. However, the embodiments are not necessarily limited to the above examples, and the identified combination can be one of multiple combinations in which at least one of the energy and data processing cycles required for the DNN accelerator 200 to perform operations falls within a preset level.

[0069] According to an embodiment, the global controller 220 can control the GLB 210 based on commands received from the host 260. The global controller 220 can determine at least one of the data tile shape and data stream of the neural network data allocated to the GLB 210 based on data processing commands received from the host 260. For example, in the operation of computing one layer of a multi-layered deep neural network, the data processing command from the host 260 may include information indicating that the data stream of the neural network data is a WS data stream. The data processing command from the host 260 may include information indicating that the input data tiles of the neural network data allocated to the GLB 210 are (5×5×1×1) tensors, the weight data tiles are (2×2×1×1) tensors, the output data tiles are (2×2×1×1) tensors, or a hardware mapping value containing this information. However, the embodiments are not necessarily limited to the above examples, and the above description is equally applicable to at least one of other memory controllers that receive data processing commands from the host 260, such as off-chip memory controllers, local controllers, and register controllers. Therefore, overlapping descriptions will be omitted.

[0070] Therefore, the DNN accelerator 200 according to the embodiment can flexibly change the hardware mapping and data flow of neural network data allocated to multiple levels of memory in order to perform optimized data processing. A detailed method for updating neural network data between adjacent levels of memory so that the DNN accelerator 200 can perform DNN workloads based on flexibly changed hardware mappings and flexibly changed data flows will now be described with reference to the accompanying drawings.

[0071] Figure 3 It is a view used to interpret hardware mapping values ​​according to an embodiment.

[0072] refer to Figure 3 It can examine neural network data of convolutional layers in one of the multiple memory levels allocated to a DNN accelerator.

[0073] According to an embodiment, the neural network operation to generate output data 303 by performing a convolution operation on input data 301 and weight data 302 can be performed in a convolutional layer. At this time, for each data type, there may be neural network parameter values ​​related to the type of each neural network data. The neural network parameter values ​​can refer to hardware mapping values ​​determined by hardware mapping. In other words, the hardware mapping values ​​of the neural network data allocated to a specific level of memory can include information about the shape of the data tiles allocated to the neural network data at that specific level of memory.

[0074] In detail, the neural network parameters may include multiple input data parameters related to the factors of input data 301, multiple weight data parameters related to the factors of weight data 302, and multiple output data parameters related to the factors of output data 303.

[0075] The input data parameters may include at least one of the following parameters related to the batch size B, input channel C, group size G, input height H, and input width W of the input data 301.

[0076] The weight data parameters may include at least one of the following parameters related to the weight channel C, group size G, weight count K, weight height R, and weight width S of the weight data 302.

[0077] The output data parameters may include at least one of the following parameters related to the batch size B, group size G, output count K, output height Q, and output width P of the output data 303.

[0078] According to an embodiment, at least one of the multiple parameters of the neural network data can be derived from at least one other parameter. For example, such as Figure 3 As shown, the parameters related to the height H and width W of the input data 301 can be derived from at least one other parameter. For example, the height H and width W of the input data 301 can be calculated using the equations W = (P-1) × stride + R and H = (Q-1) × stride + S. However, embodiments of the present invention are not limited thereto.

[0079] Figure 4 This is a flowchart of the operation method of the DNN accelerator according to an embodiment. Figure 3 The operation shown can be performed by Figure 1 The DNN accelerator 100 shown Figure 2 The DNN accelerator 200 shown Figure 13 The DNN accelerator 1300 shown, and Figure 13 At least one processor 1310 is shown to execute this.

[0080] In operation S410, based on the data stream of neural network data allocated to the first-level memory and the hardware mapping values, the DNN accelerator can calculate multiple offsets representing the starting components of multiple data tiles of the neural network data. According to an embodiment, the DNN accelerator can store the calculated multiple offsets in a queue data type structure. The multiple data tiles can refer to one or more components of the neural network data allocated to the first-level memory that are updated between the first-level and second-level memories in response to an update request from the second-level memory.

[0081] According to an embodiment, based on the hardware mapping value of each of the multiple data types of neural network data, the DNN accelerator can calculate multiple offsets for the multiple data types of neural network data. The DNN accelerator can then identify multiple components of the updated data tile based on the calculated offsets for the data type of the updated data tile.

[0082] According to embodiments, the first-level memory and the second-level memory can be one of multiple levels of memory in the memory hierarchy of the DNN accelerator, and the second-level memory can be a higher-level memory than the first-level memory. For example, the first-level memory and the second-level memory can be GLB and LB. However, embodiments are not necessarily limited to the above examples; the first-level memory and the second-level memory can be LB and registers, or they can be DRAM and GLB.

[0083] According to an embodiment, the data stream may include information about the data type of neural network data reused in the second-level memory, among multiple data types of neural network data. According to an embodiment, the hardware mapping value may include information about the shape of the data tiles of the neural network data allocated to the first-level and second-level memories. According to an embodiment, at least one of the data stream and the hardware mapping value may be determined based on at least one of the energy required for the operation of the DNN accelerator and the data processing cycle.

[0084] In operation S420, based on receiving an update request for neural network data from the second-level memory, the DNN accelerator identifies the data type of the update data tile corresponding to the received update request among multiple data tiles. According to an embodiment, the DNN accelerator can receive update requests for neural network data from the second-level memory, and based on the received update request, can identify the data type of the update data tile to be updated according to the received update request among multiple data tiles of neural network data allocated to the first-level memory.

[0085] According to an embodiment, the DNN accelerator can calculate the required number of updates and the cumulative number of updates for multiple data types of neural network data. The required number of updates can refer to the number of times the neural network data needs to be updated to perform a workload on the neural network data stored in the first-level memory. The cumulative number of updates to the neural network data can refer to the number of times the neural network data has been updated in response to update requests while performing a workload on the neural network data allocated to the first-level memory. According to an embodiment, the required number of updates and the cumulative number of updates can be calculated differently for data types corresponding to the data stream and for data types other than those corresponding to the data stream. According to an embodiment, the DNN accelerator can identify the data type of the updated data tile based on the required number of updates and the cumulative number of updates at the point in time when the update request is received.

[0086] In operation S430, the DNN accelerator identifies one or more components in the updated data tile based on the data type of the updated data tile and the offset of the updated data tile among multiple calculated offsets. The one or more components of the updated data tile may include at least one component of the neural network data allocated to the first-level memory.

[0087] According to an embodiment, the DNN accelerator can calculate the distance to the offset based on a hardware mapping value of an identified data type. The distance to the offset can include information about how far one or more components of the updated data tile are from the offset, based on the data tile allocated to the first-level memory. According to an embodiment, the DNN accelerator can identify components that are separated from the offset by the distance to the offset as one or more components of the updated data tile.

[0088] In operation S440, the DNN accelerator can update one or more components of neural network data, identified, between the first-level memory and the second-level memory. Multiple storage addresses exist corresponding to multiple components of data tiles allocated to the first-level memory, and the DNN accelerator can identify one or more components of the updated data tile to identify the storage address where the neural network data of that component is stored. The DNN accelerator can update the updated data tile between the first-level memory and the second-level memory by accessing the storage address where one or more components of the updated data tile are stored.

[0089] Figure 5 It is a diagram used to explain the neural network data updated between GLB and LB according to an embodiment.

[0090] refer to Figure 5Neural network data can be assigned to the GLB 510. For example, input data tile 511 can be assigned to the GLB 510 in the form of (5×5×1×1), weight data tile 512 can be assigned in the form of (2×2×1×1), and output data tile 513 can be assigned in the form of (4×4×1×1). This includes assigning data to... Figure 5 The multiple numbers in the data tiles of the GLB 510 shown can refer to the index of each of the multiple components that make up the data tile. The input data tile 511, weight data tile 512, and output data tile 513 assigned to the GLB 510 can be neural network data that is updated between DRAM (not shown) and the GLB 510, which is a lower-level memory than the GLB 510.

[0091] According to an embodiment, a global controller controlling the GLB 510 can receive update requests for neural network data from a first local controller controlling a first LB 520 included in a first processing element and a second local controller controlling a second LB 530 included in a second processing element. In response to the update request, the global controller can group some neural network data allocated to the GLB 510 into update data tiles, and can update the update data tiles among the GLB 510, the first LB 520 included in the first processing element, and the second LB 530 included in the second processing element. For example, as... Figure 5 As shown, in Seq.#0, the neural network data of the input data tile 511 (0, 1, 2, 5, 6, 7), the weight data tile 512 (0, 1), and the output data tile 513 (0, 1, 4, 5) can be updated between GLB 510 and the first LB 520. In Seq.#0, the neural network data of the input data tile 511 (5, 6, 7, 10, 11, 12), the weight data tile 512 (2, 3), and the output data tile 513 (0, 1, 4, 5) can be updated between GLB 510 and the second LB 530. The set of multiple components of the neural network data to be updated can be referred to as the updated data tile.

[0092] According to an embodiment, when the workloads for updating neural network data in the first LB 520 and the second LB 530 are both executed, the first local controller and the second local controller can send a request to the global controller 220 to update the neural network data in order to execute the next workload. For example, when all the workloads for updating neural network data between GLB 510 and the first LB 520 and the second LB 530 are executed in Seq.#0, another neural network data can be updated between GLB 510 and the first LB 520 and the second LB 530 in Seq.#1, thereby allowing the next workload to be executed.

[0093] According to an embodiment, the order in which neural network data is updated between GLB 510 and the first LB 520 and the second LB 530 can be determined based on the data flow of the neural network data. For example, suppose the data flow of the neural network data is as follows: Figure 5 The WS data stream shown only updates the input and output data until the workload for updating the weight data in the first LB 520 and the second LB 530 is fully executed, and the weight data is not updated. In other words, in response to the update request at Seq.#0, the input data, weight data, and output data are updated between GLB 510 and the first LB 520 and the second LB 530, and in response to the update request at Seq.#1, the input data and output data are updated.

[0094] According to an embodiment, the DNN accelerator can identify multiple components of the update data tile corresponding to the update request and update the neural network data of the identified multiple components. Details will now be described with reference to the following figures.

[0095] Figure 6 This is a view used to explain the detailed structure of the memory controller according to an embodiment.

[0096] refer to Figure 6 The memory controller 600 may include an offset calculator 610, a sorting manager 620, and an address calculator 630. The memory controller 600 can control the operation of at least one of the multi-level memories in the memory hierarchy of the DNN accelerator. For example, the memory controller 600 may be at least one of a register controller, a local controller, a global controller, and a DRAM controller.

[0097] Before performing operations on a specific layer of the DNN accelerator, the offset calculator 610 can calculate multiple offsets for that specific layer. According to an embodiment, the offset calculator 610 can calculate the multiple offsets based on the hardware mapping values ​​of the first-level memory and the second-level memory included in the data processing command 601 received by the memory controller 600. The offset calculator 610 can store the calculated multiple offsets in a queue 611, and can send the queue 611 containing the multiple offsets to the sorting manager 620. References below... Figure 7 This describes the detailed operation of the offset calculator 610.

[0098] The sorting manager 620 can determine the movement order of neural network data to be updated based on the update request. According to an embodiment, in response to an update request for neural network data received by the memory controller 600, the sorting manager 620 can identify the data type 621 of the update data tile corresponding to the received update request. The sorting manager 620 can obtain the offset 622 of the update data tile from a plurality of offsets stored in the queue 611 received from the offset calculator 610. The sorting manager 620 can send the identified data type 621 of the update data tile and the obtained offset 622 of the update data tile to the address calculator 630. (Refer to below...) Figure 8 This describes the detailed operation of the sorting manager 620.

[0099] Address calculator 630 can calculate the storage address where the neural network data of the updated data tile corresponding to the update request is stored. According to an embodiment, address calculator 630 can identify the storage address 632 to which one or more components 631 and one or more components in the updated data tile are assigned, based on the data type 621 and offset 622 of the updated data tile received from sorting manager 620. Memory controller 600 can update the neural network data of the updated data tile between first-level memory and second-level memory based on the identified storage address. (Refer to below...) Figure 9 Describe the detailed operation of the Address Calculator 630.

[0100] Figure 7 This is pseudocode describing the operation of the offset calculator 610 according to an embodiment. Figure 7 The pseudocode is when the data stream is assumed to be a WS data stream. Figure 6 An example of an algorithm for calculating multiple offsets using the offset calculator 610. When the data stream is an IS data stream or an OS data stream, a for loop statement based on the hardware mapping value associated with the data stream can be located outside, and a for loop statement based on the hardware image value not associated with the data stream can be located inside. Figure 7 The hardware mapping values ​​included can correspond to the reference above. Figure 3 The described hardware mapping value.

[0101] According to an embodiment, based on the hardware mapping value of each of a plurality of data types, the offset calculator 610 can calculate multiple offsets for the plurality of data types. (See reference) Figure 7 In rows 2 and 3, the offset calculator 610 can calculate multiple offsets using equation 1.

[0102] [Equation 1]

[0103]

[0104] Among them, "Channel" lower Height lower and Width lower "can refer to the channel, height, and width of the weight data tile assigned to the first-level memory, and (n, c, h, w) can refer to variables determined based on the corresponding hardware mapping values ​​of multiple data types of neural network data assigned to the first-level and second-level memories."

[0105] For example, such as Figure 5 As shown, assume that the weight data tiles are allocated as (1×1×2×2) to the first-level memory, and the weight data tiles are allocated as (1×1×1×2) to the second-level memory. In this case, the "K" value, which is the hardware mapping value of the weight data of the GLB, is... lower C lower R lower , and S lower " can be "1,1,2,2", and "K" is the hardware mapping value of the LB weight data. upper C upper R upper , and S upper "Can be "1,1,2,2". Furthermore, the "Channel" in the offset calculation equation... lower Height lower and Width lower "Can be "1,2,2", which represents the channel, height, and width of the weighted data tiles assigned to the first-level memory. (See reference) Figure 7 In lines 9, 10, 13, and 14, the offset calculator 610 can substitute (0, 0, 0, 0) and (0, 0, 1, 0) into the variables (k, c, r, s) of the offset calculation equation using a for loop, based on the hardware mapping values ​​of the first-level memory and the second-level memory. In this case, multiple offsets "0, 2" of the weighted data can be calculated, and these multiple offsets of the weighted data can indicate... Figure 5The starting component of the weight data tile that is updated between GLB 510 and the first LB 520 and the second LB 530 in response to an update request, among the multiple components of the weight data stored in GLB 510.

[0106] According to an embodiment, the offset calculator 610 can calculate the required number of updates for multiple data types of neural network data. (Reference) Figure 7 In pseudocode lines 12 and 16, the offset calculator 610 can increment an iterative value representing the required number of updates for multiple data types by the number of times the corresponding offsets for each data type are calculated. The required number of updates for a data type corresponding to the data stream can refer to the number of updates required to perform all workloads on the neural network data allocated to the first-level memory, and the required number of updates for data types other than the data type corresponding to the data stream can refer to the number of updates for each other data type required to perform one update of the neural network data of the data type corresponding to the data stream. For example, according to... Figure 5 In an embodiment, the offset calculator 610 can calculate the required number of updates for the weight data as 2, since the multiple offsets of the weight data corresponding to the data stream are calculated twice in total, and can calculate the required number of updates for the input data and the required number of updates for the output data as 4, since the offsets of the input data and the output data are calculated a total of 4 times for one weight data update.

[0107] According to an embodiment, each of a plurality of offsets of a data type corresponding to a data stream can be grouped using at least one offset of each of other data types calculated together with each offset. For example, according to Figure 5 In one embodiment, the offsets of the input data "0, 2, 10, 12" and the output data "0, 2, 8, 10" are calculated simultaneously with the offset of the weight data "0". Furthermore, the offsets of the input data "5, 7, 15, 17" and the output data "0, 2, 8, 10" are calculated simultaneously with the offset of the weight data "2". In this case, the offsets of the weight data "0", the input data "0, 2, 10, 12", and the output data "0, 2, 8, 10" can be grouped into a first group, and the offsets of the weight data "2", the input data "5, 7, 15, 17", and the output data "0, 2, 8, 10" can be grouped into a second group.

[0108] According to an embodiment, when parallel neural network data is updated, multiple updated data tiles corresponding to multiple offsets belonging to the same group can be updated for the same upper-level memory. For example, as Figure 5As shown, update data tiles corresponding to multiple offsets belonging to the first group can be updated relative to the first LB520, and update data tiles corresponding to multiple offsets belonging to the second group can be updated relative to the second LB530.

[0109] Figure 8 This is pseudocode describing the operation of the sorting manager according to an embodiment. Figure 8 The pseudocode is when the data stream is assumed to be a WS data stream. Figure 6 An example of an algorithm for a sorting manager 620 to determine the movement order of neural network data. When the data stream is an IS data stream or an OS data stream, the for loop statement based on the hardware mapping values ​​associated with the data stream can be located externally, and the positions of the condition statements and variables associated with each data stream can be changed, and indications are included. Figure 8 The required number of updates in the iter I iter W and iter O It can correspond to the instruction Figure 8 The required number of iterations for calculating multiple data types. I ,iteration W and iteration O .

[0110] According to an embodiment, the sorting manager 620 can calculate the cumulative number of updates to the neural network data. When updating the neural network data in response to an update request, the sorting manager 620 can increment the cumulative number of updates for the data type of the updated neural network data. For example, refer to... Figure 8 In pseudocode lines 8, 11, and 14, the sorting manager 620 can increment the cnt value, representing the cumulative update count of multiple data types, whenever the neural network data is updated. The cumulative update count of data types other than those corresponding to the data stream can be initialized whenever the cumulative update count of the data type corresponding to the data stream increases.

[0111] According to an embodiment, the sorting manager 620 can identify the data type of the updated data tile based on the required number of updates for multiple data types and the cumulative number of updates at the time the update request is received. The sorting manager 620 can identify the data type of the updated data tile as one of the following: a data type corresponding to the data stream, a data type other than the data type corresponding to the data stream, or one of all multiple data types, by comparing the required number of updates for multiple data types with the cumulative number of updates at the time the update request is received.

[0112] For example, assuming according to Figure 5 and Figure 7The required number of updates for the input data, weight data, and output data calculated in the above embodiments is "4, 2, 4".

[0113] When the cumulative update count of the input data, weight data, and output data at the time the update request is received is "0, 0, 0", a workload based on the new neural network data needs to be executed in the second-level memory where the update request was already sent at the time the update request was received. Therefore, as Figure 8 As shown in lines 7 and 8, the sorting manager 620 can identify the data type of the updated data tile as input data, weight data, and output data, and can increment the cumulative update count of the input data, weight data, and output data to be updated by 1.

[0114] When the cumulative update count of the input data, weight data, and output data at the time the update request is received is "2, 1, 2", the weight data updated in secondary memory at the time the update request is received is reused, so it may not be necessary to update the weight data. Therefore, as... Figure 8 As shown in lines 10 and 11, the sort manager 620 can identify the data type of the updated data tile as input data and output data, and can increment the input data to be updated and the cumulative update count of the input data by 1.

[0115] When the cumulative update count of the input data, weight data, and output data at the time the update request is received is "4, 1, 4", the workload of updating the weight data in the second-level memory at the time the update request was received has been fully executed, therefore, new weight data needs to be updated. Therefore, as... Figure 8 As shown in lines 13 and 14, the sorting manager 620 can identify the data type of the updated data tile as weighted data, increment the cumulative update count of the weighted data by 1, and initialize the cumulative update counts of the input and output data to 0. Therefore, the input and output data associated with the weighted data that is updated again in response to the next update request can be updated.

[0116] According to an embodiment, the sorting manager 620 can send an update request for neural network data to a memory controller, which controls a memory at a lower level than the first-level memory, based on the required update counts for multiple data types and the cumulative update count at the time the update request was received. The sorting manager 620 can compare the required update counts for multiple data types with the cumulative update count at the time the update request was received, and when the cumulative update counts for multiple data types at the time the update request for neural network data is received equal to the required update counts, an update request for new neural network data can be sent. For example, in the above embodiment, when the cumulative update counts for input data, weight data, and output data are "4, 2, 4", the update of the neural network data allocated to the first-level memory has been fully executed at the time the update request was received, therefore the first-level memory needs to be updated with new neural network data. Therefore, the sorting manager 620 can initialize the required update counts for multiple data types to 0, such as... Figure 8 As shown in lines 16, 17, and 18, an update request for neural network data is sent to a memory at a lower level than the first-level memory, and the process waits until the update of the neural network data corresponding to the update request is performed.

[0117] Figure 9 This is pseudocode describing the operation of the address calculator according to an embodiment. Figure 9 The pseudocode is an example of such an algorithm, where Figure 6 The address calculator 630 identifies the storage address of one or more components of the updated data tile. Figure 9 The terms "input data transmission," "output data transmission," and "weight data transmission" can refer to data that has already been transmitted. Figure 8 When transmitting data (input), transmitting data (output), and transmitting data (weight).

[0118] According to an embodiment, the address calculator 630 can calculate the distance to the offset based on the hardware mapping value of the updated data tile. The address calculator 630 can calculate the distance by substituting variables determined based on the hardware mapping value of the neural network data allocated to the second-level memory. Figure 7 The variables (n, c, h, and w) in Equation 1 are used to calculate the distance to the offset. (See reference...) Figure 9 In lines 2 and 3, the address calculator 630 can identify and calculate one or more components of the updated data tile by using equation 2.

[0119] [Equation 2]

[0120]

[0121] The distance to the offset corresponds to the distance in equation 2. According to an embodiment, the address calculator 630 can identify components that are separated from the offset by a distance to the offset as one or more components of the updated data tile. The address calculator 630 can identify the storage address of the first-level memory to which the identified one or more components have been allocated, and can update the neural network data stored at the identified storage address between the first-level memory and the second-level memory.

[0122] For example, suppose the neural network data has been allocated to the first-level memory and the second-level memory, such as Figure 5 As shown in the above embodiment. In this case, the hardware mapping value "B" of the input data of the second-level memory. upper C upper H upper and W upper "Can be "1, 1, 2, 3". (See reference) Figure 9 In lines 16, 17, and 18, when the address calculator 630 receives information from the sorting manager 620 indicating that the data type of the updated data tile is the input data type and the offset of the updated data tile is "0", the address calculator 630 can substitute (0, 0, 0, 0), (0, 0, 0, 1), (0, 0, 0, 2), (0, 0, 1, 0), (0, 0, 1, 1), and (0, 0, 1, 2) into the variables (b, c, h, w) of the distance calculation equation based on the hardware mapping values ​​of the input data allocated to the second-level memory. In this case, "0, 1, 2, 5, 6, 7" can be obtained by adding the distance to the offset "0, 1, 2, 5, 6, 7" to the offset "0", and the address calculator 630 can identify the component of the input data tile whose index value "0, 1, 2, 5, 6, 7" corresponds to the distance to the offset "0, 1, 2, 5, 6, 7" at the offset "0" of the input data tile as one or more components of the updated data tile. When the address calculator 630 receives information from the sorting manager 620 indicating that the data type of the updated data tile is the input data type and the offset of the updated data tile is "2", the address calculator 630 can identify the components corresponding to 5, 6, 7, 10, 11, and 12 of the input data tile as one or more components of the updated data tile according to the aforementioned method. The address calculator 630 can identify the storage addresses of the first-level memory to which the components corresponding to the index values ​​0, 1, 2, 5, 6, and 7 of the input data tiles have been assigned, as well as the storage addresses of the first-level memory to which the components corresponding to the index values ​​5, 6, 7, 10, 11, and 12 of the input data tiles have been assigned, and can update the neural network data stored at the identified storage addresses between the first-level memory and the second-level memory.

[0123] Figure 10This is a table used to compare the specifications of a DNN accelerator according to an embodiment with other DNN accelerators;

[0124] Figure 10 Examples are given for the specification of the DNN accelerator "Fornax" according to the embodiment, as well as the specifications of other DNN accelerators "Eyeriss", "FlexFlow" and "MAERI", each of which has an interconnection network.

[0125] According to embodiments, "Fornax" may be equipped with a GLB and a LB, the size of which maximizes energy efficiency and performance. For example, the GLB may be a uniform buffer with a size of 128KB, and the LB may include individual buffers with a size of 32 bytes each for neural network data of multiple data types. However, this is an example of an empirically chosen size considering the energy efficiency and performance of the neural network accelerator, and other sizes may be chosen based on at least one of the hardware design cost, energy efficiency, and performance of the DNN accelerator 200.

[0126] According to the embodiments, "Fornax" may not have an interconnect network. On the other hand, other DNN accelerators such as "Eyeriss," "FlexFlow," and "MAERI" may have interconnect networks. In other words, because "Fornax" has no interconnect network, its hardware complexity is lower compared to "Eyeriss," "FlexFlow," and "MAERI." Therefore, "Fornax" can achieve optimal energy efficiency and performance, while also improving design efficiency and reducing design costs.

[0127] According to an embodiment, in "Fornax", Figure 10 The GLB, LB, and other memory levels not included in the above can support a variety of data streams. In other words, because Fornax supports a variety of data streams, it can perform computations by selecting and applying data streams that ensure optimal energy efficiency and / or optimal performance. On the other hand, in Eyeriss, FlexFlow, and MAERI, specific memory levels may support only a single data stream, or may support only two data types out of multiple data types, even if they support multiple streams.

[0128] Figure 11A and 11B This is a graph showing a comparison between the respective performance of the DNN accelerator according to the embodiment and other DNN accelerators. Figure 11A and 11BMeasurements based on experimental results are shown, which identify the optimal execution method for minimizing energy or cycles for all DNN layers using a time-cycle analysis approach. The DNN accelerator "Fornax" according to the embodiment was designed and synthesized using a SAED 32nm technology prototyping compiler to evaluate hardware complexity. "Fornax" is implemented to operate at a clock frequency of 200MHz and an operating voltage of 0.95V.

[0129] refer to Figure 11A The DNN accelerator "Fornax" according to the embodiment is compared with other DNN accelerators "Eyeriss", "FlexFlow" and "MAERI" in terms of the energy required to operate on various DNN layers.

[0130] According to the embodiments, when the data stream and hardware mapping that achieves optimal energy efficiency are applied to each of the multiple DNN accelerators, Fornax can reduce energy consumption by an average of 11.9%, 10.8%, and 8.2% compared to Eyeriss, FlexFlow, and MAERI, respectively. Figure 11A It can be shown that Fornax offers superior energy efficiency compared to other DNN accelerators for all DNN layers. Because Eyeriss, FlexFlow, and MAERI do not support the flexibility of full data flow across all levels of memory, despite the existence of interconnect networks between different PEs (hereinafter referred to as interconnect networks), FlexFlow and MAERI may offer reduced energy efficiency for various DNN layer operations compared to Fornax, which supports the flexibility of full data flow.

[0131] refer to Figure 11B The DNN accelerator "Fornax" according to the embodiment was compared with other DNN accelerators "Eyeriss", "FlexFlow" and "MAERI" in terms of the data processing cycles required for operation of various DNN layers.

[0132] According to embodiments, when optimal energy-efficient data flow and hardware mapping are applied to each of multiple DNN accelerators, Fornax achieves an average data processing performance of 1.11x, 1.06x, and 1.09x higher than Eyeriss, FlexFlow, and MAERI, respectively. For certain DNN layers, MAERI achieves higher data processing performance than Fornax. However, this is because MAERI is able to apply data flow and hardware mapping, thereby ensuring optimal performance for the corresponding DNN layer through the interconnect network. On the other hand, even without an interconnect network, this application can achieve similar data processing performance to MAERI through the flexibility of supporting the complete data flow, even for the corresponding DNN layer.

[0133] Figure 12A and 12B This is a graph showing a comparison of performance and design area with or without the interconnect network of the DNN accelerator according to an embodiment.

[0134] exist Figure 12A In this study, the energy required to operate the DNN accelerator "Fornax" without interconnection networks according to the embodiment is compared with the energy required to operate the DNN accelerator "Fornax-N" including interconnection networks. Figure 12A The diagram illustrates a minimum energy scheduling solution for executing YOLO layers based on a brute-force time-loop analysis method. The gray lines in the diagram represent clustered data points, and each data point indicates the energy required for the DNN accelerator to compute possible execution methods. OS dataflow diagram 1210, IS dataflow diagram 1220, and WS dataflow diagram 1230 show the energy required for each execution method computed by the DNN accelerator when "Fornax" and "Fornax-N" select the same dataflow in their local buffers. Diagram 1240, showing different dataflow selections, illustrates the energy required for each execution method computed by the DNN accelerator when "Fornax" and "Fornax-N" select different dataflows to achieve optimal energy efficiency.

[0135] refer to Figure 12AThis can be confirmed by OS dataflow diagram 1210: "Fornax" and "Fornax-N" utilize OS dataflows for energy optimization, and in this case, each DNN accelerator generates the exact same energy cost difference. It can also be confirmed by IS dataflow diagram 1220 and WS dataflow diagram 1230: compared to "Fornax-N", "Fornax" generates 0% (none) and 1.8% energy cost differences. It can also be confirmed by different dataflow selection diagram 1240: "Fornax" and "Fornax-N" select different dataflows for minimum energy execution; however, when comparing the execution methods that generate the actual minimum energy cost for the two DNN accelerators, "Fornax" generates a 1.2% energy cost difference compared to "Fornax-N".

[0136] refer to Figure 12B The figure shows the design area of ​​the DNN accelerator "Fornax" excluding the interconnect network and the DNN accelerator "Fornax-N" including the interconnect network, as well as the design area of ​​the detailed components included in the two DNN accelerators.

[0137] refer to Figure 12B Compared to Fornax, Fornax-N requires more design area based on the PE inter-network configuration and additional components for implementing the PE inter-network. Therefore, it can be confirmed that the total design area of ​​Fornax-N is increased by 8.3% compared to Fornax.

[0138] Therefore, inter-PE networks can increase the complexity and cost of hardware design. However, the DNN accelerator according to the embodiment supports flexible hardware mapping and flexible data flow, thereby achieving optimal energy efficiency and performance without inter-PE networks.

[0139] Figure 13 This is a block diagram showing the detailed configuration of a DNN accelerator according to an embodiment. (See reference) Figure 13 The DNN accelerator 1300 according to an embodiment may include at least one processor 1310 and a memory 1320. However, the components of the DNN accelerator 1300 are not limited to those described above. Figure 13 The components shown, and the DNN accelerator 130 can be used with more Figure 13 The components shown are implemented using more components.

[0140] At least one processor 1310 controls all operations of the DNN accelerator 1300. According to an embodiment, at least one processor 1310 may include a memory controller that controls the operation of the memory 1300, and the memory controller may correspond to the above reference. Figures 1 to 12 describes the memory controller. According to an embodiment, at least one processor 1310 may include a host that sends data processing commands to the memory controller, and this host may correspond to the one described above. Figures 1 to 1 2. The host described.

[0141] The memory 1320 may include a first-level memory 1321 and a second-level memory 1322. However, the components of the memory 1320 are not limited to those specified above. Figure 13 The components shown, and the memory 1320 can be used with more Figure 13 The components shown are implemented with more components. For example, memory 1320 may include memory at a higher level than first-level memory 1321 or at a lower level than second-level memory 1322, or may include a memory controller that controls the operation of each level of memory. According to an embodiment, memory 1320 may store programs or instructions for processing and control by at least one processor 1310. According to an embodiment, memory 1320 may store neural network data required for deep neural network computation of DNN accelerator 1300.

[0142] According to an embodiment, based on the data stream of neural network data allocated to the first-level memory and hardware mapping values, at least one processor can calculate multiple offsets representing the start components of multiple data tiles of the neural network data. According to an embodiment, in response to an update request for the neural network data, at least one processor can identify the data type of the updated data tile corresponding to the received update request among the multiple data tiles. According to an embodiment, the at least one processor can identify the storage address of the first-level memory to which the updated data tile is allocated based on the data type of the updated data tile and the offset of the updated data tile among the multiple offsets. According to an embodiment, at least one processor can update the neural network data stored at the identified storage address between the first-level memory and the second-level memory. According to an embodiment, the data stream may include information about the data type of the neural network data reused by the second-level memory among the multiple data types of the neural network data. According to an embodiment, the hardware mapping values ​​may include information about the shape of the data tiles of the neural network data allocated to the first-level memory and the second-level memory.

[0143] According to an embodiment, based on the hardware mapping value of each of a plurality of data types, at least one processor can calculate multiple offsets for multiple data categories.

[0144] According to an embodiment, the at least one processor can calculate the required number of updates and the cumulative number of updates for the plurality of data types, and can identify the data type of the updated data tile based on the required number of updates and the cumulative number of updates at the time the update request was received.

[0145] According to an embodiment, at least one processor can calculate the distance to the offset based on a hardware mapping value of the identified data type, and can identify the component of the distance to the offset that is separated from the offset as at least one component of the updated data tile.

[0146] According to an embodiment, at least one of the data stream and hardware mapping value can be determined based on at least one of the energy required to operate the DNN accelerator and the data processing cycle.

[0147] According to an embodiment, the second-level memory may be a memory at a higher level than the first-level memory in the memory hierarchy of the DNN accelerator.

[0148] According to an embodiment, the first-level memory may include a GLB consisting of a unified buffer that shares the region where neural network data of multiple data types are allocated.

[0149] According to an embodiment, the second-level memory may include separate buffers of the same size for multiple data types.

[0150] According to an embodiment, in a DNN accelerator, the interconnect network used for sending and receiving neural network data may not exist between components that each include different secondary memories.

[0151] Therefore, since the DNN accelerator 1300 according to the embodiment can perform neural network computations based on hardware mappings and data flows that satisfy at least one of optimal energy efficiency and optimal performance, while eliminating interconnect networks, energy efficiency and performance can be ensured, design efficiency can be improved, and design costs can be reduced. Because the hardware size is reduced by eliminating interconnect networks, the application areas of the DNN accelerator 1300 can be further expanded.

[0152] According to an embodiment, energy efficiency and performance can be maximized by providing a flexible data stream to the controller that performs the workload.

[0153] According to an embodiment, the necessity of heavy NoC can be effectively eliminated by providing a flexible data flow for the controller that performs the workload.

[0154] According to an embodiment, by providing a flexible data flow for the accelerator to the controller performing the workload, hardware design costs and complexity can be reduced.

[0155] The foregoing description provided in this disclosure is intended to enable any person skilled in the art to perform or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to various modifications without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples described herein, but rather to provide the widest scope consistent with the principles and novel features disclosed herein.

[0156] While exemplary implementations may refer to the use of aspects of the currently disclosed subject matter in the context of one or more independent computer systems, the subject matter is not limited thereto, but can be implemented in conjunction with any computing environment, such as a networked or distributed computing environment. Furthermore, aspects of the subject matter of this disclosure can be implemented in or among multiple processing chips or devices, and storage can similarly be affected among multiple devices. These devices may include PCs, network servers, and handheld devices.

[0157] Embodiments of this disclosure can also be embodied as storage media including computer-executable instructions, such as computer-executable program modules. Computer-readable media can be any available medium accessible by a computer and includes all volatile / non-volatile and removable / non-removable media. Furthermore, computer-readable media can include all computer storage and communication media. Computer storage media includes all volatile / non-volatile and removable / non-removable media embodied by specific methods or techniques for storing information such as computer-readable instruction code, data structures, program modules, or other data. Communication media typically include computer-readable instructions, data structures, or other data in modulated data signals, such as program modules.

[0158] Furthermore, computer-readable storage media can be provided in the form of non-transitory storage media. "Non-transitory storage media" is a tangible device, meaning only that it does not contain signals (such as electromagnetic waves). This term does not distinguish between cases where data is stored semi-permanently in the storage medium and cases where data is temporarily stored. For example, a non-transitory recording medium may include a buffer in which data is temporarily stored.

[0159] According to embodiments of this disclosure, the methods according to various disclosed embodiments can be provided by including them in a computer program product. The computer program product, as a commodity, can be traded between a seller and a buyer. The computer program product is distributed in the form of a device-readable storage medium (e.g., an optical disc read-only memory (CD-ROM)), or it can be distributed online (e.g., downloaded or uploaded) directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable application) can be stored at least temporarily in a device-readable storage medium, such as the memory of a manufacturer's server, an app store's server, or a relay server, or it can be temporarily generated.

[0160] Although one or more embodiments have been described with reference to the accompanying drawings, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of this disclosure as defined by the following claims.

Claims

1. A control method for optimizing a deep neural network (DNN) accelerator for data processing, the control method comprising: Based on the data stream and hardware mapping values ​​of the neural network data allocated to the first-level memory, multiple offsets representing the start components of multiple data tiles in the neural network data are calculated. Based on the update request for the neural network data received from the second-level memory, the data type of the updated data tile corresponding to the received update request among the plurality of data tiles is identified; One or more components of the updated data tile are identified based on the data type of the updated data tile and the offset of the updated data tile among multiple calculated offsets; and Update the neural network data of one or more identified components between the first-level memory and the second-level memory. The data stream includes information about the data type of the neural network data that is reused by the second-level memory, among a plurality of data types of the neural network data. The hardware mapping values ​​include information about the shape of the data tiles of the neural network data allocated to the first-level memory and the second-level memory.

2. The control method according to claim 1, wherein, The calculation of multiple offsets includes: calculating multiple offsets for the multiple data types based on the hardware mapping value of each of the multiple data types.

3. The control method according to claim 1 further includes calculating the required number of updates and the cumulative number of updates for the plurality of data types. The data types of the identified update data tiles include: The data type of the updated data tile is identified based on the required number of updates and the cumulative number of updates at the time the update request is received.

4. The control method according to claim 1, wherein, One or more components of the identifier update data tile include: The distance to the offset is calculated based on the hardware mapping value of the identified data type, including information about how far the offset is from the data tile allocated to the first-level memory and one or more components of the updated data tile; and The component that is separated from the offset by the distance to the offset is identified as at least one component of the updated data tile.

5. The control method according to claim 1, wherein, At least one of the data stream and the hardware mapping value is determined based on at least one of the energy required for the operation of the DNN accelerator and the data processing cycle.

6. The control method according to claim 1, wherein, The second-level memory is a memory at a higher level than the first-level memory in the memory hierarchy of the DNN accelerator.

7. The control method according to claim 6, wherein, The first-level memory includes a unified buffer that shares the region where neural network data of the multiple data types are allocated.

8. The control method according to claim 6, wherein, The second-level memory includes individual buffers of the same size to which the various data types of the neural network data are respectively allocated.

9. The control method according to claim 1, wherein, In the DNN accelerator, the interconnect network for sending and receiving neural network data does not exist between components that each include a different secondary memory.

10. A deep neural network (DNN) accelerator for optimizing data processing, the DNN accelerator comprising: First-level memory; Second-level memory; and At least one processor is configured to control the operation of the first-level memory and the operation of the second-level memory. The at least one processor is further configured to: Based on the data stream and hardware mapping values ​​of the neural network data allocated to the first-level memory, multiple offsets representing the start components of multiple data tiles in the neural network data are calculated; Based on the update request for the neural network data received from the second-level memory, the data type of the updated data tile corresponding to the received update request among the plurality of data tiles is identified; One or more components of the updated data tile are identified based on the data type of the updated data tile and the offset of the updated data tile among multiple calculated offsets; and Update the neural network data of one or more identified components between the first-level memory and the second-level memory. The data stream includes information about the data type of the neural network data that is reused by the second-level memory, among a plurality of data types of the neural network data. The hardware mapping values ​​include information about the shape of the data tiles of the neural network data allocated to the first-level memory and the second-level memory.

11. The DNN accelerator according to claim 10, wherein, The at least one processor is further configured to calculate multiple offsets of the multiple data types based on the hardware mapping value of each of the multiple data types.

12. The DNN accelerator according to claim 10, wherein, The at least one processor is further configured to calculate the required number of updates and the cumulative number of updates for the plurality of data types, and to identify the data type of the updated data tile based on the required number of updates and the cumulative number of updates at the point in time when the update request is received.

13. The DNN accelerator according to claim 10, wherein, The at least one processor is also configured to calculate the distance to the offset based on a hardware mapping value of the identified data type, and to identify the component of the distance to the offset that is separated from the offset as at least one component of the updated data tile.

14. The DNN accelerator according to claim 10, wherein, At least one of the data stream and the hardware mapping value is determined based on at least one of the energy required for the operation of the DNN accelerator and the data processing cycle.

15. The DNN accelerator according to claim 10, wherein, The second-level memory is a memory at a higher level than the first-level memory in the memory hierarchy of the DNN accelerator.

16. The DNN accelerator according to claim 15, wherein, The first-level memory includes a global buffer, which consists of a unified buffer that shares the regions where neural network data of the multiple data types are allocated.

17. The DNN accelerator according to claim 15, wherein, The second-level memory includes individual buffers of the same size to which the various data types of the neural network data are respectively allocated.

18. The DNN accelerator according to claim 10, wherein, In the DNN accelerator, the interconnect network for sending and receiving neural network data does not exist between components that each include a different secondary memory.

19. A non-transient computer-readable recording medium having a program recorded thereon, which, when executed by a computer, performs a control method for a deep neural network (DNN) accelerator, the control method comprising: Based on the data stream and hardware mapping values ​​of the neural network data allocated to the first-level memory, multiple offsets representing the start components of multiple data tiles in the neural network data are calculated. Based on the update request for the neural network data received from the second-level memory, the data type of the updated data tile corresponding to the received update request among the plurality of data tiles is identified; One or more components of the updated data tile are identified based on the data type of the updated data tile and the offset of the updated data tile among multiple calculated offsets; and Update the neural network data of one or more identified components between the first-level memory and the second-level memory. in The data stream includes information about the data type of the neural network data that is reused by the second-level memory, among multiple data types of the neural network data. The hardware mapping values ​​include information about the shape of the data tiles of the neural network data allocated to the first-level memory and the second-level memory.

Images