Machine learning accelerator, computing device including machine learning accelerator, and method for loading data to machine learning accelerator

Abstract

To provide a machine learning accelerator that loads compressed weight data for a shorter period of time.SOLUTION: In a computing device 10, an accelerator includes: a first data controller (CT) that stores original length information and decompresses first data having a first length; a first accelerator core (C) that uses second data having an original length as first weight data to execute a first machine learning base operation; a second data controller (CT) that stores original length information and decompresses third data having a second length shorter than the first length; and a second accelerator core that uses fourth data having the original length as second weight data to execute a second machine learning base operation. Each of the first and second data controllers monitors timing when the decompression is completed on the basis of the original length and ends the decompression at timing when the decompression is completed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to electronic devices, and more particularly to a machine learning accelerator with improved data loading speed, a computing device including a machine learning accelerator, and a method for loading data into a machine learning accelerator. [Background technology]

[0002] The machine learning accelerator is configured to load weight data to perform machine learning-based operations. As machine learning and related technologies develop, the amount of data loaded into the machine learning accelerator increases. Therefore, the time required to load weight data into the machine learning accelerator also increases.

[0003] To reduce the time it takes to load the weight data into the machine learning accelerator, the weight data may be compressed, and the compressed weight data may be transmitted to the machine learning accelerator in a shorter time. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent No. 10,546,393 [Patent Document 2] U.S. Patent No. 11,669,736 [Patent Document 3] US Patent Application Publication No. 2023 / 0059970 [Patent Document 4] US Patent Application Publication No. 2022 / 0398441 [Patent Document 5] Chinese Patent Publication No. 111047020 [Patent Document 6] US Patent Application Publication No. 2018 / 0082181 [Patent Document 7] U.S. Patent No. 11,632,129 [Patent Document 8] US Patent Application Publication No. 2021 / 0287074 Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention has been made in view of the above-described conventional techniques, and an object of the present invention is to provide a machine learning accelerator that can load compressed weight data in a shorter time, a computing device including a machine learning accelerator, and a method for loading data into a machine learning accelerator. [Means for solving the problem]

[0006] According to at least one embodiment of the present invention, a machine learning accelerator includes: a first data controller configured to store original length information indicating the original length of the data, receive first data having a first length, decompress the first data having the first length to generate second data having the original length, and output the second data having the original length; a second data controller configured to store the original length information, receive third data having a second length shorter than the first length, decompress the third data having the second length to generate fourth data having the original length, and output the fourth data having the original length; a first accelerator core configured to receive the second data having the original length and perform a first machine-learning-based operation using the second data as first weight data; and a second accelerator core configured to receive fourth data having the original length and perform a second machine-learning-based operation using the fourth data as second weight data, wherein each of the first data controller and the second data controller is configured to monitor when decompression is complete based on the original length and to terminate decompression when decompression is complete. Machine Learning Accelerator

[0007] In accordance with at least one embodiment of the present invention, a computing device includes: a memory that stores first data having a first length and second data having a second length shorter than the first length; and a machine learning accelerator configured to receive from the memory the first data and third data having the first length and including the second data, wherein the machine learning accelerator is configured to decompress the first data having the first length to generate first weight data, convert the third data having the first length to second data having a second length, decompress the second data having the second length to generate second weight data, and perform machine learning-based operations based on the first weight data and the second weight data.

[0008] According to at least an embodiment of the present invention, there is provided a method for a processor to load data into a machine learning accelerator, the method including: the processor simultaneously programming two or more Direct Memory Access (DMA) masters with a first starting address and first length information; the two or more DMA masters reading data in parallel from a memory based on the first starting address and the first length information; and transmitting the parallel read data to the machine learning accelerator, wherein the machine learning accelerator is configured to: decompress first data corresponding to the first length information among the parallel read data to generate first weight data; convert second data corresponding to the first length information to generate third data having a second length that is shorter than the first length indicated by the first length information; decompress the third data having the second length to generate second weight data; and perform a machine learning-based operation based on the first weight data and the second weight data. [Effects of the Invention]

[0009] According to embodiments of the present invention, direct memory access (DMA) masters that load compressed weight data into a machine learning accelerator can be simultaneously programmed with the same starting address and the same length information. Thus, a machine learning accelerator, a computing device including a machine learning accelerator, and a method for loading data into a machine learning accelerator are provided that reduce the time it takes to load compressed weight data. [Brief explanation of the drawings]

[0010] [Figure 1] 1 illustrates a computing device in accordance with at least one embodiment of the present invention. [Figure 2] 1 illustrates an example of a compressed embodiment of weight data being loaded into an accelerator. [Figure 3] 1 illustrates an example of how a computing device loads compressed weight data into an accelerator. [Figure 4] 1 illustrates an example of how a computing device loads compressed weight data from a storage device into memory. [Figure 5] An example of the process of loading weight data compressed by the method of FIG. 4 from a storage device into memory is shown. [Figure 6] 1 illustrates an example of how a computing device loads compressed weight data into an accelerator. [Figure 7] 1 illustrates a data controller and associated components in accordance with at least one embodiment of the present invention. [Figure 8] An example of how the drain circuit operates is shown. [Figure 9] 1 illustrates a data controller and associated components in accordance with at least one embodiment of the present invention. [Figure 10] 1 illustrates a data controller and associated components in accordance with at least one embodiment of the present invention. [Figure 11] 11 illustrates an example of how the validity monitors of each of the data controllers of FIG. 10 operate. [Figure 12] 10 shows another example of a compressed form of weight data loaded into an accelerator. [Figure 13] An example of the process of loading weight data compressed by the method of FIG. 4 from a storage device into memory is shown. [Figure 14] 1 illustrates a computing device according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] In the following, embodiments of the present invention will be described clearly and in detail so as to enable those skilled in the art to easily practice the present invention.

[0012] 1 illustrates a computing device 10 in accordance with at least one embodiment of the present invention. Referring to FIG. 1, computing device 10 may include a system bus 110, a processor 120, an accelerator 130, a modem 140, a memory controller 150, memory 160, a DMAC 170 (Direct Memory Access Controller), and a storage device 180.

[0013] System bus 110 may be configured to enable communication between components of computing device 10, for example, by providing channels between the components of computing device 10. For example, system bus 110 may provide channels based on one or more of a variety of communication protocols, such as Peripheral Component Interconnect express (PCIe), Non-Volatile Memory express (NVMe), Dual In-line Memory Module (DIMM), Advanced eXtensible Interface (AXI), and the like.

[0014] Processor 120 may be configured to execute an operating system and various applications. Processor 120 may be configured to control components of computing device 120 as required by the operating system and various applications. For example, processor 120 may include a central processing unit (CPU) and / or an application processor (AP) that includes one or more processing cores.

[0015] Accelerator 130 may be a machine learning accelerator that performs machine-learning-based learning or executes a trained machine learning module. For example, the machine learning may be based on a variety of algorithms, including convolutional neural networks (CNNs), deep neural networks (DNNs), and generative adversarial networks (GANs). In at least some embodiments, for example, the trained machine learning module may be configured to perform speech recognition, speech synthesis, image recognition, image classification, and / or image processing using a neural network, a tablet device, a smart TV, an augmented reality (AR) device, an Internet of things (IoT) device, an autonomous vehicle, a robot, a medical device, a drone, an advanced driver assistance system (ADAS), an image display device, a data processing server, a measurement device, etc., and / or may be incorporated into one of various types of electronic devices.

[0016] The accelerator 130 may include multiple accelerator cores C. As at least one example, the accelerator 130 may include four accelerator cores C. The accelerator cores C may be configured to operate in parallel to perform machine learning operations. The machine learning operations may be at least a portion of a variety of operations including training, inference, and / or classification of a machine learning module.

[0017] The accelerator 130 may include data controllers CT each corresponding to an accelerator core C. For example, the accelerator 130 may include four data controllers CT. The accelerator controllers CT may be configured to operate in parallel. The data controllers CT may decompress compressed data transmitted to the accelerator 130 and transmit the decompressed data to the corresponding accelerator cores C. The data controllers CT may also transmit uncompressed data transmitted to the accelerator 130 to the accelerator cores C.

[0018] Modem 140 may be configured to communicate with an external device. For example, modem 140 may communicate with an external device based on various wired and / or wireless communication protocols such as IEEE 802.11, Ethernet, 5G, Bluetooth, etc. Modem 140 may transmit data stored in memory 160 to an external device or store data transmitted from an external device in modem 140.

[0019] Memory controller 150 may be configured to access memory 160. For example, memory controller 150 may access memory 160 in response to a request from processor 120 and / or a direct memory access controller (DMAC) 170. For example, memory controller 150 may write data to memory 160 or read data from memory 160. Memory controller 150 may perform various operations for managing data stored in memory 160, such as various background operations including a refresh operation and a row hammer prevention operation.

[0020] The memory 160 may be a random access memory. For example, the memory 160 may be implemented with one or more of various random access memories, such as static random access memory (DRAM), static RAM (SRAM), phase-change RAM (PRAM), magnetic RAM (MRAM), ferroelectric RAM (FRAM), and resistive RAM (RRAM).

[0021] DMAC 170 may be configured, in accordance with a program executed by processor 120, to request memory controller 150 to write data output from at least one of the components of computing device 10 to memory 160. For example, DMAC 170 may request memory controller 150 to output data stored in memory 160 in accordance with a program executed by processor 120 to at least one of the components of computing device 10. DMAC 170 may include multiple DMA masters M. As at least one example, DMAC 170 may include four DMA masters M. Each DMA master M may independently request memory controller 150 to access, e.g., write or read, memory 160.

[0022] As at least one example, DMA master M may request memory controller 150 to write or read memory 160 in parallel, independently, or simultaneously. In response to the request of DMA master M, memory controller 150 may perform write or read operations on memory 160 in parallel, independently, or simultaneously. For example, multiple parallel, independent, or simultaneous channels may be provided between DMA master M and memory controller 150. Note that multiple parallel, independent, or simultaneous channels may be provided between memory controller 150 and memory 160.

[0023] Storage device 180 may be a secondary storage device of computing device 10. For example, storage device 180 may be a hard disk drive (HDD), a solid state drive (SSD), an embedded storage device, or the like.

[0024] 2 illustrates an example of a compressed form of weight data (e.g., compressed weight data) loaded into accelerator 130. Referring to FIGS. 1 and 2, as at least one example, accelerator 130 may execute a machine learning module including three layers. First compressed layer weight data L1_WD through third compressed layer weight data L3_WD (corresponding to the three layers) are shown, but examples are not limited thereto. For example, in at least one embodiment, the number of layers may be more or less than three, and the number of compressed weight data may correspond to the number of layers included in the machine learning module.

[0025] The first compressed layer weight data L1_WD may be compressed weight data used in the first layer of the machine learning module, and may include 1a compressed weight data WD1a, 1b compressed weight data WD1b, 1c compressed weight data WD1c, and 1d compressed weight data WD1d corresponding to the four accelerator cores C, respectively.

[0026] The second compressed layer weight data L2_WD may be compressed weight data used in the second layer of the machine learning module. The second compressed layer weight data L2_WD may include 2a compressed weight data WD2a, 2b compressed weight data WD2b, 2c compressed weight data WD2c, and 2d compressed weight data WD2d corresponding to the four accelerator cores C, respectively.

[0027] The third compressed layer weight data L3_WD may be compressed weight data used in the third layer of the machine learning module. The third compressed layer weight data L3_WD may include 3a compressed weight data WD3a, 3b compressed weight data WD3b, 3c compressed weight data WD3c, and 3d compressed weight data WD3d corresponding to the four accelerator cores C, respectively.

[0028] As at least one example, the compressibility of the weight data may vary depending on the pattern of the weight data. The smaller the difference between the ratio of "0" and "1" in the weight data, the lower the compressibility of the weight data. The larger the difference between the ratio of "0" and "1" in the weight data, the higher the compressibility of the weight data. The more concentrated the "0"s or "1"s are in the weight data, the higher the compressibility of the weight data. The more dispersed the "0"s or "1"s are in the weight data, the lower the compressibility of the weight data.

[0029] When the compression ratios of the weight data are different from each other, the sizes of the compressed weight data may be different. An example in which the sizes of the compressed weight data are different is shown in FIG.

[0030] The first compressed layer weight data L1_WD, the second compressed layer weight data L2_WD, and the third compressed layer weight data L3_WD may be stored in storage device 180. For example, the first compressed layer weight data L1_WD, the second compressed layer weight data L2_WD, and the third compressed layer weight data L3_WD may be obtained (e.g., from an external device via modem 140) and stored in memory 160 and / or stored in storage device 180. As another example, the first compressed weight data L1_WD, the second compressed layer weight data L2_WD, and the third compressed layer weight data L3_WD are stored in a removable storage device and coupled to computing device 10 as part of storage device 180.

[0031] The first compressed layer weight data L1_WD may further include 1a length information LI1a, 1b length information LI1b, 1c length information LI1c, and 1d length information LI1d indicating the lengths of the 1a compressed weight data WD1a, 1b compressed weight data WD1b, 1c compressed weight data WD1c, and 1d compressed weight data WD1d, respectively. The 1a length information LI1a, 1b length information LI1b, 1c length information LI1c, and 1d length information LI1d may include information on the starting addresses on the storage device 180 where the 1a compressed weight data WD1a, 1b compressed weight data WD1b, 1c compressed weight data WD1c, and 1d compressed weight data WD1d are stored, as well as the length (or size) of the addresses of the 1a compressed weight data WD1a, 1b compressed weight data WD1b, 1c compressed weight data WD1c, and 1d compressed weight data WD1d.

[0032] The second compressed layer weight data L2_WD may further include 2a length information LI2a, 2b length information LI2b, 2c length information LI2c, and 2d length information LI2d indicating the lengths of the 2a compressed weight data WD2a, 2b compressed weight data WD2b, 2c compressed weight data WD2c, and 2d compressed weight data WD2d, respectively. The 2a length information LI2a, 2b length information LI2b, 2c length information LI2c, and 2d length information LI2d may include information on the starting addresses on storage device 180 at which the 2a compressed weight data WD2a, 2b compressed weight data WD2b, 2c compressed weight data WD2c, and 2d compressed weight data WD2d are stored, as well as the length (or size) of the addresses of the 2a compressed weight data WD2a, 2b compressed weight data WD2b, 2c compressed weight data WD2c, and 2d compressed weight data WD2d.

[0033] The third compressed layer weight data L3_WD may further include 3a length information LI3a, 3b length information LI3b, 3c length information LI3c, and 3d length information LI3d indicating the lengths of the 3a compressed weight data WD3a, the 3b compressed weight data WD3b, the 3c compressed weight data WD3c, and the 3d compressed weight data WD3d, respectively. The 3a length information LI3a, the 3b length information LI3b, the 3c length information LI3c, and the 3d length information LI3d may include information on the starting addresses on the storage device 180 where the 3a compressed weight data WD3a, the 3b compressed weight data WD3b, the 3c compressed weight data WD3c, and the 3d compressed weight data WD3d are stored, as well as the length (or size) of the addresses of the 3a compressed weight data WD3a, the 3b compressed weight data WD3b, the 3c compressed weight data WD3c, and the 3d compressed weight data WD3d.

[0034] 3 shows an example of how computing device 10 loads compressed weight data to accelerator 130. Referring to FIGS. 1, 2, and 3, in step S110, computing device 10 may load compressed weight data WD from storage device 180 to memory 160 based on the first type length information. The weight data WD may include 1a-compressed weight data WD1a, 1b-compressed weight data WD1b, 1c-compressed weight data WD1c, and 1d-compressed weight data WD1d of the first compressed layer weight data L1_WD, 2a-compressed weight data WD2a, 2b-compressed weight data WD2b, 2c-compressed weight data WD2c, and 2d-compressed weight data WD2d of the second compressed layer weight data L2_WD, and 3a-compressed weight data WD3a, 3b-compressed weight data WD3b, 3c-compressed weight data WD3c, and 3d-compressed weight data WD3d of the third compressed layer weight data L3_WD.

[0035] The first type length information may include 1a length information LI1a, 1b length information LI1b, 1c length information LI1c, and 1d length information LI1d of the first compressed layer weight data L1_WD, 2a length information LI2a, 2b length information LI2b, 2c length information LI2c, and 2d length information LI2d of the second compressed layer weight data L2_WD, and 3a length information LI3a, 3b length information LI3b, 3c length information LI3c, and 3d length information LI3d of the third compressed layer weight data L3_WD.

[0036] The processor 120 can read the compressed weight data WD from the storage device 180 and load it into the memory 160 and / or can program the DMAC 170 to read the compressed weight data WD from the storage device 180 and load it into the memory 160.

[0037] In step S120, the computing device 10 loads the compressed weight data WD from the memory 160 to the accelerator 130 based on the second type length information. For example, the second type length information may be part of the first type length information. By loading the compressed weight data WD to the accelerator 130 using the second type length information, the computing device 10 improves the speed at which the compressed weight data WD is loaded to the accelerator 130 and reduces the loading time.

[0038] As at least one example, processor 120 may program DMAC 170 to read compressed weight data WD stored in memory 160 and load it into accelerator 130. Accelerator 130 may receive the compressed weight data WD and decompress the compressed weight data WD to generate weight data. Accelerator 130 may load the weight data into accelerator core C.

[0039] In step S130, the computing device 10 executes the accelerator 130. For example, the processor 120 may control the accelerator 130 so that the accelerator 130 performs a machine learning operation (e.g., learning, inference, classification, etc.). The accelerator core C of the accelerator 130 may perform the machine learning operation based on weight data loaded into the accelerator core C and data transmitted from the processor 120.

[0040] 4 illustrates an example of a method (e.g., step S110 of FIG. 3) by which computing device 10 loads compressed weight data WD from storage device 180 to memory 160. FIG. 5 illustrates an example of a process for loading compressed weight data WD from storage device 180 to memory 160 according to the method of FIG. 4.

[0041] 1, 4, and 5, in step S210, the computing device 10 may read (read) the length information LI. For example, the processor 120 may read (read) the 1a length information LI1a, 1b length information LI1b, 1c length information LI1c, 1d length information LI1d, 2a length information LI2a, 2b length information LI2b, 2c length information LI2c, 2d length information LI2d, 3a length information LI3a, 3b length information LI3b, 3c length information LI3c, 3d length information LI3d, and original length information OLI from the first length information area LIA1 of the storage device 180. The original length information OLI may indicate the length (or size) of each weight data before compression.

[0042] In step S220, the computing device 10 may read the compressed weight data WD. Each of the 1a length information LI1a, 1b length information LI1b, 1c length information LI1c, 1d length information LI1d, 2a length information LI2a, 2b length information LI2b, 2c length information LI2c, 2d length information LI2d, 3a length information LI3a, 3b length information LI3b, 3c length information LI3c, and 3d length information LI3d may include information on the start address (e.g., see the upward arrow) and length (or size) (e.g., see the horizontal arrow) of the storage device 180 where the corresponding compressed weight data is stored.

[0043] The processor 120 or the DMAC 170 can read the compressed weight data WD from the storage device 180 using the length information LI.

[0044] In step S230, computing device 10 can load the compressed weight data WD based on the same address into memory 160. For example, processor 120 and / or DMAC 170 can load 1a compressed weight data WD1a, 1b compressed weight data WD1b, 1c compressed weight data WD1c, and 1d compressed weight data WD1d of the first compressed layer weight data L1_WD in parallel into storage spaces of memory 160 having sequential addresses, starting from the same first starting address.

[0045] 5, the 1a-compressed weight data WD1a, 1b-compressed weight data WD1b, 1c-compressed weight data WD1c, and 1d-compressed weight data WD1d of the first compressed layer weight data L1_WD may have different lengths (or sizes). For example, in the first compressed layer weight data L1_WD, the lengths (or sizes) of the 1a-compressed weight data WD1a, 1b-compressed weight data WD1b, and 1d-compressed weight data WD1d may be smaller than the length (or size) of the 1c-compressed weight data WD1c.

[0046] The processor 120 and / or the DMAC 170 can load the 2a compressed weight data WD2a, the 2b compressed weight data WD2b, the 2c compressed weight data WD2c, and the 2d compressed weight data WD2d of the second compressed layer weight data L2_WD in parallel starting from the same second starting address into storage spaces of the memory 160 having sequential addresses.

[0047] For example, the second start address may be an address immediately following the end address of the 1c compressed weight data WD1c having the longest length (or size) among the 1a compressed weight data WD1a, the 1b compressed weight data WD1b, the 1c compressed weight data WD1c, and the 1d compressed weight data WD1d of the first compressed layer weight data L1_WD. In other words, the location of the second start address in the storage space of the memory 160 may be based on the longest length (or size) among the 1a compressed weight data WD1a, the 1b compressed weight data WD1b, the 1c compressed weight data WD1c, and the 1d compressed weight data WD1d of the first compressed layer weight data L1_WD.

[0048] Therefore, the end address of the 1a-compressed weight data WD1a may be smaller than the end address of the 1c-compressed weight data WD1c. The processor 120 or the DMAC 170 may fill the storage space of the memory 160 starting from the address immediately following the end address of the 1a-compressed weight data WD1a to the end address of the 1c-compressed weight data WD1c with dummy data, a random pattern, a predetermined pattern, or no data. The storage space not filled with data may have a fixed value, such as "0" or "1."

[0049] 5, the 2a-compressed weight data WD2a, the 2b-compressed weight data WD2b, the 2c-compressed weight data WD2c, and the 2d-compressed weight data WD2d of the second compressed layer weight data L2_WD may have different lengths (or sizes). For example, in the second compressed layer weight data L2_WD, the lengths (or sizes) of the 2a-compressed weight data WD2a, the 2b-compressed weight data WD2b, and the 2d-compressed weight data WD2d may be smaller than the length (or size) of the 2c-compressed weight data WD2c.

[0050] The processor 120 and / or the DMAC 170 can load the 3a compressed weight data WD3a, the 3b compressed weight data WD3b, the 3c compressed weight data WD3c, and the 3d compressed weight data WD3d of the third compressed layer weight data L3_WD in parallel into storage spaces of the memory 160 having sequential addresses, starting from the same third starting address.

[0051] For example, the third start address may be the address immediately following the end address of the 2c compressed weight data WD2c having the longest length (or size) among the 2a compressed weight data WD2a, the 2b compressed weight data WD2b, the 2c compressed weight data WD2c, and the 2d compressed weight data WD2d of the second compressed layer weight data L2_WD. In other words, the location of the third start address in the storage space of the memory 160 may be based on the longest length (or size) among the 2a compressed weight data WD2a, the 2b compressed weight data WD2b, the 2c compressed weight data WD2c, and the 2d compressed weight data WD2d of the second compressed layer weight data L2_WD.

[0052] Therefore, the end address of the 2a-compressed weight data WD2a may be smaller than the end address of the 2c-compressed weight data WD2c. The processor 120 or the DMAC 170 may fill the storage space of the memory 160 starting from the address immediately following the end address of the 2a-compressed weight data WD2a to the end address of the 2c-compressed weight data WD2c with dummy data, a random pattern, a predetermined pattern, or no data. The storage space not filled with data may have a fixed value, such as "0" or "1."

[0053] 5, in the third compressed layer weight data L3_WD, the 3a compressed weight data WD3a, the 3b compressed weight data WD3b, the 3c compressed weight data WD3c, and the 3d compressed weight data WD3d may have different lengths (or sizes). For example, in the third compressed layer weight data L3_WD, the lengths (or sizes) of the 3a compressed weight data WD3a, the 3b compressed weight data WD3b, and the 3d compressed weight data WD3d may be smaller than the length (or size) of the 3c compressed weight data WD3c.

[0054] The end address of the 3a-compressed weight data WD3a may be smaller than the end address of the 3c-compressed weight data WD3c. The processor 120 and / or DMAC 170 may fill the storage space of the memory 160 starting from the address immediately following the end address of the 3a-compressed weight data WD3a to the end address of the 3c-compressed weight data WD3c with dummy data, a random pattern, a predetermined pattern, or no data. The unfilled storage space may have a fixed value, such as a value of "0" or "1."

[0055] In step S240, the computing device 10 may generate layer length information LLI, which may include first layer length information LLI1 including information on the start address (see upward arrow) and length (or size) (see horizontal arrow) of the memory 160 where the first compressed layer weight data L1_WD is stored.

[0056] The layer length information LLI may further include second layer length information LLI2 including information on the starting address (see upward arrow) of memory 160 where the second compressed layer weight data L2_WD is stored, and the length (or size) (see horizontal arrow).

[0057] The layer length information LLI may further include third layer length information LLI3, which includes information on the starting address (see upward arrow) of memory 160 where the third compressed layer weight data L3_WD is stored, and the length (or size) (see horizontal arrow).

[0058] In step S250, the computing device 10 may load the original length information OLI and the layer length information LLI into the memory 160. For example, the processor 120 and / or the DMAC 170 may store the original length information OLI and the layer length information LLI in a second length information area LIA2 of the memory 160.

[0059] 6 shows an example of a method by which the computing device 10 loads the compressed weight data WD into the accelerator 130 (e.g., step S120 of FIG. 2). Referring to FIGS. 1, 5, and 6, in step S310, the computing device 10 may read the first type length information and the second type length information from the memory 160. For example, the first type length information may be original length information OLI containing length information for each weight data. The second type length information may be layer length information LLI containing information for each layer. The processor 120 and / or the DMAC 170 may read the first type length information from the memory 160 and transmit it to the data controller CT of the accelerator 130.

[0060] In step S320, computing device 10 may simultaneously program DMA master M. For example, processor 120 may program DMA master M with the second type location information.

[0061] The processor 120 can program the DMA (direct memory access) master M to read data from the memory 160 using the first layer length information LLI1 and transfer the read data to the accelerator 130 (see FIG. 5). The DMA master M can read data in parallel or simultaneously from a memory space corresponding to the length (or size) indicated by the first layer length information LLI1, starting from a first start address indicated by the first layer length information LLI1, and transfer the data to the data controller CT of the accelerator 160, respectively. The data controller CT can convert the data transferred from the memory 160 into 1a-compressed weight data WD1a, 1b-compressed weight data WD1b, 1c-compressed weight data WD1c, and 1d-compressed weight data WD1d based on the original length information OLI.

[0062] In step S330, the processor 120 may determine whether the data transmitted to the accelerator 130 is the last compressed layer weight data L_WD. For example, the processor 120 may determine whether the data transmitted to the accelerator 130 is data including the third compressed layer weight data L3_WD. If the data transmitted to the accelerator 130 corresponds to the last compressed layer weight data L_WD, the processor 120 may finish loading the compressed weight data WD into the accelerator 130.

[0063] If the data communicated to accelerator 130 does not correspond to the last compressed layer weight data L_WD, processor 120 may load compressed layer weight data of the next layer into accelerator 130. For example, processor 120 may load second compressed layer weight data L2_WD into accelerator 130 by programming DMA master M with second layer length information LLI2. Note that processor 120 may load third compressed layer weight data L3_WD into accelerator 130 by programming DMA master M with third layer length information LLI3.

[0064] As described above, computing device 10 according to at least one embodiment of the present invention can use multiple DMA masters M to load compressed weight data WD into accelerator 130. Therefore, the speed at which compressed weight data WD is loaded into accelerator 130 is improved, and the loading time is reduced.

[0065] Note that the computing device 10 according to at least one embodiment of the present invention can load all of the weight data of each layer using the length information of the maximum length (or size) among the weight data of each layer having different lengths (or sizes) into the accelerator 130. Therefore, the number of times the DMA master M needs to be programmed is reduced, thereby improving the speed at which the compressed weight data WD is loaded into the accelerator 130 and shortening the loading time.

[0066] 7 illustrates a data controller CT and related components according to at least one embodiment of the present invention. Referring to FIGS. 1, 5, 6, and 7, the DMA master M can request the memory controller 150 to read weight data WD stored in the memory 160. The DMA master M can receive data from the memory 160 and the memory controller 150 according to a defined accelerator unit and transfer the received data to the data controller CT. For example, the DMA master M can be programmed to read data of a length (or size) corresponding to the layer length information LLI. For example, the DMA master M can be programmed with the same layer length information LLI.

[0067] Each of the data controllers CT may include a drain circuit DC, a decompression circuit DCC, and a validity monitor VM (or validity monitor circuit). One DMA master M is coupled to one accelerator core C via one drain circuit DC, one decompression circuit DCC, and one validity monitor VM of one data controller CT. As an example, one DMA master M, one drain circuit DC, one decompression circuit DCC, and one validity monitor VM may form a data chain that converts data read from memory 180, i.e., data including compressed weight data, into weight data and transmits it to one accelerator core C.

[0068] Each of the drain circuits DC can receive data from a corresponding one of the DMA masters M. Each of the drain circuits DC can receive a drain signal DS from a corresponding one of the validity monitors VM. Each of the drain circuits DC can be configured to transmit data transmitted from the corresponding DMA master M to a corresponding one of the decompression circuits DCC when the drain signal DS is inactive. Each of the drain circuits DCC can be configured to drain data transmitted from the corresponding DMA master M without transmitting it to the corresponding decompression circuit DCC when the drain signal DS is active. For example, each of the drain circuits DC can ignore or discard data transmitted from the corresponding DMA master M when the drain signal DS is active.

[0069] Each of the decompression circuits DCC may be configured to perform decompression on data transmitted from a corresponding drain circuit DC, and each of the decompression circuits DCC may transmit the decompressed data, i.e., weight data, to a corresponding validity monitor VM.

[0070] Each of the validity monitors VM can receive weight data (e.g., decompressed weight data) from a corresponding decompression circuit DCC. The validity monitors VM can commonly receive original length information OLI. Each of the validity monitors VM can monitor whether the length (or size) of the data (e.g., decompressed weight data) received from the corresponding decompression circuit DCC reaches the length (or size) indicated by the original length information OLI.

[0071] If the length (or size) of the data received from the corresponding decompression circuit DCC does not reach the length (or size) indicated by the original length information OLI, each of the validity monitors VM can deactivate the drain signal DS and transmit the data transmitted from the decompression circuit DCC to the corresponding one of the accelerator cores C. If the length (or size) of the data received from the corresponding decompression circuit DCC reaches the length (or size) indicated by the original length information OLI, each of the validity monitors VM can activate the drain signal DS.

[0072] 8 shows an example of how the drain circuit DC operates. Referring to FIGS. 1, 7, and 8, in step S410, the drain circuit DC may receive input data. For example, the drain circuit DC may receive data including compressed weight data from a corresponding DMA master M.

[0073] In step S420, the drain circuit DC may determine whether the drain signal DS is active. If the drain signal DS is not active, in step S430, the drain circuit DC may transmit data. For example, the drain circuit DC may transmit input data received from the corresponding DMA master M to the corresponding decompression circuit DCC. Then, step S450 is performed.

[0074] If the drain signal is active, the drain circuit DC may drain the input data in step S440. For example, the drain circuit DC may ignore or discard the input data. Then, step S450 is performed.

[0075] In step S450, the drain circuit DC can determine whether the data is finished. For example, the drain circuit DC can determine that the data is finished when no more input data is received from the corresponding DMA master M. If it is determined that the data is finished, the drain circuit DC can end the process. If it is determined that the data is not finished, the drain circuit DC can receive step S410 again.

[0076] As described above, a computing device 10 according to at least one embodiment of the present invention can store compressed weight data having different sizes in memory 160 in storage spaces having the same starting address and the same layout length information. The computing device 10 can simultaneously program the DMA master M to transfer weight data in parallel to the accelerator 130 using the same starting address and the same layout length information. The drain circuit DC and validity monitor VM of the data controller CT of the accelerator 130 can extract the decompressed weight data and transfer it to the accelerator core C. Because the number of times the DMA master M needs to be programmed is reduced, the speed at which the computing device 10 loads compressed weight data into the accelerator 130 can be improved, and the loading time can be reduced.

[0077] 9 shows a data controller CTa and related components according to an embodiment of the present invention. Compared with the data controller CT of FIG. 8, the data controller CTa of FIG. 9 may further include a demultiplexer DX and a multiplexer MS.

[0078] A demultiplexer DX is provided between the DMA master and the drain circuit DC. A multiplexer MX is provided between the validity monitor VM and the accelerator core C. The demultiplexer DX and the multiplexer MX can operate in response to a mode signal MS.

[0079] As at least one example, computing device 10 may provide a compressed mode and an uncompressed mode. In the compressed mode, compressed weight data may be provided to storage device 180. Computing device 10 may load the compressed weight data into accelerator 130 according to the methods described with reference to FIGS. 1-8. In the compressed mode, mode signal MS may have a first value. In response to mode signal MS having the first value, demultiplexer DX may communicate data communicated from DMA master M to drain circuit DC. In response to mode signal MS having the first value, multiplexer MX may communicate data communicated from validity monitor VM to accelerator core C.

[0080] In uncompressed mode, uncompressed weight data is provided to storage device 180. The uncompressed weight data is loaded into memory 180 in the same size. Computing device 10 can program DMA master M with the same starting address and the same layout length information. DMA master M can communicate the uncompressed weight data stored in memory 180 to accelerator 130.

[0081] In the uncompressed mode, the mode signal MS may have a second value different from the first value. In response to the mode signal MS having the second value, the demultiplexer DX may transmit data communicated from the DMA master M to the multiplexer MX. In response to the mode signal MS having the second value, the multiplexer MX may transmit data communicated from the demultiplexer DX to the accelerator core C.

[0082] 10 illustrates a data controller CTb and related components according to at least one embodiment of the present invention. Referring to FIG. 10, the data controller CTb may include a decompression circuit DCC and a validity monitor VM.

[0083] Each decompression circuit DCC can decompress data, i.e., data including compressed weight data, transmitted from a corresponding one of the DMA masters M. Each decompression circuit DCC can transmit the decompressed data to a corresponding one of the validity monitors M, VM.

[0084] Each validity monitor VM can determine whether the length (or size) of the data transmitted from the corresponding decompression circuit DCC reaches the length (or size) indicated by the original length information OLI. If the length (or size) of the data transmitted from the decompression circuit DCC does not reach the length (or size) indicated by the original length information OLI, each validity monitor VM can deactivate the reset signal RS. Each validity monitor VM can transmit the data transmitted from the decompression circuit DCC to a corresponding one of the accelerator cores C.

[0085] When the length (or size) of the data transmitted from the decompression circuit DCC reaches the length (or size) indicated by the original length information OLI, each of the validity monitors VM can activate a reset signal RS. The reset signal RS can be transmitted to a corresponding one of the DMA masters M. In response to determining that the reset signal RS is activated, the corresponding DMA master M can be reset.

[0086] As at least one example, the reset signal RS may also be provided to the decompression circuit DSS. In response to determining that the reset signal RS is activated, the corresponding decompression circuit DCC may be reset.

[0087] As at least one example, the data controller CTb may further include a demultiplexer DX and a multiplexer MX, as described with reference to Figure 9. In the uncompressed mode, the demultiplexer DX and the multiplexer MX can bypass data communicated from the DMA master M to the accelerator core C.

[0088] Figure 11 shows an example of how each validity monitor VM of the data controller CTb of Figure 10 operates. Referring to Figures 1, 10 and 11, at step S510, the validity monitor VM may receive input data. For example, the validity monitor VM may receive weight data from the corresponding decompression circuit DCC.

[0089] In step S520, the validity monitor VM may determine whether the data is over (or whether the end of the data has been reached). For example, the validity monitor VM may determine that the data is over if the length (or size) of the received weight data reaches the length (or size) pointed to by the original length information OLI. If it is determined that the data is over, in step S530, the validity monitor VM may reset the corresponding DMA master M among the DMA masters M. Thereafter, the validity monitor VM may terminate the process.

[0090] In step S540, the validity monitor VM may communicate the input data to the corresponding one of the accelerator cores C and continue monitoring. Thereafter, the validity monitor VM may start again from step S410.

[0091] As described above, a computing device 10 according to at least one embodiment of the present invention can store compressed weight data having different sizes in memory 160 in storage spaces having the same starting address and the same layout length information. The computing device 10 can simultaneously program the DMA master M to transfer weight data in parallel to the accelerator 130 using the same starting address and the same layout length information. The validity monitor VM of the data controller CT of the accelerator 130 can extract the decompressed weight data and transfer it to the accelerator core C. Because the number of times the DMA master M needs to be programmed is reduced, the speed at which the computing device 10 loads compressed weight data into the accelerator 130 can be improved, and the loading time can be reduced.

[0092] 12 illustrates another example of a compressed form of weight data (e.g., compressed weight data) being loaded into accelerator 130. Referring to FIGS. 1 and 12, as at least one example, accelerator 130 may execute a machine learning module that includes four layers.

[0093] The fourth compressed weight data WD4, the fifth compressed weight data WD5, the sixth compressed weight data WD6, and the seventh compressed weight data WD7 may each correspond to one layer of the machine learning module. Fourth length information LI4 indicating the length of the fourth compressed weight data WD4, fifth length information LI5 indicating the length of the fifth compressed weight data WD5, sixth length information LI6 indicating the length of the sixth compressed weight data WD6, and seventh length information LI7 indicating the length of the seventh compressed weight data WD7 are stored in storage device 180 together with the fourth compressed weight data WD4, the fifth compressed weight data WD5, the sixth compressed weight data WD6, and the seventh compressed weight data WD7.

[0094] 13 shows an example of a process for loading weight data WD compressed by the method of FIG. 4 from storage device 180 to memory 160. Referring to FIGS. 1, 4 and 13, fourth length information LI4, fifth length information LI5, sixth length information LI6, seventh length information LI7 and original length information OLI are stored in third length information area LIA3 of storage device 180.

[0095] The fourth compressed weight data WD4, the fifth compressed weight data WD5, the sixth compressed weight data WD6 and the seventh compressed weight data WD7 are stored in memory spaces at sequential addresses.

[0096] The fourth length information LI4 may include information on the start address (see the upward arrow) and length (see the horizontal arrow) of the fourth compressed weight data WD4. The fifth length information LI5 may include information on the start address (as indicated by the upward arrow) and length (see the horizontal arrow) of the fifth compressed weight data WD5.

[0097] The sixth length information LI6 may include information on the start address (see upward arrow) and length (see horizontal arrow) of the sixth compressed weight data WD6. The seventh length information LI7 may include information on the start address (see upward arrow) and length (see horizontal arrow) of the seventh compressed weight data WD7.

[0098] The processor 120 or the DMAC 170 can load the fourth compressed weight data WD4, the fifth compressed weight data WD5, the sixth compressed weight data WD6, and the seventh compressed weight data WD7 in parallel starting from the same starting address into storage spaces of the memory 160 having sequential addresses.

[0099] As shown in FIG. 13, the lengths (or sizes) of the fourth compressed weight data WD4, the fifth compressed weight data WD5, and the sixth compressed weight data WD6 may be smaller than the length (or size) of the seventh compressed weight data WD7.

[0100] The fifth compressed weight data WD5, the sixth compressed weight data WD6, and the seventh compressed weight data WD7 loaded into memory 160 can be loaded into accelerator 130 by the method described with reference to Figures 1 to 11.

[0101] 14 illustrates a computing device 20 according to another embodiment of the present invention. Referring to FIG. 14, the computing device 20 may include a system bus 210, a processor 220, an accelerator 230, a modem 240, a memory controller 250, a memory 260, and a storage device 280. The system bus 210, the processor 220, the modem 240, the memory controller 250, the memory 260, and the storage device 280 may be the same as or substantially similar to the system bus 110, the processor 120, the modem 140, the memory controller 150, the memory 160, and the storage device 180. Therefore, a repeated description thereof will be omitted.

[0102] The accelerator 230 may include an accelerator core C, a data controller CT, and a DMA master M. The DMA master M may be dedicated to the accelerator 230. Except that the DMA master M is included in the accelerator 230, loading the compressed weight data into the accelerator 230 may be performed in the same manner as described with reference to FIGS.

[0103] As at least one example, the DMA master M included in the accelerator 230 is referred to as an AXI master.

[0104] As some examples, computing device 10 or 20 has been described as including one accelerator 130 or 240. However, examples are not limited thereto, and computing device 10 or 20 may include two or more accelerators. Computing device 10 or 20 may load the compressed weight data into at least one accelerator using a DMA master M included in at least one of the two or more accelerators. Alternatively, computing device 10 or 20 may load the compressed weight data into at least one accelerator using a DMA master M of a separate DMA controller.

[0105] In the above-described embodiments, terms such as "first," "second," and "third" are used to describe components according to the technical concept of the present invention. However, terms such as "first," "second," and "third" are used to distinguish components from one another and do not limit the present invention. For example, terms such as "first," "second," and "third" do not imply any numerical meaning in terms of order or any form.

[0106] In the above-described embodiments, components according to embodiments of the invention are referenced using terminology directed to functional blocks and their functional block elements (e.g., including processor, accelerator, etc.). The blocks may be implemented as processing circuitry such as various hardware devices (e.g., integrated circuits (ICs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), etc.), firmware running on hardware devices, software (e.g., applications), and / or combinations of hardware devices and software. As at least some examples, processing circuitry may specifically be included in (and / or enabled by) a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a system-on-chip (SoC), a programmable logic unit, a microprocessor, an application-specific integrated circuit (ASIC), etc., and / or may include active and / or passive electronic components such as transistors, resistors, capacitors, etc., and / or electronic circuits including one or more of these components. Note that a block may include a circuit made of semiconductor elements within an IC or a circuit registered as Intellectual Property (IP). For example, a circuit may include active and / or passive electronic components such as transistors, resistors, capacitors, etc.

[0107] The above is a specific embodiment for carrying out the present invention. The present invention also includes embodiments that are not limited to the above-described embodiments, but are merely designed or can be easily modified. The present invention also includes techniques that can be easily implemented by modifying the embodiments. Therefore, the scope of the present invention should not be limited to the above-described embodiments, but should be defined not only by the claims below but also by equivalents to the claims of the present invention. [Explanation of symbols]

[0108] 10: Computing equipment 110: System bus 120: Processor 130: Accelerator 140: Modem 150: Memory controller 160:Memory 170:DMAC 180: Storage device

Claims

1. a first data controller configured to store original length information indicating an original length of data, receive first data having a first length, decompress the first data having the first length to generate second data having the original length, and output the second data having the original length; a second data controller configured to store the original length information, receive third data having a second length shorter than the first length, decompress the third data having the second length to generate fourth data having the original length, and output the fourth data having the original length; a first accelerator core configured to receive the second data having the original length and perform a first machine learning based operation using the second data as first weight data; a second accelerator core configured to receive the fourth data having the original length and perform a second machine-learning based operation using the fourth data as second weight data; Each of the first data controller and the second data controller monitoring when decompression is completed based on the original length; configured to terminate the decompression when the decompression is completed; Machine learning accelerator.

2. The machine learning accelerator includes: the first data controller and the second data controller: receiving the first data and the second data in parallel; configured to output the second data and the fourth data in parallel; The machine learning accelerator of claim 1 .

3. At least one of the first data controller and the second data controller a drain circuit configured to receive corresponding data of the first data and the second data, and configured to output the corresponding data when a corresponding drain signal is in an inactive state; a decompression circuit configured to receive the corresponding data from the drain circuit, decompress the corresponding data, and output the decompressed data; a monitor circuit configured to activate the corresponding drain signal when decompression of the data indicated by the original length information is completed; In response to the corresponding drain circuit being activated, the drain circuit is configured to cease outputting the corresponding data. The machine learning accelerator of claim 1 .

4. a first DMA master configured to read the first data having the first length and communicate the first data to the decompression circuit of the first data controller; a second DMA master configured to read fifth data including the second data and having the first length and to communicate the fifth data to the decompression circuit of the second data controller; the first DMA master and the second DMA master are simultaneously programmed based on the same starting address and the first length; The machine learning accelerator of claim 3 .

5. Each of the first data controller and the second data controller a decompression circuit configured to receive corresponding ones of the first data and the second data, decompress the corresponding data, and output the decompressed data; a monitor circuit configured to activate a reset signal when the decompression of the corresponding data indicated by the original length information is completed; In response to determining that the reset signal is activated, the decompression circuit is configured to cease the decompression of the corresponding data. The machine learning accelerator of claim 4 .

6. the monitor circuit is configured to transmit the reset signal to a corresponding one of an external first DMA (Direct Memory Access) master and an external second DMA master; The machine learning accelerator of claim 5 .

7. a first DMA master configured to read the first data having the first length and communicate the first data to the decompression circuit of the first data controller; a second DMA master configured to read fifth data including the second data and having the first length and to transmit the fifth data to the decompression circuit of the second data controller; In response to a determination by the monitor circuit of the first data controller that the reset signal is activated, the first DMA master is configured to stop reading the first data; In response to a determination by the monitor circuit of the second data controller that the reset signal is activated, the second DMA master is configured to stop reading the fifth data. The machine learning accelerator of claim 5 .

8. the first DMA master and the second DMA master are simultaneously programmed based on the same starting address and the first length; The machine learning accelerator of claim 7 .

9. a memory for storing first data having a first length and second data having a second length shorter than the first length; a machine learning accelerator configured to receive from the memory the first data and third data having the first length and including the second data; The machine learning accelerator decompressing the first data having the first length to generate first weight data; converting the third data having the first length to the second data having the second length; decompressing the second data having the second length to generate second weight data; configured to perform a machine learning based action based on the first weight data and the second weight data. Computing equipment.

10. 1. A method for a processor to load data into a machine learning accelerator, comprising: the processor simultaneously programming two or more DMA masters with a first starting address and first length information; the two or more DMA masters reading data in parallel from a memory based on the first starting address and the first length information, and transmitting the data read in parallel to the machine learning accelerator; The machine learning accelerator decompressing first data corresponding to the first length information among the parallel read data to generate first weight data; converting second data corresponding to the first length information to generate third data having a second length shorter than the first length indicated by the first length information; decompressing the third data having the second length to generate second weight data; configured to perform a machine learning based action based on the first weight data and the second weight data. method.

Images