Storage device and method of operating the same

Abstract

Disclosed is a method of operating a memory device including a neural network processor and a memory device, the method including: outputting, by a controller device, a trigger signal indicating a neural network processor to perform a neural network operation in response to a command of a host device; requesting, by the neural network processor, target model data regarding a parameter of a target model and instruction data for indicating the neural network operation from a memory device storing the target model data and the instruction data in response to the trigger signal; receiving, by the neural network processor, the target model data and the instruction data from the memory device; and outputting, by the neural network processor, inference data based on the target model data and the instruction data.

Description

[0001] Cross-references to related applications

[0002] This application is based on and claims priority to Korean Patent Application No. 10-2020-0117046, filed on September 11, 2020, with the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present invention relates to a storage device, and more specifically, to a storage device including a neural network processor and a method of operating the storage device. Background Technology

[0004] The storage device is configured to perform neural network operations based on deep learning patterns that require numerous operations for learning and inference. When mobile devices struggle to provide the computational power needed for deep learning, they can send data to a cloud server, which can then perform the deep learning operations. The mobile device can then receive the results of these operations and provide the deep learning inference data to the user. However, this process may encounter issues such as the ongoing cost of using cloud services and the inability to receive inference data offline from the cloud server.

[0005] Therefore, a neural processing unit (NPU) dedicated to neural network operations is built into the mobile device, so the mobile device itself can perform neural network operations. Summary of the Invention

[0006] This invention provides a storage device capable of efficiently performing neural network operations by improving the commands between the storage device's controller and the neural network processor. For example, according to one aspect of this invention, neural network operations can be performed by eliminating unnecessary commands between the storage device's controller and the neural network processor.

[0007] According to one aspect of this disclosure, a method is provided for operating a storage device including a neural network processor, the method comprising: a controller device outputting a signal instructing the neural network processor to perform neural network operations based on a command from a host device; the neural network processor requesting target model data and instruction data from a memory device storing target model data and instruction data based on the signal, the target model data corresponding to parameters of a target model, and the instruction data for performing neural network operations based on the target model; the neural network processor receiving the target model data and instruction data from the memory device; and the neural network processor outputting inference data based on the target model data and instruction data.

[0008] According to another aspect of this disclosure, a storage device is provided, comprising: a controller device configured to output a first signal instructing neural network operation based on a command from a host device; a memory device configured to store target model data corresponding to parameters of a target model in a first page region of a page in the memory device, and to store instruction data for instructing neural network operation in a second page region of the same page in the memory device; and a neural network processor configured to: output a second signal to the memory device based on the first signal, the second signal including a request from the memory device for the target model data and instruction data, receiving the target model data and instruction data from the memory device, and outputting inference data based on the target model data and instruction data.

[0009] According to another aspect of this disclosure, a memory device is provided, comprising: an array of memory cells including a plurality of pages; and a memory controller configured to: perform a read operation on a target page among the plurality of pages based on a read command for the target page, and output the target page, wherein each of the plurality of pages includes: a first page region configured to store target model data corresponding to a target model on which neural network operations will be performed; and a second page region configured to store instruction data for instructing neural network operations on the target model.

[0010] According to another aspect of this disclosure, a method executed by a neural network processor is provided, the method comprising: receiving a command to execute a command of a neural network; requesting, based on the command, target model data and instruction data from a memory device storing target model data and instruction data, the target model data corresponding to parameters of a target model, and the instruction data for performing neural network operations based on the target model; receiving the target model data and instruction data from the memory device; generating inference data based on the target model data and instruction data; and outputting the inference data.

[0011] According to another aspect of this disclosure, a storage device is provided, comprising: a memory storing one or more instructions; and a processor configured to execute one or more instructions, thereby: receiving a command to execute a neural network; based on the command output, requesting target model data and instruction data from the memory device storing target model data and instruction data, the target model data corresponding to parameters of a target model, and the instruction data for performing neural network operations based on the target model; receiving the target model data and instruction data from the memory device; generating inference data based on the target model data and instruction data; and outputting the inference data. Attached Figure Description

[0012] Embodiments of the inventive concept will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, wherein:

[0013] Figure 1 This is a schematic block diagram of a neural network system according to an example embodiment;

[0014] Figure 2 This is a block diagram illustrating the components of a storage device and an application processor (AP) according to a comparative embodiment, as well as the control signals between the storage device and the AP;

[0015] Figure 3 This is a flowchart of a method for operating a storage device according to an example embodiment;

[0016] Figure 4 This is a diagram illustrating a neural network layer according to an example embodiment;

[0017] Figure 5 This is a flowchart of a method for generating inference data by performing neural network operations on multiple layers, according to an example embodiment;

[0018] Figure 6 This is a diagram of a neural network processor according to an example embodiment;

[0019] Figure 7 It is a diagram showing multiple pages, including the first page area and the second page area;

[0020] Figure 8 This is a diagram illustrating an example where page groups (PGs) are mapped to multiple layers.

[0021] Figure 9 This is a diagram illustrating an example of receiving data corresponding to each layer from memory when multiple pages are specified for each layer;

[0022] Figure 10 This is a flowchart illustrating an example of determining the memory device for storing layer data based on the computational intensity of each layer;

[0023] Figure 11 It shows the basis Figure 10 The example embodiment shows a diagram of the mapping for tables at each determined storage location; and

[0024] Figure 12 This is a flowchart illustrating an example of determining whether to partition and store data based on the size of the data to be stored in the memory device. Detailed Implementation

[0025] The embodiments will now be described more fully with reference to the accompanying drawings.

[0026] Figure 1 This is a schematic block diagram of a neural network system according to an example embodiment.

[0027] Neural network systems can be trained to work with neural networks. According to example embodiments, a neural network system can analyze input data and infer information contained within the input data using a neural network. For example, a neural network system can learn using a neural network and training data. Based on the inferred information, the neural network system can determine the status of components in an electronic device equipped with the neural network system or control components of that electronic device. For example, neural network systems can be applied to smartphones, tablets, smart TVs (TVs), augmented reality (AR) devices, Internet of Things (IoT) devices, autonomous vehicles, robots, medical devices, drones, advanced driver assistance systems (ADAS), image display devices, and measuring devices that use neural networks to perform speech recognition, image recognition, image classification, and image processing operations. Furthermore, neural network systems can be installed in one of various types of electronic devices.

[0028] Reference Figure 1 The neural network system may include a storage device 10, an application processor (AP) 20, and random access memory (RAM) 30. The neural network system may also include input / output (I / O) modules, a security module, and a power control device, and further include various processors. In some example embodiments, some or all of the components of the neural network system may be formed on a single semiconductor chip. For example, the neural network system may be implemented as a system-on-a-chip (SoC). However, this disclosure is not limited thereto; thus, according to another example embodiment, the neural network system may be formed on multiple semiconductor chips.

[0029] AP 20 may include a central processing unit (CPU) 21 and a neural processing unit (NPU) 22. CPU 21 controls all operations of AP 20. CPU 21 may include a single core or multiple processor cores (or multiple cores). CPU 21 can process or execute programs and / or data stored in storage (e.g., storage device 10) using RAM 30. NPU 22 of AP 20 can update the weight parameters of a neural network model by training it and generate inference data based on input data received from external devices. For example, CPU 21 can execute an application and control NPU 22 to perform the required tasks based on the neural network, depending on the execution of the application.

[0030] Storage device 10 can receive commands from host device and provide data required by host device based on the commands. Storage device 10 may include controller device 11, neural network processor 12, and multiple memory devices (e.g., first memory device 13a and second memory device 13b). Controller device 11 can receive commands from host device other than storage device 10, analyze the commands, and send control signals and address signals to neural network processor 12 or multiple memory devices.

[0031] The neural network processor 12 can perform neural network operations based on received input data. Furthermore, the neural network processor 12 can generate information signals based on the results of the neural network operations. The neural network processor 12 can be implemented as a neural network operation accelerator, coprocessor, digital signal processor (DSP), or application-specific integrated circuit (ASIC), and performs operations on at least one of various types of neural network models, such as convolutional neural networks (CNNs), region-based neural networks with convolutional neural networks (R-CNN), region proposal networks (RPNs), recurrent neural networks (RNNs), stacked deep neural networks (S-DNNs), state-space dynamic neural networks (S-SDNNs), deconvolutional networks, deep belief networks (DBNs), restricted Boltzmann machines (RBMs), fully convolutional networks, long short-term memory (LSTM) networks, classification networks, planar residual networks, dense networks, hierarchical pyramid networks, and fully convolutional networks.

[0032] The plurality of memory devices may include at least one of volatile memory and non-volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), and / or flash memory. Volatile memory may include dynamic RAM (DRAM), static RAM (SRAM), and / or synchronous DRAM (SDRAM). According to an example embodiment, the plurality of memory devices may include at least one of hard disk drive (HDD), solid-state drive (SSD), compact flash (CF) card, secure digital card (SD) card, micro SD card, mini SD card, extreme digital (xD) card, and Memory Stick.

[0033] Multiple memory devices may include a first memory device 13a and a second memory device 13b. The first memory device 13a and the second memory device 13b may be different types of memory devices, and may be memory devices classified according to data processing rate or data transfer rate. For example, the first memory device 13a may be a high-bandwidth NAND (HBN) memory, and the second memory device 13b may be a NAND memory. Although Figure 1The example embodiments shown only illustrate two memory devices as the plurality of memory devices, but the inventive concept of this disclosure is not limited thereto. Therefore, according to other example embodiments, the plurality of memory devices may include two or more memory devices, and may include two or more types of memory devices. According to yet another example embodiment, memory device 10 may include one memory device.

[0034] The multiple memory devices may include multiple memory blocks, each of which includes multiple pages. The neural network processor 12 can specify the page to be read from the memory devices and send the address of the specified page to the memory devices; therefore, the memory devices can send data about the specified page to the neural network processor 12. The multiple memory devices may store target model data in the first page area of ​​each page and instruction data in the second page area of ​​each page.

[0035] The target model data can be data about the parameters of the target model on which the neural network operation will be performed, and the instruction data can be data instructing the neural network operation. The following will refer to... Figure 4 The process described is to perform a series of neural network operations by reading data from a memory device that stores target model data in a first page area and instruction data in a second page area.

[0036] Figure 2 This is a block diagram illustrating the control signals between the storage device 10 and the AP 20 of the comparison neural network system, as well as the configuration of the storage device 10 and the AP 20.

[0037] Reference Figure 2 AP 20 can send command 210 to storage device 10 along with input data for performing neural network operations. Command 210 can be a command signal for instructing neural network operations. According to another example embodiment, command 210 can be a signal for specifying a target neural network model for performing operations and a command signal for instructing neural network operations. Input data can be data for generating inference results by performing neural network operations. For example, input data can be image data for generating inference results of a target object. Alternatively, according to a comparative embodiment, AP 20 can send instruction data to storage device 10 to perform neural network operations on a target model. The controller device 11 of storage device 10, which receives the instruction data, can decode the instruction data and generate instruction signal 220a.

[0038] The controller device 11 of the storage device 10 can receive command 210 and send instruction signal 220a to the neural network processor 12 to instruct the neural network processor 12 to perform neural network operations on the input data. According to a comparative neural network system, the instruction signal 220a may include a trigger signal for instructing the neural network processor 12 to perform neural network operations on a target model and signals for decoding instruction data and instructing a series of operations on each of the multiple layers.

[0039] For example, controller device 11 can instruct neural network processor 12 to perform neural network operations on the target model, and neural network processor 12 can receive instruction signal 220a and begin performing neural network operations on the first layer. In this case, neural network processor 12 can perform a series of operations to output result data about the first layer. To perform each operation, neural network processor 12 must receive instruction signals 220a sequentially from controller device 11. Neural network processor 12 can perform a first operation on the first layer and notify controller device 11 that the first operation has been completed. Controller device 11 can be notified that the first operation has been completed and can instruct neural network processor 12 to perform a second operation. That is, when m operations are performed on n layers to generate inference results (where m and n are integers), controller device 11 of the comparison neural network system must send and receive a total of 2×n×m instruction signals 220a and a total of 2×n×m completion signals 220b.

[0040] According to the comparison, the memory device 13 of the neural network system can store only the target model data, and in response to the model data request signal 230 of the neural network processor 12, provide the target model data to the neural network processor 12 without sending instruction data to the neural network processor 12. That is, according to Figure 2 In the comparative neural network system shown, memory device 13 does not provide instruction data to neural network processor 12.

[0041] Conversely, the storage device 10 according to the present invention can store the target model data and instruction data together in a memory (e.g., a first memory device 13a or a second memory device 13b), receive the target model data and instruction data together, and perform a series of operations without sending the instruction signal 220a and the completion signal 220b to the controller device 11 and receiving the instruction signal 220a and the completion signal 220b from the controller device 11.

[0042] Figure 3 This is an operation based on the example embodiment. Figure 1 A flowchart of the method for using storage device 10. (Refer to...) Figure 1 describe Figure 3 .

[0043] According to an example embodiment, the neural network processor 12 of the storage device 10 can receive signals from the controller device 11, receive target model data and instruction data together from the memory device based on trigger signals, and output inference data. According to an example embodiment, the neural network processor 12 of the storage device 10 can receive trigger signals from the controller device 11. According to an example embodiment, the neural network processor 12 can receive target model data and instruction data from either the first memory device 13a or the second memory device 13b.

[0044] In operation S10, the controller device 11 of the storage device 10 may output a signal to the neural network processor 12 in response to a command received from the host device. The signal may be a trigger signal instructing the neural network processor 12 to perform neural network operations. Alternatively, the controller device 11 may send both a signal instructing neural network operations and a signal specifying the model on which the neural network operations will be performed from among a plurality of neural network models.

[0045] In operation S20, the neural network processor 12 of the storage device 10 can receive commands from the controller device 11 and request target model data and instruction data from the memory device. According to an example embodiment, the neural network processor 12 can request target model data and instruction data from either the first memory device 13a or the second memory device 13b. The neural network processor 12 of the example embodiment can determine the type of input data or the computational complexity of the input data and select a target model from multiple neural network models to perform the operation. However, the neural network processor 12 is not limited to this; thus, according to another example embodiment, the neural network processor 12 can receive a signal from a trigger signal instructing the target model to perform an operation and select the target model to perform the operation.

[0046] The neural network processor 12 can receive page addresses stored in an internal buffer memory or a target memory stored in an internal controller, and receive target model data and instruction data stored in the received page addresses from the memory devices. According to an example embodiment, the neural network processor 12 can receive target model data and instruction data stored in the received page addresses from a first memory device 13a or a second memory device 13b.

[0047] In operation S30, the neural network processor 12 can output inference data about the input data based on the target model data and instruction data. The target model data may include data from multiple layers. The neural network processor 12 can output result data corresponding to each layer based on the layer data. The result data of the current layer can be input to the next layer; therefore, inference data can be generated based on the output result data of the last layer.

[0048] To generate result data for each layer, the neural network processor 12 can perform multiple operations. Each of these operations can be performed based on instruction data. As an example, to generate result data for each layer, the neural network processor 12 can perform a multiplication operation on the input data and weight parameters. In this case, the controller of the neural network processor 12 can operate the processing circuitry based on instruction data received from the memory device to output the result of the multiplication operation on the input data and weight parameters.

[0049] Figure 4 This is a diagram illustrating a neural network layer according to an example embodiment. (Refer to...) Figure 1 describe Figure 4 .

[0050] The neural network model according to the example embodiment may include multiple layers (e.g., first to fourth layers L1, L2, L3, and L4). The neural network processor 12 may output result data for each of the first to fourth layers L1, L2, L3, and L4, receive result data for previous layers, and generate inference data pieces OUT1 and OUT2 based on the last layer. (Refer to...) Figure 4 A neural network model can include four layers, for example, layers L1, L2, L3, and L4. Layer L1 can be the input layer, layers L2 and L3 can be hidden layers, and layer L4 can be the output layer. Although in Figure 4 The example embodiment illustrates four layers L1, L2, L3, and L4, but this disclosure is not limited thereto. Thus, according to another example embodiment, a different number of layers may be provided.

[0051] according to Figure 4 In the illustrated example embodiment, the neural network processor 12 can provide input data IN1, IN2, and IN3 to a first layer L1 and receive output data from the first layer L1. Then, the neural network processor 12 can provide the output data from the first layer L1 as input data to a second layer L2 and receive output data from the second layer L2. Next, the neural network processor 12 can provide the output data from the second layer L2 as input data to a third layer L3 and receive output data from the third layer L3. Finally, the neural network processor 12 can provide the output data from the third layer L3 as input data to a fourth layer L4, which is the final layer, and receive output data from the fourth layer L4. The neural network processor 12 generates inference data slices OUT1 and OUT2 based on the output data from the fourth layer.

[0052] The storage device 10 can store data about the trained neural network model as model data, and weight parameters updated by training for each layer as layer data. That is, the model data can include multiple layer data and structural data about the multiple layers included in the neural network model.

[0053] Figure 5 This is a flowchart of a method for generating inference data by performing neural network operations on multiple layers, according to an example embodiment. Figure 6 This is a diagram of a neural network processor 12 according to an example embodiment.

[0054] Reference Figure 6 The neural network processor 12 may include a communication interface 121, a compiler 122, a controller 123, a buffer memory 124, and processing circuitry 125. The communication interface 121 can receive trigger signals from the controller device 11 of the storage device 10, or send request signals for requesting target model data and instruction data to the storage device. Additionally, the communication interface 121 can receive target model data and instruction data from the storage device and send the generated inference data to an external host device or storage device.

[0055] Reference Figure 5 When performing neural network operations on a target model comprising multiple layers, the neural network processor 12 can perform operations on each layer and generate inference data. The neural network processor 12 can, in response to a received signal, first perform neural network operations on the first layer.

[0056] In operation S210, the neural network processor 12 can request layer data and instruction data regarding the nth layer from the memory device. According to an example embodiment, the neural network processor 12 can request layer data and instruction data regarding the nth layer from either the first memory device 13a or the second memory device 13b. Although the instruction data corresponding to each piece of layer data can vary depending on the layer, the instruction data used to perform the neural network operations can be similar when the operations on each layer are similar.

[0057] In operation S310, the memory device that has received the request signal can send the nth layer data and instruction data corresponding to the request signal to the neural network processor 12. According to the example embodiment, the nth layer data can be stored as weight parameters corresponding to the nth layer. The neural network processor 12 can generate instruction signals to drive the processing circuit 125 of the neural network processor 12 to perform operations on the weight parameters and input data based on the instruction data.

[0058] In operation S320, the neural network processor 12 can perform multiple operations on the nth layer and generate result data. For example, the multiple operations may include multiplication, accumulation, and a next-layer call operation. However, the multiple operations are not limited to these, but may include all operations determined to generate result data for each neural network model. The result data may be data computed after all operations determined in each layer have been performed, and may be input data for the next layer. The neural network processor 12 can generate result data for each layer, store the result data in a buffer memory 124, and call the next layer in response to the result data being stored in the buffer memory 124.

[0059] In operation S330, the neural network processor 12 can determine whether the nth layer of the generated result data is the last layer. The last layer can refer to... Figure 4 The output layer of the nth layer can be a layer without a next layer. Based on the determination that the nth layer has a next layer, the neural network processor 12 can request layer data and instruction data for the (n+1)th layer and perform neural network operations on the (n+1)th layer.

[0060] In operation S340, when it is determined that the nth layer is the last layer, the neural network processor 12 can generate result data about the nth layer as inference data. The neural network processor 12 can store the generated inference data in a memory device, or provide the generated inference data to the host device through the communication interface 121 of the neural network processor 12.

[0061] Reference Figure 6 The compiler 122 of the neural network processor 12 can convert data received through the communication interface 121 into assembly language that can be processed by the neural network processor 12, and store the assembly language in the buffer memory 124. (See reference...) Figure 6 The compiler 122 may be included in the neural network processor 12 and converts the received target model data or instruction data into assembly language, but the inventive concept is not limited thereto. In another embodiment, the compiler 122 may be included in a memory device and receive the assembly language converted from data received through interface 121.

[0062] The controller 123 of the neural network processor 12 can perform all control operations on the neural network processor 12. For example, the controller 123 of the neural network processor 12 can generate instruction signals for driving the processing circuit 125 based on instruction data stored in the buffer memory 124. In addition, when multiple operations are to be performed to generate result data, the controller 123 can instruct the next operation after each operation is performed.

[0063] The buffer memory 124 can at least temporarily store data received through the communication interface 121 and data generated by the processing circuit 125. For example, the data received through the communication interface 121 may include target model data and instruction data, and the data generated by the processing circuit 125 may include result data generated for each layer and intermediate data generated for each operation. The buffer memory 124 can at least temporarily store result data and provide result data to the processing circuit 125 in response to the execution of an operation at the next layer. In addition, when the neural network processor 12 receives data for each page of the memory device, the instruction data and target model data stored together for each page can be separated and stored in separate regions.

[0064] The processing circuit 125 can receive at least some of the input data, intermediate data, and target model data from the buffer memory 124, perform operations in response to instruction signals, and generate intermediate data in each operation. The processing circuit 125 can use the intermediate data generated in the previous operation to perform the next operation.

[0065] Figure 7 This is a diagram of multiple pages, each of which includes a first page area "Area 1" and a second page area "Area 2".

[0066] A memory device (which may be a non-volatile memory device) may include multiple memory cell blocks, each of which may include multiple pages. See reference. Figure 7 Each page may include a first page area "Area 1" and a second page area "Area 2", and multiple data of different types may be stored in the first page area "Area 1" and the second page area "Area 2" respectively. For example, target model data may be stored in the first page area "Area 1", and instruction data may be stored in the second page area "Area 2". The first page area "Area 1" may be a set of memory cells configured to store 512 bits, 2k bits, and 4k bits, and the second page area "Area 2" may be a set of memory cells configured to store 16 bits, 64 bits, and 128 bits. However, the memory size of the first page area "Area 1" and the second page area "Area 2" is not limited to this, and the ratio of the first page area "Area 1" to the second page area "Area 2" may vary depending on the page.

[0067] When the neural network processor (reference) Figure 6 In step 12), when performing neural network operations by receiving data about multiple pages, the instruction data may include address information about the next page to be read after the current page, and the neural network processor 12 may load the next page from the memory device based on the address information about the next page.

[0068] According to another example embodiment, when the neural network processor 12 loads multiple pages, identification bits indicating the page range can be written in the first and last pages of the multiple pages. The neural network processor 12 can load pages sequentially from the memory device based on the identification bits of the first page of the multiple pages, and end the loading of pages based on the identification bits of the last page of the multiple pages. By using the above method to load multiple pages, the target model data and instruction data for performing neural network operations can be separated and stored in a buffer memory.

[0069] Figure 8 This is a diagram illustrating an example where page groups PG1, PG2, PG3, and PG4 are mapped to multiple layers. (Refer to...) Figure 1 describe Figure 8 .

[0070] Reference Figure 8 When performing neural network operations on model A in multiple neural network models, pages about multiple layers can be mapped. Although Figure 8 The illustration shows a page mapping based on model A, but the inventive concept is not limited to this, and pages can be mapped to each layer for each of multiple neural network models. (See also...) Figure 8 Model A may include n layers, and the layer data and instruction data corresponding to each layer may be stored in page groups PG separated by the layers. For example, the first page group PG1 may include layer data and instruction data for the first layer, and the first page group PG1 and the second page group PG2 may correspond to addresses that are separate from each other.

[0071] According to an example embodiment, the controller device 11 or the neural network processor 12 of the storage device 10 can store the address range of page groups PG corresponding to each layer. When the address range of page groups PG for each layer is stored in the controller device 11 of the storage device 10, the controller device 11 can send a trigger signal to the neural network processor 12, and simultaneously send the address range information corresponding to each layer of the target model. When the address range of page groups PG for each layer is stored in the neural network processor 12, the neural network processor 12 can load the pages corresponding to the address range of each layer. When it is determined that the operation on the corresponding layer is completed, the neural network processor 12 can load the pages corresponding to the next layer.

[0072] Therefore, the neural network processor 12 according to the example embodiment can receive layer data and instruction data for multiple layers by receiving a trigger signal only once from the controller device 11 of the storage device 10. Furthermore, the neural network processor 12 can generate inference data without sending additional signals to and receiving additional signals from the controller device 11.

[0073] Figure 9 This is a diagram illustrating an example of receiving data corresponding to each level from memory when multiple pages are specified for each level. (See also...) Figure 1 , Figure 3 and Figure 9 When the neural network processor 12 according to the example embodiment receives layer data and instruction data corresponding to multiple pages, the neural network processor 12 can receive data about the multiple pages in sequence. Figure 3 In operation S10, the neural network processor 12 may specify a target model in response to receiving a trigger signal from the controller device 11 of the storage device 10, and load the address range corresponding to multiple layers of the target model from the controller device 11 or the buffer memory of the storage device 10.

[0074] Reference Figure 9 In operation S211, the neural network processor 12 can specify the page address range of the layer on which neural network operations will be performed. As an example, a page group storing layer data and instruction data for the nth layer can be located in the address range A1 to A2. k Internally, the neural network processor 12 can request addresses A1 to A2 from the memory device. k Corresponding data.

[0075] In operation S311, the memory device can begin reading data from the page corresponding to address A1, and read data about the page corresponding to the specified address.

[0076] In operation S312, the neural network processor 12 can load data about the page corresponding to the address, and partition and store layer data and instruction data. According to an embodiment, the neural network processor 12 can partition a series of bits received based on the identification bits included in each of the layer data and instruction data into a portion corresponding to the layer data and a portion corresponding to the instruction data, and store the partitioned portions in a buffer memory.

[0077] In operation S313, the neural network processor 12 can determine whether the address of the received page is the last page of the page address range of the corresponding layer. As an example, when the nth layer is in A1 to A... k When the address is within the specified page address range, the neural network processor 12 can determine whether the address of the received page is A. k When the address of the received page is the last page, the neural network processor 12 can perform neural network operations on the nth layer; however, when the address of the received page is not the last page, the neural network processor 12 can load data about the page corresponding to the next address.

[0078] Figure 10This is a flowchart illustrating an example of determining the memory device for storing layer data based on the computational intensity of each layer. Figure 11 It shows the basis Figure 10 The embodiment maps a diagram of tables for each defined storage location. (Refer to...) Figure 1 describe Figure 10 and Figure 11 .

[0079] According to the example embodiment, the storage device 10 can store target model data and instruction data together in each page of the memory device, such as Figure 7 As in the example embodiment. The storage device 10 may store layer data in a first page region of a memory page of the memory cell array and instruction data in a second page region of a memory page of the memory cell array. The storage device 10 may load data from the page and store mutually separate identification bits together in the layer data and instruction data to identify each of the layer data and instruction data.

[0080] The target model data may include layer data corresponding to each of the multiple layers, and the storage device 10 may store multiple pages containing the layer data corresponding to each of the multiple layers as a page group in the memory device. The controller device 11 or the neural network processor 12 may map each of the multiple layers to the address of each layer in the page group and store the mapped layers and addresses.

[0081] Reference Figure 10 The storage device 10 according to the example embodiment may include a plurality of memory devices. The storage device 10 may determine from the plurality of memory devices which memory devices will store layer data and instruction data for each layer based on the computational intensity required to perform neural network operations. That is, assuming the plurality of layers includes a first layer and an nth layer as the last layer, the storage device 10 may determine which memory device will store layer data and instruction data for the first layer to the memory device storing layer data and instruction data for the nth layer.

[0082] In operation S40, the storage device 10 can determine the computational intensity required to perform neural network operations on the nth layer and determine whether the computational intensity exceeds a threshold. The computational intensity can correspond to the computational resource requirements for performing the neural network operations. According to an example embodiment, the computational intensity can be a measurement of floating-point operations performed using code portions related to the amount of memory access required to perform the operation on the corresponding layer. The computational intensity can be proportional to the amount of computation required for the corresponding layer. That is, because the computational intensity is proportional to the amount of computation relative to the model size, the computational intensity can decrease as the model size increases and the amount of computation decreases.

[0083] When the storage device 10 determines that the computation intensity corresponding to the nth layer is less than a threshold, the process can proceed to operation S50. Otherwise, when the storage device 10 determines that the computation intensity is greater than or equal to the threshold, the process can proceed to operation S60.

[0084] In operation S50, because the arithmetic intensity corresponding to the nth layer is less than a threshold, the neural network processor 12 can determine that the nth layer has a small amount of computation for a given size, and stores the nth layer data in the first memory device 13a. Conversely, in operation S60, because the computation intensity corresponding to the nth layer is greater than or equal to a threshold, the neural network processor 12 can determine that the nth layer has a large amount of computation for a given size, and stores the nth layer data in the second memory device 13b.

[0085] According to the example embodiment, the first memory device 13a of the storage device 10 may be a memory device capable of transmitting a large amount of information per second, while the second memory device 13b may be a memory device capable of transmitting a smaller amount of information per second compared to the first memory device 13a. For example, the first memory device 13a may be an HBN memory capable of transmitting and receiving data at 10Gbps, and the second memory device 13b may be a NAND memory capable of transmitting and receiving data at 4Gbps. That is, the first memory device 13a may be an HBN memory capable of transmitting and receiving data at 10Gbps, and the second memory device 13b may be a NAND memory capable of transmitting and receiving data at 4Gbps.

[0086] The storage device 10 according to the example embodiment may be a storage device installed in a mobile device. Therefore, it may be necessary to efficiently utilize the memory device to perform neural network operations in the mobile device. Thus, in the example embodiment, only some layer data determined to have a large data size compared to the computational load can be stored in the HBN memory, while the remaining layer data can be stored in ordinary NAND memory. Therefore, the HBN memory installed in the mobile device can be utilized efficiently.

[0087] In operation S70, the storage device 10, where the nth layer data is stored in the first memory device 13a or the second memory device 13b, can determine whether the nth layer is the last layer. When it is determined that the nth layer is the last layer, the storage device 10 can terminate the data storage because the storage of the target model data has been completed. Otherwise, when it is determined that the nth layer is not the last layer and there are more layers of data to be stored, the storage device 10 can proceed to the operation of storing the layer data and instruction data for the next layer.

[0088] Reference Figure 11 Storage device 10 can map and store the storage location of each layer. According to... Figure 11In an example embodiment, layer data and instruction data for each of the first, second, and fourth layers can be stored in the first memory device 13a, and layer data and instruction data for each of the third and nth layers can be stored in the second memory device 13b. Storage device 10 can store information regarding... Figure 1 Mapping information of the storage location for each layer.

[0089] When the neural network processor 12 performs neural network operations by loading target model data, layer data for each layer can be loaded based on mapping information about the storage location of its corresponding layer. As an example, layer data and instruction data for the first layer can be loaded from the first memory device 13a, and layer data and instruction data for the third layer can be loaded from the second memory device 13b.

[0090] according to Figure 10 and Figure 11 In some exemplary embodiments, layer data for each layer may be stored in either of two memory devices (e.g., a first memory device 13a and a second memory device 13b), but the number of memory devices is not limited thereto. In other embodiments, layer data for each layer may be stored in any of at least three memory devices.

[0091] Figure 12 This is a flowchart illustrating an example of determining whether to partition and store data based on the size of the data to be stored in the memory device. (Refer to...) Figure 1 describe Figure 12 .

[0092] Reference Figure 12 According to the example embodiment, the storage device 10 can determine whether to partition the data to be stored and to store the partitioned data in multiple pages. As an example, when the layer data comprises a series of bits, the partitioning of the layer data can be determined based on whether the number of bits of the layer data and instruction data is greater than the number of bits that the memory device can store in one page. The neural network processor 12 can load all the layer data by loading the layer data partitioned into multiple pages in page order.

[0093] In operation S41, storage device 10 can determine whether the total data size, including instruction data and layer data, is greater than the page size. When the total data size is greater than the page size, all data cannot be stored in one page. Therefore, storage device 10 can determine that data needs to be partitioned and proceed to operation S51. Conversely, when it is determined that all data is less than the page size (i.e., when the total data size is less than the page size), storage device 10 can determine that data partitioning is not necessary and proceed to operation S71.

[0094] In operation S51, the storage device 10 can divide the layer data by subtracting the instruction data size from the page size. For example, when the page size is 4kb and the instruction data size is 1kb, the storage device 10 can divide the layer data in 3kb units. When the layer data size is 24kb, it can be divided into a total of 8 pages in 3kb units.

[0095] In operation S61, the storage device 10 can store the divided layer data and instruction data together on a single page. As an example, in operation S51, the data divided into 8 pages can be stored together with the instruction data. In operation S71, when the total data size is less than the page size, the storage device 10 can store the layer data and instruction data together on a single page.

[0096] Although the inventive concept has been specifically shown and described with reference to exemplary embodiments thereof, it should be understood that various changes in form and detail may be made without departing from the spirit and scope of the appended claims.

Claims

1. A method of operating a storage device including a neural network processor, the method comprising: The controller device outputs a signal instructing the neural network processor to perform neural network operations based on a command from the host device; The neural network processor selects a target model from multiple neural network models based on the signal, requests the target model data and the instruction data from a memory device storing target model data and instruction data, wherein the target model data corresponds to the parameters of the target model, and the instruction data is used to perform the neural network operation based on the target model; The neural network processor receives the target model data and the instruction data from the memory device; as well as The neural network processor outputs inference data based on the target model data and the instruction data. The target model data and the instruction data are stored together in each page of the memory device. The step of storing the target model data and the instruction data includes storing the layer data corresponding to each layer of the target model and the instruction data in each page group.

2. The method according to claim 1, wherein, The steps of requesting the target model data and the instruction data include: Specify the page address of the memory device, and The request is to write the target model data and the instruction data together in the page address.

3. The method according to claim 2, wherein, The steps of requesting the target model data and the instruction data include: When the instruction data written to the specified page address includes information corresponding to the next page address, the target model data and the instruction data are requested from the next page address.

4. The method according to claim 1, wherein, The step of outputting the inference data includes: performing a series of operations on the input data based on the instruction data, and outputting result data corresponding to each layer of the target model.

5. The method according to claim 4, wherein, The step of requesting the target model data and the instruction data includes: requesting first-layer data and the instruction data corresponding to the first layer of the target model from the memory device based on the signal.

6. The method according to claim 5, wherein, The steps for outputting the inferred data include: Receives the first layer data and the instruction data from the memory device, and generates first result data by performing the neural network operation on the first layer; and The neural network processor requests second-layer data from the memory device based on the first result data.

7. The method according to claim 6, wherein, The steps for outputting the inferred data include: The second result data is generated by performing the neural network operation on the second layer using the first result data as input data; and The second result data is generated as the inference data based on the determination that the second layer is the last layer of the target model.

8. The method according to claim 1, wherein, The steps of outputting the inference data include: dividing the target model data and the instruction data; and storing the target model data and the instruction data in separate buffer memories.

9. The method according to claim 1, wherein, The step of storing the target model data and the instruction data includes: identifying a memory that will store layer data and instruction data corresponding to each layer of the target model based on the computational intensity of the neural network operation performed on each layer of the target model.

10. The method according to claim 9, wherein, The step of identifying the memory to store the layer data and the instruction data includes: Based on the fact that the first computational intensity of the first layer is less than a threshold, the first layer data is stored in the first memory, and Based on the fact that the first operational strength of the first layer is greater than or equal to the threshold, the data of the first layer is stored in the second memory.

11. The method according to claim 1, wherein, The step of storing the target model data and the instruction data includes: based on the instruction data, which includes page order information corresponding to multiple pages, writing an identifier bit indicating the page range to which the target model data is written in the first and last pages of the multiple pages.

12. The method according to claim 1, wherein, Storing the target model data and the instruction data includes storing the address of a page containing the target model data in the controller device or the neural network processor.

13. A storage device, comprising: A controller device configured to instruct neural network operation based on a first signal output from a host device; A memory device configured to store target model data corresponding to parameters of a target model in a first page region of a page in the memory device, and to store instruction data instructing the operation of the neural network in a second page region of the page in the memory device; and The neural network processor is configured as follows: Based on the first signal, the target model is selected from multiple neural network models, and a second signal is output to the memory device, the second signal including a request from the memory device for the target model data and the instruction data. Receive the target model data and the instruction data from the memory device, and Based on the target model data and the instruction data, inference data is output. The memory device stores the layer data corresponding to each layer of the target model and the instruction data in each page group.

14. The storage device according to claim 13, wherein, The memory device is also configured to: Specify page address, and The target model data and instruction data corresponding to the specified page address are provided to the neural network processor.

15. The storage device according to claim 13, wherein, The neural network processor is also configured to: Based on the first-layer data corresponding to the first layer of the target model and the instruction data, a series of operations are performed on the input data. Generate result data corresponding to the first layer of the target model, and In response to the generation of the resulting data, a second layer of data is requested from the memory device.

16. A memory device, comprising: An array of memory cells comprising multiple pages; as well as The memory controller device is configured as follows: Based on the read command for the target page, a read operation is performed on the target page among the plurality of pages, and Output the target page. Each of the plurality of pages includes: The first page area is configured to store target model data corresponding to the target model on which neural network operations will be performed; and The second page area is configured to store instruction data for instructing neural network operations on the target model. In this context, multiple layers of each of the multiple neural network models, including the target model, can be mapped to the multiple pages. The memory cell array includes multiple page groups, each of which is configured to store layer data and instruction data corresponding to each layer of the target model.

17. The memory device according to claim 16, wherein, The plurality of page groups store layer data and instruction data corresponding to layers whose computational intensity is less than a threshold for performing the neural network operations.

Images