Memory device, operating method, and electronic device

Abstract

The memory device is configured to perform neural network learning, the memory device comprising a first dedicated memory corresponding to a first link included in a neural network and configured to store a first forward propagation weight of the first link and at least one first candidate weight, and a first processing element (PE), a first processing element (PE) is configured to perform a multiplication operation between an input and a first forward propagation weight stored in a first dedicated memory for a first link, wherein the first forward propagation weight stored in the first dedicated memory is configured to be updated with one of the at least one first candidate weight after a multiplication operation of the first link corresponding to the first dedicated memory.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to Korean Patent Application No. 10-2024-0110768, filed on August 19, 2024, with the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure relates to memory devices, methods of operation, and electronic devices. Background Technology

[0004] Applications such as high-performance and / or graphics algorithms are data-intensive and computationally intensive. Machine learning applications (such as neural networks) can include a large number of operations and memory requirements involving a large number of computations. Machine learning applications require computing systems with large operational and memory capabilities to more accurately train or learn from diverse datasets.

[0005] Neural networks can include multiple layers. To obtain more accurate results, the number of layers included in a neural network is increasing, and the time elapsed for each operation may increase due to the increased amount of work. To perform forward and backward propagation to train and / or derive the results of the neural network, each of the multiple layers needs to receive the results of operations in the previous layer. Therefore, it may be inefficient to use processing power, and the time elapsed for neural network operations may increase. Thus, there is a need to reduce the time elapsed for neural network operations by using processing power more efficiently. Summary of the Invention

[0006] Typically, in some aspects, this disclosure relates to a memory device, a method of operating a memory device, and an electronic device including the memory device, the memory device being configured to reduce the time elapsed during neural network operation and including a plurality of dedicated memories configured to store weights corresponding to a plurality of links included in the neural network, respectively.

[0007] According to some embodiments, this disclosure relates to a memory device configured to perform neural network learning, the memory device including a first dedicated memory and a first processing element (PE), the first dedicated memory corresponding to a first link included in the neural network and configured to store first forward propagation weights and at least one first candidate weight for the first link, the first processing element (PE) being configured to perform a multiplication operation between an input and the first forward propagation weights stored in the first dedicated memory for the first link, wherein the first forward propagation weights stored in the first dedicated memory are configured to be updated using one of the at least one first candidate weight after the multiplication operation for the first link corresponding to the first dedicated memory.

[0008] According to some embodiments, this disclosure relates to an operation method of a memory device, the method comprising: performing a first operation on a first link included in a neural network based on a first forward propagation weight among a plurality of weights stored in a first dedicated memory; after the first operation, updating the weights other than the first forward propagation weight among the plurality of weights stored in the first dedicated memory to the first forward propagation weight; performing a second operation on a second link included in the neural network based on a second forward propagation weight among a plurality of weights stored in a second dedicated memory and the result of the first operation; after the second operation, updating the weights other than the second forward propagation weight among the plurality of weights stored in the second dedicated memory to the second forward propagation weight; generating training weights for the second link based on the result of the second operation; storing the training weights for the second link as second candidate weights in the second dedicated memory; and generating training weights for a first link based on the training weights of the second link; storing the training weights of the first link as first candidate weights in the first dedicated memory.

[0009] According to some embodiments, this disclosure relates to a memory device configured to perform neural network learning, the memory device including a memory and a processor, the memory including a plurality of dedicated memory regions corresponding respectively to a plurality of links included in the neural network, the processor being configured to perform operations required for training the neural network, wherein the plurality of dedicated memory regions are configured to store a plurality of weights for the corresponding links respectively, the plurality of dedicated memory regions are logically separated from each other, and the processor is further configured to perform operations based on one of the plurality of weights stored respectively in the plurality of dedicated memory regions, and to remove weights used in the operations from the plurality of dedicated memory regions. Attached Figure Description

[0010] The exemplary embodiments will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings.

[0011] Figure 1This is a diagram illustrating an example of a system including a memory device that performs internal processing operations according to some embodiments.

[0012] Figure 2 This is a block diagram illustrating an example of a memory device according to some implementations.

[0013] Figure 3 This is a block diagram used to illustrate an example of a neural network according to some implementations.

[0014] Figure 4 This is a diagram used to illustrate examples of neural networks according to some implementations and the weights corresponding to the links included in the neural network.

[0015] Figures 5A to 5H This is a diagram used to illustrate example operations of a neural network according to some implementations.

[0016] Figure 6 This is a block diagram used to illustrate an example of an electronic device according to some implementations.

[0017] Figure 7 This is a block diagram used to illustrate an example of an electronic device according to some implementations.

[0018] Figure 8 This is a flowchart illustrating an example of an operation method for a memory device according to some implementation methods.

[0019] Figure 9 This is a block diagram illustrating an example of a system for describing an electronic device including a memory device, according to some implementation methods. Detailed Implementation

[0020] In the following text, exemplary embodiments will be explained in detail with reference to the accompanying drawings.

[0021] The processing element (PE) described herein may include processing units such as neural processing units (NPU), central processing units (CPU), digital signal processors (DSP), graphics processing units (GPU), encryption processing units, physical processing units, machine learning processing units, or logic circuits.

[0022] The PE (Programmable Execution Environment) may include various operation layers that perform processing in memory (PIM) operations. The first (e.g., lowest) level of the operation layer may include bit vector operations (e.g., basic logic operations, which may be referred to as "primitive" operations). The second (e.g., middle) level of the operation layer may include composite operations comprising multiple bit vector operations. For example, composite operations may include mathematical operations such as addition, multiplication, etc., which may include multiple logical AND, OR, XOR, shift, etc. The third (e.g., top) level of the operation layer may include control flow operations (e.g., loops, branches, etc.) associated with program execution involving processing performed using the PE 510. However, the above description is intended to aid in understanding the PE, and the PE conceived according to the present invention is not limited thereto.

[0023] Figure 1 This is a diagram illustrating an example of a system including a memory device performing internal processing operations according to some embodiments. Figure 1 In this context, system 100 can be configured to perform applications for high-performance computing, graphics manipulation, etc., or applications for learning systems (such as deep neural networks). These applications require high computational and memory capabilities to collaboratively execute jobs or tasks in parallel, train different datasets, and learn with high accuracy, where power efficiency and low latency are important.

[0024] System 100 may include host device 110 and memory device 120. Host device 110 may be used to solve an overall job or task through a parallel processing method, wherein the overall job or task is divided into smaller jobs executed in parallel by a large number of computing entities (e.g., processor 112, cores in processor 112, and processing in memory (PIM) circuitry 122).

[0025] Host device 110 can be communicatively connected to memory device 120 via bus 130. Host device 110 can be, for example, a computing system, such as a computer, laptop computer, server, workstation, portable communication terminal, personal digital assistant (PDA), portable multimedia player (PMP), smartphone, and wearable device. In some embodiments, host device 110 can be one of the components included in the computing system, such as a graphics card.

[0026] The host device 110 may include a processor 112 for performing general computer operations in system 100 and a memory controller 114 for managing data transmission / reception to / from memory device 120. The processor 112 is the main component of system 100; it processes and manages instructions and is primarily responsible for executing the operating system and applications. Furthermore, the processor 112 enables workloads to be distributed across multiple computing entities for parallel processing to solve complex jobs or tasks. The processor 112 may include processing units such as an NPU, CPU, DSP, GPU, cryptographic processing unit, physical processing unit, machine learning processing unit, etc.

[0027] Processor 112 can improve efficiency by distributing the execution of various computational tasks, instructions, or cores to other processors or offloading them to memory device 120. A core is defined as one or more instructions grouped together to perform a task or a definable subtask. PIM circuitry 122 of memory device 120 can perform computational processing via cores offloaded by processor 112. PIM circuitry 122 can perform various types of computational processing operations. For example, at least some neural network operations related to artificial intelligence can be executed by PIM circuitry 122. For example, processor 112 can control memory device 120 via memory controller 114 such that at least some neural network operations can be executed by PIM circuitry 122.

[0028] The memory controller 114 can access the memory device 120 according to a memory request from the host device 110. The memory controller 114 can control write or read operations on the memory device 120 by providing the memory device 120 with commands CMD and addresses ADDR. Furthermore, the memory controller 114 and the memory device 120 can send and receive data DQ to be written and data DQ to be read. Such memory access operations can be performed via the bus 130 between the memory controller 114 and the memory device 120.

[0029] Bus 130 can operate under the control of host device 110 (e.g., memory controller 114) according to PIM and / or High Bandwidth Memory (HBM) specifications. For simplicity, the command / address signal lines and data lines are illustrated as a single line between memory controller 114 and memory device 120, but in reality, the command / address signal lines and data lines can be multiple signal lines. Bus 130 can be implemented as a channel or multiple channels comprising multiple signal lines. Bus 130 can be referred to as a channel, and in some embodiments described below, bus 130 and channel can be used interchangeably.

[0030] The memory device 120 can write or read data under the control of the memory controller 114. For example, the memory device 120 can be a DDR synchronous dynamic random access memory (SDRAM) device. However, the invention is not limited thereto, and the memory device 120 can be any of the volatile memory devices, such as LPDDR SDRAM, wide I / O DRAM, high bandwidth memory (HBM), and hybrid memory cube (HMC). According to some embodiments, the memory device 120 can be any of the non-volatile memory devices, such as flash memory, phase-change RAM (PRAM), magnetic RAM (MRAM), resistive RAM (RRAM), and ferroelectric RAM (FRAM).

[0031] Memory device 120 may include a memory cell array (MCA) 121, PIM circuitry 122, and control logic circuitry 126. The MCA 121 may be grouped into memory banks and may include multiple word lines, multiple bit lines, and multiple memory cells formed at the intersections of the word lines and bit lines. Each memory cell may include a DRAM cell, which includes an access transistor and a storage capacitor. However, this disclosure is not limited thereto.

[0032] In some implementations, MCA 121 can store a dataset for training the neural network. Memory device 120 can perform neural network training based on the dataset stored in MCA 121. The dataset can be divided into multiple batches, and memory device 120 can train the neural network based on multiple batches. Therefore, memory device 120 can train the neural network multiple times based on a single dataset.

[0033] PIM circuitry 122 may include one or more processing elements 123 configured to perform kernel operations offloaded from host device 110. Host device 110 (e.g., memory controller 114) may initiate processing operations of PIM circuitry 122 by issuing PIM commands via bus 130. PIM circuitry 122 may be hardware with processing capabilities, similar to processor 112 included in host device 110.

[0034] In some implementations, the PIM circuit 122 may include a plurality of PEs 123. Each of the plurality of PEs 123 may perform operations in a neural network. For example, each of the plurality of PEs 123 may perform the following (see reference) Figure 4 The description describes the multiplication operations of weights and activations. In some implementations, the number of multiple PEs 123 may be equal to or greater than the number of links included in the neural network, but the inventive concept is not limited thereto. For example, the number of multiple PEs 123 may be greater than the number of links between two consecutive layers.

[0035] In some embodiments, the PIM circuit 122 may include a plurality of dedicated memories (DMs) 124. Each of the plurality of dedicated memories 124 may correspond to a plurality of links included in a neural network. In other words, the number of the plurality of dedicated memories 124 may be equal to the number of links included in the neural network. Hereinafter, it is described that the number of the plurality of dedicated memories 124 is equal to the number of links included in the neural network, but the number of the plurality of dedicated memories 124 according to this disclosure is not limited thereto. For example, the number of the plurality of dedicated memories 124 may be equal to the number of the plurality of PEs 123.

[0036] Multiple dedicated memories 124 may correspond to multiple links and store multiple weights for each corresponding link. In some embodiments, the multiple weights may include forward propagation weights, candidate weights, and backpropagation weights. Forward propagation weights refer to the weights used in the forward propagation for training or outputting the results of the neural network; candidate weights refer to weights that can be updated as forward propagation weights or backpropagation weights; and backpropagation weights may refer to the weights used in the backpropagation for training the neural network. The following references... Figure 5A The accompanying figure provides a more detailed description of the forward propagation weights, at least one candidate weight, and the backpropagation weights.

[0037] In some implementations, the memory device 120 can accelerate the training of the neural network by including multiple dedicated memories, each corresponding to a multiple link. For example, the multiple dedicated memories can each include multiple weights for a corresponding link. Based on the multiple weights, the memory device 120 can perform forward propagation for other batches even before updating the multiple weights according to training, thereby reducing the time elapsed for training the neural network.

[0038] The control logic circuit 126 can control the overall operation of the memory device 120. The control logic circuit 126 can generate control signals based on the command CMD received from the memory controller 114 to perform write operations, read operations, refresh operations and / or internal processing operations of the memory device 120.

[0039] In some implementations, the control logic circuit 126 may update at least one of the candidate weights to a forward propagation weight after the forward propagation and / or backward propagation operations, depending on the operation of the neural network. For example, the control logic circuit 126 may update the forward propagation weights by changing the address of the location storing the forward propagation weights to the address of the location storing the candidate weights. However, this disclosure is not limited thereto, and the control logic circuit 126 may also update the forward propagation weights by copying and writing the candidate weights.

[0040] Although it has been referenced Figure 1The description describes the execution of operations in a neural network according to some embodiments via PIM circuit 122, but the memory device 120 is not limited thereto. For example, see the following references. Figure 7 As mentioned above, the operations in the neural network can be executed by the processor 112.

[0041] Figure 2 This is a block diagram illustrating an example of a memory device according to some implementations. Figure 2 In the above reference, PIM circuit 122, multiple PEs 123_1 to 123_9, and multiple dedicated memories 124_1 to 124_9 correspond to the above reference. Figure 1 The PIM circuit, PE, and dedicated memory are described, and descriptions identical to those given above are omitted.

[0042] Reference Figure 2 The PIM circuit 122 may include multiple PEs 123_1 to 123_9, multiple dedicated memories 124_1 to 124_9, and data input / output circuit 125.

[0043] In some implementations, multiple PEs 123_1 to 123_9 can each perform operations for training and / or outputting the results of the neural network. The multiple PEs 123_1 to 123_9 can each perform some of the operations in the neural network, and the remaining operations in the neural network can be performed by a processor (…). Figure 1 112) is executed based on the results of operations performed by multiple PEs 123_1 to 123_9. For example, each of the multiple PEs 123_1 to 123_9 can independently perform multiplication operations between activations and forward propagation weights in the neural network, and the remaining operations in the neural network (e.g., operations for summing the results of multiplication operations, operations for applying biases, etc.) can be performed by the processor ( Figure 1 112) Execution.

[0044] Multiple dedicated memories 124_1 to 124_9 may each store forward propagation weights, at least one candidate weight, and backpropagation weights corresponding to each of the multiple links included in the neural network. When forward propagation and / or backpropagation are performed, the weights stored in the multiple dedicated memories 124_1 to 124_9 may be updated. For example, forward propagation weights stored in the multiple dedicated memories 124_1 to 124_9 may be used for forward propagation of the neural network. The used forward propagation weights may be deleted from the dedicated memory (e.g., 124_1), and at least one of the candidate weights may be determined as a new forward propagation weight (or updated as a forward propagation weight). Similarly, backpropagation weights may be used for backpropagation in the neural network. However, unlike forward propagation weights, backpropagation weights used in backpropagation may be updated as candidate weights without being deleted, and may later be updated as forward propagation weights. Weights generated based on backpropagation used to train the neural network (hereinafter referred to as training weights) may be stored as candidate weights in the dedicated memory (e.g., 124_1).

[0045] PIM circuit 122 can receive data stored in MCA through data input / output circuit 125. Figure 1 The data input / output circuit 125 can divide the received dataset into multiple batches and perform neural network training based on these multiple batches, as described above. However, this disclosure is not limited thereto.

[0046] In some implementations, multiple dedicated memories 124_1 to 124_9 configured to store multiple weights can be associated with an MCA (Multi-Channel Association) configured to store a dataset for training. Figure 1 121) is logically and physically separated. In addition, multiple dedicated memories 124_1 to 124_9 can be logically and physically separated from each other, and multiple dedicated memories 124_1 to 124_9 can not share PE with each other.

[0047] For ease of illustration, each of the plurality of PEs 123_1 to 123_9 is shown to include a dedicated memory, but this disclosure is not limited thereto. For example, the plurality of dedicated memories 124_1 to 124_9 may be logically allocated to the plurality of PEs 123_1 to 123_9 respectively, and may not be physically adjacent or directly connected.

[0048] also, Figure 2 The number of multiple PEs 123_1 to 123_9 and multiple dedicated memories 124_1 to 124_9 shown in the figure are merely examples, and this disclosure is not limited thereto.

[0049] Figure 3 This is a block diagram used to illustrate examples of neural networks according to some implementations. Figure 3 In this document, the neural network (NN) is intended to help understand the neural network structure (or neural network model structure) according to the present disclosure, but the present disclosure is not limited thereto.

[0050] exist Figure 3 In a neural network (NN), the structure can include an input layer, hidden layers, and an output layer. The NN can perform operations based on the received input data I1 and I2, and generate output data O1 and O2 based on the results of the operations. For example, as mentioned above, the dataset can be divided into multiple batches, and the NN can use one batch as input data I1 and I2 to generate output data O1 and O2.

[0051] The neural network NN can be a deep neural network (DNN) or an n-layer neural network including two or more hidden layers. For example, the neural network NN can be a DNN including an input layer 10, a first hidden layer 12, a second hidden layer 14, and an output layer 16. Multiple layers can be implemented as convolutional layers, fully connected layers, softmax layers, etc. For example, a convolutional layer can include convolution operations, pooling operations, activation function operations, etc. In some embodiments, convolution operations, pooling operations, and activation function operations can each constitute a layer. However, the neural network model according to this disclosure is not limited thereto.

[0052] The outputs of multiple layers (i.e., input layer 10, first hidden layer 12, second hidden layer 14, and output layer 16) can be referred to as features (or feature maps). These multiple layers (i.e., input layer 10, first hidden layer 12, second hidden layer 14, and output layer 16) can receive features generated by previous layers as input features and generate output features or output signals by manipulating the input features. Features can refer to data that expresses various characteristics of the input data that can be recognized by the neural network (NN).

[0053] When a neural network (NN) has a DNN structure, it can include more layers from which useful information can be extracted, and it can handle complex datasets. While the NN shown here comprises four layers—input layer 10, first hidden layer 12, second hidden layer 14, and output layer 16—this is merely an example, and a NN can include fewer or more layers. Furthermore, a NN can include layers with various structures beyond those shown in Figure 5.

[0054] The neural network NN comprises multiple layers, namely, the input layer 10, the first hidden layer 12, the second hidden layer 14, and the output layer 16, each of which may also include multiple neurons. Multiple neurons can correspond to multiple artificial nodes referred to as units or similar terms. For example, as... Figure 3 As shown, the input layer 10 may include two neurons (nodes), and the first hidden layer 12 and the second hidden layer 14 may each include three neurons (nodes). However, this is just an example, and the layers included in a neural network NN may include a variety of numbers of neurons (nodes).

[0055] Neurons in multiple layers of a neural network NN (i.e., input layer 10, first hidden layer 12, second hidden layer 14, and output layer 16) can connect to each other and exchange data. Neurons can receive data from other neurons, perform operations on the data, and output the results of the operations to other neurons.

[0056] The inputs and outputs of each neuron (nodes, such as N1, N2, and N3) can be referred to as input activation and output activation. In other words, activation can be a parameter corresponding to both the neuron's output and the inputs of neurons in the next layer. Simultaneously, each neuron can be activated based on the outputs received from neurons included in previous layers (e.g., (etc.), weights (e.g., (etc.) and bias (e.g., The loss function determines the output activation of each neuron. Weights and biases are parameters (often called weight parameters) used to calculate the output activation from each neuron, where weights refer to the values ​​assigned to the connections between neurons, and biases represent the coefficients associated with individual neurons. A neural network (NN) can determine parameters such as weights and biases such that the loss value generated by the loss function is minimized.

[0057] A neural network (NN) can include multiple links. Each of these links can represent elements contained in a previous layer (e.g., ...). The activation and weights of neuron outputs in (e.g., Multiplication operations between () are performed. Therefore, the multiplication operations corresponding to the link can be obtained from the above reference. Figure 2 The described PE (e.g., 123_1) is executed. At this time, the weights used in the multiplication operation (e.g., forward propagation weights) can be weights stored in dedicated memory 124_1.

[0058] Figure 4 This is a diagram used to illustrate examples of neural networks according to some implementations and the weights corresponding to the links included in the neural network. See also... Figure 3 describe Figure 4And the same descriptions as those given above can be omitted.

[0059] exist Figure 4 In this neural network, an input layer 40, a first layer 41, a second layer 42, and an output layer 43 may be included. The first layer 41 and the second layer 42 may be hidden layers as described above. Figure 4 For ease of explanation, each of the input layer 40, the first layer 41, the second layer 42, and the output layer 43 is shown to include one neuron, and a link is shown between two consecutive layers. This is merely for ease of explanation, and the neural network according to this disclosure may include more hidden layers, and multiple neurons may be included in each of the multiple layers, and multiple links may exist between two consecutive layers.

[0060] The neural network may include a first layer 41 and a second layer 42, and may include a first link 44 between the first layer 41 and the second layer 42. Similarly, the neural network may include a second layer 42 and an output layer 43, and may include a second link 45 between the second layer 42 and the output layer 43.

[0061] First dedicated memory (e.g., Figure 2 The 124_1) can store multiple weights W1_1 to W1_4 corresponding to the first link 44, i.e., the first weight table WT1. Similarly, the second dedicated memory (e.g., Figure 2 124_2) can store multiple weights W2_1 to W2_4 corresponding to the second link 45, i.e., the second weight table WT2. The number of multiple weights stored in the dedicated memory according to this disclosure can be determined based on the number of multiple hidden layers included in the neural network. For example, the number of multiple weights stored in each of the multiple dedicated memories can be twice or more the number of multiple layers included in the neural network. Figure 4 Since there are two hidden layers, the number of weights stored in each of the multiple dedicated memories can be four or more.

[0062] exist Figure 4In the first weight table WT1, multiple gradients G1_1 to G1_4 may be included along with multiple weights W1_1 to W1_4. The multiple gradients G1_1 to G1_4 may each correspond to multiple weights W1_1 to W1_4. The multiple gradients G1_1 to G1_4 can be determined based on the multiple weights W1_1 to W1_4. For example, a memory device according to the present invention can determine the multiple gradients G1_1 to G1_4 as the reciprocals of the multiple weights W1_1 to W1_4. Similarly, the second weight table WT2 may include multiple gradients G2_1 to G2_4 along with multiple weights W2_1 to W2_4, and the reciprocals of the multiple weights W2_1 to W2_4 can be determined as the multiple gradients G2_1 to G2_4. The weights included in the weight table can be used for the forward propagation of the neural network, and the gradients included in the weight table can be used for the backward propagation of the neural network.

[0063] However, in some implementations, the weight table is not limited to Figure 4 The weight table shown is an example (e.g., WT1). For example, the memory device may store only the weights corresponding to the multiple links included in the neural network in dedicated memory, and may not store the multiple gradients corresponding to the multiple weights. The stored weights can be used when performing the forward propagation operation of the neural network, and the inverses of the stored weights can be used when performing the backward propagation operation of the neural network.

[0064] Figures 5A to 5H This is a diagram used to illustrate an example of the operation of a neural network according to some implementations. See also... Figure 3 and Figure 4 describe Figures 5A to 5H And the description that is the same as the one given above will be omitted. Figures 5A to 5H The input layer 50, first layer 51, second layer 52, output layer 53, first link 54, and second link 55 shown correspond to respectively Figure 4 The input layer 40, first layer 41, second layer 42, output layer 43, first link 44, and second link 45 are shown respectively, and descriptions identical to those given above will be omitted. It can be understood that the first weight table WT1 is stored logically assigned to the first PE (e.g., Figure 2 The first dedicated memory of 123_1) (e.g., Figure 2 In 124_1), and the second weight table WT2 is stored in the logically assigned second PE (e.g., Figure 2 The second dedicated memory of 123_2) (e.g., Figure 2 In section 124_2). See below for reference. Figures 5A to 5H Describes the neural network operation of a memory device according to this disclosure. Figures 5A to 5HIn the following, it is assumed that the dataset used to train the neural network is divided into six batches, but the number of batches used to train the neural network according to this disclosure may be more or less than six.

[0065] exist Figure 5A During the first time period Time 1, the memory device can perform operations for forward propagation. During Time 1, the memory device can perform forward propagation operations for the neural network on the first batch #1 based on the first forward propagation weight W1_1. For example, the memory device (or the first PE) can perform a multiplication operation between the activation of the first batch #1 output from the first layer 51 and the first forward propagation weight W1_1. Simultaneously, during Time 1, the memory device cannot perform operations corresponding to the second link 55 because there is no output from the second layer 52. After performing forward propagation on the first batch #1, the first forward propagation weight W1_1 can be updated to one of a plurality of first candidate weights W1_2 to W1_4. For example, the first forward propagation weight W1_1 can be deleted from the first dedicated memory, and the first forward propagation weight can be updated with the first candidate weight W1_2.

[0066] exist Figure 5A In this process, the order of the multiple first candidate weights W1_2 to W1_4 stored in the first dedicated memory can be changed. For example, the order of the first candidate weights W1_2 can be changed from second to first, and thus the first forward propagation weights can be updated with the first candidate weights W1_2. As described above, since multiple gradients can be determined based on the corresponding weights, the order of gradients can be changed similarly after the forward propagation operation of the neural network. In other words, weights and gradients corresponding to weights form pairs, and the order of corresponding weights and gradients in the weight table can be the same.

[0067] Figure 5A The initial weights stored in the first weight table WT1 and the initial weights stored in the second weight table WT2 can be identical. For example, before performing the forward propagation operation for the neural network, all weights W1_1 to W1_4 stored in the first weight table WT1 can have the same value. Similarly, before performing the forward propagation operation for the neural network, all weights W2_1 to W2_4 stored in the second weight table WT2 can have the same value. As described above, since multiple gradients can be determined based on the corresponding weights, all gradients stored in the weight tables before the forward propagation operation for the neural network can have the same value.

[0068] exist Figure 5BDuring the second time period (Time2), the memory device can perform operations for forward propagation. During Time2, the memory device (or the first PE) can perform forward propagation operations for the neural network on the second batch (Batch#2) based on the first forward propagation weight W1_2. For example, the memory device can perform a multiplication operation between the activations of the second batch (Batch#2) output from the first layer 51 and the first forward propagation weight W1_2. Furthermore, during the second time period (Time2), the memory device (or the second PE) can perform forward propagation operations for the neural network on the first batch (Batch#1) based on the second forward propagation weight W2_1. For example, the memory device can perform a multiplication operation between the activations of the first batch (Batch#1) output from the second layer 52 and the second forward propagation weight W2_1. The above-mentioned operations can be performed simultaneously. Figure 5B The operations performed by the first PE and the operations performed by the second PE are described. This can be understood through the above reference. Figure 5A The given description is used to understand the updated weights, so its detailed description is omitted.

[0069] In some implementations, the memory device can perform forward propagation based on the training of the neural network, using multiple dedicated memories configured to store multiple weights, without waiting for the weights to be updated. (See above reference...) Figure 5B The memory device can simultaneously perform operations based on the first forward propagation weights W1_2 for the second batch (Batch #2) and operations based on the second forward propagation weights W2_1 for the first batch (Batch #1). Therefore, the training time for the neural network can be reduced.

[0070] exist Figure 5B and Figure 5C In the process, after the operation corresponding to the second link 55, a second candidate weight W2_5 can be generated based on the output data of the output layer 53 as the result of training the first batch Batch#1, and the second candidate weight W2_5 can be added to the second weight table WT2. As described above, the gradient G2_5 corresponding to the second candidate weight W2_5 can be added to the second weight table WT2 together with the second candidate weight W2_5 and used for the backpropagation operation. However, this disclosure is not limited to this, and the gradient G2_5 can be calculated based on the second candidate weight W2_5 during the backpropagation operation and then used for the backpropagation operation.

[0071] exist Figure 5CDuring the third time period (Time3), the memory device can perform operations for forward propagation. During Time3, the memory device (or the first PE) can perform forward propagation operations for the neural network on the third batch (Batch#3) based on the first forward propagation weights W1_3. For example, the memory device can perform a multiplication operation between the activations of the third batch (Batch#3) output from the first layer (51) and the first forward propagation weights W1_3. Furthermore, during Time3, the memory device (or the second PE) can perform forward propagation operations for the neural network on the second batch (Batch#2) based on the second forward propagation weights W2_2. For example, the memory device can perform a multiplication operation between the activations of the second batch (Batch#2) output from the second layer (52) and the second forward propagation weights W2_2. Additionally, during Time3, the memory device (or the second PE) can perform backpropagation operations based on the second candidate weights W2_5, which are the results of training the first batch (Batch#1). The second candidate weight W2_5 can be called the training weight because it is based on the results of training the first batch Batch#1 as described above.

[0072] exist Figure 5C In this context, the second candidate weight W2_5 can be referred to as the backpropagation weight. The operation used for backpropagation can mean changing the weights corresponding to the links (to minimize the error in the neural network's output), through which the neural network can be trained. The order of the weights included in the weight table can be changed after the operation used for forward propagation. The memory device can update the second candidate weight W2_5 to the candidate weights after performing the backpropagation operation based on the second candidate weight W2_5.

[0073] As mentioned above, refer to Figure 5C The memory devices can simultaneously perform operations for the third batch (Batch #3) based on the first forward propagation weights W1_3, and operations for the second batch (Batch #2) based on the second forward propagation weights W2_2. Therefore, the training time for the neural network can be reduced.

[0074] You can refer to the above. Figures 5A to 5C To understand the given description Figures 5D to 5H Therefore, descriptions identical to those already given above will be omitted.

[0075] exist Figure 5DDuring the fourth time period (Time 4), the memory device can perform operations for forward propagation. During Time 4, the memory device (or the first PE) can perform forward propagation operations for the neural network on the fourth batch (Batch #4) based on the first forward propagation weight W1_4. For example, the memory device can perform a multiplication operation between the activations of the fourth batch (Batch #4) output from the first layer 51 and the first forward propagation weight W1_4. Furthermore, during the fourth time period (Time 4), the memory device (or the second PE) can perform forward propagation operations for the neural network on the third batch (Batch #3) based on the second forward propagation weight W2_3. For example, the memory device can perform a multiplication operation between the activations of the third batch (Batch #3) output from the second layer 52 and the second forward propagation weight W2_3. Furthermore, during the fourth time period (Time4), the memory device (e.g., the first PE) can perform backpropagation based on the first candidate weights W1_5, which are the results of training the first batch (Batch#1), and the memory device (e.g., the second PE) can perform backpropagation based on the second candidate weights W2_6, which are the results of training the second batch (Batch#2).

[0076] exist Figure 5EDuring the fifth time period (Time 5), the memory device can perform operations for forward propagation. During Time 5, the memory device (or the first PE) can perform forward propagation operations for the neural network on the fifth batch (Batch #5) based on the first forward propagation weight W1_5. For example, the memory device can perform a multiplication operation between the activations of the fifth batch (Batch #5) output from the first layer 51 and the first forward propagation weight W1_5. As mentioned above, the first forward propagation weight W1_5 is a weight based on the result of training the neural network on the first batch (Batch #1). Furthermore, during the fifth time period (Time 5), the memory device (or the second PE) can perform forward propagation operations for the neural network on the fourth batch (Batch #4) based on the second forward propagation weight W2_4. For example, the memory device can perform a multiplication operation between the activations of the fourth batch (Batch #4) output from the second layer 52 and the second forward propagation weight W2_4. Furthermore, during the fifth time period (Time 5), the memory device (e.g., the first PE) can perform backpropagation based on the first candidate weight W1_6, which is the result of training on the second batch (Batch #2), and the memory device (e.g., the second PE) can perform backpropagation based on the second candidate weight W2_7, which is the result of training on the third batch (Batch #3). The first candidate weight W1_7 can be generated through backpropagation performed based on the second candidate weight W2_7.

[0077] exist Figure 5FDuring the sixth time period (Time 6), the memory device can perform operations for forward propagation. During Time 6, the memory device (or the first PE) can perform forward propagation operations for the neural network on the sixth batch (Batch #6) based on the first forward propagation weights W1_6. For example, the memory device can perform a multiplication operation between the activations of the sixth batch (Batch #6) output from the first layer 51 and the first forward propagation weights W1_6. It is understood that, as described above, the first forward propagation weights W1_6 are weights based on the results of training the neural network on the second batch (Batch #2). Furthermore, during the sixth time period (Time 6), the memory device (or the second PE) can perform forward propagation operations for the neural network on the fifth batch (Batch #5) based on the second forward propagation weights W2_5. For example, the memory device can perform a multiplication operation between the activations of the fifth batch (Batch #5) output from the second layer 52 and the second forward propagation weights W2_5. Furthermore, during the sixth time period (Time 6), the memory device (e.g., the first PE) can perform backpropagation based on the first candidate weights W1_7, which are the result of training on the third batch (Batch #3), and the memory device (e.g., the second PE) can perform backpropagation based on the second candidate weights W2_8 (i.e., determined as backpropagation weights), which are the result of training on the fourth batch (Batch #4). The first candidate weights W1_8 can be generated through backpropagation performed based on the second candidate weights W2_8.

[0078] exist Figure 5G During the seventh time period (Time 7), the memory device can perform operations for forward propagation. During Time 7, the memory device (or the second PE) can perform forward propagation operations for the neural network on the sixth batch (Batch #6) based on the second forward propagation weights W2_6. For example, the memory device can perform a multiplication operation between the activations of the sixth batch (Batch #6) output from the second layer 52 and the second forward propagation weights W2_6. Furthermore, during Time 7, the memory device (e.g., the first PE) can perform backpropagation operations based on the first candidate weights W1_8, which are the result of training on the fourth batch (Batch #4), and the memory device (e.g., the second PE) can perform backpropagation operations based on the second candidate weights W2_9 (i.e., determined as backpropagation weights), which are the result of training on the fifth batch (Batch #5). The first candidate weights W1_9 can be generated through backpropagation performed based on the second candidate weights W2_9.

[0079] exist Figure 5H During the eighth time period (Time 8), a memory device (e.g., a first PE) can perform backpropagation operations based on a first candidate weight W1_9, which is the result of training on the fifth batch (Batch #5), and a memory device (e.g., a second PE) can perform backpropagation operations based on a second candidate weight W2_10 (i.e., determined as the backpropagation weight), which is the result of training on the sixth batch (Batch #6). The first candidate weight W1_10 can be generated through backpropagation performed based on the second candidate weight W2_10.

[0080] Figure 6 This is a block diagram used to illustrate examples of memory devices according to some implementations. Figure 6 In this context, the first storage device 61, the second storage device 62, and the third storage device 63 can be electrically connected to each other. Since the first storage device 61, the second storage device 62, and the third storage device 63 each correspond to the above-mentioned reference... Figure 1 The memory device 120 described will therefore omit the same description as that already given above.

[0081] exist Figure 6 In the first memory device 61, the second memory device 62 and the third memory device 63 may include memories 64, 66 and 68 configured to store datasets for training neural networks and PIM circuits 65, 67 and 69 configured to perform operations for forward propagation and backward propagation of neural networks, respectively.

[0082] Neural networks can include multiple layers to generate accurate output data about input data. As the number of layers increases, the accuracy of the output data can improve. However, because the number of links included in the neural network may increase, the operation time may also increase. As described above, the memory device according to this disclosure ( Figure 1 The 120) may include multiple dedicated memories (e.g., corresponding to multiple links included in the neural network) Figure 2 (124_1 to 124_9). As described above, according to this disclosure, multiple dedicated memories (e.g., Figure 2 (124_1 to 124_9) can store multiple weights for the corresponding links respectively.

[0083] However, the number of links included in the neural network can be greater than the number of dedicated memories included in memory devices 61, 62, and 63. Furthermore, as mentioned above, the multiple dedicated memories can be physically and logically separated from the memory used to store the dataset, and the multiple dedicated memories can be physically and logically separated from each other. Therefore, as... Figure 6 As shown, multiple memory devices 61, 62 and 63 can be connected to each other to perform forward and backward propagation operations for the neural network described above.

[0084] After training is complete, the memory device can determine combined (e.g., final) weights based on multiple weights stored in dedicated memory. The memory device can determine the combined (e.g., final) weights by combining multiple weights. For example, the memory device can determine the combined (e.g., final) weights by averaging multiple weights. The memory device can then generate the output of the neural network using the combined (e.g., final) weights.

[0085] Figure 7 This is a block diagram used to illustrate examples of electronic devices according to some embodiments. Figure 7 In this context, the electronic device 70 may include a processor 112a and a memory 71. The processor 112a may correspond to the one described above. Figure 1 The processor 112 is described, and the same descriptions as those already given above will be omitted.

[0086] The memory 71 may include multiple memory regions. These multiple memory regions included in the memory 71 may be logically separated from each other. The memory 71 according to the present invention may include a data memory region 72 configured to store a dataset for training a neural network and multiple dedicated memory regions 73 to 76 configured to store multiple weights corresponding to links in the neural network.

[0087] Although it has been referenced Figures 1 to 6 Operations for forward and / or backward propagation of neural networks based on PE and dedicated memory are described, but this disclosure is not limited thereto. For example, such as Figure 7 As shown, the processor 112a included in the electronic device 70 can be used as the aforementioned PE. Furthermore, the multiple dedicated memory (DM) regions 73 to 76, which are logically separate from each other, can each correspond to a multiple dedicated memory.

[0088] Figure 8 This is a flowchart illustrating an example of a method for operating a memory device according to some implementation schemes. Figure 8 In operation S100, the memory device can perform a first operation on a first link included in a neural network based on a first forward propagation weight among a plurality of weights stored in a first dedicated memory.

[0089] In operation S200, after the first operation, the memory device can update the weights other than the first forward propagation weight among the multiple weights stored in the first dedicated memory to the first forward propagation weight. Here, the memory device can delete the first forward propagation weight used in the first operation from the first dedicated memory.

[0090] In operation S300, the memory device can perform a second operation on a second link included in the neural network based on a second forward propagation weight among a plurality of weights stored in a second dedicated memory and the result of the first operation.

[0091] In operation S400, after the second operation, the memory device can update the weights stored in the second dedicated memory, excluding the second forward propagation weights, to the second forward propagation weights. Here, the first dedicated memory and the second dedicated memory can be physically and logically separated from each other. Furthermore, as described above, the neural network includes N hidden layers (N is an integer greater than or equal to 1), and the number of weights stored in each of the first and second dedicated memories can be 2X or greater. Additionally, the memory device can delete the second forward propagation weights used in the second operation from the second dedicated memory.

[0092] In operation S500, the memory device can generate training weights for the second link based on the result of the second operation, and store the training weights for the second link as second candidate weights in the second dedicated memory.

[0093] In operation S600, the memory device can generate training weights for the first link based on the training weights for the second link, and store the training weights for the first link as first candidate weights in the first dedicated memory.

[0094] The dataset used to train the neural network includes a first batch and a second batch that is consecutive to the first batch, and the neural network can learn from the first batch and then learn from the second batch, wherein the second operation on the first batch and the first operation on the second batch can be performed together.

[0095] After training the neural network is complete, the combined (e.g., final) weights of each of the first and second links can be determined based on multiple weights stored in each of the first and second dedicated memories.

[0096] Figure 9 This is a block diagram illustrating an example of a system for describing an electronic device including a memory device, according to some embodiments. Figure 9In this system, system 3000 may include camera 3100, display 3200, audio processor 3300, modem 3400, DRAM 3500a and 3500b, flash memory 3600a and 3600b, I / O devices 3700a and 3700b, and application processor (AP) 3800. System 3000 can be implemented as a laptop computer, mobile phone, smartphone, tablet PC, wearable device, healthcare device, or IoT device. Furthermore, system 3000 can be implemented as a server or PC.

[0097] Camera 3100 can capture still images or videos according to user control, and can store the captured image / video data or send the captured image / video data to display 3200. Audio processor 3300 can process audio data or network content included in flash memory 3600a and 3600b. Modem 3400 can send modulated signals for wired / wireless data transmission / reception to receiver, and the modulated signals can be demodulated by receiver to recover the original signal. I / O devices 3700a and 3700b may include devices that provide digital input and / or digital output functions, such as Universal Serial Bus (USB), storage devices, digital cameras, Secure Digital (SD) cards, Digital Universal Discs (DVDs), network adapters, touch screens, etc.

[0098] AP 3800 can control the overall operation of system 3000. AP 3800 may include controller 3810, accelerator or accelerator chip 3820, and interface 3830. AP 3800 can control display 3200, causing a portion of the contents stored in flash memories 3600a and 3600b to be displayed on display 3200. When user input is received through I / O devices 3700a and 3700b, AP 3800 can perform control operations corresponding to the user input. AP 3800 may include an accelerator block, which is circuitry dedicated to computing artificial intelligence (AI) data, or may include an accelerator chip 3820 separate from AP 3800. DRAM 3500b may be additionally provided in the accelerator block or accelerator chip 3820. Accelerator blocks are function blocks that specifically perform specific functions of the AP 3800 and may include graphics processing units (GPUs) (which are function blocks that specifically process graphics data), neural processing units (NPUs) (which are blocks that are specifically used for AI computing and inference), and data processing units (DPUs) (which are blocks that are specifically used for data transmission).

[0099] System 3000 may include multiple DRAMs 3500a and 3500b. AP 3800 can establish a DRAM interface protocol and communicate with DRAMs 3500a and 3500b to control them via commands compliant with JEDEC standards and Mode Register (MRS) settings, or to utilize company-specific features such as low-voltage / high-speed / reliability and Cyclic Redundancy Check (CRC) / Error Correction Code (ECC) functions. For example, AP 3800 can communicate with DRAM 3500a via JEDEC-compliant interfaces such as LPDDR4 and LPDDR5, and accelerator block or accelerator chip 3820 can be configured and use a new DRAM interface protocol to control DRAM 3500b used for the accelerator, which has a larger bandwidth than DRAM 3500a.

[0100] although Figure 9 Only DRAMs 3500a and 3500b are shown, but this disclosure is not limited thereto. Any memory such as PRAM, SRAM, MRAM, RRAM, FRAM, or hybrid RAM can be used, provided the bandwidth, response speed, and voltage conditions of the AP 3800 or accelerator chip 3820 are met. DRAMs 3500a and 3500b have relatively smaller latency and bandwidth than I / O devices 3700a and 3700b or flash memories 3600a and 3600b. When the system 3000 is powered on and OS and application data are loaded onto it, DRAMs 3500a and 3500b are initialized, and DRAMs 2500a and 2500b can be used as temporary storage devices for OS and application data, or as execution space for various software codes.

[0101] In DRAMs 3500a and 3500b, four arithmetic operations (i.e., addition, subtraction, multiplication, and division), vector computation, address computation, or Fast Fourier Transform (FFT) computation can be performed. Furthermore, in DRAMs 3500a and 3500b, functions for inference operations can be performed. Here, inference can be performed in deep learning algorithms using artificial neural networks. Deep learning algorithms can include training operations for learning a model from various data and inference operations for recognizing data using the trained model. According to some implementations, images captured by a user via camera 3100 are signal processed and stored in DRAM 3500b, and the accelerator block or accelerator chip 3820 can use the data stored in DRAM 3500b and the functions for inference to perform AI data computation for recognizing data.

[0102] System 3000 may include multiple storage devices or flash memories 3600a and 3600b having a larger capacity than DRAM 3500a and 3500b. Accelerator block or accelerator chip 3820 may use flash memories 3600a and 3600b to perform training operations and AI data computations. According to some embodiments, flash memories 3600a and 3600b may include a memory controller 3610 and flash memory 3620, and training operations and inference AI data computations performed by AP 3800 and / or accelerator chip 3820 may be performed more efficiently by using an arithmetic unit included in memory controller 3610. Flash memories 3600a and 3600b may store images captured by camera 3100 or data transmitted via a data network. For example, flash memories 3600a and 3600b may store augmented reality / virtual reality content, high-definition (HD) content, or ultra-high-definition (UHD) content.

[0103] In System 3000, DRAM 3500a and 3500b may include the above reference. Figures 1 to 8 The memory device described.

[0104] While this disclosure contains numerous details of specific implementations, these should not be construed as limiting the scope of any claims that may be claimed, their equivalents, or the claims described later. Certain features described in this disclosure in the context of individual implementations may also be implemented in combination in a single implementation. Conversely, various features described in the context of individual implementations may also be implemented individually or in any suitable sub-combination in multiple implementations. Furthermore, although features may be described above as functioning in certain combinations, in some cases, one or more features from a combination may be removed from the combination, and the combination may involve sub-combinations or variations thereof.

Claims

1. A memory device configured to perform neural network learning, the memory device comprising: A first dedicated memory, corresponding to a first link included in the neural network, and configured to store a first plurality of weights, wherein the first plurality of weights includes a first forward propagation weight for the first link and at least one first candidate weight; and A first processing element PE is configured to perform a multiplication operation between the input and the first forward propagation weights stored in the first dedicated memory for the first link. The memory device is configured to update the first forward propagation weight stored in the first dedicated memory using one of the at least one first candidate weights after the multiplication operation for the first link corresponding to the first dedicated memory.

2. The memory device according to claim 1, further comprising a plurality of dedicated memories corresponding to a plurality of links included in the neural network, in, The multiple dedicated memories are physically and logically separated from each other.

3. The memory device of claim 1, further comprising a plurality of PEs configured to perform forward propagation in the neural network. in, The neural network includes two consecutive layers and multiple links between the two consecutive layers. The number of PEs is greater than the number of links between the two consecutive layers.

4. The memory device according to claim 1, in, The neural network includes N hidden layers, where N is an integer greater than or equal to 1, and... The number of the plurality of weights stored in the first dedicated memory is 2xN or greater.

5. The memory device according to claim 1, in, The neural network also includes a second link that shares neurons with the first link. The memory device further includes: A second dedicated memory is configured to store a second plurality of weights, wherein the second plurality of weights includes a second forward propagation weight for the second link and at least one second candidate weight; and The second PE is configured to perform a multiplication operation between the input and the second forward propagation weights stored in the second dedicated memory for the second link, and The second PE is configured as follows: By training the neural network, additional weights are generated from the at least one second candidate weight as second backpropagation weights, and Based on the second backpropagation weights, training weights for the first link are generated, wherein the first dedicated memory is configured to store the training weights.

6. The memory device according to claim 5, wherein, The memory device is configured to remove the first forward propagation weight from the first dedicated memory after the multiplication operation for the first link.

7. The memory device according to claim 6, wherein, The memory device is configured to determine the weights for the first link based on a combination of weights stored in the first dedicated memory and the second dedicated memory after the training of the neural network is completed.

8. The memory device according to claim 1, further comprising: Multiple dedicated memories, each corresponding to a different link included in the neural network; and Multiple PEs, each corresponding to one of the multiple dedicated memories.

9. The memory device of claim 1, further comprising a data memory configured to store a dataset for training the neural network. in, The first PE is physically and logically separated from the data storage.

10. The memory device according to claim 1, in, The first dedicated memory is configured to store the forward propagation weight gradient corresponding to the first forward propagation weight and the at least one candidate weight gradient corresponding to the at least one first candidate weight, respectively. and The first PE is configured to perform backpropagation based on the at least one candidate weight gradient.

11. A method of operating a memory device, the method comprising: Based on a first forward propagation weight among a plurality of weights stored in a first dedicated memory, a first operation is performed on a first link included in a neural network. After performing the first operation, the first forward propagation weight is updated using different weights among the plurality of weights stored in the first dedicated memory; A second operation is performed on a second link included in the neural network, based on a second forward propagation weight among a plurality of weights stored in a second dedicated memory and based on the result of the first operation. After performing the second operation, the second forward propagation weights are updated using different weights among the plurality of weights stored in the second dedicated memory; Based on the result of the second operation, training weights for the second link are generated, and the training weights for the second link are stored as second candidate weights in the second dedicated memory. and Training weights for the first link are generated based on the training weights for the second link, and the training weights for the first link are stored as first candidate weights in the first dedicated memory.

12. The method according to claim 11, wherein, The first dedicated memory and the second dedicated memory are physically and logically separated from each other.

13. The method according to claim 11, in, The neural network comprises N hidden layers, where N is an integer equal to or greater than 1, and... The number of weights stored in each of the first dedicated memory and the second dedicated memory is 2xN or greater.

14. The method according to claim 11, in, Updating the first forward propagation weights using different weights from the plurality of weights stored in the first dedicated memory includes deleting the first forward propagation weights used in the first operation from the first dedicated memory, and Updating the second forward propagation weights using different weights from the plurality of weights stored in the second dedicated memory includes deleting the second forward propagation weights used in the second operation from the second dedicated memory.

15. The method of claim 14, further comprising: After training the neural network is completed, weights for each of the first link and the second link are determined based on a combination of weights stored in each of the first dedicated memory and the second dedicated memory.

16. The method according to claim 11, in, The dataset used to train the neural network includes a first batch and a second batch. The neural network is configured to perform learning on the second batch after learning on the first batch, and The method includes performing a second operation for the first batch and a first operation for the second batch together.

17. A memory device configured to perform neural network learning, the memory device comprising: The memory includes multiple dedicated memory regions, each corresponding to a different link included in the neural network. and The processor is configured to perform operations to train the neural network. The multiple dedicated memory regions are configured to store multiple weights for the corresponding links, respectively. The plurality of dedicated memory regions are logically separated from each other, and The processor is further configured as follows: The operation is performed based on at least one of a plurality of weights stored respectively in the plurality of dedicated memory regions, and The weights used in the operation are removed from the plurality of dedicated memory regions.

18. The memory device of claim 17, wherein, The number of the plurality of dedicated memory regions is greater than or equal to the number of the plurality of links included in the neural network.

19. The memory device according to claim 17, in, The neural network includes N hidden layers, where N is an integer greater than or equal to 1, and... The number of weights stored in each of the plurality of dedicated memory regions is 2xN or greater.

20. The memory device of claim 17, wherein, The processor is further configured to, after training the neural network is completed, determine weights for multiple combinations corresponding to each of the multiple links based on combinations of the multiple weights stored respectively in the multiple dedicated memory regions.

Images