Apparatus and method to update weight for on-device learning of neural network model

Abstract

The present invention relates to a weight update method in a weight update device for on-device learning of a neural network model, wherein the weight matrix of the neural network model is divided into columns and mapped to each page of a flash memory for storage, weight columns requiring updating according to the learning of the neural network model are selected, the pages of the flash memory corresponding to the selected weight columns are copied to SRAM (Static Random Access Memory) to update the weights, and a rewrite operation is performed on the pages of the flash memory.

Description

Technology Field

[0001] The present invention relates to a weight update device and method for on-device learning of a neural network model. Background Technology

[0003] With the recent rise in interest in edge devices that perform AI neural network training locally rather than on a server, research on edge devices capable of on-device training is actively underway.

[0004] These edge devices typically utilize Micro Controller Units (MCUs) for on-device learning, and the memory types within MCUs—Static Random Access Memory (SRAM) and Flash Memory—each have the following advantages and disadvantages. Specifically, SRAM offers fast access speeds and enables rapid read / write operations, but it has the disadvantage of high cost and limited capacity. In contrast, Flash Memory is suitable for storing large amounts of data and retaining permanent data, but it has the disadvantage of significant limitations when performing write operations.

[0005] Meanwhile, for on-device learning, it is essential to update parameters stored in flash memory, such as weights; however, as previously explained, flash memory is subject to many constraints when performing write operations. In particular, erasing data from flash memory requires high voltage and incurs significant costs, and to write data, regardless of size, the corresponding page must first be erased before the data can be written. Additionally, while it is possible to perform partial weight updates to reduce flash memory access overhead, this leads to the problem of reduced inference accuracy in on-device learning.

[0006] Therefore, research is needed on a method to minimize the loss in inference accuracy of such on-device learning and update the weights required for training using only a minimum amount of memory. Prior art literature

[0008] Korean Registered Patent Publication No. 10-2505946 The problem to be solved

[0009] The present invention has been devised to solve the above-mentioned problems, and the objective of the present invention is to provide a weight update device and method for on-device learning of a neural network model. means of solving the problem

[0011] A weight update method in a weight update device for on-device learning of a neural network model according to an embodiment of the present invention for achieving the above objective comprises: a step of dividing a weight matrix of the neural network model into columns and mapping and storing it in each page of a flash memory; a step of selecting a weight column that requires updating according to the learning of the neural network model; and a step of copying the page of the flash memory corresponding to the selected weight column to a Static Random Access Memory (SRAM) to update the weight and performing a rewrite operation on the page of the flash memory.

[0012] A weight update device for on-device learning of a neural network model according to an embodiment of the present invention for achieving the above objective comprises: a flash memory that stores the weights of the neural network model in page units; a static random access memory (SRAM) that updates the weights by copying the weights stored in the flash memory; and a linker that divides the matrix of the weights into columns and maps them to each page of the flash memory. The weight update device selects a column of weights that requires updating according to the learning of the neural network model, and controls the update of only a portion of the weights by copying only the page of the flash memory corresponding to the selected column of weights to the SRAM. Effects of the invention

[0014] According to one aspect of the present invention described above, by providing a weight update device and method for on-device learning of a neural network model, it is possible to maintain the inference accuracy of on-device learning and minimize the use of relatively expensive SRAM. Brief explanation of the drawing

[0016] FIG. 1 is a diagram showing the internal structure of a weight update device for on-device learning according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of a mapping operation of a linker included in a weight update device for on-device learning according to an embodiment of the present invention. FIG. 3 is a diagram showing an example of a weight update algorithm for on-device learning according to an embodiment of the present invention. FIG. 4 is a diagram showing an example of a weight update operation for on-device learning according to an embodiment of the present invention. FIG. 5 is a graph showing an example of determining a threshold value applied in a weight update method for on-device learning according to an embodiment of the present invention, And, FIG. 6 is a flowchart illustrating a weight update method of a weight update device for on-device learning according to an embodiment of the present invention. Specific details for implementing the invention

[0017] The following detailed description of the invention refers to the accompanying drawings, which illustrate specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that various embodiments of the invention are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in relation to one embodiment. It should also be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not intended to be limiting, and the scope of the invention is limited only by the appended claims, including all equivalents to those claimed therein, provided appropriately described. Similar reference numerals in the drawings refer to the same or similar functions across various aspects.

[0018] The components according to the present invention are defined by functional distinction rather than physical distinction, and can be defined by the functions each performs. Each component may be implemented as hardware or as program code and processing units that perform each function, and the functions of two or more components may be included and implemented in a single component. Therefore, it should be noted that the names assigned to the components in the following embodiments are not intended to physically distinguish each component but are assigned to imply the representative function performed by each component, and that the technical concept of the present invention is not limited by the names of the components.

[0019] Preferred embodiments of the present invention will be described in more detail below with reference to the drawings.

[0020] FIG. 1 is a diagram showing the internal structure of a weight update device for on-device learning according to an embodiment of the present invention.

[0021] The illustrated weight update device (100) includes a core (101), a flash memory (103), an SRAM (105), and an on-chip bus (107), and each of the core (101), the flash memory (103), and the SRAM (105) is connected to the on-chip bus (107). Here, the weight update device may be an MCU.

[0022] The above core (101) executes a learning program for a neural network model for on-device learning and stores the neural network model weights (110) generated from the neural network model in an on-chip memory, namely a flash memory (103) and an SRAM (105).

[0023] However, while flash memory (103) is suitable for storing large amounts of data and storing permanent data, it has the limitation that writing operations must be performed in page units. That is, to store data in flash memory (103), it must be deleted in page units before storing. In addition, SRAM (105) has the disadvantage that while reading / writing operations are unrestricted, it is expensive and has a small capacity.

[0024] Accordingly, in the embodiments of the present invention described below, a method for updating only a portion of the neural network model weights (110) using minimal memory while minimizing the loss of inference accuracy of the neural network model learning will be explained.

[0025] First, the weight update method according to an embodiment of the present invention includes a process of storing neural network model weights (110) in flash memory (103) in page units through a linker during the learning program compilation stage of the neural network model. Below, the operation of the linker storing neural network model weights (110) in flash memory (103) in page units will be explained in more detail through FIG. 2.

[0026] FIG. 2 is a diagram showing an example of a mapping operation of a linker included in a weight update device for on-device learning according to an embodiment of the present invention.

[0027] When the illustrated linker (120) receives the weights of a neural network model, for example, a weight matrix (110), it divides the weight matrix (110) into columns and maps them to each page of the flash memory (103). At this time, the linker (120) divides the weight matrix (110) by referring to another input, a linker script.

[0028] The weight matrix (110) divided into the above column units is mapped and stored in each page of the flash memory through the linker (120). That is, it is sequentially mapped and stored in the pages occupied starting from the starting address of the flash memory.

[0029] During the training process of a neural network model, weight parameters that overflow due to the capacity of SRAM are stored in flash memory. Since data stored in SRAM is freely accessible for reading and writing, updating weights during the training process is cost-effective. However, since modifying data stored in flash memory requires deleting the entire page corresponding to the address where the data exists, there are limitations on updating weights during the training process.

[0030] In addition, since the training program code and data (i.e., weights) of the neural network model are stored together in the flash memory, any deletion of pages to modify this data may affect the execution of the code. Therefore, in an embodiment of the present invention, the weight matrix (110) of the input layer is stored in columns in each flash memory page to update only selected weights.

[0031] FIG. 2 illustrates, as an example, the process of storing a weight matrix (110) in Section A, where the start and end addresses of the flash memory are _start_A and _end_A, respectively, through a linker (120). Here, the linker (120) refers to an array that manages the start addresses of the pages constituting the flash memory to access the section where the weights are stored during runtime, and uses the compiler's _attribute_ keyword to map each column constituting the weight matrix (110) of the input layer to a single page.

[0032] In an embodiment of the present invention, columns of a weight matrix (110) stored in flash memory can be modified through the mapping operation of such a linker (120), and if it is possible to determine pixels requiring weight updates from an input image, only the weights associated with those pixels can be partially updated.

[0033] FIG. 3 is a diagram showing an example of a weight update algorithm for on-device learning according to an embodiment of the present invention.

[0034] Algorithm 1 described represents a process in which, when a page requiring an update is determined among the pages of flash memory, only the weight stored in the page requiring an update is partially updated.

[0035] The i-th page among the pages of flash memory, that is, page up If an update is needed for the weights stored in [i], (line 7) page up [i], i.e., page up Update the weights by copying the weights stored in [i] to SRAM, (lines 8-9) page up The weight stored in [i] is replaced with the weight updated through the above SRAM (line 10). That is, page up Perform a rewrite operation on [i] to store the updated weights through the SRAM.

[0036] In order to update only a portion of the weights stored in flash memory as described above, it is crucial to select weights that minimize the loss in inference accuracy of the neural network model and optimize memory usage.

[0037] Below, through Fig. 4, the process of determining the page that needs to be updated in Algorithm 1 of Fig. 3, that is, the process of selecting the weight column that needs to be updated from the weight matrix stored in flash memory, will be explained in more detail.

[0038] FIG. 4 is a diagram showing an example of an operation to select and update weights for on-device learning according to an embodiment of the present invention.

[0039] A weight update device according to an embodiment of the present invention calculates the amount of change Δavg_grad between gradient average vectors used to select weight columns requiring update according to the following procedure.

[0040] That is, the weight update device calculates the average gradient vector avg_grad [t] for the current time point t and the average gradient vector avg_grad [t-1] for the previous time point t-1, which are generated according to the learning of the neural network model, by multiplying the error vector (error2) generated in the hidden layer of the neural network model and the result vector (a1) of the input layer activation function of the neural network model. Here, l1 represents the size of the input layer and l2 represents the size of the hidden layer.

[0041] Multiplication of the error vector (error2) and the result vector (a1) of the activation function generates a gradient matrix of size l2x l1, and the nth element of the result vector (a1) of the activation function and the error vector (error2) are multiplied by MAC (Multiply-Accumulate) and divided by l2 to calculate the average gradient vector avg_grad[t] for time t.

[0042] Then, the weight update device compares the slope average vector avg_grad[t] calculated above for t with the slope average vector avg_grad[t-1] for t-1 and calculates the change amount Δavg_grad[t] between the slope average vectors as shown in Equation 1 below.

[0043]

[0044] A weight update device that calculates the change amount Δavg_grad[t] of the gradient average vector through mathematical formula 1 selects weight columns that need to be updated from the weight matrix stored in flash memory based on the calculated Δavg_grad[t] and a predetermined threshold value TH. At this time, the weight update device selects columns where the calculated Δavg_grad[t] exceeds the threshold value TH as weight columns that need to be updated.

[0045] Conventional Fully Connected (FC) layer backpropagation required runtime memory of size l1 x l2 for gradient matrix calculation, but the weight update method according to an embodiment of the present invention requires only memory to store Δavg grad[t-1] at time t, thus reducing runtime memory usage to l 1 x Reduce to N. Here, N represents the number of columns in the weight matrix selected as update targets that exceed the threshold TH.

[0046] In addition, the weight update method according to an embodiment of the present invention minimizes runtime memory usage through in-place operation during the process of calculating updates to Δavg_grad and selected weight columns based on the comparison of Δavg_grad and threshold TH. Here, the memory usage can be calculated as shown in Equation 2 below.

[0047]

[0048] Subsequently, the weight update device updates the weights by copying the page corresponding to the selected weight column from the weight matrix stored in the flash memory to the SRAM as described in FIG. 3, and then performs a page rewrite operation of the flash memory to store the updated weight column through the SRAM.

[0049] Below, we will explain in more detail how to determine the threshold value applied when selecting the weight column to be updated through Fig. 5.

[0050] FIG. 5 is a graph showing an example of determining a threshold value applied in a weight update method for on-device learning according to an embodiment of the present invention.

[0051] The weight update method according to an embodiment of the present invention partially updates only selected weights, so while it can reduce training costs, there is a concern that inference accuracy may also decrease.

[0052] Therefore, in order to select weights to be updated while maintaining inference accuracy, an embodiment of the present invention determines a threshold value applied when selecting weight columns by considering the gradient change amount propagated to each input image pixel.

[0053] Figure 5(a) shows the difference Δavg_grad between the mean gradient propagated to 784 attenuated input image features in the hidden layer and the mean gradient of the previous step during the training process of an FC model using the 28 x 28 MNIST dataset.

[0054] As can be seen in Figure 5(a), the Δavg_grad of the input image features is high across the entire input image feature location corresponding to the center of the image where the number is recorded in the actual MNIST dataset.

[0055] Figure 5(b) shows the ratio of feature indices where Δavg_grad is greater than the threshold TH when defining the threshold TH to filter out the dominant Δavg_grad propagating pixels.

[0056] As such, the present invention proposes a technique for partially updating only the weights corresponding to input image feature indices where Δavg_grad is greater than the threshold value TH. As the threshold value TH becomes smaller, the coverage to be partially updated increases, and when TH = 0.0005, the ratio to the total feature dimension is 24.5%. In this case, it can be seen that the threshold values ​​TH used are 0.0005, 0.001, 0.005, 0.01, 0.015, and 0.02, as shown in Fig. 5(a).

[0057] FIG. 6 is a flowchart illustrating a weight update method of a weight update device for on-device learning according to an embodiment of the present invention.

[0058] The weight update device divides the weight matrix of the neural network model into columns (S601) and maps the divided weight columns to each page of flash memory for storage (S603).

[0059] Then, the weight update device selects weight columns that require updating based on the training of the neural network model from the weight matrix stored in S603 (S605), and updates the weights by copying the pages of flash memory corresponding to the weight columns selected in S605 to SRAM (S607).

[0060] Subsequently, the weight update device performs a rewrite operation on the page of the flash memory and stores the weight updated via SRAM in S607 on the page of the flash memory.

[0061] The weight update method for on-device learning of a neural network model according to the present invention as described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination.

[0062] The program instructions recorded on the above-mentioned computer-readable recording medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software.

[0063] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, and flash memory.

[0064] Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0065] Although various embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the invention as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present invention.

Claims

Claim 1 A weight update method in a weight update device for on-device learning of a neural network model, comprising: a step of dividing a weight matrix of the neural network model into columns and mapping and storing it in each page of a flash memory; a step of selecting a weight column that requires updating according to the learning of the neural network model; a step of copying the page of the flash memory corresponding to the selected weight column to a Static Random Access Memory (SRAM) to update the weight and performing a rewrite operation on the page of the flash memory; wherein the step of selecting the weight column comprises: a step of calculating a gradient average vector at a current time point and a gradient average vector at a previous time point that occur according to the learning of the neural network model; a step of calculating the amount of change between the gradient average vector at the current time point and the gradient average vector at the previous time point; and a step of selecting the weight column to be updated from the weight matrix stored in the flash memory based on the calculated amount of change and a predetermined threshold value. Claim 2 delete Claim 3 A weight update method according to claim 1, characterized in that the gradient average vector is calculated as the product of an error vector generated in the hidden layer of the neural network model and a result vector of the input layer activation function of the neural network model. Claim 4 A weight update method according to claim 1, characterized in that the selected weight column is a column in which the amount of change exceeds the threshold value. Claim 5 A weight update method according to claim 1, characterized in that the threshold value is determined by considering the gradient change amount propagated to each image pixel input during the learning process of the neural network model. Claim 6 A weight update device for on-device learning of a neural network model, comprising: a flash memory that stores the weights of the neural network model in page units; a static random access memory (SRAM) that updates the weights by copying the weights stored in the flash memory; and a linker that divides the matrix of the weights into columns and maps them to each page of the flash memory; wherein the weight update device selects a column of weights that requires updating according to the learning of the neural network model, and controls the copying of only the pages of the flash memory corresponding to the selected column of weights to the SRAM to update only a portion of the weights, and wherein the selected column of weights is selected based on the amount of change between the gradient average vector at the current time point and the gradient average vector at the previous time point, which occurs according to the learning of the neural network model, and a predetermined threshold value. Claim 7 delete Claim 8 A weight update device according to claim 6, characterized in that the gradient average vector is calculated as the product of an error vector generated in the hidden layer of the neural network model and a result vector of the input layer activation function of the neural network model. Claim 9 A weight update device according to claim 6, characterized in that the selected weight column is a column in which the slope change amount exceeds the threshold value. Claim 10 A weight update device according to claim 6, characterized in that the threshold value is determined by considering the gradient change amount propagated to each image pixel input during the learning process of the neural network model.

Images