Methods and apparatus for training computer-implemented models
By setting the mask weight search range on edge devices and optimizing weight updates using multi-window data streams, the high computation and storage requirements in incremental learning are addressed, thereby improving the performance of computer vision models on edge devices.
Patent Information
- Application Number
- CN202110055314.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-07-21
- Filing Date
- 2021-01-15
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2041-01-15
AI Technical Summary
When existing computer vision models are incrementally learned on edge devices, the weight gradient calculation step differs from the calculation and data reuse pattern of the forward and backward propagation steps, resulting in high computation and storage requirements, making it difficult to apply them effectively on edge devices directly.
By setting the search range for mask weights, the target mask weights are identified and updated. Combining binary masks and pre-trained models, weight gradient calculations are performed only within the search range. Multi-window data flow and channel reordering techniques are used to optimize the weight update process.
It reduces latency and memory access requirements on edge devices, improves the inference performance of computer vision models on edge devices, and reduces unnecessary computation.
Smart Images

Figure CN113496248B_ABST
Abstract
Description
[0001] This application claims the benefit of Korean Patent Application No. 10-2020-0041638, filed on April 6, 2020, with the Korean Intellectual Property Office, and Korean Patent Application No. 10-2020-0090452, filed on July 21, 2020, the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field
[0002] The following description relates to methods and apparatus for training computer-implemented models. Background Technology
[0003] Computer vision aims to extract useful information from images. With the emergence of convolutional neural networks (CNNs) as an algorithm for computer vision tasks, numerous accelerators have been developed to build fast and efficient artificial intelligence (AI) systems in edge devices. However, the serviceability of such systems heavily relies on learning models (e.g., computer-implemented models, computer vision models) pre-trained using full datasets (e.g., ImageNet with 1000 classes) on remote servers, thus limiting the capabilities of existing inference engines in such systems.
[0004] To expand the application scope of computer-implemented models, various incremental learning (IL) techniques can be used. One goal of IL techniques is to efficiently train computer-implemented models using new tasks or a set of classes based on the knowledge of pre-trained models. However, in gradient descent-based learning processes, such techniques may require significant computational and memory accesses, making them potentially unsuitable for direct use at edge devices.
[0005] The core computation of a typical inference process consists of three steps: forward propagation (FP), backward propagation (BP), and weight gradient computation (WGC). FP and BP can have computation and data reuse patterns similar to those of inference, and thus, existing techniques utilizing unique data flows and data characteristics have been well optimized for these steps. However, the typical WGC step differs significantly from the FP and BP steps in terms of computation and data reuse patterns. Summary of the Invention
[0006] The present invention is provided in a simplified form to introduce the choice of concepts further described below in the detailed embodiments. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.
[0007] In one general aspect, a method for training a computer-implemented model includes: obtaining a first training dataset; pre-training the computer-implemented model based on the first training dataset to obtain a pre-trained model; obtaining a second training dataset; setting a search range for the mask weights based on at least one of the distribution of mask weights of a binary mask corresponding to a filter of the pre-trained model and a learning rate-related parameter set in the computer-implemented model; identifying target mask weights among the mask weights that fall within the search range; retraining the pre-trained model using the second training dataset and updating the target mask weights based on the binary mask and the pre-trained model; and updating a portion of the binary mask based on the updated target mask weights and a preset reference value.
[0008] The steps of updating the target mask weights based on a binary mask and a pre-trained model may include: updating the target mask weights by determining the gradient of the retrained target mask weights using a filter based on the binary mask and the pre-trained model.
[0009] The method may further include: repeating the steps of retraining, updating the target mask weights, and updating a portion of the binary mask a predetermined number of times; resetting the search range based on another learning rate-related parameter obtained through the repetition; identifying additional target mask weights among the mask weights that are located within the reset search range; and performing the steps of retraining, updating the target mask weights, and updating a portion of the binary mask again.
[0010] The method may further include: initializing a binary mask by initializing the mask weights of the binary mask corresponding to the filter of the pre-trained model to constants greater than a reference value; and determining the distribution of the mask weights by performing one or more retraining operations based on the binary mask and the pre-trained model to update the mask weights.
[0011] Each retraining session corresponds to one training round.
[0012] In another general aspect, an apparatus for training a computer-implemented model includes: at least one processor configured to: obtain a pre-trained model generated by pre-training the computer-implemented model based on a first training dataset; obtain a second training dataset; set a search range for mask weights based on at least one of a distribution of mask weights of a binary mask corresponding to filters of the pre-trained model and a learning rate-related parameter set in the computer-implemented model; identify target mask weights among the mask weights that fall within the search range; retrain the pre-trained model using the second training dataset; update the target mask weights based on the binary mask and the pre-trained model; and update a portion of the binary mask based on the updated target mask weights and a preset reference value; and a memory configured to: store the second training dataset and the target mask weights.
[0013] For the step of updating the target mask weights, the processor may be configured to: update the target mask weights by determining the gradient of the retrained target mask weights based on the binary mask and the filter of the pre-trained model, and record the updated target mask weights in memory.
[0014] The processor may be configured to: repeat the steps of retraining, updating the target mask weights, and updating a portion of the binary mask a predetermined number of times; reset the search range based on another learning rate-related parameter obtained through the repetition; identify additional target mask weights among the mask weights that are within the reset search range; and perform the steps of retraining, updating the target mask weights, and updating a portion of the binary mask again.
[0015] In one general aspect, a learning method for an incremental learning model includes: setting a search range for mask weights based on at least one of the distribution of mask weights of a binary mask corresponding to a filter of a pre-trained model and a parameter related to the learning rate; identifying target mask weights among the mask weights that fall within the search range; updating the target mask weights based on the binary mask and the pre-trained model; and updating a portion of the binary mask based on the updated target mask weights and a preset reference value.
[0016] The steps for updating the target mask weights based on a binary mask and a pre-trained model may include: updating the target mask weights by determining the gradient of the target mask weights for each training round using a filter based on the binary mask and the pre-trained model; and repeating the training rounds a preset number of times.
[0017] The learning method may further include: resetting the search range based on another learning rate-related parameter obtained through repetition of training rounds; and identifying additional target mask weights among the mask weights that are located within the reset search range.
[0018] The steps for updating the target mask weights may include updating the mask weights by applying the updated target mask weights to the mask weights.
[0019] Each binary mask in the mask weights can be determined by comparing the mask weights with reference values.
[0020] A binary mask may include binary values corresponding to element values included in the filters of the pre-trained model. Each binary value may be one of a first value and a second value, whereby the first value is used to activate the corresponding element in the filters of the pre-trained model, and the second value is used to deactivate the corresponding element in the filters of the pre-trained model.
[0021] For mask weights that are not within the search range, the binary mask can be set to the value of the corresponding element in the filter used to activate the pre-trained model.
[0022] The step of updating a portion of a binary mask may include: updating the element of the binary mask corresponding to the target mask weight among a plurality of element values included in the binary mask, based on the updated target mask weight and reference value.
[0023] The step of identifying target mask weights may further include: generating a target vector associated with the position of the target mask weights in the mask weights.
[0024] The step of updating the target mask weights may further include: updating the mask weights by applying the updated target mask weights to the mask weights based on the generated target vector.
[0025] The step of updating a portion of the binary mask may further include: updating the elements in the binary mask corresponding to the target mask weights based on the result of comparing the target mask weights with reference values, according to the generated target vector.
[0026] The steps of setting the search range may include at least one of the following: setting the search range based on the average value of the mask weights according to the distribution of mask weights; and setting the search range based on a learning rate-related parameter configured to determine the level of learning rate decay.
[0027] The step of setting the search range based on the average value of the mask weights according to the distribution of mask weights may further include setting the search range based on the variance of the mask weights according to the distribution of mask weights.
[0028] The learning method may further include: initializing the binary mask by initializing the mask weights of the binary mask corresponding to the filter of the pre-trained model to constants greater than reference values; and determining the distribution of the mask weights by performing training rounds based on the binary mask and the pre-trained model to update the mask weights.
[0029] In another general aspect, a learning device for an incremental learning model includes: at least one processor configured to: set a search range for mask weights based on at least one of a distribution of mask weights of a binary mask corresponding to a filter of a pre-trained model and a parameter related to the learning rate; identify at least one target mask weight among the mask weights that falls within the search range; update the target mask weights based on the binary mask and the pre-trained model; and update a portion of the binary mask based on the updated target mask weights and a preset reference value; and a memory configured to: store the target mask weights in the memory.
[0030] Regarding the step of updating the target mask weights, the processor can update the target mask weights by obtaining the gradient of the target mask weights for each training round based on the binary mask and the filter of the pre-trained model, and record the updated target mask weights in the memory; and repeat the training rounds a preset number of times.
[0031] The processor can reset the search range based on another learning rate-related parameter obtained through repetition of training rounds; and identify additional target mask weights among the mask weights that fall within the reset search range.
[0032] The binary mask can be determined as a binary value for each of the mask weights by comparing the mask weights with the reference values.
[0033] The binary value can be either a first value or a second value. The first value is used to activate the corresponding element in the filter of the pre-trained model, and the second value is used to deactivate the corresponding element in the filter of the pre-trained model.
[0034] For mask weights that are not within the search range, the binary mask can be set to the value of the corresponding element in the filter used to activate the pre-trained model.
[0035] Regarding the step of identifying target mask weights, the processor can generate a target vector associated with the position of the target mask weight in the mask weight, and record the generated target vector in memory. Regarding the step of updating the target mask weight, the processor can update the mask weight by applying the updated target mask weight to the mask weight.
[0036] The memory can store a target vector associated with the position of the target mask weight in the mask weights. For the step of updating a portion of the binary mask, the processor can update the element in the binary mask corresponding to the target mask weight based on the result of comparing the target mask weight with a reference value.
[0037] The distribution of mask weights can include the mean and standard deviation of the mask weights.
[0038] Other features and aspects will become clear from the following detailed description, drawings, and claims. Attached Figure Description
[0039] Figure 1 This is a diagram illustrating an example of learning a binary mask for each task.
[0040] Figure 2A and Figure 2B This shows the real-valued mask weights m of the convolutional layer during the learning process. r An example diagram illustrating the distribution of .
[0041] Figure 2C This is a diagram illustrating an example of a timeline for Selective Weight Update (SWU).
[0042] Figure 3 This is a diagram showing an example of SWU.
[0043] Figure 4A and Figure 4B This is a diagram illustrating an example of data flow in SWU.
[0044] Figure 5 This is a diagram illustrating an example of the architecture of a device used to perform SWU.
[0045] Figure 6 This is a flowchart illustrating an example of the learning method in an incremental learning model.
[0046] Figure 7 This is a flowchart illustrating an example of setting the search scope.
[0047] Throughout the accompanying drawings and detailed embodiments, the same reference numerals denote the same elements. The drawings may not be to scale, and for clarity, illustration, and convenience, the relative dimensions, scale, and depiction of elements may be exaggerated. Detailed Implementation
[0048] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be changed as will become clear upon understanding this disclosure, except for operations that must occur in a specific order. Furthermore, for clarity and conciseness, descriptions of features known upon understanding this disclosure may be omitted.
[0049] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein have been provided merely to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein that will be clear upon understanding the disclosure of this application.
[0050] The terminology used herein is for the purpose of describing particular examples only and is not intended to limit disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. As used herein, the term "and / or" includes any one of the relevant listed items and any combination of any two or more of the relevant listed items. As used herein, the terms "comprising," "including," and "having" indicate the presence of the stated features, quantities, operations, elements, components, and / or combinations thereof, but do not exclude the presence or addition of one or more other features, quantities, operations, elements, components, and / or combinations thereof.
[0051] Furthermore, terms (such as first, second, A, B, (a), (b), etc.) may be used herein to describe components. Each of these terms is not intended to define the nature, order, or sequence of the corresponding component, but only to distinguish the corresponding component from other components. It should be noted here that the use of the term "may" (e.g., what an example or embodiment may include or can implement) with respect to examples or embodiments indicates the existence of at least one example or embodiment that includes or implements such a feature, but all examples and embodiments are not limited thereto.
[0052] Throughout the specification, when an element (such as a layer, region, or substrate) is described as being "on" another element, "connected to," or "bonded to" another element, the element may be directly "on" said other element, "connected to," or "bonded to" said other element, or there may be one or more other elements in between. Conversely, when an element is described as being "directly on" another element, "directly connected to," or "directly bonded to" another element, there may be no other elements in between. Similarly, expressions such as "between" and "immediately between," and "adjacent" and "next to" may be interpreted as described above.
[0053] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains, and as commonly understood after understanding this disclosure. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and in this disclosure, and shall not be interpreted in an idealized or overly formalized sense.
[0054] Furthermore, in the description of the exemplary embodiments, such descriptions will be omitted when it is believed that a detailed description of the structure or function known therefrom after understanding the disclosure of this application would lead to a vague interpretation of the exemplary embodiments.
[0055] In the following description, examples will be described in detail with reference to the accompanying drawings, and the same reference numerals in the drawings always denote the same elements.
[0056] One aspect provides a selective weight update (SWU) method for reducing latency and memory access. The method may include: identifying weights to be activated from the weights of a pre-trained model in the initial phase of the learning process, and selectively performing weight gradient computation (WGC) on weights to be deactivated other than the identified weights in the remaining phase of the learning process. Another aspect provides a method for selecting some weights to be updated without degrading or diminishing the learning quality.
[0057] The difference between incremental learning (IL) processing and general learning processing is that learning begins with a model trained using a global dataset. For example, a computer-implemented model can be pre-trained based on a first training dataset, such as the global dataset, to obtain a pre-trained computer-implemented model, which can then be retrained based on a second training dataset. The computer-implemented model may include a computer vision model, in which case the training dataset may include a training image set comprising images. Various training methods can be used to implement the pre-training of the computer-implemented model according to embodiments of the invention, without limitation. Furthermore, under the assumption that the domain of the second training dataset (e.g., additionally learned images) is similar to the domain of the first training dataset (e.g., the global dataset), it can be assumed that the weights of the feature extraction layers (e.g., convolutional layers) of the pre-trained computer-implemented model will not change significantly. By learning or training a binary mask for each task without directly changing the weights of the pre-trained computer-implemented model, new tasks or a new set of classes can be successfully learned. This can instruct the feature extraction layer to adapt to the new task by deactivating 4% to 30% of the weights of the pre-trained model (i.e., masking with 0).
[0058] One aspect of this disclosure provides a selective weight update (SWU) method for reducing latency and memory access. This method identifies weights to be activated from the weights of a pre-trained model during the initial phase of the learning process, and selectively performs weight gradient computation (WGC) on weights to be deactivated other than the identified weights during the remaining phases of the learning process.
[0059] Another aspect of this disclosure provides an architecture for performing selective WGC. On the hardware side, even with limited resources, the inference capabilities of Internet of Things (IoT) devices should not be hindered; therefore, designing a dedicated architecture or placing large devices within an accelerator to perform selective WGC may be infeasible or impractical. Therefore, to perform selective WGC based on a general-purpose neural processor (NPU) architecture, a novel multi-window dataflow using a window batch device is provided. Furthermore, to allow the accelerator to efficiently handle selective WGC, two pre-epoch processing steps are introduced: mask weight compression and channel reordering.
[0060] Figure 1 This is a diagram illustrating an example of learning a binary mask for each task.
[0061] Reference Figure 1 Binary mask learning can include convolutional neural network (CNN) learning processes based on gradient descent algorithms. CNN learning processes can include three main steps: forward propagation (FP), back propagation (BP), and weight gradient computation (WGC). In one example, each of the three steps can be performed as a convolution of two different input tensors in the convolutional layer of the learning model. Binary mask learning can include additional learning processes such as thresholding, masking, and updating.
[0062] During the learning process, the binary mask matrix m b It can have a real-valued mask weight matrix m r The real-valued mask weight matrix m r It has the same size as the filter in the pre-trained learning model. For example, a binary mask can be generated using hard binarization via binary thresholding, using a predefined threshold τ. In this example, the mask can be 1 when the mask weight is greater than or equal to the predefined threshold τ, and 0 otherwise.
[0063] In one example, binary masks in FP and BP can activate or deactivate each weight in a pre-trained filter through element-wise multiplication, known as masking. The masked filter can then be used in place of the pre-trained filter during convolutional processing. In WGC, the weight gradient (WG), computed through convolution between the input feature (IF) map and the loss gradient (LG) map, can be used to update the real-valued mask weights instead of the pre-trained weights.
[0064] Convolution in WGC can have different characteristics compared to convolution in FP and BP. First, in WGC, the two inputs, IF and LG, can have almost the same size; therefore, the convolution window can be much larger than the convolution window in FP and BP. Second, both inputs (IF and LG) are three-dimensional (3D) tensors; therefore, 3D convolution can be performed instead of four-dimensional (4D) convolution. For example, when IF and LG have C0 and C1 respectively... I Channel and C O When dealing with channels, convolution can be performed on each of all channels, and C I ×C O Each weighted gradient channel can be output. The equation for WGC can be represented by Equation 1 below.
[0065] Equation 1
[0066]
[0067] In equation 1, and They represent WG respectively CI,CO (m,n) weight gradient on the channel, LG CO Loss gradient at (i,j) on channel and IF CI The input (i+m,j+n) on the channel is activated.
[0068] Figure 2A and Figure 2B This shows the real-valued mask weights m of the convolutional layer during the learning process. r An example diagram illustrating the distribution of .
[0069] Reference Figure 2A Before training, all m r The value can be initialized to a constant i greater than the threshold τ. Since the constant i is greater than the predefined threshold τ, all binary masks m b The value is 1. This means that all pre-trained weights can be activated in the initial stages of learning. For example... Figure 2A As shown, as learning progresses (e.g., from Epoch 3 to Epoch 30), m r The distribution can be transformed into a Gaussian distribution. Furthermore, as the distribution widens, a mask with zero values can be generated, and some of the pre-trained weights can be deactivated.
[0070] In short, binary masks can be learned by training m... r This involves searching for the processing of pre-trained weights that will be deactivated. For example, when the dataset for the new task is in a similar domain to the pre-training dataset, the ratio of weights to be deactivated may be high.
[0071] According to one example embodiment, this provides a method of observing m during the initial stage of learning. r The SWU technique distributes weights to narrow the search space for deactivated weights.
[0072] The range of mask weights can be divided into two: the search range and the non-search range, such as... Figure 2B As shown in the diagram. Mask weights located within the search range (e.g., less than or equal to the search bound) will be referred to as target mask weights hereinafter, and mask weights located outside the search range (e.g., greater than the search bound) will be referred to as non-target mask weights hereinafter. Before training begins, the search range can be set to (-∞, +∞). As the search bound sb of the search range moves during training, the search range can be narrowed to (-∞, sb). Unnecessary computations are eliminated by updating only the target mask weights located within the search range.
[0073] Figure 2C This is a diagram showing an example of a timeline used for SWU. (Refer to...) Figure 2C The search boundary sb can be updated in two cases to narrow the search range: after training begins or after the learning rate lr decays. The initial search boundary sb can be based on the m layers. r The distribution determines this. For example, the initial search boundary sb can be set as the m of the layer. r The average value. In this example, approximately half of m r Values can be included in the search range. Since the decaying learning rate *lr* leads to a smaller change in the mask weights, the search range can be further narrowed after the learning rate *lr* decays. Therefore, the search boundary *sb* can be updated such that the gap between the threshold *τ* and the search boundary *sb* is reduced by the learning rate decay factor.
[0074] To identify target mask weights, target vectors are generated. Before the start of each training epoch, each mask weight is compared to the layer's search boundary (sb) to generate the target vector. The "T" in the target vector indicates the position of the target mask weight within the search range, and the "U" indicates the opposite (i.e., the position of the mask weight outside the search range).
[0075] Figure 3 This is a diagram showing an example of SWU.
[0076] In one example, when the mask weights are outside the search range, since mask weights outside the search range are no longer updated, their exact values are not needed, and their binary masks are kept as 1. For example... Figure 3 As shown in the upper box 310, only the mask weight m can be compressed. rThe target mask weights 313 in 311 are stored, therefore, the mask weights m r The location information of 311 can be stored in target vector 312. For example... Figure 3 As shown in the lower box 320, only the weight gradient at position T of the target vector 321 needs to be calculated, so that element-wise m-factors can be performed. r Update. Generate the target vector and m. r Compression can be performed only once per round, so the latency used for compression can be negligible compared to the main training process.
[0077] Pseudocode for Algorithm 1 in SWU technique
[0078]
[0079] The overview of the techniques provided herein is described in Algorithm 1 above. In Algorithm 1, lines 2 through 7 may indicate the preprocessing used for SWU (such as learning rate lr and search boundary sb update, and m). r compression).
[0080] The architecture provided herein supports SWU on typical neural processor (NPU) architectures for Internet of Things (IoT) devices with inference and training capabilities. The terms "learning" and "training" are used interchangeably herein. According to one example embodiment, a multi-window MW data stream with window placement units (WPUs) and channel reordering capabilities is provided herein to perform selective WGC on a general-purpose NPU architecture with minimal overhead (such as DianNao).
[0081] Due to the aforementioned convolutional characteristics of WGC, the data flow in WGC can differ from that in FP and BP. Figure 4A This illustrates the typical data flow in WGC when the accelerator has a DianNao-like memory hierarchy and processing units (PUs) comprising a tree of multipliers and adders. Two types of on-chip buffers exist: input buffers and kernel buffers. Input buffers globally distribute data to all PUs, while kernel buffers locally multicast different data to each PU.
[0082] like Figure 4A As shown, the input buffer and kernel buffer can handle IF and LG respectively, and each PU can output the weight gradient WG in different channels. Here, the element values in the same two-dimensional (2D) convolution window can be computed together. Therefore, the buffer can simultaneously cast the element values in the same convolution window.
[0083] However, because the IF is transmitted globally to all adder trees, selective WGC cannot be implemented using existing data streams. That is, all PUs can compute the weight gradient WG at the same location on each channel. For selective WGC, each PU may need to generate the weight gradient WG independently at different locations. Therefore, a new data stream using a multi-window MW data stream with WPUs is presented here.
[0084] As mentioned above, since the two inputs are almost the same size, the convolution window in WGC can be relatively large. For example, when the size of the IF is 28×28 and the size of the weight gradient WG is 3×3, the size of its convolution window can be 26×26. This indicates that almost all element values of the IF graph (e.g., about 86% of the 28×28 graph) can be used to compute a weight gradient. The multi-window MW data stream provided here is derived from this observation.
[0085] Figure 4B This shows an example of a multi-window (MW) data stream. (See reference...) Figure 4B The sizes of IF, LG, and weight gradient WG are 6×6, 4×4, and 3×3, respectively, and the number of multipliers Tm in each PU is 6. The first PU calculates the weight gradient g. 0,0 And the second PU calculates the weight gradient g. 0,2 .
[0086] To simultaneously compute two weight gradients at different locations, an input buffer can buffer the activated input (e.g., a) regardless of location. 0,0 To a 0,5 The WPU broadcasts the loss gradient from the kernel buffer to the PU. Conversely, the WPU can match the loss gradient from the kernel buffer to the corresponding activation input based on the positional information from the target vector. For example, the first PU should not include a in its computation. 0,4 and a 0,5 Therefore, the WPU can insert 0 in the fifth and sixth multipliers. In the case of the second PU, the WPU can insert 0 in the first and second multipliers, and displace the active input in the third through sixth multipliers.
[0087] Figure 4B The example illustrates that multi-window (MW) data streams incur some overhead due to unnecessary input features mapped to the PU. However, the latency overhead is not significant due to the large convolutional window.
[0088] Table 1
[0089] IF size 56×56 28×28 14×14 7×7 Total Expenses (%) 16.67 18.59 20.00 28.57
[0090] Table 1 above indicates the latency overhead based on the size of the IF when the weight gradient WG is 3×3 and Tm = 16. Since utilization decreases when the IF size is relatively small, the maximum latency overhead occurs when the IF size is 7×7. Although the PU is not fully utilized, the total training latency can ultimately be reduced because unnecessary WGCs are skipped.
[0091] When each PU generates a different number of weight gradients through the WG channels, PU utilization can decrease due to load imbalance. Therefore, the channels may need to be reordered based on the number of weight gradients to be computed, which is the same as the number of T in the target vector. The target vector can be updated in each round, so channel reordering can be performed in each round.
[0092] Channel reordering can be performed by generating a Channel Order (CO) vector without changing the physical addresses of LG and WG. The CO vector can be a series of channel indices ordered by the number of T in the target vector. The CO vector can be transferred to the Direct Memory Access (DMA) engine to read the LG graph from external memory into the kernel buffer. Figure 4B As shown, LG 3 and LG 5 Both can compute the same number (e.g., 4) of the weight gradients. Therefore, the load on the two PUs can be balanced. The CO vector can be reused when the computed weight gradients are written back to external memory.
[0093] Furthermore, channel discarding, which discards the channel index from the CO vector, can be performed when all weight gradients of a channel are skipped (i.e., all elements of the target vector are U). With such channel discarding, the LG graph can be avoided from external memory, thus reducing the number of off-chip accesses accordingly.
[0094] Figure 5 This is a diagram illustrating an example architecture of a device used to perform SWU. The NPU architecture may include a PU array, input buffers and kernel buffers, a DMA engine, and post-processing units for batch normalization, Corrected Linear Unit (ReLU, also known as a linear rectified function) functions, pooling, and weight updates. The PU array may include Tn PUs, and each PU includes Tm multipliers. Additional components for performing selective WGC may include a WPU and a mask buffer. The mask buffer may be a storage device for binary masks for each task. Each of the multipliers may be modified to be mask-gated to support masking, and binary mask generation (or thresholding) may be performed using comparators in the post-processing units (e.g., ReLU). The NPU can perform main training processing primarily involving multiply-accumulate (MAC) operations. The main processor may be responsible for preprocessing (such as m... r Compression and channel reordering).
[0095] Figure 6 This is a flowchart illustrating an example of the learning method for an incremental learning model. The learning method for the incremental learning model described below corresponds to the learning method based on the SWU technique described above.
[0096] Reference Figure 6 The learning method of the incremental learning model includes: operation 610 of setting the search range based on at least one of the mask weight distribution and parameters related to the learning rate; operation 620 of identifying at least one target mask weight located in the search range from multiple mask weights; operation 630 of updating the target mask weight based on the binary mask and the pre-trained model; and operation 640 of updating a portion of the binary mask based on the updated target mask weight and a preset reference value.
[0097] The operation 610 of setting the search range may include setting the search range based on at least one of the mask weight distribution of the binary mask corresponding to the filter of the pre-trained model and parameters related to the learning rate. In operation 610, the filter of the pre-trained model may include pre-trained weights.
[0098] In operation 610, the binary mask may include the binary mask matrix m as described above. b A binary mask may correspond to the filters of a pre-trained model and includes binary values corresponding to element values in the filters of the pre-trained model for activating or deactivating some element values in the filters of the pre-trained model. That is, each element of the binary mask may be set to one of a first value and a second value, the first value being used to activate the corresponding element in the pre-trained filter and the second value being used to deactivate the corresponding element in the pre-trained filter. The binary mask may be configured as a matrix of the same size as the filters of the pre-trained model as described above, but is not limited to a matrix form. In the following, examples of setting the binary mask to 1 to activate elements in the filters of the pre-trained model and setting the binary mask to 0 to deactivate elements in the filters of the pre-trained model will be described.
[0099] In operation 610, the mask weights may include the real-valued mask weight matrix m described above. r Mask weights can correspond to a binary mask, and each mask weight can correspond to each of the element values included in the binary mask. As mentioned above, mask weights can be configured using a matrix of the same size as the filters of the pre-trained model, but are not limited to matrix form.
[0100] In operation 610, the mask weight distribution can be a distribution of mask weights, and may include a statistical distribution of mask weights. For example, based on the mask weight distribution, the mean, variance, and standard deviation of the mask weights can be determined. In operation 610, the parameters related to the learning rate can be the set of parameters in the learning model, and may include a learning rate decay factor. (Refer to...) Figure 7 Describe the search scope settings in detail.
[0101] In operation 620, which identifies the target mask weights, the search range can be determined using the search boundary sb as described above. In operation 620, the target mask weights can be the target mask weights described above. That is, operation 620, which identifies the target mask weights, can include compressing the mask weights m. r The operation 620, which identifies the target mask weights, may further include generating a target vector associated with the position of the target mask weight in the mask weight. The mask weights can be updated by applying the updated target mask weights to the mask weights based on the generated target vector. For example, updating the mask weights may include: identifying the positions of weights outside the search range and the positions of weights within the search range from the mask weight matrix based on the target vector, and storing the result of updating the weights within the search range in the mask weight matrix. In this case, the weight values at the positions of weights outside the search range in the mask weight matrix may not be stored.
[0102] The operation 630 of updating the target mask weights may include updating the target mask weights according to the FP, BP, and WGC steps. As described above, the operation 630 of updating the target mask weights may include updating the mask weights by performing masking to activate or deactivate each weight in the pre-trained filter using a binary mask and performing training rounds of FP, BP, and WGC using the masked filter (i.e., the masked weights). The operation 630 of updating the target mask weights may include updating the mask weights by applying the updated target mask weights to the mask weights. In one example, mask weights that are out of search range may not be updated during the training process, and therefore, mask weights that are out of search range may not be stored. That is, mask weights that are out of search range may not be stored, and the corresponding values of the binary mask may be fixed to the values set during the training process.
[0103] The training epochs associated with calculating the gradient of the target mask weights can be repeated several times. In one example, updating the target mask weights may include updating the target mask weights by obtaining the gradient of the target mask weights for each training epoch using filters and a binary mask based on the pre-trained model. That is, the target mask weights can be updated for each training epoch, and the search range can be reset based on parameters related to the learning rate obtained through repetition of training epochs. When the search range is reset, at least one target mask weight corresponding to the reset search range can be re-identified from the mask weights, and the re-identified target mask weights can be updated for each training epoch. (See also...) Figure 7 Describe in detail the reset of the search scope.
[0104] Operation 640, which updates a portion of the binary mask, may include determining the binary mask corresponding to the target mask weights by comparing the target mask weights updated in operation 630 with a preset reference value. In operation 640, the preset reference value may correspond to the predefined threshold τ described above. That is, a portion of the binary mask may indicate the binary mask corresponding to the target mask weights. The binary mask can be determined by comparing the mask weights with the reference value for each mask weight. In one example, the element of the binary mask corresponding to a weight that is not within the search range can be set to 1. That is, weights that are not target mask weights may not be updated in operation 630, and therefore the element of the binary mask corresponding to that weight may not be updated. Therefore, the set value (e.g., 1) can be fixed even when the training rounds are repeated several times according to the training process.
[0105] Figure 7 This is a flowchart illustrating an example of setting the search scope.
[0106] Reference Figure 7 The steps for setting the search range include: starting training 710, determining whether to update the search boundary sb 720, updating the search boundary sb in response to the result of determining whether to update the search boundary sb 730 in the first case and updating the search boundary sb in the second case 740, updating the target mask weights within the search range determined by the search boundary 750, and repeating the training rounds 760. The training rounds can be repeated a predetermined number of times.
[0107] The first scenario can occur in the initial stages of training. In this scenario, the mask weights can indicate a specific distribution (such as a Gaussian distribution). For example, the first scenario could be where the mask weights indicate a Gaussian distribution after the training epochs have been repeated twice. That is, the mask weights can be updated for each training epoch. The first scenario could be where the updated mask weights indicate a specific distribution. The first scenario could be where the number of training epochs with mask weights indicating a specific distribution is preset, and the training epochs are executed the preset number of times.
[0108] The learning method may include initializing the mask weights to constants greater than a reference value before training begins. Therefore, all mask weights can be greater than the reference value, and thus, the binary mask can be initialized to 1. The learning method may also include obtaining the mask weight distribution by performing training epochs to update the mask weights based on the initialized binary mask and a pre-trained model. That is, the first case can be the case where the mask weight distribution is obtained.
[0109] In the first case, the search range can be updated based on the mask weight distribution. For example, refer to... Figure 7 In the first case, the average value based on the mask weights (Mean(m)) can be represented as shown in the equation in operation 730. r )) and standard deviation (std(m r The search boundary sb is updated using k. In operation 730, k represents a preset constant. By adjusting the value of k, the narrowing of the search range can be adjusted. For example, when k is set to a large or positive value, the search boundary sb can be updated to a relatively large value; therefore, the narrowing of the search range can be smaller compared to when k is set to a small value. Furthermore, the number of target mask weights to be updated can be reduced slowly, thus the relative computational cost and relative memory usage can be high, and the relative training accuracy can be high. When k is set to a small or negative value, the opposite result can be obtained.
[0110] The second scenario involves changes in parameters related to the learning rate. For example, the learning rate decay factor might be determined based on a preset standard and then changed. In this second scenario, the search range can be updated based on parameters related to the learning rate. For example, referring to… Figure 7 In the second case, the search boundary sb can be updated based on the learning rate decay factor α, as expressed in the equation in operation 740. For example, the search boundary sb can be updated such that the gap (sb-τ) between the search boundary sb and the reference value is reduced by the learning rate decay factor α.
[0111] When the search boundary sb is not updated, the search range can remain unchanged, and the target mask weights within the search range can also remain unchanged. Therefore, the same target mask weights can be updated according to the training rounds.
[0112] Conversely, when the search boundary sb is updated in operations 730 and 740, the search range can change, and therefore, the target mask weights can also change. That is, at least one target mask weight corresponding to the changed search range can be re-identified from the mask weights. In this case, the re-identified target mask weights can be updated based on the training epochs.
[0113] According to one example embodiment, a learning method can be executed by at least one processor in the learning device described herein. Target mask weights and a binary mask can be recorded in the memory of the learning device. That is, the learning device for the incremental learning model described herein may include at least one processor and a memory for storing target mask weights. The processor can set a search range based on at least one of the mask weight distribution of the binary mask corresponding to the filters of the pre-trained model and parameters related to the learning rate; identify at least one target mask weight corresponding to the search range from the mask weights; update the target mask weights based on the binary mask and the pre-trained model; and update a portion of the binary mask based on the updated target mask weights and a preset reference value. Furthermore, the processor can generate a target vector associated with the position of the target mask weight in the mask weights, record the generated target vector in the memory, and use the target vector and target mask weight recorded in the memory to update a portion of the binary mask.
[0114] According to the example embodiments described herein, performance and energy efficiency in the WGC step can be successfully improved with minimal overhead. By compressing and storing only the mask weights that need to be learned from all mask weights in the binary mask, memory usage of the mask weights during learning can be reduced. Furthermore, by repeating training rounds while narrowing the range of mask weights whose gradients are to be computed, learning latency that may occur due to the computation of weight gradients can be reduced.
[0115] In this regard Figure 1 and Figure 5The described learning device, main processor, input buffer, kernel buffer, mask buffer, DMA, WPU, controller NPU, output buffer, DRAM, and other devices, apparatuses, units, modules, and components are implemented by hardware components. Examples of hardware components that can be used to perform the operations described in this application include, where appropriate, controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more hardware components performing the operations described in this application are implemented by computing hardware (e.g., by one or more processors or computers). The processor or computer may be implemented by one or more processing elements (such as logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field-programmable gate arrays, programmable logic arrays, microprocessors, or any other means or combination of means configured to respond to and execute instructions in a defined manner to achieve a desired result). In one example, the processor or computer includes or is connected to one or more memories storing instructions or software executed by the processor or computer. Hardware components implemented by a processor or computer can execute instructions or software (such as an operating system (OS) and one or more software applications running on the OS) for performing the operations described in this application. The hardware components can also access, manipulate, process, create, and store data in response to the execution of instructions or software. For simplicity, the singular terms "processor" or "computer" are used in the description of the examples described in this application; however, in other examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements, or multiple types of processing elements, or both. For example, a single hardware component, or two or more hardware components, may be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, may implement a single hardware component, or two or more hardware components. The hardware components can have any one or more different processing configurations, examples of which include: a single processor, a discrete processor, a parallel processor, a single instruction single data (SISD) multiprocessing, a single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.
[0116] Figures 1 to 7The methods for performing the operations described in this application, as shown, are executed by computing hardware (e.g., one or more processors or a computer), which is implemented as described above to execute instructions or software to perform the operations performed by the methods described in this application. For example, a single operation, or two or more operations, may be executed by a single processor, or two or more processors, or a processor and a controller. One or more operations may be executed by one or more processors, or a processor and a controller, and one or more other operations may be executed by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, may execute a single operation, or two or more operations.
[0117] Instructions or software for controlling a processor or computer to implement hardware components and perform the methods described above are written as computer programs, code segments, instructions, or any combination thereof to individually or collectively instruct or configure the processor or computer to operate as a machine or special-purpose computer to perform operations performed by the hardware components and methods described above. In one example, the instructions or software include machine code (such as machine code generated by a compiler) that is directly executed by the processor or computer. In another example, the instructions or software include high-level code that is executed by the processor or computer using an interpreter. Those skilled in the art can readily write the instructions or software based on the block diagrams and flowcharts shown in the accompanying drawings and the corresponding descriptions in the specification, which disclose algorithms for performing operations performed by the hardware components and methods described above.
[0118] Instructions or software used to control a processor or computer to implement hardware components and perform the methods described above, along with any associated data, data files, and data structures, are recorded, stored, or fixed in, or on, one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PRAM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage devices, hard disk drives (HDDs), solid-state drives (SDDs), flash memory, card storage (such as multimedia cards or microcards (e.g., Secure Digital (SD) or Extreme Digital (XD))), magnetic tape, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid-state drives, and any other device configured to store instructions or software and any associated data, data files, and data structures in a non-transitory manner and to provide instructions or software and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the instructions.
[0119] While this disclosure includes specific examples, it will be clear upon understanding this disclosure that various changes in form and detail may be made in these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein are to be considered descriptive only and not for limiting purposes. The description of features or aspects in each example is to be considered applicable to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order, and / or if components in the described system, architecture, apparatus, or circuit are combined in a different manner, and / or replaced or supplemented by other components or their equivalents.
[0120] Therefore, the scope of the disclosure is not limited by the specific embodiments, but by the claims and their equivalents, and all variations within the scope of the claims and their equivalents shall be interpreted as included in the disclosure.
Claims
1. A method for training a computer-implemented model, comprising: Obtain the first training dataset; The computer-implemented model is pre-trained based on the first training dataset used to perform the first task, to obtain a pre-trained model; Obtain the second training dataset; The search range of the mask weights is set based on at least one of the distribution of mask weights of the binary mask corresponding to the filter of the pre-trained model and the learning rate-related parameters set in the computer-implemented model, wherein the search range is a range less than or equal to the search boundary. Identify the target mask weights within the search range from the mask weights; The training rounds of the masked filter based on the pre-trained model are used to compute the weight gradient corresponding to the target mask weights, wherein the masked filter is obtained by activating or deactivating each weight included in the filter based on the binary mask. The pre-trained model is retrained using a second training dataset used to perform the second task, and the target mask weights are updated based on the weight gradients; a portion of the binary mask corresponding to the updated target mask weights is updated based on a preset reference value, and The masked filter is updated by applying a binary mask, including the update portion, to the masked filter for processing in the next training round of the pre-trained model. The first training dataset and the second training dataset each include a training image set, and the training image set includes images. The steps for setting the search scope include: When the distribution of the mask weights is obtained, the search range is set based on the average value of the mask weights according to the distribution of the mask weights; and When the learning rate-related parameters that determine the level of learning rate decay have been changed according to a predetermined criterion, the search range is set based on the level of learning rate decay. Specifically, only the target mask weights within the search range are updated. In this process, the non-target mask weights that are outside the search range are not updated during training, and the binary mask corresponding to the non-target mask weights is fixed to the value set during training.
2. The method of claim 1, wherein, The steps for calculating the weight gradient corresponding to the target mask weights include: The input activations of the masked filter's input channels are obtained from the forward propagation processed during the training rounds, and the loss gradients are obtained from the backpropagation processed during the training rounds; and The weight gradient corresponding to the target mask weights is calculated based on the convolution between the input activation and the loss gradient.
3. The method according to claim 2, further comprising: Repeat the steps of retraining, updating the target mask weights, and updating a portion of the binary mask a predetermined number of times; The search range is reset based on another learning rate-related parameter obtained through the repetition; Identify additional target mask weights within the reset search range from the mask weights; and Based on the identified additional target mask weights, the steps of retraining, updating the target mask weights, and updating a portion of the binary mask are performed again.
4. The method according to claim 1, wherein, The steps for updating the target mask weights include: The mask weights are updated by applying the updated target mask weights to the mask weights.
5. The method according to claim 1, in, A binary mask comprises binary values corresponding to the element values included in the filters of a pre-trained model, and the binary mask is determined for each of the mask weights by comparing the mask weights with preset reference values. Each of the binary values is one of the first value and the second value. The first value is used to activate the corresponding element in the filter of the pre-trained model, and the second value is used to deactivate the corresponding element in the filter of the pre-trained model.
6. The method according to claim 1, wherein, For non-target mask weights that are not within the search range, the binary mask is set to the value of the corresponding element in the filter used to activate the pre-trained model.
7. The method according to claim 1, wherein, The steps for updating a portion of a binary mask include: The elements of the binary mask corresponding to the target mask weights are updated among the multiple elements included in the binary mask based on the updated target mask weights and reference values.
8. The method according to any one of claims 1 to 7, wherein, The steps for identifying target mask weights include: Generate a target vector associated with the position of the target mask weights in the mask weights. The steps for updating the target mask weights also include: Based on the generated target vector, the mask weights are updated by applying the updated target mask weights to the mask weights.
9. The method according to any one of claims 1 to 7, wherein, The steps for identifying target mask weights include: Generate a target vector associated with the position of the target mask weights in the mask weights. The steps for updating the binary mask also include: Based on the generated target vector, the elements in the binary mask corresponding to the target mask weights are updated based on the result of comparing the target mask weights with the reference values.
10. The method according to any one of claims 1 to 7, wherein, The steps to set the search scope include at least one of the following: The search boundary is set based on the average value of the mask weights according to the distribution of mask weights; and The search boundary is set based on learning rate-related parameters configured to determine the level of learning rate decay.
11. The method according to claim 10, wherein, The step of setting the search boundary based on the average of the mask weights according to the distribution of mask weights also includes: The search boundary is set based on the variance of the mask weights according to the distribution of the mask weights.
12. The method according to any one of claims 1 to 7, further comprising: The binary mask is initialized by initializing the mask weights of the binary mask corresponding to the filter of the pre-trained model to constants greater than the reference value. and The distribution of mask weights is determined by performing one or more retraining operations based on a binary mask and a pre-trained model to update the mask weights.
13. The method according to any one of claims 1 to 7, wherein, Each retraining session corresponds to one training round.
14. A computer-readable storage medium storing instructions, which, when executed by a processor, cause the processor to perform the method of training a computer-implemented model as described in any one of claims 1 to 13.
15. An apparatus for training a computer-implemented model, comprising: At least one processor is configured as follows: Obtain the first training dataset. The computer-implemented model is pre-trained based on the first training dataset used to perform the first task, and a pre-trained model is obtained. Obtain the second training dataset. The search range for the mask weights is set based on at least one of the distribution of mask weights in the binary mask corresponding to the filters of the pre-trained model and the learning rate-related parameters set in the computer-implemented model, wherein the search range is a range less than or equal to the search boundary. Identify the target mask weights within the search range from the mask weights. The training rounds of the masking filter based on the pre-trained model are used to compute the weight gradients corresponding to the target mask weights. The masking filter is obtained by activating or deactivating each weight included in the filter based on a binary mask. The pre-trained model is retrained using a second training dataset used to perform the second task, and the target mask weights are updated based on the weight gradients. The binary mask is updated based on a preset reference value, corresponding to a portion of the target mask weights; and The masked filter is updated by applying a binary mask, including the update portion, to the masked filter for processing in the next training round of the pre-trained model. The first training dataset and the second training dataset each include a training image set, and the training image set includes images. Specifically, for the step of setting the search range, the processor is configured as follows: When the distribution of the mask weights is obtained, the search range is set based on the average value of the mask weights according to the distribution of the mask weights; and When the learning rate-related parameters that determine the level of learning rate decay have been changed according to a predetermined criterion, the search range is set based on the level of learning rate decay. Specifically, only the target mask weights within the search range are updated. In this process, the non-target mask weights that are outside the search range are not updated during training, and the binary mask corresponding to the non-target mask weights is fixed to the value set during training.
16. The device according to claim 15, wherein, For the step of calculating the weight gradient corresponding to the target mask weights, the processor is configured as follows: The input activations of the masked filter's input channels are obtained from the forward propagation processed during the training rounds, and the loss gradients are obtained from the backpropagation processed during the training rounds; and The weight gradient corresponding to the target mask weights is calculated based on the convolution between the input activation and the loss gradient.
17. The device according to claim 16, wherein, The processor is configured to: Repeat the steps of retraining, updating the target mask weights, and updating a portion of the binary mask a predetermined number of times; The search range is reset based on another learning rate-related parameter obtained through the repetition; Identify additional target mask weights within the reset search range from the mask weights; and Based on the identified additional target mask weights, the steps of retraining, updating the target mask weights, and updating a portion of the binary mask are performed again.
18. The device according to claim 15, in, A binary mask comprises binary values corresponding to the element values included in the filters of a pre-trained model, and the binary mask is determined for each of the mask weights by comparing the mask weights with reference values. The binary value is one of the first and second values. The first value is used to activate the corresponding element in the filter of the pre-trained model, and the second value is used to deactivate the corresponding element in the filter of the pre-trained model.
19. The device according to any one of claims 15 to 18, wherein, The processor is configured to: For the step of identifying target mask weights, a target vector associated with the position of the target mask weight in the mask weight is generated, and the generated target vector is recorded in memory; and For the step of updating the target mask weights, the mask weights are updated by applying the updated target mask weights to the mask weights.
20. The device according to any one of claims 15 to 18, wherein, The memory is configured as follows: The target vector, associated with the position of the target mask weight in the mask weight, is stored in memory. The processor is configured as follows: For a portion of the step of updating the binary mask, the elements in the binary mask corresponding to the target mask weights are updated based on the result of comparing the target mask weights with reference values.
21. The device according to any one of claims 15 to 18, wherein, Each retraining session corresponds to one training round.
Citation Information
Patent Citations
Heat Collecting Plate Fixing Device for Sprinkler
KR1020200041638A
Double canned food separable
KR1020200090452A
