Artificial neural network processing method and system

Abstract

Embodiments of the present disclosure relate to a method comprising: providing an ANN processing stage having a plurality of processing layers, the processing layers having respective parameters including at least one set of weight parameters, at least one input / output activation parameter, and at least one activation function parameter; setting a dimensionality parameter of a grid to an integer value, the grid having a plurality of grid points and being identified by a set of basis vectors; selecting a set of weight parameters of a respective processing layer; vectorizing the selected set of weight parameters, resulting in a set of weight vectors, the set of weight vectors being arranged as entries of a weight vector matrix; normalizing the weight vector matrix; applying a lattice vector quantization (LVQ) process to the normalized weight vector matrix, resulting in a codebook of code words; indexing by encoding the code words of the codebook as a function of the grid, resulting in a respective tuple of indices. The normalizing comprises solving an optimization problem, the optimization problem having a first term and a regularization term Ω L .

Description

Technical Field

[0001] For example, this specification relates to methods and systems for artificial neural network (ANN) processing (e.g., convolutional neural network (CNN) processing).

[0002] For example, one or more embodiments can be applied to a hardware accelerator engine (e.g., a neural processing unit (NPU)) configured to perform ANN processing. Background Technology

[0003] Convolutional (artificial) neural networks (CNNs) include computer-based tools that utilize deep learning algorithms to perform image processing tasks.

[0004] Classifying images of large felines as leopards or jaguars can be cited as an example of this processing.

[0005] A CNN comprises multiple layers, such as multiple (hidden) processing layers, which are coupled to an input layer and configured to apply data processing to an image tensor received therefrom. The processing layers may include, for example, convolution and / or pooling processes, and the CNN may include an output layer coupled to the input layer via hidden layers.

[0006] Convolutional processing layers use filters (also known as convolutional kernels) to perform convolution on the input image data and apply activation functions to the convolutional data to produce a feature set as the result.

[0007] Pooling layers reduce the dimensionality of feature sets by performing subsampling (e.g., max or average sampling).

[0008] The processing layer can use up to millions of parameter values ​​(also known as weights). The weight values ​​are set by training the processing layer (which may involve a large training dataset).

[0009] For example, a CNN can be trained using images of multiple animals and can be configured to identify leopards or jaguars.

[0010] As mentioned above, millions of weight values ​​can be used in CNN processing, which can consume a large memory footprint.

[0011] Processing layers can involve a large number of dot product operations between weights and feature vectors, which may result in high computational costs.

[0012] Existing solutions (e.g., discussed in Choi, Y., El-Khamy, M., & Lee, J.: “Universal deepneural network compression”, 2018, arXiv:1802.02271) investigate loss compression of deep neural networks (DNNs) through weight quantization and lossless source encoding for memory-efficient deployment, thereby introducing “universal” DNN compression through universal randomized grid quantization of DNNs. Universal randomized grid quantization randomizes the DNN weights by uniform random jitter prior to grid quantization and can perform nearly optimally for any source without relying on knowledge of its probability distribution. The paper cites a method of fine-tuning vectors that quantizes the DNN to recover performance loss after quantization. Summary of the Invention

[0013] The inventors have implemented solutions discussed by Choi et al., which exhibit various drawbacks, such as:

[0014] The accuracy of CNNs decreases after quantization;

[0015] To address this reduction in accuracy, additional, potentially computationally cumbersome codeword fine-tuning is employed; and

[0016] The risk of CNN performance degradation has increased.

[0017] Reducing the power consumption and cost of CNNs (e.g., which could facilitate Internet of Things (IoT) applications based on edge computing) is therefore a worthwhile goal to pursue.

[0018] Processing circuitry configured to perform ANN processing (such as a neural processing unit (NPU)) may include, for example, a hardware accelerator engine that includes compression / decompression functions associated with weights, thereby potentially reducing the memory storage constraints of the CNN.

[0019] The relevant hardware implementation may suffer from drawbacks, such as:

[0020] Memory bottlenecks associated with weight acquisition and activation; and / or

[0021] The cost of memory area increases.

[0022] Implementation examples can help address the shortcomings discussed above.

[0023] One or more embodiments may relate to corresponding computer program products.

[0024] Therefore, one or more embodiments may include a computer program product that can be loaded into the memory of at least one processing circuit (e.g., a computer) and includes software code portions for performing method steps when the product is run on the at least one processing circuit. As used herein, references to such computer program products are to be understood as equivalent to references to computer-readable media containing instructions for controlling a processing system, so as to coordinate the implementation of the method according to one or more embodiments. The reference to “at least one computer” is intended to highlight the possibility of one or more embodiments implemented in a modular and / or distributed manner.

[0025] One or more embodiments may relate to a corresponding system (the HW accelerator system may be an example of such a system) having weights compressed using the methods illustrated herein.

[0026] One or more embodiments may relate to corresponding methods and corresponding computer program products for decompressing such weights.

[0027] One or more embodiments can reduce the computational burden associated with performing forward pass in CNN processing.

[0028] Therefore, one or more embodiments can help avoid large parallel computational data paths associated with frequently performed operations, such as 3D tensor convolutions that can result in a large number of multiplication, addition, and accumulation operations (e.g., proportional to the number of coefficients (weights) of a neural network).

[0029] One or more embodiments may provide one or more of the following advantages:

[0030] This helps compress CNN weights, thereby reducing the memory impact of CNN processing;

[0031] The weighted storage area in the embedded architecture is preserved, thereby reducing the memory area cost that can be significant in advanced sub-nano silicon process technology;

[0032] It helps to customize optimizations while maintaining the accuracy of the original network when reducing the quantization error associated with mesh vector quantization;

[0033] It effectively offsets the performance degradation associated with memory bottlenecks in weight acquisition and activation (especially for large weight tensors, such as Fc, LSTM / GU layers);

[0034] The reduced hardware complexity allows for the use of relatively simple decompression circuit blocks, potentially leading to more efficient designs, and can also be used in CNN acceleration frameworks; and

[0035] This reduces the impact of computation on bandwidth usage.

[0036] In an embodiment, the computer-implemented method includes: providing an artificial neural network (ANN) processing level, the ANN processing level including multiple processing layers with corresponding processing layer parameters, the processing layer parameters including at least one set of weight parameters, at least one input activation parameter, at least one output activation parameter, and at least one activation function parameter; setting the dimension parameter of a grid to an integer value, the grid having multiple grid points and identified by a set of basis vectors; selecting the weight parameter set of the corresponding processing layer from the multiple processing layers of the ANN processing level; applying vectorization processing to the selected weight parameter set to produce a weight vector set as a result and arranging the weight vector set as items of a weight vector matrix; performing normalization processing on the weight vector matrix to produce a normalized weight vector matrix as a result; applying grid vector quantization (LVQ) processing to the normalized weight vector matrix to produce a codebook of codewords as a result; applying indexing processing to the generated codebook, the indexing including encoding the codewords of the codebook according to the grid L, producing a corresponding index tuple as a result; and providing the generated index tuple to a user circuit, wherein performing normalization processing on the weight vector matrix includes calculating an optimization problem having a first term and a regularization term Ω. L The first term is configured to provide normalized weight values ​​based on at least one input activation parameter of the ANN processing level, the normalized weight values ​​approximating at least one output activation parameter of the ANN processing level; and the regularization term Ω L It is configured to amplify the normalized weight values ​​of the shortest distances from the grid points of the selected grid.

[0037] In an embodiment, the method includes: implementing an artificial neural network (ANN) with multiple processing layers, each processing layer having corresponding parameters, the processing layer parameters including at least one set of weight parameters (w). i ), at least one input activation parameter (a j ), at least one output activation parameter (b) j ) and at least one activation function parameter (ρ); set the dimension parameter (d) of the grid to an integer value, the grid has multiple grid points and is identified by a basis vector set (b1, b2); select the weight parameter set (w) of the corresponding processing layer in the multiple processing layers of the ANN. i ); for the selected set of weight parameters (w iThe process involves applying vectorization to generate a set of weight vectors (u1, u2, u3) and arranging them into a weight vector matrix (U); normalizing the weight vector matrix (U) to generate a normalized weight vector matrix (U'); applying grid vector quantization (LVQ) to the normalized weight vector matrix (U') to generate a codebook of codewords; and applying indexing to the generated codebook, which involves encoding the codewords in the codebook according to the grid to generate corresponding index tuples (i...). s i a i a LUT ) and providing the generated index tuples (i) to the neural processing circuitry of the ANN. s i a i a LUT The normalization of the weight vector matrix (U) involves solving an optimization problem, which has a first term and a regularization term Ω. L (u): The first term is used to activate the ANN based on at least one input parameter (a). j This provides normalized weight values, which approximate at least one output activation parameter (b) of the ANN. j ); and the regularization term Ω L (u) is configured to amplify normalized weight values ​​that are at a distance from the grid points of the selected grid.

[0038] In an embodiment, the content of a non-transitory computer-readable medium configures a computing system to execute a method comprising: implementing an artificial neural network (ANN) having multiple processing layers, each processing layer having corresponding processing layer parameters, the processing layer parameters including at least one set of weight parameters (w). i ), at least one input activation parameter (a j ), at least one output activation parameter (b) j The grid consists of at least one activation function parameter (ρ); the grid dimension parameter (d) is set to an integer value, the grid has multiple grid points and is identified by a set of basis vectors (b1, b2); the set of weight parameters (w) for the corresponding processing layer in the multiple processing layers of the ANN is selected. i ); for the selected set of weight parameters (w iThe process involves applying vectorization to generate a set of weight vectors (u1, u2, u3) and arranging them into a weight vector matrix (U); normalizing the weight vector matrix (U) to generate a normalized weight vector matrix (U'); applying grid vector quantization (LVQ) to the normalized weight vector matrix (U') to generate a codebook of codewords; and applying indexing to the generated codebook, which involves encoding the codewords in the codebook according to the grid to generate corresponding index tuples (i...). s i a i a LUT ) and providing the generated index tuples (i) to the neural processing circuitry of the ANN. s i a i a LUT The normalization of the weight vector matrix (U) involves solving an optimization problem, which has a first term and a regularization term Ω. L (u): The first term is used to activate the ANN based on at least one input parameter (a). j This provides normalized weight values, which approximate at least one output activation parameter (b) of the ANN. j ); and the regularization term Ω L (u) is configured to amplify normalized weight values ​​that are at a distance from the grid points of the selected grid.

[0039] In an embodiment, the device includes: a memory; and a processing circuitry system coupled to the memory, wherein the processing circuitry system, in operation,: sets the dimension parameter (d) of a grid to an integer value, the grid having multiple grid points and identified by a set of basis vectors (b1, b2); selects the weight parameter set (w) of a corresponding processing layer among multiple processing layers of an artificial neural network (ANN). i An ANN has multiple processing layers, each with corresponding processing layer parameters. These parameters include at least one set of weight parameters (w). i ), at least one input activation parameter (a j ), at least one output activation parameter (b) j ) and at least one activation function parameter (ρ); for the selected set of weight parameters (w) iThe process involves applying vectorization to generate a set of weight vectors (u1, u2, u3) and arranging them into a weight vector matrix (U); normalizing the weight vector matrix (U) to generate a normalized weight vector matrix (U'); applying grid vector quantization (LVQ) to the normalized weight vector matrix (U') to generate a codebook of codewords; and indexing the generated codebook, which involves encoding the codewords in the codebook according to the grid to generate corresponding index tuples (i...). s i a i a LUT The normalization of the weight vector matrix (U) involves solving an optimization problem, which has a first term and a regularization term Ω. L (u): The first term is used to activate the ANN based on at least one input parameter (a). j This provides normalized weight values, which approximate at least one output activation parameter (b) of the ANN. j ); and the regularization term Ω L (u) is configured to amplify normalized weight values ​​that are at a distance from the grid points of the selected grid.

[0040] In one embodiment, the system includes: a memory for storing compressed, normalized weight values ​​during operation; a processing core; and a hardware accelerator engine coupled to the memory and the processing core, wherein the hardware accelerator engine, during operation,: decompresses the stored, normalized weight values ​​to produce decompressed weight values, and performs artificial neural network (ANN) operations based on the decompressed weight values, wherein the stored, compressed, normalized weight values ​​correspond to solving an optimization problem having a first term and a regularization term Ω. L (u): The first term is used to activate the ANN based on at least one input parameter (a). j This provides normalized weight values, which approximate at least one output activation parameter (b) of the ANN. j ); and the regularization term Ω L (u) is configured to amplify normalized weight values ​​that are at a distance from the grid points of the selected grid. Attached Figure Description

[0041] One or more embodiments will now be described by way of non-limiting examples and with reference to the accompanying drawings, wherein:

[0042] Figure 1 This is a diagram of the processing level of an exemplary convolutional neural network (CNN).

[0043] Figure 2This is a diagram of an exemplary hardware accelerator engine;

[0044] Figure 3A and Figure 3B These are exemplary diagrams illustrating the principles under one or more embodiments;

[0045] Figure 4 This is a flowchart of the method according to this disclosure;

[0046] Figure 5A , Figure 5B and Figure 5C These are exemplary diagrams illustrating the principles under one or more embodiments;

[0047] Figure 6A and Figure 6B It is applicable to execution Figure 4 An example graph of a function that is part of the method;

[0048] Figures 7A to 7C This is an exemplary diagram illustrating the principle under the optional features of the mesh;

[0049] Figure 8 These are exemplary diagrams illustrating the principles under one or more embodiments; and

[0050] Figure 9 This is a functional block diagram of the system according to an embodiment. Detailed Implementation

[0051] In the following description, one or more specific details are illustrated to provide a thorough understanding of examples of embodiments of this specification. Embodiments may be obtained without one or more specific details or using other methods, components, materials, etc. In other instances, known structures, materials, or operations are not illustrated or described in detail so that certain aspects of the embodiments will not be obscured.

[0052] References to "embodiment" or "one embodiment" throughout this specification are intended to indicate that a particular configuration, structure, or feature of an embodiment is included in at least one embodiment. Therefore, phrases such as "in an embodiment" or "in one embodiment" that may appear at one or more points in this specification do not necessarily refer to the same embodiment.

[0053] Furthermore, specific conformations, structures, or properties can be combined in any suitable manner in one or more embodiments.

[0054] The title / reference numerals used herein are provided for convenience only and are not intended to limit the extent or scope of protection of the embodiments.

[0055] Throughout the accompanying figures, the same parts or elements are indicated by the same reference numerals; for the sake of brevity, the corresponding descriptions for each figure are not repeated.

[0056] Furthermore, throughout this specification, the term "neural network (processing)" used, for example in expressions such as artificial neural network (ANN) processing or convolutional neural network (CNN) processing, is intended to specify machine-implemented processing of signals executed via hardware (HW) and / or software (SW) tools.

[0057] In addition to the input layer, the input layer is configured to receive an input image tensor I with a certain size, for example, an image tensor I with a size given by the image width L multiplied by the image height (e.g., equal to the image width L) multiplied by the image depth (e.g., 1) multiplied by the number of images in tensor I (e.g., 1), such as Figure 1 The convolutional neural network (CNN) 10 shown may also include multiple processing layers 100, 102, 104, 106, 108, and 110, which include:

[0058] Multiple (hidden) processing layers 100, 102, 104, 106, and 108 are coupled to the input layer and configured to apply data processing (such as, for example, convolution and / or pooling) to the image tensor O received therefrom; and

[0059] An output layer 110 is coupled to the input layer via hidden layers 100, 102, 104, 106, and 108. The output layer 110 includes, for example, a fully connected layer.

[0060] As shown in the figure, processing layers 100, 102, 104, 106, and 108 can be configured to generate corresponding feature maps F1, F2, F3, and F4. Each such feature map can have a size given by multiplying the feature map widths L1, L2, L3, and L4 by the feature map height (which can be equal to the widths L1, L2, L3, and L4) by the feature map channels (e.g., three channels of an RGB image with red, green, and blue) by the number of maps.

[0061] In one or more embodiments, processing layers 100, 102, 104, 106, 108 may have a multilayer perceptron (MLP) architecture, which includes a plurality of processing units designated as perceptrons.

[0062] A single i-th perceptron among multiple perceptrons can be identified by a tuple of values, including the weight value w. i Offset value b i and activation function ρ i .

[0063] like Figure 1 The illustrated convolutional processing layer (such as a convolutional processing layer called 102) (as an example of individual layers) includes at least one convolutional kernel (matrix) w with a size of i A convolutional kernel can be represented as:

[0064] w i ∈R C×H×T i = 1, ... K

[0065] in:

[0066] H represents the kernel height.

[0067] T represents the kernel width.

[0068] K represents the number of kernels; for example, K = 1, and...

[0069] C represents the number of input channels, which can be equal to the number of (image color) channels of the input feature map F1.

[0070] The output layer 110 may include a fully connected layer, which is a type of convolutional layer that is connected to all activations in the previous layer.

[0071] A convolutional layer such as 102 (again, as a possible example) can be configured to apply an activation function to a sliding dot product.

[0072] Such operations can be represented as, for example:

[0073]

[0074] in:

[0075] w i T It is the weight vector w i The transposed version (corresponding to the kernel),

[0076] a is, for example, the input feature vector calculated by the processing layer 100 preceding the considered layer 102.

[0077] ρ is the activation function of layer 102, and

[0078] b is the output produced by applying the activation function ρ to the product of the kernel and the input feature vector.

[0079] like Figure 2 As illustrated, the neural processing unit or circuit (NPU) 20 may include a processing circuitry system configured to perform CNN processing.

[0080] Reference EP 3 346 425 A1 discusses, for example, NPU circuits suitable for accelerating CNN processing.

[0081] like Figure 2 As illustrated, such an NPU circuit 20 includes:

[0082] The system-on-chip 200 is configured as a managed processing circuit system;

[0083] Clock signal provider block or circuit 202;

[0084] Direct memory access (DMA) units or circuits 206, 206a for accessing memory circuit blocks in a manner known per se, such as via bus interface 204;

[0085] Multiple input / output interfaces 208, 210;

[0086] The flow switching circuit block 220 is configured to manage the data flow between different components on the board;

[0087] Weight encoding / decoding circuit block 230 is configured to process weight values ​​from memory (e.g., retrieved by DMA 206 and passed by streaming switch 220);

[0088] Configure accelerator circuitry assemblies 232, 232a, and 232b; and

[0089] Other processing circuit block sets 234, 236, and 240, for example, are used for pooling, activation, and miscellaneous functions.

[0090] In particular, weight values ​​can be decoded starting from encoded weight values, which can be encoded (or compressed) to reduce the memory footprint of running CNNs on NPU 20.

[0091] For example, weight values ​​can be encoded by applying quantization to them.

[0092] Vector quantization techniques suitable for performing such quantization can be found. For example, a d-dimensional vector can be mapped to a finite vector set C = {c i :i=1,2,...,N}, where each vector c i A codeword can be designated as a "codeword". The set of all codewords can be designated as a "codebook". Each codeword can be associated with its nearest neighbor region, designated as a "Voronoi region".

[0093] In grid vector quantization (LVQ), the codebook can include grid points, each with the center of the Voronoi region as an approximate vector.

[0094] As discussed in this paper, a grid L of dimension d is composed of basis vectors b1, ..., b2 in d-dimensional Euclidean space. d The set of all linear combinations of integers can be represented as:

[0095] L(b1, ..., b) d )={∑x i b i |x i ∈Z},b i ∈R d

[0096] like Figure 3A and Figure 3B Examples include:

[0097] The first two-dimensional (d=2) grid has a first pair of basis vectors b1 and b2, which can be represented in Cartesian coordinates as: b1 = [1,0]; b2 = [0; 1]; and

[0098] The second two-dimensional (d=2) mesh, such as a hexagonal mesh, has a second pair of basis vectors b1 and b2, which can be represented in Cartesian coordinates as: b2 = [0; 1].

[0099] Compared to other vector quantization techniques, LVQ exhibits advantages such as the following:

[0100] A fixed codebook is used because grid points are determined by selecting a lattice basis; and

[0101] Rapid quantization and indexing are possible because the grid structure helps maintain organized grid points.

[0102] For simplicity, one or more embodiments have been discussed above with respect to trained CNNs with set weight values ​​(e.g., based on training settings), but it should be understood that such cases are merely exemplary and by no means limiting.

[0103] like Figure 4 The illustrated method 40 for compressing weight values ​​may include:

[0104] Select the parameter set (box 400) from the CNN parameters to encode; for example, this may involve selecting the type of processing layer, such as convolutional or fully connected layers; and / or for the grid, selecting the dimension parameter value d to use, such as an integer, an even number, such as d=2 or d=4;

[0105] Performing weight value vectorization (box 402) stores the weight values ​​as vector elements, wherein such vectorization may include receiving a weight value matrix and applying data transformation processing to the weight value matrix, thereby producing a set of weight vectors as a result, as discussed below;

[0106] The normalization process (box 404) is applied to the resulting set of weight vectors, and such normalization includes solving optimization problems as discussed below;

[0107] Grid vector quantization LVQ (box 406) is performed using the selected (integer) dimension parameter value d to select the grid used for quantization, thereby producing a codebook of codewords as the result; and

[0108] Indexing (box 408) codewords is achieved by applying LVQ. Indexing includes using index tuples (i... s i a i a LUT ) to encode the i-th element of the selected grid.

[0109] For simplicity, this article mainly discusses embodiments with respect to two-dimensional meshes (e.g., d=2), but it should be understood that such mesh dimensions are merely exemplary and by no means limiting.

[0110] It should be noted that, in one or more embodiments, choosing a value for the dimension parameter d as an integer multiple or divisor of the kernel size can be advantageous, as this can facilitate the vectorization of weight values ​​as discussed below 402.

[0111] like Figures 5A to 5C exemplified in, for example Figure 1 The weight value w of the convolutional layer indicated by 102 in the figure. i The matrix can be represented as a multidimensional grid w i Its elements are weight values, grid w i It has a grid height H, grid width T, number of channels C, and number of kernels K. For example, a multidimensional grid w can be identified using an index set. i For any element, each dimension has an index, for example, w i jlmn .

[0112] For the sake of simplicity, Figures 5A to 5C The examples refer to the case where the matrices have the same height, width and number of channels, for example, H=C=T=3, but it should be understood that such examples are merely examples and are by no means limitations.

[0113] Figures 5A to 5CThis is an example graph of performing vectorization with weight value 402. Vectorization can include:

[0114] Select the processing matrix w i The elements are governed by the order of the matrix dimensions, for example, first the channel dimension C followed by the width T. For example, each index of an element is associated with a dimension, such as associating index j with the channel dimension C, index l with the width dimension T, index m with the height dimension H, and index k with the number of kernels K.

[0115] Extract matrix w i The weighted elements are used as elements of the corresponding vectors u1, u2, u3 along the selected dimensions, resulting in a vector set u1, u2, u3. The size d of the vectors is equal to the selected dimension parameter value; for example, in the considered example, the even value d = 2.

[0116] The vector sets u1, u2, u3 are rearranged, for example, by stacking the vector sets item by item, to produce a matrix U with weight vectors u1, u2, u3 stacked item by item (e.g., column by column).

[0117] like Figures 5A to 5C As illustrated in the example, the weights w are rearranged. i The weight elements of the matrix / grid can include:

[0118] like Figure 5A As illustrated in the example, from the grid w i Extract the first set of weight values ​​u 11 u 12 And the first set of values ​​u 11 u 12 Rearranged in the first vector u1, for example, u1 = (u 11 ,u 12 ), where the first set of weight values ​​u 11 u 12 Including the first weight value u 11 Second weight value u 12 The first weight value corresponds to the grid w. i The element with the first set of index values, for example, u 11 =w i 1111 The second weight value corresponds to the grid w. i The elements in the second set of index values, where the index j of the channel dimension C is increased by 1 relative to the first set of index values, for example, u 12 =w i 2111 ;

[0119] like Figure 5BAs illustrated in the example, from the weight w i Extract the second set of weight values ​​u from the grid 21 u 22 And such a second set of weight values ​​u 21 u 22 Rearranged in the second vector u2, for example, u2 = (u 21 ,u 22 ), where the second set of weight values ​​u 21 u 22 Including the third weight value u 21 and the fourth weight value u 22 The third weight value corresponds to the grid w. i The elements in the set have a third set of index values, where the index j of the channel size C is relative to the second weight value u. 12 Increase by 1, for example, u 21 = w i 3111 And the fourth weight value corresponds to the grid w i The elements in the set have a fourth set of index values, where the channel size index j is reset to 1 due to reaching its maximum value, and the width dimension index 1 is relative to the third weight value u. 21 And by adding 1, for example, u 22 =w i 1211 ;

[0120] like Figure 5C As illustrated in the example, from the weight w i Extract the third set of weight values ​​u from the grid. 21 u 22 And this third set of values ​​u 31 u 32 Rearranged in the second vector u2, for example u3 = (u 31 ,u 32 ), where the third set of weight values ​​u 31 u 32 Including the fifth weight value u 31 and the sixth weight value u 32 The fifth weight value corresponds to the grid w. i The elements in the set have a fifth set of index values, where the index j of the channel dimension C is relative to the fourth weight value u. 12 Increase by 1, for example, u 31 = w i 2211 The sixth weight value corresponds to the grid w. i The element with the sixth set of index values, where the index j of the channel dimension C is relative to the fifth weight value u 31 An increase of 1 was added, for example, u 32 =wi 3211 .

[0121] In the example considered, the rearranged vector sets u1, u2, u3 can produce a matrix U with the first vector u1, the second vector u2, and the third vector u3 as the corresponding first, second, and third columns, which can be represented as:

[0122]

[0123] from Figures 5A to 5C The method of collecting values ​​from grid elements, as illustrated, can essentially follow a “plowing and turning” path along a certain dimension (e.g., channel dimension C), collecting weight values ​​while “plowing” in the grid and rearranging them in the vector whenever the number of “plowing” operations equal to the selected dimension parameter d has been reached.

[0124] In one or more embodiments, the matrix U resulting from the rearrangement of weight values ​​402 can be used to apply a normalization process 404, which may include using stochastic gradient descent (SGD) to solve an optimization problem, which can be represented as:

[0125]

[0126] in

[0127] S represents the set of all training samples.

[0128] a j and b j These are the selected layers (e.g., Figure 1 The input and output activations of layer 102 (shown in the image) are as follows:

[0129] ρ is the activation function (e.g., ReLU) and

[0130] Ω L It is a regularization term calculated based on the value of matrix U generated by rearranging the weight value 402.

[0131] Specifically, the regularization term Ω L (u) can be represented as:

[0132]

[0133] in

[0134] σ 2 It is the statistical variance, the value of which can be set by the user, and

[0135] cw is the codeword vector of grid L used in LVQ processing 406.

[0136] If the k-th column u of matrix U k If the codeword is also a grid L, then such a regularization term is designed to reach a minimum value.

[0137] The selection of the above optimization problem can be based on the following reasons:

[0138] Given the same input activation, the first term of the problem (represented as) This helps to produce values ​​that can adequately approximate the output activations of a CNN; and

[0139] Regularization term Ω L (u) can help to bring the weights closer (reduced distance) to the points (Voronoi centroids) of the selected grid L.

[0140] Figure 6A It is a function Ω L (u) Example of a curve in three-dimensional Cartesian space.

[0141] Figure 6B It is the projection of the previous curve onto different variance values ​​σ1, σ2, σ3, for example, σ1 = 0.25, σ2 = 0.3, σ3 = 0.4.

[0142] Optionally, when calculating the regularization term Ω L When (u), the further scaling factor λ k It can be applied to matrix U to improve the matching of the weight vector to the selected grid L.

[0143] Figures 7A to 7C This illustrates the different distributions of the elements of matrix U (represented by the filled dots in the figure) when different scaling factors λ1, λ2, and λ3 are applied to it relative to a two-dimensional (d=2) grid L. The two-dimensional (d=2) grid L has basis vectors b1 and b2, which can be represented in Cartesian coordinates (the centroid of the V region of grid L is represented by the intersection point in the figure) as: b1 = [1,0]; b2 = [0; 1], for example, λ1 = 1, λ2 = 0.5, λ3 = 0.15.

[0144] In one or more embodiments, reducing the value of the scaling factor λ can increase the element distribution density of matrix U relative to the selected grid L.

[0145] As illustrated in this paper, with or without scaling, solving the normalization problem can provide a normalization matrix U' to further processing stages (such as stage 406, which performs mesh vector quantization LVQ).

[0146] This may involve selecting a grid L with a set of basis vectors based on a chosen dimension parameter d, the grid L being configured to be used as a quantization scheme, wherein the grid L may include a finite set of points indicated as a codeword CB, for example, CB may contain 256 grid points with a minimum norm, where the grid points are codewords cw.

[0147] As illustrated in this paper, applying LVQ to normalized weights involves mapping each column of the normalized matrix to the nearest codeword in the codebook CB.

[0148] As those skilled in the art will understand, the methods discussed in the literature Conway, J., and N. Sloane: “Fast quantizing and decoding and algorithms for lattice quantizers and codes”, IEEE Transactions on Information Theory 28.2(1982):227-232 are advantageous in performing such mappings. This literature discusses a very fast algorithm for finding the grid points that are closest to any arbitrary point for each grid A_{n}(n geq 1), D_{n}(n geq 2), E_{6}, E_{7}, E_{8} and their doubly closest grid points, such that if these grids are used for vector quantization of uniformly distributed data, the algorithm finds the minimum distortion grid point, and if the grids are used as codes for a Gaussian channel, the algorithm performs maximum likelihood decoding.

[0149] Figure 8 This is a graph example of such mapping processing levels.

[0150] like Figure 8 As illustrated, any weight vector u1, u2, u3, u k Weight compression involves:

[0151] As indicated by the arrows starting from the corresponding weight vector points, the weight vectors of the grid points closest to the selected grid L are projected, with the projection representing the shift given by the optimization; and

[0152] As shown by the dashed arrow indicating the corresponding intersection point in grid L, vector quantization is performed towards the nearest grid point.

[0153] The method discussed in the literature Rault, P. and Guillemot, C.: “Indexing algorithms for Zn, An, Dn, and Dn++lattice vector quantizers”, Multimedia, IEEE Transactions on. 3.395-404, doi:10.1109 / 6046.966111 has been found to be advantageous at indexed grid point 408.

[0154] As illustrated in this paper, index grid point 408 may include encoding any grid point x as an index tuple (i s i a i a LUT This can include:

[0155] Normalize grid point x with respect to the fixed maximum norm of grid L;

[0156] Calculate the signed leader of x. s and absolute leader a ;

[0157] Based on grid point x and the calculated signature prefix l s Calculate the first index i of the index tuple of grid point z. s ;

[0158] Based on the calculated signature prefix l s and the calculated absolute first element l a Calculate the second index i of grid point x. s ;as well as

[0159] The calculated absolute first element l a It is stored in a lookup table (LUT), and the third index i is calculated based on the LUT. a LUT .

[0160] As illustrated in this paper, index 408 can be implemented such that storing the indexed codewords uses a reduced amount of memory compared to storing the complete codebook, thereby facilitating the overall compression of network weights.

[0161] The memory impact of uncompressed weight values ​​can be estimated by giving a 32-bit (assuming floating-point representation) quantity for each weight value, the number of bits multiplied by the number of weights.

[0162] The memory footprint of using method 40 and index 407 can be used only for storing tuples containing the LUT and absolute first values, as well as index values, which helps reduce the memory footprint of CNN processing. The NPU 20 can then use compressed weights to further reduce the memory used to store such weights. Tables I and II below provide estimates of the memory impact of method 40 as illustrated herein.

[0163] Table I

[0164]

[0165]

[0166] Table II

[0167]

[0168] In an alternative embodiment, the value of the first element can be generated using a generating function instead of being stored in the LUT.

[0169] As those skilled in the art will understand, examples such as Moureaux, J.-M, Loyer, P. and Antonini, M., "Low Complexity Indexing Method for Z," can be found. n and D n The method discussed in “Lattice Quantizers”, (1999), Communications, IEEE Transactions on. 46.1602–1609, doi:10.1109 / 26.737398 is suitable for this purpose.

[0170] Table III below summarizes the time (in seconds) for computationally generating the corresponding 256 and 2048 absolute principal elements in different dimensions.

[0171] Table III

[0172]

[0173] As discussed above, the circuitry 206, 230 of the NPU 20 can be configured for:

[0174] For example, via direct memory access (DMA) circuitry system 206, weight values ​​compressed using, for example, the method 40 illustrated herein can be accessed / retrieved from (smaller) memory circuitry.

[0175] For example, during flight, the retrieved normalized weight values ​​are decompressed (230); and

[0176] Improved CNN processing is performed using the normalized weight values ​​obtained as a result of decompression 230 232, 234, 236.

[0177] As illustrated herein, the NPU circuitry can be configured to perform a decompression method, such as reversing the steps of (compression) method 40. The decompression method can be configured to work in conjunction with the associated (compression) method 40, for example, using the same compression / decompression parameters (e.g., the value of dimension d for a mesh type L).

[0178] Such a decompression method has low complexity from a hardware perspective, which not only facilitates the operation of the neural processing unit by reducing the memory footprint associated with memory storage, but also positively impacts NPU performance in the following ways:

[0179] This helps reduce the amount of data transferred per weight, thereby reducing the processing latency associated with weight transfer; and

[0180] A “tighter” hardware pipeline that facilitates CNN execution.

[0181] As illustrated in this paper, decompressing weights "in flight" or dynamically can refer to decompression logic that may include performing tasks to decode the incoming stream of compressed indices that goes directly into or is attached to the hardware convolutional unit without requiring a large intermediate buffer. This can significantly improve the performance of the NPU unit 20, thereby helping to manage the critical paths that are dominated by memory bottlenecks when executing certain types of neural network workloads and operators, such as fully connected and recurrent networks (RNNs, also known as matrix / vector multiplication), long short-term memory (LSTM), or gated recurrent units (GRUs).

[0182] Figure 9 The embodiments described herein (e.g., those referenced above) can be applied. Figures 1 to 8 The described embodiment is a functional block diagram of an electronic device or system 900. System 900 includes one or more processing cores or circuits 902. Processing core 902 may include, for example, one or more processors, state machines, microprocessors, programmable logic circuits, discrete circuit systems, logic gates, registers, and various combinations thereof. The processing core can control the overall operation of system 900, execute application programs by system 900, etc.

[0183] System 900 includes one or more memories, such as one or more volatile and / or non-volatile memories, which may store, for example, all or part of instructions and data related to the control of system 900, applications and operations performed by system 900, etc. As shown, system 900 includes one or more cache memories 904, one or more primary memories 906, and one or more secondary memories 908, each including a memory management circuitry or MMU 910. One or more of memories 904, 906, and 908 may include a memory array, which may be shared by one or more processes performed by system 900 during operation.

[0184] System 900 may include one or more sensors 920 (e.g., accelerometers, pressure sensors, temperature sensors, etc.), one or more interfaces 930 (e.g., wireless communication interfaces, wired communication interfaces, etc.), one or more ANN circuits (e.g., convolutional neural network (CNN) accelerator circuit 940 as shown), and other circuits 950 (which may include other functional circuits, antennas, power supplies, etc.), as well as a main bus system 960. The main bus system 960 may include one or more data, address, power, and / or control buses coupled to various components of system 900. System 900 may also include additional bus systems, such as bus system 962 communicatively coupling cache memory 904 and processing core 902, bus system 964 communicatively coupling cache memory 904 and primary memory, bus system 966 communicatively coupling primary memory and processing core 902, and bus system 968 communicatively coupling primary memory and secondary memory 908.

[0185] One or more primary memories 906 are typically the working memory of system 900 (e.g., one or more memories for the operation of processing core 902 or one or more memories for the operation of a coprocessor such as a CNN accelerator circuit system 940), and may typically be finite-size volatile memory (e.g., DRAM) storing code and data associated with the processes executed by system 900. For convenience, references to data stored in memory herein may also refer to code stored in memory. In embodiments, non-volatile memory such as SRAM, ReRAM, PCM, MRAM, FRAM, multilevel memory, etc., may be used in one or more primary memories. Secondary memory 908 is typically non-volatile memory storing instructions and data that can be retrieved and stored in primary memory as needed by system 900. Cache memory 904 may be a relatively fast memory compared to secondary memory 908 and typically has a finite size, which may be larger than the size of primary memory 906.

[0186] Cache memory 904 temporarily stores code and data for later use by system 900. Instead of retrieving required code or data from secondary memory 908 to store in primary memory, system 900 can first check cache memory 904 to see if data or code has already been stored there. Cache memory 904 can significantly improve the performance of systems such as system 900 by reducing the time and other resources required to retrieve data and code used by system 900. When code and data are retrieved (e.g., from secondary memory 908) for system 900 use, or when data or code is written (e.g., to primary or secondary memory 908), a copy of the data or code can be stored in cache memory 904 for later use by system 900. Various cache management routines can be used to control the data stored in one or more cache memories 904.

[0187] The following outlines exemplary embodiments, with reference to the accompanying drawings. A computer-implemented method (e.g., 40) as illustrated herein includes:

[0188] An artificial neural network (ANN) processing stage (e.g., 10) is provided, comprising multiple processing layers (e.g., 100, 102, 104, 106, 108) with corresponding processing layer parameters (e.g., L1, L2, L3, L4), wherein the processing layer parameters include at least one set of weight parameters (e.g., w). i ), at least one input activation parameter (e.g., a) j ), at least one output activation parameter (e.g., b) j and at least one activation function parameter (e.g., ρ);

[0189] Set the dimension parameter (e.g., d) of the grid to an integer value. The grid has multiple grid points and is identified by a set of basis vectors (e.g., b1, b2).

[0190] Select (e.g., 400) the set of weight parameters (e.g., w) of the corresponding processing layer (e.g., 102) among the plurality of processing layers of the ANN processing level. i );

[0191] Vectorization processing (e.g., 402) is applied to the selected set of weight parameters to produce a set of weight vectors (e.g., u1, u2, u3) as a result and the set of weight vectors is arranged as items of a weight vector matrix (e.g., U).

[0192] Perform normalization processing (e.g., 404) on the weight vector matrix to produce a normalized weight vector matrix (e.g., U') as the result;

[0193] The normalized weight vector matrix is ​​processed by grid vector quantization (LVQ) to produce a codebook (e.g., CB) as the result.

[0194] The indexing process (e.g., 408) should be applied to the resulting codebook, the indexing comprising encoding the codewords of the codebook according to a grid L, thereby producing corresponding index tuples (e.g., i...). s i a i a LUT As a result; and

[0195] The generated index tuple is provided to the user circuitry (e.g., 20).

[0196] The normalization process of the weight vector matrix includes calculating an optimization problem, which has the following characteristics:

[0197] The first item is configured to provide normalized weight values ​​based on at least one input activation parameter of the ANN processing level, the normalized weight values ​​approximating at least one output activation parameter of the ANN processing level; and

[0198] Regularization term Ω L It is configured to amplify the normalized weight values ​​of the short distances from the grid points of the selected grid.

[0199] As illustrated in this article, the regularization term Ω L (u) is configured as the term u in the distance weight vector matrix. k The minimum value is reached when the distance between the grid points of the selected grid is negligible.

[0200] As illustrated in this paper, the optimization problem is expressed as:

[0201]

[0202] in

[0203] S indicates the processing layer parameters of the selected processing layer (e.g., 102) among the plurality of processing layers of the artificial neural network.

[0204] a j and b j These are the input and output activation parameters of the selected processing layer among the plurality of processing layers of the artificial neural network.

[0205] ρ is the activation function parameter of the selected processing layer among the plurality of processing layers of the artificial neural network.

[0206] u k These are terms of the weight vector matrix, and

[0207] λ is the scaling factor.

[0208] As illustrated in this article, the regularization term Ω L (u) can be represented as:

[0209]

[0210] in

[0211] σ 2 It is a statistical variance parameter, and

[0212] cw is the basis vector in the basis vector set that identifies the mesh.

[0213] As illustrated in this article, the regularization term Ω is calculated. L (u) includes applying scaling factors (e.g., λ1, λ2, λ3) to the normalized weight vector matrix, with values ​​between 0 and 1, for example.

[0214] As illustrated herein, the normalization process includes using stochastic gradient descent (SGD) to compute the optimization problem.

[0215] As illustrated in this article, performing the vectorization of the weight values ​​includes:

[0216] Processing elements (e.g., w) from at least one set of weight parameters along an ordered sequence (e.g., C, T, H, K) of a certain dimension. i );

[0217] Weight values ​​are extracted from at least one set of weight parameters along an ordered sequence along the said dimension; and

[0218] A set of weight vectors (e.g., u1, u2, u3) is provided by arranging the extracted weight values ​​as elements of the corresponding vectors in the set of weight vectors, wherein the weight vectors in the set of weight vectors have corresponding vector dimensions equal to the integer value of the dimension parameter (e.g., d).

[0219] As illustrated herein, the LVQ processing includes:

[0220] Based on the integer value of the dimension parameter (e.g., d), select a grid and a set of basis vectors with multiple grid points; and

[0221] Quantization is applied to the weight vector set using the base vector set of the selected grid.

[0222] As illustrated in this article, the ANN processing level is a Convolutional Neural Network (CNN) processing level.

[0223] The computer program product illustrated herein includes instructions that, when executed by a computer, cause the computer to perform the methods illustrated herein.

[0224] The computer-readable medium illustrated herein has been stored, in which normalized weight values ​​are obtained using methods illustrated herein.

[0225] As illustrated in this article, methods for manipulating hardware accelerator engines configured to perform artificial neural network (ANN) processing based on weight values ​​include:

[0226] Access (e.g., 206) the normalized weight values ​​obtained using the method illustrated herein;

[0227] Decompress the accessed normalized weight values ​​(e.g., 230) to produce decompressed weight values; and

[0228] Artificial neural network (ANN) processing is performed based on the decompressed weight values ​​(e.g., 10).

[0229] The computer program product illustrated herein includes instructions that, when executed by a computer, cause the computer to perform methods for operating a hardware accelerator engine.

[0230] As illustrated herein, computer-readable media include instructions that, when executed by a computer, cause the computer to perform methods of operating a hardware accelerator engine.

[0231] The hardware accelerator engine circuitry illustrated herein (e.g., 20) includes a memory circuitry system in which the following is stored:

[0232] The normalized weight values ​​obtained using the method; and

[0233] When executed in the hardware accelerator engine, it causes the hardware accelerator engine to execute the following instructions:

[0234] Access (e.g., 206) the normalized weight values;

[0235] Decompress the accessed normalized weight values ​​(e.g., 230) to produce decompressed weight values; and

[0236] ANN processing is performed based on the decompressed weight values ​​(e.g., 10).

[0237] It should be understood that the various implementations illustrated throughout the accompanying drawings are not necessarily intended to be employed in the same combinations illustrated in the drawings. Therefore, one or more embodiments may employ these (non-mandatory) options individually and / or in different combinations relative to the combinations illustrated in the drawings.

[0238] Details and embodiments may vary significantly from those described by way of example only, without prejudice to the underlying principles and without departing from the scope of protection. The scope of protection is defined by the appended claims.

[0239] In an embodiment, the computer-implemented method includes: providing an artificial neural network (ANN) processing level, the ANN processing level including multiple processing layers with corresponding processing layer parameters, the processing layer parameters including at least one set of weight parameters, at least one input activation parameter, at least one output activation parameter, and at least one activation function parameter; setting the dimension parameter of a grid to an integer value, the grid having multiple grid points and identified by a set of basis vectors; selecting the weight parameter set of the corresponding processing layer from the multiple processing layers of the ANN processing level; applying vectorization processing to the selected weight parameter set to produce a weight vector set as a result and arranging the weight vector set as items of a weight vector matrix; performing normalization processing on the weight vector matrix to produce a normalized weight vector matrix as a result; applying grid vector quantization (LVQ) processing to the normalized weight vector matrix to produce a codebook of codewords as a result; applying indexing processing to the generated codebook, the indexing including encoding the codewords of the codebook according to the grid L, producing a corresponding index tuple as a result; and providing the generated index tuple to a user circuit, wherein performing normalization processing on the weight vector matrix includes calculating an optimization problem having a first term and a regularization term Ω. L The first term is configured to provide normalized weight values ​​based on at least one input activation parameter of the ANN processing level, the normalized weight values ​​approximating at least one output activation parameter of the ANN processing level; and the regularization term Ω L It is configured to amplify the normalized weight values ​​of the shortest distances from the grid points of the selected grid.

[0240] In an embodiment, the method includes: implementing an artificial neural network (ANN) with multiple processing layers, each processing layer having corresponding parameters, the processing layer parameters including at least one set of weight parameters (w). i ), at least one input activation parameter (a j ), at least one output activation parameter (b) j) and at least one activation function parameter (ρ); set the dimension parameter (d) of the grid to an integer value, the grid has multiple grid points and is identified by a basis vector set (b1, b2); select the weight parameter set (w) of the corresponding processing layer in the multiple processing layers of the ANN. i ); for the selected set of weight parameters (w i The process involves applying vectorization to generate a set of weight vectors (u1, u2, u3) and arranging them into a weight vector matrix (U); normalizing the weight vector matrix (U) to generate a normalized weight vector matrix (U'); applying grid vector quantization (LVQ) to the normalized weight vector matrix (U') to generate a codebook of codewords; and applying indexing to the generated codebook, which involves encoding the codewords in the codebook according to the grid to generate corresponding index tuples (i...). s i a i a LUT ) and providing the generated index tuples (i) to the neural processing circuitry of the ANN. s i a i a LUT The normalization of the weight vector matrix (U) involves solving an optimization problem, which has a first term and a regularization term Ω. L (u): The first term is used to activate the ANN based on at least one input parameter (a). j This provides normalized weight values, which approximate at least one output activation parameter (b) of the ANN. j ); and the regularization term Ω L (u) is configured to amplify normalized weight values ​​that are at a distance from the grid points of the selected grid.

[0241] In the embodiment, in the term u of the weight vector matrix (U) k When the distance between the grid points of the selected grid is negligible, the regularization term Ω L (u) reaches its minimum value. In the embodiment, the optimization problem is expressed as:

[0242]

[0243] in

[0244] S indicates the processing layer parameters of the selected processing layer among the plurality of processing layers of the ANN.

[0245] a j It is the input activation parameter of the selected processing layer among multiple processing layers of an ANN, and bj It outputs the activation parameter.

[0246] ρ is the activation function parameter of the selected processing layer among the multiple processing layers of an ANN.

[0247] u k These are terms of the weight vector matrix, and

[0248] λ is the scaling factor.

[0249] In the embodiment, the regularization term Ω L (u) can be represented as:

[0250]

[0251] in

[0252] σ 2 It is a statistical variance parameter, and

[0253] cw is the basis vector in the basis vector set that identifies the mesh.

[0254] In one embodiment, solving the optimization problem includes applying a second scaling factor to the normalized weight vector matrix (U). In another embodiment, the value of the second scaling factor is between 0 and 1. In yet another embodiment, the normalization process includes using stochastic gradient descent (SGD) to compute the optimization problem. In yet another embodiment, performing the vectorization of the weight values ​​includes processing at least one set of weight parameters (w) along an ordered sequence of dimensions. i Elements in ); an ordered sequence along the dimension, from at least one set of weight parameters (w i The method extracts weight values ​​from the vector set (u1, u2, u3); and provides a weight vector set (u1, u2, u3) by arranging the extracted weight values ​​as elements of the corresponding vectors in the vector set (u1, u2, u3), wherein the weight vectors in the weight vector set (u1, u2, u3) have corresponding vector dimensions equal to an integer value of the dimension parameter (d). In an embodiment, the LVQ process includes: selecting a grid and a base vector set having multiple grid points based on the integer value of the dimension parameter (d); and applying quantization to the weight vector set (u1, u2, u3) using the base vector set of the selected grid. In an embodiment, the ANN is a convolutional neural network (CNN). In an embodiment, the method includes: accessing normalized weight values ​​in a normalized weight vector matrix; decompressing the accessed normalized weight values ​​to produce decompressed weight values; and performing operations based on the decompressed weight values ​​using a hardware accelerator engine of the ANN.

[0255] In an embodiment, the content of a non-transitory computer-readable medium configures a computing system to execute a method comprising: implementing an artificial neural network (ANN) having multiple processing layers, each processing layer having corresponding processing layer parameters, the processing layer parameters including at least one set of weight parameters (w). i ), at least one input activation parameter (a j ), at least one output activation parameter (b) j The grid consists of at least one activation function parameter (ρ); the grid dimension parameter (d) is set to an integer value, the grid has multiple grid points and is identified by a set of basis vectors (b1, b2); the set of weight parameters (w) for the corresponding processing layer in the multiple processing layers of the ANN is selected. i ); for the selected set of weight parameters (w i The process involves applying vectorization to generate a set of weight vectors (u1, u2, u3) and arranging them into a weight vector matrix (U); normalizing the weight vector matrix (U) to generate a normalized weight vector matrix (U'); applying grid vector quantization (LVQ) to the normalized weight vector matrix (U') to generate a codebook of codewords; and applying indexing to the generated codebook, which involves encoding the codewords in the codebook according to the grid to generate corresponding index tuples (i...). s i a i a LUT ) and providing the generated index tuples (i) to the neural processing circuitry of the ANN. s i a i a LUT The normalization of the weight vector matrix (U) involves solving an optimization problem, which has a first term and a regularization term Ω. L (u): The first term is used to activate the ANN based on at least one input parameter (a). j This provides normalized weight values, which approximate at least one output activation parameter (b) of the ANN. j ); and the regularization term Ω L (u) is configured to amplify normalized weight values ​​that are at distance from the grid points of the selected grid. In an embodiment, the optimization problem is expressed as:

[0256]

[0257] in

[0258] S indicates the processing layer parameters of the selected processing layer among the plurality of processing layers of the ANN.

[0259] aj It is the input activation parameter of the selected processing layer among multiple processing layers of an ANN, and b j It outputs the activation parameters.

[0260] ρ is the activation function parameter of the selected processing layer among the multiple processing layers of an ANN.

[0261] u k These are terms of the weight vector matrix, and

[0262] λ is the scaling factor.

[0263] In one embodiment, the method includes: accessing normalized weight values ​​in a normalized weight vector matrix; decompressing the accessed normalized weight values ​​to produce decompressed weight values; and using a hardware accelerator engine of an ANN to perform operations based on the decompressed weight values. In another embodiment, the content includes instructions that, when executed by a computing system, cause the computing system to perform the method.

[0264] In an embodiment, the device includes: a memory; and a processing circuitry system coupled to the memory, wherein the processing circuitry system, in operation,: sets the dimension parameter (d) of a grid to an integer value, the grid having multiple grid points and identified by a set of basis vectors (b1, b2); selects the weight parameter set (w) of a corresponding processing layer among multiple processing layers of an artificial neural network (ANN). i An ANN has multiple processing layers, each with corresponding processing layer parameters. These parameters include at least one set of weight parameters (w). i ), at least one input activation parameter (a j ), at least one output activation parameter (b) j ) and at least one activation function parameter (ρ); for the selected set of weight parameters (w) i The process involves applying vectorization to generate a set of weight vectors (u1, u2, u3) and arranging them into a weight vector matrix (U); normalizing the weight vector matrix (U) to generate a normalized weight vector matrix (U'); applying grid vector quantization (LVQ) to the normalized weight vector matrix (U') to generate a codebook of codewords; and indexing the generated codebook, which involves encoding the codewords in the codebook according to the grid to generate corresponding index tuples (i...). s i a i a LUT The normalization of the weight vector matrix (U) involves solving an optimization problem, which has a first term and a regularization term Ω. L(u): The first term is used to activate the ANN based on at least one input parameter (a). j This provides normalized weight values, which approximate at least one output activation parameter (b) of the ANN. j ); and the regularization term Ω L (u) is configured to amplify normalized weight values ​​that are at a distance from the grid points of the selected grid. In an embodiment, in the weight vector matrix (U), item u... k When the distance between the grid points of the selected grid is negligible, the regularization term Ω L (u) reaches its minimum value. In the embodiment, the optimization problem is expressed as:

[0265]

[0266] in

[0267] S indicates the processing layer parameters of the selected processing layer among the plurality of processing layers of the ANN.

[0268] a j It is the input activation parameter of the selected processing layer among multiple processing layers of an ANN, and b j It outputs the activation parameter.

[0269] ρ is the activation function parameter of the selected processing layer among the multiple processing layers of an ANN.

[0270] u k These are terms of the weight vector matrix, and

[0271] λ is the scaling factor.

[0272] In the embodiment, the regularization term Ω L (u) can be represented as:

[0273]

[0274] in

[0275] σ 2 It is a statistical variance parameter, and

[0276] cw is the basis vector that identifies the grid in the basis vector set.

[0277] In one embodiment, solving the optimization problem includes applying a second scaling factor to the normalized weight vector matrix (U). In another embodiment, the value of the second scaling factor is between 0 and 1. In yet another embodiment, the normalization process includes using stochastic gradient descent (SGD) to compute the optimization problem. In yet another embodiment, performing the vectorization of the weight values ​​includes processing at least one set of weight parameters (w) along an ordered sequence of dimensions.i Elements in ); an ordered sequence along the dimension, from at least one set of weight parameters (w i The LVQ process extracts weight values ​​from the vector set (u1, u2, u3); and provides a weight vector set (u1, u2, u3) by arranging the extracted weight values ​​as elements of the corresponding vectors in the vector set (u1, u2, u3), wherein the weight vectors in the weight vector set (u1, u2, u3) have corresponding vector dimensions equal to an integer value of the dimension parameter (d). In an embodiment, the LVQ process includes: selecting a grid and a base vector set having multiple grid points based on the integer value of the dimension parameter (d); and applying quantization to the weight vector set (u1, u2, u3) using the base vector set of the selected grid. In an embodiment, the ANN is a convolutional neural network (CNN). In an embodiment, the processing circuitry system operates by: accessing normalized weight values ​​in a normalized weight vector matrix; decompressing the accessed normalized weight values ​​to produce decompressed weight values; and performing the ANN operation based on the decompressed weight values.

[0278] In one embodiment, the system includes: a memory for storing compressed, normalized weight values ​​during operation; a processing core; and a hardware accelerator engine coupled to the memory and the processing core, wherein the hardware accelerator engine, during operation,: decompresses the stored, normalized weight values ​​to produce decompressed weight values, and performs artificial neural network (ANN) operations based on the decompressed weight values, wherein the stored, compressed, normalized weight values ​​correspond to solving an optimization problem having a first term and a regularization term Ω. L (u): The first term is used to activate the ANN based on at least one input parameter (a). j This provides normalized weight values, which approximate at least one output activation parameter (b) of the ANN. j ); and the regularization term Ω L (u) is configured to amplify normalized weight values ​​that are distanced from the grid points of the selected grid. In an embodiment, the processing kernel operates by: generating stored compressed normalized weight values, the generation of which includes: setting the dimension parameter (d) of the grid to an integer value, the grid having multiple grid points and identified by a set of basis vectors (b1, b2); selecting the weight parameter set (w) of the corresponding processing layer among the multiple processing layers of the ANN. i An ANN has multiple processing layers, each with corresponding processing layer parameters. These parameters include at least one set of weight parameters (w). i ), at least one input activation parameter (a j ), at least one output activation parameter (b) j) and at least one activation function parameter (ρ); for the selected set of weight parameters (w) i The process involves applying vectorization to generate a set of weight vectors (u1, u2, u3) and arranging them into a weight vector matrix (U); normalizing the weight vector matrix (U) to generate a normalized weight vector matrix (U'); applying grid vector quantization (LVQ) to the normalized weight vector matrix (U') to generate a codebook of codewords; and indexing the generated codebook, which involves encoding the codewords in the codebook according to the grid to generate corresponding index tuples (i...). s i a i a LUT The normalization of the weight vector matrix (U) involves solving the optimization problem.

[0279] In one embodiment, a system includes: a memory for storing compressed, normalized weight values ​​in operation; and a processing circuitry system coupled to the memory. The processing circuitry system in operation: implements an artificial neural network; decompresses the stored normalized weight values ​​to produce decompressed weight values; and performs ANN operations based on the decompressed weight values. The stored compressed, normalized weight values ​​correspond to solving an optimization problem having: a first term for providing normalized weight values ​​based on at least one input activation parameter of the ANN, the normalized weight values ​​approximating at least one output activation parameter of the ANN; and a regularization term for amplifying normalized weight values ​​that are at a distance from grid points of a selected grid. The system uses the ANN to classify images, the classification including performing ANN operations. In an embodiment, the processing circuitry generates the stored compressed, normalized weight values ​​during operation, including: setting the dimension parameter (d) of the grid to an integer value, the grid having multiple grid points and identified by a base vector set (b1, b2); selecting the weight parameter set (w) of the corresponding processing layer from among the multiple processing layers of the ANN. i An ANN has multiple processing layers, each with corresponding processing layer parameters. These parameters include at least one set of weight parameters (w). i ), at least one input activation parameter (a j ), at least one output activation parameter (b) j ) and at least one activation function parameter (ρ); for the selected set of weight parameters (w) iThe process involves applying vectorization to generate a set of weight vectors (u1, u2, u3), arranging these vectors into a weight vector matrix (U); performing normalization on the weight vector matrix (U) to generate a normalized weight vector matrix (U'); applying grid vector quantization (LVQ) to the normalized weight vector matrix (U') to generate a codebook of codewords; and indexing the generated codebook, which involves encoding the codewords of the codebook according to the grid to generate corresponding index tuples (i...). s i a i a LUT The normalization process of the weight vector matrix (U) includes solving an optimization problem. In an embodiment, the processing circuitry includes a processing core and a hardware accelerator engine coupled to the memory and the processing core, wherein the hardware accelerator engine decompresses the stored normalized weight values ​​and performs ANN operations during operation.

[0280] Some embodiments may take the form of a computer program product. For example, according to one embodiment, a computer-readable medium including a computer program adapted to perform one or more of the methods or functions described above is provided. The medium may be a physical storage medium, such as, for example, a read-only memory (ROM) chip, or a disk such as a digital multifunction disk (DVD-ROM), optical disk (CD-ROM), hard disk, memory, network, or a portable media article readable by a suitable drive or via a suitable connection, including portable media articles encoded in one or more barcodes or other related codes stored on one or more such computer-readable media and readable by a suitable reading device.

[0281] Furthermore, in some embodiments, some or all of the methods and / or functions may be implemented or provided in other ways, such as being implemented at least in part in firmware and / or hardware, including but not limited to one or more application-specific integrated circuits (ASICs), digital signal processors, discrete circuit systems, logic gates, standard integrated circuits, controllers (e.g., by executing appropriate instructions, and including microcontrollers and / or embedded controllers), field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and devices employing RFID technology and various combinations thereof.

[0282] The various embodiments described above can be combined to provide further embodiments. As needed, aspects of the embodiments can be modified to incorporate concepts from various patents, applications, and publications to provide other embodiments.

[0283] In view of the foregoing description, these and other changes may be made to the embodiments. Generally, the terminology used in the appended claims should not be construed as limiting the claims to the specific embodiments disclosed in the specification and claims, but should be interpreted to include all possible embodiments claimed by such claims and the full scope of their equivalents. Therefore, the claims are not limited by this disclosure.

Claims

1. A method applied in the field of image processing, comprising: Implement an artificial neural network (ANN) with multiple processing layers, each processing layer having corresponding processing layer parameters, the processing layer parameters including at least one set of weight parameters (w). i ), at least one input activation parameter ( a j ), at least one output activation parameter ( b j ) and at least one activation function parameter ( ρ ); Set the dimension parameter (d) of the grid to an integer value. The grid has multiple grid points and is identified by a base vector set (b1, b2). Select the weight parameter set (w) of the corresponding processing layer from the plurality of processing layers of the ANN. i ); For the selected set of weight parameters (w) i The weight vector set (u1, u2, u3) is generated by applying vectorization processing, and the weight vector set (u1, u2, u3) is arranged into a weight vector matrix (U). Perform normalization on the weight vector matrix (U) to produce a normalized weight vector matrix (U'). The normalized weight vector matrix (U') is subjected to grid vector quantization (LVQ) to generate a codebook of codewords; The generated codebook is indexed, the indexing process including encoding the codewords of the codebook according to the grid to generate corresponding index tuples. i s , i a , i a LUT ); as well as The generated index tuples are provided to the neural processing circuitry of the ANN. i s , i a , i a LUT ), The normalization process of the weight vector matrix (U) includes: solving an optimization problem, wherein the optimization problem has: The first item is used to activate the ANN based on the at least one input parameter ( a j ) to provide normalized weight values, the normalized weight values ​​being approximately equal to the at least one output activation parameter of the ANN ( b j );as well as Regularization term Ω L ( u This is used to amplify the normalized weight values ​​that are at a distance from the grid points of the selected grid. The method further includes: The image is classified using the ANN, and the classification includes performing the operations of the ANN.

2. The method according to claim 1, wherein in the terms of the weight vector matrix (U) u k When the distance between the grid points of the selected grid is negligible, the regularization term Ω L ( u () Reach the minimum value.

3. The method according to claim 1, wherein the optimization problem is expressed as: in S indicates the processing layer parameter of the selected processing layer among the plurality of processing layers of the ANN. a j The input activation parameter is the processing layer parameter of the processing layer selected from the plurality of processing layers of the ANN, and b j It is the output activation parameter, ρ It is the activation function parameter of the processing layer parameter of the processing layer selected from the plurality of processing layers of the ANN. u k These are the terms of the weight vector matrix, and λ It is the scaling factor.

4. The method according to claim 3, wherein the regularization term Ω L ( u ) is represented as: in σ 2 It is a statistical variance parameter; and cw is the basis vector in the basis vector set that identifies the mesh.

5. The method according to claim 4, wherein solving the optimization problem comprises: A second scaling factor is applied to the normalized weight vector matrix (U).

6. The method of claim 5, wherein the value of the second scaling factor is between 0 and 1.

7. The method according to claim 1, wherein the normalization process comprises: The optimization problem is solved using stochastic gradient descent (SGD).

8. The method of claim 1, wherein performing the vectorization of the weight values ​​comprises: Process the at least one set of weight parameters (w) along an ordered sequence of dimensions. i Elements in ) Along the ordered sequence of said dimension, from the at least one set of weight parameters (w i Extract weight values; as well as The weight vector set (u1, u2, u3) is provided by arranging the extracted weight values ​​as elements of the corresponding vectors in the vector set (u1, u2, u3), wherein the weight vectors in the weight vector set (u1, u2, u3) have corresponding vector dimensions equal to the integer value of the dimension parameter (d).

9. The method of claim 1, wherein the LVQ processing comprises: Based on the integer value of the dimension parameter (d), select the grid having multiple grid points and a set of basis vectors; as well as Quantization is applied to the weight vector set (u1, u2, u3) using the base vector set of the selected grid.

10. The method according to claim 1, wherein the ANN is a convolutional neural network (CNN).

11. The method according to claim 1, comprising: Access the normalized weight values ​​in the normalized weight vector matrix; Decompress the accessed normalized weight values ​​to produce decompressed weight values; as well as The operation is performed using the hardware accelerator engine of the ANN, based on the decompressed weight values.

12. A non-transitory computer-readable medium having content that configures a computing device to perform a method applied in the field of image processing, the method comprising: Implement an artificial neural network (ANN) with multiple processing layers, each processing layer having corresponding processing layer parameters, the processing layer parameters including at least one set of weight parameters (w). i ), at least one input activation parameter ( a j ), at least one output activation parameter ( b j ) and at least one activation function parameter ( ρ ); Set the dimension parameter (d) of the grid to an integer value. The grid has multiple grid points and is identified by a base vector set (b1, b2). Select the weight parameter set (w) of the corresponding processing layer from the plurality of processing layers of the ANN. i ); For the selected set of weight parameters (w) i The weight vector set (u1, u2, u3) is generated by applying vectorization processing, and the weight vector set (u1, u2, u3) is arranged into a weight vector matrix (U). Perform normalization on the weight vector matrix (U) to produce a normalized weight vector matrix (U'). The normalized weight vector matrix (U') is subjected to grid vector quantization (LVQ) to generate a codebook of codewords; The generated codebook is indexed, the indexing process including encoding the codewords of the codebook according to the grid to generate corresponding index tuples. i s , i a , i a LUT ); as well as The generated index tuples are provided to the neural processing circuitry of the ANN. i s , i a , i a LUT ), The normalization process of the weight vector matrix (U) includes: solving an optimization problem, wherein the optimization problem has: The first item is used to activate the ANN based on the at least one input parameter ( a j ) to provide normalized weight values, the normalized weight values ​​being approximately equal to the at least one output activation parameter of the ANN ( b j );as well as Regularization term Ω L ( u This is used to amplify the normalized weight values ​​that are at a distance from the grid points of the selected grid. The method further includes: The image is classified using the ANN, and the classification includes performing the operations of the ANN.

13. The non-transitory computer-readable medium of claim 12, wherein the optimization problem is expressed as: in S indicates the processing layer parameter of the selected processing layer among the plurality of processing layers of the ANN. a j The input activation parameter is the processing layer parameter of the processing layer selected from the plurality of processing layers of the ANN, and b j It is the output activation parameter, ρ It is the activation function parameter of the processing layer parameter of the processing layer selected from the plurality of processing layers of the ANN. u k These are the terms of the weight vector matrix, and λ It is the scaling factor.

14. The non-transitory computer-readable medium of claim 12, wherein the method comprises: Access the normalized weight values ​​in the normalized weight vector matrix; Decompress the accessed normalized weight values ​​to produce decompressed weight values; as well as The operation is performed using the hardware accelerator engine of the ANN, based on the decompressed weight values.

15. The non-transitory computer-readable medium of claim 12, wherein the content includes instructions that, when executed by the computing system, cause the computing system to perform the method.

16. An apparatus for image classification, comprising: Memory; as well as A processing circuitry, coupled to the memory, wherein the processing circuitry implements an artificial neural network (ANN) in operation to classify images, the processing circuitry is configured to: Set the dimension parameter (d) of the grid to an integer value. The grid has multiple grid points and is identified by a base vector set (b1, b2). Select the weight parameter set (w) of the corresponding processing layer from the multiple processing layers of the ANN. i The ANN has multiple processing layers, each with corresponding processing layer parameters, including at least one set of weight parameters (w). i ), at least one input activation parameter ( a j ), at least one output activation parameter ( b j ) and at least one activation function parameter ( ρ ); For the selected set of weight parameters (w) i The weight vector set (u1, u2, u3) is generated by applying vectorization processing, and the weight vector set (u1, u2, u3) is arranged into a weight vector matrix (U). Perform normalization on the weight vector matrix (U) to produce a normalized weight vector matrix (U'). The normalized weight vector matrix (U') is subjected to grid vector quantization (LVQ) to generate a codebook of codewords; as well as The generated codebook is indexed, the indexing including encoding the codewords of the codebook according to the grid to generate corresponding index tuples ( i s , i a , i a LUT The normalization process of the weight vector matrix (U) includes: solving an optimization problem, which has: The first item is used to activate the ANN based on the at least one input parameter ( a j ) to provide normalized weight values, the normalized weight values ​​being approximately equal to the at least one output activation parameter of the ANN ( b j ); as well as Regularization term Ω L ( u This is used to amplify the normalized weight values ​​that are at a distance from the grid points of the selected grid. The processing circuit system is further configured to: The image is classified using the ANN, and the classification includes performing the operations of the ANN.

17. The device of claim 16, wherein in the terms of the weight vector matrix (U) u k When the distance between the selected grid points is negligible, the regularization term Ω L ( u () Reach the minimum value.

18. The apparatus of claim 16, wherein the optimization problem is expressed as: in S indicates the processing layer parameter of the selected processing layer among the plurality of processing layers of the ANN. a j The input activation parameter is the processing layer parameter of the processing layer selected from the plurality of processing layers of the ANN, and b j It is the output activation parameter, ρ It is the activation function parameter of the processing layer parameter of the processing layer selected from the plurality of processing layers of the ANN. u k These are the terms of the weight vector matrix, and λ It is the scaling factor.

19. The device of claim 16, wherein the regularization term Ω L ( u ) is represented as: in σ 2 It is a statistical variance parameter; and cw is the basis vector in the basis vector set that identifies the mesh.

20. The apparatus of claim 19, wherein solving the optimization problem comprises: A second scaling factor is applied to the normalized weight vector matrix (U).

21. The device of claim 20, wherein the value of the second scaling factor is between 0 and 1.

22. The apparatus of claim 16, wherein the normalization process comprises: The optimization problem is solved using stochastic gradient descent (SGD).

23. The device of claim 16, wherein performing the vectorization of the weight values ​​comprises: Process the at least one set of weight parameters (w) along an ordered sequence of dimensions. i Elements in ) Along the ordered sequence of said dimension, from the at least one set of weight parameters (w i Extract weight values; as well as The weight vector set (u1, u2, u3) is provided by arranging the extracted weight values ​​as elements of the corresponding vectors in the vector set (u1, u2, u3), wherein the weight vectors in the weight vector set (u1, u2, u3) have corresponding vector dimensions equal to the integer value of the dimension parameter (d).

24. The apparatus of claim 16, wherein the LVQ process comprises: Based on the integer value of the dimension parameter (d), select the grid and the base vector set having multiple grid points; as well as Quantization is applied to the weight vector set (u1, u2, u3) using the base vector set of the selected grid.

25. The device of claim 16, wherein the ANN is a convolutional neural network (CNN).

26. The device of claim 16, wherein the processing circuitry system, in operation: Access the normalized weight values ​​in the normalized weight vector matrix; Decompress the accessed normalized weight values ​​to produce decompressed weight values; and The ANN operation is performed based on the decompressed weight values.

27. A system for image classification, comprising: The memory stores compressed, normalized weight values ​​during operation; Processing kernel; as well as The hardware accelerator engine is coupled to the memory and the processing core, wherein the hardware accelerator engine, in operation: Decompress the stored normalized weight values ​​to produce decompressed weight values, and The artificial neural network (ANN) performs image classification based on the decompressed weight values, wherein the stored compressed and normalized weight values ​​correspond to solving an optimization problem, which has the following characteristics: The first item is used to activate the ANN based on at least one input parameter ( a j ) to provide normalized weight values, which approximate at least one output activation parameter of the ANN ( b j ); as well as Regularization term Ω L ( u This is used to amplify normalized weight values ​​that are at a distance from the grid points of the selected grid.

28. The system of claim 27, wherein the processing core, in operation, generates the stored compressed, normalized weight values, the generation comprising: The dimension parameter (d) of the grid is set to an integer value. The grid has multiple grid points and is identified by a base vector set (b1, b2). Select the weight parameter set (w) of the corresponding processing layer from the multiple processing layers of the ANN. i The ANN has multiple processing layers, each with corresponding processing layer parameters, including at least one set of weight parameters (w). i The at least one input activation parameter ( a j ), the at least one output activation parameter ( b j ) and at least one activation function parameter ( ρ ); For the selected set of weight parameters (w) i The weight vector set (u1, u2, u3) is generated by applying vectorization processing, and the weight vector set (u1, u2, u3) is arranged into a weight vector matrix (U). Perform normalization on the weight vector matrix (U) to produce a normalized weight vector matrix (U'). The normalized weight vector matrix (U') is processed by grid vector quantization (LVQ) to generate a codebook of codewords; as well as The generated codebook is indexed, the indexing including encoding the codewords of the codebook according to the grid to generate corresponding index tuples ( i s , i a , i a LUT The normalization process of the weight vector matrix (U) includes solving the optimization problem.

29. A system for image classification, comprising: The memory stores compressed, normalized weight values ​​during operation; A processing circuitry system is coupled to the memory, wherein the processing circuitry system, in operation: Implement an artificial neural network (ANN); Decompress the stored normalized weight values ​​to produce decompressed weight values, and The ANN operation is performed based on the decompressed weight values, wherein the stored compressed, normalized weight values ​​correspond to the solution of an optimization problem, which has: The first item is used to provide normalized weight values ​​based on at least one input activation parameter of the ANN, the normalized weight values ​​approximating at least one output activation parameter of the ANN; as well as The regularization term amplifies the normalized weight values ​​that are at a distance from the grid points of the selected grid. as well as The image is classified using the ANN, and the classification includes performing the operations of the ANN.

30. The system of claim 29, wherein the processing circuitry generates, during operation, the stored compressed, normalized weight values, the generation comprising: The dimension parameter (d) of the grid is set to an integer value. The grid has multiple grid points and is identified by a base vector set (b1, b2). Select the weight parameter set (w) of the corresponding processing layer from the multiple processing layers of the ANN. i The ANN has multiple processing layers, each with corresponding processing layer parameters, including at least one set of weight parameters (w). i ), at least one input activation parameter ( a j ), the at least one output activation parameter ( b j ) and at least one activation function parameter ( ρ ); For the selected set of weight parameters (w) i The weight vector set (u1, u2, u3) is generated by applying vectorization processing, and the weight vector set (u1, u2, u3) is arranged into a weight vector matrix (U). The weight vector matrix (U) is normalized to produce a normalized weight vector matrix (U'). The normalized weight vector matrix (U') is subjected to grid vector quantization (LVQ) to generate a codebook of codewords; as well as The generated codebook is indexed, the indexing including encoding the codewords of the codebook according to the grid to generate corresponding index tuples ( i s , i a , i a LUT The normalization process of the weight vector matrix (U) includes solving the optimization problem.

31. The system of claim 29, wherein the processing circuitry comprises: Processing kernel; as well as A hardware accelerator engine, coupled to the memory and the processing core, wherein the hardware accelerator engine decompresses the stored normalized weight values ​​and performs the operations of the ANN during operation.

Images