Method and system for spatial locality transform of matrices

Abstract

This invention discloses a method comprising: receiving one or more sizes of each dimension of a core that is convoluted with an input tensor to generate an output excitation; generating a control pattern for computing output values ​​for convoluted with the input tensor, wherein the control pattern is a square matrix, and each dimension is a size equal to the product of the width and height of the core. The control pattern is generated by generating a value at each position of the control pattern, the value being based on a location in the control pattern and the one or more sizes of each dimension of the core, the value indicating a position from which the value of a flattened input tensor for convoluted with the core is accessed.

Description

Technical Field

[0001] This invention is generally concerned with matrix operations, and more specifically, with the spatial transformation of matrices. Prior Technology

[0002] Modern neural networks consist of multiple layers. Each layer may contain a large number of input values, which are then transformed to produce outputs (i.e., activators) that serve as inputs to subsequent layers. Typically, these input and output values ​​are represented as matrices (e.g., arrays of values ​​with one or more dimensions). A common transformation performed on these input values ​​is a spin. A spin applies a core (which contains weighted values ​​and can also be represented as a matrix) to neighboring values ​​in the input to produce an output value. This is repeated for all values ​​in the input (as modified by the weights) to produce a set of output values. However, because the core will be computationally expensive when executed using an initial method, it will be computationally expensive due to the need to repeatedly read from neighboring values ​​and traverse or slide over the same input values ​​multiple times to produce multiple outputs.

[0003] Therefore, it is desirable to have a system that can produce an output value by spinning one of the input values ​​modified by weights through a single core operation with higher efficiency. Summary of the Invention

[0004] In one embodiment, a method includes: receiving one or more sizes of each dimension of a core that is convoluted with an input tensor to produce an output excitation; generating a control pattern for computing output values ​​for convoluted with the input tensor, the control pattern being a square matrix wherein each dimension is a size equal to the product of the width and height of the core; generating the control pattern by generating a value for each position of the control pattern, the value being based on a location in the control pattern and the one or more sizes of each dimension of the core, the value indicating a position from which the value of the flattened input tensor for convoluted with the core is accessed.

[0005] In another embodiment, a system includes a processor having a control pattern generator configured to: receive one or more sizes of the dimensions of a core that is convoluted with an input tensor to generate an output excitation; generate a control pattern for calculating an output value for convoluted with the input tensor, the control pattern being a square matrix wherein each dimension is a size equal to the product of the width and height of the core; generate the control pattern by generating a value at each position of the control pattern, the value being based on a location in the control pattern and the one or more sizes of the dimensions of the core, the value indicating a position from which the value of a flattened input tensor convoluted with the core is accessed. Simple Explanation of the Diagram

[0006] Figure 1 illustrates a system 100 according to one embodiment for rotating an input tensor via a core spatial localization transformation (SLT) to generate an output excitation.

[0007] Figure 2 is an example of a core-based rotary two-dimensional input according to an embodiment.

[0008] Figure 3 is a flowchart illustrating a method for flattening an input tensor according to one embodiment.

[0009] Figure 4 illustrates an example of flattening an input tensor in the case of a one-dimensional core according to one embodiment.

[0010] Figure 5A illustrates the first part of an example of flattening an input tensor in a two-dimensional core according to one embodiment.

[0011] Figure 5B illustrates the second part of one example of flattening an input tensor in a two-dimensional core according to an embodiment.

[0012] Figure 5C illustrates an example of flattening one of the input tensors of a plurality of input channels according to an embodiment.

[0013] Figure 6A illustrates the first part of an example of flattening an input tensor in a three-dimensional core according to one embodiment.

[0014] Figure 6B illustrates the second part of one example of flattening an input tensor in a three-dimensional core according to an embodiment.

[0015] Figure 7 is a flowchart illustrating a method for generating an extended core according to one embodiment.

[0016] Figure 8 illustrates an example of generating an extended core in the case of a one-dimensional core according to one embodiment.

[0017] Figure 9A illustrates an example of an extended core that generates one of the different two-dimensional cores according to one embodiment.

[0018] Figure 9B illustrates an instance of an extended core generated using row-first extension according to one embodiment.

[0019] Figure 9C illustrates an example of generating an extended core in the case of multiple core filters according to one embodiment.

[0020] Figure 10A illustrates the first part of one example of generating an extended core in the case of a three-dimensional core according to an embodiment.

[0021] Figure 10B illustrates a second part of an example of generating an extended core in the case of a three-dimensional core according to an embodiment.

[0022] Figure 10C illustrates a third part of an example of generating an extended core in the case of a three-dimensional core according to an embodiment.

[0023] Figure 11 is a flowchart illustrating a method for generating a control pattern according to one embodiment.

[0024] Figure 12A illustrates the first part of an example of a conceptual basis for a generation control pattern according to an embodiment.

[0025] Figure 12B illustrates the second part of an example of a conceptual basis for a generation control pattern according to an embodiment.

[0026] Figure 13A illustrates an example of a portion of the values ​​of a control pattern for generating a two-dimensional core according to an embodiment.

[0027] Figure 13B illustrates an example of a portion of the values ​​of a control pattern for generating a three-dimensional core according to an embodiment.

[0028] Figure 13C illustrates an example of a control pattern generated for a core of different dimensions according to an embodiment.

[0029] Figure 14 is a flowchart illustrating a method for generating a spinning output using a flattened input, an extended core, and a control pattern according to one embodiment.

[0030] Figure 15A illustrates the first part of one example of using a flattened input, an expanded core, and a control pattern to generate an output stimulus in the case of a one-dimensional core, according to an embodiment.

[0031] Figure 15B illustrates the second part of one example of using a flattened input, an expanded core, and a control pattern to generate an output stimulus in the case of a one-dimensional core, according to an embodiment.

[0032] Figure 15C illustrates part three of an example of generating an output stimulus using a flattened input, an expanded core, and a control pattern in the case of a one-dimensional core, according to an embodiment.

[0033] Figure 16A illustrates the first part of one example of using a flattened input, an expanded core, and a control pattern to generate an output stimulus in the case of a two-dimensional core, according to an embodiment.

[0034] Figure 16B illustrates the second part of one example of generating an output stimulus using a flattened input, an expanded core, and a control pattern in a two-dimensional core configuration, according to an embodiment.

[0035] Figure 16C illustrates the third part of an example of generating an output stimulus using a flattened input, an expanded core, and a control pattern in a two-dimensional core configuration, according to one embodiment.

[0036] Figure 16D illustrates one of the fourth examples of generating an output stimulus using a flattened input, an expanded core, and a control pattern in a two-dimensional core configuration, according to an embodiment.

[0037] Figure 16E illustrates an example of generating an output stimulus with one of multiple channels using a flattened input, an expanded core, and a control pattern in a two-dimensional core configuration, according to one embodiment.

[0038] Figure 16F illustrates a second part of an example of generating an output stimulus with one of multiple channels in a two-dimensional core configuration using a flattened input, an expanded core, and a control pattern, according to one embodiment.

[0039] Figure 17A illustrates the first part of one example of using a flattened input, an expanded core, and a control pattern to generate an output stimulus in the case of a three-dimensional core, according to an embodiment.

[0040] Figure 17B illustrates the second part of one example of using a flattened input, an expanded core, and a control pattern to generate an output stimulus in the case of a three-dimensional core, according to an embodiment.

[0041] Figure 18A illustrates a hardware diagram of an exemplary component for generating an extended core according to an embodiment.

[0042] Figure 18B illustrates a hardware diagram of an exemplary shifter circuit used to generate an extended core in an exemplary component according to an embodiment.

[0043] Figure 19 illustrates a hardware diagram of an exemplary component for generating a control pattern according to an embodiment.

[0044] Figure 20 illustrates a hardware diagram of one of the exemplary components for performing multiply-accumulate operations to generate output excitation according to an embodiment.

[0045] Figure 21 illustrates an exemplary component layout for computing output stimuli in a machine learning processor according to one embodiment.

[0046] Figure 22A illustrates an example machine learning processor according to one embodiment.

[0047] Figure 22B illustrates an instance of a machine learning processor according to one of another embodiments.

[0048] Figure 23 is a block diagram illustrating a component of an instance computing machine capable of reading instructions from a computer-readable medium and executing those instructions in a processor (or controller).

[0049] The illustrations and descriptions in the [Implementation] section are for illustrative purposes only and are not intended to limit the various embodiments described. Implementation

[0050] Cross-reference to related applications This application claims priority to U.S. Provisional Patent Application No. 62 / 769,444, filed November 19, 2018, the entire contents of which are incorporated herein by reference.

[0051] The figures and the following description are for illustrative purposes only and represent preferred embodiments. Those skilled in the art will consider alternative embodiments of the structures and methods disclosed herein as feasible alternatives that can be employed without departing from the principles of the invention.

[0052] Several embodiments are now illustrated in detail with reference to examples thereof in the accompanying drawings. It should be noted that, where feasible, similar or identical element symbols may be used in the drawings and may indicate similar or identical functionality. The drawings depict embodiments of the disclosed system (or method) for illustrative purposes only. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles described herein. [Exemplary System] [ ]

[0053] Figure 1 illustrates a system 100 according to one embodiment for revolving an input tensor via a core spatial localization transformation (SLT) to generate an output stimulus. In one embodiment, system 100 includes an input tensor 102, a core 110, an input stream generator 122, a flattened input stream 124, an extended core generator 126, an extended core 128, a control pattern generator 134, a control pattern 136, a multiply-accumulate unit 140, and an output stimulus 142. However, in other embodiments, system 100 includes different elements, and / or system 100 includes a different number of the elements shown.

[0054] The input tensor 102 is a set of input values ​​108 modified by core 110 in the spin operation to produce the output stimulus 142. In one embodiment, the input tensor 102 is represented by a matrix. The matrix may have one, two, three, or more dimensions and may be stored as an array in memory. The dimension of the array may be equal to the dimension of the matrix. In another embodiment, the input tensor 102 has a plurality of input channels 104A to 104N (collectively referred to as (a plurality of) input channels 104). Each input channel 104 contains a matrix of one or more dimensions, such as dimensions 1 to 3, 106A to 106C, of ​​the matrix in the illustrated input channel 104A.

[0055] A core 110 is applied to the input tensor 102 in a spin operation to generate output stimuli, such as output stimuli 142. Additional details regarding the spin operation will be described below with reference to Figure 2. The core 110 may be represented by one or more dimensions of a matrix. The matrix contains core values ​​114, which, in the context of a neural network, represent the weights of the input values ​​108 applied to the input tensor 102. The dimension of the input tensor 102, and specifically, the dimension of the matrices for each input channel 104, should be at least equal to or greater than the dimension of the core 110. The size (i.e., the number of elements of the core 110 across each dimension) may be less than or greater than the input tensor 102. If the core 110 is larger than the input tensor 102, the input tensor 102 may be padded so that the size of the core 110 is less than or equal to the padded input tensor 102 to allow the spin operation to occur. The resulting output stimuli 142 are matrices of the same size as the (padded) input tensor 102.

[0056] In one embodiment, core 110 includes one or more core "filters" 112A to 112M (collectively referred to as (a plurality of) core filters 112). The number of core filters 112 need not be equal to the number of input channels 104. Each core filter 112 includes a set of sub-filter cores, which are themselves filters and are equal to the number of input channels 104. A spin operation is performed on each input channel 104 using each sub-filter core, and the resulting output matrix is ​​summed to produce a single output excitation feature matrix or output features 144A to 144M (collectively referred to as (a plurality of) output features 144). This is repeated for each core filter 112. The number of output features 144 produced is equal to the number of core filters 112 present in core 110. Thus, for example, if core 110 contains one core filter 112, a single output feature 144 is produced. However, if core 110 contains five core filters 112, the output excitation 142 will have five output features 144. This allows the neural network to apply different core weights to different portions of the input (i.e., different input channels 104) and combine the results into novel outputs (i.e., different output features 144), which can then be used as further inputs in another layer of the neural network. Additional details regarding the use of the multiple input channels 104 and the multiple core filters 112 will be described below with reference to Figures 16E to 16F.

[0057] Input stream generator 122 transforms input tensor 102 into flattened input stream 124. To read input tensor 102 more efficiently and avoid reading the same value of input tensor 102 multiple times when core 110 traverses input tensor 102, input stream generator 122 transforms input tensor 102 into flattened input stream 124, which can then be efficiently fed to a buffer or other memory for access by multiply-accumulate unit 140 (described below) to significantly reduce the number of times the input value needs to be read.

[0058] To perform the transformation of the input tensor, the input stream generator 122 may first fill the input tensor 102. The input tensor 102 may be filled such that its size is a modulus (i.e., a multiple) of the size of the core 110. Additionally, the core 110 has a "focal point." This focal point is the location of the core 110 where the output value that rotates with the core 110 is generated. The input stream generator 122 fills the input tensor 102 such that, when traversing the input tensor 102, the core 110 reaches each input value in the input tensor 102. Thus, for example, if the core is a 3×3 core (where the focal point is the center of the core), the two-dimensional matrix of the input tensor 102 may be further filled around the outer boundary of the matrix using a single vector of filler values ​​after being filled to a modulus of the core, allowing the focal point of the core to slide across the input values ​​at the outer edge of the input tensor 102.

[0059] Finally, the input stream generator 122 may also pad the input tensor 102 to meet any requirements of a processor performing a spin operation. Due to factors such as bus size, number of parallel processors, memory size, or other constraints, the input stream generator 122 may further pad the input tensor 102 with padding values ​​such that the input tensor 102, after being flattened into a flattened input stream 124, meets the processor's constraints. In one embodiment, the input stream generator 122 uses padding values ​​equal to the size of core 110 (or the size of each sub-filter core of each core filter 112) to pad one side (e.g., the trailing edge) of each dimension of the input tensor 102 (or each input channel 104 of the input tensor 102). The padding values ​​described herein may be any null, zero, or standard padding values.

[0060] After filling the input tensor 102, the input stream generator 122 divides or segments the filled input tensor 102 into several blocks. Each block is the size of the core 110. Therefore, the filled matrix of the input tensor 102 is divided into several individual smaller matrices, each the size of the core 110. Although the input tensor 102 is described here as being divided, this does not mean that the input stream generator 122 must generate new matrices for each block. Rather, the input stream generator 122 may simply define the boundaries of each block within the matrix of the input tensor 102.

[0061] For the multiple input channels 104, the input stream generator 122 divides the matrix of each input channel 104 into multiple blocks of itself. The size of each block of the input channel 104 is equal to the size of the sub-filter core of each core filter 112 applied to that input channel 104.

[0062] After partitioning the input tensor 102, the input stream generator 122 identifies a flattening order. This is the order in which the values ​​of the blocks in the input tensor 102 are read. The order can be any order and can include column-priority, row-priority, diagonal-priority, aisle-priority, etc. In column-priority order, the values ​​in each column are read in a specific order (e.g., from left to right), and then in a specific order (e.g., from top to bottom). In row-priority order, rows are read in a specific order (e.g., from left to right) instead of columns, where the values ​​in each column are read in a specific order of each row (e.g., from top to bottom). In diagonal-priority order, blocks can be read along the diagonal. If a block contains more than one layer, the layers can be processed sequentially. Other orders are also possible, as long as the same sorting pattern is subsequently used to generate the extended core 128 and the control pattern 136.

[0063] Input stream generator 122 reads the values ​​of each block in a recognized flattening order and configures the values ​​as it reads the values ​​of each block in a single vector, thereby "flattening" the blocks. Input stream generator 122 reads all blocks of input tensor 102 and generates a corresponding number of vectors. The vectors are arranged parallel to each other to generate flattened input stream 124. Input stream generator 122 can read the blocks in a specific order (such as a column-major order, row-major order, etc.). Any order can be used as long as the order of the blocks is reflected when generating extension core 128 and control pattern 136, and a valid output can be generated by multiply-accumulate unit 140.

[0064] If input tensor 102 contains multiple input channels 104, the matrix of each input channel 104 is processed individually to produce one of the flattened input streams of each matrix of each input channel 104. Multiple flattened input streams can be combined together to form (combined) flattened input stream 124. Multiple flattened input streams can be combined by "stacking" them together, concatenating them, or by some other combination method.

[0065] The flattened input stream 124 is the result of the input stream generator 122 flattening the input tensor 102. Regardless of the dimension of the input tensor 102, the flattened input stream 124 of the matrices of the input tensor 102 is (at most) two-dimensional. This is because each block of the input tensor 102 is transformed into a vector (as described above) and the vectors are placed parallel to each other. In one embodiment, if the input tensor 102 contains multiple input channels 104, the flattened input streams generated from the matrices of each input channel 104 can be combined with the flattened input streams of other matrices of other input channels 104 by arranging (in computer-readable memory) the flattened input streams perpendicularly or horizontally adjacent to each other. The combined flattened input streams can also be represented in three dimensions, wherein the flattened input streams generated from each input channel are stacked vertically on top of each other. In memory, this can be represented using a depth-first storage method, where the values ​​of the depth along the three-dimensional flattened input stream are stored in priority order.

[0066] The flattened input stream 124 can be fed into a buffer or stored in memory. It can be read by the multiply-accumulate unit 140 and referenced according to the number of blocks corresponding to each vector in the flattened input stream 124. In the illustrated example, the blocks of the flattened input stream 124 range from 0 to K to J. Here, the block number 1+K represents a block in the original input tensor 102, which is the first block in a second column (or row) block. The value K represents the number of blocks in a column or row of the padded input tensor 102 (depending on the order in which the blocks are read) and varies depending on the width of the input tensor 102. For example, if the core and therefore the blocks are a 3×3 matrix, then an input tensor with a padded width of 9 will have a K value of 3. In this case, the three blocks are fitted along the width of the input tensor; therefore, the ordinal number of the first block in the second column will be the fourth block (because block counting starts from 0). If the input tensor 102 contains additional dimensions, an additional marker is indicated for the first block of each second sequence value of that dimension. For example, if the input tensor 102 contains three dimensions, a single marker M will indicate the number of blocks in a single layer of the input tensor 102, and the block number 1+M will indicate the exponent value of the first block in a second layer of the input tensor 102. These markers and block numbers can be used by the multiply-accumulate unit to reference or point to the correct block in the flattened input stream 124 using the indicator 138 in control pattern 136, as described below. Alternatively, the block number (including the markers) can be used as a clock cycle delay value to allow the multiply-accumulate unit 140 to delay reading values ​​from a buffer containing the flattened input stream 124. This allows the multiply-accumulate unit 140 to similarly reference a specific block in the flattened input stream 124.

[0067] Compared to a standard spin operation that requires reading some values ​​from the input tensor equal to one-number of values ​​in core 110, this access method and other components of system 100 (described in further detail below) allow system 100 to perform only 2^(s-1) reads of the input tensor 102 (where s is the dimension of the input), thus saving significant resources. Additional details regarding the input stream generator 122 and the flattened input stream 124 will be described below with reference to Figures 3 through 6B.

[0068] The extended core generator 126 generates an extended core 128, which, together with the flattened input stream 124 and control pattern 136, is used by the multiply-accumulate unit 140 to generate an output stimulus 142. The extended core 128 is generated from core 110. The purpose of "extending" the core is to allow a selected vector of input values ​​from the flattened input stream 124 to be multiplied by a vector (e.g., a row, column, or aisle) of the extended core 128 using a single dot product, instead of requiring the original core 110 to span the input tensor 102, as will be shown below. This significantly simplifies the generation of the output stimulus 142. The extension of core 110 follows a specific pattern involving the generation of rotational combinations of core 110 in a hierarchical manner. In the following description, references are made to a first dimension, an additional dimension, and a final dimension. This is merely one way of referring to the dimensions of the core in an organized manner (since a core can have one or more dimensions) and is not intended to indicate the arrangement or size of the dimensions. For example, if a core is three-dimensional, then the last dimension of the core does not necessarily refer to a specific three-dimensional representation of the core, but only to the core's dimension 3, as long as it has 3 different dimensions (e.g., dimension 1, dimension 2, and dimension 3).

[0069] In one embodiment, the extended core generator 126 employs a first dimension of core 110 (or a core filter 112 or a sub-filter core of core filter 112) and generates a square block containing all rotations of the single-dimensional vectors of the core. The block is generated by placing the rotations parallel to the single-dimensional vectors. For example, a 3×3 two-dimensional core will have three 3×1 single-dimensional vectors of dimension 1. All possible rotations of these vectors are generated for each vector, thus generating two additional single-dimensional vectors for each. These placements parallel to the single-dimensional vectors used to generate the additional vectors generate a square block for each single-dimensional vector. These square blocks may be referred to as cyclic matrices.

[0070] Subsequently, for each additional dimension of the core, the blocks immediately following the previous or lower dimension are divided into several groups. Each group contains blocks immediately following the previous dimension aligned with a vector parallel to the axis of that dimension. Therefore, returning to the example of the 3×3 matrix, if the previously generated square blocks are placed in the same positions as the square blocks from their source (i.e., the one-dimensional vector), then for dimension 2, a vector can pass through all the generated blocks. Therefore, the group for dimension 2 contains all the blocks generated in the previous operation. In a 3×3 core, this contains three blocks, one block for each vector of the core.

[0071] For each group generated, the extended core generator 126 produces all rotations of the blocks within that group. Using the previous example, the three blocks are in a single group of dimension 2. Therefore, the rotations of this group produce two additional combinations of the three blocks (a total of 9 blocks). Placing the two additional combinations parallel to the blocks in the group is similar to the method described above for a single-dimensional vector. Here, with all dimensions of the two-dimensional core considered, the extended core generator 126 ends the generation of the extended core and outputs the combination of 9 blocks as the extended core 128.

[0072] However, if core 110 contains further dimensions, the above procedure is repeated to result in rotations and combinations of different block groups. This ultimately results in all dimensions being considered, and the combination of all blocks with blocks of lower dimensions is output as extended core 128. Thus, the number of groups increases with the dimension. In each additional dimension, after rotating the blocks from the previous dimension aligned along the vector (as described above), the combination of rotated blocks is placed in a new block. These new blocks are used to compute the next additional dimension and are combined in various rotations. This continues until the last dimension, which has one dimension equal to the total dimensions of the core. In the last dimension, a final set of rotations is performed, and the combination of rotations is output as extended core. Thus, the number of blocks for each additional dimension processed is reduced, and after processing the rotations of the final dimension, only a single set of output blocks is retained.

[0073] The number of vectors (e.g., columns or rows) in the extended core 128 is further equal to the number of elements in the core or the number of elements in a block of the input tensor 102. The actual size of the extended core 128 itself depends on the size of the core 110. Each dimension of the two-dimensional extended core 128 has a size equal to the product of the size values ​​of each dimension of the core 110. For example, if the core is a 2×2×2 matrix, then the extended core 128 will have 8 vectors (2^3) and therefore will have a size of 8×8, since the product of the values ​​of the dimensions of the core is 8.

[0074] As described herein, the resulting extended core 128 can be used in a dot product with selected values ​​from the flattened input stream 124 to generate output excitation 142. As previously described, the values ​​in the flattened input stream 124 can be selected via a delay or index using an indicator of control pattern 136. After selecting these values, the selected values ​​can be combined with a selected vector of extended core 128 to generate an output value 146 of output excitation 142 by multiplying each vector of extended core 128 with different selected values ​​from the flattened input stream 124. Since the number of vectors in extended core 128 is equal to the number of elements in a block (which is the same size as core 110, as previously described), the number of output values ​​is also equal to the same number of elements in each block and therefore includes a block of output excitation 142 (or, in the case of multiple core filters, an output matrix of values). The position of a block in the matrix of output excitation 142 has a position corresponding to a block of the same position in the matrix of input tensor 102.

[0075] The system 100 described herein allows the computation of values ​​in each output block without having to repeatedly read the input values ​​as in an initial method (such as that described in Figure 2). Each output block is generated in a single pass through the extended core and a single selection of values ​​from the flattened input stream 124, rather than by repeatedly reading specific values ​​in the input tensor as the core 110 slides over the input tensor 102 to positions corresponding to locations in the output blocks.

[0076] For the multiple core filters 112, an extended core is generated for each core filter 112 and applied to the input, similar to the procedure described above. For the multiple input channels 104, each core filter 112 has multiple sub-filter cores corresponding to each input channel 104. In this case, an extended core is generated for each sub-filter core, and each extended core generated from each sub-filter core is applied to a relevant portion of the flattened input stream 124 corresponding to the input channel 104 (which has already been applied to the input channel 104 in an initial spin-out implementation). Additional details regarding the generation of the extended cores will be described below with reference to Figures 7 through 10C.

[0077] Control pattern generator 134 generates control pattern 136 based on information about the core size in core size input 132. The core size indicates the size of each dimension of core 110 (or each core filter). Control pattern generator 134 obtains this information and generates the same control pattern 136 for cores of the same size and dimension. Control pattern generator 134 generates the value at each location based on the coordinates (e.g., column number, row number) of the control pattern 136 and the size of the core dimension. For each location, control pattern generator 134 performs one or more test inequalities (equal to the core dimension). Each test inequality is an inequality between the modulo operation of the column number and the modulo operation of the row number at that location in control pattern 136. The result of the test inequality (i.e., true or false) is used to reference a table of control pattern values ​​to generate the value at that location in the control pattern.

[0078] Although the control pattern generator 134 is described herein as generating control patterns 136 based on core size input 132 using a stylized method, in other embodiments, the control pattern generator 134 accesses one of the control patterns 136 from memory, non-volatile memory, a stylized instruction stack, or other sources to pre-generate a pattern and selects the correct pre-generated control pattern 136 from this source based on core size input 132.

[0079] Control pattern 136 is a matrix that instructs multiply-add unit 140 which portions of the flattened input stream 124 are selected to produce vector multiplication (using dot product) with the expanded core 128 to produce selected output values. In an initial spin implementation, for each step of core 110 at a location on input tensor 102, spin is performed by summing the values ​​adjacent to the current focus representing the core and the value at the focus itself (as weighted by corresponding values ​​in the core (which are located at the same position)). The resulting sum corresponds to the output value at that focus position. Therefore, for each focus position, different values ​​from input tensor 102 are selected for summation. When the flattened input stream 124 is divided into different blocks, the output positions are calculated using values ​​from an input block at the same location as the output block or from input values ​​in adjacent input blocks, and specifically, the output positions within an output block. Therefore, for each output position within a block, control pattern 136 indicates to multiply-accumulate unit 140 the different specific input blocks from which input values ​​are extracted to perform the spin operation (i.e., the dot product mentioned above). Specifically, each vector of control pattern 136 corresponds to a different position within the output block and indicates the block from which the input matrix (from which it selects input values ​​for calculating the values ​​at the output positions) is located.

[0080] Since the output is generated block by block, control pattern 136 only needs to indicate input blocks having a position corresponding to the current output block being processed and input blocks having adjacent positions corresponding to the current output block being processed. For example, in a two-dimensional matrix of input tensor 102, the blocks are horizontally and vertically adjacent to the current output block being processed. In a three-dimensional matrix, this may include blocks in adjacent layers. For higher-dimensional matrices, this will include further "adjacent" blocks. Therefore, the number of block positions indicated by control pattern 136 is a power of 2 of the dimension of the input matrix. The size of control pattern 136 is the same as the size of the extended core 128 generated from core 110.

[0081] Control pattern 136 indicates blocks based on the block ordinal numbers of each block in the flattened input stream 124. Since blocks adjacent to an input block are not necessarily adjacent in block ordinal number, the indicator in control pattern 136 does not directly indicate blocks by block ordinal number, but rather by their relative position to the block ordinal number of the currently processed block. Therefore, for example, a relative position could indicate the current block, a column (or row) below (or to the right) of the current block, or a layer after the current block. Since the width / height and depth of one of the input tensor matrices 102 are known, the relative position can be calculated based on this information. For example, a block in a column below the current block would be the block ordinal number of the current block plus the width of the matrix. Therefore, control pattern 136 can indicate the column width value, an index, or a reference for this block. A block in a column below the current block and a column to the right of the current block would be indicated by the current block ordinal number plus the column width plus 1. Therefore, control pattern 136 can indicate either column width + 1 or column width + 1 for this block. In one embodiment, the column width is the aforementioned variable K. Each of these combinations of indicators can be indicated by one or more bits. Since control pattern 136 has a fractal pattern, each part of the matrix of control pattern 136 can be indicated by a single principal bit, wherein additional parts are indicated by additional bits and individual values ​​in that part have a second bit. The combination of individual values ​​and the parts to which those values ​​belong indicates the relative block position.

[0082] With respect to the multiple core filters 112, multiple control patterns 136 can be selected when the core filters or sub-filter cores have different dimensions. For each core with a different dimension, a corresponding control pattern 136 will be selected or generated according to the method described above. The corresponding control pattern 136 will be used for some selection values ​​of the self-flattened input stream 124. Additional details regarding the generation and characteristics of the control pattern 136 will be described below with reference to Figures 11 to 13.

[0083] Multiply-accumulate unit 140 uses flattened input stream 124, extended core 128, and control pattern 136 to perform the final operation to generate output stimulus 142. As described in detail above, multiply-accumulate unit 140 uses indicators from control pattern 136 to select values ​​from flattened input stream 124. Each vector in control pattern 136 indicates a specific block of flattened input stream 124 from which an input value will be accessed, as described above. Identifiers at each position in control pattern 136 correspond to an indication of which block (i.e., which vector) of flattened input stream 124 from which an input value will be selected. Positions within vectors in control pattern 136 also correspond to the positions of selected vectors in flattened input stream 124 containing the correct input values. By parsing the entire vector of control pattern 136, multiply-accumulate unit 140 generates a vector of selected input values.

[0084] The multiply-accumulate unit 140 further selects a corresponding vector from the extended core 128, which matches the position (e.g., a column number or row number) of the vector in the control pattern 136 used to select values ​​from the self-flattened input stream 124. The multiply-accumulate unit 140 performs a dot product between the selected vector from the extended core 128 and the vector containing the selected values ​​from the flattened input stream 124 to produce a single output value. The single output value is placed on an output block that matches the position of the currently processed input block. Furthermore, the position of the single output value in the output block corresponds to the number of positions of the vectors in the control pattern 136 (or extended core 128) used to generate that single output value.

[0085] The multiply-accumulate unit 140 repeats the procedure described herein for all vectors in control pattern 136 (and extended core 128) to produce a total number of output values ​​equal to the number of positions within the output blocks. This allows the multiply-accumulate unit 140 to generate one output block of output stimulus 142 for each input block from input tensor 102. The multiply-accumulate unit 140 further repeats this procedure for all input blocks of input tensor 102 to produce the same number of output blocks of output stimulus 142. After generating the entire set of output blocks, the multiply-accumulate unit 140 outputs the entire set of output blocks as the final output of output stimulus 142.

[0086] For the multiple input channels 104, the multiply-accumulate unit 140 uses a specific sub-filter core or core component assigned to the core 110 of each input channel 104 to generate the "pre-output" of that input channel 104. Each pre-output is generated in the same manner as the output described above. However, after generating all the pre-outputs for each input channel, the multiply-accumulate unit 140 sums the values ​​of the pre-outputs into a single output matrix of the output excitation 142.

[0087] With respect to the multiple core filters 112, the multiply-accumulate unit 140 further applies each core filter 112 to the input tensor 102, and if multiple input channels 104 exist, the multiply-accumulate unit 140 applies each core filter 112 to all input channels 104 (as described above) and sums the pre-outputs. This produces a single output feature 144 for each core filter 112. Each output feature 144 is a matrix of the same size as an input channel 104. The set of all output features 144 represents the output stimulus 142 and can be used as an input channel 104 in the next layer of the neural network (i.e., the output stimulus 142 of one layer becomes the input tensor 102 of the next layer of the neural network). Additional details regarding the multiply-accumulate unit 140 will be described below with reference to Figures 14 through 17B. Instantiated spin operation

[0088] Figure 2 is an example of a two-dimensional input with a core spiral according to one embodiment. Although a matrix is ​​used to represent the input value 202 and output value 214 described herein, they can actually be represented using only arrays, planar files, trees or other methods and do not need to be configured as a matrix, as long as the operation result is the same.

[0089] To perform a rotation on a set of input values ​​202, padding can first be applied to the input values ​​202 such that the core's "focal point" can be applied to the values ​​at the edges of the input values ​​202, allowing the output value 214 to be the same size as the input values. However, in other cases, padding is not applied to the input values ​​202, in which case the output value 214 is smaller than the input values ​​202. Compared to the size of the input values ​​202, each side of the output value 214 is reduced on the core side of the same side as the output value 214 by one-third the number of elements between the core's edges and the core's focal point. The edges of the input values ​​202 conceptually represent the positions around the outer boundary of the matrix of input values ​​202 such that no other value is farther from the center of the matrix than the value on this edge. As mentioned above, the core's "focal point" (which can also be called an "origin") is conceptually located at the core position above the current output pixel. Typically, it will correspond to one of the values / elements in the core, and for a symmetrical core (such as the 3×3 core in the example shown), the focus is usually the central element.

[0090] Therefore, to ensure that the position of the output value 214 matches the position of the input value 202, padding is added to the input value such that when crossing the first position in the input value, the core focus causes the focus to be located at the position of the edge value of the input value 202 on the matrix that matches the input value 202. Subsequently, the resulting output value 214 has a position that matches the position of such edge values. This ensures that the size of the output value 214 is the same as the size of the input value 202.

[0091] Therefore, in the illustrated example, a padding 204 of width 1 is added around the edge of the matrix representing the input value 202. Here, the padded input value 202 has a size that is a multiple of the size of the core 210, so no additional padding is needed to allow the size to become a multiple of the core size.

[0092] After filling, core 210 is applied to input value 202. Core 210 is applied by crossing (i.e., moving) core 210 across the entire input value 202. The core is moved across input value 202 according to a step size value. The step size value is a value that determines how far the core is moved for each step. If this value is 1, then the focus of the core is made to cross every possible position of input value 202. Due to filling, core 210 never exceeds the boundary of the filled input value 202 at any time, but the focus of the core may overlap with each 1 of the original value of input value 202. It should be noted that if the step size value is greater than 1, the focus of the core is not applied to each value of input value 202.

[0093] For each input value that is the focus of core 210, a spin operation 212 is performed on the input value 202 that is conceptually located below core 210. Therefore, in the illustrated example, the focus of the core is the selected spin point 208. Here, as illustrated by the heavily weighted dark box 206 surrounding the selected spin point 208, a total of nine values ​​equal in size to core 210 and surrounding the selected spin point 208 are selected. If the focus of core 210 is not at its center, this box will be shifted accordingly so that its position corresponds to the selected spin point 208. For example, if the focus is the upper left corner of core 210, the box will be shifted down one point and right one point so that the selected spin point 208 is located at the upper left corner of the box.

[0094] The spin operation 212 takes each of the input values ​​202 below the bounding box 206 and performs a dot product between the input values ​​202 below the box and the value of core 210 to produce a single output value 214. As shown in the figure, when the focus of core 210 is located at the selected spin point 208, this output of core 210 and input value 202 is "8". Here, due to filling and selecting the focus position, the position of the output value is the same as the position of the selected spin point 208 in the input value 202.

[0095] Next, the core is moved across one of the input values ​​202 (horizontally or vertically) and the spin operation 212 is repeated. The spin operation 212 is completed when the focus of the core 210 has visited all possible input values ​​202. This produces a complete set of output values ​​with the same size as the input values ​​(without padding). This spin operation allows the input values ​​to be modified by weights (cores) and combined with other values ​​in the input to produce a new output.

[0096] In mathematics, the spin operation can be represented as: (1)

[0097] Here, A can be the core, B can be the core, and c is the result of a rotation.

[0098] This spin operation allows a neural network to process complex data and generate a desired output based on that data. However, as shown here, this causes the same input value to be read multiple times when core 210 crosses input value 202, such as instance-repeated read value 218. During a single spin of the input value, instance-repeated read value 218 will be read 9 times in the illustrated instance because 9 core locations will overlap with this value. If additional cores are applied to input value 202 (as described), the same value 218 will be read even more times. Therefore, although spin operation 212 is a powerful tool in machine learning, in one of the initial methods shown here, it can generate a large number of reads (i.e., a large number of I / O operations), which becomes a problem as input value 202 grows and as the number of spins increases. Therefore, as revealed herein, a more optimized method is provided that can significantly reduce the number of repeated reads of input values. As mentioned above, the number of reads can be reduced to 2^(s-1) reads, where s is the dimension of the input. Therefore, in this example, the two-dimensional matrix of input value 202 requires only 2 reads, instead of 9 reads for each value as described above for instance value 218. [Show flat input] [string] [flow] [produce] [ ]

[0099] Figure 3 is a flowchart 300 illustrating a method for flattening an input tensor according to one embodiment. Although the flowchart illustrates a sequence of operations, the operations can be performed in any order and may have more or fewer operations. In one embodiment, the operations illustrated in Figure 3 may be performed by an input stream generator 122.

[0100] Input stream generator 122 receives 310 an input tensor that is revolved by a core. This input tensor may be input tensor 102. The core may be core 110. In one embodiment, input stream generator 122 uses padding values ​​to pad the input tensor such that an output of the revolved input tensor using the core has the same size as the input tensor. In one embodiment, the input tensor with padding values ​​has a size for each padded dimension of the input tensor (which is an integer multiple of the corresponding dimension of the core). In another embodiment, input stream generator 122 uses padding values ​​with a width equal to the size of the core in the corresponding dimension to pad one trailing edge of each dimension of the input tensor. The trailing edge of each dimension is an edge (i.e., one side or the other end) of the input tensor having a maximum exponential value. The padding value may be zero, null, or some other value.

[0101] The input stream generator 122 divides the input tensor into one or more blocks of 320, each block having a size equal to that of the core. Therefore, if the core is a 3×3 core, a 9×9 input tensor (including padding) will be divided into 9 (two-dimensional or 2D) blocks. Similarly, given a 3×3×3 core, a 9×9×9 input tensor will be divided into 27 (three-dimensional or 3D) blocks. In one embodiment, the core does not have a square size, and in this case, the input stream generator 122 divides the input tensor in an order aligned with the core across the input tensor. Therefore, if the core is crossed in a column-first manner (from left to right), the input tensor is divided along each column, before moving to the next column, and so on. Alternatively, in another embodiment, the input stream generator 122 divides the input tensor in an order orthogonal to the core across the input tensor. Therefore, in the above example, the input tensor is first divided along each row (from top to bottom, then to the next row).

[0102] Input stream generator 122 flattens 330 values ​​from one or more blocks into vectors to generate a flattened input stream. For each of the one or more blocks of the input tensor, this may include: accessing the block values ​​in a defined order; arranging the values ​​in a vector according to the defined order; and arranging one or more vectors corresponding to each of the one or more blocks into a parallel configuration to generate the flattened input stream. This defined order may be a column-priority order, a row-priority order, or an aisle-priority order. An aisle-priority order first accesses elements in a 3D block along an axis corresponding to the depth of the 3D block and then along axes corresponding to the width and height of the 3D block. Although the flattened input is presented here as two-dimensional, in other embodiments it contains more dimensions.

[0103] The flattened input stream can be stored in a buffer. The buffer can be read by a hardware-accelerated processor to perform a multiply-add operation with 1) the values ​​in the flattened input stream selected by a control pattern and 2) an extension of one of the cores to produce a spin operation output, without having to load the input tensor values ​​into the buffer multiple times, as described herein.

[0104] Additionally, the input tensor has a complex number of channels, the core has a complex number of filters, and the input channels are rotated with each core filter to produce an output with one of the complex number of output channels.

[0105] The following sections will provide additional examples of flattened input streams relative to the core of the dimensionless diagrams, Figures 4, 5A through 5C, and 6A through 6B.

[0106] Figure 4 illustrates an example 400 of flattening an input tensor in the case of a one-dimensional core according to one embodiment. In one embodiment, the procedure described herein can be executed by an input stream generator 122.

[0107] At 401, input stream generator 122 receives input 420. In the illustrated example, input 420 is represented as a 6×3 set of input values ​​in a matrix. At 402, input stream generator 122 pads input 420 based on the core size. Input stream generator 122 pads input 420 such that the focus of core 422 (which is the center value b) corresponds to the same position of input 420 to produce an output of the same size, as previously described. This causes input stream generator 122 to add padding 426A. Input stream generator 122 also pads input 420 such that it is a multiple (modulus) of the core size. This adds two rows of padding in padding 426B. Finally, in one embodiment, due to hardware requirements, input stream generator 122 uses padding equal to the width of the core to pad the trailing edge of input 420. Here, core 422 is 3-width, and therefore a 3-width padding is added to the end of input 420, resulting in three additional rows of padding at padding 426C.

[0108] At 403, the input stream generator 122 divides the padded input 424 into blocks of the same size as the core 422 to produce a tiled input 428. Since the core is a 3×1 matrix, the padded input 424 is divided into blocks of size 3×1. This results in 12 blocks. Here, the blocks are divided and sorted in column-major order, such that the padded input 424 is divided column by column. However, the blocks can also be divided row by row, in which case block 1 would be [0,0,0], block 2 would be [1,7,13], and so on. As long as subsequent operations follow the same orientation, the resulting output will be the same.

[0109] At 404, the input stream generator 122 converts the tiled input 428 into a flattened input 430. Here, the input direction 432 indicates the direction in which the flattened input 430 is input into the next step (multiply-accumulate unit 140). Therefore, block 1 is placed first, then block 2, and so on up to block 12. Each block is converted into a single vector and placed parallel to the other vectors converted from the other blocks. Since the blocks are already vectors at this point, no additional conversion occurs. However, as shown in later examples, blocks may not always be vectors, and in this case, they are flattened to become vectors.

[0110] This flattened input 430 can be stored as an array, tree, or other structure, and can be stored in a buffer, memory, or other storage medium. Values ​​in the block can be accessed using a reference pointer, memory address, or based on a clock cycle delay. Regarding the clock cycle delay, the flattened input 430 can be read in one vector at a time, and different read delays of a specific number of clock cycles can be used to access different vectors in the flattened input 430. For example, block 7 can be accessed by delaying the access by 7 clock cycles.

[0111] Figure 5A illustrates the first portion of an example 500 of flattening an input tensor in the case of a two-dimensional core according to one embodiment. Compared to the example in Figure 4, Figures 5A-5B use a two-dimensional core to result in two-dimensional partitioning. In one embodiment, the program can be executed by an input stream generator 122.

[0112] At 501, the input stream generator 122 receives the input and pads it based on the core size to produce a padded input 520. As previously mentioned, the padding satisfies three requirements: 1) it allows the output value to be the same size as the input value, and the padding can be added based on the core focus; 2) the input can be further padded so that it is a modulus of the core size; and 3) in a particular embodiment, due to hardware requirements, an additional core width of the padding is added to the trailing edge of the input.

[0113] Therefore, in the illustrated example, one vector width of the padding is added to the outside of the input, and an additional core width of the padding is added to the right and bottom (tail edge) of the input. Since core 522 is a 3×3 core, the additional padding is 3 units wide on the right and 3 units high at the bottom.

[0114] At 502, the input stream generator 122 divides the padded input 520 into blocks equal to the size of the core. Since the core is 3×3, each block is 3×3 in size. This produces a tiled input 524. A variable K is used to indicate the column size of the tiled input 524, and K is then used to point from the processed block to a block in a second column. Therefore, the first block in the second column of the tiled input 524 is block 1+K, and the blocks to its right are block 1+K+1. Note that if the first block is indicated by "0", K will be set to [column block size]+1 instead of the column block size.

[0115] The procedure is further described in Figure 5B, which illustrates the second part of one example of flattening an input tensor in a two-dimensional core according to an embodiment.

[0116] At 503, the input stream generator 122 flattens the blocks in the image into vectors according to a specific block order. Here, the flattening order sequence takes precedence, as indicated by the flattening order 532 of a single block instance 530. In other words, for each block, the values ​​in that block are read column by column (as shown by the directional arrows) and placed in one of the vectors of that block in the flattening input 528. Like the vector of the flattening input 430, this vector is placed parallel to the vectors generated for other blocks (according to the same flattening order) and used as input to the multiply-accumulate unit 140 according to the input direction 534. Although the vectors are shown as vertical, the orientation may be different in other embodiments.

[0117] Conversely, in another embodiment, the flattening order 538 is row-major, meaning that the values ​​in each block are read row by row and then placed in a single vector, parallel to the other vectors generated by the other blocks that have been filled in the input. Therefore, compared to the flattened input 528 where the vectors of block 1 are ordered as [0,0,0,0,1,2,0,11,12], the vectors of block 1 here are ordered as [0,0,0,0,1,11,0,2,12] because the values ​​in block 1 are read row by row rather than column by column. The exact flattening order 538 does not affect the output, as long as the order of other procedures corresponds to the same order when producing the output values.

[0118] Figure 5C illustrates an example of an input tensor with multiple input channels flattened according to one embodiment. In one embodiment, the input tensor 102 may have multiple input channels 104A to 104N, as described in Figure 1. The multiple input channels are polymorphically coupled with a core. The outputs of these polymorphic couplings with the core are summed to generate an output excitation. If multiple core filters are present in the core, each core filter is polymorphically coupled with multiple input channels to generate an output feature. Each input channel may correspond to a different component of the input, such as a color channel, etc. In one embodiment, the program may be executed by an input stream generator 122.

[0119] At 504, the input stream generator 122 uses the first flattened input to tile any additional input channels. For clarity, core 542 is shown as a 2×2 core. Therefore, each block is 2×2, and each vector is a 1×4 vector. Here, each input channel 540A-N (collectively referred to as input channels 540) is similar to the input from 501, therefore, the input stream generator 122 fills each input channel 540 and flattens it to produce flattened input channels 544A to 544N. In one embodiment, these are then "stacked" one on top of the other to produce a flattened input stream 546. However, in other embodiments, the flattened input channels 544A to 544N can be arranged in different combinations, such as by cascading, placing them in a 3D array, etc. In either of these two cases, the flattened input stream 546 is a combination of various flattened input channels 544, and corresponding blocks of each flattened input channel 544 can be referenced together in the flattened input stream 546. Therefore, for example, referencing a block 5 of the flattened input stream 546 (by delaying the clock cycle or otherwise) will allow referencing blocks 5 in all flattened input channels 544, including the flattened input stream 546.

[0120] Figure 6A illustrates the first portion of an example of flattening an input tensor in a three-dimensional core according to an embodiment 600. As with Figures 4 through 5C, in one embodiment, the procedure described herein can be executed by an input stream generator 122.

[0121] At 601, the input stream generator 122 receives the three-dimensional input and fills the input to produce the filled input 626, similar to the method described above with reference to Figures 4 to 5C. For clarity, only a single width of the fill 628 is shown here. Also for clarity, the actual input value 630 is not shown here because it would overlap in the perspective view. Instead, the fill value 628 is shown as a cube with a dark gray pattern, and the input value 630 is shown as a white cube. This also applies to core 632 and core value 634. Note that the filled input 626 has three dimensions 106: dimension 1 106A, dimension 2 106B, and dimension 3 106C. In some cases, dimension 1 may refer to width, dimension 2 to height, and dimension 3 to depth. Additionally, dimension 1 may refer to having rows, dimension 2 to having columns, and dimension 3 to having aisles or layers.

[0122] At 602, the input stream generator 122 tiles the input 626 to produce a block input 626, which has a size equal to that of the core 632. Since the exemplary core 632 has a size of 2×2×2 and can be represented by a three-dimensional matrix, the blocks also have a size of 2×2×2. In addition to a K variable 636 indicating the number of blocks in a column of the tiled input, the three-dimensional input also includes an M variable 638 indicating the number of blocks in a layer of the tiled input. When the operation has an output value corresponding to a position of a current block, the input requirement can come from a block in the layer following the current block and a block below the current block (as in the case of 2D input). Therefore, in addition to the K parameter which can be used to indicate the position of the block below the current block, the M parameter can be used to indicate the block following the current block. Although referenced to directions such as below and behind, the input blocks may not actually be geometrically configured as shown here, but can be indicated by a data structure abstraction. However, the same K and M parameters will apply. Since the padded input 626 has a width of 6 and a height of 6, each layer block contains 9 blocks, because the core 2×2 layer is evenly divided into 9 layers of the padded input 626 (6×6 layers). Therefore, in the illustrated example, the M parameter is 9 and the K parameter is 3.

[0123] The procedure continues in Figure 6B, which illustrates the second part of an example of flattening an input tensor in a three-dimensional core according to one embodiment. Here, at 603, the input stream generator 122 flattens each block into a vector according to the instanced flattening sequence 640 and places the vectors parallel to each other, similar to the procedure described above with reference to Figures 4 and 5B. Since a single block now has a size of 2×2×2, it contains 8 values. An order is established such that these 8 values ​​are read and arranged into a single vector according to this order. Here, the instanced flattening sequence 640 reads the single block first column-wise (column-first) and then by layer / aisle (aisle-first). This order is indicated by the bold italic numbers in the instanced flattening sequence 640 showing an exploded view of a single block. Therefore, the first read is the one by " [, 1 The value at the block location indicated by ", ]" is then read from " [, 2 The value at the block location indicated by ", ]" etc., and terminates at the location indicated by " [, 8 The value at the block location indicated by , ]". Different sorting can be established depending on the size of the block. As with previous sorting, as long as the sorting is consistent throughout the program, the output value will be the same regardless of the sorting used.

[0124] The result of this flattening by the input stream generator 122 is the flattened input 642. In each vector of the flattened input, the instance flattening order 640 is indicated by the order 644. Therefore, for each vector, the "..." indicated by the order 644... [, 1 The position , ]” has the value from the corresponding block (which is located in the instance flattening order 640 of that block) [, 1 The value at the position (in the , ]” position). Here, as in Figure 6A, the dark gray area indicates the fill value, and the white area indicates the input value. For example, block 3 in Figure 6A contains 4 input values ​​and 4 fill values. After following the instanced flattening sequence 640, this is converted into the vector indicated by block number 3 at flattened input 642, where the input value and fill value alternate. As with the flattened input in Figures 4 and 5B, this flattened input is sent to multiply-accumulate unit 140 for further processing to produce an output value. [Extended core generated] [ ]

[0125] Figure 7 is a flowchart 700 illustrating a method for generating an extended core according to one embodiment. Although the illustrated flowchart may show a sequence of operations, the illustrated operations may be performed in any order and may have more or fewer operations. In one embodiment, the operations illustrated in Figure 7 may be performed by an extended core generator 126.

[0126] The extended core generator 126 receives 710 for rotating a core of an input tensor. This core may be core 110. For a first (minimum) dimension of the core, the extended core generator 126 generates 720 square blocks (a circular matrix) of the values ​​of each one-dimensional vector of the core, which contain all rotations of that one-dimensional vector. Each one-dimensional vector here is a unique vector, which is a column of the core, a row of the core, a diagonal of the core, or a path of the core. A path of a core is a core vector aligned along an axis corresponding to the depth of the core (in this case, the third dimension).

[0127] If core 110 has multiple core filters 112, each core filter is processed individually according to the flowchart illustrated herein. If each core filter contains a sub-filter core, that sub-filter core is also processed individually according to the flowchart. Although the term "first" or "minimum" is used here, it is not intended to indicate that the dimensions of the core have a specific order or size relationship. Rather, any dimension of the core may be selected as the first dimension. For example, in the three-dimensional core of Figure 6A, the dimension along the X-axis 620 (X dimension), the dimension along the Y-axis 622 (Y dimension), or the dimension along the Z-axis 624 (Z dimension) may be selected as the first dimension.

[0128] Similarly, a core that revolves with a single-channel or multi-channel input tensor produces a single output channel. If the core contains multiple filters, each filter is applied to the input tensor to produce a single output excitation.

[0129] After processing the first dimension, at 730, for each additional dimension of the core, the extended core generator 126 groups 740 the blocks immediately following the previous dimension into several groups of blocks, each group containing a block of the immediately following previous dimension aligned with a vector parallel to the axis of the (additional) dimension. The vector parallel to the axis of the dimension is a vector parallel to the axis of the currently processed dimension. Therefore, using the above example, if the X dimension is the first dimension, then the next dimension to be processed can be the Y dimension (but it can also be the Z dimension). One or more vectors parallel to the Y-axis and aligned with blocks from the previous dimension can be drawn along the Y dimension. This is because four blocks are generated in the previous dimension (X dimension), and each of these (if conceptually "located" at the same position where these blocks were generated) forms two "rows" of blocks. Each of these "rows" is aligned with or through a vector parallel to the Y-axis. The aligned blocks are grouped together.

[0130] For the additional dimension, the extended core generator 126 further generates one or more blocks of 750 values, each block containing all rotations of the blocks within each of several groups of blocks immediately preceding the previous dimension. Thus, in the example above, each row block rotates the blocks within that row to produce a new square block of values. This results in two square blocks of values ​​in the Y dimension. To rotate a vector or a group of blocks, the extended core generator 126 rotates the values ​​in a column (e.g., barrel-shifts the values) or rotates the blocks in a group to produce a new vector or sort for each rotation. Once all possible rotations have been generated, the generated combinations of vectors or sorts of blocks are "stacked" together to produce an output block containing all rotations of the underlying vectors or underlying groups of blocks.

[0131] Continuing the example, when processing the next dimension (Z dimension), the vector is parallel to the Z-axis. In this case, rotations from the previous two "rows" of the two blocks each produce two blocks. Therefore, these two blocks are rotated while they are "positioned" in a manner that aligns them with the vector. There are no other blocks that do not pass through this vector; therefore, it is the only block required. However, if the core has more dimensions (i.e., more than 3 dimensions), there may be additional blocks that are not aligned with this vector, and therefore additional vectors are needed to group these additional blocks.

[0132] When processing all dimensions, the expanded core generator 126 outputs 760 a block corresponding to the value of the last dimension in the additional dimensions of the core as the expanded core. Using the previous example, after rotating the two blocks from the Y dimension, a single block is retained. This block is output as the expanded core.

[0133] Therefore, as in the example, the 3D core has an extended core comprising a set of first vectors, each first vector having a rotational permutation of a first block in a first block group. Each first block in the first block group corresponds to a second vector in a set of second vectors, and each second vector corresponds to a second block in a second block group. Each second block in the second block group corresponds to a vector in the core (a core vector), and each second block has a set of third vectors representing all rotational permutations of the corresponding core vector of that second block, wherein each third vector represents a rotational permutation of the corresponding core vector. Here, each set of second vectors contains all rotational permutations of the second block in the corresponding second block group. Additionally, the first vector set contains all rotational permutations of the first block in the first block group.

[0134] In one embodiment, the core is not a three-dimensional core as in the example, but a one-dimensional (1D) matrix, and the extended core comprises a plurality of parallel vectors, each of which is a different rotational permutation of one of the core's dimensions. Here, the extended core is a square matrix with a size equal to one of the dimensions of the core.

[0135] In another embodiment, the core is a two-dimensional (2D) matrix; therefore, the extended core includes a set of first vectors, each first vector having a rotational permutation of one of the blocks in a block group. Each block in the block group corresponds to one vector in the core (a core vector), and each block has a set of second vectors representing all rotational permutations of the corresponding core vector. Each second vector represents a rotational permutation of one of the corresponding core vectors, and the first vector set contains all rotational permutations of the blocks in the block group.

[0136] After being generated, the extended core is a two-dimensional square matrix with a length equal to the product of the dimensions of the corresponding core used to generate the extended core. Additional examples of extended cores for generating one-dimensional, two-dimensional, and three-dimensional cores will be described below with reference to Figures 8, 9A to 9C, and 10A to 10C.

[0137] Figure 8 illustrates an example 800 of generating an extended core in the case of a one-dimensional core according to one embodiment. In one embodiment, the procedures described herein can be executed by an extended core generator 126.

[0138] At 801, the extended core generator 126 receives core 820. In the illustrated example, core 820 has a size of 3×1 (lowercase letters indicate core values). Therefore, it is a one-dimensional core. Following the procedure described in Figure 7 above, at 802, the extended core generator 126 rotates each of the one-dimensional vectors of core 820. Since core 820 has only one single vector (i.e., [a, b, c]), only this vector can be used for rotation. Rotating this vector produces a total of three combinations: the combination [a, b, c] shown in the original core 820, the combination of rotation 822 ([c, a, b]), and the combination of rotation 824 ([b, c, a]). The method of generating the rotation is indicated by the arrow shown in 802. These combinations are grouped together to form a cyclic matrix. At 803, since core 820 has no more additional dimensions to process, the extended core generator 126 outputs the grouped vectors generated during the rotation as the extended core 826.

[0139] Because the extended core includes various rotations of the core, different columns of the extended core can be used in a single product with blocks of the flattened input stream selected by a control pattern (e.g., control pattern 136). Instead of having the original core traverse a single value in the input multiple times, the core is extended here so that it can be applied to the flattened input stream 2^(s-1) times as described above, without having to read the input multiple times equal to the number of elements in the core. Since the size of the extended core is much smaller than a possible input tensor 102, the core expansion in the procedure described here is far more efficient than multiple reads of the input tensor values.

[0140] Figure 9A illustrates an example of an extended core generated according to an embodiment 900, representing the generation of one of the different two-dimensional cores. As in Figure 8, in one embodiment, the procedure described herein can be executed by an extended core generator 126.

[0141] At 901, the extended core generator 126 receives the cores. Three instances of two-dimensional cores 920A to 920C are shown here with different sizes. Core 920A has a size of 3×2, core 920B has a size of 2×3, and core 920C has a size of 3×3. At 902A, a column priority order is used to rotate the one-dimensional vectors of core 920 in core rotations 922A to 922C; that is, the one-dimensional vectors correspond to the columns of the rotated cores. As shown, the number of one-dimensional vectors varies depending on the size of the core. For example, core 920B, with three columns, has three sets of rotations in core rotation 922B, while core 920A has only two sets of rotations.

[0142] In 903A, the extra dimension (i.e., the second dimension) is rotated. Here, as mentioned in Figure 7, a vector aligned with the axis of the second dimension is selected, and the blocks passing through each vector are rotated together. In the example of core 920A, two blocks 926A to 926B are generated in core rotation 922A, and these are aligned with a single vector along the axis of the second dimension. Block 926A is generated by the rotation of the combined column [a, b, c], and block 926B is generated by the rotation of the combined column [d, e, f]. These two blocks are rotated in core rotation 924A to generate extended core 932A. Since there are no more dimensions in the core, no further rotations are required.

[0143] A similar procedure is performed on core 920B. However, this time there are three blocks 928A to 928C. Therefore, in core rotation 924B, the three blocks 928A to 928C are rotated and combined to produce extended core 932B. For core 920C, three blocks 930A to 930C are also produced in core rotation 922C (because core 920C has three columns). These blocks 930A to 930C are rotated along the second dimension in core rotation 924C to produce extended core 932C. Each extended core has equal size along two dimensions (e.g., a square matrix), and the size along each dimension is equal to the product of the sizes of the core's dimensions. For example, core 920B has a size of 3×2, therefore, the size of each dimension of its extended core 932B is 3*2=6.

[0144] Figure 9B illustrates an instance of an extended core generated using row-major extension according to one embodiment. Instead of using column-major order as in Figure 9A, the instance shown here uses row-major order.

[0145] At 902B, the extended core generator 126 uses core 920C and rotates each one-dimensional vector at core rotation 934. However, instead, one-dimensional vectors are selected as rows of core 920C. Therefore, instead of a single vector having the value [a, b, c], the first vector shown here has the value [a, d, g] corresponding to row 1 indicated in the figure. The remaining two vectors correspond to rows 2 and 3, as indicated.

[0146] At 903A, a rotation is formed in core rotation 936, similar to core rotation 924 shown in Figure 9A, as a rotational vector from 902B to blocks 938A to 938C. This produces an extended core 940 that differs from extended core 932C (although using the same core). However, if the control pattern is also generated based on this row priority, the output value will not be different after applying the rotation.

[0147] Figure 9C illustrates an example of generating an extended core in the case of multiple core filters according to one embodiment. If the core has multiple core "filters", each core filter is extended in the same manner as previously described for a single core. If each filter has further sub-filter cores, these sub-filter cores are also extended in the same manner. Thus, the core filters 942A to 942N are extended into the extended core filters 944A to 944N illustrated.

[0148] Figure 10A illustrates a first portion of an example 1000 of generating an extended core in a three-dimensional core configuration according to one embodiment. As with Figures 8 and 9A to 9C, in one embodiment, the procedures described herein can be executed by an extended core generator 126.

[0149] At 1001, the extended core generator 126 receives core 1026. In one of the exploded views shown, it can be seen that core 1026 has layers 1028A and 1028B. Each layer has different core values ​​"a" to "h" resulting in 8 values ​​for a 2×2×2 core 1026.

[0150] At 1002, the extended core generator 126 rotates all one-dimensional vectors of core 1026 (using column precedence in this example). There are four such vectors in core 1026 corresponding to four columns in core 1026 (two on each layer). Therefore, four sets of rotations are generated by core rotation 1030.

[0151] At 1003A, the extended core generator 126 rotates the second dimension (i.e., the next dimension) by grouping the blocks generated in core rotation 1030. Here, these blocks from core rotation 1030 (which are aligned along a vector parallel to the axis of the processed current dimension (i.e., the second dimension)) are grouped together. Here, the blocks forming group 1034A are aligned along a vector (i.e., vector 1032A) parallel to the second-dimensional axis (i.e., the Y-axis 1022). The blocks forming group 1034B (which are formed by values ​​on a layer separate from the blocks in group 1032A) are also aligned along a vector (i.e., vector 1032B) parallel to the second-dimensional axis.

[0152] Although two vectors are drawn here, they are presented to illustrate the conceptual reasons behind how blocks are grouped. In reality, instead of drawing vectors, the extended core generator 126 groups these blocks by determining the exponent of the dimension from which it generates the blocks. For example, blocks in group 1034A are generated from layer 1028A (which may have an exponent "0") and blocks in group 1034B are generated from layer 1028B. Therefore, the extended core generator 126 knows to rotate blocks generated from these different layers individually. This procedure is repeated for subsequent dimensions, where the exponent value of the current dimension is used to group blocks generated in previous dimensions.

[0153] The procedure continues in Figure 10B, which illustrates the second part of one example of generating an extended core in the case of a three-dimensional core according to an embodiment.

[0154] Here, at 1003B, the extended core generator 126 rotates the blocks generated in groups 1034A to 1034B. Each group's blocks rotate independently. As shown in the figure, this generates two blocks: a block generated from the rotation of the two blocks in group 1034A and a block from the rotation of the two blocks in group 1034B.

[0155] At 1004A, the extended core generator 126 repeats step 1003A, but for the next dimension (i.e., the third dimension), which is also the last dimension of the three-dimensional core 1026. Here, the extended core generator 126 groups blocks along a vector aligned with the axis of the dimension. As shown in the figure, this vector is vector 1040, and the axis is the Z-axis 1024. The blocks generated in 1003B are generated from blocks on different layers of the core; therefore, the blocks generated in 1003B are conceptually located on different layers. Vector 1040 is aligned with the two blocks generated (i.e., through the two blocks), thus grouping these blocks into group 1038A. Since there are no other blocks for grouping, this is the only group for the third dimension.

[0156] The process continues in Figure 10C, which illustrates the third part of one example of generating an extended core in the case of a three-dimensional core according to an embodiment. Here, at 1004B, the extended core generator 126 generates all rotations of the blocks in the block group generated when processing the previous dimension. In the illustrated example, there is only one set 1038A, and therefore the blocks in this set are rotated to generate the final extended core 1044, since this is the last dimension of core 1026. [Control Pattern Generation] [ ]

[0157] Figure 11 is a flowchart 1100 illustrating a method for generating a control pattern according to one embodiment. Although the flowchart illustrates a sequence of operations, the operations can be performed in any order and may have more or fewer operations. In one embodiment, the operations illustrated in Figure 11 may be performed by a control pattern generator 134.

[0158] The control pattern generator 134 receives 1110 and a core dimension of one or more sizes from an input tensor to generate an output excitation. This core may be the core 110 in Figure 1. The core may be one-dimensional, two-dimensional, three-dimensional, or more than three-dimensional. Each dimension of the core has a size, which is received by the control pattern generator 134.

[0159] At 1120, for each position of the control pattern, the control pattern generator 134 generates a value at position 1130, which is based on one or more sizes of the position and core dimension in the control pattern.

[0160] Once the values ​​for all positions are generated, the control pattern generator 134 outputs a complete control pattern square matrix of 1140. Each dimension of the control pattern is equal to the product of the width and height of the core. Furthermore, the value at each position of the control pattern indicates a position from which the value of a flattened input tensor used for rotation with the core is accessed.

[0161] In one embodiment, each position of the control pattern has a column number and a row number, and the generation of each value includes: determining the result of a test based on one or more of the size of the column number, row number, and core dimensions for each core dimension. Each test generates a binary output indicating the relative offset of the value from its access to the flattened input stream (as described above, which is generated from the input tensor by dividing the core size of the flattened input tensor into blocks). The value of that position of the control pattern is generated as a combination of the binary outputs of the tests for each core dimension. Each test may be an inequality test that compares a first modulo operation using one of the column numbers of the position with a second modulo operation using one of the row numbers of the position. The first modulo operation is a modulo between the column number and one or more of the size of a core dimension. The second modulo operation is a modulo between the row number and one or more of the size of a core dimension. For the two-dimensional core, the first test system is r mod h < c mod w, and the second test system is floor((r mod (h * w)) / h) < floor((c mod (h * w)) / w), where r is the series number, c is the row number, h is the height of the core, and w is the width of the core.

[0162] In one embodiment, the control pattern comprises a plurality of vectors, wherein the number of vectors corresponds to the number of output value positions in a core-sized block of the output stimulus. The values ​​within each of the plurality of vectors correspond to delay values, wherein each delay value indicates the amount of delay required to access an individual input value in a flattened input stream. As described above, the flattened input stream comprises a set of parallel vectors generated from an input tensor, and the delay amount specifies one of the parallel vectors within the flattened input stream.

[0163] In one embodiment, delay values ​​are used to generate an output value at each output value location of a core-sized block of output excitation by selecting a vector from a control pattern corresponding to the output value location corresponding to the output value to be generated. Delay values ​​in the selected vector of the control pattern are accessed, wherein each delay value indicates a parallel vector within the flattened input stream from which the input values ​​used to generate the output value are accessed. A dot product is performed between the accessed input values ​​and a vector of an extended core to generate the output value at the output value location in the core-sized block of output excitation. In one embodiment, an extended core is generated by modifying the core to include a set of hierarchical blocks, wherein each block in the hierarchical block set contains all rotational permutations of the block (which are the nodes of the block). The leaves of the hierarchical block set are blocks containing all rotational permutations of the individual vectors of the core, and each block in the hierarchical block set has nodes that are aligned with a vector parallel to an axis representing the dimension of the block within the core.

[0164] In one embodiment, the control pattern is a two-dimensional square matrix having a length equal to the product of the size of the dimension corresponding to one of the input tensors.

[0165] In one embodiment, the order of delay values ​​in the configuration control pattern is based on a defined order of accessing the core size blocks of the input tensor to generate a flattened input stream.

[0166] In one embodiment, the order of priority is defined as a column priority order, a row priority order, or aisle priority order. The aisle priority order first accesses elements in the 3D block of the input tensor along an axis corresponding to the depth of the 3D core size block and then along the axes of the width and height of the 3D block.

[0167] Although the flowchart above illustrates one method of generating control patterns from the size of the core dimension, in another embodiment, various control patterns for various core sizes are stored in memory and retrieved according to the core size used. Additional details regarding the generation of control patterns will be described below with reference to Figures 12 and 13.

[0168] Figure 12A illustrates the first part of an example 1200, which is based on a conceptual foundation of a generation control pattern according to an embodiment.

[0169] As previously mentioned, the purpose of the control pattern is to select the correct block value from the flattened input stream for combination with the extended core to produce an output value. As the core navigates across input values ​​in one initial implementation of the loop, different values ​​from the input are read and combined with weights in the core to produce an output. The control pattern allows this to be simulated by selecting different input values ​​from different blocks in which they are located to produce the correct output value. However, compared to the initial implementation, the input values ​​are not read multiple times, but only a minimum number of times. The following methods can be used to determine which block of the control pattern is selected. However, although the following methods can be used to generate control patterns, in practice the hardware will likely use a separate method (such as the method described above with reference to Figure 11) because the method described herein may be inefficient. Therefore, the method described herein is more for providing a conceptual understanding of the control pattern.

[0170] At point 1201, an input is flattened, similar to the method described above with reference to Figures 4 through 6. The input tensor here has a size of 4×4, and since core 1220 has a size of 2×2, each block has a size of 2×2, and there are a total of 4 blocks (no padding is applied here). Therefore, flattened input 1218 has 4 vectors, each representing a block. The core is also expanded to produce expanded core 1224, similar to the procedure described above with reference to Figures 7 through 10.

[0171] At 1202, the first block (block 1 (1222A)) in the flattened input 1218 is multiplied by the extended core to produce an element-wise product 1226A. Therefore, the "1" in block 1 1222A is multiplied by each of the four core values ​​"a", "b", "c", and "d" in the corresponding column of the extended core 1224 to produce four products "a1", "b1", "c1", and "d1". The remaining values ​​of this block and the other blocks 2 through 4 (1222B through 1222D) are repeated. This produces the element-wise products 1226A through 1226D shown.

[0172] The procedure continues in Figure 12B, which illustrates the second part of an example of a conceptual basis for a generation control pattern according to an embodiment. Since element-wise products 1226A to 1226D have been generated, at 1203, the block values ​​required to generate the element-wise spiral product are determined for each position of the core overlapping the original input on a single block (in the illustrated example, the column-wise step of the core is used). Therefore, at position 1 (1230A) (where the core position is located at the top left corner of the input, as indicated by the core position bounding box 1232), the core values ​​"a", "b", "c", and "d" covering the input are multiplied by the underlying value of the input covered by the core value. Therefore, "a" is multiplied by "1" in the input, "b" is multiplied by "2", "c" is multiplied by "3", and "d" is multiplied by "4" to produce the element-wise product pair (1234A) at position 1 shown. In the example shown, these product pairs are displayed as rows, but they can also be transposed to be displayed as columns, depending on whether column-major or row-major format is used to produce the flattened input stream or the expanded core.

[0173] Based on the element-wise product pair 1234A, determine which of the element-wise products 1226A to 1226D can locate such an element-wise pair. In the first instance at position 1 (1230A), all such pairs can be located in the element-wise product 1226A and are marked "A". Therefore, the control pattern after position 1 is displayed as a row of all "A"s at 1236A. Here, A can represent the current block being processed, which can be generalized to block X.

[0174] Regarding position 2 (1230B), the core position 1232 shifts right by 1, although it still overlaps with the first block in the input. The element-wise product pair 1234B consists of "a2", "b5", "c4", and "d7". Since the "d" in the core is now on the value "7" in the input, this "d7" product pair cannot be found in the element-wise product 1226A, but is found in the element-wise product 1226B, as indicated by a "B". Therefore, the next line of the control pattern indicated in 1236B has a "B" at the last value corresponding to the position of "d7" in the element-wise product pair 1234B. In addition, the B here corresponds to the current block X plus 1 or block X + 1.

[0175] Regarding position 3 1230C, core position 1232 is shifted down by 1 and left by 1. The core is not shifted further to the right because if this were to happen, it would no longer overlap with the first block. This would be unnecessary because the control pattern is generated solely for producing the output value of a single block (corresponding to the same block position in the input, i.e., the current block) by flattening the input stream to determine this value. Additional blocks are computed separately. Therefore, when the system computes the output value block by block, the procedure described here only involves the position of the core where the core overlaps on the single block in question. In addition, since the focus of core 1220 in the illustrated example is the upper left corner position of core 1220 (indicated by "a"), the core position 1232 on the current block of interest is the position where the focus of the core covers each of the input values ​​of the current block. Therefore, position 3 is one of the positions where the focus of the core covers the third value in the current block (using column-by-column order). In this case, the kernel is read from the current block, but also from the block below the current block (which is block X+K, where, as previously mentioned, K is the ordinal number of the block in a column (or row). This value is combined with the kernel to produce the element-wise product pair 1234C. As shown in the element-wise product pair 1234C, "c9" and "d10" contain values ​​from the block below the current block and can be found in the element-wise product 1226C; that is, "c9" and "d10" can be among the values ​​of the element-wise product 1226C. The first two product pairs "a3" and "b4" can be found in the element-wise product 1226A because they only contain values ​​from the current block. Therefore, the next line of the control pattern after position 3 shown in 1236C has two "A"s and two "C"s, where the order of the letters corresponds to the order of the product pairs.

[0176] Finally, at position 4 (1230D), position 1232, which covers the core of the input, contacts the values ​​from the current block and the other three blocks surrounding it. Therefore, the product pair 1234D is "a4", "b7", "c10", and "d13". These can be identified in each of the element-wise products 1226A to 1226D. Therefore, the last line of the control pattern indicated in 1236D uses the letters "A" to "D" to indicate all four element-wise products 1226A to 1226D. Here, D will represent the current block X plus the column size variable K plus 1 or X + K + 1, and therefore refers to the blocks below and to the right of the current block.

[0177] As indicated here, because the value in the control pattern indicates which blocks enclose the current block that takes its value, the rotation of one of the cores on the current block is also used to select the value of the self-flattening input stream. The value is then combined with the corresponding vector of one of the extended cores to produce the output value of the corresponding focus position in the output.

[0178] Figure 13A illustrates an example 1300 of a portion of the values ​​of a control pattern for generating a two-dimensional core according to one embodiment. In one embodiment, the procedure described herein is executed by a control pattern generator 134.

[0179] At 1311, the control pattern generator 134 generates all values ​​of a control pattern 1320. At this point, most values ​​of control pattern 1320 have been generated, but the remaining value 1324 is not generated to plot the program. This value 1324 is located at column 0 and row 8, as shown in the figure. At 1312, the control pattern generator 134 uses the column and row numbers / coordinates of the position to determine the test result for that position. The test result refers to inequality test 1330. The number of tests equals the dimension of the core. Since the core is two-dimensional here, two tests are generated. Each test corresponds to one of the shift variables used to identify the tiling size of each dimension in the input tensor. For example, K represents the size or number of tiles in the first dimension (i.e., each column), and M represents the number of tiles in the second dimension (i.e., each layer). Alternatively, each shift variable of a particular dimension can be defined as the product of the tile size of that dimension and the tile size of any previous dimension. Here, the tile size indicates the number of blocks spanning a length parallel to one dimension of the input tensor.

[0180] Furthermore, the first dimension generates a test corresponding not to a shift variable, but to one of the current blocks. Therefore, a test_k and a test_1 are generated here. Alternatively, these can be referred to as delay_k and delay_1, because, as mentioned above, the shift variable can be used to indicate the amount of delay in accessing the flattened input stream.

[0181] As can be seen here, the test inequalities follow a pattern. The first dimension 1 test inequality is r mod h < c mod w, where r is the column number of the control pattern (as shown in control pattern 1320), h is the height of the core (as shown in core 1322), c is the row number, and w is the width of the core (mod is the modulo operation, and < indicates less than). The second test_k corresponding to the K variable is similar, but includes the region (i.e., h * w in the operation), because this is the second dimension. Therefore, the resulting test inequality is floor((r mod (h * w)) / h) < floor((c mod (h * w)) / w), where floor is a floor or round-down operation (to the nearest integer). Although core 1322 is a square core here, in other cases the core may not be square, therefore, both the height and width of the core are considered in the test inequality.

[0182] The results table 1332 relates to the results 1313 of the two tests. Here, depending on the true or false (i.e., 1 or 0) result of each test, a distinct value (i.e., an indicator) is placed at the corresponding position in the control pattern. Here, since there are two tests, there are a total of four possible values, which are represented here by the letters A, B, C, and D, and equivalently by the binary values ​​00, 01, 10, and 11, respectively. As described herein, this value indicates the vector / block of the flattened input stream from which the output stimulus is calculated from its access value. Here, as shown in test result 1326 (where the column number is 0 and the row number is 8), the result of both tests is true; therefore, the value at 1324 is "D" or 11, which relates to the current block x+k+1. Although the column and row numbers are sorted as shown in the figure, in another embodiment, the sorting can be reversed. The results here will be the same as long as the test inequalities are modified (e.g., by reversing them). Therefore, the procedure here is not limited to the list of control types or the specific methods that are executed.

[0183] Figure 13B illustrates an example 1301 of a portion of the values ​​of a control pattern for generating a three-dimensional core according to one embodiment. In one embodiment, the procedure described herein is executed by a control pattern generator 134.

[0184] Here, at 1351, the control pattern generator 134 generates the value of the control pattern 1360. Compared with the control pattern in Figure 13A, the control pattern 1360 is generated for a three-dimensional core 1362 with a height of 2, a width of 2, and a depth of 2 (labeled as h, w, and d, respectively).

[0185] As described in Figure 13A, each dimension of the core is associated with a test inequality. Therefore, three test inequalities are generated here in test table 1370. As in Figure 13A, these inequalities can also be referred to as delay_m, delay_k, and delay_1 instead of test_m, test_k, and test_1, respectively. test_1 and test_k are identical to the test inequalities shown in Figure 13A. test_m, corresponding to the M-shift variable (i.e., how many blocks are in one layer of the input tensor), follows a pattern similar to test_k but now includes the depth (d) of the core.

[0186] If the core will contain additional dimensions, this test inequality is further expanded to include those additional dimensions. For example, for four dimensions, the test inequality could be floor((r mod (h * w * d * t)) / (h * d * t) < floor((c mod (h * w * d * t)) / (w * d * t), where t represents the "strength" or the size of the fourth dimension of the core. This pattern continues to higher dimensions, where additional dimensions are added to the test inequality. Therefore, for a core with N dimensions, the test inequality takes the following form: floor((r mod ( [, The product of the core dimension and its size , ] )) / (h * [, Excluding the product of the core dimension and the size of the width , ] ) < floor((c mod ( [, The product of the core dimension and its size , ] )) / (w * [, Excluding the product of the core dimension of height and size , ] ), where the remaining test inequalities of the core are the test inequalities of the lower-dimensional cores N-1, N-2…1, where the test inequalities of the core with one dimension are the same as test_1 shown here (and the test inequalities of the core with two dimensions are the same as test_k shown here, etc.).

[0187] Since there are three tests in the 3D core, a total of eight combinations are shown in the results table 1372. Each combination corresponds to an indicator A through H or a three-bit binary value. As will be further described below with reference to Figure 13C, in some embodiments, the binary pattern can be used to efficiently store in the control pattern. At 1352, the control pattern generator 134 generates the final residual value 1364 at column 0 and row 7 of control pattern 1360. As shown in test result 1366, all three test inequalities result in this position being true, corresponding to indicator 1368H or binary value 111 at 1353. Therefore, H will be stored in column 0 and row 7 of control pattern 1360. However, if not all test inequalities return true, a different indicator will be selected based on the column in the results table 1372 of the three test inequality results of the selection indicator matching operation test inequality.

[0188] If the core contains additional dimensions, the size of the result table 1372 will increase, similar to the number of test inequalities. The number of columns in the result table will be equal to 2X columns, where X is the dimension of the core. It can be seen that if the results of the test inequalities are organized such that the results of the largest dimension form the most significant bits of the binary value and the results of the smallest dimension form the least significant bits, the combinations of results can be considered as binary values ​​and can be ordered sequentially in the table (e.g., in ascending order from top to bottom). The indicators associated with each combination also increase sequentially and are also associated with combinations of shift variables whose dimensions increase sequentially. For example, in the illustrated instance, A and B are not associated with shift variables, but only with the current block (i.e., the current block x and x+1), C and D involve shift variables k (i.e., x+k and x+k+1), E and F involve the next highest-dimensional shift variable m (i.e., x+m and x+m+1), and G and H involve the sum of k and m (i.e., x+m+k and x+m+k+1). If the core has an additional dimension, this will continue as a combination of shift variables of ever-increasing size. Each combination will contain two indicators, one of which is the combination itself, and the other is the combination plus 1. Therefore, for a four-dimensional core, the pattern will continue as I and J (x+t and x+t+1), K and L (x+t+k and x+t+k+1), M and N (x+t+m and x+t+m+1), and O and P (x+t+m+k and x+t+m+k+1). These will correspond to the binary values ​​1000 and 1111 (or 8 to 15), where the most significant bit corresponds to one of the test inequalities in the fourth dimension (i.e., a "test_t" or "delay_t") and the remaining bits correspond to the same three tests illustrated herein.

[0189] It should be noted that, for the purpose of calculating the test inequality, variable r replaces the column number of the control panel position that will produce one of its values ​​(0 here), variable c replaces the row number of the control panel position that will produce one of its values ​​(7 here), and variables h, w, and d replace the size of the core's height, width, and depth (2, 2, and 2 here).

[0190] Figure 13C illustrates an example 1302 of a control pattern for generating cores of different dimensions according to an embodiment. Figure 13C shows a 3×1 1D core 1380, a 3×3 2D core 1384, and a 2×2×2 3D core 1388, with the 3D core 1318 shown as an exploded view for clarity. Corresponding control patterns 1382, 1386, and 1390 are shown for these cores, respectively. They are generated according to the method described above with reference to Figure 11. For the 1D core, the identifier "A" in control pattern 1382 indicates the current block X, and "B" indicates block X+1. For control pattern 1386, "A" indicates the current block X, "B" indicates block X+1, "C" indicates block X+K (where K is the column size variable described previously), and "D" indicates block X+K+1. For control pattern 1390, "A" through "D" indicate blocks identical to those in control pattern 1316. Additionally, "E" corresponds to block X+M (the block following the current block), where M is the layer block size (i.e., the number of blocks in a layer). "F" corresponds to block X+M+1, i.e., the block to the right of the block following the current block. "G" corresponds to block X+M+K and to the blocks following and below the current block. "H" corresponds to block X+M+K+1 and to the blocks following, below, and to the right of the current block.

[0191] In one embodiment, instead of using indicators (such as letters or sequences of bits representing such letters as shown in Figure 13) to store control patterns, the fractal properties of the control patterns are utilized to store them in a more efficient manner. For each of control patterns 1382, 1386, and 1390, the identifier begins with "A" along the top and right edges, then leads to an alternating pattern between "A" and "B," and further subdivides the pattern until the leftmost value is the identifier indicating the block furthest from the current block (and, as illustrated, is the largest letter in alphabetical order). Due to this fractal property, each control pattern can be divided into one or more levels depending on the number of unique identifiers in the control pattern. The number of levels corresponds to the dimension used to generate the core of the control pattern. Thus, control pattern 1382 has a single level. Control pattern 1386 has two levels, and control pattern 1390 has three levels.

[0192] Each level comprises a square matrix containing a single binary value (i.e., a bit). The size of each dimension of the square matrix at each level is equal to the size of one dimension of the previous level multiplied by the size of the largest dimension of the core. The smallest level has a square matrix with each dimension having a size equal to the size of the largest dimension of the core. Thus, for example, in control pattern 1386, the largest level contains a square bit matrix of size 9×9, while the next smallest level has each dimension of size 3×3 (i.e., 9 divided by 3, which is the size of one dimension of the core).

[0193] This does not mean that each bit in one of the largest levels matches multiple bits in a larger level. Specifically, the larger level is divided into several regions equal to the number of bits in the next smallest level, and a bit in the next smallest level matches a bit in one of the regions of the larger level. The position of the matching bit in the next smallest level corresponds to the position of a region in the region group of the larger level. The combination of matching bits across multiple levels allows a bit sequence at each position in the control pattern to reference an identifier. Thus, in an example of control pattern 1386, each bit of the 3×3 matrix matches a 3×3 portion of a larger 9×9 matrix. The combination of bits from the 9×9 matrix and the matching bits from the 3×3 matrix together form a two-bit value, which can be used to refer to one of the four displacement indicators of control pattern 1386. In the example of control pattern 1390, since there are 8 displacement identifiers, there are 3 levels: the largest level is an 8×8 matrix, the second largest level is a 4×4 matrix, and the smallest level is a 2×2 matrix. Three bits can be used to indicate the correct one of the 8 unique identifiers in control pattern 1316 (i.e., letters "A" to "H", as shown in the figure). These bits are also previously shown in the results tables in Figures 13A and 13B.

[0194] This method allows for efficient storage of control patterns. To extract the correct bit sequence from various levels, the multiply-accumulate unit 140 can determine the position of one of the indicators in the control pattern, and then use this to determine the correct position in the highest level and the matching position in the remaining levels to determine the bit sequence corresponding to that indicator. [Output generated using flattened input, extended core, and control pattern] [ ]

[0195] Figure 14 is a flowchart 1400 illustrating a method for generating a spinning output using flattened inputs, extended cores, and control patterns according to one embodiment. Although the illustrated flowchart may show a sequence of operations, the illustrated operations can be performed in any order and may have more or fewer operations. In one embodiment, the operations illustrated in Figure 14 may be performed by a multiply-accumulate unit 140 (which may also be referred to as a multiply-accumulate unit).

[0196] The multiply-accumulate unit 140 accesses a buffer access 1410 containing one of a set of parallel vectors in a flattened input stream, where each vector represents a set of input values ​​in a unique core-sized block of an input tensor that will be spun by a core to produce an input stimulus. This flattened input stream can be a flattened input stream 124 and can be generated by the input stream generator 122 described above.

[0197] The multiply-accumulate unit 140 receives 1420 to generate an extended core by rotating and permuting the values ​​from the core, wherein the extended core has a vector corresponding to the position of the output value of one of the core size blocks of the output excitation. This can be an extended core 128 and can be generated by the extended core generator 126 described above.

[0198] The multiply-accumulate unit 140 receives 1430 a control pattern comprising one of a set of vectors, wherein each vector corresponds to the output value position of the core-size block of the output excitation, and wherein each vector contains a delay value indicating a parallel vector of the flattened input stream to access the revolved input value. The control pattern may be control pattern 136 and may be generated using the control pattern generator 134 described above.

[0199] For each output value location in each core-sized block of the output stimulus, the multiply-accumulate unit 140 generates a dot product between a first vector containing the value of the flattened input stream selected by the delay value of the corresponding vector of the control pattern and a second vector corresponding to one of the vectors in the extended core corresponding to the output value location. The multiply-accumulate unit 140 repeats this for all output values.

[0200] In one embodiment, the input tensor has a plurality of channels, and the core has a plurality of filters. Each channel of the input tensor is convolved with one or more filters of the core to produce an output excitation having one of a plurality of output characteristics. Additional examples of generating output values ​​via the multiply-accumulate unit 140 will be described below with reference to Figures 15A to 15C, Figures 16A to 16F, and Figures 17A to 17B.

[0201] Figure 15A illustrates a first portion of an example 1500 of generating an output stimulus using a flattened input, an expanded core, and a control pattern in the case of a one-dimensional core, according to one embodiment. In one embodiment, the procedure described herein may be executed by a multiply-accumulate unit 140.

[0202] At 1501A, the multiply-accumulate unit 140 processes the first block number 1 of the output stimulus (which is the same block position in the input tensor). This is the current block being processed. At 1502A, the multiply-accumulate unit 140 further processes the output position 1 of the first block 3 (where 3 is the number of values ​​in each block). In the processing of the first output position, the multiply-accumulate unit 140 at 1511A selects the value from the flattened input 1522 based on the block position (e.g., a delay amount) indicated by a vector in the control pattern 1524 of that output position. For position 1, this vector is the rightmost row in the control pattern 1524. Therefore, as shown in the figure, the control pattern vector 1528A for position 1 in an output block is all "A", indicating the current block X or block 1 from its selected value. The value in the flattened input 1522 can be selected by referring to a memory address and an exponent value, or it can be selected by delaying the clock cycle read from the flattened input 1522 by the multiply-accumulate unit 140. Here, since all values ​​of the control pattern vector 1528A indicate "A", the selected input value 1530A is selected from block 1 in the flattened input 1522 and is [0, 1, 2], as shown in the figure.

[0203] At position 1512A, the multiply-accumulate unit 140 multiplies the accumulated selected value 1530A with the extended core vector 1532A corresponding to position 1 in the output block, i.e., performs a dot product. For position 1, the extended core vector 1532A is the top column of the extended core 1526. The dot product operation results in the operation result a0+b1+c2, which produces the output 1534A at position 1, as shown in the figure.

[0204] The procedure continues in Figure 15B, which illustrates the second part of one example of using a flattened input, an expanded core, and a control pattern to generate an output stimulus in the case of a one-dimensional core, according to an embodiment.

[0205] At 1503A, the second position of the three output positions of the current block is processed. At 1513A, similar to 1511A, the control pattern vector 1536A corresponding to position 2 of a block is selected. In this case, this vector is the middle row of control pattern 1524. This control pattern vector 1536A is used to select the corresponding value from the flattened input 1522. For each identifier in control pattern vector 1536A, the block indicated by the identifier is selected from the flattened input 1522, and a value is selected from the flattened input 1522 at a position corresponding to the same position of the indicator in control pattern vector 1536A. Therefore, the first (top) position in control pattern vector 1536A has the identifier "A", which corresponds to block X (mentioned in the example) or block 1 in the flattened input 1522 in the current instance. Therefore, the value (value "0") from block 1 of the flattened input 1522 at the same first position is selected. The same operation is performed on the second position of the control pattern vector 1536A to result in a "1" selected from the second position of the first block of the flattened input 1522. For the third and final positions of the control pattern vector 1536A, the identifier is now "B", which indicates (as mentioned in the example) that the block to be referenced is block X+1 or block 2 in the current instance. Therefore, the multiply-accumulate unit 140 selects the value or "5" from the same third position from block 2. Therefore, the selected input value 1538A is [0, 1, 5], as shown in the figure.

[0206] At position 1514A, similar to the operation in 1512A, a multiply-accumulate operation is performed. However, instead of selecting the first column from the extended core 1526, the column corresponding to position 2 of the output, i.e., the vector, is selected. In the current instance, this occurs on the second column of the extended core 1526. Therefore, the extended core vector 1542A at position 2 is [c, a, b]. The dot product between this extended core vector 1542A and the selected input value 1538A is c0+a1+b5 or the output 1544A at position 2 of the output block.

[0207] The procedure continues in Figure 15C, which illustrates the third part of one example of using a flattened input, an expanded core, and a control pattern to generate an output stimulus in the case of a one-dimensional core, according to an embodiment.

[0208] At 1504A, process the last position 3 of the output block. The procedure described here is similar to that of 1503A and 1502A, but all selections for the third position are appropriately shifted.

[0209] Therefore, at position 1515A, the selected control pattern vector 1546A now corresponds to position 3 of the output block and is thus the leftmost row of control pattern 1524. This control pattern vector 1546A has one indicator for block 1 and two indicators for block 2. Therefore, the selected input value 1548A includes one "0" from block 1 and one "4" and one "5" from block 2.

[0210] At position 1516A, for position 3 corresponding to column 3 of extended core 1526 in the current instance, a multiply-accumulate operation is performed on the selected input value 1548A and the extended core vector 1552A. The output of this dot product is the output 1554A at position 3.

[0211] At 1505, this procedure is repeated for all blocks in the input to produce the same number of blocks in the output. Therefore, in the flattened input 1522 of the current instance, the operation in 1501A is repeated from 1501B for block 2 to 1501L for block 12. If the flattened input 1522 contains more blocks, the procedure shown in 1501A is repeated until all blocks are processed.

[0212] Figure 16A illustrates a first portion of an example 1600 for generating an output stimulus using a flattened input, an expanded core, and a control pattern in a two-dimensional core configuration according to one embodiment. The procedure described herein, compared to Figures 15A to 15C, pertains to a two-dimensional core rather than a one-dimensional core. In one embodiment, the procedure described herein is executed by a multiply-accumulate unit 140.

[0213] At 1601A, the first block is processed by the multiply-accumulate unit 140. Since each block has 9 positions, each position is processed to produce one of the output values ​​for that position. At 1602A, output position 1 is processed. As in the same-dimensional core instance, a control pattern vector 1628A is selected based on the output position value from control pattern 1624. Since the output position is 1, the row marked by output position 1 of control pattern 1624 at 1611A is selected as control pattern vector 1628A. This control pattern vector 1628A has "A" as its identifier. Therefore, this control pattern vector 1628A indicates that the selected input value 1630A should be entirely selected from block 1 of flattened input 1622 (because the current block X is block 1).

[0214] This procedure continues at Figure 16B, which illustrates the second part of one example of using a flattened input, an expanded core, and a control pattern to generate an output stimulus in the case of a two-dimensional core, according to an embodiment.

[0215] At position 1612A, according to the output position markings shown for extended core 1626 in Figure 16A, a multiply-accumulate operation is performed on the selected input value 1630A and an extended core vector 1632A selected from extended core 1626. In this case, output position 1 corresponds to the top column of extended core 1626. This top column is selected for the multiply-accumulate operation. Therefore, this top column forms the extended core vector 1632A. The multiply-accumulate unit 140 performs a dot product between the selected input value 1630A and the extended core vector 1632A to produce output 1634A at position 1.

[0216] At 1603A, output position 2 is processed. Output position 2 corresponds to the second row to the right of control pattern 1624 (as indicated in the figure). Note that the row order here does not need to follow the order shown. Instead, the row order can be any order as long as the correct row corresponding to the correct output position is selected. Furthermore, control pattern 1624 does not need to be ordered by row, but can also be ordered by column. As long as the correct column corresponding to the correct output position is referenced, the output will be the same. Finally, the order of the identifiers in the control panel can also be reversed. As long as the identifiers are referenced in the correct order when using them to select input values, the output will be the same. This concept also applies to the extended core 1626, as its order can differ from the order described here.

[0217] As mentioned in the example, the identifier "A" in control pattern 1624 refers to block X of flattened input 1622, where X is the current block, and the identifier "B" refers to block X+1. Control pattern vector 1636A contains both identifiers "A" and "B". For each position in control pattern vector 1636A, multiply-accumulate unit 140 determines which block the identifier refers to and selects the input value at the same position in the vector of the block in flattened input 1622 as the reference block in flattened input 1622, corresponding to the position of the identifier in control pattern vector 1636A. Therefore, for example, position 6 in control pattern vector 1636A has an identifier "B" indicating that a value should be selected from block X+1 (which is block 2 of flattened input 1622 (current block number 1+1=block 2)). The multiply-accumulate unit 140 flattens the input 1622 into block 2 vector selection, which corresponds to the value at input position 6 in position 6 of the control pattern vector 1636A, and places this value at position 6 of the selected input value 1638A. The remaining value is selected in a similar manner.

[0218] The procedure continues in Figure 16C, which illustrates the third part of one example of using a flattened input, an expanded core, and a control pattern to generate an output stimulus in the case of a two-dimensional core, according to an embodiment.

[0219] At position 1614A, the selected input value 1638A generated in the previous operation is combined with the extended core vector 1642A by a dot product to produce the output 1644A at position 2. Here, the extended core vector 1642A is again selected based on the output position number (which is 2 in this case). Therefore, the second column of the extended core 1626 is selected as the extended core vector 1642A.

[0220] The multiply-accumulate unit 140 executes the above procedure for the remaining output positions of the block. As an example, at 1606A, the procedure is executed for output position 5. Here, at 1615A, control pattern vector 1646A is selected from the row of output position 5, as shown in Figure 16A. Here, control pattern vector 1646A contains two new identifiers "C" and "D" corresponding to blocks X+K and X+K+1 respectively, where K represents the number of blocks in a column of input. Therefore, as shown in the figure, position 9 in control pattern vector 1646A is selected from position 9 of block 6 in the flattened input stream. This results in the selected input value 1648A for this output position.

[0221] The procedure continues in Figure 16D, which illustrates part four of an example of generating an output stimulus using a flattened input, an expanded core, and a control pattern in the case of a two-dimensional core, according to an embodiment.

[0222] At 1615A, a multiply-accumulate operation is performed. The multiply-accumulate unit 140 performs a dot product between the selected input value 1648A and the extended core vector 1652A selected for position 5. In the illustrated example, this is column 5 of the extended core 1626. The dot product result is output 1654A at position 5. This procedure is repeated at 1616 until all outputs in the current block are produced. Furthermore, this procedure is repeated at 1617 until all blocks are processed.

[0223] Figure 16E illustrates an example of generating an output stimulus with one of multiple channels using a flattened input, an expanded core, and a control pattern in a two-dimensional core configuration, according to one embodiment.

[0224] As previously described, an input tensor 102 may have multiple channels, each of which is a separate matrix of input values. As previously described with reference to Figure 5C, the multiple channels of an input are flattened and "stacked" one on top of the other. At 1618, each of these input channels is flattened according to the method described in Figures 4 through 6, and a selected input channel value 1658 is generated by the control pattern 1624 from the selected value of the flattened input channel 1656. Although a new matrix of the selected input channel value 1658 is shown here, no new matrix is ​​actually generated. Instead, the multiply-accumulate unit 140 can select the value of the flattened input channel 1656 in real time. Therefore, the selected input channel value 1658 is shown here to conceptually illustrate the concept.

[0225] The procedure continues in Figure 16F, which illustrates a second part of an example of generating an output stimulus with one of multiple channels in a two-dimensional core configuration, according to one embodiment.

[0226] At 1618, the multiply-accumulate unit 140 performs a 1668 dot product (i.e., multiply-accumulate) using selected input channel values ​​1658 and core filters 1660A through 1660N. Each input channel in the selected input channel values ​​1658 is spiraled with a different sub-filter core 1666, which is itself one of the cores. Each sub-filter core 1666 may have the same or different values. The spiraling procedure can be similar to the spiraling procedure described above for Figures 16A through 16D. The generated spiral outputs 1662A are summed together 1670 for a single core filter to produce output excitation channel 1664A. This procedure is repeated for each of the remaining core filters 1660B through 1660N, resulting in multiple output excitations 1664B through 1664N. These output excitations 1664A through 1664N can serve as inputs to additional spiral layers in a neural network.

[0227] Figure 17A illustrates a first part of an example of generating an output stimulus using a flattened input, an expanded core, and a control pattern in the case of a three-dimensional core, according to embodiment 1700. In one embodiment, the procedure described herein can be executed by multiply-accumulate unit 140.

[0228] At 1701, the multiply-accumulate unit 140 processes one block X of the input. At 1702, the multiply-accumulate unit 140 processes one of the output positions 8 of block X. At 1711, the multiply-accumulate unit 140 uses the selected control pattern vector 1724 of position 8 to select a value from the flattened input 1722. As described above for the one-dimensional and two-dimensional cores, a vector of control patterns corresponding to the output position (position 8 in this case) is selected as the control pattern vector 1724. The control pattern vector 1724 has eight different identifiers from "A" to "H". As shown in the illustration in Figure 17A, each identifier indicates a different block of the flattened input 1722. For example, the identifier "H" corresponds to block X+M+K+1, where M is the number of blocks in each layer and K is the number of blocks in each column of the input. For each identifier, the multiply-accumulate unit 140 selects a value at a position corresponding to the position of the identifier at the selected input value 1726 by flattening the input block of the input 1722.

[0229] The procedure continues in Figure 17B, which illustrates the second part of one example of using a flattened input, an expanded core, and a control pattern to generate an output stimulus in the case of a three-dimensional core, according to an embodiment.

[0230] At position 1712, the multiply-accumulate unit 140 combines the selected input value 1728 of position 8 with the extended core vector 1730 of position 8 by a single product to produce the output 1732 of position 8 of the current block. The selected extended core vector 1730 corresponds to the position of the output.

[0231] At position 1713, the multiply-accumulate unit 140 repeats the above procedure for all output positions of each block in the input to generate output excitation. [Example Processor Component] [ ]

[0232] Figure 18A illustrates a hardware diagram of an exemplary component for generating an extended core according to one embodiment. In some embodiments, data values ​​1810 corresponding to an array (e.g., a core of weights) are stored in a data storage circuit 1801 (e.g., a memory circuit). In some embodiments, data values ​​corresponding to an array are stored as a sequence (e.g., a single row). In this example, data values ​​1810 are coupled to a shifter circuit 1802 on one or more inputs. The shifter circuit may receive the array as a sequence of data values. For example, each data value may include one or more byte-based digital data values, such as integers, floating-point numbers, or other types of data values. Thus, for example, each data value may be coupled to a shifter circuit 1802 on a plurality of conductors (e.g., wire interconnects on an integrated circuit), wherein a group of conductors may carry a data value and a group of conductors may carry a sequence of data values. In one embodiment, the data values ​​are weights and the input data values ​​are activation values ​​of a neural network.

[0233] A conductor carrying data values ​​may be received at one input of shifter circuit 1802. The input of shifter circuit 1802 may receive a conductor to carry, for example, part or all of a sequence. As mentioned above, data values ​​corresponding to an array may be received as a sequence. In one embodiment, the sequence includes all data values ​​of the corresponding array received in parallel from data storage 1801. However, in other embodiments, the sequence includes a portion of the data values ​​of the corresponding array (e.g., receiving and permuting a row or column at a time). Shifter circuit 1802 may couple data values ​​from one or more inputs to one or more outputs in a different order to produce a permutation sequence. In some embodiments, multiple outputs each carry a sequence that may be coupled to a multiplier 1803. In other embodiments, a sequence of data values ​​received at a first input is shifted to different positions and coupled to one or more outputs. In an exemplary embodiment shown below, a sequence is received at a first input, data values ​​are shifted to different positions, and a permutation sequence is sequentially generated at a second output.

[0234] In some embodiments, the permutations generated by shifter circuit 1802 may correspond to an extended core, as described above. Therefore, in some embodiments, only one sequence, rather than multiple permutation sequences, may be stored in memory, thereby advantageously reducing memory usage, for example. For instance, a sequence received from data storage 1801 may include multiple subsequences based on rows or columns of the array. Shifter circuit 1802 may permutate sequences as follows. Data values ​​may be coupled from one or more inputs of shifter circuit 1802 to one or more outputs in the same order as the sequence. Therefore, the input sequence may include one of the sequences generated by the outputs. Furthermore, for multiple permutations of subsequences (where the position of each subsequence moves within the sequence) and for multiple permutations of data values ​​within each subsequence (where the data values ​​move within each subsequence), data values ​​are coupled from one or more inputs to one or more outputs in a different order. Examples of such permutations for different arrays have been illustrated above. In one embodiment, data values ​​are received at a first input of the shifter circuit 1802, and the data values ​​are coupled from the first input to a first output to generate an input sequence at the first output. Alternatively, the data values ​​may be coupled from the first input to a plurality of second outputs in different orders to generate a plurality of permutation sequences at the plurality of second outputs. An exemplary embodiment of this method will be illustrated below in FIG18B. In another embodiment, data values ​​are received at one or more inputs of the shifter circuit 1802, and the data values ​​are shifted to different positions between the input(s) and one or more outputs to generate an input sequence and a plurality of permutation sequences.

[0235] A multiplier circuit 1803 may be coupled to one or more outputs of a shifter circuit 1802. Therefore, a sequence generated at one or more outputs of the shifter circuit 1802 may be loaded into a plurality of register circuits in the multiplier 1803. The multiplier 1803 may further receive an input data value 1820. The input data value 1820 may be loaded into a plurality of second registers. Therefore, a plurality of multiplication results 1830 may be generated. As mentioned above, expanding the core may result in, for example, a plurality of spin-out results generated in parallel. In one embodiment, the data value corresponds to an M×N array (e.g., to be applied to one core of the input data value 1820). The shifter circuit 1802 may receive an M*N length sequence from the data storage 1810 and generate M*N different sequences, each having a length of M*N. The sequences may be coupled from the shifter circuit 1802 and loaded into the multiplier circuit 1803 (e.g., in parallel).

[0236] Figure 18B illustrates a hardware diagram of an exemplary shifter circuit used in an exemplary component to generate an extended core, according to an embodiment.

[0237] In this example, the input sequence SEQ1 corresponds to an M×N array, where M and N are integers and one or more of M and N are greater than 1. Therefore, sequence SEQ1 includes M*N data values ​​D0 to D(M*N). Sequence SEQ1 can be coupled from an input to a first output (e.g., stream S(1)) including one of the unpermuted M*N data values. In addition, sequence SEQ1 can be permuted along M*N-1 different paths into M*N-1 different sequences SEQ2 to SEQM*N provided on each of M*N-1 outputs (e.g., streams S(2) to S(M*N)) including M*N permuted data values ​​D0 to D(M*N).

[0238] More specifically, in this example, a sequence SEQ1 comprising a plurality of data values ​​D0 to D(M*N) is received on a first input stream S(1). Each data value in the sequence may include one or more bytes carried on, for example, a plurality of conductors. Here, the data values ​​are coupled from a first input (e.g., stream S(1)) to a plurality of different outputs (e.g., streams S(1) to S(M*N)) in different orders along different conductive paths to generate the original sequence SEQ1 and a plurality of permutation sequences SEQ2 to SEQM*N on the plurality of different outputs (e.g., streams S(1) to S(M*N)). For example, each data value D0 to D(M*N) in the input sequence may include one or more bytes carried on a plurality of conductive lines. Conductors carrying different data values ​​are reordered differently between an input (e.g., a serial current S(1)) and a plurality of outputs (e.g., serial currents S(2) to S(M*N)) to generate a different permutation sequence at each of the plurality of outputs. More specifically, in this example, SEQ1 on input S(1) is coupled to a plurality of rotators 1860 to 1862. Rotators 1860 to 1862 can, for example, reorder the positions of the data values ​​in each sequence by changing the positions of the conductors carrying the data values. Two exemplary rotations are illustrated at 1870 and 1871. In this example, after rotation, each permutation sequence can be coupled to one input of a multiplexer circuit. Embodiments of the invention may include an integrated circuit comprising a plurality of serial currents S. The permutation sequence generated from SEQ1 via rotators 1860 to 1862 can be coupled to adjacent serial streams S(2) to S(M*N) using, for example, multiplexers 1851 to 1853. For example, each of the reordered conductors of different permutation sequences SEQ2 to SEQM*N carrying data values ​​D0 to D(M*N) can be coupled to one input of each multiplexer. A second input of each multiplexer is coupled to one of the serial streams S(2) to S(M*N). Thus, one output of each multiplexer couples one of the permutation sequences to each serial stream. In addition, the conductors of the input of the original sequence SEQ1 carrying data values ​​D0 to D(M*N) are coupled to a first output (e.g., serial stream S(1)) in the same order to generate the original sequence on the output of the serial stream S(1). Serial streams S(1) to S(M*N) carrying SEQ1 to SEQM*N can be coupled to, for example, a multiplier circuit. Therefore, in this example, all permutations of sequence SEQ1 can be advantageously generated in parallel on the same period and loaded into the multiplier in parallel.

[0239] Figure 19 illustrates a hardware diagram of an exemplary component for generating control patterns according to one embodiment. Some embodiments may include a control pattern generator circuit 1910, which may be integrated, for example, on a semiconductor surface identical to a multiplier array circuit 1920. In one embodiment, the control pattern generator 1910 is configured (e.g., via software) to receive a core dimension, such as a height (H) and a width (W). The control pattern generator 1910 may, for example, generate two bits for each of the multiplier cells 1921 to 1926 in the multiplier array 1920. One bit may be selected between a K-delay or non-K-delay input, and the other bit may be selected between a unit delay and a non-unit delay input (e.g., 1 delay or no delay). Each multiplier cell may include a storage unit (e.g., a register) for storing, for example, one of the two pattern control bits. Two bits can be used with a selection circuit to jointly select one of four data values: x, x+1, x+K, x+K+1, where x is the current block and K is a variable indicating the block size width of the input tensor, as described above. For a core with an additional dimension, the control pattern generator 1910 generates an additional bit for each additional dimension, where the bit is selected between an additional delay by a shift variable of the additional dimension or no additional delay. It should be noted that although the term "delay" is used here, in some cases the access value of the flattened input stream is not accessed via a delay, but rather via a relative or absolute addressing system that can use the delay value as an exponent value to access memory.

[0240] In this example, the control pattern generator 1910 includes a K-delay logic circuit 1911 and a unit delay logic circuit 1912. The K-delay logic circuit 1911 can receive, for example, a single dimension of a core and generate H*H bit values ​​for selecting between a K-delay input and a non-K-delay input. In this example, the K-delay logic circuit 1911 receives a height H of a core and generates H2 bit values ​​corresponding to an H×H triangular array of control patterns. Although H is used as the input to the K-delay logic circuit 1911 in this example, other embodiments may use another dimension (such as, for example, width W) as an input to the K-delay logic circuit. Furthermore, the unit delay logic circuit 1912 can receive two dimensions, such as, for example, height H and width W. The unit delay logic circuit 1912 generates H*W bit values ​​for selecting between inputs (e.g., non-delay or K-delay) (which are, for example, unit delay (+1) or non-unit delay (e.g., 0)). In this example, the unit delay logic circuit 1912 receives a core height H and a core width W and generates H*W bit values ​​corresponding to an H*W triangular array (e.g., array 2220 in Figure 22). Bit value pairs for selecting non-delay / K-delay and unit delay / non-unit delay are coupled to multiple multiplier cells 1921 to 1926 and stored (e.g.) in a storage circuit 1930 (such as a register) in each multiplier cell. A multiplexer diagram 1931 is used here to illustrate the selection, but many different selection circuits are known to those skilled in the art. In this example, a bit from the K-delay logic circuit 1911 and a bit from the unit delay logic circuit 1912 can be stored in the storage cells of each of the multiplier cells 1921 to 1926 to select (e.g.) one of a non-delay, unit delay, K-delay, or K+1 delay input value. A pattern similar to pattern 2220 shown in Figure 22 can correspond to the first bit value in the storage circuit 1930 of the multiplier array 1920, and a pattern similar to pattern 2210 in Figure 22 can correspond to the second bit value in the storage circuit 1930 of the multiplier array 1920 to control the multiplier array selection input.

[0241] Figure 20 illustrates a hardware diagram of one exemplary component for performing multiply-accumulate operations to generate an output stimulus according to an embodiment. In this example, a value from an extended core (M*N) is loaded into a first input register of a plurality of multiply-accumulate circuits, and an input value aij is loaded into a second input register of the plurality of multiply-accumulate circuits. For example, the multiply-accumulate circuit may include a plurality of multipliers 2001, 2011, 2021, 2031, 2041, and 2051, wherein the plurality of multiplier circuits may load different values ​​wij from the extended core. Each multiplier may further load an input value from one of a plurality of blocks of the flattened input stream. For example, a11 corresponds to the value at position a11 in a first block (i.e., the first column and first row in the block), which may be coupled to an input of a selection circuit 2006 (e.g., a multiplexer or "Mux"). The a11 input can be further coupled to a delay circuit 2003 to generate a delay pattern (e.g., a one-cycle delay) of a11 at a second input of the selection circuit 2006. This delay pattern selects a unit from the current block for a block. Additionally, a11 can be coupled to a K-delay circuit 2004 (e.g., a buffer or FIFO) to generate a pattern of a11 with a delay of K cycles at another input of the selection circuit 2006. This corresponds to a block in a column below the current block. Furthermore, the output of the K-delay circuit 2004 can be received at an input of another delay circuit 2005 to generate a K+1 delay pattern of a11 at yet another input of the selection circuit 2006. Similarly, multipliers 2011 to 2021 receive, for example, values ​​w12, ..., w1 (M*N) corresponding to the first column of an extended array and input values ​​a11...aMN. Similarly, other columns of the extended array can be multiplied by input values ​​a11…aMN, up to the last column of an extended array w(M*N)1,…,w(M*N)(M*N), where the columns include, for example, the permutation values ​​described above. Multipliers 2011, 2021, 2031, 2041, and 2051 include delays 2013, 2023, 2033, 2043, and 2053, K delays 2014, 2024, 2034, 2044, and 2054, delays 2015, 2025, 2035, 2045, and 2055, and selection circuits 2016, 2026, 2036, 2046, and 2056, respectively.

[0242] A control pattern can be selected among various patterns of a11, such that an input value from an appropriate block is multiplied by a weight stored in a particular multiplier. In this example, the input value is multiplied by the sequence in the first column of the extended core, so a11w11…aMNw1(M*N) has a control pattern value corresponding to A (a current block undergoing a spin). However, in this example, computing a spin across the entire block (including spins involving adjacent blocks) may involve waiting for input values ​​from adjacent blocks to arrive. Therefore, computing a spin for a given "current block" may involve waiting for K+1 cycles of adjacent block values ​​to become available. Additional control pattern values ​​can be used as described above, depending on the core dimension. For example, to compute a spin for a current block, the system may store the input value and wait for input values ​​from other blocks to become available for computation. Therefore, for example, the rotation of a specific block can be calculated using a K+1 delayed input value for a particular block, a K delayed input value as a value in an adjacent row block, a 1 delayed value as an input value in an adjacent column block, and a non-delayed value as an input value in a block adjacent to both adjacent column and row blocks. Although this specification refers to current values ​​(e.g., for the current block or non-delayed value), unit (or 1) delay values, K delay values, and K+1 delay values, it should be understood that these delays can be relative to each other (e.g., at the input of a selection circuit). Various other delays may exist in the signal path, but are omitted here for illustrative purposes. Additionally, if the core has extra dimensions, it will include additional delay values ​​(such as an M-delay value, etc.) and corresponding circuitry.

[0243] An array of input values ​​a11 to aMN (e.g., a11 to a33) can be loaded into an array of multiplier circuits 2001, 2011, ..., 2021, each of which is loaded with different weights w11, w12, ..., w1(M*N) (e.g., w33). As described above, for example, each multiplier receives an input selected using a different control pattern input from a control pattern array having the same dimension as the extended core providing the weights. The outputs of each multiplier 2001, 2011, ..., 2021 can be coupled to one or more adder circuits (e.g., adders 2007, 2017, 2027) to sum the products and output a rounded result R1. Similarly, each column of the extended core can be loaded into a plurality of multipliers and multiplied by a vector of input values ​​masked by a corresponding column of a control pattern array. For example, here, the last column of the extended core, including values ​​w(M*N)1, w(M*N)2, ..., w(M*N)(M*N), is loaded into a row of multipliers 2031, 2041, and 2051 and combined with input values ​​a11, a12, ..., aMN controlled by the last column of the control pattern array, including values ​​at positions (M*N,1), (M*N,2), ..., (M*N,M*N). For example, the outputs of the last row of multipliers are added together in adders 2037, 2047, and 2057 to produce an M*Nth spin result R(M*N). In an equivalent alternative embodiment, a delay can be coupled to the output of the multipliers.

[0244] Figure 21 illustrates an exemplary component layout 2100 for computing output stimuli in a machine learning processor according to one embodiment. The illustrated layout represents one possible layout of components (logic and / or entities) in a machine learning processor (or other processor) for performing the operations described herein. Layout 2100 includes an array of ICU 2110, a plurality of MEM blocks 2115A to 2115N, a plurality of SXM blocks 2120A to 2120N, a plurality of NIM blocks 2125A to 2125N, and supercells 2130A-A to 2130N-N. These elements are collectively referred to in the following description by omitting their letter suffixes.

[0245] ICU 2110 issues commands to NIM block 2125. These commands instruct NIM block 2125 to configure supercell 2130 with the correct control sample values ​​to perform the spin operation described herein. Supercell 2130 is configured as a square grid. Each supercell 2130 has a size indicating how many values ​​it can process in parallel. All supercells 2130 may have the same size. For example, each supercell may be a 16×16 grid and can process 256 values.

[0246] In one embodiment, the input channels are vertically (as indicated by 2145) divided between supercells 2130, such that each input channel, after being flattened, is processed by one or more layers of supercells 2130 (layers indicated by the first letter in the naming scheme). When an input layer may be smaller than the size of a supercell 2130, or when the vertical dimension of the input channel is not uniformly divided into the vertical dimension of a supercell 2130, one or more layers may contain only a fractional portion of one layer. In the same embodiment, the output features (i.e., the different output "channels") are horizontally (as indicated by 2155) divided between supercells 2130. Therefore, one or more rows (or fractional portions) of supercells 2130 (indicated by the second letter in the naming scheme) are used to process each output feature.

[0247] Supercell 2130 itself can be similar to the multiply-accumulate unit described above. After performing a spin operation on the input value 2140, the supercell transmits its output to the supercell 2130 above it. This allows the spin operations of different input layers to be summed together to produce an output feature 2150 from multiple input layers. If there are multiple output channels, multiple sets of output features 2150 are generated using the method described above (e.g., in Figures 16E to 16F).

[0248] In a first clock cycle, ICU 2110 instructs the first NIM block 2125A to configure one or more supercells 2130 associated with that NIM block 2125A. In the illustrated example, supercells 2130 on the same layer as NIM block 2125A are configured by NIM block 2125A. These are supercells 2130A-* (where * is any letter from A to N). In a second clock cycle, NIM block 2125A uses the method described in Figures 13A to 13B to configure supercells 2130A-A with correct control pattern values. For the first supercell 2130A-A, the column and row numbers do not have an offset when generating control pattern values. However, for any other supercell 2130 besides supercells 2130A-A, the column and row numbers used to generate control pattern values ​​have an additional offset. For row numbers, this offset is equal to the number of supercells 2130 to the left ("west") of the configured supercell 2130 multiplied by the cell width of a supercell 2130. For column numbers, this offset is equal to the number of supercells 2130 below ("south") the configured supercell 2130 multiplied by the cell height of a supercell 2130. Additionally, the offset value can be filled by processing a previously used input or output channel, depending on the fraction of a supercell. This offset is used because the size of blocks, control patterns, and extended cores is not necessarily a multiple of the supercell size; therefore, different blocks can span different supercells 2130. This means that control sample values ​​can also span different supercells 2130. Therefore, it is necessary to calculate each control sample value based on the positional offset of the supercell 2130 to generate the correct control sample value corresponding to the block, several blocks, or the fraction of the block that the supercell 2130 will process.

[0249] In parallel (within the same clock cycle) with configuring supercells 2130A-A with control pattern values, NIM block 2125A transmits instructions to subsequent NIM blocks 2125B (NIM blocks above or "north" of the current NIM block 2125) that use instructions to configure their supercells 2130B-*. This instruction includes the previously mentioned offset value, causing NIM block 2125B to configure supercells 2130B-* with the correct control pattern values ​​taking the offset into account. This propagation of instructions is repeated until a number of supercells 2130 equal to the input size are configured by NIM block 2125.

[0250] During the configuration of one (or more) supercells 2130 in a superchannel, a NIM block 2125 can configure only one specific channel (i.e., one output channel) of supercell 2130 for output. The channel can be specified in instructions from ICU 2110. To configure additional channels for output of supercell 2130, ICU 2110 can issue additional instructions to NIM block 2125 per clock cycle to configure additional channels for output of supercell 2130. These instructions contain the correct offset values, ensuring that the supercell 2130 processing the output layer has the correct column and row position values ​​capable of generating the correct control pattern values.

[0251] In each clock cycle, instructions from ICU 2110 propagate "northward" from one NIM block 2125 to a subsequent NIM block 2125, while ICU 2110 continues to issue instructions to the supercell in an "eastward" direction. Therefore, the supercell 2130 is configured in a diagonal "wave" over multiple clock cycles. In this manner, the entire set of input and output channels is configured after several clock cycles. Once this is completed, the input value 2140 is received by the system described herein and processed block-by-block by the supercell 2130 in a procedure similar to the multiply-accumulate operation described above to produce the output feature 2150.

[0252] Figure 22A illustrates an example machine learning processor according to one embodiment. Features and advantages of the invention include a processor circuit capable of performing spin operations at very high speeds. In this example, a server computer 2201 (or host) may include configuration software 2210, which may receive one or more cores 2213 for performing spin operations. The processor circuit 2200 may be a machine learning processor, referring to a circuit that receives input data (e.g., stimuli, such as images) and applies a core (e.g., neural network weights) to the input data as part of, for example, a machine learning (also known as artificial intelligence) algorithm. In this example, the configuration software 2210 (sometimes referred to as a compiler) receives the neural network weights 2213. In this example, an extension core or control pattern (or both) may be generated in the software. For example, weight 2213 may be analyzed by a core generator 2212 and a control pattern generator 2211 to generate a control pattern 2225 and an extended core 2226, which may be stored in a data storage 2220 (such as, for example, memory or temporary register circuitry) in the processor circuitry 2200. Input value 2280 may be received in an input / output (I / O) circuitry 2222 and coupled to a multiply-accumulate circuitry 2221, for example, via a data bus 2224. Extended core 2226 and control pattern 2225 may also be coupled to the multiply-accumulate circuitry 2221. For example, the multiply-accumulate circuitry 2221 may produce the spin-around result described above. In some embodiments, intermediate spin-around result 2227 is stored in data storage 2220 and is applicable to new stimulus input 2280. In other embodiments, spin-around result 2290 may be provided as, for example, an output. Alternatively, the rotation result can be processed by other circuits 2223 (such as, for example, a vector processor (not shown in the figure)).

[0253] Figure 22B illustrates an example machine learning processor according to another embodiment. In this example, the core is analyzed by configuration software, but extended cores and control patterns can be generated in hardware. For example, one or more cores (e.g., neural network weights) 2262 are received in configuration software (e.g., a compiler) 2260 and analyzed by a core analyzer component 2261 to generate configuration information for generating an extended core containing a value K (which can be derived from the size of a core and other information (dimensions of the input to be processed)). Server computer 2251 can send the configuration information and the value K to processor circuitry 2250. In this example, K may correspond to, for example, the number of blocks in each column of input 2285, where the number of blocks is the same as the size of the core. The configuration information can be used to configure control pattern generator 2277 and shifter circuitry 2278. The shifter circuit 2278 can be configured to replace the core to generate an extended core, for example, from one or more cores 2275 stored in the data storage 2270.

[0254] In this embodiment, an extended core is generated in hardware. One or more cores 2275 may be stored in data memory 2270. For example, a core may be stored in data memory 2270 as a sequence of values, wherein a sequence is accessible from data memory 2270 and coupled to shift circuitry 2278 (e.g., simultaneously). Various shift circuits may be used to permutate the input sequence and generate an extended core, as described above. An example circuit for permuting the sequence to generate an extended core will be described in more detail below. The value of the extended core may be loaded into, for example, a multiply-accumulate unit 2271. Extending the core in hardware can reduce the amount of data storage required, which may be advantageous in some embodiments.

[0255] Embodiments of the present invention include configuring a K-delay circuit in series with a multiply-accumulate circuit 2271. In this example, an input value 2285 may be coupled to a delay circuit 2279, which may be a programmable delay circuit such as, for example, a programmable buffer. In one embodiment, the delay 2279 may be configured to have, for example, a delay K, which may vary depending on the core size and the number of blocks along one dimension of an input array. In one embodiment, the delay 2279 may be a first-in-first-out (FIFO) circuit in which, for example, the output is a delayed input and the delay is set to K. The input value may also be directly coupled to the multiply-accumulate circuit 2271 without delay. Thus, the multiply-accumulate circuit 2271 can receive, for example, the current input value and an input value delayed by K cycles.

[0256] The control pattern generator 2277 can generate control signals for implementing a control pattern, such that various delayed or non-delayed input data values ​​are combined with specific weights, as described above. The specific pattern control signal can correspond to, for example, a core, by which it is applied to paths for each input block and to paths for flattening blocks and cores. The output of the control pattern generator 2277 can be used to select, for example, various delay patterns of the input values, as described above.

[0257] Therefore, the multiply-accumulate circuit 2271 receives the current input value, the K-delay input value, an extension core, and a control signal for selecting different inputs. As described above, the multiply-accumulate circuit 2271 may include additional unit delay circuitry for delaying both the current input value and the K-delay input value, and the control signal from the control pattern generator 2277 can be used to select the input value suitable for multiplication with, for example, different values ​​in the extension core. The output of the multiply-accumulate circuit 2271 may be an intermediate spinner 2276, which may, for example, be stored in the data memory 2270 and spinnered with subsequent inputs 2285, or the spinner result may be output at 2295. Alternatively, the spinner result may be processed by other circuitry 2273 (such as, for example, a vector processor (not shown)). [Example of a computing machine architecture] [ ]

[0258] Figure 23 is a block diagram illustrating the components of an instance computing machine capable of reading instructions from a computer-readable medium and executing those instructions in a processor (or controller). A computer described herein may include a single computing machine shown in Figure 23, a virtual machine, a distributed computing system comprising multiple nodes of the computing machine shown in Figure 23, or any other suitable configuration of computing devices. The computer described herein may be used by any of the elements described in the preceding figures to perform the described functions.

[0259] For example, Figure 23 illustrates a graphical representation of a computing machine in an exemplary form of a computer system 2300 containing instructions 2324 (e.g., software, program code, or machine code), which may be stored in a computer-readable medium for causing the machine to execute any one or more of the programs discussed herein. In some embodiments, the computing machine operates as a stand-alone device or is connectable (e.g., networked) to other machines. In a network deployment, the machine may act as a server machine or a client machine in a server-client network environment or a peer machine in a peer-to-peer (or distributed) network environment.

[0260] The architecture of one of the computing machines described in Figure 23 can correspond to any software, hardware, or combination of components shown in the figure above. Although Figure 23 shows a variety of hardware and software components, each of the components described in Figures 1 and 2 may contain additional or fewer components.

[0261] For example, a computing machine can be a personal computer (PC), a tablet PC, a video converter (STB), a digital assistant (PDA), a cellular phone, a smartphone, a web device, a network router, an Internet of Things (IoT) device, a switch or bridge, or any machine capable of executing instructions 2324 specifying actions to be taken by the machine. Furthermore, although only a single machine is depicted in the figure, the term "machine" should also be considered as any collection of machines that individually or collectively execute instructions 2324 to perform the methods discussed herein.

[0262] The instance computer system 2300 includes one or more processors (collectively referred to as processor 2302) (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), one or more application-specific integrated circuits (ASIC), one or more radio frequency integrated circuits (RFIC), or any combination thereof), a main memory 2304, and a static memory 2306, which are configured to communicate with each other via a bus 2308. The computer system 2300 may further include a graphics display unit 2310 (e.g., a plasma display panel (PDP), a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)). The computer system 2300 may also include a digit input device 2312 (e.g., a keyboard), a cursor control device 2314 (e.g., a mouse, a trackball, a joystick, a motion sensor or other indicator), a storage unit 2316, a signal generating device 2318 (e.g., a speaker), and a network interface device 2320, which are also configured to communicate via bus 2308.

[0263] Storage unit 2316 includes a computer-readable medium 2322 on which instructions 2324 are stored to embody any one or more of the methods or functions described herein. Instructions 2324 may also reside wholly or at least partially in main memory 2304 or processor 2302 (e.g., in a processor's cache memory) during execution by computer system 2300, and main memory 2304 and processor 2302 also constitute the computer-readable medium. Instructions 2324 may be transmitted and received on a network 2326 via network interface device 2320.

[0264] Although computer-readable media 2322 is shown as a single medium in one exemplary embodiment, the term "computer-readable media" should be considered as including one or more media (e.g., a centralized or distributed database or associated cache and server) capable of storing instructions (e.g., instruction 2324). Computer-readable media may include any medium capable of storing instructions (e.g., instruction 2324) that are executed by a machine and cause the machine to perform any or more of the methods disclosed herein. Computer-readable media may include (but is not limited to) a data storage library in the form of solid-state memory, optical media, and magnetic media. Computer-readable media does not include transient media such as a signal or a carrier wave. [Additional configuration considerations] [ ]

[0265] Specific embodiments described herein include, for example, logic or components, engines, modules, or mechanisms, as illustrated in the figures above. An engine may constitute a software module (e.g., code embodied on a computer-readable medium) or a hardware module. A hardware engine is a tangible unit capable of performing a specific operation and can be configured or arranged in a particular manner. In exemplary embodiments, one or more computer systems (e.g., a standalone, client, or server computer system) or one or more hardware engines (e.g., a processor or a group of processors) of a computer system may be configured by software (e.g., an application or a portion of an application) to operate as a hardware engine to perform a specific operation, as described herein.

[0266] In various embodiments, a hardware engine may be implemented mechanically or electronically. For example, a hardware engine may include dedicated circuitry or logic permanently configured (e.g., configured as a dedicated processor, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)) to perform specific operations. A hardware engine may also include programmable logic or circuitry temporarily configured in software to perform specific operations (e.g., encompassed within a general-purpose processor or another programmable processor). It should be understood that the decision to mechanically implement a hardware engine in dedicated and permanently configured circuitry or in temporarily configured circuitry (e.g., software-configured) may be driven by cost and time considerations.

[0267] The various operations of the exemplary methods described herein may be performed, at least in part, by one or more processors (e.g., processor 2302) configured, either temporarily (e.g., by software) or permanently, to perform the relevant operations. Whether temporarily or permanently configured, these processors may constitute a processor implementation engine that operates to perform one or more operations or functions. In some exemplary embodiments, the engine mentioned herein may include a processor implementation engine.

[0268] The execution of a specific operation can be distributed across one or more processors that reside not only within a single machine but also across several machines. In some exemplary embodiments, one or more processors or processor implementation modules may be located in a single geographic location (e.g., a home environment, an office environment, or a server farm). In other exemplary embodiments, one or more processors or processor implementation modules may be distributed across several geographic locations.

[0269] Upon reading this invention, those skilled in the art will understand, through the principles disclosed herein, additional alternative structures and functional designs for similar systems or programs. Therefore, although specific embodiments and applications have been illustrated and described, it should be understood that the disclosed embodiments are not limited to the precise constructions and components disclosed herein. Various modifications, alterations, and variations, which will be apparent to those skilled in the art, can be made to the configuration, operation, and details of the methods and apparatus disclosed herein without departing from the spirit and scope defined in the appended claims.

[0270] 100: System 102: Input Tensor 104: Input Channel 104A to 104N: Input Channels 106A: Dimension 1 106B: Vitamin 2 106C: Vitamin 3 108: Input value 110: Core 112: Core Filter 112A to 112M: Core Filter 114: Core Value 122: Input Stream Generator 124: Flatten the input stream 126: Extended Core Generator 128: Extended Core 132: Core size input 134: Control Pattern Generator 136: Control Sample 138: Block Number Indicator 140: Multiply-Accumulate Unit 142: Output stimulus 144: Output Features 144A to 144M: Output Characteristics 202: Input value 204: Fill 206: Heavy weighted dark box / bounding box 208: Select the turning point 210: Core 212: Spin Operation 214: Output value 218: Repeatedly read value 300: Flowchart Steps 310 to 330 400: Instance 401: Receive Input 402: Filling input based on core size 403: Tile the input into chunks of equal size to the core. 404: Flattening the input by converting the tiled input into a stream ordered by block numbers. 420: Input 422: Core 424: Fill in the input 426A: Fill 426B: Fill 426C: Filler 428: Tile Input 430: Flatten Input 432: Input direction 500: Instance 501: Receives input and fills based on core size 502: Tiles the input into chunks of equal size to the core. 503: Flatten the image according to the block order 504: Use the first flattened input to tile the additional input channel 520: Fill in the input 522: Core 524: Tile Input 528: Flatten Input 530: Single Block 532: Flattening sequence 534: Input direction 538: Flattening sequence 540: Input Channel 540A-N: Input Channel 542: Core 544: Flatten Input Channel 544A to 544N: Flattened Input Channels 546: Flatten the input stream 600: Targeting 3D Core 601: Receives input and fills based on core size 602: Tile the input into chunks of equal size to the core. 603: Flatten the image according to the block order 620: X-axis 622: Y-axis 624: Z-axis 626: Fill Input / Block Input 628: Fill 630: Input value 632: Core 634: Core Value 636:K variable 638: Layer size ("M") 640: Flattening sequence 642: Flatten Input 644: Sequence 700: Flowchart 710 to 760: Steps 800: Instance 801: Receiver Core 802: Rotating Core 803: Generate extended core 820: Core 822: Rotation 824: Rotation 826: Extended Core 900: For 2D cores 901: Receiver Core 902A: Rotate the first dimension, and replace all rotations of the columns (column priority). 902B: Rotate the first dimension and permutate all rotations of the column (row-major order). 903A: Rotate the second dimension, replacing all rotations of the block generated by the first-dimensional rotation (column-first method). 920: Core 920A to 920C: Two-dimensional core 922A to 922C: Core Rotation 924A to 924C: Core Rotation Blocks 926A to 926B Blocks 928A to 928C Blocks 930A to 930C 932A: Extended Core 932B: Extended Core 932C: Extended Core 934: Core Rotation 936: Core Rotation Blocks 938A to 938C 940: Extended Core 942A to 942N: Core Filters 944A to 944N: Extended Core Filter 1000: Examples 1001: Receiver Core 1002: Rotate the first dimension, and replace all rotations of the columns (column priority). 1003A: Rotate the second dimension: a) Divide the blocks into vectors along the axis of the dimension. 1003B: Rotate the second dimension: 2) Generate all rotations of blocks generated from the first dimension rotation, and process each group of blocks separately (column priority order). 1004A: Rotate the third dimension: a) Divide the blocks into vectors along the axis of the dimension. 1004B: Rotate the third dimension: 2) Generate all rotations of blocks generated from the second-dimensional rotation, and process each group of blocks separately (column-by-column method). 1022: Y-axis 1024: Z-axis 1026: Core 1028: Floor 1028A: Floor 1028B: Layer 1030: Core Rotation 1032A: Vector 1032B: Vector 1034A: Group 1034B: Group 1038A: Group 1040: Vector 1044: Extended Core 1100 to 1140: Steps 1200: Examples 1201: Flatten Input 1202: Perform element-wise multiplication of flattening blocks and expanding cores. 1203: Determine which block values ​​are needed to generate the element-wise spiral product (crossing columns one by one) based on the core positions within one of the original input blocks. 1218: Flatten Input 1220: Core 1222A: Block 1 1222B: Block 2 1222C: Block 3 1222D: Block 4 1224: Extended Core 1226A: Element-wise product 1226B: Element-wise product 1226C: Element-wise product 1226D: Element-wise product 1230A: Location 1 1230B: Position 2 1230C: Position 3 1230D: Position 4 1232: Core position 1234A: Element-wise product pairs 1234B: Element-wise product pairs 1234C: Element-wise product pairs 1234D: Element-wise product pairs 1236A: Control Sample 1236B: Control Sample 1236C: Control Sample 1236D: Control Sample 1300: Instance 1301: Instance 1302: Example 1310:3×1 1D core 1311: Generate values ​​for all positions of the control pattern. 1312: Test results / control patterns for determining position using column and row numbers 1313: Refer to the table to find the indicator value. 1314:3×3 2D Core 1316: Control Sample 1318:2×2×2 3D Core 1320: Control Sample 1322: Core 1324: Remaining value 1326: Test Results 1330: Inequality Test 1332: Results Table 1351: Generate values ​​for all positions of the control pattern. 1352: Test results for determining position using column and row numbers 1353: Refer to the table to find the indicator value. 1360: Control Sample 1362: Three-dimensional core 1364: Final Remaining Value 1366: Test Results 1368H: Indicator 1370: Test Table 1372: Results Table 1380:1D Core 1382:1D Core Control Pattern 1384: 2D Core 1386:2D Core Control Pattern 1388:3D Core 1390: 3D Core Control Pattern 1400: Flowchart 1410 to 1440: Steps 1500: Examples 1501A: Processing block number 1 1502A: Further processing of output position 1 of step 3 1503A: Processing the second position 2 of 3 1504A: Processing output position 3 1505: Repeat procedure 1501A (e.g., 1501B through 1501L) for all blocks in the input. 1511A: The flattened input selection value is derived from the block position (i.e., delay amount) indicated in the control pattern of the output position. 1512A: Multiply and accumulate selected input values ​​and a vector selected from one of the extended kernels (corresponding to the position in the output block). 1513A: The flattened input selection value is derived from the block position (i.e., delay amount) indicated in the control pattern of the output position. 1514A: Multiply and accumulate selected input values ​​and a vector selected from one of the extended kernels (corresponding to the position in the output block). 1515A: The flattened input selection value is derived from the block position (i.e., delay amount) indicated in the control pattern of the output position. 1516A: Multiply and accumulate selected input values ​​and a vector selected from one of the extended kernels (corresponding to the position in the output block). 1522: Flatten Input 1524: Control Sample 1524A: Extended core vector 1526: Extended Core 1528A: Control Sample Vector 1530A: Select input value 1532A: Extended Core Vector 1534A: Output 1536A: Control Sample Vector 1538A: Select input value 1542A: Output the extended core vector at position 2 in the block. 1544A: Output 1546A: Control Sample Vector 1548A: Select input value 1552: Extended Core Vector 1552A: Output the extended core vector at position 3 in the block. 1554A: Output 1600: Example 1601A: Processing block number 1 1602A: Output position 1 of process 9 1603A: Processing output position 2 1606A: Processing output position 5 1611A: The buffer flattening input selection value is derived from the block position (i.e., delay amount) indicated in the control pattern of the output position. 1612A: Multiply and accumulate selected input values ​​and a vector selected from one of the extended kernels (corresponding to the position in the output block). 1614A: Multiply and accumulate selected input values ​​and a vector selected from one of the extended kernels (corresponding to the position in the output block). 1615A: The flattened input selection value is derived from the block position (i.e., delay amount) indicated in the control pattern of the output position. 1616: Repeat the above procedure for all output positions in block 1. 1617: Repeat the above procedure for all blocks of input. 1618: Repeat the above procedure for the input channel and the core channel. 1622: Flatten Input 1624: Control Sample 1626: Extended Core 1628A: Control Sample Vector 1630A: Select input value 1632A: Extended core vector 1634A: Output 1636A: Control Sample Vector 1638A: Select input value 1642A: Extended core vector 1644A: Output 1646A: Control Sample Vector 1648A: Select input value 1652A: Extended Core Vector 1654A: Output 1656: Flatten Input Channel 1658: Select input channel value 1660A to 1660N: Core Filter 1662A: Rotary Output 1664A to 1664N: Output Excitation Channel 1666: Sub-filter core 1668: Use extended core "sub-filters" to perform dot product operations. 1670: Sum the rotary outputs to create an output channel. 1700: Targeting the 3D kernel 1701: Process block number X 1702: Output position 8 of processing 8 1711: The buffer flattening input selection value is derived from the block position (i.e., delay amount) indicated in the control pattern of the output position. 1712: Multiply and accumulate selected input values ​​and select extended core vectors based on output block positions. 1713: Repeat for all blocks and channels 1722: Flatten Input 1724: Control Pattern Vector / Control Panel Vector 1726: Select input value 1728: Select input value 1730: Extended Core Vector 1732: Output 1801: Data storage circuit / data storage device 1802: Shifter Circuit 1803: Multiplier / Multiplier Circuit 1810: Data value 1820: Input data value 1830: Multiplication Result 1851–1853: Multiplexer 1860–1862: Rotator 1870: Rotation 1871: Rotation 1910: Control Pattern Generator Circuit / Control Pattern Generator 1911:K Delay Logic Circuit 1912: Unity Delay Logic Circuit 1920: Multiplier Array Circuit / Multiplier Array 1921–1926: Multiplier Cell 1930: Storage Circuit 1931: Multiplexer Diagram 2001: Multiplier 2003: Delay Circuits 2004:K Delay Circuit 2005: Delay Circuits 2006: Selection Circuit 2007: Adder 2011: Multiplier 2013: Delay 2014:K Delay 2015: Delay 2016: Selection Circuit 2017: Adder 2021: Multiplier 2023: Delay 2024:K Delay 2025: Delayed 2026: Selection Circuit 2027: Adder 2031: Multiplier 2033: Delay 2034:K Delay 2035: Delay 2036: Selection Circuit 2037: Adder 2041: Multiplier 2043: Delay 2044:K Delay 2045: Delay 2046: Selection Circuit 2047: Adder 2051: Multiplier 2053: Delay 2054:K delay 2055: Delay 2056: Selection Circuit 2057: Adder 2100: Component Layout 2110:ICU 2115A to 2115N: MEM Block 2120A to 2120N: SXM Block 2125: NIM Blocks 2125A to 2125N: NIM Blocks 2130: Super Cell 2130A-A to 2130N-N: Supercell 2140: Input value 2145: Vertical 2150: Output Features 2155: Level 2200: Processor Circuit 2201: Server Computer 2210: Model / Configuration Software 2211: Control Pattern Generator 2212: Core Generator 2213: Core / Neural Network Weights 2220: Array / Model / Data Storage 2221: Multiply-accumulate circuit 2222: Input / Output (I / O) Circuit 2223: Other circuits 2224: Data Bus 2225: Control Sample 2226: Extended Core 2227: Intermediate Revolving Result 2250: Processor Circuit 2251: Server computer 2260: Configuration Software 2261: Core Analyzer Component 2262: Core 2270: Data storage 2271: Multiply-accumulate / Multiply-accumulate circuit 2273: Other Circuits 2275: Core 2276: Intermediate Revolving Point 2277: Control Pattern Generator 2278: Shifter Circuit 2279: Delay circuit / delay 2280: Input value 2285: Input / Input Value 2290: Reversal Result 2295: Reversal Result 2300: Computer System 2302: Processor 2304: Main Memory 2306: Static Memory 2308: Busbar 2310: Graphics Display Unit 2312: Digital Input Device 2314: Vernier Input Device 2316: Storage Unit 2318: Signal generating device 2320: Network Interface Device 2322: Computer-readable media 2324: Instruction 2326: Network

Claims

1. A processor device comprising: A control pattern generator circuit configured to receive information about one or more dimensions of a core and to generate a control pattern at least in part based on the information, wherein the one or more dimensions include one or more of a height and a width; and a multiplier circuit configured to receive the control pattern and to select one or more input values ​​at least in part based on the control pattern.

2. The processor device of claim 1, wherein the control pattern generator circuit is configured to generate the control pattern based on one or more test inequalities.

3. The processor device of claim 2, wherein the one or more test inequalities include an inequality between a modulo operation of a column number at a position in the control pattern and a modulo operation of a row number at the position in the control pattern.

4. The processor device of claim 2, wherein the control pattern generator circuit is configured to generate the control pattern by referencing a table of control pattern values ​​based on one or more test inequalities.

5. The processor apparatus of claim 1, wherein the multiplier circuitry includes a plurality of multiplier cells, and wherein the control pattern generator circuitry is configured to provide one or more control pattern values ​​to each of the plurality of multiplier cells.

6. The processor device of claim 5, wherein the one or more control sample values ​​include one or more delay values, and wherein each multiplier cell is configured to delay one or more operations at least in part based on the one or more delay values.

7. The processor apparatus of claim 6, wherein each multiplier cell is configured to delay the one or more operations by a number of clock cycles equal to a base X plus zero or more additional amounts, wherein X is a non-negative integer, and wherein the one or more delay values ​​include at least a first bit and a second bit, the first bit indicating whether to increase the number of clock cycles to delay by K clock cycles, and the second bit indicating whether to increase the number of clock cycles to delay by L clock cycles, wherein K and L are positive integers.

8. The processor device of claim 6, wherein each multiplier cell is configured to access data from a flattened input stream at a time determined based on one or more delay values.

9. The processor device of claim 1, further comprising a shifter circuit configured to generate an extended core based on the core.

10. The processor device of claim 9, wherein the shifter circuit includes one or more rotators configured to sort data received by the rotators by controlling one or more positions of one or more conductive data paths of the one or more rotators.

11. A computing system, comprising: A plurality of processors, each processor comprising: a control pattern generator circuit configured to receive information about one or more dimensions of a core and at least partially generate a control pattern based on the information, wherein the one or more dimensions include one or more of a height and a width; and a multiplier circuit configured to receive the control pattern and at least partially select one or more input values ​​based on the control pattern.

12. The computing system of claim 11, wherein the control pattern generator circuit is configured to generate the control pattern based on one or more test inequalities.

13. The computing system of claim 11, wherein the multiplier circuit includes a plurality of multiplier cells, and wherein the control pattern generator circuit is configured to provide one or more control pattern values ​​to each of the plurality of multiplier cells.

14. The computing system of claim 13, wherein the one or more control sample values ​​include one or more delay values, and wherein each multiplier cell is configured to delay one or more operations at least in part based on the one or more delay values.

15. The computing system of claim 11, wherein each of the plurality of processor devices further includes a shifter circuit configured to generate an extended core based on the core.

16. A method for performing calculations, comprising: The computing system comprising one or more computing devices obtains information about one or more dimensions of a core, wherein the one or more dimensions include one or more of a height and a width; the computing system comprises a control pattern generator circuit, which generates a control pattern based at least in part on the information; and the computing system comprises a multiplier circuit, which multiplies two or more output values, wherein at least one of the two or more output values ​​is selected based at least in part on the control pattern.

17. The method of claim 16, wherein generating the control pattern includes generating the control pattern based on one or more test inequalities.

18. The method of claim 16, wherein the multiplier circuit includes a plurality of multiplier cells, and the method further includes providing one or more control pattern values ​​to each of the plurality of multiplier cells by means of the control pattern generator circuit.

19. The method of claim 18, wherein the one or more control sample values ​​include one or more delay values, and the method further includes delaying one or more operations of the multiplier cell by each multiplier cell at least in part based on the one or more delay values.

20. The method of claim 16, wherein the computing system includes a shifter circuit configured to generate an extended core based on the core.

Images