Rotating accumulator for vector operations

Abstract

A processing unit for generating an output vector is provided. The processing unit comprises an output vector register and a vector unit and is configured to execute machine code instructions, each instruction being an instance of a predefined set of types of instructions of an instruction set of the processing unit. The instruction set comprises vector processing instructions defined by a corresponding opcode, which cause the processing unit to: i) process at least two input vectors using the vector unit to generate a result value; ii) perform a rotation operation on a plurality of elements of the output register, wherein the result value or a value based on the result value is placed in a first end element of the output register.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a processor having an execution unit for implementing vector operations such as matrix-vector operations. BACKGROUND

[0002] The processor architecture of a given processor is designed to execute instructions instantiated from a specified instruction set. The instruction set of a processor is the basic defined set of machine code instructions of which the processor is configured to execute. These instructions include a plurality of compute instructions, for example arithmetic instructions such as addition, multiplication and the like. Each instruction executed is an instance of one of the instruction types in the instruction set. Each instruction defined in the instruction set is a machine code instruction formed of an opcode and zero or more operand fields, where the opcode specifies the operation to be performed and the operand fields, if any, are used to specify the operand or operands to be operated on by the specified operation.

[0003] Some processors are configured with vector processing instructions as part of their instruction set. A typical processor of this type includes a vector processing unit (VPU) for performing vector operations in response to corresponding machine code instructions. For example, the VPU can perform a multiply-accumulate operation on two input vectors in response to a corresponding machine code instruction, the output of which is equal to the inner (dot) product of the two input vectors. SUMMARY

[0004] The vector unit can be used to perform vector operations which form part of a larger task. However, this requires the results of each vector operation to be stored and arranged accordingly, which can require an additional operand in the instruction, memory usage to move the result values, or both. It would be desirable to provide a more streamlined mechanism of implementation.

[0005] According to a first aspect disclosed herein, there is provided a processing unit for generating an output vector, the processing unit comprising: an output register having a plurality of elements for holding different components of the output vector, the plurality of elements comprising a first end element and a second end element; and a vector unit for processing at least two input vectors to generate a corresponding result value; wherein the processing unit is configured to execute machine code instructions, each instruction being an instance of a predefined set of instruction types in an instruction set of the processing unit, wherein the instruction set comprises a vector processing instruction defined by a corresponding opcode, the vector processing instruction causing the processing unit to: i) process a first input vector and a second input vector using the vector unit to generate a result value; ii) perform a rotation operation on the plurality of elements of the output register, wherein the result value or a value based on the result value is placed in the first end element of the output register.

[0006] In an example, the value placed in the first end element of the output register as part of the rotate operation is the sum of the result value and the value present in the second end element prior to the rotate.

[0007] An advantage of this particular example is that the sequence of instructions one after the other, each requiring only an adjustment of the pointer in memory, can handle cases where multiple passes are required. This is the case, for example, when implementing matrix and / or vector greater than matrix-vector multiplication that can be handled by the vector unit (or held in input registers).

[0008] In an example, the vector unit is configured to perform the processing i) of the first and second input vectors by performing a multiply-accumulate operation on the input vectors, the result value being a scalar value resulting from the multiply-accumulate operation.

[0009] In an example, the vector unit is configured to access at least the second input vector directly from memory.

[0010] In an example, the processing unit comprises an input register for holding at least the first input vector, and the vector unit is configured to access the first input vector from the input register.

[0011] In an example, the input register is selected from a plurality of registers in a register file, and the vector processing instruction comprises an operation specifying the input register.

[0012] In an example, the input register is implicitly present in the opcode of the vector processing instruction, rather than being specified by an operand of the vector processing instruction.

[0013] In an example, the output register is selected from a plurality of registers in a register file, and the vector processing instruction comprises an operand specifying the output register.

[0014] In an example, the output register is implicitly present in the opcode of the vector processing instruction, rather than being specified by an operand of the vector processing instruction.

[0015] According to a second aspect disclosed herein, there is provided a method, performed by a processing unit, of generating an output vector at an output register having a plurality of elements for holding different components of the output vector, the plurality of elements including a first end element and a second end element, wherein the processing unit is configured to execute machine code instructions, each instruction being an instance of a predefined set of types of instructions in an instruction set of the processing unit, wherein the instruction set includes vector processing instructions defined by corresponding opcodes, the method including a vector processing instruction which causes the processing unit to perform a vector processing operation: i) to process a first input vector and a second input vector using the vector unit to generate a result value; and ii) to perform a rotation operation on the plurality of elements of the output register, wherein the result value or a value based on the result value is placed in the first end element of the output register.

[0016] In an example, the value placed in the first end element of the output register as part of the rotation operation is the sum of the result value and the value present in the second end element prior to the rotation.

[0017] In an example, the first input vector is stored in a register, and the method includes performing a plurality of said vector processing operations using the same first input vector. This can be used, for example, to perform a matrix multiplication.

[0018] In an example, the method includes, after completing a plurality of said vector processing operations, outputting the values present in the output register to a storage location.

[0019] In an example, the method includes, after completing a number of vector processing operations which is not an integer multiple of the number of elements in the output register, performing one or more rotation operations on the output register without changing the values present in the elements of the output register such that the total number of rotations is equal to an integer multiple of the number of elements in the output register.

[0020] According to a third aspect disclosed herein, there is provided a computer system comprising a processing unit according to the first aspect, programmed to perform the method according to the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0021] To assist in understanding the present disclosure and to show how embodiments can be put into effect, reference is made by way of example to the accompanying drawings, in which:

[0022] Figure 1 A computer system according to examples described herein is shown schematically;

[0023] Figure 2 A first example implementation of a vector processing instruction is shown schematically;

[0024] Figure 3a and Figure 3bTwo examples of rotate operations are shown in more detail;

[0025] Figures 4a to 4d A first example of using vector processing instructions to process matrices and vectors is shown;

[0026] Figure 5a and Figure 5b A second example of using vector processing instructions to process matrices and vectors is shown;

[0027] Figure 6 A second example implementation of a vector processing instruction is shown schematically;

[0028] Figure 7 An example method performed by a processing unit is shown schematically; and

[0029] Figure 8 Another example method performed by a processing unit is shown schematically. DETAILED DESCRIPTION

[0030] The present disclosure relates to the design of instructions and micro-architectural structures that can be used to efficiently implement matrix-vector operations.

[0031] As used herein, the term "instruction" refers to a machine code instruction, i.e. one of the basic instruction types of a processor instruction set, each instruction type being defined by a single opcode and one or more operand fields. The operands can be direct operands, i.e. the values to be operated on are directly encoded into the instruction, or the operands can take the form of indirect operands, i.e. the address of the value to be operated on can be found. For example, an add instruction can take three pointers as operands: two specifying the addresses from which to fetch the values to be added, and one specifying the destination address to which to write the result. In practice, the values in the source operand fields of vector multiplication instructions and the like are typically pointers to vector registers holding the actual vector values to be processed, rather than direct operands.

[0032] An example of a vector operation is a matrix-vector operation. Matrix-vector operations have applications in many areas of computing. For example, matrix-vector products can be used to implement convolution layers in deep learning environments such as convolutional neural networks. In this case, an array of input values (e.g. an array of pixel values for an image) can be convolved with one or more filters (also known as kernels). This involves determining the inner (dot) product of the filter and a subset of the input array known as a receptive field. When the filter is passed over the input array, an inner product is determined for each receptive field. Each inner product so determined represents a different value in an output array known as an activation map or feature map. Thus, a convolutional layer can be thought of as a multiplication of a vector (representing the filter) with a matrix (each row of the matrix representing a different receptive field). This operation can be performed more than once, e.g. on each individual colour channel of an RGB image.

[0033] Typically, determining the product of a vector V and a matrix M involves determining a plurality of inner products, since each element in the output vector A corresponds to an inner product of the input vector V and one (different) row of the input matrix M.

[0034] On a conventional microprocessor, the computation of an inner product involves a loop that in turn performs a multiplication and accumulation for each term. This can be represented, for example, as:

[0035]

[0036] Here, the "for" loop implements an element-wise multiplication and accumulation to arrive at the final inner product value ("sum").

[0037] For matrix computation, this involves two loops as follows:

[0038]

[0039] Here, the inner "for" loop (on index j) implements the inner product, and as before, the outer "for" loop (on index i) loops this inner product over each row of the matrix M. The result of the nth inner product determines the nth element A[n] of the output vector A.

[0040] This can be accelerated by using a vector processing unit (also simply referred to as a vector unit, VU). Vector units are known per se in the art. For example, a vector unit can determine the inner product of two input vectors by simultaneously performing a plurality of multiply-accumulate (VMA) operations.

[0041] As a specific example, a vector unit can compute 16 multiplications at a time, followed by an accumulation operation that determines and outputs the sum of the 16 individual multiplication operations. Such an operation can take the following form:

[0042]

[0043] This will be 16 times faster at the cost of having 16 multipliers in place (the vector unit). This operation can also be used for matrices, as follows:

[0044]

[0045] Similar to above, the inner "for" loop (on index j) implements the inner product, and the outer "for" loop (on index i) loops this inner product over each row of the matrix M.

[0046] Likewise, the result of the nth inner product determines the nth element A[n] of the output vector A. In other words, when implementing the processing as described above, the output values from the vector unit are used to construct the final output vector. That is, the result of passing the vector V and the nth row of the matrix M to the vector unit is a scalar value (the inner product) that corresponds to the nth element (element zero) of the output vector. In the example above, this includes 16 values that need to be written to the correct respective elements of the final output vector.

[0047] To ensure that the output vector is correctly constructed from the individual output values from the vector unit, there are conventionally two options.

[0048] The first option is to use a small portion of the register file to establish the individual results, i.e. to store the output values from the vector unit in regular scalar registers, and then to construct the final vector in the vector registers by reading the values from the scalar registers into the vector registers. The disadvantage of this is that more memory bandwidth is required, as the values need to be loaded repeatedly.

[0049] The second option is to use explicit indexing in the vector registers as to where to accumulate the results, i.e. to ensure that each output from the vector unit is written to the correct element in the vector registers when output from the vector unit. The disadvantage of this is that it requires an extra operand to specify where in the vector registers to write the output.

[0050] The invention provides a more efficient implementation by providing an instruction and a corresponding micro-architecture structure that enables the output vector to be automatically constructed by repeatedly applying the instruction without the need to repeatedly load values or any extra information to specify the location in the vector registers (i.e. which value corresponds to which element of the output vector). This enables efficient matrix-vector multiplication etc. to be achieved with only operands.

[0051] Typically, the matrix and vector are much larger than can be accommodated in any register bank, so the input vector can be longer than the number of elements that can be stored in the vector unit. In this case, multiple passes need to be made. This will result in more output values. Therefore, a further challenge is to load the correct bits of data in the register bank and to load the rest of the data from memory in turn. The invention provides an improvement on this, as all that is required to handle this complexity is to generate a sequence of instructions one after the other, which only requires adjusting the pointer in memory.

[0052] Figure 1An example computer system 100 is shown that includes a processor 101. The processor 101 includes a pipeline 102 that includes a series of pipeline stages. For example, the pipeline can include a fetch stage 103 that fetches instructions, a decode stage 104 that decodes instructions, and an execution unit 105. The execution unit 105 can include one or more stages, such as a register read stage that reads from a register file 106, a compute stage that performs computations, one or more memory access stages that can address memory depending on the instruction, can read and write data to memory, etc. Note that Figure 1 The specific pipeline stages shown in FIG. 1 are illustrated by way of example, but this is not limiting, and one of skill will know of other possible pipeline variants.

[0053] The processor 101 can include a scheduler (not shown) coupled to the fetch stage 103. The execution unit 105 (e.g., the memory access stages of the execution unit 105) is coupled to a data memory 107. The input of the instruction fetch stage 103 is coupled to a separate instruction memory 108. The processor 101 includes a register file 106 that includes at least one set of registers. Note also that most generally, the register file referred to here can refer to any register group up to the total collection of registers that can be addressed on the processor, and is not limited to any specific physical module or sub- partition in the register address space. In particular, the register file 106 includes one or more vector registers, as described later.

[0054] The data memory 107 is a memory in which data and computation results of a computation operation can be eventually stored. The data memory 107 can be stored on the same physical unit as the processor 101. Alternatively, the data memory 107 can be stored on a separate unit, such as an external memory. In embodiments such as Figure 1 In the embodiment shown, instructions are stored in an instruction memory 108 separate from the data memory 107, and fetched from that instruction memory 108. These can be separate memory devices or separate regions of the same memory device. Regardless, since the instruction memory 108 and the data memory 107 have non-overlapping address spaces, this means that there is no risk of instruction fetches performed by the fetch stage 103 competing with data accesses (loads or stores) performed by the memory access stages. The data memory can include, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of these memories.

[0055] The execution unit 105 is capable of performing a limited set of operations in response to instructions from a predetermined set called the instruction set. A typical instruction set can include, for example, instructions such as LOAD, ADD, STORE, etc. that the execution unit 105 is configured to understand and implement in response to the respective instructions. Thus, the execution unit 105 typically includes one or more arithmetic computation units for performing such instructions, such as fixed point arithmetic units (AUs), logical units (LUs), arithmetic logical units (ALUs), and floating point units (FPUs). Arithmetic refers to mathematical operations on numbers: e.g., multiplication, addition, division, subtraction, etc. The computations can constitute a more extensive set: e.g., if operand 0 is true, then operand 1 is stored in the target, otherwise operand 2 is stored in the target. As Figure 1 As shown, the execution unit 105 also includes a vector unit 201, which will be discussed in more detail below.

[0056] According to examples described herein, the execution unit 105 is configured to implement a novel type of instruction, referred to herein by way of example as VLMACCR (Vector-Load-Multiply-Accumulate-Reduce).

[0057] Figure 2 An example implementation of a VLMACCR operation using the processing unit 200 is shown schematically, the processing unit 200 being configured to implement the VLMACCR operation in response to the respective instruction. The VLMACCR operation itself is described below with reference to Figure 3a and Figure 3b An example implementation of a VLMACCR operation using the processing unit 200 is shown schematically, the processing unit 200 being configured to implement the VLMACCR operation in response to the respective instruction. The VLMACCR operation itself is described below with reference to

[0058] The processing unit 200 includes a vector unit 201 and an output register rA. Figure 2 The input register rV and the memory 107 described previously are also shown in FIG. 2. The input register rV can or can not form part of the processing unit 201. Similarly, the memory 107 can or can not form part of the processing unit 201.

[0059] The output register rA and the input register rV are vector registers. A vector register is a register that includes a plurality of elements for holding different components of a vector. The output register rA and the input register rV can be implemented as part of the register file 106 described previously (e.g., as specified by an operand of the VLMACCR instruction). Alternatively, one or more of the output register rA and the input register rV can be implicit registers that are automatically used by the vector unit 201 (without needing to be specified by an operand). Thus, it will be appreciated that although the registers rV, rA are shown as being specified, the registers themselves can not be fixed. That is, there can be more registers available to the processing unit 200, and one or more of rV and rA can be selected from a set of registers (e.g., in the register file 106). The vector to be used can be specified by an operand of the VLMACCR instruction (e.g., by specifying a register that stores the vector).

[0060] Note that the output register rA need not be the same length as the input register rV. The two end elements of the output register rA are referred to herein as the first end element and the second end element. It will be appreciated that these are relative terms. That is, if the first end element is the left-most element (as shown), then the second end element is the right-most element, and vice versa. Similarly, if the first end element is the top-most element, then the second end element is the bottom-most element. Figure 2

[0061] The registers rV and rA can store elements of different types and / or widths. In one example, rV can store 8-bit integers, while rA can store 32-bit integers. Other formats are possible, e.g., 32-bit floating point numbers or 16-bit floating point numbers, or the types can be mixed.

[0062] The vector unit 201 can be implemented as part of the execution unit 105 described above. Alternatively, the vector unit 201 can be implemented as a unit separate from the execution unit 105 itself. In either case, the vector unit 201 is operatively coupled to the memory 107.

[0063] The vector unit 201 is configured to perform an operation on at least two input vectors to generate a result value. In the example shown, the vector unit 201 is operatively coupled to an input register rV holding a first input vector V and to the memory 107 holding a second input vector M. In other examples, both V and M can be stored in respective registers for use by the vector unit 201. Figure 2

[0064] In the example shown, the vector unit 201 is operatively coupled to an input register rV holding a first input vector V and to the memory 107 holding a second input vector M. In other examples, both V and M can be stored in respective registers for use by the vector unit 201. Figure 2 ​​In the example shown, the operation performed by the vector unit 201 is an inner product, so the resulting value generated by the vector unit 201 from the input vectors V and M is a scalar value equal to the inner product of V and M. In one specified implementation of the inner product of 1-bit numbers, a "0" can be used to represent -1, while a "1" can be used to represent +1 (or vice versa). In this case, the product is always +1 or -1 (i.e., a non-XOR operation), and the sum of the values is simply the sum of these -1 and +1 bits, which is the population count. This is very low resolution, but an example of a highly parallel inner product. Operations other than inner products can be performed by the vector unit. For example, the vector unit can simply add two vectors together in point-wise addition (for k in range(len(V0)): V2[k] = V0[k] + V1[k]). Multiplication, subtraction, or absolute value are other examples.

[0065] Figure 3a A first example VLMACCR operation is shown schematically. In this example, the output register rA includes eight elements. The initial state 301a and the final state 301b of the output register rA are shown. In the initial state, the eight elements of the output register rA hold values A0 through A7 in order. These values can be initialized to zero, as discussed in more detail below.

[0066] The vector unit 201 processes the first input vector V and the second input vector M to generate a result value s. As discussed above, this can include the vector unit 201 determining an inner product of V and M.

[0067] The values in the elements of the output register rA are updated using the initial state values A0 through A7, as well as the result value s, as shown. That is: Figure 3a

[0068] • Each value in the output register rA is left-shifted by one bit (i.e., the value from element n is moved to element n-1).

[0069] • The value A0 in the leftmost element is added to the result value s, and the total is placed in the rightmost (nth) element of the output register rA.

[0070] Figure 3b A second example VLMACCR operation is shown schematically. This example is similar to the Figure 3a except that the values in the output register rA are shifted right. Again, the initial state 302a and the final state 302b of the output register rA are shown. In the initial state, the eight elements of the output register rA hold values A0 through A7, as before.

[0071] Again, the values in the elements of the output register rA are updated using the initial state values A0 through A7, as well as the result value s, as shown. However, in this example: Figure 3a the values are right-shifted by one bit. That is:​

[0072] • Each value in the output register rA is shifted one bit to the right (i.e. the value from element n is moved to element n+1).

[0073] • The value A in the rightmost element n-1 is added to the result value s, the total being placed in the leftmost (first) element of the output register rA.

[0074] As can be seen from the two examples above, the result value s from the vector unit 201 is added to the components of the output vector that are moved between the first and second end elements (depending on the direction of rotation). The other values (not moved from one end of the output register rA to the other) are simply shifted one element to the left (or right) without anything being added.

[0075] This step is therefore a rotation operation (also known as a circular shift) on multiple elements of the output register, in which a result value is added to the components of the output vector that are moved between the first and second end elements. In other examples, the operation performed on the result value and the components of the output vector that are moved between the first and second end elements is another mathematical or logical operation, such as multiplication, XOR, subtraction, etc.

[0076] When the values of the elements stored into the output register rA are added one at a time as the output register rA is rotated, these elements can be referred to as accumulators. For similar reasons, the output register rA itself can be referred to as an accumulator.

[0077] The rotation operation can be implemented using flops. In such an implementation, the data is simply routed around, including a combining element (e.g. an adder) on the route from the last element to the first element (depending on the way the register is to be rotated). In another implementation, a carry register can be used to temporarily store one value while the other values are shifted.

[0078] For example, the VLMACCR operation described above can be represented using the following pseudo code:

[0079]

[0080] In this example, the vectors V and M both comprise 32 components, and the output register rA comprises eight elements (accumulators) for holding the output values. M is stored in the memory 107, the first element of M being at memory address "address". The first "for" loop implements the multiply-accumulate operation (inner product). The second "for" loop implements the right rotation of the output register rA, with the result of the multiply-accumulate operation being written to the first element of the output register rA.

[0081] As mentioned above, the matrix-vector multiplication can be efficiently implemented using the processing unit 200 and the VLMACCR instruction.

[0082]

[0083] The program code is illustrated in the form of a flowchart. Figure 8

[0084] For example, successive rows of a matrix (e.g. stored in the memory 107) can be read in sequence to be multiplied with the vector V stored in rV. In another embodiment, the rows of the matrix can be stored in a second input register. The VLMACCR operation is performed on each row of the matrix and the output vector A is automatically built in the output register rA component by component.

[0085] Reference will now be made to Figures 4a to 4d An example is described in which the processing unit 201 is used to determine the product of a matrix M and a vector V. In this example, the matrix M comprises 32 columns and 8 rows. The vector unit 201 is capable of processing 16 elements at a time. The input register rV comprises 16 elements. Note that in general, the input register rV should comprise the same number of elements as the vector unit 201 is capable of processing at a time. The output register rA comprises 8 elements for holding the output vector A.

[0086] As Figure 4a shown, the values in the output register rA are first initialised to zero.

[0087] The first 16 components of the vector V are first read from the memory 107 and stored in the input register rV.

[0088] The vector unit 201 is then operated on the first 16 components of the vector V stored in the vector register rV and the first 16 elements of the first row of the matrix M read from the memory 107 to produce the output value sO in response to the first VLMACCR instruction. That is, the processing unit 200 is configured to operate on two vectors at a time, the first being the vector V and the second being the first row (and subsequent) of the matrix M. In this example, the first vector V is stored in the register rV but the second vector is read directly from the memory 107 using a pointer. To operate on subsequent rows of the matrix M, the pointer is simply adjusted. In other examples, the second vector can be stored in a vector register for the processing unit 200 to operate on (similar to the first vector V). In this case, a separate load instruction can be used to first read the second vector (or a portion thereof) from the memory 107 to a second input register.

[0089] ​A rotate operation is then applied as described above. In this case, s0 is added to the value of the first end element (leftmost element) of the output register rA and placed in the second end element (rightmost element). All other values in the output register rA are shifted left by one element.

[0090] The next steps are as shown in Figure 4b In response to the second VLMACCR instruction, the vector unit 201 operates on the first 16 components of the vector V stored in the vector register rV and the first 16 elements of the second row of the matrix M read from the memory 107 to produce an output value s1. Note that V is reused and so does not need to be reloaded into rV.

[0091] A rotate operation is then applied. In this case, s1 is added to the value of the first end element (leftmost element) from the output register rA and placed in the second end element (rightmost element). All other values in the output register rA are shifted left by one element (including the previously written s0).

[0092] The process then continues until the first 16 values in each of the 8 rows of the matrix M have been processed. In other words, the left-hand side of the matrix M is swept out by the vector unit 201 to produce a first set of "partial sums" which are stored as elements in the output register rA. The resulting state of the output vector register rA is as follows: [s0][s1][s2][s3][s4][s5][s6][s7].

[0093] As shown in Figure 4c The second 16 components of the vector V are then loaded from the memory 107 into the input register rV.

[0094] In response to the ninth VLMACCR instruction, the vector unit 201 operates on the second 16 components of the vector V stored in the vector register rV and the second 16 elements of the first row of the matrix M read from the memory 107 to produce an output value s0'.

[0095] A rotate operation is then applied as described above. In this case, s0' is added to the value s0 of the first end element (leftmost element) from the output register rA and the result s0' + s0 is placed in the second end element (rightmost element). All other values in the output register rA are shifted left by one element.

[0096] The next steps are as shown in Figure 4dThe second 16 elements of the second row of matrix M read from memory 107 are operated on by vector unit 201 with the second 16 components of vector V stored in vector register rV to produce output values s1'. Again, note that V is reused and thus need not be reloaded into rV.

[0097] A rotate operation is then applied. In this case, s1' is added to the value s1 from the first end element (the leftmost element) of output register rA and placed in the second end element (the rightmost element). All other values in output register rA are shifted left by one element.

[0098] The process then continues until the second 16 values of each of the 8 rows of matrix M have been processed. In other words, the right side of matrix M is swept out by vector unit 201 to produce a second set of "partial sums" which are added to the first set of partial sums determined previously and stored as elements in output register rA. The resulting state of output vector register rA is as follows: [s0' + s0][s1' + s1][s2' + s2][s3' + s3][s4' + s4][s5' + s5][s6' + s6][s7' + s7]. This is the final answer, as desired (i.e. the vector resulting from V multiplied by M).

[0099] Of course, the above has been described with reference to a matrix M and vector V that are exactly twice the length of input register rV (and vector unit 201 can process twice-length matrices M and vectors V), but for input vectors of any length, the same VLMACCR operation can be used to perform similar operations.

[0100] If the input vector length (and the width of matrix M) is shorter than input register rV, then the entire inner product (or other operation) can be performed at once.

[0101] If the input vector length is an integer multiple N of the length of rV, then the VLMACCR instruction can be used in the manner described above until N full rotations of output vector rA have occurred.

[0102] If the input vector length is not a non-integer multiple of the length of rV, then a combination of the above methods is applied. For example, using a 16 element vector unit 201 to process a 42 component vector operation would include two full rotations of output register rA using all 16 elements, followed by a third full rotation using only 10 elements.

[0103] Figure 5a and Figure 5bAn example is shown in which the matrix comprises 16 columns and 16 rows. The input register rV comprises 16 elements. The vector unit 201 is configured to process 16 elements at a time. The output register rA comprises 8 elements.

[0104] The implementation of the VLMACCR operation itself is similar to that described above and so is not repeated here.

[0105] In this case, the vector unit 201 is able to process the entire inner product (or other operation) between each row of the matrix M and the vector V at a time. Thus, there is no need for the repeated rotation of partial sums as before. However, the matrix contains 16 rows, which means that the final (desired) output vector will have 16 components, which is too long for the output register rA. Thus, the processing unit 200 is configured (e.g. by program code instructions) to output the values of the elements from the output register rA to a storage location, e.g. the memory 107, once the first 8 rows of the matrix M have been processed by the vector unit 201, for example as shown in Figure 5a Note that in the example code given above, it is the outer for loop which performs this operation.

[0106] The processing unit 200 then continues to process the next (last) 8 rows of the matrix M in a similar manner (after resetting the output register to zero), as shown in Figure 5b The processing unit 200 then continues to process the next (last) 8 rows of the matrix M in a similar manner (after resetting the output register to zero), as shown in Figure 5b The final result vector is then constructed by concatenating the values from the second pass to the end of the values from the first pass already stored in the storage location, for example at the storage location as shown in

[0107] In all of the examples described above, the number of rows of the matrix M is equal to an integer multiple of the number of elements in the output register rA. In the case where the matrix M comprises any other number of rows, the process is the same until there are a remaining number of rows to process which is less than the number of elements in the output register rA. These "remaining" rows can still be processed using the VLMACCR instruction as described above, however, the values present in the output register rA will need to be realigned so that the first value (so and Ao in the examples described above) is present in the first element of the output register rA. Thus, the processing unit 200 can be configured to apply an additional rotation operation. This can be done by performing a standard rotation operation, or by performing one or more "dummy" VLMACCR operations on a null vector (containing all zeros) from the memory 107 or from the input register rV.

[0108] In another example implementation, the output from the vector unit 201 is simply written to the output register rA, and the output register rA is rotated (left or right). This is shown in Figure 6

[0109] ​This still has the advantage of automatically constructing the output vector by repeated application of instructions, without the need to repeatedly load values or any additional information to specify the position in the vector register, however, it can only be used for one complete rotation of the output register rA. If the input matrix contains more rows than elements in the output register rA, additional operations are required. These issues will be explained below.

[0110] Figure 7 An example is schematically illustrated in Figure 6 The example method performed by the processing unit 200 is illustrated in the example of Fig. 7. The processing unit 200 is configured to perform the method in response to a single instruction, again referred to herein as a VLMACCR instruction.

[0111] At s701, the components of the vector M and the vector V are multiplied by the vector unit 201. That is, the vector unit 201 multiplies the value in the nth element of rM by the value in the nth element of rV. Each individual multiplication operation results in a scalar value.

[0112] At s702, the vector unit 201 accumulates the scalar values into a single result value A1. That is, the vector unit 201 adds the scalar values resulting from each multiplication operation to produce the result value A1. The vector unit 201 provides the result value A1 to the register rA.

[0113] The steps s701 and s702 together correspond to the vector unit 201 determining the inner product of the two input vectors. Thus, in the present example, the result value A1 is a scalar value. As mentioned above, operations other than inner products can be performed by the vector unit.

[0114] At s703, the output register rA performs a rotation operation (also referred to as a circular shift). Figure 6 The dashed arrows in Fig. 7 schematically illustrate the example. The rotation operation can be performed in parallel (simultaneously) with the multiplication and accumulation operations. Typically, the operation can be pipelined, e.g. one stage for memory, one stage for multiplication, and one stage for implementing an addition tree of accumulators.

[0115] The rotation operation itself is known. In Figure 6 In the example illustrated in Fig. 7, the output register rA performs a "left rotation", in which all values are shifted to the left, with the value from the first end element being written to the second end element. That is, the value present in the first element is written to the last element of the output register rA, while the values present in all other elements n are written to the n-1th element of rA. In other examples, the output register rA can perform a "right rotation", in which the values are shifted in the opposite direction (i.e. the value present in the second end element is written to the first end element, while the values present in all other elements n are written to the n+1th element). This will be described in more detail hereinafter.

[0116] At s704, the result value A1 is written to the first element of the output register rA. That is, the output register rA stores the result value received from the vector unit 201 to the first element of the output register rA. In this example, the first element of the output register rA is the left-most element, as shown. Figure 6

[0117] In the above example, the rotate operation is performed before the result value A1 is written to the first element of the output register rA. In other examples, the output register rA can be rotated (in either direction) after the output value is written to the first element.

[0118] Thus, there are four possible combinations of the order of operations (rotate first or write first) and the direction of rotation (left or right). This will now be explained with reference to a simple example in which a 4x4 matrix M is multiplied by a 4-component vector V, reusing the VLMACCR instruction. In this example, the VLMACCR operation is applied four times to produce four result values A1-A4. In this example, the size of the output register rA is also four elements.

[0119] Consider a first example in which the VLMACCR operation comprises first rotating the output register rA to the right and then writing the result from the vector unit 201 to the first element of the output register rA. After the four VLMACCR operations, the output register rA contains the final vector A: each element component A1-A4 of the vector A is present in a different element of the output register rA. Importantly, the order of the components A1-A4 is correct. The vector A can then be output, for example, to the memory 107 or some other storage location.

[0120] Consider a second example in which the VLMACCR operation comprises first writing the result from the vector unit 201 to the first element of the output register rA and then rotating the output register rA to the left. Similarly to the previous example, after the four VLMACCR operations, the output register rA contains the final vector A: each element component A1-A4 of the vector A is present in a different element of the output register rA. Importantly, the order of the components A1-A4 is correct. The vector A can then be output, for example, to the memory 107 or some other storage location. Note that in this example, the elements A1-A4 of the vector A end in the opposite order to the first example.

[0121] ​Consider a third example in which the VLMACCR operation includes first left-rotating the output register rA and then writing the result from the vector unit 201 into the first element of the output register rA. After four VLMACCR operations, the output register rA contains the components A1-A4 of the vector A, as before. However, unlike the previous example, the components A1-A4 are in the wrong order. An additional correction operation can be implemented to account for this. In this example, the correction operation includes a left rotation. Three right rotations can also be used.

[0122] Consider a fourth example in which the VLMACCR operation includes first writing the result from the vector unit 201 into the first element of the output register rA and then right-rotating the output register rA. Similar to the third example, after four VLMACCR operations, the output register rA contains the components A1-A4 of the vector A, but in the wrong order. Again, an additional correction operation can be implemented to account for this. In this example, the correction operation includes a left rotation. Three right rotations can also be used.

[0123] In all of the examples given above, it is assumed that the length of the output register rA is the same as the length of the vector A to be computed. However, the output register rA will have a fixed size (e.g., 32 elements), while the vector A to be determined can have fewer or more components than this size (e.g., fewer or more than 32 components). The case in which the vector A contains more components than the elements in the output register rA was discussed above.

[0124] In the case in which the vector A includes fewer components than the elements in the output register rA, all of the elements of the vector A will be present somewhere in the output register rA when the VLMACCR instruction is completed. Additional operations can or can not be needed to "realign" these values, similar to that discussed above.

[0125] The nature of these additional operations depends on the specific implementation of the VLMACCR instruction. This can be explained by considering the four examples discussed above, but in which the first three VLMACCR instructions are the only ones executed. This corresponds to the example in which the output register rA is four elements long and is used as part of the operation to compute a vector A that has only three components.

[0126] In the first example, the output register rA is in the following state after three VLMACCR instructions: [A3] [A2] [A1] [0]. An additional right rotation operation can be used to realign the elements of the resulting vector A.

[0127] In the second example, the output register rA is in the following state after three VLMACCR instructions: [0] [A1] [A2] [A3]. An additional left rotation operation can be used to realign the elements of the resulting vector A.

[0128] In a third example, the state of the output register rA after the three VLMACCR instructions is as follows: [A3] [0] [A1] [A2]. Two additional left rotation operations can be used to realign the elements of the result vector A.

[0129] In a fourth example, the state of the output register rA after the three VLMACCR instructions is as follows: [0] [A3] [A2] [A1]. In this case, no additional operations are needed.

[0130] Figure 8 is a flowchart illustrating a method according to examples described herein, in which the processing unit 200 is configured to use a counter to keep track of how many VLMACCR instructions are executed in the current loop. The counter can be reset at the beginning of the method.

[0131] Figure 8 The method shown in is representative of all the code in the code shown below (also given and described above) except for the outermost for loop:

[0132]

[0133] That is, the flowchart represents an example method of implementing the inner product of a single row of the matrix M with the vector V from rV.

[0134] In this example, the output vector A comprises integer multiples of 8 components, the input vector V comprises integer multiples of 16 components, and we assume that the vector unit 201 can perform 16 multiply-accumulates on an 8-element rotation accumulator rA simultaneously.

[0135] At S800, the processing unit 200 resets the counter i to 0, indicating that we are computing the first part of the output vector A.

[0136] At S801, the processing unit 200 resets the output register rA by resetting all accumulator values to zero (i.e., so that the value present in each element of the output register rA is zero).

[0137] At S802, the processing unit 200 resets the counter j to 0, indicating that we are processing the first 16 columns of the matrix M (i.e., the first 16 elements of the first row of M).

[0138] At S803, the processing unit 200 loads the first 16 values of the input vector V to be multiplied with the first 16 columns of the matrix M. In subsequent iterations, j will be changed to point to a later part of the vector, and at S803, the processing unit 200 loads the second 16, third 16, etc. of the vector to be multiplied with the second 16, third 16, etc. of the matrix.

[0139] At S804, the processing unit 200 implements eight VLMACCR instructions using the rows i, i+1, i+2,..., i+7 in the matrix.

[0140] At S805, the processing unit 200 increments the counter j by 16 in order to advance S803 and S804 to operate on the next portion of the vector V and the matrix M.

[0141] At S806, the processing unit 200 determines whether the counter j is equal to the length of the input vector V (i.e., whether the end of the portion of the matrix M has been reached). If so, there are no remaining VLMACCR instructions, and the method proceeds to S807. Otherwise, steps S803, S804, and S805 are processed again in order to iterate over the next 16 columns of the matrix M.

[0142] At S807, the processing unit 200 outputs the values present in the output register rA to another storage location, such as the memory 107. This is because at S806 it was determined that the output register rA has completed a full set of rotations, which means that all elements of the output register rA now hold values from the vector unit 201 that were computed by the current iteration.

[0143] At S808, the processing unit 200 increments the counter i by 8 in order to advance the row address used in S804 to operate on the next portion of the matrix M.

[0144] At S809, the processing unit 200 determines whether the value of the counter i is equal to the number of elements in the output vector V (“n”). That is, the processing unit 200 determines whether the output register rA has stored every portion of the output vector V since the method began at S800. If so, the method ends. If not, the method proceeds to S801.

[0145] The processor can be a pipelined processor. In a pipelined processor, the execution units are divided into a series of pipeline stages, each of which is used to perform a specified type of operation. The pipeline typically includes an instruction fetch stage, a decode stage, a register read stage, at least one compute stage, and one or more memory access stages. The instruction fetch stage fetches a first instruction from memory and sends it to the first stage of the pipeline in the next processor cycle. In the next processor cycle, the decode instruction passes down to the next stage in the pipeline, e.g., the register read stage. Meanwhile, the fetch stage fetches a second instruction from instruction memory into the decode stage. In the next successive processor cycle, the first instruction is passed to a third pipeline stage, e.g., the compute stage, while the second instruction is passed to the second pipeline stage, and a third instruction is issued to the first pipeline stage, and so on. This helps to keep the processor busy, reducing latency, because otherwise the processor would need to wait for an entire instruction to execute before issuing the next instruction to the execution unit.

[0146] The processor can be a multi-threaded processor. In a multi-threaded processor, the processor includes multiple sets of context registers, each set of context registers representing the context (i.e., program state) of a respective one of a plurality of currently executing program threads. The program state includes the program counter of the respective thread, the operands of the respective thread, and optionally the respective state information, such as whether the thread or context is currently active. The processor also includes a scheduler configured to control the instruction fetch stage to time-interleave the instructions through the pipeline, e.g., in a round-robin fashion. Threads interleaved in this manner are said to be concurrently executing. Where the execution units are pipelined, as a thread's instructions progress through the pipeline from one pipeline stage to the next, the instructions of another thread progress down the pipeline one stage behind, and so on. This interleaving approach is beneficial because it provides more opportunities to hide pipeline latency. Without interleaving, the pipeline would need a mechanism to resolve dependencies between instructions in the pipeline (a second instruction can use a result of a first instruction, which can not be ready in time), which can create pipeline bubbles during which the second and other instructions are hung up until the first instruction completes execution.

[0147] Reference is made herein to data storage for storing data. This can be provided by a single device or by multiple devices. Suitable devices include, for example, hard disks and non-volatile semiconductor memory, including, for example, solid state drives or SSDs.

[0148] The examples described herein are to be understood as illustrative examples of embodiments of the application. Further embodiments and examples are envisaged. Any feature described in relation to any one example or embodiment can be used alone or in combination with other features. Furthermore, any feature described in relation to any one example or embodiment can also be used in combination with one or more features of any other example or embodiment, or any combination of any other examples or embodiments. Furthermore, equivalents and modifications not described herein are also to be encompassed within the scope of the application, as defined in the claims.

Claims

1. A processing unit for generating an output vector, the processing unit comprising: an output register having a plurality of elements for holding different components of the output vector, the plurality of elements including a first end element and a second end element; and a vector unit for processing at least two input vectors to generate a corresponding result value; wherein the processing unit is configured to execute machine code instructions, each instruction being an instance of a predefined type of instruction set of the instruction set of the processing unit, wherein the instruction set includes a vector processing instruction defined by a corresponding opcode, the vector processing instruction causing the processing unit to: i) process a first input vector and a second input vector using the vector unit to generate the result value; ii) perform a rotation operation on the plurality of elements of the output register, wherein the result value is placed in a first end element of the output register with a sum of values that existed in the second end element prior to the rotation. the vector unit is configured to perform the processing of the first and second input vectors i) by performing a multiply-accumulate operation on the input vectors, the result value being a scalar value resulting from the multiply-accumulate operation.

2. The processing unit of claim 1, wherein, the vector unit is configured to access at least the second input vector directly from memory.

3. The processing unit of any one of claims 1-2, wherein, an input register for holding at least the first input vector, wherein the vector unit is configured to access the first input vector from the input register.

4. The processing unit according to any one of claims 1 to 3, comprising: the input register is selected from a plurality of registers in a register file, and wherein the vector processing instruction includes an operand specifying the input register.

5. The processing unit of claim 4, wherein, the input register is implicitly present in the opcode of the vector processing instruction rather than being specified by an operand of the vector processing instruction.

6. The processing unit of claim 4, wherein, the output register is selected from a plurality of registers in a register file, and wherein the vector processing instruction includes an operand specifying the output register.

7. The processing unit according to any one of claims 1 to 6, wherein, the output register is implicitly present in the opcode of the vector processing instruction rather than being specified by an operand of the vector processing instruction.

8. The processing unit according to any one of claims 1 to 6, wherein, the processing unit is configured to execute machine code instructions, each instruction being an instance of a predefined type of instruction set of the instruction set of the processing unit, wherein the instruction set includes a vector processing instruction defined by a corresponding opcode, the method comprising the vector processing instruction causing the processing unit to implement the following vector processing operations:

9. A method of generating an output vector at an output register, the method performed by a processing unit, the output register having a plurality of elements for holding different components of the output vector, the plurality of elements including a first end element and a second end element, wherein, i) process a first input vector and a second input vector using a vector unit to generate a result value; ii) perform a rotation operation on a plurality of elements of the output register, wherein the result value is placed in a first end element of the output register with a sum of values that existed in the second end element prior to the rotation. the first input vector is stored in a register, and wherein the method comprises performing a plurality of the vector processing operations using the same first input vector.

10. The method of claim 9, wherein, after completing a plurality of the vector processing operations, outputting the value present in the output register to a storage location.

11. The method of claim 10, comprising: ​ 12. The method of claim 10 or claim 11, comprising: after completing a number of vector processing operations that is not equal to an integer multiple of the number of elements in the output register, performing one or more of the rotate operations on the output register without changing the values present in the elements of the output register such that the total number of rotations is equal to an integer multiple of the number of elements in the output register.

13. A computer system comprising: The processing unit according to any of claims 1 to 8, programmed to perform the method according to any of claims 9 to 12.

Images