Instruction set architecture for matrix operation

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for re-interpreting vector instructions into matrix instructions. One of the methods is executed by a processor implementing an instruction set architecture having instructions for setting a configuration register of the processor that controls whether to interpret a vector instruction as a vector instruction or a matrix instruction. Instructions are executed to set a configuration register. Then, when one or more vector instructions are received, the one or more vector instructions are re-interpreted as matrix instructions based on the information set in the configuration register.

Description

Background Art

[0001] This specification relates to computer processors and instruction set architectures.

[0002] An instruction set architecture (ISA) is a behavioral model for a particular family of processors that is independent of the specific hardware implementation or microarchitecture details of any of the processors in the family. An ISA typically defines the types of instructions that can be executed, what fields the instructions have, the names of configuration and data registers, data types, and other features of a family of processors. An ISA provides an abstraction that allows processors with different physical characteristics and capabilities to execute the same software. Therefore, the hardware that implements the ISA can be upgraded to a newer or more powerful version without changing the software.

[0003] Some ISAs define processor support for vector operations. Vector operations operate on vectors of arbitrary length and free software developers or compilers from explicitly expressing iterations of elements of vectors. Instead, processors implementing the ISA will automatically iterate vectors based on vector sizes that can be specified at runtime rather than hard-coded. Processors implementing such vector instructions typically utilize specialized vector processing hardware components with multiple cores for parallelizing vector operations.

[0004] The ISA that defines vector operations may define a special vector register set for supporting vector operations. Then, vector instructions may reference vector registers as operands. The implementation of vector operations will implement vector instructions without software specifying explicit iteration instructions. In order to use such vector operations, software may specify various configuration information about vectors and their elements, such as the number of elements in the vector, and the size and type of each element in the vector.

[0005] However, while vector operations provide great flexibility for single-dimensional data sets, such arbitrary-length vector operations are often inefficient when processing multi-dimensional data sets such as matrices. One problem is that since matrices have two-dimensional indices, it is highly possible that the processor may run out of vector register resources when trying to iterate over a two-dimensional matrix of arbitrary size. When this happens, other mitigation measures that slow down computational performance must be invoked, such as the slow process of writing data out to memory to free up resources in vector registers.

[0006] This phenomenon is a substantial bottleneck for machine learning operations, which often require very intensive matrix computations. Summary of the invention

[0007] This specification describes an instruction set architecture (ISA) having instructions that are particularly useful for and improve the performance of matrix operations and related machine learning applications. To this end, the ISA defines a new configuration register (CR) for matrix operations and a companion instruction set for setting the value of the CR.

[0008] Setting the value of CR for matrix operations effectively overrides the meaning of the vector multiplication instruction, so that the instruction causes the processor to perform matrix multiplication arithmetic. In doing so, the processor implementing the ISA will reinterpret the vector register operand as a vector of small matrices rather than a vector of single elements. For example, instead of performing a processor operation on a 256-element vector of scalar values, the processor may reinterpret the data as a quarter-length vector of a 2x2 matrix.

[0009] This arrangement provides significantly higher computational intensity without fundamentally changing existing vector instructions.

[0010] Specific embodiments of the subject matter described in this specification may be implemented to achieve one or more of the following advantages. The instruction set architecture described in this specification improves the performance of processors that perform matrix operations, which makes such processors more efficient and faster when executing machine learning applications that rely on such matrix applications. The matrix extensions are also fully backward compatible, so that older software written for vector-only operations will still execute on newer processors that implement the matrix extensions. According to an embodiment, a processor is provided that is configured to implement an instruction set architecture having an instruction that, in operation, sets a configuration register of the processor with one or more values ​​that cause the processor to reinterpret one or more vector instructions as matrix instructions.

[0011] Matrix expansion itself is scalable and does not require the processor implementation to use a specific matrix size. Furthermore, in a heterogeneous processing environment with performance and efficiency cores, it is conceivable that the cores can support different matrix sizes as long as the OS takes care not to migrate threads from a core with higher performance to a core with lower performance during matrix processing.

[0012] The processor may be configured to perform vector arithmetic on the matrix sequence to reinterpret the vector instructions as matrix instructions.

[0013] Reinterpreting the vector instructions as matrix instructions may include reinterpreting data in the vector registers as a matrix sequence.

[0014] Reinterpreting the data in the vector register as a sequence of matrices may include reinterpreting the data in the vector register as a sequence of 2x2, 4x4, 8x8, or 16x16 matrices.

[0015] The configuration register may have a field representing the matrix width.

[0016] The field representing the matrix width may represent the exponent N for a matrix having a width given by 2^N.

[0017] The configuration register may have a field indicating the order of the matrix data.

[0018] The configuration register may have a field indicating the widening mode.

[0019] The configuration register may have a field representing a horizontal accumulation stride, wherein the processor is configured to interpret the value of the horizontal accumulation stride as a directive to use a pre-addition instruction during a multiply-accumulate operation.

[0020] The instruction set architecture may specify an enable bit in a second different configuration register that specifies whether the processor is to interpret one or more vector instructions as referencing vector inputs or matrix inputs.

[0021] According to a further embodiment, a method is provided for execution by a processor implementing an instruction set architecture, the instruction set architecture having instructions for setting a configuration register of the processor, the configuration register controlling whether vector instructions are reinterpreted as matrix instructions, the method comprising: executing instructions to set the configuration register; receiving one or more vector instructions; and reinterpreting the one or more vector instructions as matrix instructions based on information set in the configuration register.

[0022] One or more computer storage media are also provided, encoded with instructions of an instruction set architecture having instructions for setting a configuration register to control whether a processor implementing the instruction set architecture will reinterpret vector instructions as matrix instructions, wherein the instructions are executed by a processor implementing the instruction set architecture to cause the processor to perform operations, the operations comprising: executing the instructions to set the configuration register; receiving one or more vector instructions; and as a result, reinterpreting the one or more vector instructions as matrix instructions.

[0023] The following optional features may be applied to the above-described method or computer storage medium.

[0024] Reinterpreting vector instructions as matrix instructions may include performing vector arithmetic on a sequence of matrices.

[0025] Reinterpreting the vector instructions as matrix instructions may include reinterpreting data in the vector registers as a matrix sequence.

[0026] Reinterpreting the data in the vector register as a sequence of matrices may include reinterpreting the data in the vector register as a sequence of 2x2, 4x4, 8x8, or 16x16 matrices.

[0027] The execution instruction can set the field representing the matrix width in the configuration register.

[0028] The field representing the matrix width may represent the exponent N for a matrix having a width given by 2^N.

[0029] The execution instruction may set a field in the configuration register that indicates the order of the matrix data.

[0030] The execution instruction may set a field in the configuration register indicating the widening mode.

[0031] Executing the instruction may set a field in a configuration register indicating a horizontal accumulate stride, and further include interpreting the value of the horizontal accumulate stride as a directive to use a pre-add instruction during a multiply-accumulate operation.

[0032] The instruction set architecture may specify an enable bit in a second different configuration register that specifies whether the processor is to interpret one or more vector instructions as referencing vector inputs or matrix inputs.

[0033] The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 An example processor implementing an example instruction set architecture (ISA) is shown.

[0035] Figure 2A An example interpretation of a matrix multiplication instruction is shown.

[0036] Figure 2B Shows Figure 2A Example operation of the matrix multiplication instruction.

[0037] Figure 2C Shows Figure 2A Example results for the matrix instructions.

[0038] Figure 3 is a flow chart illustrating an example process 300 for reinterpreting vector instructions as matrix instructions. DETAILED DESCRIPTION

[0039] Figure 1 An example processor 102 is shown for implementing an example instruction set architecture (ISA). Processor 102 includes an instruction decode module 110, a standard processing subsystem 130, a configuration subsystem 120, a vector processing subsystem 140, and a matrix multiplier 150. These are example components that may be used to implement the ISA described in this specification.

[0040] The processor 102 is configured to implement the ISA described in this specification. The ISA may include multiple instructions. Each instruction may cause the processor to perform one or more operations. The ISA may have one or more matrix instructions that cause the processor 102 to perform matrix operations. The ISA may include instructions for setting the configuration register 125 of the processor 102 with one or more values, and the one or more values ​​cause the processor to reinterpret one or more vector instructions as matrix instructions. The difference between a matrix instruction and a vector instruction is that the operand of the matrix instruction is a two-dimensional data set, while the operand of the vector instruction is a one-dimensional data set.

[0041] Instruction decode module 110 has logic circuitry that can decode each of the instructions in the ISA and cause subsystems of processor 102 to perform operations necessary to implement the instructions.

[0042] The ISA may have one or more vector instructions that cause the processor 102 to perform vector operations or matrix operations. The ISA also has instructions for setting configuration registers to control such vector operations or matrix operations. The instruction decoding module 110 may route the configuration register instructions to the configuration subsystem 120, and may route the vector instructions to the vector processing subsystem 140. The vector processing subsystem 140 may include one or more vector registers 145 and other appropriate hardware for implementing vector instructions. Each vector register may store data for vector processing.

[0043] Vector instructions are instructions that cause the processor 102 to perform one or more vector operations. For example, a vadd instruction, when executed by the vector processing subsystem 140, can fill a vector register with the element-by-element addition of two other vector registers. In some implementations, the processor can use parallel processing hardware to execute vector instructions. For example, the vector processing subsystem 140 can have an array of processing elements that can perform the operations of vector addition instructions in parallel. Therefore, vector instructions can cause the processor 102 to operate on multiple pairs of data specified by the operands of the instruction. The vector register 145 can, for example, store a one-dimensional array of integers, logical values, characters, or floating-point numbers, just to name a few examples. Vector instructions can operate on vectors of arbitrary length.

[0044] Vector instructions may include instructions for performing vector operations. In some implementations, vector instructions may reference vector registers 145 as operands. In order to use such vector operations, configuration registers 125 store data that specifies various configuration information about the vector and its elements, such as the number of elements in the vector and the size and type of each element in the vector.

[0045] For example, the ISA may include instructions to set a vector register with data describing an M-length vector of 1s, instructions to set a vector register with data describing an M-length vector of numbers 1 to M, and instructions to multiply two vectors together. The vector processing subsystem may set an operand in a vector register to represent a vector of 1s, and set an operand in another vector register to represent a vector of numbers 1 to M. The vector processing subsystem 140 may then multiply the two vectors together.

[0046] The ISA may also have instructions to set the configuration registers 125 of the processor 102 to reinterpret one or more vector instructions as matrix instructions. Matrix instructions are instructions that cause the processor to perform operations on two-dimensional data sets of arbitrary size. The instruction decoding module 110 sends the instructions to the configuration subsystem 120. The configuration subsystem 120 includes one or more configuration registers 125. For one or more of the configuration registers 125, the ISA may define a configuration register (CR) for matrix operations and a companion instruction set for setting the value of the CR.

[0047] Setting the value of CR 125 for matrix operations effectively overrides the meaning of the vector multiplication instruction, so that the instruction causes the processor to perform matrix multiplication arithmetic. In doing so, the processor implementing the ISA will reinterpret the vector register operand as a vector of small matrices rather than a vector of single elements. For example, instead of performing processor operations on a vector of scalar values, the processor may reinterpret the data as a quarter-length vector of a 2x2 matrix.

[0048] An example of a configuration register for matrix operations will now be described. The example configuration register has the name vtypex, which has the following fields and abbreviations: selected matrix width (vsmw), matrix data order (vmdo), widening mode (vnwmode), and horizontal accumulation span (vhspan).

[0049] The selected matrix width field represents the width of the matrix to be referenced by the vector instruction. In some implementations, the selected matrix width is specified as an exponent in the expression 2^N. In other words, a value of 0 represents a width of 1, a value of 4 represents a width of 16, and so on. For example, if the vector register 145 of the processor 102 holds 16 values, the selected matrix width 0 will be interpreted as 16 scalar values, the selected matrix width 1 will be interpreted as a vector register that holds four 2x2 matrices, and the selected matrix width 2 will be interpreted as a vector register that holds a 4x4 matrix.

[0050] The matrix data order field specifies whether the values ​​in the vector register are arranged in row-major or column-major order. This capability effectively provides free transposition when performing matrix multiplication. In some implementations, the matrix data order field can be set to specify z-ordering or Morton ordering, which effectively interleaves the x and y coordinates.

[0051] The widening mode field specifies the bit width of the computed output. In a typical multiplication operation of two 8-bit numbers, the result can be up to a double-widened 16-bit number. However, for machine learning applications that rely on accumulation, 16 bits are usually not enough. Therefore, setting the widening mode field allows the processor to allocate more bits to the output result than would normally be the case. Thus, the result of multiplying two 8-bit numbers can be stored in a quadruple-widened 32-bit output register. Conversely, the widening mode field can also be used to narrow the output, such as when the result needs to be shifted and truncated.

[0052] The horizontal accumulation stride field affects the operation of the matrix multiplication operation. In fact, this field provides a second addition step after the multiplication but before the accumulation. This function improves a disadvantage of the output quadruple widening, that is, you must write the output to an output register that is twice as wide as the input, which can be complicated to implement in hardware. Instead, after the multiplication, this field specifies the horizontal reduction summation of the group of matrices (e.g., a group of 2 matrices, a group of 4 matrices, or a group of 8 matrices), which reduces the number of outputs that need to be written.

[0053] The ISA may also specify an enable bit (veml) that controls whether the vector instruction is executed in vector mode or matrix mode. In some implementations, the enable bit is the value of the control vector operation in a second different configuration register 125. Placing the enable bit in this second register allows full backward compatibility with previous programs that did not take into account matrix extensions.

[0054] To set the values ​​of the matrix configuration registers, the ISA may define a new instruction for doing so, for example, an instruction named vsetvxi. The new instruction may have fields that specify the values ​​to be written to the matrix configuration registers, and software may change these values ​​as needed at runtime.

[0055] When encountering a vector operation with the enable bit set, the processor 102 will therefore treat the input operands as representing matrix arrays rather than scalar vectors.

[0056] If the enable bit indicates that the vector instruction is being executed in matrix mode, the instruction decode module 110 sends the instruction to the matrix multiplier 150. The matrix multiplier 150 includes appropriate hardware to perform matrix arithmetic on vector register operands using the data in the vector registers 145, for example, processing the data in the registers into a sequence of matrices and multiplying the matrices. If the enable bit indicates that the vector instruction is being executed in vector mode, the instruction decode module 110 instead sends the instruction to be executed by the vector processing subsystem 140.

[0057] The ISA may also have one or more standard (e.g., non-vector and non-matrix) instructions, such as load, store, add, and branch. The instruction decoding module 110 may route the standard instructions to the standard processing subsystem 130. The standard processing subsystem 130 includes appropriate hardware for implementing the standard instructions. For example, the standard processing subsystem 130 may execute a load instruction by issuing a command to a memory for data at a specific address specified by the load instruction.

[0058] Figure 2A An example interpretation of a matrix multiplication instruction is shown. The matrix multiplication instruction may be implemented on any suitable processor (e.g., Figure 1 is implemented on a processor 102).

[0059] In this example, the processor has two vector registers, each with sixteen elements. The first vector register 210 includes elements V0, V1, V2, ..., V15, and the second vector register 220 includes elements V16, V17, V18, ..., V31. For example, the elements can store data representing integers or floating point numbers.

[0060] With the appropriate configuration register set, the processor can be configured to interpret instructions in matrix mode rather than vector mode. The processor can be configured to interpret vector register operands as vectors of matrices of a specified size. The processor can reinterpret vector register operands as vectors of matrices of a specified size rather than vectors of single scalar elements. In this example, the processor can reinterpret the data as a vector of 2x2 matrices rather than a vector of length 16 of a single element.

[0061] The matrix width can be specified by a mathematical expression. In some implementations, the matrix width is specified as an exponent in the expression 2^N. More specifically, a value of 0 represents a width of 1, a value of 4 represents a width of 16, and so on. In this example, the vector register holds 16 values. Therefore, the selected matrix width of 1 will be interpreted as each vector register holding four 2x2 matrices.

[0062] In this example, the first four elements of the first vector register 210 are interpreted as a 2x2 matrix 212. Each position in the matrix can be represented as (r, c), where r ranges from 0 to the total row -1, and c ranges from 0 to the total column -1. In this example, r ranges from 0 to 1, and c also ranges from 0 to 1. The processor interprets the matrix 212 as having an element V1 in the (0,0) position, an element V2 in the (0,1) position, an element V3 in the (1,0) position, and an element V4 in the (1,1) position.

[0063] The processor may similarly interpret the remaining elements of the first vector register 210 as another three 2x2 matrices 214 (for elements V4 to V7), 216 (for elements V8 to V11), and 218 (for elements V12 to V15). The processor may also interpret the elements of the second vector register 220 as four 2x2 matrices 222 (for elements V16 to V19), 224 (for elements V20 to V23), 226 (for elements V24 to V27), and 228 (for elements V28 to V31) in the same manner.

[0064] In this example, the processor receives a matrix instruction. The matrix instruction reads 'vmul VR3, VR2, VR1'. This instruction can be decoded to instruct the processor that the vector registers should be interpreted as storing a matrix having properties defined by the configuration registers, multiplying the elements of the first vector register 210 (i.e. VR1) by the elements of the second vector register 220 (i.e. VR2), and storing the result in the third vector register 230 (i.e. VR3).

[0065] Figure 2B Shows Figure 2A Example operation of a matrix multiplication instruction. The matrix multiplication instruction may be performed on a processor (e.g., Figure 1 is implemented on a processor 102).

[0066] Because in this example, the processor is configured to interpret instructions in matrix mode, the processor can interpret vector registers 210 and 220 as vectors of 2x2 matrices. The processor can interpret the matrix instruction 'vmul VR3, VR2, VR1' as performing a matrix multiplication between the matrices 212, 214, 216 and 218 of the first vector register and the matrices 222, 224, 226 and 228 of the second vector register.

[0067] The processor may multiply the first matrix 212 of the first vector register 210 by the first matrix 222 of the second vector register 220. The matrix 212 has V0 in the (0,0) position, V1 in the (0,1) position. V2 is located at the (1,0) position, and V4 is located at the (1,1) position. The matrix 222 has V16 in the (0,0) position, V17 in the (0,1) position. V18 is located at the (1,0) position, and V19 is located at the (1,1) position.

[0068] The result of multiplying the 2x2 matrix 212 with the 2x2 matrix 222 is another 2x2 result matrix 232. After performing the matrix multiplication, the (0,0) position of the result matrix 232 may include the result of V0×V16+V1×V18. The (0,1) position of the result matrix 232 includes the result of V0×V17+V1×V19. The (1,0) position of the result matrix 232 includes the result of V2×V16+V3×V18. The (1,1) position of the result matrix 232 includes the result of V2×V17+V3×V19.

[0069] The processor may multiply each remaining matrix in the first vector register 210 with the matrix of the same index in the second vector register 220 to produce a result matrix. Specifically, the processor may multiply the second 2x2 matrix 214 of the first vector register with the second 2x2 matrix 224 of the second vector register to produce a result 2x2 matrix 234. Similarly, the processor may multiply the matrix 216 with the matrix 226 to produce a result matrix 236, and multiply the matrix 218 with the matrix 228 to produce a result matrix 238.

[0070] Figure 2C Shows Figure 2A Matrix multiplication instructions can be used in processors (e.g., Figure 1 is implemented on a processor 102).

[0071] The processor may interpret the matrix instruction 'vmul VR3, VR2, VR1' as performing a matrix multiplication between the matrix in the first vector register 210 and the matrix in the second vector register 220 and storing the result in the third vector register 230. The third vector register 230 has the same dimensions as the first vector register 210 and the second vector register 220.

[0072] In this example, the third vector register 230 is a vector of 16 elements. The third vector register 230 stores the values ​​of the result matrices 232, 234, 236, and 238 of the vector multiplication operation. The first result matrix 232 is the result of multiplying the first 2x2 matrix of the first vector register 210 by the first 2x2 matrix of the second vector register 220. The elements of the first result matrix 232 fill the first four elements of the third vector register 230. Specifically, the first element of the third vector register 230 is the (0,0) index of the first result matrix 232, for example, V0×V16+V1×V18. The second element of the third register is the (0,1) index of the first result matrix 232, and the third element and the fourth element are filled by the (1,0) and (1,1) indexes, respectively.

[0073] In this mode, the elements of the second result matrix 234 fill the fifth to eighth elements of the third vector register 230. The next four elements are filled with elements of the third result matrix 236, and the last four elements are filled with elements of the fourth result matrix 238. Thus, four result matrices are represented as the third vector register 230.

[0074] Figure 3 is a flow chart illustrating an example process 300 for reinterpreting vector instructions as matrix instructions. Process 300 may be performed by a processor (e.g., Figure 1 Executed by processor 102).

[0075] The processor executes an instruction to set the configuration register to reinterpret the vector instruction as a matrix instruction (step 310). Setting the configuration register for matrix operations effectively overrides the meaning of the vector multiplication instruction so that the instruction causes the processor to perform matrix multiplication arithmetic. In doing so, the processor will reinterpret the vector register operand as a vector of matrices rather than a vector of individual elements.

[0076] Configuration instructions may be related to matrix width. In some implementations, the execution instruction sets a field in the configuration register that represents the matrix width. The matrix width field may represent the width of the matrix to be referenced by the vector instruction. In some implementations, the selected matrix width is specified as an exponent in the expression 2^N.

[0077] Configuration instructions may be related to matrix data order. In some implementations, execution instructions set a field in a configuration register that represents the matrix data order. The matrix data order field may specify whether the arrangement of values ​​in the vector register is row-major or column-major. In some implementations, the matrix data order field may be set to specify z-ordering or Morton ordering, which effectively interleaves the x and y coordinates.

[0078] Configuration instructions may be associated with widening mode. In some implementations, the execution instruction sets a field in a configuration register that indicates widening mode. The widening mode field may specify the bit width of the calculated output. Setting the widening mode field may cause the processor to allocate more bits to the output result. Conversely, the widening mode field may also be used to narrow the output, such as when the result needs to be shifted and truncated.

[0079] Configuration instructions may be related to the horizontal accumulation span. In some implementations, the execution instruction sets the field representing the horizontal accumulation span in the register. The horizontal accumulation span field may affect the operation of matrix multiplication and accumulation operations. In fact, the field specifies that the second addition step is performed after the multiplication but before the accumulation. In some examples, the execution instruction causes the processor to interpret the value of the horizontal accumulation span as a guide to use the pre-addition instruction during the multiplication-accumulation operation. The value of the horizontal accumulation span may represent the size of each matrix group that should be used as an input to the pre-addition operation. For example, if the value of the horizontal accumulation span is 2, each pair of matrices will be added together to become a single matrix to be used in the accumulation. The horizontal accumulation span effectively reduces the number of outputs that need to be written during the multiplication-accumulation operation.

[0080] The configuration instruction may be associated with an enable bit. In some examples, the execution instruction may specify the enable bit in the second configuration register. The enable bit may specify whether the processor will interpret the vector instruction as referencing a vector input or a matrix input.

[0081] The processor receives a vector instruction that references two vector registers (step 320). The vector registers can hold vector data for processing. The vector registers can have a specified number of elements. The vector registers can represent, for example, a one-dimensional array of integers, logical values, characters, or floating point numbers.

[0082] A vector instruction may cause a processor to perform an operation on two vector registers. For example, a vector instruction may cause a processor to multiply an element of a first vector with an element of the same index of a second vector, for example, multiplying a first element of a first vector register with a first element of a second vector register, multiplying a second element of a first vector register with a second element of a second vector register, and so on. As another example, a vector instruction may cause a processor to add elements of two vector registers together. In some implementations, a vector instruction may reference more than two vector registers. For example, an instruction may indicate that the result of multiplying (or adding, etc.) data in a vector register should be stored in a third vector register.

[0083] The processor reinterprets the vector instruction as a matrix instruction about the matrix stored in the two vector registers (step 330). The processor reinterprets the vector register as a vector of matrices of the specified size. For example, if the vector register has 16 elements and the specified size is 2x2, the processor reinterprets the vector register as a vector of 4 2x2 matrices. The first element of the vector becomes a matrix containing the first four elements of the original vector register. In some examples, the data in the vector register can be reinterpreted as a sequence of 2x2, 4x4, 8x8 or 16x16 matrices.

[0084] The processor may perform vector arithmetic on a sequence of matrices. For example, if the vector instruction is to multiply elements of a first vector with elements of the same index of a second vector, the processor may multiply the first matrix of the first reinterpreted vector register with the first matrix of the second reinterpreted vector register, and so on.

[0085] For example, suppose the processor receives a vector multiplication instruction that references two input vectors and a third output vector. If the configuration register specifies that the input is a 2x2 matrix, the processor will interpret each sequential group of four elements in the input vector register as a 2x2 matrix instead of four scalars, and will perform a matrix multiplication with the corresponding group of four values ​​in another input vector register. This strategy effectively doubles the performance of each execution channel by reusing each data input twice in two multiplication operations, which can have significant performance improvements.

[0086] Certain novel aspects of the subject matter of this specification are set forth in the following claims.

[0087] Embodiments of the subject matter and functional operations described in this specification may be implemented in digital electronic circuit systems, in tangibly embodied computer software or firmware, in computer hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of them. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory program carrier for execution by a data processing device or for controlling the operation of a data processing device. Alternatively or additionally, program instructions may be encoded on an artificially generated propagated signal (e.g., a machine-generated electrical, optical, or electromagnetic signal) that is generated to encode information to be sent to a suitable receiver device for execution by a data processing device. A computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. However, a computer storage medium is not a propagated signal.

[0088] The term "data processing apparatus" covers all kinds of equipment, devices and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. The equipment may include special-purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, the equipment may also include code that creates an execution environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0089] A computer program (which may also be referred to or described as a program, software, software application, module, software module, script, or code) may be written in any form of programming language, including compiled or interpreted languages ​​or declarative or procedural languages, and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). A computer program may be deployed to execute on one computer located at one site or on multiple computers distributed across multiple sites and interconnected by a communication network.

[0090] The processes and logic flows described in this specification may be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and the apparatus may also be implemented as, a special purpose logic circuit system, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0091] Computers suitable for executing computer programs include, by way of example, central processing units that can be based on general or special microprocessors or both, or any other kind of. Typically, the central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more large-capacity storage devices for storing data, such as a magnetic disk, a magneto-optical disk, or an optical disk, or be operatively coupled to receive data from it or transfer data to it or both. However, a computer need not have such a device. In addition, a computer may be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name a few.

[0092] Computer-readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices); magnetic disks (e.g., internal hard disks or removable disks); magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0093] Although this specification contains many specific implementation details, these details should not be interpreted as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features that may be specific to a particular embodiment of a particular invention. Certain features described in this specification in the context of a single embodiment may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. In addition, although features may be described above as working in certain combinations and even initially claimed as such, one or more features from the claimed combination may be deleted from the combination in some cases, and the claimed combination may involve a sub-combination or a change in the sub-combination.

[0094] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or that all of the operations shown be performed, to achieve the desired result. In some contexts, multitasking and parallel processing can be advantageous. In addition, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0095] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the actions recited in the claims can be performed in a different order and still achieve the desired results. As an example, the processes depicted in the accompanying drawings do not necessarily require the particular order shown or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous.

Claims

1. A processor configured to implement an instruction set architecture having instructions that, in operation, set a configuration register of the processor with one or more values ​​that cause the processor to reinterpret one or more vector instructions as matrix instructions.

2. The processor of claim 1, wherein the processor is configured to perform vector arithmetic on a sequence of matrices to reinterpret the vector instructions as matrix instructions.

3. The processor of any one of claims 1 to 2, wherein reinterpreting the vector instruction as a matrix instruction comprises reinterpreting data in the vector register as a matrix sequence.

4. The processor of claim 3, wherein reinterpreting the data in the vector register as a sequence of matrices comprises reinterpreting the data in the vector register as a sequence of 2x2, 4x4, 8x8, or 16x16 matrices.

5. The processor according to any one of claims 1 to 4, wherein the configuration register has a field indicating a matrix width. 6 . The processor of claim 5 , wherein the field representing the matrix width represents an exponent N for a matrix having a width given by 2^N.

7. The processor according to any one of claims 1 to 6, wherein the configuration register has a field indicating the order of matrix data.

8. The processor according to any one of claims 1 to 7, wherein the configuration register has a field indicating a widening mode.

9. A processor according to any one of claims 1 to 8, wherein the configuration register has a field representing a horizontal accumulation stride, wherein the processor is configured to interpret the value of the horizontal accumulation stride as a directive to use a pre-addition instruction during a multiply-accumulate operation.

10. A processor according to any one of claims 1 to 9, wherein the instruction set architecture specifies an enable bit in a second different configuration register, the enable bit specifying whether the processor is to interpret the one or more vector instructions as referencing vector inputs or matrix inputs.

11. A method performed by a processor implementing an instruction set architecture, the instruction set architecture having instructions for setting a configuration register of the processor, the configuration register controlling whether vector instructions are reinterpreted as matrix instructions, the method comprising: executing the instruction to set the configuration register; receiving one or more vector instructions; and reinterpreting the one or more vector instructions as matrix instructions based on the information set in the configuration register.

12. The method of claim 11, wherein reinterpreting the vector instructions as matrix instructions comprises performing vector arithmetic on a sequence of matrices.

13. The method of any one of claims 11 to 12, wherein reinterpreting the vector instructions as matrix instructions comprises reinterpreting data in the vector registers as a matrix sequence.

14. The method of claim 13, wherein reinterpreting the data in the vector register as a sequence of matrices comprises reinterpreting the data in the vector register as a sequence of 2x2, 4x4, 8x8, or 16x16 matrices.

15. The method of any one of claims 11 to 14, wherein executing the instruction sets a field in the configuration register representing a matrix width.

16. The method of claim 15, wherein the field representing the matrix width represents an exponent N for a matrix having a width given by 2^N.

17. The method of any one of claims 11 to 16, wherein executing the instruction sets a field in the configuration register representing matrix data order.

18. The method of any one of claims 11 to 17, wherein executing the instruction sets a field in the configuration register indicating a widening mode.

19. The method of any one of claims 11 to 18, wherein executing the instruction sets a field in the configuration register representing a horizontal accumulation stride, and further comprising interpreting the value of the horizontal accumulation stride as a directive to use a pre-add instruction during a multiply-accumulate operation.

20. The method of any one of claims 11 to 19, wherein the instruction set architecture specifies an enable bit in a second different configuration register, the enable bit specifying whether the processor is to interpret the one or more vector instructions as referencing vector inputs or matrix inputs.

21. One or more computer storage media encoded with instructions of an instruction set architecture having instructions for setting a configuration register to control whether a processor implementing the instruction set architecture will reinterpret vector instructions as matrix instructions, wherein the instructions are executed by the processor implementing the instruction set architecture to cause the processor to perform operations comprising: executing the instruction to set the configuration register; receiving one or more vector instructions; and reinterpreting the one or more vector instructions as matrix instructions based on the information set in the configuration register, reinterpreting the one or more vector instructions as matrix instructions.

22. The one or more computer storage media of claim 21, wherein reinterpreting the vector instructions as matrix instructions comprises performing vector arithmetic on a sequence of matrices.

23. The one or more computer storage media of any one of claims 21 to 22, wherein reinterpreting a vector instruction as a matrix instruction comprises reinterpreting data in a vector register as a matrix sequence.

24. The one or more computer storage media of claim 23, wherein reinterpreting the data in the vector register as a sequence of matrices comprises reinterpreting the data in the vector register as a sequence of 2x2, 4x4, 8x8, or 16x16 matrices.

25. The one or more computer storage media of any one of claims 21 to 24, wherein executing the instruction sets a field in the configuration register representing a matrix width.

26. The one or more computer storage media of claim 25, wherein the field representing the matrix width represents an exponent N for a matrix having a width given by 2^N.

27. One or more computer storage media according to any one of claims 21 to 26, wherein executing the instructions sets a field in the configuration register representing matrix data order.

28. One or more computer storage media according to any one of claims 21 to 27, wherein executing the instructions sets a field in the configuration register indicating a widening mode.

29. One or more computer storage media according to any one of claims 21 to 28, wherein executing the instruction sets a field in the configuration register representing a horizontal accumulation stride, and further comprises interpreting the value of the horizontal accumulation stride as a directive to use a pre-addition instruction during a multiply-accumulate operation.

30. One or more computer storage media according to any one of claims 21 to 29, wherein the instruction set architecture specifies an enable bit in a second different configuration register, the enable bit specifying whether the processor is to interpret the one or more vector instructions as referencing vector inputs or matrix inputs.

Images