Execution of a computer instruction for matrix multiplication

Abstract

This invention provides an improved method, system, and computer program for performing matrix multiplication computer instructions. [Solution] The method is a processor having a control register and a vector register whose length is indicated in the control register, in which a matrix operation A × B = C is performed, where A is an M × K matrix with elements of a first bit width, B is a K × N matrix with elements of a second bit width, and C is an M × N matrix with elements of a third bit width. The instruction has three parameters that encode the type of multiplication to be performed and indicate which vector register to sequentially store the elements of each matrix in. The method also obtains the length from the control register, decodes the instruction to obtain the multiplication type, determines the first and second bit widths according to the multiplication type, determines the values ​​of M, N, and K according to the length and the first and second bit widths, and performs matrix multiplication considering the three parameters, the first and second bit widths, and the values ​​of M, N, and K.

Description

Technical Field

[0001] The present disclosure relates to a computer-implemented method for performing computer instructions to multiply a first matrix and a second matrix, as well as a computer program and a system suitable for implementing such a computer-implemented method.

Background Art

[0002] Matrix multiplication is known to be a type of operation widely used in various computer-based applications related to, for example, robotics, computer graphics, image and signal processing, artificial intelligence (AI) and machine learning (ML). For example, AI and ML have become established in computing devices of all kinds in recent years. Modern ML models consist of hundreds of convolutional or fully connected layers that can be efficiently implemented as matrix multiplications. To meet the demanding performance goals of large-scale ML models, computer architectures have added instructions for efficiently processing matrices.

[0003] Processor architectures are known that support various data types with the ability to access / address and process vector registers. An example of the above architecture is RISC-V, an open standard instruction set architecture (ISA) that can be freely used. It is a clean and modern ISA that is easy to extend and provides a solid foundation for building processors. Due to its flexibility, it has become an ISA that is popular in many applications.

[0004] Today, it is desirable and challenging to improve the implementation of matrix multiplication on any type of processor available for implementing computer-based solutions that rely on matrix multiplication. Such computer-based solutions can be more efficient and powerful in proportion to any improvement in efficiency of the implementation of matrix multiplication. The more efficient the matrix multiplication computation, the more efficient and powerful the applications that rely on it will be. [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] Therefore, the object of this disclosure is to provide novel methods, systems, and computer programs aimed at improving current methods for implementing computer instructions for matrix multiplication. [Means for solving the problem]

[0006] In a first embodiment, a computer implementation method is provided for performing a computer instruction to multiply a first matrix and a second matrix in a processor having an architecture including a set or array of vector registers and a first control register, each of the vector registers having a vector length indicated in the first control register. The first matrix consists of first numeric elements having a first bit width arranged in M ​​rows and K columns, the second matrix consists of second numeric elements having a second bit width arranged in K rows and N columns, and the third matrix obtained as a result of multiplying the first and second matrices consists of third numeric elements having a third bit width arranged in M ​​rows and N columns. A computer instruction (also referred herein as a matrix multiplication instruction) encodes an operation code indicating the type of multiplication to be performed and has a first matrix parameter, a second matrix parameter, and a third matrix parameter. The first matrix parameter indicates which vector registers sequentially store the first numeric elements. The second matrix parameter indicates which vector registers sequentially store the second numeric elements. The third matrix parameter indicates which vector registers the third numerical element (of the result) will be sequentially stored in (or should be sequentially stored in).

[0007] These computer implementations (also referred to herein as matrix multiplication methods) include, at runtime, obtaining a vector length from a first control register, and at runtime, decoding a computer instruction to obtain the operation code encoded in the computer instruction, and thus the type of multiplication to be performed. Such a matrix multiplication method further includes, at runtime, determining a first bit width and a second bit width depending on the type of multiplication to be performed obtained, and at runtime, determining values ​​for M, N, and K depending on the obtained vector length and the determined first and second bit widths. These matrix multiplication methods also further include, at runtime, performing a multiplication of the first and second matrices, taking into account a first matrix parameter, a second matrix parameter, and a third matrix parameter, the determined first and second bit widths, and the determined values ​​for M, N, and K.

[0008] Given that the first, second, and third matrix parameters (i.e., if the matrices involved are located in a vector register), the first and second bit widths (i.e., the length of the bit block to be accessed / addressed corresponding to the elements of the matrices involved), and the values ​​of M, N, and K (i.e., the dimensions of the matrices involved) are available, the multiplication of the first and second matrices can be performed in any known way based on the available data. Since matrix multiplication algorithms and how they can be implemented are well known in the art at present, details therein are not provided here. Somewhere in this disclosure, for reasons of completeness only, a specific way of doing so is provided via source code 1.

[0009] The matrix multiplication method proposed herein provides an efficient implementation for matrix multiplication, which is an essential kernel in many modern computer-based applications. This efficiency stems, among other things, from the fact that it is used to store matrices involving existing vector registers, thus requiring no new architecturally visible state. This means that existing tools and operating systems of the adopted processor architecture (e.g., RISC-V architecture) will function without requiring any modifications. Furthermore, it can support a wide variety of data types.

[0010] In some examples, the architecture may be a RISC-V architecture. In this case, the set or array of vector registers may be a vector register file (VRF) of the RISC-V architecture, and the first control register may be a vector length register (VLEN) of the RISC-V architecture. Other processor architectures on which the matrix multiplication method according to this disclosure can be implemented may be, for example, x86, ARM, MIPS, etc.

[0011] Depending on the implementation, the architecture may further include a second control register, thereby allowing the operation code (opcode) indicating the type of multiplication to be performed to be encoded in combination with the computer instruction (or its name or format) and some or all of the contents of the second control register. Thus, in these implementations, the type of multiplication to be performed can be determined at runtime by decoding the computer instruction along with some or all of the contents of the second control register to obtain the operation code encoded in the computer instruction. In this way, if it is not possible to encode all desired types of matrix multiplication to be performed by the instruction name or format alone, an extended opcode range may be defined by a combination of the instruction name or format and some or all of the contents of the second control register.

[0012] In RISC-V-based implementations, this second control register may be a RISC-V architecture vector type register (vtype). In particular, three bits in vtype[5:3] may indicate whether the first and second bit widths are 16 bits or 8 bits, which, in combination with the instruction name or format, can encode a wider range of opcodes than the limited opcodes that can be encoded by the instruction name or format alone. Other bits in vtype, such as one bit in vtype

[10] , may indicate the number type of the first and second numeric elements, e.g., Brain floating-point (BF16) or IEEE 754 16-bit floating-point (FP16), so that the opcodes that can be encoded by the instruction name or format alone are extended thanks to the further information provided by the aforementioned one bit in vtype

[10] . Details on how to encode an extended opcode range by combining the full or partial contents of the instruction name / format and the second control register (e.g., vtype) are provided in other parts of this disclosure, referencing, for example, Table 2.

[0013] The first matrix parameter may indicate the first initial vector register of a first sequence of consecutive vector registers in which the first numerical elements are sequentially stored, starting from the first initial vector register. The second matrix parameter may indicate the second initial vector register of a second sequence of consecutive vector registers in which the second numerical elements are sequentially stored, starting from the second initial vector register. The third matrix parameter may indicate the third initial vector register of a third sequence of consecutive vector registers in which the third numerical elements are sequentially stored (or should be stored), starting from the third initial vector register.

[0014] In some implementations, the first and second bit widths may be the same bit width, equal to 4 bits, 8 bits, or 16 bits, depending on the opcode obtained or the type of multiplication to be performed. In the example, the third bit width may be 32 bits, which may correspond to one of the following numeric representations, depending on the opcode obtained or the type of multiplication to be performed: a 32-bit floating-point number, FP32; a 32-bit signed integer, INT32; or a 32-bit unsigned integer, UINT32.

[0015] If the first and second bit widths are 16 bits, then these 16 bits represent one of the following numerical representations, depending on the opcode obtained or the type of multiplication to be performed: • 16-bit floating-point numbers, FP16, for both the first and second numeric elements. • 16-bit Brain floating-point numbers, BF16, for both the first and second numerical elements. • 16-bit signed integers, INT16, for both the first and second numeric elements. • Both the first and second numeric elements are 16-bit unsigned integers, UINT16. The first numeric element is a 16-bit signed integer, INT16, and the second numeric element is a 16-bit unsigned integer, UINT16. It can be addressed.

[0016] If the first and second bit widths are 8 bits, then these 8 bits represent one of the following numerical representations, depending on the opcode obtained or the type of multiplication to be performed: • For both the first and second numeric elements, an 8-bit signed integer, INT8, • For both the first and second numeric elements, an 8-bit unsigned integer, UINT8, The first numeric element is an 8-bit signed integer, INT8, and the second numeric element is an 8-bit unsigned integer, UINT8. It can be addressed.

[0017] If the first and second bit widths are 4 bits, then these 4 bits represent one of the following numerical representations, depending on the opcode obtained or the type of multiplication to be performed: • For both the first and second numeric elements, a 4-bit signed integer, INT4, • For both the first and second numeric elements, a 4-bit unsigned integer, UINT4, The first numeric element is a 4-bit signed integer, INT4, and the second numeric element is a 4-bit unsigned integer, UINT4. It can be addressed.

[0018] Details relating to possible different variations or types of matrix multiplication that can be performed by the matrix multiplication methods of this disclosure are provided elsewhere in this specification, in particular with respect to Table 2, the variations including combinations of the aforementioned number types, namely FP16, BF16, INT16, UINT16, or INT8, UINT8, or INT4, UINT4 for both the first and second number elements, and FP32, INT32, UINT32 for the third number element (of the result).

[0019] In the example of the matrix multiplication method, the values ​​of M and N may be determined at runtime based only on the obtained vector length, while the value of K may be determined at runtime based on the obtained vector length as well as the further determined first and second bit widths.

[0020] In some examples, determining the value of K according to the acquired vector length and the determined first and second bit widths may include establishing a 16-bit operating bit width (at runtime), determining operation K according to the acquired vector length and established operating bit width (at runtime), and determining the value of K according to the determined operation K and the determined first and second bit widths (at runtime). In particular, the value of K may be set at runtime to be equal to the determined operation K if the determined first and second bit widths are 16 bits, or equal to the determined operation K multiplied by 2 if the determined first and second bit widths are 8 bits, or equal to the determined operation K multiplied by 4 if the determined first and second bit widths are 4 bits.

[0021] In some configurations, the processor may support functions that address (or handle) 8-bit or 16-bit bit packets instead of 4-bit bits in a vector register. With this in mind, performing a multiplication of a first matrix and a second matrix may involve, at runtime, accessing the first and second numeric elements to be processed for multiplication via an addressing function, either in direct mode if the first and second bit widths are 8-bit or 16-bit, or indirect mode if the first and second bit widths are 4-bit. Direct mode may involve using an addressing function to access each of the 8-bit or 16-bit numeric elements being processed without any access-related post-processing. Indirect mode may involve using an addressing function to access the 8-bit bit packet containing each of the 4-bit numeric elements being processed, and, along with access-related post-processing of the accessed / addressed 8-bit bit packet, identifying its four MSBs (most significant bits) or four LSBs (least significant bits) as the 4-bit numeric element being processed.

[0022] In some implementations, the first matrix can be a submatrix of a first complete matrix composed of various submatrices including the first submatrix, and the second matrix can be a submatrix of a second complete matrix composed of various submatrices including the second submatrix, whereby some matrix multiplication methods according to the present disclosure can correspond to the submatrix multiplication accumulation method. These methods by accumulation can include, at runtime, accumulating the multiplication of the first submatrix and the second submatrix within a (resulting) third matrix as a partial result of the multiplication of the first complete matrix and the second complete matrix. Thus, the multiplication of the first complete matrix and the second complete matrix can be implemented by executing one of these submatrix multiplication accumulation methods as many times as necessary according to the different submatrices into which the first complete matrix and the second complete matrix are splittable or separable or divisible.

[0023] In a further aspect, a computer program comprising program instructions for causing a processor system to implement any of the matrix multiplication methods disclosed herein is provided. The computer program (also referred to herein as the matrix multiplication computer program) may be embodied on a storage medium and / or carried on a carrier signal. The matrix multiplication computer program is suitable for implementing a matrix multiplication method as described in other parts of the present disclosure. Thus, for example, aspects of the matrix multiplication method such as functional, structural, advantageous considerations, etc. can similarly be attributed to the matrix multiplication computer program.

[0024] In yet a further aspect, a system is provided for performing a computer instruction that multiplies a first matrix and a second matrix in a processor having an architecture that includes a set or array of vector registers and a first control register, each of the vector registers having a vector length indicated in the first control register. The first matrix is composed of first numerical elements having a first bit width arranged in M rows and K columns, the second matrix is composed of second numerical elements having a second bit width arranged in K rows and N columns, and a third matrix resulting from the multiplication of the first matrix and the second matrix is composed of third numerical elements having a third bit width arranged in M rows and N columns. The computer instruction encodes an operation code indicating the type of multiplication to be performed and has a first matrix parameter, a second matrix parameter, and a third matrix parameter. The first matrix parameter indicates in which vector registers the first numerical elements are sequentially stored. The second matrix parameter indicates in which vector registers the second numerical elements are sequentially stored. The third matrix parameter indicates in which vector registers the (resulting) third numerical elements are sequentially stored (or are to be sequentially stored).

[0025] Such a system (also referred to herein as a matrix multiplication system) comprises a length module, a decoding module, a bit width module, a dimension module, and a multiplication module. The length module is configured to obtain the vector length from a first control register at runtime. The decoding module is configured to decode a computer instruction at runtime to obtain the operation code encoded in the computer instruction, and therefore the type of multiplication to be performed. The bit width module is configured to determine a first bit width and a second bit width at runtime, depending on the type of multiplication to be performed obtained. The dimension module is configured to determine the values ​​of M, N, and K at runtime, depending on the obtained vector length and the determined first and second bit widths. The multiplication module is configured to perform the multiplication of a first matrix and a second matrix at runtime, taking into account a first matrix parameter, a second matrix parameter, a third matrix parameter, the determined first and second bit widths, and the determined values ​​of M, N, and K. These matrix multiplication systems are suitable for performing matrix multiplication methods as described in other parts of this disclosure. Therefore, the above-mentioned aspects of matrix multiplication methods, such as functional, structural, and advantageous considerations, can similarly be attributed to matrix multiplication systems.

[0026] In a further embodiment, a processor system is provided having an architecture comprising a set / array of vector registers and one or more control registers, each of which has a vector length indicated in one of the control registers, and also comprising either (or a combination thereof) a matrix multiplication system or a computer program according to the present disclosure incorporated into the processor system. [Brief explanation of the drawing]

[0027] Non-limiting examples of this disclosure are described below with reference to the attached drawings. [Figure 1]This is a schematic block diagram illustrating a system for executing a computer instruction to multiply a first matrix by a second matrix, as an example. [Figure 2] This flowchart schematically illustrates a computer implementation method for performing a computer instruction that multiplies a first matrix and a second matrix, using an example. [Modes for carrying out the invention]

[0028] Figure 1 is a schematic block diagram of a system 100 for implementing a computer instruction to multiply a first matrix and a second matrix, as an example. This computer instruction may also be referred to herein as a matrix multiplication instruction. The matrix multiplication system 100 may be contained in a processor 116 having an architecture including a set or array of vector registers 112 and a first control register 114, and therefore may operate in the processor 116. Each of the vector registers 112 may have a vector length 108, as indicated by the contents of the first control register 114. The architecture may be any processor architecture in which the matrix multiplication method according to this disclosure can be implemented and performed, such as RISC-V, x86, ARM, or MIPS. In a RISC-V implementation, the set or array of vector registers 112 may be a vector register file (VRF) of the RISC-V architecture, and the first control register 114 may be a vector length register (VLEN) of the RISC-V architecture.

[0029] The first matrix may consist of a first number of elements arranged in M ​​rows and K columns. Each of the first number of elements may have a first bit width. The second matrix may consist of a second number of elements arranged in K rows and N columns. Each of the second number of elements may have a second bit width. The result of multiplying the first matrix and the second matrix, or the third matrix obtained as a result thereof, may consist of a third number of elements arranged in M ​​rows and N columns. Each of the third number of elements may have a third bit width. The first and second bit widths may be the same bit width, for example, 16 bits, 8 bits, or 4 bits. The third bit width may, in principle, be 32 bits, which is sufficiently large for the purpose of preventing overflow.

[0030] A matrix multiplication instruction (to be performed) can encode an operation code (opcode) indicating the type of matrix multiplication to be performed, and may have first, second, and third matrix parameters. The first matrix parameter may indicate which of the vector registers 112 will sequentially store the first number of elements. For example, the first matrix parameter may indicate the first initial vector register of a first sequence of consecutive vector registers (in the array of vector registers 112), and the first number of elements will be stored sequentially starting from the first initial vector register. The second matrix parameter may indicate which of the vector registers 112 will sequentially store the second number of elements. For example, the second matrix parameter may indicate the second initial vector register of a second sequence of consecutive vector registers (in the array of vector registers 112), and the second number of elements will be stored sequentially starting from the second initial vector register. The third matrix parameter may indicate which of the vector registers 112 will sequentially store (or should store) the third number of elements. For example, the third matrix parameter may indicate a third initial vector register of a third sequence of consecutive vector registers (in the array of vector registers 112), and the third number of elements are stored (or should be stored) sequentially, starting from the third initial vector register.

[0031] The matrix multiplication method may be triggered during the execution of the application program in response to a call that invokes the execution of a matrix multiplication instruction, which is included in the application program. Prior to the above call, the application program may generate the necessary execution preconditions that enable the execution of the matrix multiplication instruction. The generation of the above execution preconditions may include, for example, providing a vector length 108 to a first control register 114, loading a first matrix into a required or desired vector register 112, loading a second matrix into a required or desired vector register 112, and reserving a required / desired vector register 112 to accommodate a third matrix (i.e., the result of the matrix multiplication). Once these execution preconditions are generated by the running application program, a call to the matrix multiplication instruction may be triggered to perform the multiplication of the first matrix and the second matrix. This call may be programmed within the application program with an appropriate opcode and first, second, and third parameters that perfectly match the previously generated execution preconditions. The first and second number elements (making up the first and second matrices, respectively) may need to be sequentially loaded into the corresponding vector registers in a specific order suitable for the algorithm in which the matrix multiplication is implemented. This is a well-known fact in the current technological field, so I will not go into detail here.

[0032] The matrix multiplication system 100 may comprise a length module 101, a decoding module 102, a bit width module 103, a dimension module 104, and a multiplication module 105.

[0033] The length module 101 may be configured at runtime to obtain or retrieve the vector length 108 from the first control register 114. The vector length 108 may have been input to the first control register 114 via an appropriate instruction(s) in the application program prior to the call to the matrix multiplication instruction. The decoding module 102 may be configured at runtime to decode the (called) matrix multiplication instruction and extract from it the opcode 106 and therefore the type 107 of the multiplication to be performed. The call to the matrix multiplication instruction may be made after the necessary consistent execution preconditions have been established by the running application program, as described above.

[0034] The bit width module 103 may be configured to determine the first and second bit widths 109 at runtime, depending on the type of multiplication to be performed, which has been previously determined, 107. The dimension module 104 may be configured to determine the values ​​of M, N, and K at runtime, depending on the vector length 108 and the first and second bit widths 109, which have been previously determined, 108. The multiplication module 105 may also be configured to perform the multiplication of the first and second matrices at runtime, taking into account the first, second, and third matrix parameters 111, the first and second bit widths 109 and the values ​​of M, N, and K 110.

[0035] The values ​​of M, K, and N (i.e., matrix dimensions) 110 may depend on the vector length 108 and the first and second bit widths 109 (i.e., the bit widths of the first and second number elements that make up the first and second matrices, respectively). For example, if vector lengths from 64 bits to 16384 bits are supported, the values ​​of M, K, and N may be as shown in Table 1 below, with VLEN being a vector length of 108. [Table 1]

[0036] If the number of defined or to be defined different opcodes 106 exceeds the opcode capacity of the available instruction format within the processor architecture being used, the extended range of different opcodes can be defined using all or part of the second control register (sCR) 115. In a RISC-V implementation, this second control register 115 may be, for example, the vector type register (vtype) of the RISC-V architecture.

[0037] For example, Table 2 below shows possible combinations of instruction formats and the sCR portion that encodes the extended range of the opcode, where vd is the third matrix parameter, vs1 is the first matrix parameter, vs2 is the second matrix parameter, FP16 is a 16-bit floating-point number, FP32 is a 32-bit floating-point number, BF16 is a 16-bit brain floating-point number, INT16 is a 16-bit signed integer, INT32 is a 32-bit signed integer, UINT16 is a 16-bit unsigned integer, UINT32 is a 32-bit unsigned integer, INT8 is an 8-bit signed integer, UINT8 is an 8-bit unsigned integer, INT4 is a 4-bit signed integer, and UINT4 is a 4-bit unsigned integer. [Table 2]

[0038] In a RISC-V implementation using vtype as a second control register 115, vtype[5:3] may be sCR-part1 whose 3-bit content can indicate, for example, whether the first and second bit widths 109 are 16 bits or 8 bits, and / or vtype

[10] may be sCR-part2 whose 1-bit content can indicate, for example, whether the first and second number elements are BF16 or FP16. In these implementations, the execution prerequisites necessary to enable the execution of matrix multiplication instructions (as described in other parts of this disclosure) may further include providing appropriate content to sCR-part1 and sCR-part2, for example, according to Table 2.

[0039] In the specific case of instruction format = "vfmxm vd,vs1,vs2", sCR-part1=16, and sCR-part2=0 (first row of Table 2), the above combination corresponds to a type of matrix multiplication in which the first and second matrices consist of FP16 elements and the resulting third matrix consists of FP32 elements. In the specific case of instruction format = "vmxmsu4 vd,vs1,vs2" (last row of Table 2), sCR-part1 and sCR-part2 are not used to encode the corresponding opcode, but instruction format = "vmxmsu4 vd,vs1,vs2" alone specifies a type of matrix multiplication in which the first matrix consists of INT4 elements, the second matrix consists of UINT4 elements and the resulting third matrix consists of INT32 elements.

[0040] Given the first, second, and third matrix parameters 111, the first and second bit widths 109, and the values ​​of M, N, and K 110, the multiplication module 105 may address / access each of the first number elements constituting the first matrix, each of the second number elements constituting the second matrix, and each of the third number elements constituting the resulting third matrix. The above addressing of the first, second, and third number elements may be programmed, for example, in an indexed or index-based manner. The following source code 1 represents one of many possible software-based implementations of matrix multiplication, which is based on a "three nested FOR statements" technique with index-based addressing of the first, second, and third number elements. [Table 3]

[0041] In this source code 1, A is the first matrix, B is the second matrix, and C is the resulting third matrix. The outermost FOR and index i are exclusive to traverse dimension M, the intermediate FOR and index j are exclusive to traverse dimension N, and the innermost FOR and index k are exclusive to traverse dimension K. The get_dimensions function is dedicated to determining matrix dimensions M, N, and K, and details of its calculation are provided in other parts of the explanation. The specific matrix multiplication method to be implemented corresponds to one of the submatrix multiplication accumulation methods, the details of which are provided in other parts of this disclosure.

[0042] Regarding the determination of the matrix dimensions M, K, and N110, the values ​​of M and N may be determined solely according to the vector length 108 according to the criteria in Table 1, for example, while K may be determined according to the vector length 108 and the first and second bit widths 109. In particular, K may be determined by establishing a 16-bit operating bit width, determining operation K according to the vector length 108 and the (16-bit) operating bit width according to the criteria in Table 1, for example, and determining K according to operation K and the first and second bit widths 109. Specifically, K may be determined to be equal to operation K when the first and second bit widths 109 are 16 bits, or equal to operation K multiplied by 2 when the first and second bit widths 109 are 8 bits, or equal to operation K multiplied by 4 when the first and second bit widths 109 are 4 bits.

[0043] The processor 116 may support address functions that address or handle 8, 16, or 32-bit bit packets instead of 4 bits in the vector register 112 (e.g., the index-based address function in source code 1). This is the case, for example, in the RISC-V architecture. If the first, second, and third bit widths 109 are 8, 16, or 32 bits, the first, second, and third number elements can be directly addressed or handled using the address functions described above. If, instead, a portion of the bit width 109 is 4 bits, each of the 4-bit number elements can be addressed / handled by, firstly, addressing the 8-bit packet containing it using the address function, and secondly, identifying the four MSBs (most significant bits) or four LSBs (least significant bits) within the addressed 8-bit packet as 4-bit number elements to be processed for matrix multiplication.

[0044] In the example, the first matrix may be a submatrix of a first complete matrix that can be divided into various submatrices including the first submatrix, and the second matrix may be a submatrix of a second complete matrix that can be divided into various submatrices including the second submatrix, in which case the matrix multiplication method may support cumulative submatrix multiplication. To implement the above support, the matrix multiplication method may further include accumulating the multiplications of the first and second submatrices in a third matrix as a partial result of multiplying the first and second complete matrices at runtime. Thus, the multiplication of the first and second complete matrices can be performed by executing one of such cumulative submatrix multiplication methods as many times as necessary, depending on the various submatrices from which the first and second complete matrices can be divided. The specific implementation represented by source code 1 supports cumulative submatrix multiplication to compute C = A * B + C (instead of just C = A * B), where A is the first submatrix, B is the second submatrix, C to the right of the equals sign is the partial multiplication of the first and second complete matrices accumulated immediately before the current execution, and C to the left of the equals sign is the partial multiplication of the first and second complete matrices accumulated at the end of the current execution (of the cumulative submatrix multiplication method).

[0045] Figure 2 is a schematic flowchart illustrating a computer implementation method for performing a computer instruction to multiply a first matrix by a second matrix, by example. Since such a matrix multiplication method can be performed by the matrix multiplication system shown in Figure 1, the number references from Figure 1 above can be reused in the following description of Figure 2. Thus, the same or similar functional and operational principles and aspects described with reference to Figure 1 may be attributable to the matrix multiplication method shown in Figure 2, but the difference is that Figure 1 represents such principles and aspects from the perspective of a (matrix multiplication) system, while Figure 2 represents them from the perspective of a (matrix multiplication) method or process flow. Figure 2 shows arrows indicating a sequence of executions and logically connected method steps or actions from the perspective of a (matrix multiplication) method / process flow. Each sequence of executions shown in Figure 2 follows a specific execution order, but other logical execution orders are possible whenever the conditions before and after each method step / action involved are met.

[0046] As shown in Figure 2, the matrix multiplication method may be initiated, for example, in response to a call that invokes a matrix multiplication instruction contained in an application program being executed by processor 116 (for example, in block 200). The considerations regarding matrix multiplication instructions and application programs described above with respect to Figure 1 also apply to this description of Figure 2.

[0047] The matrix multiplication method may include obtaining the vector length from the first control register 114 at runtime (for example, in block 201). This acquisition function, which can be implemented, for example, in method block 201, may be carried out by the length module 101 described above, with reference to Figure 1, for example. Thus, the functional details and considerations described with respect to module 101 may similarly apply to method block 201.

[0048] The matrix multiplication method may further include, at runtime (for example, in block 202), decoding the computer instruction to obtain the operation code 106 encoded in the computer instruction, and therefore the type 107 of the multiplication to be performed. This decoding function, which can be implemented, for example, in / by method block 202, may be implemented by the decoding module 102 described above, with reference to Figure 1, for example. Thus, the functional details and considerations described with respect to module 102 may similarly be attributed to method block 202.

[0049] The matrix multiplication method may further include determining the first and second bit widths 109 at runtime (for example, in block 203) depending on the acquired type 107 of the multiplication to be performed. This bit width function, which can be implemented in / by method block 203, for example, may be implemented by the bit width module 103 mentioned above with reference to Figure 1, for example. Thus, the functional details and considerations described with respect to module 103 may similarly be attributed to method block 203.

[0050] The matrix multiplication method may further include determining the values ​​of M, N, and K110 at runtime (for example, in block 204) according to the obtained vector length 108 and the determined first and second bit widths 109. This matrix dimension function, which can be implemented in / by method block 204, for example, can be implemented by the dimension module 104 described above with reference to Figure 1, for example. Thus, the functional details and considerations described with respect to module 104 can similarly be attributed to method block 204.

[0051] The matrix multiplication method may further include, at runtime (for example, in block 205), performing the multiplication of the first and second matrices, taking into account the first, second and third matrix parameters 111, the determined first and second bit widths 109, and the determined values ​​of M, N, and K 110. This matrix multiplication function, which can be implemented, for example, in method block 205, may be implemented by the multiplication module 105 described above, for example with reference to Figure 1. Thus, the functional details and considerations described with respect to module 105 may similarly apply to method block 205.

[0052] The matrix multiplication method may be terminated at runtime (for example, in block 206) by storing the third matrix, i.e., the result of the matrix multiplication (i.e., transferring it from vector register 112 to memory), so that the running application program has the third matrix available in memory and can handle it as if it were programmed within the application program.

[0053] As used herein, the term “module” should be understood to refer to software, firmware, hardware, and / or various combinations thereof. It should be noted that modules are illustrative. Modules may be combined, integrated, separated, and / or duplicated to support various applications. Furthermore, functions described herein as being performed by a particular module may be performed by one or more other modules and / or one or more other devices instead of, or in addition to, the functions performed by the described particular module.

[0054] A module may be implemented across multiple devices, associated with or linked to the corresponding matrix multiplication method / system proposed herein and / or other components which may be local or remote to each other. Additionally, a module may be moved from one device and added to another, and / or included in both devices, associated with the corresponding matrix multiplication method / system proposed herein. Any software implementation may be tangibly embodied in one or more storage media, such as a memory device, a floppy disk, a compact disc (CD), a digital versatile disc (DVD), or other devices capable of storing computer code.

[0055] The matrix multiplication system provided herein may be implemented by computational means, electronic means, or a combination thereof. The computational means may be a set of instructions (e.g., a computer program), and the matrix multiplication system may use memory and a processor to embody the set of instructions stored in memory and executable by the processor. These instructions may include one or more functions for performing corresponding matrix multiplication methods, such as those described in other parts of this disclosure.

[0056] When a matrix multiplication system is implemented solely by electronic means, the controller of the matrix multiplication system may be, for example, a CPLD (composite programmable logic device), an FPGA (field programmable gate array), or an ASIC (application-specific integrated circuit).

[0057] If the matrix multiplication system is a combination of electronic and computational means, the computational means may be a set of instructions (e.g., a computer program), and the electronic means may be any electronic circuit capable of implementing the corresponding method steps of the matrix multiplication methods proposed herein, such as those described in other parts of this disclosure.

[0058] Computer programs may be implemented on a storage medium (e.g., a CD-ROM, DVD, USB drive, computer memory, or read-only memory) or carried on a carrier signal (e.g., an electrical or optical carrier signal).

[0059] The computer program(s) may be in the form of source code, object code, code intermediate source and object code, for example, partially compiled code, or any other form suitable for use in implementing the matrix multiplication method of this disclosure. The carrier may be any entity or device capable of carrying the computer program(s).

[0060] For example, the carrier may include a storage medium such as ROM, for example, a CD-ROM or semiconductor ROM, or a magnetic recording medium such as a hard disk. Furthermore, the carrier may be a transmittable carrier such as an electrical or optical signal that can be carried via electrical or optical cables, or by wireless or other means.

[0061] If a computer program(s) is embodied as a signal that can be directly carried by a cable or other device or means, the carrier may consist of such a cable or other device or means. Alternatively, the carrier may be an integrated circuit in which the computer program(s) is embedded, and the integrated circuit is adapted to carry out or to be used in carrying out the matrix multiplication method proposed herein.

[0062] Although only a few examples are disclosed herein, other substitutions, modifications, uses, and / or equivalents are possible. Furthermore, all possible combinations of the examples described are also covered. Therefore, the scope of this disclosure should not be limited by any particular example, but should be determined solely by a fair reading of the appended claims.

Claims

1. A computer implementation method for performing a computer instruction to multiply a first matrix and a second matrix in a processor having an architecture including a set or array of vector registers and a first control register, wherein each of the vector registers has a vector length indicated in the first control register, The first matrix is ​​composed of first numeric elements having a first bit width arranged in M ​​rows and K columns, the second matrix is ​​composed of second numeric elements having a second bit width arranged in K rows and N columns, and the third matrix obtained as a result of multiplying the first matrix and the second matrix is ​​composed of third numeric elements having a third bit width arranged in M ​​rows and N columns. The computer instruction encodes an operation code indicating the type of multiplication to be performed, and has the following parameters: a first matrix parameter indicating which vector registers the first numeric element is sequentially stored in; a second matrix parameter indicating which vector registers the second numeric element is sequentially stored in; and a third matrix parameter indicating which vector register the third numeric element is or should be sequentially stored in, and the computer implementation method is During execution, the vector length is obtained from the first control register, During execution, the computer instruction is decoded to obtain the operation code encoded in the computer instruction, and therefore the type of multiplication to be performed. At runtime, the first bit width and the second bit width are determined according to the type of multiplication to be performed obtained, During execution, the values ​​of M, N, and K are determined according to the acquired vector length and the determined first and second bit widths. During execution, the multiplication of the first matrix and the second matrix is ​​performed, taking into account the first matrix parameter, the second matrix parameter, the third matrix parameter, the determined first bit width and second bit width, and the determined values ​​of M, N, and K. A computer implementation method that performs a computer instruction to multiply a first matrix and a second matrix, including the following.

2. The architecture further includes a second control register, and the operation code indicating the type of multiplication to be performed is encoded in the computer instruction together with some or all of the contents of the second control register, and the decoding of the computer instruction is A computer implementation method for performing a computer instruction to multiply a first matrix and a second matrix according to claim 1, comprising decoding the computer instruction together with the contents of part or all of the second control register at runtime to obtain the operation code encoded in the computer instruction, and therefore the type of multiplication to be performed.

3. The first matrix parameter indicates the first initial vector register of a first sequence of consecutive vector registers in which the first numerical elements are sequentially stored, starting from the first initial vector register. The second matrix parameter indicates the second initial vector register of a second sequence of consecutive vector registers in which the second numerical elements are sequentially stored, starting from the second initial vector register. A computer implementation method for performing a computer instruction to multiply a first matrix and a second matrix according to claim 1 or 2, wherein the third matrix parameter indicates the third initial vector register of a third sequence of consecutive vector registers in which the third numerical elements are sequentially stored, starting from the third initial vector register, or which should be sequentially stored.

4. A computer implementation method for performing a computer instruction to multiply a first matrix and a second matrix according to any one of claims 1 to 3, wherein the third bit width is 32 bits corresponding to one of the following numerical representations, namely a 32-bit floating-point number, FP32, a 32-bit signed integer, INT32, a 32-bit unsigned integer, or UINT32, depending on the type of multiplication to be performed obtained.

5. The value of M, the value of N, and the value of K are determined as described above. A computer implementation method for performing a computer instruction to multiply a first matrix and a second matrix according to any one of claims 1 to 4, comprising determining the value of M and the value of N according to the acquired vector length only, and determining the value of K according to the acquired vector length and further determined first bit width and second bit width.

6. A computer implementation method for performing a computer instruction to multiply a first matrix and a second matrix according to any one of claims 1 to 5, wherein the first bit width and the second bit width are the same bit width, equal to 4 bits, 8 bits, or 16 bits, depending on the type of multiplication to be performed obtained.

7. The value of K is determined according to the acquired vector length and the first and second bit widths determined. At runtime, establish a 16-bit operating bit width, During execution, the operation K is determined according to the acquired vector length and the established operation bit width, During execution, the value of K is determined according to the determined operation K and the determined first bit width and second bit width, A computer implementation method for performing a computer instruction to multiply a first matrix and a second matrix according to claim 5 or 6, including the above.

8. The value of K is determined according to the determined operation K and the determined first bit width and second bit width. A computer implementation method for performing a computer instruction to multiply a first matrix and a second matrix according to claim 7, comprising determining the value of K such that, at runtime, if the determined first bit width and second bit width are 16 bits, the value of K is equal to the determined operation K, or if the determined first bit width and second bit width are 8 bits, the value of K is equal to the determined operation K multiplied by 2, or if the determined first bit width and second bit width are 4 bits, the value of K is equal to the determined operation K multiplied by 4.

9. The processor supports a function that addresses or handles 8-bit or 16-bit bit packets instead of 4-bit bit packets in the vector register, and performs the multiplication of the first matrix and the second matrix. During execution, the first and second numeric elements to be processed for performing the multiplication via the addressing function are accessed in direct mode if the first and second bit widths are 8 bits or 16 bits, or indirect mode if the first and second bit widths are 4 bits. The direct mode includes using the addressing function to access each of the 8-bit or 16-bit numeric elements being processed without any post-processing related to their access, A computer implementation method for performing a computer instruction to multiply a first matrix and a second matrix according to any one of claims 6 to 8, wherein the indirect mode includes accessing an 8-bit bit packet containing each of the 4-bit numeric elements to be processed using the addressing function, and, together with the access-related post-processing of the accessed 8-bit bit packet, identifying the four most significant bits or the four least significant bits therein as the 4-bit numeric elements to be processed.

10. If the first bit width and the second bit width are 16 bits, the 16 bits are one of the following numerical representations, depending on the type of multiplication to be performed: - For both the first and second numerical elements, 16-bit floating-point numbers, FP16, - For both the first and second numerical elements, 16-bit Brain floating-point numbers, BF16, - For both the first and second numeric elements, a 16-bit signed integer, INT16, - For both the first and second numeric elements, a 16-bit unsigned integer, UINT16, - The first numeric element is a 16-bit signed integer, INT16, and the second numeric element is a 16-bit unsigned integer, UINT16. A computer implementation method that performs a computer instruction to multiply a first matrix and a second matrix according to any one of claims 6 to 9, corresponding to the above.

11. If the first bit width and the second bit width are 8 bits, then the 8 bits are one of the following numerical representations, depending on the type of multiplication to be performed, namely, - For both the first and second numeric elements, an 8-bit signed integer, INT8, - For both the first and second numeric elements, an 8-bit unsigned integer, UINT8, The first numeric element is an 8-bit signed integer, INT8, and the second numeric element is an 8-bit unsigned integer, UINT8. A computer implementation method that performs a computer instruction to multiply a first matrix and a second matrix according to any one of claims 6 to 10, corresponding to the above.

12. If the first bit width and the second bit width are 4 bits, then the 4 bits are one of the following numerical representations, depending on the type of multiplication to be performed, namely, - For both the first and second numeric elements, a 4-bit signed integer, INT4, - For both the first and second numeric elements, a 4-bit unsigned integer, UINT4, The first numeric element is a 4-bit signed integer, INT4, and the second numeric element is a 4-bit unsigned integer, UINT4. A computer implementation method for performing a computer instruction to multiply a first matrix and a second matrix according to any one of claims 6 to 11, corresponding to the above.

13. The first matrix is ​​a submatrix of a first complete matrix, which is composed of various submatrices including the first submatrix, and the second matrix is ​​a submatrix of a second complete matrix, which is composed of various submatrices including the second submatrix. Performing the multiplication of the first matrix and the second matrix, taking into account the first matrix parameter, the second matrix parameter, the third matrix parameter, the determined first bit width and second bit width, and the determined values ​​of M, N, and K, A computer implementation method for performing a computer instruction to multiply a first matrix and a second matrix according to any one of claims 1 to 12, further comprising accumulating the multiplication of the first submatrix and the second submatrix in the third matrix as a partial result of the multiplication of the first matrix and the second matrix, such that at runtime the multiplication of the first matrix and the second matrix can be performed by performing the method considering the various submatrixes that constitute the first matrix and the second matrix.

14. A computer program comprising computer program code means adapted to perform all steps of the method described in any one of claims 1 to 13 when executed on a computer system.

15. A matrix multiplication system for executing a computer instruction to multiply a first matrix and a second matrix in a processor having an architecture including a set or array of vector registers and a first control register, wherein each of the vector registers has a vector length indicated in the first control register, The first matrix is ​​composed of first numeric elements having a first bit width arranged in M ​​rows and K columns, the second matrix is ​​composed of second numeric elements having a second bit width arranged in K rows and N columns, and the third matrix obtained as a result of multiplying the first matrix and the second matrix is ​​composed of third numeric elements having a third bit width arranged in M ​​rows and N columns. The computer instruction encodes an operation code indicating the type of multiplication to be performed, and has the following parameters: a first matrix parameter indicating which vector registers the first numeric elements are sequentially stored in; a second matrix parameter indicating which vector registers the second numeric elements are sequentially stored in; and a third matrix parameter indicating which vector registers the third numeric elements are, or should be, sequentially stored in, and the matrix multiplication system is, A length module configured to obtain the vector length from the first control register at runtime, A decoding module configured to decode the computer instruction at runtime to obtain the operation code encoded in the computer instruction, and therefore the type of multiplication to be performed, A bit width module configured to determine the first bit width and the second bit width at runtime according to the type of multiplication to be performed obtained, A dimensional module configured to determine the values ​​of M, N, and K at runtime according to the acquired vector length and the determined first and second bit widths, A multiplication module configured to perform the multiplication of the first matrix and the second matrix at runtime, taking into account the first matrix parameter, the second matrix parameter, the third matrix parameter, the determined first bit width and second bit width, and the determined values ​​of M, N, and K, A matrix multiplication system equipped with the following features.

Images