Interruptible and restartable matrix multiply instructions, processors, methods, and systems

Abstract

This application discloses interruptible and restartable matrix multiplication instructions, a processor, a method, and a system. One aspect of the processor includes a decoding unit for decoding matrix multiplication instructions. The matrix multiplication instructions indicate a first memory location for a first source matrix, a second memory location for a second source matrix, and a third memory location for storing the result matrix. The processor also includes an execution unit coupled to the decoding unit. The execution unit is configured to, in response to the matrix multiplication instructions, perform partial multiplication of the first and second source matrices before an interrupt, and store a completion progress indicator in response to the interrupt. The completion progress indicator indicates the amount of progress made before the interrupt in multiplying the first and second source matrices and storing the corresponding result data in the third memory location.

Description

[0001] This invention patent application is a divisional application of the invention patent application with international application number PCT / US2017 / 036038, international application date of June 6, 2017, and Chinese national phase application number 201780034999.3, entitled "Interruptible and Restartable Matrix Multiplication Instructions, Processor, Method and System". Technical Field

[0002] The embodiments described herein generally relate to processors. Specifically, the embodiments described herein generally relate to matrix multiplication in a processor.

[0003] Background Information

[0004] Many processors employ a Single Instruction Multiple Data (SIMD) architecture. Such processors can have instruction sets that include various types of compact data instructions. Compact data instructions can be used to operate on multiple compact data elements, or multiple pairs of compact data elements, simultaneously and / or in parallel. Multiple data elements can be compacted as compact data within a register or memory location, where the bits of the register or memory location are logically divided into a sequence of data elements. The processor may have parallel execution hardware that responds to compact data instructions to operate on multiple compact data elements simultaneously and / or in parallel.

[0005] A specific example of such instructions is the compact data multiplication instruction. Another specific example is the compact data multiplication and accumulation instructions. These instructions can be used in a wide variety of algorithms, including matrix multiplication. Compared to scalar instructions that typically operate on only a single data element or a pair of data elements, these compact data or SIMD instructions generally tend to help improve the performance of the various algorithms in which they are used, through the SIMD data parallelism they provide. Attached Figure Description

[0006] The invention can be best understood by referring to the following description and the accompanying drawings, which illustrate various embodiments. In the drawings:

[0007] Figure 1 This is a block diagram of an embodiment of a computer system.

[0008] Figure 2 This is a flowchart of an embodiment of a method for executing matrix multiplication instructions.

[0009] Figure 3 This is a block diagram of a first example embodiment of a suitable set of operands for matrix multiplication instructions.

[0010] Figure 4This is a block diagram of a second example embodiment of a suitable set of operands for matrix multiplication instructions.

[0011] Figure 5 This is a block diagram of a detailed example embodiment of a matrix multiplication and accumulation execution unit that can be used to execute or implement matrix multiplication instructions.

[0012] Figure 6 This is a block diagram of an example embodiment of fragmentation.

[0013] Figure 7 This is a flowchart illustrating the fast path and slow path processing according to an embodiment.

[0014] Figure 8A -C is a block diagram of three different embodiment processors suitable for executing the matrix multiplication instructions disclosed herein.

[0015] Figure 9A This is a block diagram illustrating an embodiment of an ordered pipeline and an embodiment of an out-of-order release / execution pipeline with register renaming.

[0016] Figure 9B This is a block diagram of an embodiment of a processor core, which includes a front-end unit coupled to an execution engine unit, and both the front-end unit and the execution engine unit are coupled to a memory unit.

[0017] Figure 10A It is a block diagram of an embodiment of a single processor core, its connection to the on-die interconnect network, and a local subset of its Level 2 (L2) cache.

[0018] Figure 10B yes Figure 10A A block diagram of an embodiment of a processor core portion.

[0019] Figure 11 This is a block diagram of an embodiment of a processor that may have more than one core, may have an integrated memory controller, and may have an integrated graphics device.

[0020] Figure 12 This is a block diagram of a first embodiment of a computer architecture.

[0021] Figure 13 This is a block diagram of a second embodiment of a computer architecture.

[0022] Figure 14 This is a block diagram of a third embodiment of a computer architecture.

[0023] Figure 15 This is a block diagram of the fourth embodiment of the computer architecture.

[0024] Figure 16This is a block diagram illustrating the use of a software instruction converter to convert binary instructions in a source instruction set into binary instructions in a target instruction set, according to an embodiment of the present invention. Detailed Implementation

[0025] This document discloses embodiments of matrix multiplication instructions, embodiments of processors for executing instructions, embodiments of methods executed by a processor when instructions are executed, embodiments of systems including one or more processors for executing instructions, and embodiments of machine-readable media storing or otherwise providing instructions. In some embodiments, a processor may have: a decoding unit or other logic for receiving instructions and / or decoding the instructions, and an execution unit or other logic for executing the instructions. Numerous specific details (e.g., specific instruction operations, instruction parameters, data formats, methods of specifying matrices, processor configurations, microarchitectural details, operation sequences, etc.) are set forth in the following description. However, embodiments may be practiced without these specific details. In other instances, well-known circuits, structures, and techniques have not been shown in detail to avoid obscuring the understanding of this specification.

[0026] Figure 1 This is a block diagram of an embodiment of computer system 100. In various embodiments, the computer system may represent a desktop computer, laptop computer, notebook computer, workstation, server, or other computer system. The computer system includes a processor 102 and system memory 120. The processor and system memory may be coupled to each other or communicate with each other through conventional coupling mechanisms (e.g., via one or more buses, hubs, memory controllers, chipset components, etc.). The system memory may include one or more different memory devices and / or one or more different types of memory.

[0027] In some embodiments, processor 102 may be a general-purpose processor (e.g., a general-purpose microprocessor or central processing unit (CPU) of the type used in desktop computers, laptop computers, or other computers). Alternatively, the processor may be a dedicated-purpose processor. Examples of suitable dedicated-purpose processors include, but are not limited to, network processors, communication processors, cryptographic processors, graphics processors, coprocessors, and digital signal processors (DSPs). In some embodiments, the processor may be disposed on at least one integrated circuit or semiconductor die. In some embodiments, the processor may include at least some hardware (e.g., transistors, capacitors, diodes, circuits, non-volatile memory storing microcode, etc.).

[0028] During operation, processor 102 may receive matrix multiplication instruction 106. For example, the instruction may be fetched from system memory or otherwise received via a bus or other interconnect. The instruction may represent a macro instruction, machine code instruction, machine language instruction, or other instructions or control signals in the processor's instruction set. In some embodiments, the matrix multiplication instruction may be a matrix multiplication instruction that does not additionally perform matrix accumulation. In other embodiments, the matrix multiplication instruction may be a matrix multiplication instruction that also performs matrix accumulation using an accumulation matrix. Unless otherwise specified, the term matrix multiplication instruction is used broadly / generally herein to refer to any of these variations.

[0029] Instruction 106 may specify (e.g., explicitly specify) or otherwise indicate (e.g., implicitly indicate) a first storage location 122 where the first source matrix (A) 124 is stored, may specify or otherwise indicate a second storage location 126 where the second source matrix (B) 128 is stored, and may specify or otherwise indicate a third storage location 130 where the result matrix (C) 132 in response to the instruction will be stored. As shown, in some embodiments, each of the first, second, and third storage locations may optionally be in system memory 120.

[0030] The first, second, and third storage locations 122, 126, and 130 may be indicated in different ways in different embodiments. As an example, in some embodiments, instruction 106 may provide first and second source and result matrix indicators 114. For example, in the case where the storage location is in system memory, these indicators may represent memory pointers or other memory address information. Different types of memory address information are suitable for different types of addressing modes that can be used. Depending on the specific addressing mode, in some embodiments, such memory address information from indicator 114 may be combined with other memory address information (e.g., in a data segment register, extended segment register, or other register, or in the encoding of an instruction, or elsewhere) to obtain a memory address for accessing system memory.

[0031] As shown, in some embodiments, these indicators 114 may optionally be stored in a set 112 of the processor's registers (e.g., general-purpose registers or scalar registers). Each of the registers may represent an on-die (or integrated circuit) storage location operable to store data. Registers may represent architectural registers that are architecturally visible or visible to software and / or programmers, and / or registers indicated by instructions in the processor's instruction set to identify operands. These architectural registers contrast with other non-architectural registers in a given microarchitecture (e.g., temporary registers, reorder buffers, retirement registers, etc.). These registers may be implemented in different microarchitectures and in different ways, and they are not limited to any particular type of design. Examples of suitable types of registers include, but are not limited to, dedicated physical registers, dynamically allocated physical registers using register renaming, and combinations thereof.

[0032] In some cases, instruction 106 may optionally explicitly specify one or more of the registers 112 among the storage indicators 114. For example, the instruction may optionally have one or more source and / or destination operand specification fields (e.g., consecutive or non-consecutive bits in instruction encoding) each operable to specify one of the registers. As another example, one or more of the registers 112 among the storage indicators 114 may optionally be implicit with respect to the instruction (e.g., implicit with respect to the instruction's opcode). For example, when the processor recognizes the instruction (e.g., when it decodes the opcode), it may implicitly or inherently understand to view these implicit registers without the instruction having any non-opcode bits for explicitly specifying the registers. Alternatively, one or more of these indicators 114 may optionally be stored in another storage location.

[0033] In some embodiments, the instructions may optionally provide one or more matrix dimension indicators 116, but this is not mandatory. Matrix dimension indicators 116 may specify or otherwise indicate the dimensions (e.g., the number of rows, the number of columns, the dimensionality, or the order) associated with the first source matrix (A) 124, the second source matrix (B) 128, and, in some cases, the result matrix (C) 132. As a particular example, three different matrix dimension indicators may exist to specify or otherwise indicate three different dimensions associated with the first source matrix (A) and the second source matrix (B). As will be further explained below, the full dimensions of the first and second source matrices to be multiplied and the result matrix (C) can be fully specified by only three different dimensions (e.g., because in matrix multiplication, one dimension of the source matrices is the same, and the dimension of the result matrix (C) depends on the dimension of the source matrices). Advantageously, matrix dimension indicators can allow the instructions to operate on matrices of flexible and / or arbitrary sizes, the dimensions of which can be provided by the matrix dimension indicators. Various different sizes of matrices can be specified. Furthermore, the size of the matrix can range from relatively small to potentially extremely large, such as matrices that can be multiplied over time ranging from less than seconds to several hours, days, weeks, months or more when multiplied on general-purpose microprocessors of the type widely used in existing computers.

[0034] Alternatively, instead of supporting fully flexible and / or arbitrary-sized matrices, one or more sets of predetermined and / or fixed-sized matrices may be optionally supported. In this case, a single indicator 116 may be optionally used to select any one of multiple such sets of predetermined and / or fixed-sized matrices. As an example, two, three, four, eight, sixteen, or other numbers of different predetermined and / or fixed-sized matrices may be optionally supported. As another option, only a single set of predetermined and / or fixed-sized matrix dimensions may be optionally supported, and these may be optionally fixed or implicit for instructions (e.g., for their opcodes). In this case, the matrix dimension indicator 116 may be optionally omitted entirely. With such embodiments, the software algorithm may divide a large matrix into a set of smaller fixed-sized matrices for use with instructions, and the software algorithm may be responsible for merging the results together. Using this set of predetermined and / or fixed-sized matrices can help simplify implementation and / or allow for optimization of execution logic, but this may also tend to reduce the flexibility of instructions.

[0035] Refer again Figure 1The processor includes a decoding unit or decoder 108. The decoding unit can receive and decode matrix multiplication instructions. The decoding unit can output one or more relatively low-level instructions or control signals (e.g., one or more microinstructions, microoperations, microcode entry points, decoded instructions or control signals, etc.) that reflect or represent relatively high-level matrix multiplication instructions and / or derived from relatively high-level matrix multiplication instructions. In some embodiments, the decoding unit may include one or more input structures (e.g., ports, interconnects, interfaces) for receiving matrix multiplication instructions, instruction recognition and decoding logic coupled to the input structures for recognizing and decoding matrix multiplication instructions, and one or more output structures (e.g., ports, interconnects, interfaces) coupled to the instruction recognition and decoding logic for outputting lower-level instructions or control signals. Various different mechanisms can be used to implement the decoding unit, including but not limited to microcode read-only memory (ROM), lookup tables, hardware implementations, programmable logic arrays (PLAs), and other mechanisms suitable for implementing the decoding unit. In some embodiments, the decoding unit may be included on a die (e.g., on a die having execution unit 110). In some embodiments, the decoding unit may include at least some hardware (e.g., transistors, integrated circuits, or firmware on the die).

[0036] In some embodiments, as an alternative to directly providing matrix multiplication instructions to the decoding unit, an instruction emulator, transformer, modifier, interpreter, or other instruction conversion module may optionally be used. Various types of instruction conversion modules may be implemented in software, hardware, firmware, or a combination thereof. In some embodiments, the instruction conversion module may be located external to the processor, such as, for example, on a separate die and / or in memory (e.g., as a static, dynamic, or runtime emulation module). As an example, the instruction conversion module may receive matrix multiplication instructions, which may be instructions in a first instruction set, and the instruction conversion module may emulate, transform, modify, interpret, or otherwise convert the matrix multiplication instructions into one or more corresponding intermediate instructions or control signals, which may be in a second, different instruction set. These one or more intermediate instructions or control signals in the second instruction set may be provided to a decoding unit (e.g., decoding unit 108), which may decode them into one or more lower-level instructions or control signals executable by the processor's native hardware (e.g., one or more execution units).

[0037] Refer again Figure 1The execution unit 110 is coupled to the decoding unit 108 and the register 112. In some embodiments, the execution unit may be on a die or integrated circuit (e.g., on a die or integrated circuit having a decoding unit). The execution unit may receive instruction or control signals representing matrix multiplication instruction 106 and / or one or more decoded or otherwise converted instructions or control signals derived from the matrix multiplication instruction 106. During deployment and use, the execution unit may also be operable to couple to system memory 120. The execution unit may also receive a first source matrix (A) 124 and a second source matrix (B) 128 as input. In embodiments where the matrix multiplication instruction is also used to perform accumulation, the execution unit may also receive an accumulation matrix stored in a third storage location 130, where the result matrix (C) 132 will be stored. In this case, the third storage location can be used as a source / destination operand for the instruction, as it can be initially used as a source operand for accumulating the matrix and can subsequently be used as a destination operand for the result matrix (C) 132.

[0038] Execution unit 110 may be operable to store the result matrix (C) 132 in a third or destination storage location 130 indicated by the instruction in response to matrix multiplication instruction 106 and / or as a result of matrix multiplication instruction 106 (e.g., in response to one or more instructions or control signals decoded from the instruction and / or in response to the decoded instruction and / or in response to an instruction provided to the decoder). In some embodiments, if the matrix multiplication instruction does not perform accumulation, the result matrix (C) 132 may represent the matrix multiplication product of a first source matrix (A) 124 and a second source matrix (B) 128. Matrix multiplication involves multiplying two input matrices to produce another output matrix. In other embodiments, if the matrix multiplication instruction does not perform accumulation, an additional accumulation matrix may be added to the matrix multiplication product, and the result matrix (C) 132 may represent the multiplication product of a first source matrix (A) 124 and a second source matrix (B) 128 plus the accumulation matrix. Typically, the accumulator matrix can be a matrix that is initially stored in the third storage location 130 (where the result matrix (C) will be stored subsequently) just before the matrix multiplication instruction is executed, and when the matrix multiplication instruction is completed, the result matrix (C) can be stored on top of the accumulator matrix (and may become the accumulator matrix for another matrix multiplication instruction).

[0039] In some embodiments, different opcodes may be provided for matrix multiplication instructions with and without matrix accumulation. In other embodiments, the same opcode may be optionally used for both variants, and the instruction may have one or more bits or fields indicating whether the instruction will be performed with or without matrix accumulation. For example, a single bit may have a first value (e.g., binary zero) indicating that the instruction will be performed without matrix accumulation, or a second value (e.g., binary one) indicating that the instruction will be performed with matrix accumulation. The reverse binary convention is also suitable. At least conceptually, the bit may represent a coefficient that can be multiplied by the accumulation matrix to result in accumulation (e.g., in the case of bit one) or not to result in accumulation (e.g., in the case of bit zero), but in practice, such multiplication may not actually be performed.

[0040] As discussed above, in some embodiments, the sizes of the first source matrix (A) 124 and the second source matrix (B) 128 may optionally be flexible or arbitrary. Furthermore, in some embodiments, the first source matrix (A), the second source matrix (B), and the result matrix (C) may all potentially / optionally be large, or even extremely large. Primarily depending on the size of the matrices, the time required to complete an instruction when executed on a general-purpose microprocessor of the type widely used in existing computers can range from relatively short time intervals (e.g., approximately seconds to minutes) to long or even extremely long time intervals (e.g., approximately from several hours to months or more), but the scope of the invention is not limited to any particular amount of time.

[0041] Especially when the completion time is relatively long, errors, exceptions, interruptions, traps, or other abnormal conditions or interruptions may occur before the matrix multiplication instructions complete. As used herein, "abnormal condition" broadly refers to various types of errors, exceptions, traps, aborts, etc., that may stop or interrupt the execution of the matrix multiplication instructions. As used herein, "interruption" broadly refers to these types of abnormal conditions, as well as other possible types of events or conditions that may stop or otherwise interrupt the execution of the matrix multiplication instructions. As an example, in the event that an exception is detected (e.g., in response to a privilege violation, page fault, memory protection error, etc.), the execution of the matrix multiplication instructions can be stopped, and an automatically generated control flow transfer to a handler routine (e.g., an operating system module) can be made.

[0042] In some embodiments, to help account for the possibility of such exceptional conditions or other interruptions, the matrix multiplication instruction 106 may be operable to be interruptible and / or restartable. In some embodiments, if an interruption occurs before the matrix multiplication instruction completes and is committed, the matrix multiplication instruction may be operable to cause the execution unit 110 and / or processor 102 to store a completion progress indicator 118. The completion progress indicator may broadly represent any of a variety of different types of values ​​that can be used to indicate (e.g., to the execution unit and / or processor) how much progress has been made in the execution of the matrix multiplication instruction at or approximately at the time of the interruption and / or the level or amount of work done toward the completion of the matrix multiplication instruction. In different embodiments, this amount may be expressed in different ways. For example, the completion progress indicator may indicate the amount of progress already completed in multiplying the first and second source matrices as the amount of multiplications already performed and / or the amount of result data based on such multiplications that has been stored in memory. As an example, a completion progress indicator may represent any one or more of the following: a value that is incremented or otherwise updated as the computation is performed, an evaluation of an equation reflecting completion progress, an estimate of the level of progress or completion, or another value or data operable to indicate completion progress. When matrix multiplication is performed, the completion progress indicator may represent the destination operand of the matrix multiplication instruction that can be stored if the matrix multiplication stops before completion (e.g., due to an exception or other interruption). Subsequently, when the matrix multiplication instruction is resumed or restarted, the completion progress indicator may represent the source operand of the matrix multiplication instruction that can be read and used to restart or resume the computation. In some embodiments, computation may be resumed at a point based on and / or dependent on the completion progress indicator.

[0043] In one example embodiment, the completion progress indicator 118 can be initialized to a start value before the matrix multiplication instruction is executed. As the matrix multiplication calculation is executed, the completion progress indicator can be changed or adjusted (e.g., substantially continuously or at discrete points in time). If the matrix multiplication instruction is executed completely to completion, the completion progress indicator can have a final or stop value. Upon such successful completion, the matrix multiplication instruction can be allowed to retire or otherwise complete, and the instruction pointer can be allowed to advance to the next instruction to be executed. However, if an interruption occurs at any intermediate point between the start and stop values ​​of the completion progress indicator, the current intermediate value of the completion progress indicator can be stored (e.g., somewhere between the start and stop values). Later, when the matrix multiplication instruction is resumed, the completion progress indicator can be retrieved and used to restart them where the calculation stopped. In some cases, the start value can be zero, and the completion progress indicator can be incremented as the calculation is executed. In other cases, the start value can be a value indicating the progress to be made and / or the total amount of work to be performed, and the completion progress indicator can be decremented as the calculation is executed.

[0044] As shown, in some embodiments, the completion progress indicator 118 may optionally be stored in one of the registers 112. Matrix multiplication 106 may specify (e.g., explicitly specify by one or more bits or fields of an instruction) or otherwise indicate (e.g., implicitly indicate) such a register. Alternatively, the completion progress indicator may optionally be stored in system memory or in another storage location.

[0045] In some embodiments, if such an interruption occurs, intermediate or incomplete computation results may also be stored. Typically, such results may be stored in system memory 120 (e.g., in result matrix (C) 132). When instructions are resumed later, these intermediate or incomplete computation results may be received and used to restart them where the computation stopped.

[0046] Execution unit 110 and / or processor 102 may include specific or particular logic (e.g., transistors, integrated circuits, or other hardware and / or firmware (e.g., instructions stored in non-volatile memory) and / or software operable to execute matrix multiplication instructions and / or store results in response to matrix multiplication instructions and / or as their results (e.g., in response to one or more instructions or control signals decoded from the matrix multiplication instructions). In some embodiments, the execution unit may include at least some hardware and may include hardware and / or firmware that is primarily potentially combined with some software. In some embodiments, the execution unit may include a fused multiplier-adder circuit of a two-dimensional array. Various different relative amounts of circuitry or other hardware and / or microcode or firmware may be used (e.g., depending on specific cost and performance objectives of a particular implementation). For example, relatively more hardware may be used to help provide a relatively higher performance approach, or relatively more firmware may be used to help provide a relatively lower cost approach.

[0047] Advantageously, matrix multiplication instructions can help provide a relatively high level of arithmetic processing within the scope of a single instruction's execution. Even compared to current wide SIMD instructions, matrix multiplication instructions can typically provide significantly more arithmetic operations. This can help distribute the associated energy consumption of fetching and decoding instructions, as well as retrieving and storing data, across a significantly larger number of arithmetic operations. This, in turn, can help reduce the energy consumption per arithmetic processing unit. Furthermore, in some embodiments, matrix multiplication instructions can be interruptible and / or restartable. For example, when an instruction is stopped (e.g., due to an exception or other abnormal condition or other interruption), a completion progress indicator 118 and intermediate or incomplete calculation results can be stored. This can help ensure continued forward progress in the face of possible interruptions and can be particularly advantageous for matrix multiplication instructions with long or extremely long completion times.

[0048] To avoid obscuring the description, a relatively simple processor 102 has been shown and described. However, the processor may optionally include other processor components. For example, various different embodiments may include various different combinations and configurations of the components shown and described in any of the figures in Figures 9-11. All components of the processor may be coupled together to allow them to operate as intended. As an example, consider Figure 9B Instruction cache 934 can cache instructions, instruction fetch unit 938 can fetch instructions, decoding unit 940 can decode instructions, scheduler unit 956 can schedule related operations, execution unit 962 can execute instructions, decommissioning unit 954 can decommission instructions, and so on.

[0049] Figure 2This is a flowchart illustrating an embodiment of method 233 for executing matrix multiplication instructions. In various embodiments, the method may be executed by a processor, instruction processing apparatus, digital logic device, or integrated circuit. In some embodiments, method 233 may be performed by… Figure 1 The processor 102 and / or utilizing Figure 1 The processor 102 and / or using Figure 1 The matrix multiplication instruction 106 is used for execution. The components, features, and specific optional details described herein with respect to processor 102 and / or matrix multiplication instruction 106 are also optionally applied to method 233. Alternatively, method 233 may be executed by similar or different processors or devices and / or within similar or different processors or devices and / or using similar or different matrix multiplication instructions. Furthermore, processor 102 may execute methods similar to or different from method 233.

[0050] The method includes receiving a matrix multiplication instruction at block 234. In various aspects, the instruction may be received at a processor or a portion of a processor (e.g., an instruction fetch unit, a decoding unit, a bus interface unit, etc.). In various aspects, the instruction may be received from a source outside the processor and / or outside the die (e.g., from memory, interconnects, etc.) or from a source on the processor and / or on the die (e.g., from an instruction cache, an instruction queue, etc.). In some embodiments, the matrix multiplication instruction may specify or otherwise indicate a first memory location of a first source matrix, a second memory location of a second source matrix, and a third memory location where the result matrix will be stored.

[0051] At box 235, a determination can be made as to whether an exception, page fault, other exceptional condition, or other interrupt has occurred. For example, if a portion of the memory operand being operated on is inaccessible, such as in the event of a page fault, the instruction may be interrupted. If no such interruption occurs (i.e., it is determined to be "no"), the method can proceed to box 236.

[0052] At box 236, portions of the first and second source matrices can be multiplied. At box 237, a portion of the resulting data can be stored in a third memory location. Without accumulation, this represents the product of multiplying portions of the first and second source matrices. With accumulation, this portion of the resulting data represents the portion of the product to which accumulated data is added.

[0053] At box 238, a determination can be made as to whether more data will be processed. If more data exists that will be processed (i.e., determined to be "yes"), the method can revisit box 235. More data can be retrieved, assuming no page faults or other interruptions, and the additional data can be processed.

[0054] Alternatively, if it is determined at box 238 that there is no more data to process (i.e., it is determined to be "no"), the method can proceed to optional box 239. At optional box 239, a completion progress indicator indicating the complete completion of the matrix multiplication instruction can optionally be stored (e.g., in a register or other architecture-visible storage location).

[0055] Referring again to box 235, if at some point it is determined that an interrupt has occurred (i.e., determined to be "yes"), the method can proceed to box 240. At box 240, a completion progress indicator indicating incomplete progress can be stored (e.g., in a register or other architecture-visible storage location). The completion progress indicator can be similar to or the same as those described elsewhere herein. For example, the completion progress indicator could indicate the amount of progress made prior to the interrupt in multiplying the first and second source matrices and storing the corresponding result data in a third memory location.

[0056] The illustrated method involves architectural operations (e.g., those operations visible from a software perspective). In other embodiments, the method may optionally include one or more microarchitectural operations. As examples, an instruction can be fetched and decoded, a source matrix can be accessed, an execution unit can perform multiple microarchitectural operations to implement the instruction, and so on. In some embodiments, the method may also optionally include decomposing matrix operands into relatively small portions (e.g., slices or blocks). In some embodiments, the method may also optionally include managing relatively small portions (e.g., slices or blocks) within one or more caches. In some embodiments, the method may also optionally include prefetching source matrix data. In some embodiments, the method may also optionally include performing a relatively “slower” path for partial slices and / or after an interruption involving multiplications of individual data elements—AND-AND—and conversely, performing a relatively “faster” path for complete slices and / or when no interruption is detected for complete slices.

[0057] Figure 3 This is a block diagram of a first example embodiment of a suitable set of operands 342 for a matrix multiplication instruction (e.g., matrix multiplication instruction 106). The operands include a set of source and destination matrix indicators 314. In the illustrated embodiment, these indicators include memory address information 344 for a first source matrix (A), memory address information 345 for a second source matrix (B), and memory address information 346 for a destination matrix (C), which in some embodiments may be a source / destination matrix (e.g., initially used as an accumulation matrix).

[0058] The operands also include matrix dimension indicators 316. In the illustrated embodiment, these indicators include the number of rows 347 of the first source matrix (A), the number of columns 348 of the first source matrix (A), and the number of columns 349 of the second source matrix (B). In other embodiments, other combinations of dimensions may optionally be used to specify the dimensions of the matrix, as described elsewhere herein. Advantageously, including these matrix dimension indicators allows matrix multiplication instructions to be used to process matrices of various sizes and / or arbitrary sizes.

[0059] The operands also include a completion progress indicator 318. The completion progress indicator can be used as a destination operand for matrix multiplication instructions that have been interrupted and / or as a source operand for matrix multiplication instructions that have been resumed or restarted.

[0060] Figure 4 This is a block diagram of a second example embodiment of a suitable set of operands 442 for a matrix multiplication instruction (e.g., matrix multiplication instruction 106). The operands include a set of source and destination matrix indicators 414. In the illustrated embodiment, these indicators include memory address information 444 for a first source matrix (A), memory address information 445 for a second source matrix (B), and memory address information 446 for a destination matrix (C), which in some embodiments may be a source / destination matrix.

[0061] The operand also includes a matrix dimension indicator 416. In this example embodiment, the matrix dimension indicator 416 includes a set of multiplication matrix dimension indicators 450 and a set of memory layout dimension indicators 452. The multiplication matrix dimension indicator 450 includes the number of rows 447 of the first source matrix (A), the number of columns 448 of the first source matrix (A), and the number of columns 449 of the second source matrix (B). In other embodiments, other combinations of dimensions may be optionally used to specify the dimensions of the matrix, as described elsewhere herein.

[0062] The operands also include memory layout dimension indicators 452. These indicators can be used to indicate the dimensions of potentially / optionally larger matrices containing the matrices to be multiplied (e.g., matrices whose dimensions are indicated according to multiplication matrix dimension indicator 450), as they are arranged or stored in memory. As an example, multiplication matrix dimension indicator 450 may correspond only to slices, blocks, or other portions of the larger matrix that corresponds to memory layout dimension indicator 452. As another example, the larger matrix corresponding to memory layout dimension indicator 452 may include padding (e.g., zero padding), such as to help achieve alignment with cache line boundaries, etc. Combinations of these are also possible. Furthermore, in different embodiments, memory layout dimension indicators may be in column-major or row-major format. For example, in column-major format, the memory layout dimension may include the distance between columns of an optional / potentially larger matrix having a first source matrix (A) (e.g., in 8-bit bytes or 16-bit words) 453, the distance between columns of an optional / potentially larger matrix having a second source matrix (B) 454, and the distance between columns of an optional / potentially larger matrix having a destination matrix (C) 455. Alternatively, in row-major format, the memory layout dimension may include the distance between rows of an optional / potentially larger matrix having a first source matrix (A), the distance between rows of an optional / potentially larger matrix having a second source matrix (B), and the distance between rows of an optional / potentially larger matrix having a destination matrix (C).

[0063] The operands also include a completion progress indicator. The completion progress indicator can be used as the destination for matrix multiplication instructions that are interrupted in progress, and as the source operand for matrix multiplication instructions when the matrix multiplication instructions are restarted.

[0064] In different embodiments, operands 342 and 442 may be provided in different ways. As an example, each of these operands may optionally be stored in a different register (e.g., a 32-bit or 64-bit general-purpose register) specified or otherwise indicated by the matrix multiplication instruction. Alternatively, a memory location or other storage location may optionally be used. As another option, a matrix dimension indicator 316 and / or a multiplication matrix dimension indicator 450 and / or a memory layout dimension indicator 452 may optionally be provided within the encoding of the matrix multiplication instruction (e.g., in an immediate value). As a specific illustrative example, a 32-bit immediate value may optionally be used, and bits [9:0] may be used to specify a first dimension, bits [20:10] may be used to specify a second dimension, and bits [31:21] may be used to specify a third dimension.

[0065] Figure 5 This is a block diagram of a detailed example embodiment of a matrix multiplication and accumulation execution unit 510 that can be used to execute or implement matrix multiplication instructions (e.g., matrix multiplication instruction 106). During operation, when deployed in a system, the execution unit can be operable to be coupled to memory 520. The memory may have a first source matrix (A) 524, a second source matrix (B) 528, and a source and destination (SRC / DST) accumulation matrix (C) 532 stored therein. The source and destination accumulation matrix (C) can be initially used as a source operand from which a source accumulation matrix can be read, and can then subsequently used as a destination operand from which a result accumulation matrix can be stored (e.g., the result accumulation matrix can overwrite the source accumulation matrix initially stored in the same storage location).

[0066] Each of the matrices has multiple rows (in the vertical direction shown) and multiple columns (in the horizontal direction shown). The number of rows or columns can also be referred to by other names in the art, such as, for example, the dimension, size, or order of the matrix. Specifically, the first source matrix (A) has a number of rows (rowsA) and a number of columns (colsA). Similarly, the second source matrix (B) has a number of rows (rowsB) and a number of columns (colsB). In matrix multiplication, colsA and rowsB represent the common, identical, or equal dimensions of the two matrices. The source and destination accumulation matrix (C) has the same number of rows (rowsC) as the first source matrix (rowsA) and the same number of columns (colsC) as the second source matrix (colsB). That is, the number of rows and columns of the source and destination accumulation matrix (C) can be derived from the dimensions of the first and second source matrices. Due to these dependencies, various different combinations of dimensions can be used to specify all the required dimensions of these matrices, and the matrix multiplication instructions disclosed herein can utilize any sufficient combination.

[0067] Execution unit 510 may be operable to receive source and result matrix indicators 514. These indicators may be similar to or the same as the previously described indicator 114. As an example, the indicators may include memory address information for identifying the memory locations of the stored source and destination matrices in memory.

[0068] Execution unit 514 is also operable to receive multiplication matrix dimension indicator 450. As shown in the illustrated embodiment, the multiplication matrix dimension indicator may include three different indicators for three different dimensions sufficient to specify all dimensions of the three matrices. In the illustrated example, these dimensions include the number of rows 547 of the first source matrix (A), the number of columns 548 of the first source matrix (A) (which is the same as the number of rows of the second source matrix (B)), and the number of columns 549 of the second source matrix (B). It will be understood that other possible ways of indicating the same information exist, such as, for example, by indicating the dimensions of the source and destination accumulation matrix (C), from which some dimensions of the source matrix can be derived.

[0069] The execution unit is also operable to receive memory layout dimension indicators 452, which have potentially / optionally larger matrices A, B, and C, respectively. These dimensions can be expressed in column-major or row-major format, as previously described.

[0070] As shown, in some embodiments, the execution unit may optionally include fragmentation logic 556. In some embodiments, in response to a matrix multiplication instruction, fragmentation logic 556 and / or execution unit 510 and / or a processor having the execution unit may be operable to perform fragmentation. Fragmentation may broadly refer to dividing, splitting, or otherwise decomposing a relatively large matrix into a plurality of non-overlapping smaller matrices referred to as slices or blocks. As an example, in some embodiments, in response to a matrix multiplication instruction, fragmentation logic and / or execution unit and / or processor may be operable to fragment relatively large source matrices A, B, and C (e.g., dimensions indicated according to multiplication matrix dimension indicator 450) into relatively smaller slices of at least one size.

[0071] If most slices have dimensions that are powers of 2, it can tend to improve efficiency. Slices can optionally be square, but this is not mandatory. For example, a relatively large source matrix can be partitioned along its largest dimension, where one dimension is a power of 2. Typically, the outer edges of a relatively large matrix (and / or relatively large slices), i.e., those portions that are last sliced ​​(or further sub-sliced), may tend to have slices where dimensions are not always powers of 2 and / or dimensions may be rectangular rather than square. This slice, executed in response to a matrix multiplication instruction, can be higher than and exceed any optional / potential slices executed by the software algorithm outside the scope of the matrix multiplication instruction's execution.

[0072] In some embodiments, sharding may optionally be performed to divide a relatively large source matrix into at least two different sizes of shards, but this is not mandatory. For example, relatively large source matrices A, B, and C may first be decomposed into relatively large shards, and then the relatively large shards may be decomposed into relatively smaller shards, and this process may optionally be repeated for one or more or even smaller sizes. In some embodiments, there may be two, three, four, five, or even more shards of different levels or sizes. As an example, different sizes of shards may be selected in part based on the different storage capacities of the on-die storage structures (e.g., registers, caches, scratch memory, dedicated buffers, etc.) used to store the shards, such that the shards are appropriately adapted to these different structures. As an example, shard data may be copied from a higher level in the memory hierarchy, the shard data may then be manipulated, and the results may then be stored back to a higher level in the memory hierarchy, and this may be performed on each level in the memory hierarchy.

[0073] To further illustrate certain concepts, Figure 6 This is a block diagram of an example embodiment of sharding. Execution unit 610 can receive matrix multiplication instruction 606. The execution unit includes sharding logic 656 and fused multiplication and addition logic 658. Matrix 624 can be stored in memory 620. In response to the matrix multiplication instruction, the sharding logic can shard the matrix to divide it into four shards (in this illustrative example), which are labeled shards "1", "2", "3", and "4". Shard "1" can be accessed and stored in level 2 (L2) cache 670. The size of shard "1" can optionally be selected to fit the size of the L2 cache (e.g., including any double or triple buffering, as will be discussed further below).

[0074] In response to a matrix multiplication instruction, the sharding logic can further shard slice "1" to divide it into four additional or even smaller slices (in this illustrative example), which are labeled slices "1.1", "1.2", "1.3", and "1.4". Slice "1.1" can be stored in the Level 1 (L1) cache 668. The size of slice "1.1" can be optionally selected to fit the size of the L1 cache (e.g., including any double or triple buffering, as will be discussed further below).

[0075] In response to a matrix multiplication instruction, the sharding logic can perform further sharding on shard "1.1" to divide shard "1.1" into four additional or even smaller shards (in this illustrative example), which are labeled shards "1.1.1", "1.1.2", "1.1.3", and "1.1.4". These smaller shards can be processed by fused matrix multiplication and addition logic 658, which may have an array of fused multipliers and adders to handle shards of this size. As shown, shard "1.1.1" can be provided to the fused matrix multiplication and addition logic. It will be understood that this is only an illustrative example. In other embodiments, matrices and / or shards can optionally be divided into fewer or more shards. Furthermore, fewer or more shards of different levels and sizes can optionally be used.

[0076] In some embodiments, slices may optionally be double-buffered or triple-buffered within a cache. Double-buffering and triple-buffering may refer to having two or three copies of a slice, respectively. For example, in some embodiments, slices for matrices A and B may optionally be double-buffered, while slices for matrix C may optionally be double-buffered or triple-buffered. One copy may be used for the source data to be multiplied (and in some cases, accumulated), while another copy may be used to collect the arithmetic results.

[0077] In this example, a cache has already been used to store the slices. In other embodiments, alternatively, separate or dedicated storage locations may be used. For example, one or more levels of scratch memory may be optionally used to store these slices. This might be the case, as an example, when instructions are executed by a dedicated matrix multiplication accelerator that may not have these existing cache levels.

[0078] Refer again Figure 5 The slices to be arithmetically processed by the execution unit can be provided to the fused matrix multiplication and addition logic 558. As an example, the fused matrix multiplication and addition logic may include a fused multiplier and adder array of two-dimensional arrays. As an example, this may include 64 to 128 double-precision floating-point fused multiplier-adders per clock cycle per CPU core, or 128 to 256 single-precision fused multiplier-adders. The fused matrix multiplication and addition logic can perform matrix multiplication, and in this embodiment, it can also perform matrix accumulation. Alternatively, matrix accumulation may be optionally omitted or disabled (e.g., by not performing addition, by multiplying the accumulated matrix by zero coefficients, etc.).

[0079] To further illustrate certain concepts, the following pseudocode can be used to represent a simple illustrative example of how logic 560, which combines matrix multiplication and addition logic 558, can optionally be implemented to perform an algorithm for matrix multiplication with matrix accumulation:

[0080]

[0081] The algorithm consists of three nested loops. Specifically, an outer loop with loop counter "i" is used for all rows of the first source matrix (A) (i.e., "rowsA"), a middle loop with loop counter "j" is used for all columns of the second source matrix (B) (i.e., "colsB"), and a third innermost loop with loop counter "k" is used for the common dimension ("comm"). Multiplication and addition are performed within all these nested loops.

[0082] It will be understood that this is merely an illustrative example of a suitable algorithm. Other algorithms may optionally add additional loops (e.g., for slices, for slices of different sizes across multiple levels, etc.). Furthermore, the order in which data is accessed may optionally differ from that shown in this algorithm. This can be partly due to the specific way slices are implemented. Generally, it may be appropriate not to change the order of the innermost "k" loops along the common dimension, as changing their order may tend to slightly modify the final result value, partly due to the ordering dependency on floating-point rounding. However, for implementations where such rounding fluctuations are acceptable, the order of the innermost loops may optionally be changed if necessary.

[0083] If the matrix multiplication instruction completes successfully, the execution unit may be operable to store the resulting matrix in the source / destination accumulation matrix (C) 532. However, if an exception, page fault, or other such exceptional condition or interruption occurs before completion, a completion progress indicator (CPI) 518 may be stored. The execution unit and / or completion progress indicator calculation logic 562 may be operable to store the completion progress indicator. For example, it may optionally be stored in a general-purpose register 512, in memory, or in another suitable storage location. The completion progress indicator may optionally be similar to or the same as those described above.

[0084] As a specific illustrative example, for the algorithm with three nested loops shown above, the completion progress indicator logic 562 may include logic 564 for calculating the completion progress indicator (CPI) based on and / or satisfying the following Equation 1:

[0085] CPI = i*colsB*comm + j*comm + k Equation 1

[0086] In this equation, "i" represents the current loop counter used for rows of matrix A at the time of interruption, "j" represents the current loop counter used for columns (colsB) of matrix B at the time of interruption, and "k" represents the current loop counter used for the common dimension (comm) at the time of interruption. Different sizes of sequence numbers can be used in different embodiments. For example, sequence numbers can be expressed as 32-bit, 64-bit, or 128-bit values, to name just a few. Generally, 32-bit sequence numbers tend to be suitable for medium to large-sized matrices, while 64-bit sequences can be used for very large matrices (e.g., matrix multiplication with two weeks), and 128-bit sequence numbers can be used for extremely large matrices.

[0087] After a restart, the calculations can be resumed at the point where they stopped due to the interruption. A completion progress indicator can be used for this purpose. For example, the loop counter values ​​can be restored to their values ​​at the time of the interruption by using a completion progress indicator. As an example, for a completion progress indicator calculated according to Equation 1, the loop counter values ​​i, j, and k can have values ​​that conform to the following Equations 2-4:

[0088] i = CPI / comm / colsB Equation 2

[0089] j = (CPI / comm)%colsB Equation 3

[0090] k = CPI%comm%colsB Equation 4

[0091] In these equations, "%" represents the modulo or remainder operator that produces an integer. Another suitable example of a completion progress indicator is the concatenation of different loop counter values. Furthermore, they can optionally be stored separately instead of being concatenated, but can logically represent yet another example of a completion progress indicator.

[0092] After the matrix multiplication instruction is successfully completed, the completion progress indicator can have a value that conforms to the value calculated according to the following equation 5:

[0093] CPI = rowsA * comm * colsB (Equation 2)

[0094] This is merely an illustrative example of a suitable completion progress indicator and an illustrative way to restart computation using that indicator. In other embodiments, more than three nested loops may optionally be used. Furthermore, these loops may not proceed sequentially. In some embodiments, execution unit 510 and / or a processor including execution unit 510 may be allowed to read data portions of matrices A, B, and (for accumulation) C in an order different from the specific order shown in the pseudocode above. Furthermore, in some embodiments, data portions of matrix C may optionally be written in an order different from that shown in the pseudocode.

[0095] Correspondingly, execution units and / or processors may be allowed to generate and store completion progress indicators based on and conforming to equations or methods different from the specific illustrative example shown above. The final result matrix may be essentially defined architecturally for the instructions (e.g., minor variations due to order-dependent floating-point rounding may be allowed). However, the intermediate order of processing matrix data, the corresponding method of calculating completion progress indicators, and the values ​​of the completion progress indicators may not be defined architecturally. Instead, these aspects may be allowed to be flexible. This allows the specific order and manner of processing matrix data (e.g., the specific way of accessing data, how to implement sharding, etc.) to vary from one implementation to another and / or be customized for different design goals. In some embodiments, intermediate values ​​of completion progress indicators may optionally be meaningless to the software. In some embodiments, the software may not know how to interpret or use these completion progress indicators to resume matrix multiplication after an interruption. Instead, the execution unit and / or processor may be responsible for using such intermediate values ​​of completion progress indicators.

[0096] Similarly, slices or intermediate data stored within the processor (e.g., registers, caches, temporary memory, etc.) prior to instruction completion may optionally not be architecturally defined and / or understandable by software. Optionally, such slices or intermediate data may not be saved and restored during context switching and / or after an interruption of a matrix multiplication instruction. Furthermore, in some embodiments, such data may optionally not be monitored (e.g., by other cache coherence agents, other cache proxies, other hardware threads or cores, etc.). In some embodiments, the loading and storing of matrix data by matrix multiplication instructions may be ordered only by reference to the preceding and subsequent instructions. Even though matrix multiplication instructions may be able to read source operands and write destination operands in an order different from, for example, the three nested loops shown in the pseudocode above, normal memory ordering rules should generally apply to matrix multiplication instructions (e.g., in general), the preceding instructions in program order, and the subsequent instructions in program order.

[0097] In some embodiments, matrix multiplication instructions as disclosed herein may optionally support two or more different slice algorithms, mechanisms, or methods. For example, these different slice methods may differ in the number of slices of different sizes, different slice sizes, or other aspects associated with slice, or combinations thereof. As an example, the initial execution of the matrix multiplication instruction may use a relatively simpler slice algorithm, and later the slice algorithm may evolve or change over time (e.g., to include more complex features, more levels of slices, etc.). As a result, the slice aspects associated with the execution of the matrix multiplication instruction may change over time or with processor generations. In some embodiments, in addition to storing a completion progress indicator, the execution unit may also be operable to store an indication of the slice algorithm used when an interrupt occurs. For example, a single bit may be used to distinguish between two different slice algorithms, or two or more bits may be used to allow for a greater variety of possible slice algorithms to be used in the future. This identifier may be stored if the instruction is interrupted. In some cases, it may be stored together with the completion progress indicator, such as in a different bit field, or stored separately from the completion progress indicator (e.g., in another register). Furthermore, in some embodiments, one or more additional bits may optionally be added, each for different implementation aspects. As an example, values ​​assuming four to eight bits can be used to specify multiple different possible implementations in the sharding algorithm and / or other ways. This can allow for the use of different implementations, including indications for the implementation to change over time and for the execution unit to use a previous implementation when resuming after an interruption. For example, if two implementations are possible, the execution unit can store an indication of which implementation is used, such that the same implementation can be reused after resuming interrupted instructions.

[0098] In some embodiments, the fused matrix multiplication and addition logic 558 and / or execution unit 510 can be designed, customized, or optimized to be relatively faster and / or more efficient for matrix data of a certain size or size range (e.g., a certain slice size or range). As an example, a first implementation can be designed, customized, or optimized for slices of one size (e.g., a 16-row matrix A, a 16-column matrix A, and a 16-column matrix B), for example, with approximately 32 to 64 fused multiply-add (FMA) clocks. A second implementation can be designed, customized, or optimized for slices of another size (e.g., a 32-row matrix A, a 32-column matrix A, and a 32-column matrix B), for example, with approximately 256 to 512 FMA clocks. The third implementation can be designed, customized, or optimized for use with another size of chip (e.g., a 64-row matrix A, a 64-column matrix A, and a 64-column matrix B), and the fourth implementation can be used for chips with yet another different size (e.g., a 128-row matrix A, a 128-column matrix A, and a 128-column matrix B), for example, with approximately 16,000 to 32,000 FMA clocks.

[0099] In some embodiments, instructions may implicitly perform arithmetic operations on slices or data of fixed size, but this is not mandatory. In other embodiments, instructions may implicitly perform arithmetic operations on slices or data of flexible size, but this is not mandatory. In still other embodiments, both forms may be optionally supported. One possible advantage of using fixed size (when appropriate) is that it can help improve performance. For example, in the case of fixed size, the fixed size can be known at instruction fetch and / or decoding, rather than only later (e.g., during execution). By knowing the fixed size approximately at decoding time, an appropriate number of micro-operations or other operations can be introduced into the pipeline to help reduce the number of bubbles or empty slots in the execution pipeline. This can help improve performance. In contrast, if the size is only known later, it may be too late to avoid such bubbles or empty slots in the execution pipeline. In some embodiments, the fixed-size form may optionally use immediate values ​​to specify a multiplication matrix dimension indicator (e.g., multiplication matrix dimension indicator 450). Instructions of this form can be used when the compiler is aware or knows that such a fixed size will be used, and these multiplication matrix dimension indicators can be passed via the immediate value of the instruction. In other embodiments, when the size is not fixed, or if it is unknown whether the size is fixed, a flexible size form can be used, and the multiplication matrix dimension indicators can be specified in a register (e.g., a general-purpose register). Such performance improvements typically tend to be significant only for relatively small-sized matrices, because for larger-sized matrices, the inefficiency of the initial bubbles is quickly eliminated and only represents a small portion of all computations.

[0100] In some embodiments, matrix multiplication instructions, when executed, may also be operable to cause the execution unit and / or processor to prefetch matrix data. For example, matrix data may be prefetched from a more distant level of the memory hierarchy to a more recent level (e.g., near the processor core). Such data prefetching operations may overlap with or occur concurrently with arithmetic operations used for instructions. This can be useful in helping to ensure that the execution unit has sufficient data to process. In some embodiments, such prefetching operations may optionally be implemented as prefetch hints that are not architecturally guaranteed to complete and that the processor is free to ignore or disregard (e.g., if it deems it should do something else instead). In some embodiments, an earlier matrix multiplication instruction in the program sequence may be operable to cause matrix data prefetching to prefetch data that will be used by subsequent matrix multiplication instructions.

[0101] Some processors may already have existing data prefetch units. In some embodiments, such existing data prefetch units may optionally be reused for matrix multiplication instructions. However, existing data prefetch units may be adapted to be aware that they are fetching matrix data and operable to perform prefetching accordingly in a manner suitable for such matrix data. For example, this may include prefetching based on two-dimensional data considerations instead of only one-dimensional data considerations, prefetching complete slices or other discrete portions of matrix data, prefetching along slice boundaries, and so on. In other embodiments, additional or dedicated data prefetch units or logic may be optionally included and may be primarily dedicated to the execution of matrix multiplication instructions. Such dedicated data prefetch units or logic may also be aware that they are fetching matrix data and operable to perform prefetching accordingly in a manner suitable for such matrix data.

[0102] When executed, matrix multiplication may repeatedly access data from memory in some cases. When accessing successive portions of a matrix from memory, page boundaries may be crossed. Page faults may or may not be present. Typically, if no page fault is present, a complete slice of all source operands can be available. However, if a page fault is present, only a partial slice of one of the source operands can be available. A partial slice can also exist when instruction execution resumes at an intermediate point where a portion of the slice has already been processed and / or along the periphery of a large matrix.

[0103] Figure 7 This is a flowchart of an example embodiment of method 780 for performing matrix multiplication in a relatively fast or relatively slow mode. The terms "faster" and "slower" are relative terms (i.e., relative to each other) rather than absolute terms. The method includes attempting to access a complete slice of a source operand from memory at block 781. At block 782, a determination can be made as to whether only a portion of the slice is available for at least one of the source operands.

[0104] If the complete slice is available for all source operands (e.g., determined to be "no"), then at box 783, multiplication can be performed in a relatively faster mode, involving relatively more concurrent multiplications per unit of time. As an example, vectors or arrays of data elements can be multiplied simultaneously. Typically, the faster mode is the common scenario, except in cases of page faults, interrupts, and slices along the outer perimeter of large matrices.

[0105] Alternatively, if only a partial slice is available (e.g., determined to be "yes"), the multiplication can be performed in a relatively slower mode at box 784, involving relatively fewer concurrent multiplications per unit of time. In some cases, this may involve performing individual element-by-element multiplications, or at least fewer multiplications simultaneously than for the faster mode. Once the partial slice has been completed, execution can usually be eagerly resumed to the faster mode.

[0106] In some embodiments, assuming a complete slice is available and faster mode is executed, the execution unit may initiate matrix multiplication instructions and switch to slower mode if a partial slice is detected (e.g., by inducing a micro-anomaly). In some embodiments, assuming a completion progress indicator indicates that the amount of prior progress is zero and faster mode is executed, the execution unit may initiate matrix multiplication instructions and switch to slower mode (e.g., by inducing a micro-anomaly) when the completion progress indicator indicates that the amount of prior progress is not zero.

[0107] Figure 8A -C is a block diagram of three different embodiment processors suitable for executing the matrix multiplication instructions disclosed herein. Figure 8A A first processor 802A is shown in this example, having four isomorphic cores 890A-1 to 890A-4. As an example, each of these cores may have substantially the same architecture, architectural features, and execution resources. In some embodiments, each of the four cores may each include logic 891A-1 to 891A-4 for performing embodiments of matrix multiplication disclosed herein.

[0108] Figure 8B A second processor 802B with three heterogeneous cores is shown in this example. Specifically, there is an ordered SIMD core 893, an ordered scalar core 894, and an out-of-order (OOO) core 895. These cores can generally be heterogeneous in at least some architectural and microarchitectural aspects. For example, only the OOO core may have OOO logic, or at least more than OOO logic. As another example, the SIMD core may have SIMD execution resources, but the scalar core may not, and so on. In some embodiments, one of the cores, such as the SIMD core 893, may include 891B for performing embodiments of matrix multiplication disclosed herein. The OOO core may alternatively or additionally have such logic.

[0109] Figure 8C A third processor 802C with two isomorphic cores 890C-1 to 890C-2 is shown in this example. As an example, each of these cores may have substantially the same architecture, architectural features, and execution resources. The third processor also has a matrix multiplication accelerator 896. In some embodiments, the matrix multiplication accelerator may include logic for performing embodiments of matrix multiplication disclosed herein.

[0110] Exemplary core architectures, processors, and computer architectures

[0111] Processor cores can be implemented in different ways, for different purposes, and in different processors. For example, implementations of such cores may include: 1) general-purpose ordered cores intended for general-purpose computing; 2) high-performance general-purpose out-of-order cores intended for general-purpose computing; and 3) dedicated cores intended primarily for graphics and / or scientific (throughput) computing. Implementations of different processors may include: 1) CPUs, which include one or more general-purpose ordered cores and / or one or more general-purpose out-of-order cores intended for general-purpose computing; and 2) coprocessors, which include one or more dedicated cores intended primarily for graphics and / or scientific (throughput) computing. These different processors result in different computer system architectures, which may include: 1) a coprocessor on a separate chip from the CPU; 2) a coprocessor in the same package as the CPU but on a separate die; 3) a coprocessor on the same die as the CPU (in which case such a coprocessor is sometimes referred to as dedicated logic or a dedicated core, such as integrated graphics and / or scientific (throughput) logic); and 4) a system-on-a-chip that may include the described CPU (sometimes referred to as application cores or application processors), the coprocessors described above, and additional functionality on the same die. Exemplary core architectures are then described, followed by exemplary processors and computer architectures.

[0112] Exemplary core architecture

[0113] Ordered and disordered kernel diagrams

[0114] Figure 9A This is a block diagram illustrating an exemplary ordered pipeline and an exemplary out-of-order release / execution pipeline for register renaming according to various embodiments of the present invention. Figure 9B This is a block diagram illustrating exemplary embodiments of ordered architecture cores to be included in a processor according to various embodiments of the present invention, and exemplary out-of-order release / execution architecture cores with register renaming. Figures 9A-9B The solid-line boxes in the diagram illustrate ordered pipelines and ordered cores, while the optional additions to the dashed boxes illustrate register renaming, out-of-order release / execution pipelines, and cores. Since the ordered aspect is a subset of the out-of-order aspect, the out-of-order aspect will be described.

[0115] exist Figure 9A In the processor pipeline 900, there are fetch stage 902, length decoding stage 904, decoding stage 906, allocation stage 908, renaming stage 910, scheduling (also known as dispatch or issue) stage 912, register read / memory read stage 914, execution stage 916, write-back / memory write stage 918, exception handling stage 922, and commit stage 924.

[0116] Figure 9BA processor core 990 is shown, which includes a front-end unit 930 coupled to an execution engine unit 950, and both the front-end unit 930 and the execution engine unit 950 are coupled to a memory unit 970. Core 990 can be a Reduced Instruction Set Computing (RISC) core, a Complex Instruction Set Computing (CISC) core, a Very Long Instruction Word (VLIW) core, or a hybrid or alternative core type. Alternatively, core 990 can be a dedicated core, such as, for example, a network or communication core, a compression engine, a coprocessor core, a general-purpose computing graphics processing unit (GPGPU) core, a graphics core, etc.

[0117] Front-end unit 930 includes branch prediction unit 932 coupled to instruction cache unit 934 coupled to instruction translation lookup buffer (TLB) 936 coupled to instruction fetch unit 938 coupled to decoding unit 940. Decoding unit 940 (or decoder) decodes instructions and generates one or more micro-operations, microcode entry points, microinstructions, other instructions, or other control signals as output, which are decoded from, or otherwise reflect, the original instructions or derived from them. Decoding unit 940 can be implemented using various different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memory (ROM), etc. In one embodiment, core 990 includes microcode ROM or other media (e.g., in decoding unit 940, or otherwise within front-end unit 930) storing microcode for certain macro instructions. The decoding unit 940 is coupled to the rename / allocator unit 952 in the execution engine unit 950.

[0118] Execution engine unit 950 includes a rename / allocator unit 952, which is coupled to a retirement unit 954 and a set 956 of one or more scheduler units. The scheduler units 956 represent any number of different schedulers, including reservation stations, central instruction windows, etc. The scheduler units 956 are coupled to physical register file units 958. Each physical register file unit in the physical register file units 958 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integers, scalar floating-point numbers, compressed integers, compressed floating-point numbers, vector integers, vector floating-point numbers, status (e.g., an instruction pointer as the address of the next instruction to be executed), etc. In one embodiment, the physical register file units 958 include vector register units, write mask register units, and scalar register units. These register units may provide architectural vector registers, vector mask registers, and general-purpose registers. Multiple physical register file units 958 are overlapped by retirement units 954 to illustrate various ways in which register renaming and out-of-order execution can be implemented (e.g., using multiple reorder buffers and multiple retirement register files; using multiple future files, multiple history buffers, and multiple retirement register files; using register mappings and register pools, etc.). Retirement units 954 and multiple physical register file units 958 are coupled to multiple execution clusters 960. Multiple execution clusters 960 include a set 962 of one or more execution units and a set 964 of one or more memory access units. Execution units 962 can perform various operations (e.g., shift, addition, subtraction, multiplication) and can perform operations on various data types (e.g., scalar floating-point, compressed integer, compressed floating-point, vector integer, vector floating-point). While some embodiments may include multiple execution units dedicated to a particular function or set of functions, other embodiments may include only one execution unit or multiple execution units that all perform all functions. The scheduler unit 956, physical register file unit 958, and execution cluster 960 are shown as possibly multiple, because some embodiments create separate pipelines for certain types of data / operations (e.g., scalar integer pipelines, scalar floating-point / compact integer / compact floating-point / vector integer / vector floating-point pipelines, and / or memory access pipelines each having their own scheduler unit, physical register file unit, and / or execution cluster—and in the case of separate memory access pipelines, some embodiments are implemented where only the execution cluster of that pipeline has multiple memory access units 964). It should also be understood that, in the case of using separate pipelines, one or more of these pipelines may be out-of-order deployment / execution, and the remaining pipelines may be ordered.

[0119] A set of memory access units 964 is coupled to a memory unit 970, which includes a data TLB unit 972, which is coupled to a data cache unit 974, which is coupled to a Level 2 (L2) cache unit 976. In one exemplary embodiment, the memory access unit 964 may include a load unit, a memory address unit, and a memory data unit, each of which is coupled to the data TLB unit 972 in the memory unit 970. An instruction cache unit 934 is also coupled to the Level 2 (L2) cache unit 976 in the memory unit 970. The L2 cache unit 976 is coupled to one or more other levels of cache and ultimately to main memory.

[0120] As an example, the exemplary register renaming out-of-order release / execution core architecture can implement pipeline 900 as follows: 1) Instruction fetch 938 executes fetch stage 902 and length decoding stage 904; 2) Decoding unit 940 executes decoding stage 906; 3) Rename / allocator unit 952 executes allocation stage 908 and rename stage 910; 4) (multiple) scheduler units 956 execute scheduling stage 912; 5) (multiple) physical register file units 958 and memory unit 970 execute register read / memory read stage 914; execution cluster 960 executes execution stage 916; 6) memory unit 970 and (multiple) physical register file units 958 execute write-back / memory write stage 918; 7) each unit may involve exception handling stage 922; and 8) retirement unit 954 and (multiple) physical register file units 958 execute commit stage 924.

[0121] Core 990 may support one or more instruction sets (e.g., the x86 instruction set (with some extensions added with newer versions); the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc., Sunnyvale, California (with optional additional extensions such as NEON)), including the instructions described herein. In one embodiment, Core 990 includes logic for supporting compressed data instruction set extensions (e.g., AVX1, AVX2), thereby allowing the use of compressed data to perform operations used by many multimedia applications.

[0122] It should be understood that a core can support multithreading (execution of two or more parallel operations or a set of threads), and this multithreading can be accomplished in various ways, including time-division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each thread in the simultaneous multithreading of the physical core), or combinations thereof (e.g., time-division fetching and decoding, and subsequent operations such as...). Simultaneous multithreading in hyper-threading technology.

[0123] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can also be used in ordered architectures. While the illustrated processor embodiment also includes separate instruction and data cache units 934 / 974 and a shared L2 cache unit 976, alternative embodiments may have a single internal cache for both instructions and data, such as, for example, a Level 1 (L1) internal cache or multiple levels of internal caches. In some embodiments, the system may include a combination of internal caches and external caches located outside the core and / or processor. Alternatively, all caches may be located outside the core and / or processor.

[0124] Specific exemplary ordered core architecture

[0125] Figures 10A-10B The diagram illustrates a more specific exemplary ordered core architecture, where the core is one of several logic blocks within a chip (including other cores of the same and / or different types). Depending on the application, the logic block communicates with some fixed functional logic, memory I / O interfaces, and other necessary I / O logic via a high-bandwidth interconnect network (e.g., a ring network).

[0126] Figure 10AThis is a block diagram of a single processor core according to an embodiment of the invention, its connection to an on-die interconnect network 1002, and a local subset 1004 of its second-level (L2) cache. In one embodiment, the instruction decoder 1000 supports the x86 instruction set with a compact data instruction set extension. The L1 cache 1006 allows low-latency access to cache memory into scalar and vector units. Although in one embodiment (for design simplification), scalar unit 1008 and vector unit 1010 use separate register sets (scalar register 1012 and vector register 1014, respectively), and data transferred between these registers is written to memory and subsequently read back from the first-level (L1) cache 1006, alternative embodiments of the invention may use different methods (e.g., using a single register set or including a communication path that allows data to be transferred between these two register sets without being written and read back).

[0127] The local subset 1004 of the L2 cache is part of the global L2 cache, which is divided into multiple separate local subsets, one for each processor core. Each processor core has a direct access path to its own local subset 1004 of the L2 cache. Data read by a processor core is stored in its L2 cache subset 1004 and can be accessed quickly in parallel with other processor cores accessing their own local L2 cache subsets. Data written by a processor core is stored in its own L2 cache subset 1004 and is dumped from other subsets if necessary. A ring network ensures the consistency of shared data. The ring network is bidirectional to allow agents such as processor cores, L2 caches, and other logic blocks to communicate with each other within the chip. Each ring data path is 1012 bits wide in each direction.

[0128] Figure 10B This is according to an embodiment of the present invention. Figure 10A An expanded diagram of a portion of the processor core. Figure 10B This includes the L1 data cache 1006A portion of L1 cache 1004, and further details regarding vector unit 1010 and vector register 1014. Specifically, vector unit 1010 is a 16-wide vector processing unit (VPU) (see 16-wide ALU 1028) that executes one or more of integer, single-precision floating-point, and double-precision floating-point instructions. This VPU supports mixing of register inputs via mixing unit 1020, numerical conversion via value conversion units 1022A-B, and copying of memory inputs via copy unit 1024. Write mask register 1026 allows the writing of predicted vectors.

[0129] Processor with integrated memory controller and graphics device

[0130] Figure 11 This is a block diagram of a processor 1100 according to an embodiment of the present invention, which may have more than one core, may have an integrated memory controller, and may have an integrated graphics device. Figure 11 The solid-line box illustration shows a processor 1100 having a single core 1102A, a system agent 1110, and a set of one or more bus controller units 1116, while the optional additional illustration with dashed boxes shows an alternative processor 1100 having multiple cores 1102A-N, a set of one or more integrated memory controller units 1114 from the system agent units 1110, and dedicated logic 1108.

[0131] Therefore, different implementations of processor 1100 may include: 1) a CPU, wherein dedicated logic 1108 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores), and cores 1102A-N are one or more general-purpose cores (e.g., general-purpose ordered cores, general-purpose out-of-order cores, or a combination of both); 2) a coprocessor, wherein cores 1102A-N are a large number of dedicated cores designed primarily for graphics and / or scientific (throughput); and 3) a coprocessor, wherein cores 1102A-N are a large number of general-purpose ordered cores. Thus, processor 1100 may be a general-purpose processor, coprocessor, or dedicated processor, such as, for example, a network or communication processor, a compression engine, a graphics processor, a GPGPU (General-Purpose Graphics Processing Unit), a high-throughput integrated many-core (MIC) coprocessor (including 30 or more cores), an embedded processor, etc. The processor may be implemented on one or more chips. Processor 1100 may be part of one or more substrates, and / or may be implemented on one or more substrates using any of a variety of process technologies (e.g., BiCMOS, CMOS, or NMOS).

[0132] The memory hierarchy includes one or more cache levels within the core, a set 1106 of one or more shared cache units, and external memory (not shown) coupled to a set 1114 of integrated memory controller units. The set 1106 of shared cache units may include one or more intermediate-level caches, such as Level 2 (L2), Level 3 (L3), Level 4 (L4), or other levels of cache, Last Level Cache (LLC), and / or combinations thereof. While in one embodiment, ring-based interconnect units 1112 interconnect integrated graphics logic 1108, the set 1106 of shared cache units, and system proxy units 1110 / (multiple) of integrated memory controller units 1114, alternative embodiments may use any number of known techniques to interconnect such units. In one embodiment, consistency is maintained between one or more cache units 1106 and cores 1102A-N.

[0133] In some embodiments, one or more cores 1102A-N can be multithreaded. System agent 1110 includes those components that coordinate and operate core 1102A-N. System agent unit 1110 may include, for example, a power control unit (PCU) and a display unit. The PCU may be, or may include, the logic and components required to regulate the power state of core 1102A-N and integrated graphics logic 1108. The display unit is used to drive one or more externally connected displays.

[0134] The 1102A-N cores can be homogeneous or heterogeneous in terms of their instruction set architecture; that is, two or more cores in the 1102A-N cores may be able to execute the same instruction set, while other cores may be able to execute only a subset of that instruction set or a different instruction set.

[0135] Exemplary computer architecture

[0136] Figures 12-15 This is a block diagram of an exemplary computer architecture. Other system designs and configurations known in the art for laptops, desktops, handheld PCs, personal digital assistants, engineering workstations, servers, network devices, network hubs, switches, embedded processors, digital signal processors (DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, cellular phones, portable media players, handheld devices, and various other electronic devices are also suitable. Generally, a wide variety of systems or electronic devices capable of incorporating processors and / or other execution logic as disclosed herein are generally suitable.

[0137] Now for reference Figure 12The diagram illustrates a system 1200 according to an embodiment of the present invention. System 1200 may include one or more processors 1210, 1215 coupled to a controller hub 1220. In one embodiment, the controller hub 1220 includes a graphics memory controller hub (GMCH) 1290 and an input / output hub (IOH) 1250 (which may be on separate chips); the GMCH 1290 includes memory and a graphics controller, to which a memory 1240 and a coprocessor 1245 are coupled; the IOH 1250 couples an input / output (I / O) device 1260 to the GMCH 1290. Alternatively, one or both of the memory and the graphics controller may be integrated within a processor (as described herein), with the memory 1240 and the coprocessor 1245 directly coupled to the processor 1210, and the controller hub 1220 and the IOH 1250 residing on a single chip.

[0138] The optionality of the additional processor 1215 is in Figure 12 The processors 1210 and 1215 are indicated by dashed lines. Each processor 1210 and 1215 may include one or more of the processing cores described herein and may be a version of processor 1100.

[0139] The memory 1240 may be, for example, dynamic random access memory (DRAM), phase-change memory (PCM), or a combination of both. In at least one embodiment, the controller hub 1220 communicates with the processors(s)1210,1215 via a multi-branch bus such as a front-side bus (FSB), a point-to-point interface such as a fast path interconnect (QPI), or a similar connection 1295.

[0140] In one embodiment, the coprocessor 1245 is a dedicated processor, such as, for example, a high-throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, and so on. In one embodiment, the controller hub 1220 may include an integrated graphics accelerator.

[0141] There can be various differences between physical resources 1210 and 1215 in terms of a range of quality metrics, including architecture, microarchitecture, thermal and power consumption characteristics.

[0142] In one embodiment, processor 1210 executes instructions that control general-type data processing operations. Embedded within these instructions may be coprocessor instructions. Processor 1210 identifies these coprocessor instructions as having a type that should be executed by an attached coprocessor 1245. Therefore, processor 1210 issues these coprocessor instructions (or control signals representing coprocessor instructions) to coprocessor 1245 on a coprocessor bus or other interconnect. Coprocessors 1245 (multiple) receive and execute the received coprocessor instructions.

[0143] See now Figure 13 The diagram shown is a block diagram of a first, more specific, exemplary system 1300 according to an embodiment of the present invention. Figure 13 As shown, the multiprocessor system 1300 is a point-to-point interconnect system and includes a first processor 1370 and a second processor 1380 coupled via a point-to-point interconnect 1350. Each of processors 1370 and 1380 may be a version of processor 1100. In one embodiment of the invention, processors 1370 and 1380 are processors 1210 and 1215, respectively, and coprocessor 1338 is coprocessor 1245. In another embodiment, processors 1370 and 1380 are processor 1210 and coprocessor 1245, respectively.

[0144] Processors 1370 and 1380 are shown as including integrated memory controller (IMC) units 1372 and 1382, respectively. Processor 1370 also includes point-to-point (PP) interfaces 1376 and 1378 as part of its bus controller unit; similarly, the second processor 1380 includes PP interfaces 1386 and 1388. Processors 1370 and 1380 can exchange information via PP interface 1350 using point-to-point (PP) interface circuitry 1378 and 1388. Figure 13 As shown, IMC 1372 and 1382 couple the processor to the corresponding memory, namely memory 1332 and memory 1334, which may be portions of the main memory locally attached to the corresponding processor.

[0145] Processors 1370 and 1380 can exchange information with chipset 1390 via respective PP interfaces 1352 and 1354 using point-to-point interface circuits 1376, 1394, 1386, and 1398. Chipset 1390 can optionally exchange information with coprocessor 1338 via high-performance interface 1339. In one embodiment, coprocessor 1338 is a dedicated processor, such as, for example, a high-throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, etc.

[0146] A shared cache (not shown) may be included in either processor or external to both processors but connected to them via a PP interconnect, such that if the processors are in a low-power mode, the local cache information of either or both processors may be stored in the shared cache.

[0147] Chipset 1390 can be coupled to first bus 1316 via interface 1396. In one embodiment, first bus 1316 may be a peripheral component interconnect (PCI) bus or a bus such as PCI Fast Bus or another third-generation I / O interconnect bus, but the scope of the invention is not limited thereto.

[0148] like Figure 13 As shown, various I / O devices 1314 may be coupled to a first bus 1316 along with a bus bridge 1318, which couples the first bus 1316 to a second bus 1320. In one embodiment, one or more additional processors 1315, such as a coprocessor, a high-throughput MIC processor, a GPGPU, an accelerator (such as, for example, a graphics accelerator or digital signal processing (DSP) unit), a field-programmable gate array, or any other processor, are coupled to the first bus 1316. In one embodiment, the second bus 1320 may be a low pin count (LPC) bus. In one embodiment, various devices may be coupled to the second bus 1320, including, for example, a keyboard and / or mouse 1322, a communication device 1327, and a storage unit 1328, such as a disk drive that may include instruction / code and data 1330, or other mass storage devices. Additionally, audio I / O 1324 may be coupled to the second bus 1320. Note that other architectures are possible. For example, instead of… Figure 13 The point-to-point architecture allows the system to implement multi-branch buses or other similar architectures.

[0149] Now for reference Figure 14 The diagram shown is a block diagram of a second, more specific, exemplary system 1400 according to an embodiment of the present invention. Figure 13 and 14 Similar elements in the figure use similar reference numerals, and from Figure 14 The middle part is omitted Figure 13 certain aspects to avoid confusion Figure 14 Other aspects.

[0150] Figure 14The illustrated processors 1370 and 1380 may include integrated memory and I / O control logic (“CL”) 1372 and 1382, respectively. Therefore, CL 1372 and 1382 include an integrated memory controller unit and I / O control logic. Figure 14 The diagram shows that not only are the memories 1332 and 1334 coupled to CLs 1372 and 1382, but the I / O device 1414 is also coupled to the control logic 1372 and 1382. The conventional I / O device 1415 is coupled to the chipset 1390.

[0151] Now for reference Figure 15 The diagram shown is a block diagram of a SoC 1500 according to an embodiment of the present invention. Figure 11 Similar elements in the figure use similar reference numerals. Additionally, dashed boxes are optional features on more advanced SoCs. Figure 15 In this configuration, multiple interconnect units 1502 are coupled to: an application processor 1510, which includes an assembly 142A-N of one or more cores and multiple shared cache units 1106; a system proxy unit 1110; multiple bus controller units 1116; multiple integrated memory controller units 1114; an assembly 1520 of one or more coprocessors, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 1530; a direct memory access (DMA) unit 1532; and a display unit 1540 for coupling to one or more external displays. In one embodiment, the multiple coprocessors 1520 include dedicated processors, such as, for example, network or communication processors, compression engines, GPGPUs, high-throughput MIC processors, or embedded processors, etc.

[0152] The various embodiments of the mechanisms disclosed herein can be implemented in hardware, software, firmware, or a combination of such implementations. Embodiments of the invention can be implemented as computer programs or program code executable on a programmable system including at least one processor, a storage system (including volatile and non-volatile memories and / or storage elements), at least one input device, and at least one output device.

[0153] Program code (such as, Figure 13 The code 1330 shown in the figure is applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor, such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0154] The program code can be implemented using a high-level procedural or object-oriented programming language to communicate with the processing system. Assembly or machine language can also be used if needed. In fact, the mechanisms described herein are not limited to any particular programming language. In any case, the language can be a compiled or interpreted language.

[0155] One or more aspects of at least one embodiment can be implemented by representational instructions stored on a machine-readable medium, which represent various logics in a processor, and which, when read by a machine, cause the machine to manufacture logic for performing the techniques described herein. Such representations, referred to as “IP cores,” can be stored on tangible machine-readable media and can be supplied to various customers or production facilities for loading into manufacturing machines that actually manufacture the logic or processor.

[0156] Such machine-readable storage media can include, but are not limited to, non-transitory, tangible arrangements of articles made or formed by a machine or device, including storage media such as hard disks; any other type of disk, including floppy disks, optical disks, compact disc read-only memory (CD-ROM), rewritable compact discs (CD-RW), and magneto-optical disks; semiconductor devices such as read-only memory (ROM), random access memory (RAM) such as dynamic random access memory (DRAM) and static random access memory (SRAM), erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM); phase-change memory (PCM); magnetic cards or optical cards; or any other type of medium suitable for storing electronic instructions.

[0157] Therefore, embodiments of the invention also include a non-transitory tangible machine-readable medium containing instructions or design data, such as a hardware description language (HDL), that defines the architectures, circuits, devices, processors, and / or system features described herein. These embodiments are also referred to as program products.

[0158] Simulation (including binary transformation, code transformation, etc.)

[0159] In some cases, instruction translators can be used to translate instructions from a source instruction set to a target instruction set. For example, an instruction translator can transform instructions (e.g., using static binary transformation, including dynamically compiled binary transformation), modify, emulate, or otherwise convert them into one or more other instructions to be processed by the kernel. Instruction translators can be implemented in software, hardware, firmware, or a combination thereof. Instruction translators can be on the processor, off the processor, or partially on and partially off the processor.

[0160] Figure 16 This is a block diagram illustrating the use of a software instruction converter to convert binary instructions in a source instruction set into binary instructions in a target instruction set, according to an embodiment of the present invention. In the illustrated embodiment, the instruction converter is a software instruction converter; however, alternatively, the instruction converter can be implemented using software, firmware, hardware, or various combinations thereof. Figure 16 This illustrates the use of an x86 compiler 1604 to compile a program of the form of a high-level language 1602 to generate x86 binary code 1606 that can be natively executed by a processor 1616 having at least one x86 instruction set core. A processor 1616 having at least one x86 instruction set core refers to any processor that performs substantially the same function as an Intel processor having at least one x86 instruction set core by compatibly executing or otherwise performing: 1) an essential portion of the instruction set of an Intel x86 instruction set core, or 2) a version of object code for an application or other software aimed at running on an Intel processor having at least one x86 instruction set core to achieve substantially the same results as an Intel processor having at least one x86 instruction set core. The x86 compiler 1604 refers to a compiler operable to generate x86 binary code 1606 (e.g., object code) that can be executed on a processor 1616 having at least one x86 instruction set core, with or without additional linking processing. Similarly, Figure 16 A program in the form of a high-level language 1602 can be compiled using an alternative instruction set compiler 1608 to generate alternative instruction set binary code 1610 that can be natively executed by a processor 1614 that does not have at least one x86 instruction set core (e.g., a processor with a core that executes the MIPS instruction set of MIPS Technologies, Inc., Sunnyvale, California, and / or the ARM instruction set of ARM Holdings, Inc., Sunnyvale, California). An instruction converter 1612 is used to translate the x86 binary code 1606 into code that can be natively executed by the processor 1614 that does not have an x86 instruction set core. This translated code is unlikely to be identical to the alternative instruction set binary code 1610 because an instruction converter capable of doing so would be difficult to manufacture; however, the translated code will perform general operations and consists of instructions from the alternative instruction set. Therefore, the instruction converter 1612 represents, through emulation, simulation, or any other process, software, firmware, hardware, or a combination thereof that allows a processor or other electronic device without an x86 instruction set processor or core to execute the x86 binary code 1606.

[0161] The components, features, and details described for any of the processors disclosed herein may optionally be applied to any of the methods disclosed herein that may optionally be executed by and / or using such processors in embodiments. Any of the processors described herein in embodiments may optionally be included in any of the systems disclosed herein. Any of the instructions disclosed herein in embodiments may optionally be executed by and / or using any of the processors disclosed herein, optionally having any of the microarchitectures shown herein in some embodiments, and optionally included in any of the systems shown herein in some embodiments. Therefore, the features and details described for any of the instructions disclosed herein may thus optionally be applied in some embodiments to any of the processors and / or systems disclosed herein that may be used to execute those instructions.

[0162] The processor components disclosed herein are operable, configurable, capable of, or able to be used to perform operations. For example, a decoder can be used to decode instructions, an execution unit can be used to store results, and so on. For clarity, these expressions should be understood not to imply that the processor components are in operation or use, but rather to indicate what they are capable of or able to be used for when they are in operation, but which are not in operation in the apparatus claims.

[0163] The terms “coupled” and / or “connected” and their derivatives may be used in the specification and claims. These terms are not intended to be synonymous with each other. Rather, in various embodiments, “connected” can be used to indicate that two or more elements are in direct physical and / or electrical contact with each other. “Coupled” can mean that two or more elements are in direct physical and / or electrical contact with each other. However, “coupled” can also mean that two or more elements are not in direct contact with each other, but still cooperate or act on each other. For example, an execution unit may be coupled to a register and / or decoding unit via one or more intermediate components. In the drawings, arrows are used to illustrate connection and coupling.

[0164] The components disclosed herein and the methods depicted in the foregoing figures can be implemented using logic, modules, or units, including hardware (e.g., transistors, gates, circuits, etc.), firmware (e.g., non-volatile memory storing microcode or control signals), software (e.g., stored on a non-transitory computer-readable storage medium), or combinations thereof. In some embodiments, these logic, modules, or units may comprise at least some or primarily a mixture of hardware and / or firmware potentially combined with some optional software.

[0165] The term "and / or" may have been used. As used herein, the term "and / or" means one or the other or both (e.g., A and / or B means: A or B, or both A and B).

[0166] In the foregoing description, specific details have been set forth in order to provide a thorough understanding of the embodiments. However, other embodiments may be practiced without some of these specific details. The scope of the invention is not determined by the specific examples provided above, but only by the appended claims. In other instances, well-known circuits, structures, devices, and operations have been shown in block diagram form and / or without detail to avoid obscuring the understanding of the specification. Where deemed appropriate, reference numerals or the closing portions of reference numerals have been repeated between the figures to indicate corresponding or similar elements that may optionally have similar or identical characteristics, unless otherwise specified or obvious.

[0167] Certain operations may be performed by hardware components or embodied in machine-executable or circuit-executable instructions that can be used to cause and / or lead a machine, circuit, or hardware component (e.g., a processor, a portion of a processor, circuitry, etc.) to be programmed with instructions that perform these operations. These operations may also optionally be performed by a combination of hardware and software. The processor, machine, circuit, or hardware may include dedicated or specific circuitry or other logic (e.g., hardware possibly combined with firmware and / or software) for executing and / or processing instructions and storing results in response to those instructions.

[0168] Some embodiments include an article of manufacture (e.g., a computer program product) that includes a machine-readable medium. The medium may include mechanisms for providing (e.g., storing) information in a machine-readable form. The machine-readable medium may provide or store instructions or sequences of instructions that, if and / or when executed by a machine, may be used to cause the machine to perform and / or cause the machine to perform one or more operations, methods, or techniques disclosed herein.

[0169] In some embodiments, a machine-readable medium may include tangible and / or non-transitory machine-readable storage media. For example, a non-transitory machine-readable storage medium may include floppy disks, optical storage media, optical disks, optical data storage devices, CD-ROMs, magnetic disks, magneto-optical disks, read-only memory (ROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), flash memory, phase-change memory, phase-change data storage materials, non-volatile memory, non-volatile data storage devices, non-transitory memory, or non-transitory data storage devices, etc. A non-transitory machine-readable storage medium does not consist of transiently propagated signals. In some embodiments, the storage medium may include tangible media, including solid substances or materials such as, for example, semiconductor materials, phase-change materials, magnetic solid materials, solid data storage materials, etc. Alternatively, non-tangible transient computer-readable transmission media may be used, such as, for example, electrical, optical, acoustic, or other forms of propagated signals—such as carrier waves, infrared signals, and digital signals.

[0170] Examples of suitable machines include, but are not limited to, general-purpose processors, special-purpose processors, digital logic circuits, integrated circuits, etc. Other examples of suitable machines include computer systems or other electronic devices that contain processors, digital logic circuits, or integrated circuits. Examples of such computer systems or electronic devices include, but are not limited to, desktop computers, laptop computers, notebook computers, tablet computers, netbooks, smartphones, cellular phones, servers, network equipment (e.g., routers and switches), mobile internet devices (MIDs), media players, smart TVs, internet access devices, set-top boxes, and video game controllers.

[0171] For example, references throughout this specification to “one embodiment,” “embodiment,” “one or more embodiments,” and “some embodiments” indicate that specific features may be included in the implementation of the invention, but are not necessarily required to be. Similarly, in this description, to facilitate the flow of this disclosure and aid in the understanding of the various inventive aspects, various features are sometimes grouped together in a single embodiment, drawing, or description thereof. However, the disclosed method should not be construed as reflecting an intention that the invention requires more features than expressly recited in each claim. Rather, as reflected in the appended claims, the inventive aspect lies in fewer than all features of a single disclosed embodiment. Therefore, the claims following the detailed description are expressly incorporated into this detailed description, and each claim exists independently as a separate embodiment of the invention.

[0172] Example 1 is a processor including a decoding unit for decoding matrix multiplication instructions. The matrix multiplication instructions indicate a first memory location for a first source matrix, a second memory location for a second source matrix, and a third memory location for storing the result matrix. The processor also includes an execution unit coupled to the decoding unit. The execution unit is configured to, in response to the matrix multiplication instructions, perform partial multiplication of the first and second source matrices before an interrupt, and store a completion progress indicator in response to the interrupt. The completion progress indicator indicates the amount of progress made before the interrupt in multiplying the first and second source matrices and storing the corresponding result data in the third memory location.

[0173] Example 2 includes the processor of Example 1, wherein optionally, the execution unit is used to store a completion progress indicator that is not defined in the architecture in response to a matrix multiplication instruction.

[0174] Example 3 includes the processor of Example 1, wherein optionally, the execution unit is configured to store a completion progress indicator that will not be understood by the software in response to a matrix multiplication instruction.

[0175] Example 4 includes the processor of Example 1, wherein optionally, the execution unit is configured to, in response to the resumption of matrix multiplication instructions after an interrupt, perform the following operations: receive a completion progress indicator, and use the completion progress indicator to resume the multiplication of the first and second source matrices without repeating the multiplication of portions of the first and second source matrices that would have been multiplied before the interrupt.

[0176] Example 5 includes the processor of Example 1, wherein optionally, a decoding unit is used to decode matrix multiplication instructions, which are used to indicate matrix dimension indicators.

[0177] Example 6 includes the processor of Example 1, wherein optionally, a decoding unit is configured to decode a matrix multiplication instruction, the matrix multiplication instruction being configured to indicate: the number of rows of a first source matrix; the number of columns of a second source matrix; and at least one of the following: (a) the number of columns of the first source matrix; and (b) the number of rows of the second source matrix.

[0178] Example 7 includes the processor of Example 6, wherein optionally, the first source matrix, the second source matrix, and the result matrix are each stored in memory in column-major format, and wherein optionally, a decoding unit is configured to decode matrix multiplication instructions for indicating the distances between columns of the larger matrix including the first source matrix, the distances between columns of the larger matrix including the second source matrix, and the distances between columns of the larger matrix including the result matrix.

[0179] Example 8 includes the processor of Example 6, wherein optionally, the first source matrix, the second source matrix, and the result matrix are each stored in memory in row-major format, and wherein optionally, a decoding unit is configured to decode matrix multiplication instructions for indicating the distances between rows of the larger matrix including the first source matrix, the distances between rows of the larger matrix including the second source matrix, and the distances between rows of the larger matrix including the result matrix.

[0180] Example 9 includes the processor of Example 1, wherein optionally, the execution unit is configured to decompose a first source matrix into multiple slices in response to a matrix multiplication instruction.

[0181] Example 10 includes the processor of Example 9, wherein optionally, the execution unit is configured to decompose a first source matrix into a plurality of slices having a first size in response to a matrix multiplication instruction, and to decompose at least one slice having the first size into a plurality of slices having a second size smaller than the first size.

[0182] Example 11 includes the processor of Example 9, wherein optionally, the execution unit is configured to store at least two copies of each of the chips in a cache in response to a matrix multiplication instruction.

[0183] Example 12 includes the processor of Example 11, wherein optionally, the execution unit is configured to, in response to a matrix multiplication instruction, perform the following operations: decompose each of the first source matrix, the second source matrix, and the accumulation matrix into multiple slices, the accumulation matrix being initially stored in a third memory location; store two copies of each of the slices from the first and second source matrices in a cache; and store three copies of each of the slices from the accumulation matrix in a cache.

[0184] Example 13 includes a processor of any of Examples 1 to 12, wherein the execution unit is configured to perform the following operations in response to a matrix multiplication instruction: (1) determine whether a given slice is a complete slice or a partial slice; (2) optionally perform relatively more concurrent multiplications when the given slice is a complete slice; or (3) optionally perform relatively fewer concurrent multiplications when the given slice is a partial slice.

[0185] Example 14 includes a processor of any one of Examples 1 to 12, wherein the execution unit is configured to perform the following operations in response to the resumption of matrix multiplication instructions after an interrupt: (1) speculatively begin execution of the matrix multiplication instructions with the assumption that the prior progress indicator indicates that the prior progress quantity is none; and (2) optionally cause a micro-anomaly or other signal when the completion progress indicator is used to indicate that the prior progress quantity is not none.

[0186] Example 15 includes a processor of any one of Examples 1 to 12, wherein a decoding unit is configured to decode a matrix multiplication instruction having an immediate number indicating the size of each of the first and second source matrices, and wherein optionally, the processor is configured to introduce the operation into the front end of the processor's pipeline based on the indicated size of each of the first and second source matrices.

[0187] Example 16 includes a processor of any of Examples 1 to 12, wherein the execution unit is configured to perform a multiplication in response to a matrix multiplication instruction, initially adding a portion of an accumulated matrix stored in a third memory location to portions of the first and second source matrices.

[0188] Example 17 includes a processor of any of Examples 1 to 12, further includes a general-purpose register, and wherein the execution unit is configured to optionally store a completion progress indicator in the general-purpose register in response to a matrix multiplication instruction.

[0189] Example 18 is a method executed by a processor, comprising receiving a matrix multiplication instruction at the processor. The matrix multiplication instruction indicates a first memory location of a first source matrix, a second memory location of a second source matrix, and a third memory location where the result matrix will be stored. In response to the matrix multiplication instruction, partial multiplication of the first and second source matrices is performed before an interrupt. A completion progress indicator is stored in response to both the matrix multiplication instruction and the interrupt. The completion progress indicator indicates the amount of progress made before the interrupt in multiplying the first and second source matrices and storing the corresponding result data in the third memory location.

[0190] Example 19 includes the method of Example 18, wherein the storage includes a completion progress indicator that is not architecturally defined and cannot be understood by the software.

[0191] Example 20 includes the method of Example 18, further including, in response to the resumption of matrix multiplication instructions after an interruption, performing the following operations: (1) receiving a completion progress indicator; and (2) using the completion progress indicator to resume the multiplication of the first and second source matrices without repeating the multiplication of portions of the first and second source matrices that were multiplied before the interruption.

[0192] Example 21 includes the method of Example 18, wherein receiving includes receiving a matrix multiplication instruction, the matrix multiplication instruction being used to indicate: the number of rows of a first source matrix; optionally the number of columns of a second source matrix; and optionally at least one of the following: (a) the number of columns of the first source matrix; and (b) the number of rows of the second source matrix.

[0193] Example 22 includes the method of Example 18, and further includes decomposing the first source matrix into multiple slices in response to a matrix multiplication instruction.

[0194] Example 23 includes the method of Example 18, further including, in response to the resumption of matrix multiplication instructions after an interruption, performing the following operations: (1) optionally starting to execute the matrix multiplication instructions speculatively by assuming that the completion progress indicator indicates that the prior progress quantity is none; and (2) optionally causing a micro-anomaly when the completion progress indicator indicates that the prior progress quantity is not none.

[0195] Example 24 is a computer system including an interconnect and a processor coupled to the interconnect. The processor is used to receive matrix multiplication instructions. The matrix multiplication instructions are used to indicate a first memory location for a first source matrix, a second memory location for indicating a second source matrix, and a third memory location for storing the result matrix. The processor is used in response to the matrix multiplication instructions to perform the following operations: partially multiplying the first and second source matrices before an interrupt, and storing a completion progress indicator in an architecturally visible memory location in response to the interrupt. The completion progress indicator is used to indicate the amount of progress made before the interrupt in multiplying the first and second source matrices and storing the corresponding result data in the third memory location. The system also includes dynamic random access memory (DRAM) coupled to the interconnect. DRAM storing instructions other than one or more instances of the matrix multiplication instructions is not used to access or use the completion progress indicator.

[0196] Example 25 includes the computer system of Example 24, wherein the processor is used to store a completion progress indicator that is not defined in the architecture in response to a matrix multiplication instruction.

Claims

1. A processor (102), comprising: Decoder (108) is used for matrix multiplication instructions (106); 606) Decoding, the matrix multiplication instruction (106; 606) has a first field associated with the first source matrix (124), a second field associated with the second source matrix (128), a destination field (132), and an opcode for identifying the matrix multiplication instruction (106; 606); An execution unit (110) is coupled to the decoder (108), the execution unit (110) being configured to respond to the matrix multiplication instruction (106); 606) and perform the operation, the operation including: The first source matrix (124) is divided into a first plurality of slices, each of the first plurality of slices including a specified number of non-overlapping data elements, and The second source matrix (128) is divided into a second plurality of slices, each of which includes a specified number of non-overlapping data elements. The execution unit (110) includes fused matrix multiplication and addition logic (558), which is used to perform parallel fused multiplication and accumulation operations using data elements from a first slice in the first plurality of slices and data elements from a second slice in the second plurality of slices. At least one of the fused multiplication-accumulation operations is used for: The data elements from the first slice are multiplied by the corresponding data elements from the second slice to generate multiple products, and one or more of the multiple products are added to the corresponding data elements from the accumulation matrix to generate the result matrix (132). The corresponding result value in 532).

2. The processor of claim 1, wherein, The first field is used to indicate the position for the first source matrix (124), the second field is used to indicate the position for the second source matrix (128), and the destination field is used to indicate the position for the result matrix (132; 532).

3. The processor of claim 1, wherein, The opcode corresponds to the size of the first source matrix (124), the second source matrix (128), and the result matrix (132; 532).

4. The processor (102) of claim 1, wherein, The fusion matrix multiplication and addition logic (558) includes: Multiple multipliers, each multiplier multiplying a data element from the first slice with a data element from the second slice to generate one of the multiple products, and Multiple adders, each adder being used to add one or more of the multiple products to the corresponding data element from the accumulation matrix to generate the corresponding result value in the result matrix (132; 532).

5. The processor (102) of claim 4, wherein, The decoder (108) is configured to output a plurality of micro-operation or control signals in response to the matrix multiplication instruction, wherein a first plurality of micro-operation or control signals indicates a multiplication to be performed by the plurality of multipliers.

6. The processor (102) as claimed in claim 4, wherein, The size of the first slice and the size of the second slice are based on the number of the plurality of multipliers.

7. The processor (102) as claimed in claim 4, wherein, The plurality of multipliers includes multiple multipliers for optimizing slices having one or more predetermined sizes.

8. The processor (102) as claimed in claim 7, wherein, The one or more predetermined sizes include sheets with 16 rows and 16 columns, sheets with 32 rows and 32 columns, or sheets with 64 rows and 64 columns.

Images