Matrix multiplier cache

Abstract

Techniques related to integrated circuits supporting matrix operations are disclosed. In various embodiments, an integrated circuit includes a dot product accumulation circuit including: a dot product circuit configured to determine a dot product of a first vector and a second vector; and an adder circuit coupled to an output of the dot product circuit and configured to add a result of the dot product to the accumulated value. The integrated circuit also includes an accumulator cache coupled to an input of the adder circuit and an output of the adder circuit. The accumulator cache is configured to provide the accumulated value to the adder circuit, and store the result of the addition as a subsequent accumulated value for a subsequent point accumulation addition operation.

Description

Technical Field

[0001] This disclosure relates generally to processors, and more specifically to performing matrix operations using processor hardware. Background Technology

[0002] Some computational tasks heavily rely on matrix operations. These tasks can include those related to graphics processing, such as rendering, shading, lighting, and texturing. Matrix operations are also frequently used in various machine learning algorithms, such as those involving different types of neural networks. To provide support for these types of tasks, designers of central processing units (CPUs) and / or graphics processing units (GPUs) can define instructions in their instruction set architectures (ISAs) for performing matrix operations. As the complexity of these tasks increases, the need for efficient matrix operations on the underlying hardware also increases. Attached Figure Description

[0003] Figure 1 This is a block diagram illustrating an exemplary integrated circuit, which includes a matrix multiplier having a point accumulation circuit using an accumulator cache.

[0004] Figure 2 This is a block diagram illustrating an exemplary arrangement of point accumulator circuitry and its corresponding accumulator cache within a matrix multiplier.

[0005] Figure 3 This is a block diagram illustrating an example of a component within a point accumulation circuit.

[0006] Figure 4 This is a block diagram illustrating an exemplary accumulator cache with multiple entries for storing results.

[0007] Figure 5 This is a block diagram illustrating an example of using the compiler's intelligent scheduling, which provides hints to support the use of the accumulator cache.

[0008] Figure 6 This is a flowchart illustrating an exemplary method performed by a matrix multiplier.

[0009] Figure 7 This is a flowchart illustrating an exemplary method executed by the compiler.

[0010] Figure 8 This is a block diagram illustrating an exemplary computing device that implements the functionality described herein.

[0011] Figure 9 These are illustrations of exemplary applications of systems and devices that implement the functionality described herein.

[0012] Figure 10This is a block diagram illustrating an exemplary computer-readable medium for storing circuit design information for implementing a device having the functionality described herein. Detailed Implementation

[0013] In linear algebra, matrix multiplication typically involves performing a dot product on each combination of rows in a first matrix and columns in a second matrix, where 1) each value in a given row is multiplied by the corresponding value in a given column, and 2) the resulting products are then summed. It may also be desirable to further add offset / accumulated values ​​to this sum. For example, in neural networks, computing a perceptron involves adding a bias ω0 to the dot product of the input vector and a vector of ω weights. This type of operation is called dot accumulation and is supported by instructions defined by the Instruction Set Architecture (ISA) in some GPU architectures.

[0014] In some cases, workloads involve interdependent matrix multiplications, where the output of one dot-accumulator is used as the accumulated input operand for another. For example, in a recurrent neural network (RNN), the first dot product of the input vector and the weight vector can be added to the result of the second dot product of the history vector and the weight vector. However, if instructions that depend on dot-accumulators are scheduled sequentially, pipeline stalls may occur in the processor executing the instructions because the first dot-accumulator is written back and then retrieved from the register file, which is a time-consuming process.

[0015] This disclosure describes an implementation in which a cache is used to locally store the result of a first point accumulation, such that the result is immediately available for a second dependent point accumulation without the latency increase caused by retrieving the result from a register file. As will be described below in various embodiments, the integrated circuit may include point accumulation circuitry and an accumulator cache. The point accumulation circuitry may include a dot product circuitry configured to determine the dot product of a first vector and a second vector; and an adder circuitry configured to add the result of the dot product to an accumulated value. The accumulator cache is configured to store the result of the addition as an accumulated value for subsequent point accumulation operations. Thus, when there is a dependency for a second point accumulation, the accumulator cache is configured to provide the accumulated value to the adder circuitry without having to access a register file storing, for example, the first and second vectors.

[0016] Now go to Figure 1 A block diagram of an integrated circuit 10 configured to support matrix multiplication is depicted. In an illustrated embodiment, the integrated circuit 10 includes a matrix multiplier 100 comprising a plurality of dot product accumulation circuits 110. Circuit 110 further includes a dot product circuit 112, an adder 114, and an accumulator cache 116. In some embodiments, the integrated circuit 10 may be implemented differently from that shown, such as including multiple additional components, such as Figure 2 and Figure 8 The subject of discussion.

[0017] Integrated circuit (IC) 10 may correspond to any suitable circuit configured to perform matrix-related operations, such as dot-matrix addition. In some embodiments, integrated circuit 10 is a central processing unit (CPU), application-specific integrated circuit (ASIC), system-on-a-chip (SoC), field-programmable gate array (FPGA), etc. In some embodiments, IC 10 is a GPU that can perform graphics-related tasks (e.g., rendering) by executing matrix multiplication instructions. In some embodiments, IC 10 is a neural engine that can execute matrix multiplication instructions in parallel while training a machine learning model. Integrated circuit 10 may also be included in any suitable computing device (such as a desktop computer, laptop computer, tablet computer, mobile computing device, or hereinafter referred to as such). Figure 9 (Any other device among the other devices discussed).

[0018] Matrix multiplier 100 is a circuit configured to perform various matrix-related operations, which can be executed by IC 10 in response to instructions defined by a specific ISA. In an illustrated embodiment, matrix multiplier 100 uses dot-accumulator circuit 110 to perform dot-accumulator operations. As shown, dot-accumulator circuit 110 includes dot-product circuit 112 and adder 114. Dot-product circuit 112 is a circuit configured to determine the initial dot-product portion of a dot-accumulator operation between a first vector A and a second vector B, which can be vectors within the larger matrices A and B multiplied by matrix multiplier 100. Adder 114 is a circuit configured to add the dot-product output generated by circuit 112 to an offset / accumulated value. As noted above, in some cases, matrix multiplier 100 may receive successive instructions to perform two interdependent dot-accumulation additions, such that the output of the first dot-accumulation addition (shown as result C) is used as the input accumulation value for the second dot-accumulation addition (shown as result C is fed back into adder 114). For example, matrix multiplier 100 may receive successive instructions to compute the first dot product of the input vector and weight vector in a recurrent neural network (RNN), as noted above, and add the resulting value to the second dot product of the history vector and weight vector in the RNN. If matrix multiplier 100 writes the result of the first dot product back to memory (e.g., in conjunction with the following...), Figure 2 If the data register file 220 under discussion is then read back and the result is subsequently read to calculate the second dot product, the resulting travel time of reusing that value may cause pipeline stalls and performance degradation in the matrix multiplier 100.

[0019] However, in the illustrated implementation, the accumulator cache 116 is coupled to the output of the dot product accumulation circuit 110 to store the intermediate result of the first instruction and to provide this intermediate result back to circuit 110 when executing the second dependent instruction. Therefore, stalling is avoided when executing a set of dependent instructions. As shown, if circuit 110 executes the second instruction based on the result of the first instruction, the accumulator cache 116 provides the dot product from the first instruction to adder 114, which adds the output of the dot product circuit 112 for subsequent instructions to the cached result C to determine the subsequent dot product accumulation. Relative to Figure 3 The accumulator cache 116 is discussed in more detail.

[0020] Now relative to Figure 2 The arrangement of multiple multiplication circuits and corresponding caches is discussed.

[0021] Now go to Figure 2 A block diagram of additional components in integrated circuit 10 is shown. In the illustrated embodiment, integrated circuit 10 includes a scheduler 210, a data register file 220, and a matrix multiplier 100, which includes multiple point accumulator circuits 110 and an accumulator cache 116. In some embodiments, IC 10 is implemented in a different manner than shown. As an example, matrix multiplier 100 may include two separate accumulator caches 116 for integer and floating-point circuits 110.

[0022] Scheduler 210 is circuitry configured to schedule instructions for execution on various execution units, such as matrix multiplier 100 including point-specific accumulator circuitry 110. In some implementations, because scheduler 210 resides in a single-instruction multiple-data (SIMD) processor, scheduler 210 may have fewer capabilities than a CPU scheduler that is allowed to occupy a larger portion of the die space. For example, scheduler 210 may lack the ability to identify instruction dependencies and schedule accordingly (or have limited capabilities), which may reduce the ability to utilize accumulator cache 116. However, as will be discussed below... Figure 5 As discussed, scheduler 210 may support the ability to receive "hints" generated by the compiler that identify dependencies. Thus, when the compiler determines that there is a dependency between two or more instructions, it will provide an indication to scheduler 210, which can then schedule the two instructions to be executed sequentially to utilize cache 116.

[0023] In the illustrated implementation, matrix multiplier 100 includes thirty-two execution channels, each having an available point-accumulator circuit 110 and a corresponding accumulator cache 116. When executing instructions, matrix multiplier 100 receives matrices A and B from registers in a data register file 220, which includes an array of registers for data storage. In some implementations, matrix multiplier 100 may subsequently load elements from matrices A and B into a source cache for a specific channel before performing matrix operations. To support integer and floating-point operations, in various implementations, matrix multiplier 100 includes separate integer (int) point-accumulator circuits 110 and floating-point (fp) point-accumulator circuits 110, each having logic / circuit for handling these different data types. In the illustrated implementation, a given accumulator cache 116 is shared between the integer and floating-point point-accumulator circuits 110 because a given data path can be used to execute instructions of only one data type at a time. In other embodiments, the accumulator cache 116 may be included within the point accumulator circuit 110, such that the floating-point point accumulator circuit has a separate accumulator cache 116 from the cache 116 included within the integer point accumulator circuit 110.

[0024] Because die space can limit the total number of available channels / circuit 110, in various embodiments, matrix multiplier 100 can perform some matrix multiplications by performing multiple passes via circuit 110, where each pass processes different parts of matrices A and B. For example, in one embodiment, Figure 2 The 32 circuits 110 depicted can operate on 128 elements of a matrix at a given time. However, if an instruction to multiply a 16×16 matrix (containing a total of 256 elements) has been received, the matrix multiplier 100 can perform a first pass, whereby it passes a first portion (the initial 128 elements of each matrix) of the first and second matrices to the dot-accumulator circuit 110 to compute the first partial result set, and passes a second portion (the remaining 128 elements) of the first and second matrices to the dot-accumulator circuit 110 to compute the second partial result set. In some embodiments, the first and second portions are pipelined such that the multiplier 100 can pass the second portion of the first and second matrices to the dot-accumulator circuit 110 while the first partial result set is stored in the accumulator cache 116.

[0025] Compared to Figure 3 The matrix operations performed by the point accumulation circuit 110 are further described.

[0026] Now go to Figure 3A block diagram of a dot product accumulator circuit 110 is depicted. In an illustrated embodiment, circuit 110 includes a dot product circuit 112, an adder 114, a multiplexer 340, and an accumulator cache 116. As further depicted, the dot product circuit 112 includes multiple latches 310, multipliers 320, and adders 330. In some embodiments, circuit 110 is implemented in a different manner than shown. For example, although... Figure 3 Circuit 110 is depicted to support dot product accumulation between two vectors A and B, each having four elements. However, circuit 110 may support dot products with more (or fewer) elements, such as 8-way dot product accumulation, 16-way dot product accumulation, etc. As another example, circuit 110 may include cache 116.

[0027] As previously discussed, the dot-matrix accumulator circuit 110 receives vectors A and B from data register file 220. As shown, input vector A may include elements A0, A1, A2, and A3; input vector B may include elements B0, B1, B2, and B3, which can be integers or floating-point values ​​of any suitable size. After latch 310 receives elements A0-3 and B0-3, latch 310 releases these elements to multiplier 320. Before these elements reach multiplier 320, the permutation network in circuit 112 transposes the elements from vectors A and B to prepare for multiplier 320, such that elements from vector A are paired with their corresponding elements from vector B.

[0028] Multiplier 320 is configured to perform a multiplication operation in which the elements of a row from vector A are multiplied by the corresponding elements of a column from vector B. For example, the first element from vector A (e.g., A0) is multiplied by the first element from vector B (e.g., B0), and the second element from vector A (e.g., A1) is multiplied by the second element from vector B (e.g., B1). As shown, circuit 112 includes multipliers 320 for each element pair (A0 and B0, A1 and B1, A2 and B2, A3 and B3) to produce results P[0], P[1], P[2], and P[3], respectively. In other embodiments, circuit 110 may include a different number of multipliers 320 to facilitate multiplication based on the number of element pairs from vectors A and B. When an output is generated from one of the multipliers 320, the output is temporarily stored in a second latch 310 until each multiplier 320 has produced its corresponding output. After receiving all the outputs from multiplier 320, second latch 310 releases the result to adder 330.

[0029] Adder 330 is configured to perform addition operations in which the outputs of multiplier 320 are summed. As shown, P[0] and P[1] are used as inputs to the first adder 330 and produce an output of Sum[0]; P[2] and P[3] are used by the second adder 330 and produce an output of Sum[1]. In other embodiments, circuit 112 may include a different number of adders 330 based on the number of outputs received from multiplier 320. After all outputs have been received from the initial two adders 330, third latch 310 releases the result to final adder 330 to add Sum[0] and Sum[1] of circuit 112, thereby producing a dot product of vectors A and B.

[0030] Adder 114 is configured to perform an addition operation in which it adds the output of dot product circuit 112 to a cached accumulated value, represented as result C. In instances where consecutive dependent instructions are being executed, the cached result C from cache 116 is the previous output of adder 114 after being passed to cache 116 via multiplexer 340 for storage and reuse. However, in other cases, multiplexer 340 is configured to select different inputs to route different values ​​(such as the previously refreshed result C or some other value added to the dot product result produced by dot product circuit 112) from data register file 220 to accumulator cache 116.

[0031] Compared to Figure 4 The accumulator cache 116 is described in more detail.

[0032] Now go to Figure 4 A block diagram of accumulator cache 116 is depicted. In the illustrated embodiment, accumulator cache 116 includes a plurality of entries 410, including entries 410A and 410B. In some embodiments, accumulator cache 116 is implemented in a different manner than shown. For example, accumulator cache 116 may store additional entries from dot-matrix accumulator circuitry 110, including additional read ports to read portions of entries 410 at a finer granular level, etc.

[0033] As previously discussed, accumulator cache 116 is a local memory for receiving and storing elements from the dot product (e.g., result C) generated from circuit 110. Because it takes time for accumulator cache 116 to write the dot product back to the registers of data register file 220, the result C of a subsequent dot product may become available for storage in cache 116 before the write-back is complete. To avoid pipeline stalls during write-backs to registers of data register file 220, accumulator cache 116 is double-buffered, allowing cache 116 to read the current result C (e.g., entry 410B) while performing a write-back of a previous result C (e.g., entry 410A). This double buffering can also be used for cache loads, in which cache 116 can read a previously refreshed result C (or some other value used as the accumulation value) from registers of data register file 220 while simultaneously writing the previous result C back to another register. In this case, one entry in entry 410 can provide input to adder 114 for the first instruction, while the second entry 410 is used to write back the output of the second instruction.

[0034] As shown in the figure, each entry 410 can be further divided into a high-order memory bank 402A and a low-order memory bank 402B, which allows for greater read / write granularity matching. In some implementations, this ability to read the high-order and low-order memory banks 402 independently can also allow for earlier eviction of data from the cache 116.

[0035] Now go to Figure 5 A block diagram of intelligent scheduling 500 is depicted. In the illustrated embodiment, intelligent scheduling 500 includes a compiler 510 and a scheduler 210. In some embodiments, intelligent scheduling 500 is implemented in a different manner than shown.

[0036] Compiler 510 is executable software used to compile program instructions 502 written in a high-level language into instructions 512 defined by an ISA supported by IC 10. As noted above, scheduler 210 may not have the ability to identify instruction dependencies (or may have only a limited ability). In the illustrated implementation, compiler 510 is able to execute dot product addition instructions to identify dependencies between them and provides cache hints 514 indicating dependencies on the scheduler to scheduler 210. As shown, compiler 510 receives program instructions 502 that have matrix multiplication dependencies (such as dot product addition dependencies). In response to compiling instruction 502 and identifying the dependency between the first and second instructions, compiler 510 provides corresponding cache hints 514 that indicate that the first and second instructions 512 should be scheduled consecutively to immediately reuse accumulator values. Any suitable method can be used to convey hints 412, such as modifying the opcode of the instruction, modifying operand inputs, etc. In response to receiving a prompt 514 associated with instructions 512A and 512B, scheduler 210 is configured to schedule instructions 512A and 512B one after another on the same point accumulation circuit 110 to ensure that cached results C are reused.

[0037] Now go to Figure 6 A flowchart of method 600 is depicted. Method 600 is one embodiment of a method that can be executed by an integrated circuit device such as integrated circuit 10. In many cases, execution of method 600 can reduce the likelihood of pipeline stalls when a matrix multiplier circuit (e.g., matrix multiplier 100) is executing a dependent dot product addition instruction.

[0038] In step 610, the dot product circuit (e.g., dot product circuit 112) determines the dot product of the first vector and the second vector. In various embodiments, the matrix multiplier circuit includes multiple dot accumulation circuits (e.g., dot accumulation circuit 110). The matrix multiplier circuit passes a first portion of the first matrix and the second matrix to the dot accumulation circuit to compute a first partial result set. While storing the first partial result set in multiple accumulator caches, the matrix multiplier passes a second portion of the first matrix and the second matrix to the dot accumulation circuit to compute a second partial result set. In some embodiments, the dot accumulation circuit performs integer dot accumulation, and the second dot accumulation circuit performs floating-point dot accumulation. In some embodiments, the integrated circuit is a single instruction multiple data (SIMD) processor. In other embodiments, the integrated circuit is a graphics processing unit.

[0039] In step 620, the adder circuit (e.g., adder 114) adds the result of the dot product to the accumulated value. In some embodiments, step 620 may be performed after step 630. In step 630, the accumulator cache (e.g., an accumulator buffer) provides the accumulated value to the adder circuit. In some embodiments, the integrated circuit includes a scheduler circuit (e.g., scheduler 210). The scheduler circuit receives compiled first program instructions (e.g., first ISA instruction 512A) and second program instructions (e.g., second ISA instruction 512B) from a compiler (e.g., compiler 510), having an indication (e.g., cache hint 514) that the second program instructions depend on the dot product result of the first program instructions. The scheduler circuit sequentially schedules the first and second program instructions for execution by the dot product circuit, thereby providing the dot product result to the accumulator cache as an input operand for executing the second program instructions.

[0040] In step 640, the accumulator cache stores the result of the addition as a subsequent accumulated value for use in subsequent point-accumulation operations. The accumulator cache stores the result of the addition in a first entry (e.g., entry 410B) while simultaneously writing the previously stored result from a second entry (e.g., entry 410A) of the accumulator cache back to the register file (e.g., data register file 220). The accumulator cache provides the stored result to the adder circuitry in both point-accumulation circuits. The register file circuitry stores the values ​​of a first matrix and a second matrix, which include the first and second vectors. The accumulator cache is located closer to the adder than the register file circuitry.

[0041] Now go to Figure 7 A flowchart of method 700 is depicted. Method 700 is one embodiment of a method that can be executed by an integrated circuit device such as integrated circuit 10. In many cases, execution of method 700 can reduce the likelihood of pipeline failure when a matrix multiplier circuit (e.g., matrix multiplier 100) is executing a dependent dot product addition instruction.

[0042] In step 710, the compiler (e.g., compiler 510) receives program instructions (e.g., program instruction 502) for including an operation that performs matrix multiplication. In step 720, the compiler determines that the specific implementation of the operation includes performing a second point accumulation that depends on the first point accumulation. In step 730, based on this determination, the compiler provides compiled instructions that include indications that the second point accumulation will be scheduled sequentially after the first point accumulation (e.g., cache hint 514) so ​​that a cache coupled to the point accumulation circuitry that performs the first point accumulation provides the result of the first point accumulation to the point accumulation circuitry as an input operand for the second point accumulation.

[0043] Exemplary computer system

[0044] Now for reference Figure 8 The diagram illustrates a block diagram of an example embodiment of device 800. In some embodiments, device 800 may include (or correspond to) integrated circuit 10 and / or implement the functionality of matrix multiplier 100. In some embodiments, the components of device 800 may be included within a system-on-a-chip. In some embodiments, device 800 may be included in a mobile computing device that may be battery-powered. Therefore, the power consumption of device 800 may be an important design consideration. In the illustrated embodiment, device 800 includes a structure 810, a computing complex 820, an input / output (I / O) bridge 860, a cache / memory controller 830, a graphics unit 840, and a display unit 850. In some embodiments, as supplements to or alternatives to the illustrated components, device 800 may also include other components (not shown), such as video processor encoders and decoders, image processing or recognition elements, computer vision elements, etc.

[0045] Structure 810 may include various interconnects, buses, MUXs, controllers, etc., and may be configured to facilitate communication between various components of device 800. In some embodiments, portions of structure 810 may be configured to implement various different communication protocols. In other embodiments, structure 810 may implement a single communication protocol, and components coupled to structure 810 may internally switch from a single communication protocol to other communication protocols.

[0046] In the illustrated implementation, computing complex 820 includes a bus interface unit (BIU) 822, a cache 824, and cores 826A through 826B. In various implementations, computing complex 820 may include a variety of numbers of processors, processor cores, and caches. For example, computing complex 820 may include one, two, four, or any other suitable number of processor cores. In one implementation, cache 824 is a set-associative L2 cache. In some implementations, cores 826A through 826B may include internal instruction and data caches. In some implementations, coherence units (not shown) in architecture 810, cache 824, or elsewhere in device 800 may be configured to maintain coherence between the various caches of device 800. BIU 822 may be configured to manage communication between computing complex 820 and other elements of device 800. Processor cores such as cores 826A through 826B may be configured to execute instructions of a specific instruction set architecture (ISA), which may include operating system instructions and user application instructions. These instructions can be stored in a computer-readable medium, such as memory coupled to the memory controller 830 discussed below.

[0047] As used herein, the term "coupled to" can indicate one or more connections between elements, and coupling may include intermediate elements. For example, in Figure 8 In this context, the graphics unit 840 can be described as being "coupled" to memory via structure 810 and cache / memory controller 830. In contrast, in... Figure 8 In the exemplary implementation, the graphics unit 840 is "directly coupled" to the structure 810 because there are no intermediate elements.

[0048] The cache / memory controller 830 can be configured to manage data transfer between the structure 810 and one or more caches and memories. For example, the cache / memory controller 830 may be coupled to an L3 cache, which in turn may be coupled to system memory. In other embodiments, the cache / memory controller 830 may be directly coupled to memory. In some embodiments, the cache / memory controller 830 may include one or more internal caches. The memory coupled to the controller 830 can be any type of volatile memory, such as dynamic random access memory (DRAM), synchronous DRAM (SDRAM), dual data rate (DDR, DDR2, DDR3, etc.) SDRAM (including mobile versions of SDRAM, such as mDDR3, etc., and / or low-power versions of SDRAM, such as LPDDR4, etc.), RAMBUS DRAM (RDRAM), static RAM (SRAM), etc. One or more memory devices may be coupled onto a circuit board to form a memory module, such as a single in-line memory module (SIMM), a dual in-line memory module (DIMM), etc. Alternatively, these devices can be mounted with integrated circuits in a chip-stacked configuration, a package-stacked configuration, or a multi-chip module configuration. The memory coupled to the controller 830 can be any type of non-volatile memory, such as NAND flash memory, NOR flash memory, nanometer RAM (NRAM), magnetoresistive RAM (MRAM), phase-change RAM (PRAM), raceway memory, memristor memory, etc. As noted above, the memory can store program instructions (such as compiler 510) that can be executed by the computing complex 820 to cause the device 800 to perform the functionalities described herein.

[0049] The graphics unit 840 may include one or more processors, such as one or more graphics processing units (GPUs). For example, the graphics unit 840 may receive graphics-oriented instructions, such as OpenGL. ® Metal ® Or DIRECT3D ®Instructions. The graphics unit 840 can execute dedicated GPU instructions or perform other operations based on received graphics-oriented instructions. The graphics unit 840 is typically configured to process large blocks of data in parallel and can build an image in a frame buffer for output to a display, which may be included in a device or may be a separate device. The graphics unit 840 may include a transformation engine, a lighting engine, a triangle engine, and a rendering engine in one or more graphics processing pipelines. The graphics unit 840 can output pixel information for displaying the image. In various embodiments, the graphics unit 840 may include programmable shader circuitry, which may include highly parallel execution cores configured to execute graphics programs, which may include pixel tasks, vertex tasks, and computation tasks (which may be graphics-dependent or may not be graphics-dependent). In some embodiments, the graphics unit 840 includes the matrix multiplier 100 discussed herein.

[0050] Display unit 850 can be configured to read data from a frame buffer and provide a stream of pixel values ​​for display. In some embodiments, display unit 850 can be configured as a display pipeline. Additionally, display unit 850 can be configured to blend multiple frames to produce an output frame. Furthermore, display unit 850 may include one or more interfaces (e.g., MIPI) for coupling to a user display (e.g., a touchscreen or an external display). ® Or embedded display port (eDP)).

[0051] I / O bridge 860 may include various components configured to implement functionalities such as Universal Serial Bus (USB) communication, security, audio, and low-power always-on connectivity. I / O bridge 860 may also include interfaces such as pulse-width modulation (PWM), general-purpose input / output (GPIO), serial peripheral interface (SPI), and internal integrated circuit (I2C). Various types of peripheral devices and equipment can be coupled to device 800 via I / O bridge 860.

[0052] In some implementations, device 800 includes network interface circuitry (not explicitly shown) that can be connected to structure 810 or I / O bridge 860. The network interface circuitry can be configured to communicate via various networks, which can be wired networks, wireless networks, or both. For example, the network interface circuitry can be configured to communicate via a wired local area network (LAN), a wireless LAN (e.g., via Wi-Fi), or Wi-Fi. ™ The network interface circuitry communicates via one or more cellular networks using one or more radio access technologies, or wide area networks (e.g., the Internet or a virtual private network). In some embodiments, the network interface circuitry is configured to communicate via one or more cellular networks using one or more radio access technologies. In some embodiments, the network interface circuitry is configured to use device-to-device communication (e.g., Bluetooth). ®or Wi-Fi ™ Communication can be achieved through direct connection, etc. In various implementations, this network interface circuitry can provide device 800 with connectivity to various types of other devices and networks.

[0053] Example Application

[0054] Now go to Figure 9 This can include various types of systems, including any of the circuits, devices, or systems discussed above. The system or device 900, which may incorporate or otherwise utilize one or more of the techniques described herein, can be used in a wide range of fields. For example, the system or device 900 can be used as part of the hardware of a system such as a desktop computer 910, a laptop computer 920, a tablet computer 930, a cellular or mobile phone 940, or a television 950 (or a set-top box coupled to a television).

[0055] Similarly, the disclosed components can be used in wearable devices 960, such as smartwatches or health monitoring devices. In many embodiments, a smartwatch can perform a variety of different functions—for example, access to email, cellular services, calendars, health monitoring, etc. Wearable devices can also be designed to perform only health monitoring functions, such as monitoring a user's vital signs, performing epidemiological functions such as contact tracing, providing communication to emergency medical services, etc. Other types of devices are also envisioned, including devices worn around the neck, implantable devices, and glasses or helmets designed to provide computer-generated reality experiences, such as those based on augmented reality and / or virtual reality.

[0056] System or device 900 can also be used in a variety of other contexts. For example, system or device 900 can be used in the context of a server computer system (such as a dedicated server) or on shared hardware implementing cloud-based services 970. Furthermore, system or device 900 can be implemented in a wide range of dedicated everyday devices, including common household devices 980 such as refrigerators, thermostats, security cameras, etc. The interconnection of such devices is often referred to as the “Internet of Things” (IoT). Components can also be implemented in various modes of transportation. For example, system or device 900 can be used in control systems, guidance systems, entertainment systems, etc., of various types of vehicles 990.

[0057] Figure 9 The applications illustrated are merely exemplary and are not intended to limit the potential future applications of the disclosed systems or devices. Other example applications include, but are not limited to, portable gaming devices, music players, data storage devices, and unmanned aerial vehicles.

[0058] Example computer-readable media

[0059] Various example circuits have been described in detail above in this disclosure. It is intended that this disclosure cover not only embodiments including such circuits, but also computer-readable storage media that include design information specifying such circuits. Therefore, this disclosure is intended to support claims that cover not only devices including the disclosed circuits, but also storage media specifying such circuits in a format recognized by a computing system configured to generate a simulation model of hardware circuitry, a manufacturing system configured to generate hardware (e.g., integrated circuits) including the disclosed circuits. Claims regarding such storage media are intended to cover entities that, for example, generate circuit designs but do not themselves perform complete operations (such as design simulation, design synthesis, circuit fabrication, etc.).

[0060] Now go to Figure 10 The diagram depicts a block diagram of an example non-transitory computer-readable storage medium storing circuit design information. In an illustrated embodiment, a computing system 1040 is configured to process the design information. This may include executing instructions included in the design information, interpreting instructions included in the design information, compiling, transforming, or otherwise updating the design information. Thus, in some embodiments, the design information (e.g., through programming the computing system 1040) controls the computing system 1040 to perform the various operations discussed below.

[0061] In the illustrated example, computing system 1040 processes the design information to generate both a computer simulation model of hardware circuitry 1060 and lower-level design information 1050. In other embodiments, computing system 1040 may generate only one of these outputs, may generate other outputs based on the design information, or both. Regarding the computational simulation, computing system 1040 may execute instructions in a hardware description language, which includes register-transfer-level (RTL) code, behavioral code, structural code, or some combination thereof. The simulation model may perform the functionality specified by the design information, facilitate verification of the functional correctness of the hardware design, generate power consumption estimates, generate timing estimates, etc.

[0062] In the illustrated example, computing system 1040 also processes the design information to generate lower-level design information 1050 (e.g., gate-level design information, netlists, etc.). As shown, this may include synthesis operations such as building multi-level networks, optimizing the network using technology-independent techniques, technology-dependent techniques, or both, and outputting a gate network (with potential constraints based on a library of available gate pairs, size adjustments, delays, power, etc.). Based on the lower-level design information 1050 (and potentially other inputs), semiconductor manufacturing system 1020 is configured to manufacture integrated circuit 1030 (which may correspond to the functionality of simulation model 1060). Note that computing system 1040 may generate different simulation models based on design information at various descriptive levels (including information 1050, 1015, etc.). Data representing design information 1050 and model 1060 may be stored on medium 1010 or one or more other media.

[0063] In some implementations, lower-level design information 1050 controls (e.g., programs) the semiconductor manufacturing system 1020 to manufacture integrated circuit 1030. Therefore, when processed by the manufacturing system, the design information can program the manufacturing system to manufacture circuits including the various circuits disclosed herein.

[0064] The non-transitory computer-readable storage medium 1010 may include any of a variety of suitable types of memory devices or storage devices. The non-transitory computer-readable storage medium 1010 may be an installation medium, such as a CD-ROM, floppy disk, or magnetic tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory; magnetic media, such as hard disk drives or optical storage devices; registers, or other similar types of memory elements. The non-transitory computer-readable storage medium 1010 may also include other types of non-transitory memory or combinations thereof. Therefore, the non-transitory computer-readable storage medium 1010 may include two or more memory media; such media may reside in different locations—for example, in different computer systems connected via a network.

[0065] Design information 1015 can be specified using any of a variety of suitable computer languages, including hardware description languages ​​such as, but not limited to, VHDL, Verilog, SystemC, SystemVerilog, RHDL, M, MyHDL, etc. The formats of various design information can be recognized by one or more applications executed by computing system 1040, semiconductor manufacturing system 1020, or both. In some embodiments, design information may also include one or more cell libraries specifying the composition, layout, or both of integrated circuit 1030. In some embodiments, design information is specified wholly or partially in the form of a netlist specifying cell library elements and their connectivity. Individually, the design information discussed herein may or may not include sufficient information for manufacturing the corresponding integrated circuit. For example, design information may specify circuit elements to be manufactured but not their physical layout. In this case, design information may need to be combined with layout information to actually manufacture the specified circuit.

[0066] In various implementations, the integrated circuit 1030 may include one or more custom macrocells, such as memory and analog or mixed-signal circuitry. In this case, design information may include information associated with the included macrocells. Such information may include, but is not limited to, schematic capture databases, mask design data, behavioral models, and device or transistor-level netlists. Mask design data may be formatted according to a Graphical Data System (GDSII) or any other suitable format.

[0067] The semiconductor manufacturing system 1020 may include any of the various suitable elements configured to manufacture integrated circuits. This may include elements for, for example, depositing semiconductor material (e.g., on a wafer that may include a mask), removing material, changing the shape of the deposited material, modifying the material (e.g., by doping the material or by using ultraviolet treatment to modify the dielectric constant), etc. The semiconductor manufacturing system 1020 may also be configured to perform various tests on the manufactured circuits to ensure proper operation.

[0068] In various embodiments, integrated circuit 1030 and model 1060 are configured to operate according to a circuit design specified by design information 1015, which may include performing any of the functionalities described herein. For example, integrated circuit 1030 may include Figures 1 to 8 Any of the various components shown. Additionally, integrated circuit 1030 can be configured to perform the various functions described herein in conjunction with other components. Furthermore, the functionality described herein can be performed by multiple interconnected integrated circuits.

[0069] As used herein, a phrase in the form of “design information specifying a circuit configured to…” does not imply that the circuit in question must be manufactured to satisfy the element. Rather, the phrase indicates that the design information describes a circuit that, when manufactured, will be configured to perform the indicated actions or will include the specified components. Similarly, stating that “instructions of a hardware description programming language” are “executable” to program a computing system to generate a computer simulation model does not mean that the instructions must be executed to satisfy the element, but rather specifies the characteristics of those instructions. In this context, additional features associated with the model (or the circuit represented by the model) can similarly relate to the characteristics of these instructions. Therefore, an entity selling a computer-readable medium having instructions that satisfy the stated characteristics may provide an infringing product even if another entity actually executes those instructions on the medium.

[0070] It should be noted that, at least in the context of digital logic, a given design can be implemented using a variety of different gate arrangements, circuit techniques, etc. However, once a digital logic design is specified, those skilled in the art do not need to perform extensive experiments or studies to determine those specific implementations. Instead, they understand the process for reliably and predictably producing one or more circuit implementations that provide the functionality described by the design information. Different circuit implementations may affect the performance, area, power consumption, etc., of a given design (potentially involving trade-offs between different design goals), but the logical function remains unchanged between different circuit implementations of the same circuit design.

[0071] In some implementations, instructions included in the design information instructions provide RTL information (or other higher-level design information) and can be executed by a computing system to synthesize a gate-level netlist representing the hardware circuitry based on the RTL information as input. Similarly, these instructions provide behavioral information and can be executed by the computing system to synthesize netlists or other lower-level design information. This lower-level design information can be used to program the manufacturing system 1020 to manufacture the integrated circuit 1030.

[0072] ***

[0073] This disclosure includes references to “one implementation” or “a group of implementations” (e.g., “some implementations” or “various implementations”). An implementation is a different specific embodiment or instance of the disclosed concepts. References to “implementation,” “one implementation,” and “specific implementation,” etc., do not necessarily refer to the same implementation. A large number of possible implementations are contemplated, including those specifically disclosed, as well as modifications or alternatives that fall within the substance or scope of this disclosure.

[0074] This disclosure may discuss potential advantages that may arise from the disclosed embodiments. Not all specific implementations of all these embodiments will necessarily exhibit any or all of the potential advantages. Whether a particular embodiment achieves an advantage depends on many factors, some of which are outside the scope of this disclosure. In fact, there are many reasons why an embodiment falling within the scope of the claims may not exhibit some or all of any of the disclosed advantages. For example, a particular embodiment may include other circuitry outside the scope of this disclosure, in conjunction with one embodiment of the disclosed embodiments, which negates or diminishes one or more of the disclosed advantages. Furthermore, suboptimal design execution of a particular embodiment (e.g., the implementing technique or tool) may also negate or diminish the disclosed advantages. Even assuming an implementation of the technique, the realization of advantages may still depend on other factors, such as the environmental circumstances in which the implementation is deployed. For example, the inputs provided to a particular embodiment may prevent one or more problems addressed in this disclosure from occurring in a particular context, and as a result, the benefits of its solution may not be realized. In view of the existence of possible factors outside this disclosure, it is hereby expressed that any potential advantages described herein should not be construed as a limitation of the claims that must be satisfied in order to prove infringement. Rather, the identification of such potential advantages is intended to illustrate the types of improvements available to the designer who benefits from this disclosure. Permanently describing such advantages (e.g., stating that a particular advantage "may occur") is not intended to convey a question about whether such advantages can actually be realized, but rather to recognize that the realization of such advantages often depends on the technological reality of additional factors.

[0075] Unless otherwise stated, the embodiments are non-limiting. That is, the disclosed embodiments are not intended to limit the scope of the claims drafted based on this disclosure, even where only a single example is described with respect to a particular feature. The disclosed embodiments are intended to be illustrative and not restrictive, without any statement to the contrary in this disclosure. Therefore, this application is intended to allow for claims covering the disclosed embodiments, as well as such alternatives, modifications, and equivalents, which will be apparent to those skilled in the art to the advantage of this disclosure.

[0076] For example, features in this application can be combined in any suitable manner. Therefore, new claims may be made for any such combination of features during the filing of this application (or an application claiming priority thereto). Specifically, referring to the appended claims, features of dependent claims may be combined with features of other dependent claims, including claims dependent on other independent claims, where appropriate. Similarly, features from the respective independent claims may be combined where appropriate.

[0077] Thus, although the appended dependent claims may be drafted such that each dependent claim depends from a single other claim, additional dependencies are also contemplated. Any combination of dependent features consistent with the present disclosure is contemplated, and such combinations may be claimed in this application or in another application. In short, the combinations are not limited to those specifically recited in the appended claims.

[0078] In appropriate circumstances, it is also contemplated that a claim drafted in one format or statutory type (e.g., apparatus) is intended to support a corresponding claim in another format or statutory type (e.g., method).

[0079] ***

[0080] Since the present disclosure is a legal document, various terms and phrases may be subject to regulatory and judicial interpretation. Notice is hereby given that the following paragraphs, as well as the definitions provided throughout the present disclosure, will be used to determine how claims drafted based on the present disclosure are to be interpreted.

[0081] References to items in the singular form (i.e., a noun or noun phrase preceded by "a," "an," or "the") are intended to mean "one or more" unless the context clearly dictates otherwise. Thus, in the absence of accompanying context, a reference to an "item" in a claim does not exclude additional instances of that item. A "plurality" of items refers to a collection of two or more items in the set of items.

[0082] The word "may" is used herein in an enabling sense (i.e., having the potential to be able to), rather than in a mandatory sense (i.e., must).

[0083] The terms "comprising" and "including" and their forms are open-ended and mean "including but not limited to."

[0084] When the term "or" is used in the present disclosure with respect to a list of options, it will generally be understood to be used in an inclusive sense unless the context provides otherwise. Thus, the statement "x or y" is equivalent to "x or y, or both," and thus encompasses 1) x but not y, 2) y but not x, and 3) both x and y. On the other hand, phrases such as "either x or y, but not both" make clear that "or" is used in an exclusive sense.

[0085] The expressions “w, x, y, or z, or any combination thereof” or “...at least one of w, x, y, and z” are intended to cover all possibilities involving a single element up to the total number of elements in the set. For example, given the set [w, x, y, z], these phrases cover any single element in the set (e.g., w but not x, y, or z), any two elements (e.g., w and x, but not y or z), any three elements (e.g., w, x, and y, but not z), and all four elements. The phrase “...at least one of w, x, y, and z” therefore refers to at least one element in the set [w, x, y, z], thus covering all possible combinations of that list of elements. This phrase should not be interpreted as requiring the existence of at least one instance of w, at least one instance of x, at least one instance of y, and at least one instance of z.

[0086] In this disclosure, various “labels” may precede nouns or noun phrases. Unless the context otherwise provides, different labels used for features (e.g., “first circuit,” “second circuit,” “specific circuit,” “given circuit,” etc.) refer to different instances of the feature. Additionally, unless otherwise stated, the labels “first,” “second,” and “third” do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) when applied to features.

[0087] The phrase "based on" is used to describe one or more factors that influence the determination. This term does not exclude the possibility that additional factors might influence the determination. That is, the determination may be based solely on the specified factors or on the specified factors along with other unspecified factors. Consider the phrase "A is determined based on B." This phrase specifies that B is a factor used to determine A or that B influences the determination of A. This phrase does not exclude the possibility that the determination of A may also be based on some other factor, such as C. This phrase is also intended to cover implementations where A is determined solely based on B. As used herein, the phrase "based on" is synonymous with the phrase "at least partially based on."

[0088] The phrases “responding to” and “responding” describe one or more factors that trigger an effect. This phrase does not exclude the possibility that additional factors may influence or otherwise trigger the effect, whether these factors are used in conjunction with or independently of the specified factor. That is, the effect may respond solely to these factors, or it may respond to the specified factor as well as other unspecified factors. Consider the phrase “responding to B and executing A.” This phrase specifies that B is a factor that triggers the execution of A or triggers a specific result of A. This phrase does not exclude that the execution of A may also respond to certain other factors, such as C. This phrase also does not exclude that the execution of A may be jointly executed in response to B and C. This phrase is also intended to cover implementation schemes where A is executed solely in response to B. As used herein, the phrase “responding” is synonymous with the phrase “at least partially responding to.” Similarly, the phrase “responding to” is synonymous with the phrase “at least partially responding to.”

[0089] ***

[0090] Within this disclosure, different entities (which may be referred to differently as “units,” “circuits,” other components, etc.) may be described or protected by the claims as being “configured” to perform one or more tasks or operations. This expression—[entity] configured to [perform one or more tasks]—is used herein to refer to a structure (i.e., a tangible thing). More specifically, this expression is used to indicate that the structure is arranged to perform one or more tasks during operation. A structure may be considered “configured” to perform a task even if the structure is not currently being operated. Therefore, an entity described or stated as being “configured” to perform a task refers to tangible things such as devices, circuits, systems with processor units, and memory storing program instructions executable to perform the task. This phrase is not used herein to refer to intangible things.

[0091] In some cases, various units / circuits / components may be described herein as a collection of entities that perform tasks or operations. It should be understood that these entities are "configured" to perform those tasks / operations, even if not specifically stated otherwise.

[0092] The term "configured as" is not intended to mean "able to be configured as." For example, an unprogrammed FPGA is not considered "configured as" to perform a specific function. However, the unprogrammed FPGA can be "able to be configured as" to perform that function. After proper programming, the FPGA can then be considered "configured as" to perform a specific function.

[0093] For the purposes of this U.S. patent application based on this disclosure, the statement in the claims that a structure is “configured” to perform one or more tasks is expressly intended not to invoke 35 USC § 112(f) for that claim element. If an applicant wishes to invoke part 112(f) during the filing of a U.S. patent application based on this disclosure, it will use the “component for [performing a function]” structure to state the elements of the claims.

[0094] Different “circuits” may be described in this disclosure. These circuits, or “circuits”, constitute hardware that includes various types of circuit elements, such as combinational logic, clock storage devices (e.g., flip-flops, registers, latches, etc.), finite state machines, memories (e.g., random access memory, embedded dynamic random access memory), programmable logic arrays, etc. Circuits may be custom-designed or taken from standard libraries. In various specific implementations, circuits may include digital components, analog components, or a combination of both, depending on the circumstances. Certain types of circuits may be commonly referred to as “cells” (e.g., decoding units, arithmetic logic units (ALUs), functional units, memory management units (MMUs), etc.). Such cells also refer to circuits or circuitry.

[0095] Therefore, the circuits / units / components and other elements illustrated in the accompanying drawings and described herein include hardware elements, such as those described in the preceding paragraphs. In many cases, the internal arrangement of hardware elements in a particular circuit can be specified by describing the function of that circuit. For example, a particular “decoding unit” can be described as having the function of executing “the opcode of a processing instruction and routing that instruction to one or more of a plurality of functional units,” meaning that the decoding unit is “configured” to perform that function. To those skilled in the art of computers, this functional specification is sufficient to suggest a set of possible structures for the circuit.

[0096] In various implementations, as discussed in the preceding paragraphs, circuits, cells, and other elements can be defined by the functions or operations they are configured to perform. The arrangement of these circuits / cells / components relative to each other and the manner in which they interact form a microarchitecture definition of hardware, which is ultimately manufactured in an integrated circuit or programmed into an FPGA to form a physical implementation of the microarchitecture definition. Therefore, a microarchitecture definition is considered by those skilled in the art to be a structure from which many physical implementations can be derived, all of which fall within the broader structure described by the microarchitecture definition. That is, those skilled in the art, with the microarchitecture definition provided according to this disclosure, can implement this structure without excessive experimentation and using the application of a person of ordinary skill in the art, by encoding the description of the circuits / cells / components in a hardware description language (HDL) such as Verilog or VHDL. The HDL description is often expressed in a way that can be revealed as functional. However, for those skilled in the art, the HDL description is a way of translating the structure of a circuit, cell, or component into the details of the next level of implementation. Such HDL descriptions can take the following forms: behavioral code (which is typically non-synthesizable), Register Transfer Language (RTL) code (which is typically synthesizable compared to behavioral code), or structural code (e.g., a netlist specifying logic gates and their connectivity). HDL descriptions can be sequentially synthesized against a library of cells designed for a given integrated circuit manufacturing technology and can be modified for timing, power, and other reasons to obtain a final design database that is sent to the factory to generate masks and ultimately produce integrated circuits. Some hardware circuitry or portions thereof can also be custom-designed in a schematic editor and captured into the integrated circuit design along with the synthesized circuitry. The integrated circuit may include transistors and other circuit elements (e.g., passive components such as capacitors, resistors, inductors, etc.), as well as interconnects between transistors and circuit elements. Some implementations may implement multiple integrated circuits coupled together to implement the hardware circuitry, and / or discrete components may be used in some implementations. Alternatively, the HDL design can be synthesized into a programmable logic array such as a Field Programmable Gate Array (FPGA) and implemented within the FPGA. This decoupling between the design of a set of circuits and their subsequent low-level implementations often results in a situation where the circuit or logic designer never specifies a particular set of structures for the low-level implementation that goes beyond a description of what the circuit is configured to do, because that process is performed at different stages of the circuit implementation process.

[0097] The fact that a circuit can be implemented to the same specifications using many different low-level combinations of circuit elements results in a large number of equivalent circuit structures. As noted, these low-level circuit implementations can vary depending on the manufacturing technology, the foundry chosen to manufacture the integrated circuit, the cell library provided for a particular project, and so on. In many cases, the choice of different design tools or methods to produce these different implementations can be arbitrary.

[0098] Furthermore, for a given implementation, a single concrete implementation of the circuit's specific functional specifications typically involves a large number of devices (e.g., millions of transistors). Therefore, the shearing volume of this information makes it impractical to provide a complete description of the low-level structure used to implement a single implementation, let alone a large number of equivalent possible implementations. To this end, this disclosure describes the structure of a circuit using functional abbreviations commonly used in industry.

Claims

1. An integrated circuit, the integrated circuit comprising: Point accumulation circuit, the point accumulation circuit includes: Dot product circuit, the dot product circuit being configured to determine the dot product of a first vector and a second vector; and An adder circuit, coupled to the output of the dot product circuit and configured to add the result of the dot product to the accumulated value; and An accumulator cache, coupled to the input and output of the adder circuit, wherein the accumulator cache is configured as follows: The accumulated value is provided to the adder circuit; and The result of the addition is stored as a subsequent accumulated value for use in subsequent point-to-point accumulation operations.

2. The integrated circuit of claim 1, wherein the accumulator cache is configured as follows: The result of the addition is stored in the first entry of the accumulator cache, while the previously stored result is written back to the register file from the second entry of the accumulator cache.

3. The integrated circuit according to claim 1, further comprising: A matrix multiplier circuit configured to multiply a first matrix and a second matrix, wherein the matrix multiplier circuit includes multiple point accumulation circuits.

4. The integrated circuit of claim 3, wherein the matrix multiplier circuit is configured as follows: The first matrix and a first portion of the second matrix are transmitted to the point accumulation circuit to compute a first set of results; and While storing the first partial result set in multiple accumulator caches, the second part of the first matrix and the second matrix are transmitted to the point accumulation circuit to compute the second partial result set.

5. The integrated circuit according to claim 1, further comprising: Scheduler circuit, the scheduler circuit being configured to: Receives compiled first program instructions and second program instructions, the compiled first program instructions and second program instructions having an indication from the compiler of a point-accumulation result of the second program instructions depending on the first program instructions; as well as The first program instruction and the second program instruction are continuously scheduled for execution by the point accumulation circuit, thereby providing the point accumulation result in the accumulator cache as an input operand for executing the second program instruction.

6. The integrated circuit according to claim 1, further comprising: Register file circuitry, wherein the register file circuitry is configured as follows: Store the values ​​of a first matrix and a second matrix, which include the first vector and the second vector; and The accumulator cache is located closer to the adder circuit than the register file circuit.

7. The integrated circuit of claim 1, wherein the point accumulation circuit is configured to perform integer point accumulation; and The integrated circuit further includes a second point accumulation circuit configured to perform floating-point point accumulation.

8. The integrated circuit of claim 7, wherein the accumulator cache is configured as follows: The stored result is provided to the adder circuit in the two-point accumulation circuit.

9. The integrated circuit of claim 1, wherein the integrated circuit is a single instruction multiple data (SIMD) processor.

10. The integrated circuit of claim 1, wherein the integrated circuit is a graphics processing unit.

11. A method, the method comprising: Point accumulation is performed by a computing device, the point accumulation including: Determine the dot product of the first and second vectors; and The dot product result is added to an accumulated value by an adder circuit, wherein the accumulated value is provided by an accumulator cache coupled to the adder circuit; and The result of the addition is stored in the accumulator cache as the subsequent accumulation value for use in subsequent point accumulation operations.

12. The method of claim 11, wherein the storage comprises: The result of the addition is stored in the first entry of the accumulator cache, while the previously stored result is written back from the second entry of the accumulator cache to the register file circuit of the computing device.

13. The method of claim 12, wherein the accumulator cache is located closer to the adder circuit than the register file circuit.

14. The method according to claim 11, further comprising: The computing device multiplies a first matrix and a second matrix, which include the first vector and the second vector, wherein the multiplication includes performing the point accumulation.

15. The method according to claim 11, further comprising: The computing device receives compiled first program instructions and second program instructions, the compiled first program instructions and second program instructions having an indication from the compiler of a point-accumulation result of the second program instructions depending on the first program instructions; as well as The computing device continuously schedules the first program instructions and the second program instructions for execution, thereby providing the point accumulation result in the accumulator cache as an input operand for executing the second program instructions.

16. A non-transitory computer-readable medium storing instructions of a hardware description programming language, which, when processed by a computing system, program the computing system to generate a computer simulation model, wherein the model represents hardware circuitry comprising: Point accumulation circuit, the point accumulation circuit includes: Dot product circuit, the dot product circuit being configured to determine the dot product of a first vector and a second vector; and An adder circuit, coupled to the output of the dot product circuit and configured to add the result of the dot product to the accumulated value; and An accumulator cache, coupled to the input and output of the adder circuit, wherein the accumulator cache is configured as follows: The accumulated value is provided to the adder circuit; and The result of the addition is stored as a subsequent accumulated value for use in subsequent point-to-point accumulation operations.

17. The computer-readable medium of claim 16, wherein the accumulator cache is configured as follows: The result of the addition is stored in the first entry of the accumulator cache, while the previously stored result is written back to the register file from the second entry of the accumulator cache.

18. The computer-readable medium of claim 16, wherein the hardware circuitry comprises: A matrix multiplier circuit configured to multiply a first matrix and a second matrix, wherein the matrix multiplier circuit includes multiple point accumulation circuits.

19. The computer-readable medium of claim 16, wherein the hardware circuitry comprises: Scheduler circuit, the scheduler circuit being configured to: Receive from the compiler an indication of the point-accumulation result of the second program instruction depending on the first program instruction; as well as The first program instruction and the second program instruction are continuously scheduled for execution by the point accumulation circuit.

20. The computer-readable medium of claim 16, wherein the hardware circuitry comprises: Register file circuitry, wherein the register file circuitry is configured as follows: The first vector and the second vector are stored, wherein the accumulator cache is closer to the adder circuit than the register file circuit.

Images