Streaming engine using early and late address and loop count registers to track architectural state

Abstract

The present application discloses a stream engine that uses early and late address and loop count registers to track architectural state. A stream engine (125, 200) for use in a digital data processor (100) specifies a fixed, read-only data stream defined by multiple nested loops. An address generator (1901) generates addresses for data elements. A stream head register (2718, 2728) stores the data element to be provided next to a functional unit for use as an operand. The stream engine (125, 2700) stores, for each of the nested loops, an early address of the data element to be retrieved next and a late address of the data element in the stream head register (2718, 2728). The stream engine (125, 2700) stores, for each of the nested loops, an early loop count (3701-3704) of the data element to be retrieved next and a late loop count (3705-3708) of the data element in the stream head register (2718, 2728).

Description

[0001] This application is a divisional application of Chinese patent application 201711379376.8, filed on December 20, 2017, entitled “Streaming engine using early and late address and loop count registers to track architectural state”.

[0002] Related applications

[0003] The patent application of this application is an improvement to U.S. patent application serial number 14 / 331,986, filed on July 15, 2014, entitled "HIGHLY INTEGRATED SCALABLE, FLEXIBLE DSP MEGAMODULE ARCHITECTURE," which claims priority to U.S. provisional patent application serial number 61 / 846,148, filed on July 15, 2013. Technical Field

[0004] The technical field of the invention is digital data processing, and more particularly the control of streaming engines for operand retrieval. Background Art

[0005] Modern digital signal processors (DSPs) face multiple challenges. Workloads continue to grow, requiring ever-increasing bandwidth. System-on-chip (SoCs) continue to increase in size and complexity. Memory system latency severely impacts certain classes of algorithms. As transistors get smaller, memory and registers become less reliable. And as software stacks grow larger, the number of potential interactions and errors increases.

[0006] Memory bandwidth and scheduling are issues for digital signal processors operating on real-time data. A digital signal processor operating on real-time data typically receives an input data stream, performs a filtering function (such as encoding or decoding) on ​​the data stream and outputs a converted data stream. The system is called real-time because if the converted data stream is not available for output when scheduled, the application fails. Typical video encoding requires a predictable but non-contiguous input data pattern. The corresponding memory access is often difficult to implement within the available address generation and memory access resources. A typical application requires memory access to load data registers in a data register file and then provide it to the functional unit that performs the data processing. Summary of the Invention

[0007] The present invention is a stream engine for use in a digital signal processor. A fixed data stream sequence is specified by storing corresponding parameters in control registers. The data stream comprises multiple nested loops. Once started, the data stream is read-only and cannot be written to. This is often required for real-time filtering operations.

[0008] The stream engine includes an address generator that generates data element addresses and a stream head register that stores the data elements to be provided to the functional unit as operands. The stream engine retrieves the stream data from the stream buffer before it is used by the CPU core.

[0009] The stream engine includes an early address register that stores the stream memory address of the data element to be retrieved from the memory next, and a late address register that stores the stream memory address corresponding to the data element stored in the stream head register. Both the early address register and the late address register store addresses for nested loops.

[0010] The stream engine includes an early count register that stores a count for each nested loop corresponding to a data element to be next retrieved from memory, and a late count register that stores a count for each nested loop corresponding to a data element stored in the stream head register. Both the early count register and the late count register initially store data corresponding to a loop count that changes the memory address by an amount equal to the loop count of the corresponding nested loop specified in a stream template register. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] These and other aspects of the invention are illustrated in the accompanying drawings, in which:

[0012] Figure 1 A dual scalar / vector datapath processor according to one embodiment of the present invention is shown;

[0013] Figure 2 Shown Figure 1 Registers and functional units in a dual scalar / vector datapath processor as shown in;

[0014] Figure 3 The global scalar register file is shown;

[0015] Figure 4 shows a local scalar register file shared by the arithmetic functional units;

[0016] Figure 5shows a local scalar register file shared by the multiplication functional units;

[0017] Figure 6 shows a local scalar register file shared by the load / store units;

[0018] Figure 7 The global vector register file is shown;

[0019] Figure 8 shows a predicate register file;

[0020] Figure 9 shows a local vector register file shared by the arithmetic functional units;

[0021] Figure 10 shows a local vector register file shared by the multiplication and related functional units;

[0022] Figure 11 shows the pipeline stages of a central processing unit according to a preferred embodiment of the present invention;

[0023] Figure 12 16 instructions for a single fetch packet are shown;

[0024] Figure 13 An example of instruction encoding of instructions used by the present invention is shown;

[0025] Figure 14 The bit encoding of condition code extension slot 0 is shown;

[0026] Figure 15 The bit encoding of condition code extension slot 1 is shown;

[0027] Figure 16 shows the bit encoding of constant extension slot 0;

[0028] Figure 17 is a partial block diagram illustrating constant expansion;

[0029] Figure 18 Shown is a carry control for SIMD operations according to the present invention;

[0030] Figure 19 A conceptual diagram of the stream engine of the present invention is shown;

[0031] Figure 20 A first example of lane allocation in a vector is shown;

[0032] Figure 21 A second example of channel allocation in a vector is shown;

[0033] Figure 22 A basic two-dimensional flow is shown;

[0034] Figure 23 Shown Figure 21 The order of elements within the example stream;

[0035] Figure 24 The extraction of a smaller rectangle from a larger rectangle is shown;

[0036] Figure 25 shows how the stream engine retrieves the stream for this example at a transposed granularity of 4 bytes;

[0037] Figure 26 shows how the stream engine retrieves the stream for this example at a transposed granularity of 8 bytes;

[0038] Figure 27 Shows details of the stream engine of the present invention;

[0039] Figure 28 Shown is the stream template register of the present invention;

[0040] Figure 29 Shows the subfield definition of the flag field of the stream template register of the present invention;

[0041] Figure 30 A partial schematic diagram showing a stream engine supply of data displaying the present invention is shown;

[0042] Figure 31 A conceptual diagram illustrating one embodiment of a storage data flow;

[0043] Figure 32 Shows details of the cache architecture of the data buffer in the stream engine (prior art);

[0044] Figure 33 is a partial schematic diagram illustrating generation of an early address and an early loop count for loop 0;

[0045] FIG34 is a flow chart showing the operation of an early address generator;

[0046] Figure 35 is a partial schematic diagram illustrating generation of a late address and a late cycle count for cycle 0;

[0047] Figure 36 is a flow chart illustrating the operation of the post-address generator;

[0048] Figure 37 shows an exemplary bit assignment of an extended control register storing an early cycle count and a late cycle count;

[0049] Figure 38shows an exemplary bit assignment of an extended control register storing early and late addresses;

[0050] Figure 39 shows an exemplary bit assignment for an extended control register that stores a fetched virtual address that generated an error in a stream engine;

[0051] Figure 40 shows an exemplary bit assignment for an extended control register that stores data detailing the nature of a detected error;

[0052] Figure 41 The error detection and correction of the present invention is shown;

[0053] Figure 42 is a schematic diagram illustrating a multi-processor multi-memory system to which the fetch hysteresis of the present invention is applicable;

[0054] Figure 43 is a partial schematic diagram illustrating hardware of an exemplary embodiment of the present invention including a priority technique employing hysteresis; and

[0055] Figure 44 is a flow chart illustrating a priority technique employing hysteresis. DETAILED DESCRIPTION

[0056] Figure 1 A dual scalar / vector datapath processor according to a preferred embodiment of the present invention is shown. The processor 100 includes a separate level 1 instruction cache (L1I) 121 and a level 1 data cache (L1D) 123. The processor 100 includes a level 2 combined instruction / data cache (L2) 130 that stores both instructions and data. Figure 1 The connection between the level 1 instruction cache 121 and the level 2 combined instruction / data cache 130 is shown (bus 142). Figure 1 The connection (bus 145) between the first level data cache 123 and the second level combined instruction / data cache 130 is shown. In the preferred embodiment of the processor 100, the second level combined instruction / data cache 130 stores both instructions for backing up the first level instruction cache 121 and data for backing up the first level data cache 123. In the preferred embodiment, the second level combined instruction / data cache 130 is stored in a manner that is consistent with the first level instruction cache 121. Figure 1 The CPU core 110, the L1 instruction cache 121, the L1 data cache 123, and the L2 combined instruction / data cache 130 are formed on a single integrated circuit. The single integrated circuit may optionally include other circuits.

[0057] Central processing unit core 110 retrieves instructions from level 1 instruction cache 121 as controlled by instruction fetch unit 111. Instruction fetch unit 111 determines the next instruction to be executed and calls a fetch packet size for such an instruction set. The nature and size of the fetch packet are described in further detail below. As is known in the art, upon a cache hit (if the instructions are stored in level 1 instruction cache 121), the instructions are fetched directly from level 1 instruction cache 121. Upon a cache miss (if the specified instruction fetch packet is not stored in level 1 instruction cache 121), the instructions are searched in level 2 combined cache 130. In a preferred embodiment, the size of the cache line in level 1 instruction cache 121 is equal to the size of the fetch packet. The memory location of the instructions is either a hit or a miss in level 2 combined cache 130. A hit is serviced by level 2 combined cache 130. A miss is serviced by a higher level cache (not shown) or by main memory (not shown). As is known in the art, the requested instruction may be provided to both the L1 instruction cache 121 and the CPU core 110 simultaneously for accelerated use.

[0058] In a preferred embodiment of the present invention, the central processing unit core 110 includes multiple functional units that perform the data processing tasks specified by the instructions. The instruction dispatch unit 112 determines the target functional unit for each retrieved instruction. In a preferred embodiment, the central processing unit 110 operates as a very long instruction word (VLIW) processor capable of operating multiple instructions simultaneously in corresponding functional units. Preferably, the compiler organizes instructions in execution packets that are executed together. The instruction dispatch unit 112 directs each instruction to its target functional unit. The functional unit assigned to the instruction is entirely specified by the instructions generated by the compiler. The hardware of the central processing unit core 110 does not participate in this functional unit allocation. In a preferred embodiment, the instruction dispatch unit 112 can operate on multiple instructions in parallel. The number of such parallel instructions is set by the size of the execution packet. This will be further described below.

[0059] Part of the dispatching task of the instruction dispatch unit 112 is to determine whether to execute on a functional unit in the scalar datapath side A 115 or the vector datapath side B 116. An instruction bit in each instruction, called the s bit, determines which datapath the instruction controls. This will be described in further detail below.

[0060] Instruction decode unit 113 decodes each instruction in the currently executing packet. Decoding includes identifying the functional unit that executes the instruction, identifying registers in a possible register file that provide data for the corresponding data processing operation, and identifying the register destination for the result of the corresponding data processing operation. As explained further below, an instruction may include a constant field in place of a register number operand field. The result of this decoding is a signal used to control the target functional unit to perform the data processing operation specified by the corresponding instruction on the specified data.

[0061] The central processing unit core 110 includes control registers 114. The control registers 114 store information for controlling functional units in the scalar datapath side A 115 and the vector datapath side B 116 in a manner not relevant to the present invention. This information may be mode information or the like.

[0062] The decoded instruction from instruction decode 113 and the information stored in control register 114 are provided to scalar datapath side A 115 and vector datapath side B 116. As a result, the functional units within scalar datapath side A 115 and vector datapath side B 116 perform the data processing operation specified by the instruction on the instruction specified data and store the result in one or more data registers specified by the instruction. Each of scalar datapath side A 115 and vector datapath side B 116 includes multiple functional units that preferably operate in parallel. These will be discussed below in conjunction with Figure 2 More specifically, there is a data path 117 between the scalar data path side A 115 and the vector data path side B 116 that allows data to be exchanged.

[0063] The CPU core 110 further includes non-instruction-based modules. An emulation unit 118 allows the machine state of the CPU core 110 to be determined in response to instructions. This functionality is typically used for algorithm development. An interrupt / exception unit 119 enables the CPU core 110 to respond to external asynchronous events (interrupts) and to attempts to perform incorrect operations (exceptions).

[0064] The central processing unit core 110 includes a stream engine 125. The stream engine 125 provides two data streams from predetermined addresses, typically cached in the L2 combined cache 130, to the register file of the vector datapath side B. This provides control data movement from memory (such as cache in the L2 combined cache 130) directly to the operand inputs of the functional units. This is further detailed below.

[0065] Figure 1Example data widths of buses between various components are shown. Level 1 instruction cache 121 provides instructions to instruction fetch unit 111 via bus 141. Bus 141 is preferably a 512-bit bus. Bus 141 is unidirectional from Level 1 instruction cache 121 to central processing unit core 110. Level 2 combined cache 130 provides instructions to Level 1 instruction cache 121 via bus 142. Bus 142 is preferably a 512-bit bus. Bus 142 is unidirectional from Level 2 combined cache 130 to Level 1 instruction cache 121.

[0066] The L1 data cache 123 exchanges data with the register file in scalar datapath side A 115 via bus 143. Bus 143 is preferably a 64-bit bus. The L1 data cache 123 exchanges data with the register file in vector datapath side B 116 via bus 144. Bus 144 is preferably a 512-bit bus. Buses 143 and 144 are shown as bidirectionally supporting both data reads and data writes from the CPU core 110. The L1 data cache 123 exchanges data with the L2 combining cache 130 via bus 145. Bus 145 is preferably a 512-bit bus. Bus 145 is shown as bidirectionally supporting cache services for both data reads and data writes from the CPU core 110.

[0067] The L2 combined cache 130 provides data from the first data stream to the stream engine 125 via bus 146. Bus 146 is preferably a 512-bit bus. The stream engine 125 provides data from the first data stream to the functional units of vector datapath side B 116 via bus 147. Bus 147 is preferably a 512-bit bus. The L2 combined cache 130 provides data from the second data stream to the stream engine 125 via bus 148. Bus 148 is preferably a 512-bit bus. The stream engine 125 provides data from the second data stream to the functional units of vector datapath side B 116 via bus 149. Bus 149 is preferably a 512-bit bus. According to a preferred embodiment of the present invention, buses 146, 147, 148, and 149 are shown as unidirectional, from the L2 combined cache 130 to the stream engine 125 and vector datapath side B 116.

[0068] In a preferred embodiment of the present invention, both the L1 data cache 123 and the L2 combined cache 130 may be configured as a selected amount of cache or directly addressable memory in accordance with U.S. Patent No. 6,606,686, entitled UNIFIED MEMORY SYSTEM ARCHITECTURE INCLUDING CACHE AND DIRECTLY ADDRESSABLE STATIC RANDOM ACCESS MEMORY.

[0069] Figure 2 Further details of the functional units and register files within scalar datapath side A 115 and vector datapath side B 116 are shown. Scalar datapath side A 115 includes a global scalar register file 211, an L1 / S1 local register file 212, an M1 / N1 local register file 213, and a D1 / D2 local register file 214. Scalar datapath side A 115 includes an L1 unit 221, an S1 unit 222, an M1 unit 223, an N1 unit 224, a D1 unit 225, and a D2 unit 237. Vector datapath side B 116 includes a global vector register file 231, an L2 / S2 local register file 232, an M2 / N2 / C local register file 233, and a predicate register file 234. Vector datapath side B 116 includes an L2 unit 241, an S2 unit 242, an M2 unit 243, an N2 unit 244, a C unit 245, and a P unit 246. There are restrictions on which functional units can read from or write to which register files. These are detailed below.

[0070] Scalar datapath side A 115 includes an L1 unit 221. The L1 unit 221 typically accepts two 64-bit operands and produces a 64-bit result. Each operand is retrieved from an instruction-specific register in the global scalar register file 211 or the L1 / S1 local register file 212. The L1 unit 221 preferably performs the following instruction-selected operations: 64-bit addition / subtraction operations; 32-bit minimum / maximum operations; 8-bit single instruction multiple data (SIMD) instructions, such as absolute sum, minimum and maximum determination; rotated minimum / maximum operations; and various move operations between register files. The result may be written to an instruction-specific register in the global scalar register file 211, the L1 / S1 local register file 212, the M1 / N1 local register file 213, or the D1 / D2 local register file 214.

[0071] Scalar datapath side A 115 includes an S1 unit 222. The S1 unit 222 typically accepts two 64-bit operands and produces a 64-bit result. The two operands are each retrieved from an instruction-specific register in the global scalar register file 211 or the L1 / S1 local register file 212. The S1 unit 222 preferably performs the same type of operations as the L1 unit 221. There may be slight variations in the data processing operations supported by the L1 unit 221 and the S1 unit 222. The result may be written to an instruction-specific register in the global scalar register file 211, the L1 / S1 local register file 212, the M1 / N1 local register file 213, or the D1 / D2 local register file 214.

[0072] Scalar datapath side A 115 includes an M1 unit 223. The M1 unit 223 typically accepts two 64-bit operands and produces a 64-bit result. Each operand is retrieved from an instruction-specific register in the global scalar register file 211 or the M1 / N1 local register file 213. The M1 unit 223 preferably performs the following instruction-selected operations: 8-bit multiplication; complex dot product; 32-bit bit count; complex conjugate multiplication; and bitwise logical operations, shifts, additions, and subtractions. The result can be written to an instruction-specific register in the global scalar register file 211, the L1 / S1 local register file 212, the M1 / N1 local register file 213, or the D1 / D2 local register file 214.

[0073] Scalar datapath side A 115 includes an N1 unit 224. The N1 unit 224 typically accepts two 64-bit operands and produces a 64-bit result. The two operands are each called from an instruction-specific register in the global scalar register file 211 or the M1 / N1 local register file 213. The N1 unit 224 preferably performs the same type of operation as the M1 unit 223. Some dual operations (called dual-issue instructions) can use both the M1 unit 223 and the N1 unit 224. The result can be written to an instruction-specific register in the global scalar register file 211, the L1 / S1 local register file 212, the M1 / N1 local register file 213, or the D1 / D2 local register file 214.

[0074] Scalar datapath side A 115 includes a D1 unit 225 and a D2 unit 237. Each of the D1 and D2 units 225 and 237 typically accepts two 64-bit operands and produces a 64-bit result. The D1 and D2 units 225 and 237 typically perform address calculations and corresponding load and store operations. The D1 unit 225 is used for 64-bit scalar loads and stores, while the D2 unit 237 is used for 512-bit vector loads and stores. The D1 and D2 units 225 and 237 also preferably perform: data swapping, packing, and unpacking for loads and stores; 64-bit SIMD arithmetic operations; and 64-bit bitwise logical operations. The D1 / D2 local register file 214 typically stores the base address and offset address used in the corresponding load and store address calculations. The two operands are each retrieved from instruction-specific registers in the global scalar register file 211 or the D1 / D2 local register file 214. The calculation result may be written into an instruction-specified register of the global scalar register file 211 , the L1 / S1 local register file 212 , the M1 / N1 local register file 213 , or the D1 / D2 local register file 214 .

[0075] Vector datapath side B 116 includes an L2 unit 241. The L2 unit 241 typically accepts two 512-bit operands and produces a 512-bit result. The two operands are each called from an instruction-specific register in the global vector register file 231, the L2 / S2 local register file 232, or the predicate register file 234. The L2 unit 241 preferably executes instructions similar to the L1 unit 221, except for the wider 512-bit data. The result can be written to an instruction-specific register in the global vector register file 231, the L2 / S2 local register file 222, the M2 / N2 / C local register file 233, or the predicate register file 234.

[0076] Vector datapath side B 116 includes an S2 unit 242. The S2 unit 242 typically accepts two 512-bit operands and produces a 512-bit result. Each operand is called from an instruction-specific register in the global vector register file 231, the L2 / S2 local register file 232, or the predicate register file 234. The S2 unit 242 preferably executes instructions similar to the S1 unit 222, except for the wider 512-bit data. The result is written to an instruction-specific register in the global vector register file 231, the L2 / S2 local register file 222, the M2 / N2 / C local register file 233, or the predicate register file 234. Some dual operations (called dual-issue instructions) may utilize both the L2 unit 241 and the S2 unit 242. The result is written to an instruction-specific register in the global vector register file 231, the L2 / S2 local register file 232, or the M2 / N2 / C local register file 233.

[0077] Vector datapath side B 116 includes an M2 unit 243. The M2 unit 243 typically accepts two 512-bit operands and produces a 512-bit result. The two operands are each called from an instruction-specific register in the global vector register file 231 or the M2 / N2 / C local register file 233. The M2 unit 243 preferably executes instructions similar to the M1 unit 222, except for the wider 512-bit data. The result can be written to an instruction-specific register in the global vector register file 231, the L2 / S2 local register file 232, or the M2 / N2 / C local register file 233.

[0078] Vector datapath side B 116 includes an N2 unit 244. The N2 unit 244 typically accepts two 512-bit operands and produces a 512-bit result. The two operands are each called from an instruction-specific register in the global vector register file 231 or the M2 / N2 / C local register file 233. The N2 unit 244 preferably performs the same type of operation as the M2 unit 243. Some dual operations (called dual-issue instructions) can use both the M2 unit 243 and the N2 unit 244 together. The result can be written to an instruction-specific register in the global vector register file 231, the L2 / S2 local register file 232, or the M2 / N2 / C local register file 233.

[0079] Vector datapath side B 116 includes a C unit 245. The C unit 245 typically accepts two 512-bit operands and produces a 512-bit result. The two operands are each called from an instruction-specified register in the global vector register file 231 or the M2 / N2 / C local register file 233. The C unit 245 preferably executes: "rake" and "search" instructions; up to 512 2-bit PN*8-bit multiplications by I / Q complex multiplications per clock cycle; 8-bit and 16-bit sum of absolute differences (SAD) calculations, up to 512 SADs per clock cycle; horizontal add and horizontal min / max instructions; and vector permute instructions. The C unit 245 also includes four vector control registers (CUCR0 through CUCR3) for controlling certain operations of the C unit 245 instructions. Control registers CUCR0 through CUCR3 are used as operands in certain C unit 245 operations. Control registers CUCR0 to CUCR3 are preferably used to: control the general permute instruction (VPERM); and as masks for SIMD multiple DOT (dot) product operations (DOTPM) and SIMD multiple sum of absolute differences (SAD) operations. Control register CUCR0 is preferably used to store the polynomial for the Galois Field Multiply operation (GFMPY). Control register CUCR1 is preferably used to store the Galois Field polynomial generator function.

[0080] Vector datapath side B 116 includes a P-unit 246. The P-unit 246 performs basic logic operations on registers of the local predicate register file 234. The P-unit 246 has direct access to read and write to the predicate register file 234. These operations include AND, ANDN, OR, XOR, NOR, BITR, NEG, SET, BITCNT, RMBD, BIT Decimate, and Expand. Typical intended uses of the P-unit 246 include operations on SIMD vector compare results used to control another SIMD vector operation.

[0081] Figure 3 The global scalar register file 211 is shown. There are 16 independent 64-bit wide scalar registers, labeled A0 through A15. Each register in the global scalar register file 211 can be read from or written to as 64-bit scalar data. All scalar datapath side A 115 functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 237) can read from or write to the global scalar register file 211. The global scalar register file 211 can be read as 32-bit or 64-bit and can only be written to as 64-bit. The instruction execution determines the size of the data read. Subject to restrictions described in detail below, the vector datapath side B 116 functional units (L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245, and P unit 246) can read from the global scalar register file 211 via the cross path 117.

[0082] Figure 4 The D1 / D2 local register file 214 is shown. There are 16 independent 64-bit wide scalar registers, labeled D0 through D16. Each register in the D1 / D2 local register file 214 can be read from or written to as 64-bit scalar data. All scalar datapath-side A115 functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 237) can write to the global scalar register file 211. Only the D1 unit 225 and D2 unit 237 can read from the D1 / D1 local scalar register file 214. It is expected that the data stored in the D1 / D2 local scalar register file 214 will include the base address and offset address used in address calculations.

[0083] Figure 5 An L1 / S1 local register file 212 is shown. Figure 5 The embodiment shown has eight independent 64-bit wide scalar registers, labeled AL0 through AL7. Figure 13 ) allows the L1 / S1 local register file 212 to include up to 16 registers. Figure 5 The embodiment implements only eight registers to reduce circuit size and complexity. Each register of the L1 / S1 local register file 212 can be read from or written to as 64-bit scalar data. All scalar datapath-side A115 functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 237) can write to the L1 / S1 local scalar register file 212. Only the L1 unit 221 and S1 unit 222 can read from the L1 / S1 local scalar register file 212.

[0084] Figure 6 The M1 / N1 local register file 213 is shown. Figure 6 The embodiment shown has eight independent 64-bit wide scalar registers, labeled AM0 through AM7. Figure 13 ) allows the M1 / N1 local register file 213 to include up to 16 registers. Figure 6 The embodiment implements only eight registers to reduce circuit size and complexity. Each register of the M1 / N1 local register file 213 can be read from or written to as 64-bit scalar data. All scalar datapath-side A115 functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 237) can write to the M1 / N1 local scalar register file 213. Only the M1 unit 223 and N1 unit 224 can read from the M1 / N1 local scalar register file 213.

[0085] Figure 7 A global vector register file 231 is shown. There are 16 independent 512-bit wide scalar registers. Each register in the global vector register file 231 can be read from or written to as 64-bit scalar data, labeled B0 through B15. Each register in the global vector register file 231 can be read from or written to as 512-bit vector data, labeled VB0 through VB15. The instruction type determines the data size. All vector datapath side B 116 functional units (L2 unit 241, S2 unit 242, M3 unit 243, N2 unit 244, C unit 245, and P unit 246) can read from or write to the global scalar register file 231. Subject to the limitations detailed below, the scalar datapath side A 115 functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 237) can read from the global vector register file 231 via the crossbar path 117.

[0086] Figure 8The P local register file 234 is shown. There are eight independent 64-bit wide registers, labeled P0 through P15. Each register in the P local register file 234 can be read or written as 64-bit scalar data. The vector datapath side B 116 functional units, L2 unit 241, S2 unit 242, C unit 245, and P unit 246, can write to the P local register file 234. Only the L2 unit 241, S2 unit 242, and P unit 246 can read from the P local scalar register file 234. Commonly intended uses of the P local register file 234 include: writing a one-bit SIMD vector compare result from the L2 unit 241, S2 unit 242, or C unit 245; operating on the SIMD vector compare result by the P unit 246; and using the result of the operation to control another SIMD vector operation.

[0087] Figure 9 An L2 / S2 local register file 232 is shown. Figure 9 The embodiment shown in has eight independent 512-bit wide scalar registers. Figure 13 ) allows the L2 / S2 local register file 232 to include up to 16 registers. Figure 9 The embodiment implements only eight registers to reduce circuit size and complexity. Each register of the L2 / S2 local vector register file 232 can be read from or written to as 64-bit scalar data, labeled BL0 through BL7. Each register of the L2 / S2 local vector register file 232 can be read from or written to as 512-bit vector data, labeled VBL0 through VBL7. The instruction type determines the data size. All vector datapath side B 116 functional units (L2 unit 241, S2 unit 242, M2 unit 233, N2 unit 244, C unit 245, and P unit 246) can write to the L2 / S2 local vector register file 232. Only the L2 unit 241 and S2 unit 242 can read from the L2 / S2 local vector register file 232.

[0088] Figure 10 The M2 / N2 / C local register file 233 is shown. Figure 10 The embodiment shown in has eight independent 512-bit wide scalar registers. The preferred instruction encoding (see Figure 13 ) allows the L1 / S1 local register file 212 to include up to 16 registers. Figure 10The embodiment implements only eight registers to reduce circuit size and complexity. Each register of the M2 / N2 / C local vector register file 233 can be read from or written to as 64-bit scalar data labeled BM0 through BM7. Each register of the M2 / N2 / C local vector register file 233 can be read from or written to as 512-bit vector data labeled VBM0 through VBM7. All vector datapath side B 116 functional units (L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245, and P unit 246) can write to the M2 / N2 / C local vector register file 233. Only the M2 unit 233, N2 unit 244, and C unit 245 can read from the M2 / N2 / C local vector register file 233.

[0089] Providing a global register file accessible to all functional units of a side and local register files accessible by only some functional units of a side is a design choice.The present invention can be implemented with only one type of register file corresponding to the disclosed global register file.

[0090] The crossbar path 117 allows limited data exchange between the scalar datapath side A 115 and the vector datapath side B 116. During each operation cycle, one 64-bit data word can be called from the global scalar register file A 211 for use as an operand for one or more functional units of the vector datapath side B 116, and one 64-bit data word can be called from the global vector register file 231 for use as an operand for one or more functional units of the scalar datapath side A 115. Any scalar datapath side A 115 functional unit (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 237) can read a 64-bit operand from the global vector register file 231. The 64-bit operand is the least significant bits of the 512-bit data in the register of the global vector register file 232 being accessed. Multiple scalar datapath side A 115 functional units can use the same 64-bit cross-path data as an operand during the same operating cycle. However, in any single operating cycle, only one 64-bit operand is transferred from vector datapath side B 116 to scalar datapath side A 115. Any vector datapath side B 116 functional unit (L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245, and P unit 246) can read a 64-bit operand from the global scalar register file 211. If the corresponding instruction is a scalar instruction, the cross-path operand data is treated as any other 64-bit operand. If the corresponding instruction is a vector instruction, the upper 448 bits of the operand are filled with zeros. Multiple vector datapath side B 116 functional units can use the same 64-bit cross-path data as an operand during the same operating cycle. In any single operation cycle, only one 64-bit operand is transferred from the scalar datapath side A 115 to the vector datapath side B 116 .

[0091] The stream engine 125 transfers data under certain restrictions. The stream engine 125 controls two data streams. A stream consists of a sequence of elements of a specific type. Programs operating on the streams read the data sequentially, performing operations on each element in turn. Each stream has the following basic properties: Stream data has well-defined start and end times. Stream data has a fixed element size and type throughout the stream. Stream data has a fixed sequence of elements. Therefore, programs cannot randomly search within the stream. Stream data is read-only while active. Programs cannot write to a stream while simultaneously reading from it. Once a stream is opened, the stream engine 125: calculates addresses; retrieves defined data types from the L2 unified cache (which may require cache servicing from higher-level memory); performs data type operations such as zero extension, sign extension, and data element sorting / swapping, such as matrix transformations; and passes data directly to the programmed data register file within the CPU core 110. Therefore, the stream engine 125 is useful for real-time digital filtering operations on well-behaved data. The stream engine 125 releases these memory retrieval tasks from the corresponding central processing unit core 110, thereby enabling other processing functions.

[0092] The stream engine 125 provides the following benefits. The stream engine 125 allows multi-dimensional memory access. The stream engine 125 increases the available bandwidth of the functional units. Because the stream buffer bypasses the level 1 data cache 123, the stream engine 125 minimizes the number of cache miss stalls. The stream engine 125 reduces the number of scalar operations required to maintain loops. The stream engine 125 manages address pointers. The stream engine 125 handles address generation, automatically freeing up address generation instruction slots and the D1 unit 224 and D2 unit 237 for other computations.

[0093] The CPU core 110 operates on an instruction pipeline. Instructions are retrieved from fixed-length instruction packets, described further below. All instructions require the same number of pipeline stages to retrieve and decode, but require varying numbers of execution stages.

[0094] Figure 11 The following pipeline stages are shown: a program fetch stage 1110, a dispatch and decode stage 1110, and an execute stage 1130. The program fetch stage 1110 includes three stages for all instructions. The dispatch and decode stage includes three stages for all instructions. The execute stage 1130 includes one to four stages depending on the instruction.

[0095] The fetch phase 1110 includes a program address generation phase 1111 (PG), a program access phase 1112 (PA), and a program receive phase 1113 (PR). During the program address generation phase 1111 (PG), a program address is generated in the CPU core 110, and a read request is sent to the memory controller of the L1I instruction cache. During the program access phase 1112 (PA), the L1I instruction cache processes the request, accesses the data in its memory, and sends a fetch packet to the CPU core 110 boundary. During the program receive phase 1113 (PR), the CPU core 110 registers the fetch packet.

[0096] Instructions are always fetched in 16 32-bit wide slots, one at a time, forming a fetch packet. Figure 12 Sixteen instructions 1201 to 1216 are shown for a single fetch packet. The fetch packet is aligned on a 512-bit (16-word) boundary. The preferred embodiment uses a fixed 32-bit instruction length. Fixed-length instructions are advantageous for several reasons. Fixed-length instructions enable simple decoder alignment. Properly aligned instruction fetches allow multiple instructions to be loaded into parallel instruction decoders. This properly aligned instruction fetch can be achieved by predetermined instruction alignment when stored in memory coupled to fixed instruction packet fetch (fetch packets aligned on 512-bit boundaries). Aligned instruction fetches allow parallel decoders to operate on instruction-sized fetch bits. Variable-length instructions require an initial step to locate each instruction boundary before they can be decoded. Fixed-length instruction sets generally allow for a more regular layout of instruction fields. This simplifies the construction of each decoder, which is advantageous for widely distributed VLIW central processors.

[0097] The execution of individual instructions is controlled in part by a p-bit in each instruction. This p-bit is preferably bit 0 of a 32-bit wide slot. The p-bit determines whether an instruction is executed in parallel with the next instruction. Instructions are scanned from lower to higher addresses. If the p-bit of an instruction is 1, the next instruction (higher memory address) is executed in parallel with the instruction (in the same cycle as the instruction). If the p-bit of an instruction is 0, the next instruction is executed in the cycle after the instruction.

[0098] The CPU core 110 and the L1I instruction cache 121 pipelines are decoupled from each other. A fetch packet from the L1I instruction cache 121 can take different numbers of clock cycles to return, depending on external circumstances such as whether there is a hit in the L1I instruction cache 121 or a hit in the L2 combined cache 130. Therefore, the program access stage 1112 (PA) may require several clock cycles instead of the single clock cycle required in other stages.

[0099] Instructions executed in parallel constitute an execution packet. In a preferred embodiment, an execution packet can include up to sixteen instructions. No two instructions in an execution packet can use the same functional unit. A slot is one of five types: 1) a self-contained instruction that executes on one of the functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, D2 unit 237, L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245, and P unit 246) of the CPU core 110; 2) a unitless instruction, such as a NOP (no operation) instruction or multiple NOP instructions; 3) a branch instruction; 4) a field extension of a constant; and 5) a conditional code extension. Some of these slot types are explained further below.

[0100] The dispatch and decode stage 1110 includes the instruction dispatch to the appropriate execution unit stage 1121 (DS), the instruction pre-decode stage 1122 (D1), and the instruction decode and operand fetch stage 1222 (D2). During the instruction dispatch to the appropriate execution unit stage 1121 (DS), fetch packets are divided into execution packets and distributed to the appropriate functional units. During the instruction pre-decode stage 1122 (D1), the source registers, destination registers, and associated paths are decoded to execute the instruction in the functional unit. During instruction decode, the operand fetch stage 1222 (D2) performs unit decoding in more detail and reads operands from the register file.

[0101] The execution stage 1130 includes execution stages 1131 through 1135 (E1 through E5). Different types of instructions require different numbers of these stages to complete execution. These pipeline stages play an important role in understanding the state of the device at the cycle boundaries of the CPU core 110.

[0102] During the execution 1 phase 1131 (E1), the instruction conditions are evaluated and the operands are operated on. Figure 11 As shown, the execute 1 stage 1131 can receive operands from one of the stream buffers 1141 and the register file, schematically shown as 1142. For load and store instructions, address generation is performed and address modifications are written to the register file. For branch instructions, the branch fetch packet of the PG stage is affected. Figure 11 As shown, load and store instructions access memory, schematically shown here as memory 1151. For single-cycle instructions, the result is written to the destination register file. This assumes that any condition of the instruction evaluates to true. If the condition evaluates to false, then after execute 1 stage 1131, the instruction does not write any result or have any pipeline operation.

[0103] During execute 2 stage 1132 (E2), load instructions send addresses to memory. Store instructions send addresses and data to memory. Single-cycle instructions that saturate results set the SAT bit in the control status register (CSR) when saturation occurs. For two-cycle instructions, the result is written to the destination register file.

[0104] During execution stage 3 1133 (E3), data memory accesses are performed. Any multiplication instruction that saturates the result sets the SAT bit in the control status register (CSR) when saturation occurs. For instructions that take 3 cycles, the result is written to the destination register file.

[0105] During execute 4 stage 1134 (E4), the load instruction brings the data to the CPU core 110 boundary. For a 4 cycle instruction, the result is written to the destination register file.

[0106] During execution stage 5 1135 (E5), a load instruction writes data to the register. Figure 11 Schematically shown as input from memory 1151 to execution 5 stage 1135.

[0107] Figure 13 An example of an instruction encoding 1300 for a functional unit instruction used in the present invention is shown. Those skilled in the art will recognize that other instruction encodings are possible and within the scope of the present invention. Each instruction consists of 32 bits and controls the operation of one of the individually controllable functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, D2 unit 237, L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245, and P unit 246). The bit fields are defined as follows.

[0108] The creg field 1301 (bits 29 to 31) and the z bit 1302 (bit 28) are optional fields used in conditional instructions. These bits are used in conditional instructions to identify the predicate register and the condition. The z bit 1302 (bit 28) indicates whether the prediction is based on a zero or non-zero in the predicate register. If z = 1, the test is for an equation with zero. If z = 0, the test is for a non-zero. The cases of creg = 0 and z = 0 are considered to be always true to allow unconditional instruction execution. The creg field 1301 and the z field 1302 are encoded in the instruction as shown in Table 1.

[0109]

[0110] Table 1

[0111] The execution of conditional instructions depends on the value stored in a specified data register. This data register is in the global scalar register file 211 for all functional units. Note that the "z" in the z bit column refers to the zero / non-zero comparison selection mentioned above, and "x" is a don't care state. This encoding can only designate a subset of the 16 global registers as assertion registers. This selection is to preserve bits in the instruction encoding. Note that unconditional instructions do not have these optional bits. For unconditional instructions, these bits in fields 1301 and 1302 (28 to 31) are preferably used as additional opcode bits.

[0112] The dst field 1303 (bits 23 to 27) specifies a register in the corresponding register file as the destination for the instruction result.

[0113] The src2 / cst field 1304 (bits 18 to 22) has multiple meanings depending on the instruction opcode field (bits 4 to 12 for all instructions and additional bits 28 to 31 for unconditional instructions). The first meaning is to specify a register of the corresponding register file as the second operand. The second meaning is a direct constant. Depending on the instruction type, this is treated as an unsigned integer and zero-extended to the specified data length, or treated as a signed integer and sign-extended to the specified data length.

[0114] The src1 field 1305 (bits 13 to 17) specifies a register in the corresponding register file as the first source operand.

[0115] The operation code (opcode) field 1306 (bits 4 to 12) of all instructions (and additional bits 28 to 31 for unconditional instructions) specifies the type of instruction and specifies the appropriate instruction options. This includes explicit specification of the functional unit to be used and the operation to be performed. A detailed explanation of the opcode, other than the instruction options detailed below, is beyond the scope of this invention.

[0116] The e bit 1307 (bit 2) is used only for direct constant instructions that can extend constants. If e=1, the direct constant is extended in the manner detailed below. If e=0, the direct constant is not extended. In this case, the direct constant is specified by the src2 / cst field 1304 (bits 18 to 22). Note that this e bit 1307 is only used for certain instructions. Therefore, with appropriate encoding, this e bit 1307 can be omitted from instructions that do not need it, and the bit serves as an additional opcode bit.

[0117] The s bit 1307 (bit 1) specifies either the scalar datapath side A 115 or the vector datapath side B 116. If s=0, then the scalar datapath side A 115 is selected. This limits the functional unit to Figure 2The L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225 and D2 unit 237 and the corresponding register files are shown. Similarly, s=1 selects the vector datapath side B 116, limiting the functional units to Figure 2 Shown are an L2 unit 241 , an S2 unit 242 , an M2 unit 243 , an N2 unit 244 , a P unit 246 and corresponding register files.

[0118] The p bit 1308 (bit 0) marks an execution packet. The p bit determines whether an instruction is executed in parallel with the following instruction. The p bit is scanned from low to high address. If p = 1 for the current instruction, the next instruction is executed in parallel with the current instruction. If p = 0 for the current instruction, the next instruction is executed in the cycle after the current instruction. All instructions executed in parallel constitute an execution packet. An execution packet can include up to twelve instructions. Each instruction in an execution packet must use a different functional unit.

[0119] There are two different condition code extension slots. Each execute packet can include each of these unique 32-bit condition code extension slots, which include the 4-bit creg / z field of the instructions in the same execute packet. Figure 14 shows the encoding of condition code extension slot 0, and Figure 15 The encoding of condition code extension slot 1 is shown.

[0120] Figure 14The encoding of the condition code extension slot 0 with 32 bits is shown. Field 1401 (bits 28 to 31) specifies the four creg / z bits allocated to instructions in the same execution packet for the L1 unit 221. Field 1402 (bits 27 to 24) specifies the four creg / z bits allocated to instructions in the same execution packet for the L2 unit 241. Field 1403 (bits 19 to 23) specifies the four creg / z bits allocated to instructions in the same execution packet for the S1 unit 222. Field 1404 (bits 16 to 19) specifies the four creg / z bits allocated to instructions in the same execution packet for the S2 unit 242. Field 1405 (bits 12 to 15) specifies the four creg / z bits allocated to instructions in the same execution packet for the D1 unit 225. Field 1406 (bits 8 to 11) specifies the four creg / z bits allocated to instructions in the same execution packet for the D2 unit 245. Field 1407 (bits 6 and 7) is unused / reserved. Field 1408 (bits 0 through 5) is encoded as a unique set of bits (CCEX0) to identify condition code extension slot 0. Once this unique ID for condition code extension slot 0 is detected, the corresponding creg / z bits are used to control the conditional execution of any L1 unit 221, L2 unit 241, S1 unit 222, S2 unit 242, D1 unit 224, and D2 unit 225 instructions in the same execution packet. The interpretation of these creg / z bits is shown in Table 1. If the corresponding instruction is conditional (includes creg / z bits), the corresponding bits in condition code extension slot 0 override the condition code bits in the instruction. Note that an execution packet cannot have more than one instruction directed to a specific execution unit. An execution packet for an instruction cannot include more than one condition code extension slot 0. Therefore, the mapping of creg / z bits to functional unit instructions is unambiguous. Setting the creg / z bits equal to "0000" makes the instruction unconditional. Thus, a suitably encoded condition code extension slot 0 can make some corresponding instructions conditional and some instructions unconditional.

[0121] Figure 15The encoding of the condition code extension slot 1 with 32 bits is shown. Field 1501 (bits 28 to 31) specifies the four creg / z bits allocated to the M1 unit 223 instructions in the same execution packet. Field 1502 (bits 27 to 24) specifies the four creg / z bits allocated to the M2 unit 243 instructions in the same execution packet. Field 1503 (bits 19 to 23) specifies the four creg / z bits allocated to the C unit 245 instructions in the same execution packet. Field 1504 (bits 16 to 19) specifies the four creg / z bits allocated to the N1 unit 224 instructions in the same execution packet. Field 1505 (bits 12 to 15) specifies the four creg / z bits allocated to the N2 unit 244 instructions in the same execution packet. Field 1506 (bits 5 to 11) is unused / reserved. Field 1507 (bits 0 through 5) is encoded as a unique set of bits (CCEX1) to identify condition code extension slot 1. Once this unique ID for condition code extension slot 1 is detected, the corresponding creg / z bits are used to control the conditional execution of any M1 unit 223, M2 unit 243, C unit 245, N1 unit 224, and N2 unit 244 instructions in the same execution packet. The interpretation of these creg / z bits is shown in Table 1. If the corresponding instruction is conditional (includes creg / z bits), the corresponding bits in condition code extension slot 1 override the condition code bits in the instruction. Note that an execution packet cannot have more than one instruction directed to a specific execution unit. An execution packet for an instruction cannot include more than one condition code extension slot 1. Therefore, the mapping of creg / z bits to functional unit instructions is unambiguous. Setting the creg / z bits equal to "0000" makes the instruction unconditional. Thus, a properly encoded condition code extension slot 1 can make some instructions conditional and some instructions unconditional.

[0122] Condition code extension slot 0 and condition code extension slot 1 can both include p bits to define the above combination of Figure 13 In a preferred embodiment, as Figure 14 and Figure 15 As shown, condition code extension slot 0 and condition code extension slot 1 preferably have bit 0 (p bits) always encoded as 1. Therefore, neither condition code extension slot 0 nor condition code extension slot 1 can be located in the last instruction slot of an execution packet.

[0123] There are two different constant expansion slots. Each execution packet may include each of these unique 32-bit constant expansion slots, which include 27 bits to be connected as high-order bits to 5-bit constant fields 1305 to form 32-bit constants. As pointed out in the above instruction encoding description, only some instructions define src2 / cst fields 1304 as constants rather than source register identifiers. At least some of these instructions can use the constant expansion slots to expand this constant to 32 bits.

[0124] Figure 16 The fields for constant extension slot 0 are shown. Each execution packet may include one instance of constant extension slot 0 and one instance of constant extension slot 1. Figure 16 Constant extension slot 0 1600 is shown to include two fields. Field 1601 (bits 5 to 31) constitutes the most significant 27 bits of the extended 32-bit constant, including the target instruction scr2 / cst field 1304 as the five least significant bits. Field 1602 (bits 0 to 4) is encoded as a unique set of bits (CSTX0) to identify constant extension slot 0. In a preferred embodiment, constant extension slot 0 1600 can only be used to extend a constant in one of the L1 unit 221 instructions, D1 unit 225 instructions, S2 unit 242 instructions, D2 unit 237 instructions, M2 unit 243 instructions, N2 unit 244 instructions, branch instructions, or C unit 245 instructions within the same execution packet. Constant extension slot 1 is similar to constant extension slot 0, except that bits 0 to 4 are encoded as a unique set of bits (CSTX1) to identify constant extension slot 1. In a preferred embodiment, constant extension slot 1 can only be used to extend a constant in one of the L2 unit 241 instructions, data in the D2 unit 237 instructions, offset in the S1 unit 222 instructions, D1 unit 225 instructions, M1 unit 223 instructions, or N1 unit 224 instructions in the same execution package.

[0125] The method of using constant extension slot 0 and constant extension slot 1 is as follows. The target instruction must be a type that allows constant specifications. As known in the art, this is achieved by replacing an input operand register designation field with the least significant bit of the constant, as described above with respect to scr2 / cst field 1304. The instruction decoder 113 determines this situation, which is called a direct field from the instruction opcode bit. The target instruction also includes a constant extension bit (e position 1307) dedicated to signaling whether to not expand the specified constant (preferably, constant extension bit=0) or to expand the constant (preferably, constant extension bit=1). If the instruction decoder 113 detects constant extension slot 0 or constant extension slot 1, it further checks other instructions in the execution packet for instructions corresponding to the detected constant extension slot. Constant extension is only performed when only a corresponding instruction has a constant extension bit (e position 1307) equal to 1.

[0126] Figure 17 is a partial block diagram 1700 illustrating constant expansion. Figure 17Assume that instruction decoder 113 detects constant extension slot and corresponding instruction in the same execution packet. Instruction decoder 113 provides 27 extension bits from constant extension slot (bit field 1601) and 5 constant bits (bit field 1305) from corresponding instruction to connector 1701. Connector 1701 forms a single 32-bit word from these two parts. In a preferred embodiment, 27 extension bits from constant extension slot (bit field 1601) are the most significant bits, and 5 constant bits (bit field 1305) are the least significant bits. The combined 32-bit word is provided to one input of multiplexer 1702. 5 constant bits from corresponding instruction field 1305 provide the second input to multiplexer 1702. The selection of multiplexer 1702 is controlled by the state of constant extension bit. If constant extension bit (e position 1307) is 1 (extended), multiplexer 1702 selects the connected 32-bit input. If the constant extension bit is 0 (not extended), then multiplexer 1702 selects 5 constant bits from the corresponding instruction field 1305. Multiplexer 1702 provides this output to the input of sign extension unit 1703.

[0127] Sign extension unit 1703 forms the final operand value from the input from multiplexer 1702. Sign extension unit 1703 receives control inputs scalar / vector and data size. The scalar / vector input indicates whether the corresponding instruction is a scalar or vector instruction. The functional units on datapath side A 115 (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 237) can only execute scalar instructions. Any instruction directed to one of these functional units is a scalar instruction. The functional units on datapath side B (L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, and C unit 245) can execute either scalar or vector instructions. The instruction decoder 113 determines whether the instruction is a scalar or vector instruction based on the opcode bits. The P unit 246 can only execute scalar instructions. The data size can be 8 bits (byte B), 16 bits (halfword H), 32 bits (word W), or 64 bits (doubleword D).

[0128] Table 2 lists the operation of the sign extension unit 1703 for various options.

[0129]

[0130]

[0131] Table 2

[0132] Both constant extension slot 0 and constant extension slot 1 may include p bits to define the above combination of Figure 13In a preferred embodiment, as in the case of the conditional code extension slots, constant extension slot 0 and constant extension slot 1 preferably have bit 0 (p bits) always encoded as 1. Therefore, constant extension slot 0 and constant extension slot 1 cannot be located in the last instruction slot of an execution packet.

[0133] It is technically feasible to include constant extension slots 0 or 1 and more than one corresponding instruction marked as constant extension (e bit = 1) in an execution packet. For constant extension slot 0, this would mean that more than one of the L1 unit 221 instruction, the data in the D1 unit 225 instruction, the S2 unit 242 instruction, the offset in the D2 unit 237 instruction, the M2 unit 243 instruction, or the N2 unit 244 instruction in the execution packet has an e bit of 1. For constant extension slot 1, this would mean that more than one of the L2 unit 241 instruction, the data in the D2 unit 237 instruction, the S1 unit 222 instruction, the offset in the D1 unit 225 instruction, the M1 unit 223 instruction, or the N1 unit 224 instruction in the execution packet has an e bit of 1. Providing the same constant extension to more than one instruction is not considered a useful feature. Therefore, in one embodiment, the instruction decoder 113 can determine that this situation is an invalid operation and is not supported. Alternatively, this combination can be supported by applying the extension bit of the constant extension slot of each corresponding functional unit instruction marked as constant extension.

[0134] Special vector predicate instructions use registers in the predicate register file 234 to control vector operations. In the current embodiment, all of these SIMD vector predicate instructions operate on selected data sizes. Data sizes can include byte (8-bit) data, halfword (16-bit) data, word (32-bit) data, doubleword (64-bit) data, quadword (128-bit) data, and half-vector (256-bit) data. Each bit of the predicate register controls whether a SIMD operation is performed on the corresponding data byte. Operation of the P unit 246 allows for various complex vector SIMD operations based on more than one vector comparison. For example, two comparisons can be used to determine a range. A candidate vector is compared to a first vector reference having the minimum value of a range packed in a first data register. A second comparison of the candidate vector is performed using a second reference vector having the maximum value of the range packed in a second data register. Logical combinations of the two resulting predicate registers allow vector conditional operations to determine whether each data portion of the candidate vector is within or outside the range.

[0135] The L1 unit 221, S1 unit 222, L2 unit 241, S2 unit 242, and C unit 245 typically operate in single instruction multiple data (SIMD) mode. In this SIMD mode, the same instruction is applied to packed data from two operands. Each operand stores multiple data elements set in predetermined slots. SIMD operations are enabled by carry control at data boundaries. This carry control can operate on different data widths.

[0136] Figure 18 Carry control is shown. AND gate 1801 receives the carry output of bit N within the operand-wide arithmetic logic unit (64-bit for the scalar datapath side A 115 functional unit and 512-bit for the vector datapath side B 116 functional unit). AND gate 1801 also receives a carry control signal, which will be explained further below. The output of AND gate 1801 is provided to the carry input of bit N+1 of the operand-wide arithmetic logic unit. AND gates such as AND gate 1801 are positioned between each pair of bits at a possible data boundary. For example, for 8-bit data, such an AND gate would be located between bits 7 and 8, between bits 15 and 16, between bits 23 and 24, and so on. Each such AND gate receives a corresponding carry control signal. If the data size is the minimum value, each carry control signal is 0, effectively preventing carry propagation between adjacent bits. If the selected data size requires two arithmetic logic unit sections, the corresponding carry control signal is 1. Table 3 below shows example carry control signals for a 512-bit wide operand, which can be used by, for example, the vector datapath side B 116 functional unit that is divided into 8, 16, 32, 64, 128, or 256 bits. In Table 3, the upper 32 bits control the carry out of the upper bits (bits 128 to 511), and the lower 32 bits control the carry out of the lower bits (bits 0 to 127). There is no need to control the carry out of the most significant bit, so only 63 carry control signals are required.

[0137]

[0138] Table 3

[0139] For integer powers of 2 (2 N ) is common in the art. However, this carry control technique is not limited to integer powers of 2. Those skilled in the art will understand how to apply this technique to other data sizes and other operand widths.

[0140] Figure 19 A conceptual diagram of the stream engine of the present invention is shown. Figure 19The process of a single stream is shown. Stream engine 1900 includes a stream address generator 1901. Stream address generator 1901 sequentially generates addresses for the elements of the stream and provides these element addresses to system memory 1910. Memory 1910 calls the data (data elements) stored at the element addresses and provides these data elements to data first-in, first-out (FIFO) memory 1902. Data FIFO 1902 provides buffering between memory 1910 and CPU 1920. Data formatter 1903 receives data elements from data FIFO memory 1902 and provides data formatting according to the stream definition. This process will be described below. Stream engine 1900 provides the formatted data elements from data formatter 1903 to CPU 1920. The program on CPU 1920 consumes the data and generates output.

[0141] Stream elements typically reside in normal memory. Memory itself does not impose a particular structure on a stream. A program imposes structure by defining a stream by specifying the following stream attributes: the address of the first element of the stream; the size and type of the elements in the stream; the formatting of the data in the stream; and the sequence of addresses associated with the stream.

[0142] The stream engine defines the address sequence of the elements of a stream based on pointers that traverse memory. Multiple levels of nested loops control the paths taken by the pointers. The iteration count of a loop level indicates the number of times the level is repeated. The dimension gives the distance between the pointer positions of that loop level.

[0143] In a basic forward flow, the innermost loop always consumes physically contiguous elements from memory. The implicit dimension of this innermost loop is 1 element. The pointer itself moves from one element to another in a sequentially increasing order. At each level outside the inner loop, the loop moves the pointer to a new position based on the size of the dimension at that loop level.

[0144] This form of addressing allows the program to specify a regular path through memory in a small number of parameters. Table 4 lists the addressing parameters for elementary streams.

[0145]

[0146] Table 4

[0147] The above definition maps successive elements of the stream to increasing addresses in memory. This works for most algorithms, but not all. Some algorithms are better served by reading elements in the reverse order of the stream, with decreasing memory addresses. For example, discrete convolution computes the dot product of vectors, according to the formula:

[0148]

[0149] In most DSP code, f[] and g[] represent arrays in memory. For each output, the algorithm reads f[] in the forward direction, but reads g[] in the reverse direction. The actual filter restricts the index range of [x] and [tx] to a finite number of elements. To support this pattern, the stream engine supports reading elements in descending address order.

[0150] Matrix multiplication presents unique challenges for the streaming engine. Each element in a matrix product is the vector dot product between the rows of the first matrix and the columns of the second. Programs typically store matrices in either row-major or column-major order. Row-major order stores all elements of a single row contiguously in memory. Column-major order stores all elements of a single column contiguously in memory. Matrices are typically stored in the same order as the language's default array order. As a result, only one of the two matrices in a matrix multiplication maps to the streaming engine's two-dimensional stream definition. In a typical example, the first index traverses the columns of the first array but the rows of the second array. This problem is not unique to the streaming engine. The access pattern for matrix multiplication deviates significantly from most general-purpose memory hierarchies. Some software libraries transpose one of the two matrices so that both can be accessed row-wise (or column-wise) during the multiplication. The streaming engine supports implicit matrix transposition using transposed streams. Transposed streams avoid the cost of explicitly transforming the data in memory. Instead of accessing data in strict contiguous element order, the streaming engine effectively swaps the inner two loop dimensions in traversal order, fetching elements back into contiguous vector lanes along the second dimension.

[0151] This algorithm is efficient but impractical to implement for small element sizes. Some algorithms work on matrix tiles which are multiple columns and multiple rows. Therefore, the streaming engine defines a separate transpose granularity. The hardware imposes a minimum granularity. The transpose granularity must also be at least as large as the element size. The transpose granularity causes the streaming engine to retrieve one or more consecutive elements along dimension 0 before moving along dimension 1. When the granularity is equal to the element size, this will result in retrieving a single column from the row-major array. Otherwise, the granularity specifies retrieving 2, 4 or more columns at a time from the row-major array. This also applies to column-major layouts by swapping rows and columns in the description. The parameter GRANULE indicates the transpose granularity in bytes.

[0152] Another common matrix multiplication technique swaps the two innermost loops of the matrix multiplication. The resulting inner loop no longer reads the columns of one matrix but the rows of another. For example, the algorithm can hoist a term outside the inner loop and replace it with a scalar value. On a vector machine, the innermost loop can be implemented very efficiently with a single scalar-by-vector multiplication followed by a vector addition. The central processing unit core 110 of the present invention does not have a scalar-by-vector multiplication. Instead, the program must copy the scalar value over the entire vector length and use vector-by-vector multiplication. The stream engine of the present invention directly supports this and related usage models with an element copy mode. In this mode, the stream engine reads a particle smaller than the full vector size and copies the particle to fill the next vector output.

[0153] The streaming engine treats each complex number as a single element with two subelements, which give the real and imaginary parts (rectangular) or the magnitude and angle (polar) of the complex number. Not all programs or peripherals agree on the order in which these subelements should appear in memory. Therefore, the streaming engine provides the ability to swap two subelements of a complex number at no cost. This feature swaps halves of an element without interpreting its contents and can be used to swap pairs of subelements of any type, not just complex numbers.

[0154] Algorithms often prefer to work with high precision, but high precision values ​​require more storage and bandwidth than low precision values. Often, a program will store data in memory with low precision, promote these values ​​to higher precision for calculations, and then demote these values ​​to reduced precision for storage. The stream engine directly supports this by allowing algorithms to specify promotion to a first level type. In a preferred embodiment of the invention, for integer types, each subelement can be promoted to the size of the next larger type with sign or zero extension. The stream engine can support floating point promotion, and it is also feasible to promote 16-bit floating point values ​​and 32-bit floating point values ​​to 32-bit format and 64-bit format respectively.

[0155] The stream engine defines a stream as a discrete sequence of data elements, and the CPU core 110 consumes these data elements packed consecutively into vectors. Vectors are very similar to streams in that they contain multiple homogeneous elements with some implicit sequence. Because the stream engine reads streams, but the CPU core 110 consumes vectors, the stream engine must map streams to vectors in a consistent manner.

[0156] A vector consists of channels of equal size, each containing one subelement. The CPU core 110 designates the rightmost channel of a vector as channel 0, regardless of the device's current endian mode. Channel numbers increase from right to left. The actual number of channels in a vector varies depending on the length of the vector and the data size of its subelements.

[0157] Figure 20 A first example of lane allocation in a vector is shown. Vector 2000 is divided into eight 64-bit lanes (8x64 bits = 512-bit vector length). Lane 0 includes bits 0 to 63; lane 1 includes bits 64 to 125; lane 2 includes bits 128 to 191; lane 3 includes bits 192 to 255, lane 4 includes bits 256 to 319, lane 5 includes bits 320 to 383, lane 6 includes bits 384 to 447, and lane 7 includes bits 448 to 511.

[0158] Figure 21 A second example of lane allocation in a vector is shown. Vector 2100 is divided into 16 32-bit lanes (16x32 bits = 512-bit vector length). Lane 0 includes bits 0 to 31; row 1 includes bits 32 to 63; lane 2 includes bits 64 to 95; lane 3 includes bits 96 to 127; lane 4 includes bits 128 to 159; lane 5 includes bits 160 to 191; lane 6 includes bits 192 to 223; lane 7 includes bits 224 to 255; lane 8 includes bits 256 to 287; row 9 occupies bits 288 to 319; lane 10 includes bits 320 to 351; lane 11 includes bits 352 to 387; lane 12 includes bits 388 to 415; lane 13 includes bits 416 to 447; lane 14 includes bits 448 to 479; and lane 15 includes bits 480 to 511.

[0159] The stream engine maps the innermost stream dimension directly to the vector lanes. It maps earlier elements in that dimension to lower lane numbers, and later elements to higher lane numbers. This is true regardless of whether the particular stream advances in increasing or decreasing address order. Regardless of the order of the stream definitions, the stream engine stores elements in the vector in increasing lane order. For non-complex data, it places the first element retrieved by the first vector CPU core 110 in lane 0, the second in lane 1, and so on. For complex data, the stream engine places the first element in lanes 0 and 1, the second in lanes 2 and 3, and so on. Subelements within an element maintain the same relative order regardless of stream direction. For unswapped complex elements, this places the subelement with the lower address in each pair in the even-numbered lanes and the subelement with the higher address in each pair in the odd-numbered lanes. Swapping complex elements reverses this mapping.

[0160] The stream engine populates each vector CPU core 110, retrieving as many elements as possible from the innermost stream dimension. If the innermost dimension is not a multiple of the vector length, the stream engine stretches that dimension with zeros to an integer multiple of the vector length. Therefore, for high-dimensional streams, the first element of each iteration from the outer dimensions arrives in lane 0 of the vector. The stream engine always maps the innermost dimensions to consecutive lanes in the vector. For transposed streams, the innermost dimension consists of groups of subelements along dimension 1, not dimension 0, because the transposition swaps these two dimensions.

[0161] Compared to a one-dimensional stream, a two-dimensional stream exhibits significant variation. A basic two-dimensional stream retrieves smaller rectangles from a larger one. A transposed two-dimensional stream reads rectangles column by column rather than row by row. A recurrent stream, where the second dimension overlaps, first performs a repetitive cycle of finite impulse response (FIR) filter taps or FIR filter samples that provide a sliding window of input samples.

[0162] Figure 22 A basic two-dimensional stream is shown. The two internal dimensions represented by ELEM_BYTES, ICNT0, DIM1, and ICNT1 provide enough flexibility to describe retrieving a smaller rectangle 2220 having dimensions 2221 and 2222 from a larger rectangle 2210 having dimensions 2211 and 2212. In this example, rectangle 2220 is a 9 by 13 rectangle of 64-bit values, and rectangle 2210 is a larger 11 by 19 rectangle. The following stream parameters define this stream:

[0163] ICNT0=9

[0164] ELEM_BYTES=8

[0165] ICNT1=13

[0166] DIM1 = 88 (11 times 8)

[0167] Therefore, the iteration count in the 0-dimension 2221 is 9. The iteration count in the 1-dimension 2222 is 13. Note that ELEM_BYTES scales only the innermost dimension. The first dimension has ICNT0 elements of size ELEM_BYTES. The stream address generator does not scale the outer dimensions. Therefore, DIM1 = 88, or 11 elements, each scaled by 8 bytes.

[0168] Figure 23 The order of elements in this example stream is shown. The stream engine retrieves the elements of the stream in the order shown in sequence 2300. The first 9 elements come from the first row of rectangle 2220, jumping from 1 to 8 from left to right. The 10th to 24th elements come from the second row, and so on. When the stream moves from the 9th element to the 10th element ( Figure 23 At jump 9 in the inner loop, the stream engine calculates the new position based on the pointer position at the start of the inner loop, rather than ending at the position where the pointer was at the end of the first dimension. This makes DIM1 independent of ELEM_BYTES and ICNT0. DIM1 always represents the distance between the first bytes of each consecutive row.

[0169] Before dimension 0, the transposed stream is accessed along dimension 1. The following example shows several transposed streams, varying the transposition granularity. Figure 24 It is shown that a smaller rectangle 2420 (12x8) having dimensions 2421 and 2422 is extracted from a larger rectangle 2410 (14x13) having dimensions 2411 and 2412. Figure 24 , ELEM_BYTES is equal to 2.

[0170] Figure 25 The figure shows how the stream engine retrieves the stream of this example with a transposed granularity of 4 bytes; the retrieval pattern 2500 retrieves pairs of elements from each row (because the granularity of 4 is twice the ELEM_BYTES of 2), otherwise moving down the column. Once the bottom of a pair of columns is reached, the pattern is repeated with the next pair of columns.

[0171] Figure 26 26. The stream engine fetches 4 elements from each row (since a granularity of 8 is four times the ELEM_BYTES of 2) before moving on to the next row in the column, as shown in fetch pattern 2600.

[0172] So far, the streams examined have simply read each element from memory exactly once. A stream can read a given element from memory multiple times, effectively looping over a segment of memory. FIR filters have two common looping patterns. FIRs reread the same filter taps for each output. FIRs also read input samples from a sliding window. Two consecutive outputs will require input from two overlapping windows.

[0173] Figure 27 Details of stream engine 2700 are shown. Stream engine 2700 includes three main components: stream 0 2710; stream 1 2720; and shared L2 interface 2730. Stream 0 2710 and stream 1 2720 both include the same hardware operating in parallel. Stream 0 2710 and stream 1 2720 both share L2 interface 2730. Each stream 2710 and 2720 provides up to 512 bits per cycle to CPU core 110. The stream engine architecture achieves this through its dedicated stream paths and shared dual L2 interfaces.

[0174] Each stream engine 2700 includes a dedicated four-dimensional stream address generator 2711 / 2721, each of which can generate a new unaligned request per cycle. The address generator 2711 / 2721 outputs a 512-bit aligned address that overlaps with an element in the sequence defined by the stream parameters. This will be further described below.

[0175] Each address generator 2711 / 2711 is connected to a dedicated micro-table look-aside buffer (μTLB) 2712 / 2722. μTLB 2712 / 2722 translates a single 48-bit virtual address into a 44-bit physical address each cycle. Each μTLB 2712 / 2722 has eight entries, encompassing a minimum of 32kB for a 4kB page or a maximum of 16MB for a 2MB page. Each address generator 2711 / 2721 generates two addresses per cycle. μTLB 2712 / 2722 translates only one address per cycle. To maintain throughput, stream engine 2700 exploits the fact that most stream references will be within the same 4kB page. Therefore, address translations do not modify bits 0 through 11 of the address. If the aout0 and aout1 lines are in the same 4kB page (aout0[47:12] is the same as aout1[47:12]), the μTLB 2712 / 2722 only translates aout0 and reuses the upper bits of both addresses for translation.

[0176] The translated addresses are queued in the command queue 2713 / 2723. These addresses are aligned with information from the corresponding storage allocation and tracking blocks 2714 / 2724. The stream engine 2700 does not explicitly manage the μTLB 2712 / 2722. The system memory management unit (MMU) invalidates the μTLB as needed during context switches.

[0177] Storage allocation and tracking 2714 / 2724 manages the internal storage of the stream, detects data reuse, and tracks the life cycle of each data segment. This will be further described below.

[0178] The reference queues 2715 / 2725 store the reference sequences generated by the corresponding address generators 2711 / 2721. This information drives the data formatting network so that it can present data to the CPU core 110 in the correct order. Each entry in the reference queue 2715 / 2725 includes the information needed to read the data from the data store and align it with the CPU core 110. The reference queues 2715 / 2725 maintain the following information listed in Table 5 in each slot:

[0179]

[0180] Table 5

[0181] When address generator 2711 / 2721 generates a new address, storage allocation and tracking 2714 / 2724 inserts a reference into reference queue 2715 / 2725. When data becomes available and there is space in the stream-holding register, storage allocation and tracking 2714 / 2724 removes the reference from reference queue 2715 / 2725. When storage allocation and tracking 2714 / 2724 removes a slot reference from reference queue 2715 / 2725 and formats the data, it checks whether the reference represents the last reference to the corresponding slot. Storage allocation and tracking 2714 / 2724 compares the reference queue 2715 / 2725 removal pointer with the last reference recorded for the slot. If they match, storage allocation and tracking 2714 / 2724 marks the slot inactive after the data is completed.

[0182] The stream engine 2700 has data storage 2716 / 2737 for any number of elements. Deep buffering allows the stream engine to fetch ahead in the data stream, hiding memory system latency. The correct amount of buffering may vary between product generations. In the currently preferred embodiment, the stream engine 2700 uses 32 slots for each stream. Each slot holds 64 bytes of data.

[0183] The butterfly network 2717 / 2727 consists of a 7-stage butterfly network. The butterfly network 2717 / 2727 receives 128 bytes of input and produces 64 bytes of output. The first stage of the butterfly is actually half a stage. It collects matching unaligned retrieved bytes from the two slots and merges them into a rotated 64-byte array. The remaining 6 stages form a standard butterfly network. The butterfly network 2717 / 2727 performs the following operations: rotates the next element down into byte lane 0; promotes the data type by a power of 2 if necessary; swaps the real and imaginary components of complex numbers if necessary; and converts from big endian to little endian if the CPU core 110 is currently in big endian mode. The user specifies the element size, type promotion, and real / imaginary swapping as part of the stream parameters.

[0184] The stream engine 2700 attempts to retrieve and format data ahead of the CPU core 110's demand for it so that it can maintain full throughput. The holding registers 2718 / 2728 provide a small amount of buffering to keep the process fully pipelined. Aside from the fact that the stream engine 2700 provides full throughput, the holding registers 2718 / 2728 are not directly architecturally visible.

[0185] The two streams 2710 / 2720 share a pair of independent L2 interfaces 2730: L2 Interface A (IFA) 2733 and L2 Interface B (IFB) 2734. Each L2 interface provides 512 bits / cycle throughput directly to the L2 controller, for an aggregate bandwidth of 1024 bits / cycle. The L2 interfaces use a credit-based Multicore Bus Architecture (MBA) protocol. The L2 controller allocates its own pool of command credits to each interface. The pool should contain enough credits so that each interface can issue requests sufficient to achieve the full read return bandwidth when reading from L2 RAM, L2 cache, and Multicore Shared Memory Controller (MSMC) RAM.

[0186] To maximize performance, both flows can use both L2 interfaces, allowing a single flow to send a peak command rate of 2 requests / cycle. Each interface prefers one flow over the other, but this preference changes dynamically based on the request. The IFA 2733 and IFB 2734 always prefer the opposite flow; when the IFA 2733 prefers flow 0, the IFB 2734 prefers flow 1, and vice versa.

[0187] The arbitrator 2731 / 2732 before each interface 2733 / 2734 applies the following basic protocol every cycle with available credit. Arbitrator 2731 / 2732 checks whether the preferred stream has a command ready to send. If so, arbitrator 2731 / 2732 selects that command. Arbitrator 2731 / 2732 next checks whether the backup stream has at least two requests ready to send or a command but no credit. If so, arbitrator 2731 / 2732 retrieves the command from the backup stream. If either interface issues a command, the preferred and backup streams exchange their intentions for the next request. Using this simple algorithm, the two interfaces dispatch requests as quickly as possible while maintaining fairness between the two flows. The first rule ensures that each flow can send a request every cycle with available credit. The second rule provides a mechanism for one flow to borrow from the other interface when the second interface is idle. The third rule spreads the bandwidth requirements of each flow across the two interfaces, ensuring that neither interface itself becomes a bottleneck.

[0188] Coarse-grained rotators 2735 / 2736 enable the stream engine 2700 to support a transposed matrix addressing mode. In this mode, the stream engine 2700 swaps the two innermost dimensions of its multidimensional loops. This accesses the array column-by-column rather than row-by-row. Rotators 2735 / 2736 are architecturally invisible except when this transposed access mode is enabled.

[0189] A stream definition template provides the complete structure of a stream, including data. The iteration count and dimensions provide the bulk of the structure, while various flags provide the remaining details. The stream engine defines a single stream template for all streams that include data. All supported stream types fit within this template. The number above each field indicates the number of bytes within a 256-bit vector. The stream engine defines a four-level loop nest for addressing elements in the stream. Most fields in the stream template map directly to parameters in the algorithm. Figure 28 Flow Template Register 2800 is shown. The number above the field is the number of bytes within the 512-bit vector. Table 6 shows the flow field definitions for the flow template.

[0190]

[0191] Table 6

[0192] In the current example, DIM0 is always equal to ELEM_BYTES, which defines physically contiguous data. The stream template primarily consists of 32-bit fields. The stream template limits ICNT3 to 8 bits and the FLAGS field to 24 bits. The stream engine 2700 interprets all iteration counts as unsigned integers and all dimensions as unscaled signed integers. The above template fully specifies the type of element, the length and dimensions of the stream. The stream instruction specifies the starting address alone. This is typically specified via a scalar register in the scalar register file 211, which stores the starting address. This allows a program to open multiple streams using the same template.

[0193] Figure 29 2900 shows the subfield definitions of the flag field 2900. Figure 29 As shown, the flags field 2900 is 3 bytes or 24 bits. Figure 29 The number of bits in the field is shown in Table 7.

[0194]

[0195] Table 7

[0196] The Element Type (ELTYPE) field 2901 defines the data type of the element in the stream. The four-bit encoding of this field is defined as shown in Table 8.

[0197]

[0198]

[0199] Table 8

[0200] The subelement size determines the type used for type promotion and vector lane width. For example, when a stream requests type promotion, a 16-bit subelement is promoted to a 32-bit subelement. The vector lane width is important when the CPU core 110 operates in big-endian mode because it always arranges vectors in little-endian order.

[0201] The total element size determines the minimum granularity of the stream. In the stream addressing model, it determines the number of bytes the stream retrieves per iteration of its innermost loop. A stream always reads whole elements, whether in increasing or decreasing order. Therefore, the innermost dimension of a stream spans ICNT0 × total element size bytes.

[0202] The real-complex type determines whether the stream engine treats each element as a real number or the two parts of a complex number (real / imaginary or magnitude / angle). This field also specifies whether the two parts of the complex number are swapped. The total element size of a complex type is twice the size of its sub-element. Otherwise, the sub-element size is equal to the total element size.

[0203] DIR bit 2902 determines the fetch direction of the inner loop (Loop 0). If DIR bit 2902 is 0, then Loop 0 fetches in the forward direction toward increasing addresses. Inner loop (Loop 0). If DIR bit 2902 is 1, then Loop 0 fetches in the reverse direction toward decreasing addresses.

[0204] The TRANSPOSE field 2903 determines whether the stream engine accesses the stream in transposed order. Transposed order swaps two internal addressing levels. The TRANSPOSE field also indicates the granularity at which it transposes the stream. The four-bit encoding of this field is defined as shown in Table 9.

[0205]

[0206] Table 9

[0207] The stream engine 2700 actually transposes at a granularity different from the element size. This allows the program to retrieve multiple columns of elements from each row. The transposition granularity must not be smaller than the element size.

[0208] The PROMOTE field 2904 controls whether the stream engine promotes sub-elements in the stream and the type of promotion. When enabled, the stream engine 2700 promotes types by a single power of 2. The encoding of the two bits of this field is defined as shown in Table 10.

[0209] PROMOTE describe 00 No improvement 01 Unsigned integer promotion, zero extension. 10 Signed integer promotion, sign extension. 11 Floating-point promotion

[0210] Table 10

[0211] When the stream specifies "no promotion", each subelement will occupy a vector channel with a width equal to the size specified by ELTYPE. Otherwise, each subelement occupies a vector channel twice as large. When PROMOTE is 00, the stream engine retrieves half the data from memory to satisfy the same number of stream retrievals.

[0212] Promotion modes 01b and 10b treat the incoming sub-elements as unsigned and signed integers, respectively. For unsigned integers, the stream engine promotes by filling the new bits with zeros. For signed integers, the stream engine promotes by filling the new bits with a copy of the sign bit. Positive signed integers have their most significant bit equal to 0. When a positive signed integer is promoted, the new bits are filled with zeros. Negative signed integers have their most significant bit equal to 1. When a negative signed integer is promoted, the new bits are filled with 1s.

[0213] Promotion mode 11b treats the passed child elements as floating-point numbers. Floating-point promotion treats each child element as a floating-point type. The stream engine supports two types of floating-point promotion: short float (16-bit) to single-precision float (32-bit); and single-precision float (32-bit) to double-precision float (64-bit).

[0214] The THROTTLE field 2905 controls how aggressively the stream engine fetches before the CPU core 110. The encoding of the two bits of this field is defined as shown in Table 11.

[0215] THROTTLE describe 00 Minimize throttling and maximize early retrieval 01 Less throttling, more early retrieval 10 More throttling, less early retrieval 11 Maximize throttling and minimize premature retrieval

[0216] Table 11

[0217] THROTTLE does not change the meaning of the stream and serves only as a hint. The streaming engine may ignore this field. Programs should not rely on specific throttling behavior for program correctness, as the architecture does not specify the precise throttling behavior. THROTTLE allows programmers to provide hints to the hardware about the program's own behavior. By default, the streaming engine attempts to stay as far ahead of the CPU core 110 as possible to hide as much latency as possible while providing full streaming throughput to the CPU core 110. While several key applications require this level of throughput, it can result in suboptimal system-level behavior for others. For example, the streaming engine discards all data retrieved via context switches. Therefore, in systems with a large number of context switches, aggressive early fetching can result in wasted bandwidth. Aggressive early fetching in these systems only makes sense if the CPU core 110 consumes data very quickly.

[0218] The CPU core 110 exposes the stream engine to programs through a small number of instructions and dedicated registers. The STROPEN instruction opens a stream. The STROPEN command specifies a stream number, indicating whether to open stream 0 or stream 1. STROPEN specifies the stream template register that stores the stream template described above. Table 12 lists a summary of the STROPEN instruction.

[0219] Summary describe Stream start address register Scalar register storing the stream start address Serial Number Stream 0 or Stream 1 Stream Template Register Vector registers storing stream template data

[0220] Table 12

[0221] The stream start address register is preferably a scalar register in the general scalar register file 211. The STROPEN instruction specifies stream 0 or stream 1 through its opcode. The stream template register is preferably a vector register in the general vector register file 221. If the specified stream is active, the STROPEN instruction closes the previous stream and replaces it with the specified stream.

[0222] The STRCLOSE command closes a stream. The STRCLOSE command specifies the stream number of the stream to be closed.

[0223] The STRSAVE instruction captures enough state information about a specified stream to restart the stream in the future. The STRRSTR instruction resumes a previously saved stream. The STRSAVE instruction does not save any data for a stream. The STRSAVE instruction saves only metadata. The stream retrieves data in response to the STRRSTR instruction.

[0224] A stream engine is in one of three states: inactive; active; or frozen. When inactive, the stream engine does nothing. Any attempt to retrieve data from an inactive stream engine is an error. A stream engine is inactive until a program opens a stream. It becomes inactive after a program consumes all elements from a stream or closes the stream. Programs that use streams explicitly activate and deactivate the stream engine. In combination with its own explicit save and restore operations, the operating environment manages streams across context switch boundaries through the stream engine's implicit freeze behavior.

[0225] An active stream engine has a stream associated with it. A program can retrieve new stream elements from an active stream engine. A stream engine remains active until one of the following occurs. When a stream retrieves the last element from the stream, it becomes inactive. When a program explicitly closes a stream, it becomes inactive. A stream engine freezes when the central processing unit core 110 responds to an interrupt or exception. When the stream engine freezes, the frozen stream engine captures all state required to resume the stream. The stream engine freezes in response to interrupts and exceptions. This is combined with special instructions to save and restore the frozen stream context so that the operating environment can switch contexts cleanly. A frozen stream is reactivated when the central processing unit core 110 returns to the context of the interrupt.

[0226] Figure 30 is a partial schematic diagram 3000 illustrating the above-described stream input operand encoding. Figure 30 The src1 field 1305 of one instruction is shown decoding the corresponding src1 input of the functional unit 3020. These same circuits are replicated for the src2 / cst field 1304 and the src2 input of the functional unit 3020. Additionally, these circuits are replicated for each instruction within an execution packet that can be dispatched simultaneously.

[0227] The instruction decoder 113 receives bits 13 to 17 of the src1 field 1305 of the instruction. The opcode field (bits 4 to 12 of all instructions and bits 28 to 31 of additional unconditional instructions) explicitly specifies the corresponding functional unit 3020. In this embodiment, the functional unit 3020 can be the L2 unit 241, the S2 unit 242, the M2 unit 243, the N2 unit 244, or the C unit 245. Figure 30 The relevant portion of the instruction decoder 113 shown in FIG. 1 decodes the src1 bit field 1305. The sub-decoder 3011 determines whether the src1 bit field 1305 is within the range of 00000 to 01111. If so, the sub-decoder 3011 provides the corresponding register number to the global vector register file 231. In this example, the register field is the four least significant bits of the src1 bit field 1305. The global vector register file 231 calls the data stored in the register corresponding to the register number and provides the data to the src1 input of the functional unit 3020. This decoding is generally known in the art.

[0228] Subdecoder 3012 determines whether src1 bit field 1305 is within the range of 10000 to 10111. If so, subdecoder 3012 provides the corresponding register number to the corresponding local vector register file. If the instruction is directed to L2 unit 241 or S2 unit 242, the corresponding local vector register file is local vector register field 232. If the instruction is directed to M2 unit 243, N2 unit 244, or C unit 245, the corresponding local vector register file is local vector register field 233. In this example, the register field is the three least significant bits of src1 bit field 1305. Local vector register file 231 calls the data stored in the register corresponding to the register number and provides the data to the src1 input of functional unit 3020. The corresponding local vector register file 232 / 233 calls the data stored in the register corresponding to the register number and provides the data to the src1 input of functional unit 3020. This decoding is generally known in the art.

[0229] Subdecoder 3013 determines if src1 bit field 1305 is 11100. If so, subdecoder 3013 provides a stream0 read signal to stream engine 2700. Stream engine 2700 then provides the stream0 data stored in holding register 2718 to the src1 input of functional unit 3020.

[0230] Subdecoder 3014 determines whether src1 bit field 1305 is 11101. If so, subdecoder 3014 provides a stream0 read signal to stream engine 2700. Stream engine 2700 then provides the stream0 data stored in holding register 2718 to the src1 input of functional unit 3020. Subdecoder 3014 also provides an advance signal to stream0. As previously described, stream engine 2700 advances to store the next sequential data element of stream0 in holding register 2718.

[0231] Subdecoder 3015 determines if src1 bit field 1305 is 11110. If so, subdecoder 3015 provides a stream 1 read signal to stream engine 2700. Stream engine 2700 then provides the stream 1 data stored in holding register 2728 to the src1 input of functional unit 3020.

[0232] Subdecoder 3016 determines whether src1 bit field 1305 is 11111. If so, subdecoder 3016 provides a stream 1 read signal to stream engine 2700. Stream engine 2700 then provides the stream 1 data stored in holding register 2728 to the src1 input of functional unit 3020. Subdecoder 3014 also provides an advance signal to stream 1. As previously described, stream engine 2700 advances to store the next sequential data element of stream 2 in holding register 2728.

[0233] Similar circuitry is used to select data provided to the scr2 input of functional unit 3002 in response to the bit encoding of src2 / cst field 1304. Constant inputs may be provided to the src2 input of functional unit 3020 in the manner described above.

[0234] The exact number of instruction bits dedicated to operand specification and the number of data registers and data streams are design choices. Those skilled in the art will recognize that other quantity options described in this application are feasible. In particular, the specification of a single global vector register file and the omission of local vector register files are feasible. The present invention uses a bit encoding of the input operand select field to specify stream read and another bit encoding to specify stream read and forward flow.

[0235] Figure 3127 is a partial block diagram showing the components that support cache storage of stream data for retrieval of stream 0. The components for stream 1 are similar. Data memory 2716 is conceptually a data first-in-first-out (FIFO) buffer. In a preferred embodiment, this FIFO is implemented as a cache, as described below.

[0236] Early address generator 3101 generates addresses for memory fetches. Late address generator 3102 generates the addresses of the next element(s) to be loaded into stream head register 2718. Stream data memory 2716 buffers stream data between its retrieval from memory and its provision to the functional units of CPU core 110. Data movement is controlled with respect to data stored in reference queue 2515.

[0237] Figure 32 Detail of multiple cache lines used in stream data memory 2716 is shown. Cache structure 3200 includes cache lines 3210, 3220, and 3220, representing the internal structure. Cache lines 3210, 3220, and 3230 include: an address tag 3211; a valid bit 3212; a ready bit 3213; an action bit 3214; a least recently used (LRU) indicator 3215; a parity bit 3216; and multiple data words 3217. The bit length of data word 3217 is set by the minimum addressable data size of central processing unit core 110. This is typically 8 bits / 1 byte.

[0238] The cache structure 3200 stores data from more distant memory that is accessed by multi-bit addresses. The cache structure 3200 is organized to facilitate this storage and to help find such data in the cache. Each cache line 3210, 3220, and 3230 typically stores 2 N 3201 within the corresponding cache lines 3210, 3220, and 3230. The location of the data word 3217 along dimension 3201 within the corresponding cache lines 3210, 3220, and 3230 serves as a proxy for the least significant bits of the corresponding address.

[0239] To determine whether a memory access is to data cached within cache structure 3200 (a cache hit), cache structure 3200 compares the address tags 3211 of all cache lines with the most significant bits of the accessed memory location. When a match is detected, the location within the cache line along dimension 3201 corresponds to the least significant bits of the address that allow identification of the accessed data word.

[0240] Table 13 records the characteristics of the valid, ready, active, and last reference fields.

[0241] Effective A single bit indicating whether the tag address is valid. Ready A single bit indicating that data has arrived at that address. Active A single bit indicating whether there are any outstanding references to the data. Final reference Indicates the value of the latest reference for this slot in the reference queue

[0242] Table 13

[0243] Each cache line 3210, 3220, and 3230 includes a corresponding valid bit 3212. A first state of the valid bit 3212 indicates that the corresponding data word 3217 of the cache line is valid. The opposite state of the valid bit indicates that the corresponding data word 3217 is not valid. There are several situations in which data stored in the cache structure 3200 is not valid. After the initial activation of the digital signal processor system 100, the cache structure 3200 will not be loaded. Therefore, the cache structure 3200 will not store valid data. Therefore, all cache lines are initially marked as not valid. During a cache access, a comparison of the requested address with the address tag 3211 will not detect a match unless the corresponding valid bit 3212 indicates that the data is valid.

[0244] Each cache line 3210, 3220, and 3230 includes a corresponding ready bit 3213. A first state of the ready bit 3213 indicates that no data for the corresponding cache line has arrived. A second state of the ready bit 3213 indicates that fetched data for the cache line has arrived.

[0245] Each cache line 3210, 3220, and 3230 includes a corresponding active bit 3214. A first state of the active bit 3214 indicates that a reference to the data of the corresponding cache line is pending. A second state of the active bit 3214 indicates that a reference to the data of the cache line is pending.

[0246] Each cache line 3210, 3220, and 3230 includes a corresponding last reference field 3215. The last reference field 3215 is typically a plurality of bits. The last reference field 3215 indicates the most recent reference to that slot in the reference queue. As described in further detail below, this information is used to allocate cache lines.

[0247] Each cache line 3210, 3220, and 3230 includes a corresponding parity bit 3216. The generation and use of the parity bits 3216 will be described below.

[0248] Table 14 details the interaction of the Valid, Ready, and Active bits.

[0249]

[0250] Table 14

[0251] Using this metadata, storage allocation and tracking 2714 / 2724 can identify data reuse opportunities in the stream. Storage allocation and tracking 2714 / 2724 performs the following steps for each address. It compares the address to the associated tag in the tag array. On a hit, it cancels the command associated with the address. On a miss, it allocates a free slot, sets Active = 1, Ready = 0, and updates the outgoing command to direct the data being retrieved to that slot. In either case, a slot number is associated with the address. Storage allocation and tracking 2714 / 2724 inserts the reference into the reference queue. Storage allocation and tracking 2714 / 2724 sets Active = 1 and updates Last Reference to the position of the reference in the reference queue. This is the value of the reference queue's insertion pointer at the time of insertion. This process converts the generated address into a slot number representing the data. From this point on, the stream engine does not need to track addresses directly.

[0252] To maximize reuse and minimize stalls, the stream engine 2700 allocates slots in the following order: if available in FIFO order, the slot after the most recently allocated one; if available, the lowest numbered available slot; if no slot is available, then stall and repeat these two steps until a successful allocation is made. This will tend to allocate slots in FIFO order, but avoid stalls if a particular reuse pattern works against that order.

[0253] Figure 33 A partial schematic diagram of an early address generator 3101 is shown. Early address generator 3102 forms the address for retrieving the next element in the defined stream of the corresponding stream engine. Start address register 3201 stores the starting address of the data stream. As previously described, start address register 3301 is preferably a scalar register in the global scalar register file 211, specified by the STROPEN instruction, which opens the corresponding stream. As is known in the art, the start address can be copied from the specified scalar register and stored locally at the corresponding address generator 2711 or 2721. The first loop of the stream utilizes Loop0 count register 3311, adder 3312, multiplier 3313, and comparator 3314. Loop0 count register 3311 stores a working copy of the iteration count of the first loop (Loop0). For each iteration of Loop0, adder 3312 is triggered by the Next Address signal, incrementing the loop count by 1, which is stored in Loop0 count register 3311. Multiplier 3313 multiplies the current loop count by the number ELEM_BYTES. ELEM_BYTES is the size (in bytes) of each data element in the loop. Loop0 iterates over physically consecutive data elements in memory with a step size of ELEM_BYTES.

[0254] Comparator 3314 compares the count stored in Loop0 Count Register 3311 (after being incremented by adder 3313) with the value of ICNT0 2810 from the corresponding Flow Template Register 2800. When the output of adder 3312 equals the value of ICNT0 2810 of Flow Template Register 2800, the iteration of Loop0 is complete. Comparator 3314 generates an active Loop0 End signal. Loop0 Count Register 3311 is reset to 0, and an iteration of the next higher loop, in this case Loop1, is triggered.

[0255] The circuit for higher loops (Loop1, Loop2, Loop3) is similar to Figure 33 The circuit shown. Each loop includes a corresponding working cycle count register, an adder, a multiplier, and a comparator. The adder of each loop is triggered by the loop end signal of the previous loop. The second input of each multiplier is the corresponding dimension DIM1, DIM2, and DIM3 of the corresponding flow template. The comparator of each loop compares the working cycle register count with the corresponding iteration value ICNT1, ICTN2, and ICTN3 of the corresponding flow template register 2800. The loop end signal generates the iteration of the next higher loop. The loop end signal from loop3 ends the flow.

[0256] Figure 33 The generation of the early loop0 count is shown. The early loop0 count is equal to the update data stored in the corresponding working count register 3311. The early loop0 count is updated at each change in the working loop0 count register 3311. The early loop counts of the higher loops (loop1, loop2, loop3) are generated similarly.

[0257] Figure 33 The generation of the early loop0 address is shown. The early loop0 address is equal to the data output from the multiplier 3313. The early loop0 address is updated every time the working loop0 count register 3311 changes. Similar circuits for Loop1, Loop2, and Loop3 generate corresponding early loop addresses.

[0258] Figure 34A and Figure 34B3400 is a flow chart illustrating a data retrieval method for a flow using early address generator 3101. Flowchart 3400 begins at entry point A to test block 3401. Test block 3401 determines whether the next retrieval from memory is required. A preferred method for determining this is described in detail below. If test block 3201 determines that a memory retrieval is not required (no at test block 3201), the process loops back to test block 3401 for another determination. The process remains at test block 3401 until a memory retrieval is required (yes at test block 3401).

[0259] When a memory fetch is required (yes at test block 3401), a tag comparison is performed at block 3402. The most significant bit of the next fetch address is compared with the address tag 3211 of each cache line in the stream data memory 2716. As previously described, a match is not confirmed unless the corresponding valid bit 3212 indicates valid.

[0260] Test block 3403 determines whether the address comparison of block 3402 found a match (hit). If test block 3403 determines a match (hit) (yes at test block 3402), flow proceeds to block 3404. Block 3404 associates the current fetch address with the cache line / slot number that generated the hit. Once a hit occurs, the stream data for the current fetch address has been stored in stream data memory 2717. Block 3155 updates the reference queue. The corresponding reference queue 2715 or 2725 stores data indicating the location of the data within the cache. As previously described, this data stored in the reference queue allows the vector in the corresponding first register 2718 / 2728 to be called and aligned with it.

[0261] Block 3406 generates the next address in the stream. This is typically the next element in loop 0. However, the current stream element may be the end of one of the nested loops. The next address is generated in the manner described above based on the specified stream data pattern. Flow returns to test block 3401 via entry point A.

[0262] In the event of a miss (no at test block 3403), the data corresponding to the current fetch address is not stored in the corresponding stream data memory 2616 or 2726. Process 3400 attempts to allocate a new cache line to the current fetch address. Test block 3407 determines whether the cache line is available for allocation. As described below, unused cache lines may not be available for allocation. Once allocated, a cache line can only be deallocated by consuming the stream data elements stored in the cache line read from the corresponding first register 2718 / 2728 via the central processing unit core 110. Due to the nature of data streaming, all previously allocated cache lines store data to be used by the central processing unit core 110 before the data corresponding to the current fetch address. Therefore, cache lines will not be evicted because they store data required before the current data. Therefore, the cache line cannot be allocated until all stream data stored in the cache line has been consumed by the central processing unit core 110. As described below, such data consumption is a necessary but not sufficient condition for a cache line to be reallocated. If no cache line is available for allocation (no at test block 3407), block 3408 stalls the process for one memory cycle. The process remains in this loop until at least one cache line is available for allocation (yes at test block 3407).

[0263] If at least one cache line is available for allocation (yes at test block 3407), test block 3409 determines whether the cache line following the last used cache line is available for allocation. If the cache line is available for allocation (yes at test block 3409), block 3410 allocates the cache line. If the cache line is not available for allocation (no at test block 3409), block 3411 allocates the lowest available cache line. The preferred mode of allocation is first-in-first-out (FIFO) order. Therefore, the preferred cache line to be allocated is the cache line following the last used cache line. Therefore, active data will tend to rotate within the cache line allocated after consumption by the central processing unit core 110. This allocation pattern tends to allocate slots in FIFO order. The allocation pattern described above (allocating the lowest available cache line if the cache line following the last used cache line is not available) avoids stalls if a particular data reuse pattern operates against this FIFO order.

[0264] When a cache line is allocated for the current fetch address, flow proceeds to block 3412. Block 3412 sets the valid bit of the newly allocated cache line to 1, the ready bit to 0, and the active bit to 1. This indicates that the newly allocated cache line is valid, not ready, and data is in flight. The next block 3413 requests cache service for the newly allocated cache line. In a preferred embodiment of the present invention, streaming data bypasses L1D cache 123 and requests data directly from L2 cache 130. This design choice prevents streaming data from interfering with other data used by CPU core 110.

[0265] Test block 3414 waits for data to be returned to stream data memory 2716. The latency of the data return is unknown and variable. This newly retrieved data may be stored in L2 cache 130 and available quickly. This data may not be stored in L2 cache 130 and must therefore be retrieved from another level of memory. This other level of memory may be shared among several requesters. In this case, there is typically arbitration between the requesters for access. Therefore, memory latency is unknown and variable.

[0266] Once the requested data is returned (yes at test block 3414), block 3415 stores the data in the allocated cache line. Block 3416 changes the corresponding ready bit to 1, indicating that the data is ready for consumption.

[0267] Process flow proceeds to block 3404. Block 3404 associates the current fetch address with the cache line / slot number that generated the hit. Block 3405 updates the reference queue. The corresponding reference queue 2715 or 2725 stores data indicating the location of the data within the cache. Block 3406 generates the next address in the flow. Flow returns to test block 3401 via entry point A.

[0268] Figure 35 Similar to Figure 33 , shows an exemplary circuit for generating a late Loop0 count and a late Loop0 address. The first loop of this flow uses a Loop0 count register 3511, an adder 3512, a multiplier 3513, and a comparator 3514. The multiplier 3513 has a second input that receives ELEM_BYTES. Figure 35Each iteration is triggered by a stream advance signal. Therefore, the count value of the later counting circuit and the address value of the later address reflect the next data to be provided to the functional unit. This will be the stream element stored in the corresponding first register 2218 / 2228. The circuits of loops Loop1, Loop2, and Loop3 are similar, except that the second input of the corresponding multiplier is the corresponding dimension DIM1, DIM2, and DIM3 of the corresponding stream template. The loop end signal generates the next iteration of the higher loop.

[0269] As described above, each stream engine maintains two copies of loop counts and loop addresses: an early copy indicating the current position of the stream engine; and a late copy indicating the last iteration submitted by the CPU core 110. In most cases, the late copy provides the clearest indication of the current state of the program. The early copy simply indicates how far ahead of the CPU core 110 the stream engine has fetched. The difference between the early and late copies reflects the amount of stream data stored in data memory 2217 / 2237. The count data and address data are related. Any particular embodiment may include only the count data, only the address data, or both.

[0270] It is known in the art that Figure 33 and Figure 35 Any count shown can be represented as a count down or a count up. Figure 33 and Figure 35 An up-counting embodiment is shown. Those skilled in the art will understand how to modify the initialization and comparison to implement the present invention in a down-counting circuit. Additionally, those skilled in the art will understand how to modify the comparison to implement the described addressing mode.

[0271] Figure 36Flowchart 3600 illustrates how to submit a stream head register using the late address generator 3102. Flowchart 3600 begins at test block 3601. Test block 3601 determines whether the next data element of the data stream is to be loaded into the corresponding stream head register. As previously described, in a preferred embodiment, the stream data remains in the stream head register, where it can be retrieved multiple times by multiple functional units until the stream engine encounters a read and advance operand encoding. After detecting the read and advance operand encoding, the stream engine provides the current data in the stream head register to the requesting functional unit and replaces the data in the stream head register with the next sequential data element in the data stream. As previously described, depending on the relationship between the specified stream data size and the vector size, the stream engine may fill the stream head register with more than one stream data element. If the stream head register data is not to be replaced (no at test block 3601), control loops back to test block 3601 for another determination. Flow remains at test block 3601 until the data stored in the stream head register needs to be replaced (yes at test block 3601).

[0272] When the first register needs to be replaced (yes at test block 3601), block 3602 obtains the address of the data for the next first register fill. In a preferred embodiment, this determination is made via the corresponding reference queue 2715 / 2725. Text block 3603 determines whether the cache line just accessed is valid. If the cache line is valid (yes at text block 3603), block 3604 calls the data into a temporary buffer. If the cache line is not valid (no at text block 3603), block 3605 zero-fills the temporary buffer.

[0273] In either case, block 3606 rotates and formats the data. This process has been described above. Data rotation aligns the data elements of the stream with the appropriate slots in the header register. As previously described, data formatting includes complex sub-element swapping, data promotion, transposition, and replication. After rotation and formatting, block 3607 stores the data in the corresponding header register 2718 / 2728 in the vector slot as previously described.

[0274] Figure 37An exemplary bit assignment for an extended control register (ECR), designated SEn_ICNT 3700, is shown. According to a preferred embodiment, stream engine 2700 includes two such ECRs: SE0_INCT for stream 0 and SE1_INCT for stream 1. Each SEn_ICNT stores, for each open stream, the early loop count and late loop count of the loop defined in the corresponding stream template. SEn_ICNT 3700 allows programmers to determine the stream engine's position within a given stream when debugging a program. In a preferred embodiment, SEn_ICNT 3700 is accessible only by a debugger. Any other attempt to access SEn_ICNT 3700 generates a permission error. SEn_ICNT 3700 includes an early loop0 count 3701, an early loop1 count 3702, an early loop2 count 3703, an early loop3 count 3704, a late loop0 count 3705, a late loop1 count 3706, a late loop2 count 3707, and a late loop3 count 3708. Each count value field has a bit width at least as large as the bit width of the corresponding iteration count of the corresponding stream template register. The corresponding early count and late count are similar, but updated at different times, as will be further described below.

[0275] Figure 38 An exemplary bit assignment for an ECR, designated SEn_ADDR 3800, is shown. According to a preferred embodiment, stream engine 2700 includes two such ECRs: SE0_ADDR for stream 0 and SE1_ADDR for stream 1. Each SEn_ADDR stores, for each open stream, the early and late addresses of the loop defined in the corresponding stream template. SEn_ADDR 3800 allows programmers to determine the stream engine's location within a given stream when debugging a program. In a preferred embodiment, SEn_ADDR 3800 is accessible only by a debugger. Any other attempt to access SEn_ADDR 3800 generates a permission error. SEn_ADDR 3800 includes an early Loop 0 address 3801, an early Loop 1 address 3802, an early Loop 2 address 3803, an early Loop 3 address 3804, a late Loop 0 address 3805, a late Loop 1 address 3806, a late Loop 2 address 3807, and a late Loop 3 address 3808. Each address field has a bit width at least as large as the bit width of the address of processor 100. The corresponding early and late addresses are similar but updated at different times, as will be described further below.

[0276] The Stream Engine 2700 reports a range of faults, including: programming errors; memory translation faults; bit errors detected in the storage array; bus errors reported by the system; and function failures. The Stream Engine reports errors synchronously, aligned with program execution.

[0277] The stream engine 2700 synchronously reports faults by marking data from the stream engine with a flag indicating the fault. This marked data triggers an internal CPU core 110 exception event if and only if the program attempts to consume the data. Any instruction that attempts to consume data marked as faulty generates an internal CPU core 110 exception event. The relationship between the fault identified by the stream engine 2700 and the instruction that experienced the exception is independent of the stall time.

[0278] When stream engine 2700 detects a fault to be reported, it marks the affected data in internal memory 2716 / 2726. In parallel, stream engine 2700 records the details of the error in the corresponding extended control registers: fault address register SEn_FAR and fault source register SEn_FSR. Each stream has its own set of extended control registers SEn_FAR and SEn_FSR. Extended control registers SEO_FAR and SEO_FSR store data for stream 0, and extended control registers SE1_FAR and SE1_FSR store data for stream 1. Depending on the error, stream engine 2700 can also disable the generation of further requests for the stream. In a preferred embodiment, fault address register SEn_FAR and fault source register SEn_FSR are accessible only by a debugger.

[0279] The stream engine 2700 records the virtual address that triggers the error in the corresponding extended control register SEn_FAR. Figure 39 , an exemplary SEn_FAR register 3900 is shown. In the current embodiment, each SEn_FAR register is 64 bits. The first field 3901 stores the virtual address retrieved by the stream engine that generated the fault. In a preferred embodiment, this virtual address is 49 bits (bits 0 to 40). In the current example, the remaining bits of the SEn_FAR register (second field 3902) are reserved and unused. If the virtual address of an embodiment exceeds 49 bits, these bits can be used.

[0280] The flow engine 270 records the fault status in the corresponding register SEn_FSR. Figure 40An exemplary SEn_FSR register 4000 is shown in FIG. In the current embodiment, each SEn_FSR register is 64 bits. The first field 4001 stores details of the fault encountered. In the current exemplary embodiment, the second field 4002 (the remaining bits of the SEn_FSR register 4000) is reserved and unused. Table 15 shows the encoding of the details field 4001 in the SEn_FSR register 4000.

[0281]

[0282]

[0283] Table 15

[0284] As shown in Table 15, bit 12, the most significant bit of bit field 4001, is 0 if no error has occurred and 1 if an error has occurred. One set of codes in bit field 4001 is reserved to indicate memory management unit errors. Memory management unit errors include memory translation failures such as: lack of permissions; lack of a valid translation entry for the provided virtual address; and actual physical errors encountered during the translation process. A second set of codes in bit field 4001 is reserved to indicate a L2 cache error. Other codes indicate streaming engine errors.

[0285] The encoding of "10000000000001" indicates a data error in the stream engine's internal memory. This indicates an error in the corresponding data memory 2716 / 2726. In a preferred embodiment, parity bits are stored along with the data bits. When used, a new parity is calculated based on the data and compared to the stored parity. A difference between the new parity and the stored parity indicates an internal memory error.

[0286] The code of "1000000000010" indicates a stream template error in the stream engine. When opening a stream, the CPU core 110 indicates the stream template registers that store the parameters for the data stream. The stream template registers ( Figure 28 and Figure 29 Some possible data encodings generate combinations of options not supported by the preferred embodiment. These include: element size after promotion and element repetition exceeds the vector length; transpose granule size after promotion, element repetition, and decimation exceeds the vector length; invalid dimension or address alignment of the transpose stream; decimation enabled while promotion is disabled; out-of-range iteration count for the transpose stream; reserved encoding used in any field; and non-zero value in any reserved field. The latter two errors are indicative of an incorrectly specified template.

[0287] The encoding of "1000000000011" indicates a stream engine activity mismatch. This occurs when the CPU core 110 attempts an operation that is inconsistent with the current stream state (inactive, active, frozen).

[0288] The encoding of "1000000000100" instructs the CPU core 110 to falsely read when the stream is invalid. An invalid stream cannot be read. Attempting to read an invalid stream generates this error.

[0289] The encoding of "1000000000101" indicates that address bit 48 is flipped. Although a virtual address has 49 bits, it is typically stored in a 64-bit scalar register. The CPU core 110 sign-extends the 49-bit virtual address to 64 bits for storage. Therefore, changes to bit 49 (the sign bit) of the virtual address are not allowed. Any streaming data retrieval operation triggers bit 49 of the virtual address, generating this error.

[0290] The encoding of "1000000000110" indicates a L2 cache fragmented data return. A streaming data fetch that results in a fragmented data return from the combined L2 cache 130 generates this error.

[0291] The encoding is "1000000000111" is reserved and not used.

[0292] Upon detecting any of these stream engine faults, the stream engine attempts to store the virtual address of the data fetch that generated the error in SEn_FAR and the corresponding error code in SEn_FSR, as described above. The stream engine is not always able to associate an address with every fault. In a preferred embodiment, the stream engine cannot report the error address in the following situations: invalid stream state (bad STRRSTR) when activating frozen stream; bit errors in the data memory array during transposition of stream; bus errors reported by the system during transposition of stream. In those cases, the stream engine clears SEn_FAR to 0. Because the stream engine can generate multiple addresses in parallel to satisfy a single data stage assigned to the CPU core 110, the stream engine selects one of these addresses to report if both addresses encounter a fault. The stream engine does not define an architectural order among these parallel addresses; rather, the address the stream engine chooses to report a fault when two faults occur in parallel is implementation-defined.

[0293] When these errors occur, the stream engine does not immediately trigger an error to the CPU core 110. As the data is consumed by the CPU core 110, the fault status flows through the stream engine data memory 2716 / 2726 along with the affected data. If and when the program actually consumes the data, the CPU core 110 triggers an internal exception event. Due to the pipelined nature of the CPU core 110, many cycles may elapse between when the stream engine detects the fault and when the CPU core 110 takes the exception. If the program never consumes the affected data, the CPU core 110 will never generate an exception event. For example, if a program opens a stream with an invalid template and then closes the stream without reading any data from it, the stream engine will not cause the CPU core 110 to take an exception. This behavior treats the stream engine fetch as speculative until the program running on the CPU core 110 actually consumes the data. If the fault is recoverable, such as a page fault, the stream will resume at the point of the data item that triggered the exception when the CPU core 110 returns from the exception event handler.

[0294] Figure 41 The error detection and correction of the present invention is shown. Figure 1 and Figure 27 Components shown in FIG. 1 are given the same reference numerals. Figure 41 Connections between the central processing unit core 110 , a level one instruction cache (L1I) 121 , a level one data cache (L1D) 123 , and a level two combined instruction / data cache (L2) 130 are shown.

[0295] Figure 41 The program memory controller (PMC) 4110 is shown, including the L1I cache 121 and error correction hardware. The L1I cache 121 receives data from the L2 SRAM / cache 130 via a 2x256-bit correction unit 4141 and a 16x32-bit parity generator 4142. When an instruction is provided to the CPU core 110, the parity bits stored in the L1I cache 121 are compared with the newly calculated parity bits from the 16x32-bit parity detector 4111. If they match, the instruction is provided to the CPU core 110 via a multiplexer 4112. If they do not match, the instruction is recalled from the L2 SRAM / cache 130, and the parity test is performed again. On a cache miss in the L1I cache 121 , the L2 SRAM / cache provides the requested instruction to the L1I cache 121 via a 2×256-bit correction unit 4141 and a 16×32-bit parity generator 4142 , and directly in parallel via a multiplexer 4122 .

[0296] Figure 41 The data memory controller (DMC) 4120 is shown, including the L1D cache 123 and error correction hardware. The L1D cache 123 receives data from the L2 SRAM / cache via a 2x256-bit correction unit 4121 and a 16x32-bit parity generator 4122. When data is provided to the CPU core 110, the parity bits stored in the L1D cache 123 are compared with the newly calculated parity bits from the 16x32-bit parity detector 4123. If they match, the data is provided to the CPU core 110 via a multiplexer 4124. If they do not match, the data is retrieved from the L2 SRAM / cache 130 and then parity tested again. On a cache miss in L1D cache 123 , L2 SRAM / cache provides the requested data to L1D cache 123 via 2×256-bit correction unit 4121 and 16×32-bit parity generator 4122 , and directly to central processing unit core 110 in parallel via multiplexer 4124 .

[0297] Writes from the central processing unit core 110 are subject to parity generation in the 16×32-bit syndrome generator 4125 . The data received from the central processing unit core 110 and the calculated parity bits are stored in the L1D cache 123 .

[0298] When writing back from the L1D cache 123, the newly calculated parity bits are compared with the stored parity in the 2x256-bit syndrome generator 4143. If these match, the data is stored in the L2 SRAM / cache 130. If they do not match, the 2x256-bit syndrome generator 4145 attempts correction. If correction is achieved, the corrected data is stored in the L2 SRAM / cache 130. If correction fails, a fault is generated.

[0299] Figure 41Stream buffer 125 is shown as including separate stream 0 2710 and stream 1 2720. Stream 0 2710 receives data from L2 SRAM / cache via a 2x256-bit correction unit 4151 and a 16x32-bit parity generator 4152. When data is provided to CPU core 110, the parity bits stored in stream 0 2710 are compared with the newly calculated parity bits in a 16x32-bit parity detector 4131. If they match, the data is provided to CPU core 110. If they do not match, a fault exists. Stream 1 2720 operates similarly, with a 2x256-bit correction unit 4153, a 16x32-bit parity generator 4154, and a 16x32-bit parity detector 4132.

[0300] Figure 41 The unified memory controller (UMC) 4140 is shown, including the L2 SRAM / cache 130 and error correction hardware. The L2 SRAM / cache 130 receives data from the multicore shared memory controller (MSMC) 4160 via a 2x256-bit syndrome generator 4144. New parity is generated for storage in the L2 SRAM / cache 130, and correction is attempted if necessary. In the event of a mismatch and correction failure, the data is retrieved from the MSMC 4160 and parity tested again. Writes from the L2 SRAM / cache 130 to the MSMC 4160 do not have parity checking or correction.

[0301] Periodically, 2x256-bit syndrome generation 4145 and 2x256 correction 4146 are performed on the data stored in L2 SRAM / cache 130. If necessary, the data and parity are recalled, new parity is generated, and a checksum correction is attempted. If the data is correct, it is not changed in L2 SRAM / cache 130. If the data is correct, the corrected data is stored back in L2 SRAM / cache 130. If the data correction fails, a fault is generated.

[0302] High-performance computing systems, such as the aforementioned streaming engines, require low-latency, high-bandwidth access to pools of random-access memory to properly utilize the available hardware. Memory performance scaling has not kept pace with advances in computer architecture and processes, resulting in memory bottlenecks for many applications. These state-of-the-art memories, particularly off-chip memories, require spatial and temporal locality to efficiently utilize data buses and access data arrays. In multicore systems, fast memory bandwidth resources are often scarce, and access must be shared among many computing elements. Arbitrators that provide this resource allocation typically attempt to provide a fair distribution of resources to similarly prioritized services. This fairness can actually hinder memory efficiency by undermining the spatial and temporal locality required for efficient memory utilization.

[0303] As an example of such a system, consider hosts A and B attempting to access memory Z. As greedy requesters, A and B will send as many requests as quickly as possible until they have reserved all available data buffer capacity. They will then stall until some of the previous requests return. In most systems with large off-chip memories, the memory return bandwidth is lower than the requester-issued capacity. Consequently, data requests return slower than they were initially sent. For the greedy hosts, as soon as one request comes back, a new one goes out. Once the data buffer is full, requests are issued and return at the speed of the memory return bandwidth, leaving time gaps between requests. This results in an initial stream of back-to-back requests until the data buffer is full, followed by a steady state of slow, one-by-one requests matching the memory return bandwidth.

[0304] The decrease in request rates from requesters A and B reduces the temporal and spatial locality of each request. This, in turn, reduces the efficiency of memory accesses. This reduced memory efficiency further reduces the request return rate, leading to larger time gaps between requests, further reducing locality. This feedback loop significantly degrades system performance, especially as more requesters enter the pool to access memory Z.

[0305] A greedy host will exploit a request algorithm based on the number of requests it needs to send (N), based on the host's program or functionality, and based on the number of requests the host is capable of sending (M), based on buffering, availability, or other constraints. This algorithm typically looks like this:

[0306] 1. If N == 0, no request is sent (not needed).

[0307] 2. (N>0) and (N≤M), send N requests.

[0308] 3. (N>0) and (N>M), send M requests.

[0309] As previously mentioned, these greedy hosts reduce the temporal and spatial locality of requests handled by memory. This typically results in more memory page changes and increased latency required to swap address translation entries. The present invention introduces a lag in the greedy host request rate. This lag adds the lag command number (R) to the calculation and looks like this:

[0310] 1. If N == 0, no request is sent (not needed).

[0311] 2. If (N>0) and (N≤M), send N requests.

[0312] 3. If (N > 0) and (N > M):

[0313] a. If (M < R), wait until more buffer is available;

[0314] b. If (M ≥ R), issue R requests.

[0315] In the second case (N ≤ M), there is more available buffer than requests to send. Therefore, all outstanding requests should be sent. In the third case (N > M), there are more requests than available buffer. In this case, the present invention uses hysteresis. This hysteresis does not send any requests until a group of R requests can be sent together. This ensures that whenever more than R requests are to be sent, the requesters group R requests together. Forcing clustering may increase the latency of any single command. However, this forced clustering ensures better temporal and spatial locality in memory to maximize efficiency. Since the entire system is bottlenecked on memory bandwidth, maximizing utilization results in a system with higher overall performance.

[0316] Figure 42 A multi-processor multi-memory system 4200 to which the fetch hysteresis of the present invention can be applied is shown. System 4200 includes multiple data processors. These include data processors 4201 and 4202, and each data processor includes a processor 100 as described above. Each of processors 4201 and 4202 includes a scalar data path 115, a vector data path 116, a level-1 instruction cache (L1I) 121, a level-1 data cache (L1D) 123, a stream engine 125, and a level-2 combined instruction / data cache (L2) 130. These elements are connected and operate as described above. As Figure 42 shown, system 4200 may include additional data processors of the same type. System 4200 includes multiple reduced instruction set computer (RISC) central processing units (CPUs) 4211 to 4213. These data processor types are merely used as examples of the present invention, and other larger or smaller numbers of data processor types are feasible.

[0317] The multi-processor multi-memory system 4200 includes multiple memories of multiple types. These include shared memory type A 4231 and 4232 and shared memory type B 4241 and 4342. System 4200 also includes a shared resource 4251. The shared resource 4251 may include an input device, an output device, or other types of devices shared among data processors 4201, 4202, 4211, and 4213. These memory types are merely examples, and other memories and more or fewer numbers of shared resource types are feasible.

[0318] The multi-core shared memory controller 4220 mediates data movement between data processors 4201, 4202, 4211, ... 4213 and resources 4231, 4232, 4241, 4342, ... 4251. This mediation includes prioritization and sorting, address translation, and matching of data flow types between the various processors and memories.

[0319] The hysteresis of this aspect of the invention assumes that access latency to at least some shared memory is enhanced when accesses are temporally and spatially localized rather than uncorrelated. For example, memory that supports page-mode access exhibits reduced latency when accesses to the same page are temporally adjacent. Memory accesses from a single source are more likely to exhibit this temporal and spatial locality than temporally adjacent accesses from different sources. Therefore, techniques that group accesses from a single source to feed a shared resource can reduce latency.

[0320] Figure 43 and Figure 44 It is shown that this hysteresis is applied to the stream engine of the present application. Figure 43 A partial schematic diagram of hardware implementing the invention is shown. Figure 44 The process of this aspect of the present invention is shown. This description assumes that bursty packets occur between the flow engine and the corresponding L2 combined cache. In this case, the L2 combined cache will generally maintain burstiness when seeking service from higher-level memory. Those skilled in the art will recognize that the hysteresis of the present invention can be implemented at other locations within the multiprocessor system. For example, the multi-core shared memory controller 4220 can provide hysteresis for memory requests from RISC CPUs 4211 and 4213.

[0321] Figure 43 The calculation of the number of pending requests N, the number of available buffers M and the hysteresis value R is shown. The number of pending requests N is calculated using a plurality of multipliers 4301, 4301, 4303 and 4304. The first multiplier 4301 receives as a first input the number ELEM_BYTES. This is the size of the data elements of the defined data stream. As described above, ELEM_BYTES is defined by the stream definition template. In a preferred embodiment, the stream definition template includes the field EFLAGS, a subset of which is the ELTYPE field. As shown in Table 8, the ELTYPE field encodes the number ELEM_BYTES. This encoding is merely an example and those skilled in the art will recognize other feasible techniques for setting the number ELEM_BYTES. The second input to the multiplier 4301 is from Figure 33 The multiplier 4302 counts the amount of the early count register 3311 shown in Loop0. Figure 331, which is not shown in the figure, but is similar to the product of Loop0 Count register 3311, Loop1 Count multiplied by multiplier 4301. Similarly, multiplier 4303 multiplies the product of Loop2 Count from early Count register 3331 by multiplier 4302, and multiplier 4304 multiplies the product of Loop3 Count from early Count register 3341 by multiplier 4303. Since early loop count registers 3311, 3321, 3331, and 3341 store the remaining number of their corresponding loops, this number is the number of data elements required to be retrieved for the specified data stream. Multiplying by ELEM_BYTES generates the remaining amount of data to be retrieved by the data stream. This number is designated as N.

[0322] Block 4311 generates the number of unallocated cache lines within the corresponding data memory 2716 / 2726. Multiplier 4312 multiplies the cache line size by this product. This cache line size must be expressed in the same units as ELEM_BYTES. The product output of multiplier 4312 is the number of available data buffers M. When a data stream is initially opened, no cache lines will be allocated, and the number of available data buffers M will be at its maximum value. The number of available data buffers M decreases when cache lines are allocated to new data and increases when cache lines are de-allocated as the CPU core 110 consumes data.

[0323] Register 4321 stores a hysteresis constant R. This is preferably a memory mapped register that is writable by the central processing unit core 110. The data in register 4321 is read for the retrieval hysteresis of the present invention.

[0324] The retrieval controller 4331 forms a retrieval determination from the N, M and R inputs. Figure 44 The retrieval determination is performed as shown. This retrieval determination is used in the test block 3401 of Figure 34 to determine whether a retrieval request from the stream engine has been issued. The retrieval controller 4331 counts the number of retrievals issued and controls the test block 3401 to retrieve the issuance accordingly. The retrieval controller 4330 can be implemented as a finite state machine or as a properly programmed logic controller in an asynchronous logic machine.

[0325] Figure 44 4400 is a flow chart illustrating a retrieval delay process. Test block 4401 determines whether at least one retrieval request is ready. In a stream engine embodiment, this determines whether an open stream has not yet retrieved and stored all data elements. A closed stream does not require retrieval. If all specified data elements have been retrieved and stored, no retrieval is required. If no request is ready (no at test block 4401), test block 4401 repeats until at least one request is ready.

[0326] If at least one request is ready (yes at test block 4401), block 4402 compares the number of ready requests, N, with the number of currently available data buffers, M. In a stream engine embodiment, the number of ready requests, R, is equal to the number of data elements in the corresponding data stream that have not yet been retrieved, multiplied by the size of the data elements. In a stream engine embodiment, the amount of currently available buffers, M, is equal to the number of unallocated cache lines, multiplied by the cache line size. If the number of ready requests, R, is less than or equal to the amount of currently available data buffers, M (yes at test block 4402), block 4403 dispatches all N ready requests.

[0327] If the number of ready requests N is greater than the number of data buffers currently available M (no at test block 4402), test block 4404 determines whether the amount of data buffers currently available M is less than the hysteresis R. If the amount of data buffers currently available M is less than the hysteresis R (yes at test block 4404), block 4405 enters a wait state for one memory cycle. The process then returns to test block 4404 to determine whether the amount of data buffers currently available M is less than the hysteresis R. In a stream engine embodiment, the amount of data buffers M depends on the number of unallocated cache lines. As described above, when the central processing unit 110 processes reads and advance instruction encodings, cache lines in the data memory 2716 / 2725 are deallocated. In other embodiments, the amount of data buffers will generally decrease as the retrieved data is consumed. Process 4400 remains in this loop until the amount of data buffers M is greater than or equal to the hysteresis R.

[0328] If the amount M of data buffer currently available is greater than or equal to the hysteresis R (no at test block 4404), then block 4406 dispatches R requests. As noted above in the stream engine embodiment, this allows process 3400 to override test block 3401 in retrieving data streams.

[0329] The hysteresis of this aspect of the invention requires that requests not be sent until a cluster of R requests can be sent together. This will group the R requests together as long as there are more than R requests ready. Forced clustering may increase the latency of any individual command. However, this forced clustering ensures better temporal and spatial locality in memory to maximize efficiency. The memory system accessed may include further address translations, or may include memory with reduced latency for back-to-back identical page accesses. Forced clustering can better utilize these memory resources. Forced clustering reduces the likelihood of needing to change table lookup entries in the address translation hardware because clustered requests are more likely to have spatial locality. This tends to avoid the need for address translation entry swaps. Since the latency of such address translation entry swaps is uncertain and can be long, the average latency is reduced. The higher likelihood of spatial locality of such clustered requests is more likely to allow back-to-back access to the same memory page. This also tends to reduce average latency.

Claims

1. A device comprising: Processing kernel; Memory; as well as a memory controller coupled to the processing core and the memory, wherein the memory controller comprises: a first address generator configured to generate a first set of addresses; a memory interface coupled to the memory and configured to retrieve a set of data elements based on the first set of addresses; a buffer configured to store the set of data elements; a holding register coupled to the processing core and configured to store a first data element of the set of data elements; as well as a second address generator configured to generate an address corresponding to a second data element in the set of data elements; The memory controller is configured as follows: receiving a read instruction from the processing core; and Based on the read instruction: providing the first data element to the processing core; causing the second address generator to generate the address; and The second data element is caused to be provided from the buffer to the holding register and replace the first data element.

2. The apparatus according to claim 1, wherein the first address generator comprises: Multiple loop count registers; a plurality of adders coupled to the plurality of loop count registers; a plurality of comparators coupled to the plurality of adders, and each comparator being configured to: comparing an output of a corresponding adder of the plurality of adders with a threshold value; and Based on the output of the corresponding adder of the plurality of adders being equal to the threshold, a loop end signal configured to cause an iteration of a next higher loop is provided.

3. The device according to claim 2, wherein: The first address generator includes a plurality of registers configured to store the threshold values ​​for the plurality of comparators; and The threshold is based on a flow definition template.

4. The apparatus according to claim 2, wherein the first address generator comprises: A plurality of multipliers are coupled to the plurality of adders, and each multiplier is configured to multiply an output of a corresponding adder of the plurality of adders by a data element size to determine an address in the first set of addresses.

5. The device according to claim 2, wherein: The processing core has a debugger function; and The values ​​of the plurality of loop count registers are accessible by the processing core via the debugger function.

6. The apparatus of claim 1 , further comprising an address register coupled to the first address generator and the memory interface and configured to store a first address in the first set of addresses, the first address corresponding to a next data element to be retrieved from the memory by the memory interface. 7 . The apparatus of claim 1 , wherein the memory comprises a level 2 cache (L2 cache), and the memory interface is configured to retrieve the set of data elements directly from the L2 cache.

8. The apparatus according to claim 1, comprising a first level cache (L1 cache) and a second level cache (L2 cache), wherein: The memory includes the L2 cache; and The memory interface is configured to retrieve the set of data elements from the L2 cache via a data path that does not include the L1 cache.

9. The apparatus of claim 1 , wherein the memory controller comprises a data formatting circuit coupled between the buffer and the holding register and configured to format a subset of the set of data elements including the second data element such that the formatted subset of the set of data elements is stored in the holding register.

10. The apparatus of claim 9, wherein the data formatting circuitry is configured to format the subset of the set of data elements according to a flow definition template.

11. The apparatus of claim 9 , further comprising a reference queue coupled to the data formatting circuit and configured to store, for each element in the set of data elements: The address at which the element is stored in the buffer; and A format for the element to be applied by the data formatting circuitry.

12. The apparatus of claim 1, wherein the buffer is a first-in-first-out buffer.

13. An apparatus comprising: Processing kernel; The first level cache memory is L1 cache memory; Second level cache memory is L2 cache memory; as well as a memory controller coupled to the processing core and the L2 cache memory, wherein the memory controller comprises: a first address generator configured to generate a first set of addresses; a memory interface coupled to the L2 cache memory and configured to retrieve a set of data elements based on the first set of addresses from the L2 cache memory via a data path that does not include the L1 cache memory; a buffer configured to store the set of data elements; a holding register coupled to the processing core and configured to store a first data element; and a second address generator configured to generate an address associated with a second data element; The memory controller is configured as follows: providing the first data element from the holding register to the processing core; and Based on the first data element being provided: causing the second address generator to generate the address; and Based on the address, the second data element is caused to be provided from the buffer to the holding register and replace the first data element.

14. The apparatus according to claim 13, wherein the first address generator comprises: Multiple loop count registers; a plurality of adders coupled to the plurality of loop count registers; a plurality of comparators coupled to the plurality of adders, and each comparator being configured to: comparing an output of a corresponding adder of the plurality of adders with a threshold value; and Based on the output of the corresponding adder of the plurality of adders being equal to the threshold, a loop end signal configured to cause an iteration of a next higher loop is provided.

15. The apparatus according to claim 14, wherein: The first address generator includes a plurality of registers configured to store the threshold values ​​for the plurality of comparators; and The threshold is based on a flow definition template.

16. The apparatus of claim 15, wherein the first address generator comprises: A plurality of multipliers are coupled to the plurality of adders, and each multiplier is configured to multiply an output of a corresponding adder of the plurality of adders by a data element size to determine an address in the first set of addresses.

17. The apparatus of claim 14, wherein: The processing core has a debugger function; and The values ​​of the plurality of loop count registers are accessible by the processing core via the debugger function.

18. The apparatus of claim 13 , wherein the memory controller comprises a data formatting circuit coupled between the buffer and the holding register and configured to format a subset of the set of data elements including the second data element such that the formatted subset of the set of data elements is stored in the holding register.

19. The apparatus of claim 18, wherein the data formatting circuitry is configured to format the subset of the set of data elements according to a flow definition template.

20. The apparatus of claim 18, further comprising a reference queue coupled to the data formatting circuit and configured to store, for each element in the set of data elements: The address at which the element is stored in the buffer; and A format for the element to be applied by the data formatting circuitry.

Images