Inserting predefined padding values into vector stream

Abstract

The invention relates to inserting predefined padding values into a vector stream. Software instructions are executed on a processor within a computer system to configure a streaming engine with flow parameters (4700) to define a multidimensional array. The stream parameters define a size and a fill value indicator for each dimension of the multi-dimensional array. Data is retrieved (4702) from a memory coupled to the streaming engine in response to the streaming parameter. A vector stream of the multi-dimensional array in response to the stream parameter is formed (4703) from the data retrieved from memory. A padded flow vector is formed (4706) that includes a specified padded value without accessing the padded value from system memory.

Description

[0001] Information related to divisional application

[0002] This invention is a divisional application of the invention patent application filed on May 25, 2020, with application number 202080037266.7 and title "Inserting Predefined Fill Values ​​into a Vector Stream". Technical Field

[0003] This invention relates to using a streaming engine to insert padding values ​​into a vector stream. Background Technology

[0004] Digital signal processors (DSPs) are optimized for processing data streams derived from various input signals, such as sensor data, video streams, voice channels, radar signals, and biomedical signals. DSPs operating on real-time data can receive input data streams, perform filtering functions (e.g., encoding or decoding) on ​​the data streams, and output transformed data streams. The system is called real-time because if the transformed data stream is unavailable for output during scheduling, the application fails. Some video coding requires predictable but non-sequential input data patterns. Such applications require memory access to load data registers into a data register file and then feed data from the data registers to the functional units that perform data processing.

[0005] One or more DSP processing cores can be combined with various peripheral circuits, memory blocks, etc., on a single integrated circuit (IC) die to form a system-on-a-chip (SoC). These systems can contain multiple interconnected processors sharing on-chip and off-chip memory. The processors can include a combination of an instruction cache (ICache) and a data cache (DCache) for improved processing. Furthermore, multiple processors with shared memory can be integrated into a single embedded system. Processors can physically share the same memory without accessing data or executing code located in the same memory location, or a portion of the shared memory can be used as common shared memory. Summary of the Invention

[0006] Methods and apparatus are provided to execute software instructions on a processor within a computer system to configure a streaming engine with streaming parameters to define a multidimensional array. The streaming parameters define the size of each dimension of the multidimensional array and specify padding value indicators. Data is retrieved from memory coupled to the streaming engine in response to the streaming parameters. A vector stream of the multidimensional array in response to the streaming parameters is formed from the data retrieved from memory. A padded stream vector containing specified padding values ​​is formed without accessing the padding values ​​from system memory. Attached Figure Description

[0007] Figure 1This section describes an example of a dual scalar / vector data path processor.

[0008] Figure 2 illustrate Figure 1 The registers and functional units in the dual scalar / vector data path processor are described in the document.

[0009] Figure 3 Explain the global scalar register file.

[0010] Figure 4 This describes the local scalar register file shared by the arithmetic function unit.

[0011] Figure 5 This describes the local scalar register file shared by the multiplication function unit.

[0012] Figure 6 This describes a local scalar register file shared by the load / store unit.

[0013] Figure 7 Explain the global vector register file.

[0014] Figure 8 Explain the assertion register file.

[0015] Figure 9 This describes the local vector register file shared by the arithmetic function unit.

[0016] Figure 10 This describes the local vector register file shared by the multiplication and related functional units.

[0017] Figure 11 Describe the pipeline stages of the processing unit.

[0018] Figure 12 This describes the 16 instructions for acquiring a single packet.

[0019] Figure 13 This describes an example of instruction encoding for an instruction.

[0020] Figure 14 Explain the bit encoding of condition code extension slot 0.

[0021] Figure 15 Explain the bit encoding of condition code extension slot 1.

[0022] Figure 16 Explain the bit encoding of constant extension slot 0.

[0023] Figure 17 This is a partial block diagram illustrating the extension of the constant.

[0024] Figure 18 Explain carry control in SIMD operations.

[0025] Figure 19 A conceptual diagram illustrating a streaming engine.

[0026] Figure 20 This describes the formatting operation sequence.

[0027] Figure 21 Illustrate an example of channel allocation in a vector.

[0028] Figure 22 Illustrate an example of channel allocation in a vector.

[0029] Figure 23 Explain the basic two-dimensional (2D) flow.

[0030] Figure 24 illustrate Figure 23 The order of elements within the instance stream.

[0031] Figure 25 This demonstrates how to extract a smaller rectangle from a larger rectangle.

[0032] Figure 26 This demonstrates how the instance streaming engine acquires streams with a 4-byte transpose granularity.

[0033] Figure 27 This demonstrates how the instance streaming engine acquires streams with an 8-byte transpose granularity.

[0034] Figure 28 This section describes the details of the instance streaming engine.

[0035] Figure 29 Explain the instance stream template register.

[0036] Figure 30 illustrate Figure 29 The subfield definition of the flag field of the instance stream template register.

[0037] Figure 31 This describes an instance of vector length masking / group copying blocks.

[0038] Figure 32 This is a partial diagram illustrating an instance of a streaming engine valid or invalid indication being generated.

[0039] Figure 33 This is a partial schematic diagram illustrating the streaming engine address generator that generates loop addresses and loop counts.

[0040] Figure 34 This illustration shows a partial diagram of the streaming engine supplying the data for this example.

[0041] Figure 35 The illustration shows a partial diagram of the streaming engine supplying valid data to the assertion unit.

[0042] Figure 36 It includes matrix multiplication accelerators and Figure 28 A block diagram of the streaming engine system.

[0043] Figure 37 Illustrate an example of matrix multiplication.

[0044] Figure 38 yes Figure 28 A more detailed block diagram of a part of the streaming engine.

[0045] Figure 39 , 40 Explanation of points 41, 42, and 43: Figure 28 Instances of the streaming engine are linear streams.

[0046] Figure 44A , 44B Together, we explain how the submatrix is ​​passed through Figure 28 The streaming engine is used to amplify empty vectors for matrix multiplication.

[0047] Figure 45 This indicates that an empty vector will be added to the stream.

[0048] Figures 46 to 47 Explanation passed Figure 28 The streaming engine inserts zeros or predefined data vectors to form a stream.

[0049] Figure 48 It includes Figure 28 A block diagram of a multiprocessor system for a streaming engine. Detailed Implementation

[0050] In the diagram, to maintain consistency, identical components are indicated by the same component symbol.

[0051] Digital signal processors (DSPs) are optimized for processing data streams derived from various input signals, such as sensor data, video streams, voice channels, radar signals, and biomedical signals. DSPs are concerned with memory bandwidth and scheduling when operating on real-time data. Example DSPs include streaming engines to improve memory bandwidth and data scheduling.

[0052] One or more DSPs can be combined with various peripheral circuits, memory blocks, etc., on a single integrated circuit (IC) die to form a system-on-a-chip (SoC). See, for example, "66AK2Hx MulticoreKeystone" incorporated herein by reference. TM DSP+ARM ® System-on-a-Chip (66AK2Hx Multicore Keystone) TM DSP+ARM ®System-on-Chip (2013).

[0053] In some instance processors, the autonomous streaming engine (SE) is coupled to the DSP. In this instance, the streaming engine comprises two tightly coupled streaming engines capable of managing two data streams simultaneously. In another instance, the streaming engine can manage only a single stream, while in still others, it can handle more than two streams. In each case, for each stream, the streaming engine includes an address generation phase, a data formatting phase, and a storage for the formatted data awaiting processor consumption. In the instance described herein, the address is derived from an algorithm that may involve multidimensional loops, with each dimension maintaining an iteration count. In one instance, the streaming engine supports six levels of nested iterations. In other instances, more or fewer levels of iterations are supported.

[0054] In some instance processors, the control logic and alignment network enable the streaming engine to form empty or padded vectors and insert them into the stream without accessing empty data or padded values ​​from system memory.

[0055] Several examples of forming empty vectors and filled vectors. Figures 36 to 48 More detailed description.

[0056] refer to Figures 1 to 18 Describe an example DSP processor. A reference to an example streaming engine capable of managing two data streams using a six-dimensional nested loop. Figures 19 to 35 describe.

[0057] Figure 1 This describes an instance processor 100 containing dual scalar / vector data paths 115 and 117. As used herein, the term "vector" refers to an array of one-dimensional data elements that can be accessed and operated on as a unit. Processor 100 includes a streaming engine 125, which is described in more detail herein. Processor 100 includes a separate Level 1 Instruction Cache (L1I) 121 and a Level 1 Data Cache (L1D) 123. Processor 100 includes a Level 2 (L2) combined instruction / data cache 130 that stores both instructions and data. Figure 1 Explain the connection between the L1I cache and the L2 combined instruction / data cache 130, and the 512-bit bus 142. Figure 1 This describes the connection between L1D cache 123 and L2 combined instruction / data cache 130, using a 512-bit bus 145. In the instance processor 100, L2 combined instruction / data cache 130 stores both instructions supporting L1I cache 121 and data supporting L1D cache 123. In this instance, L2 combined instruction / data cache 130 uses... Figure 1Known or later-developed memory system technologies not described herein are further connected to higher-level caches and / or main memory. As used herein, the term "higher-level" memory or cache refers to the next level in the memory hierarchy further away from the processor, while the term "lower-level" memory or cache refers to the level in the memory hierarchy closer to the processor. In various instances, L1I cache 121, L1D cache 123, and L2 cache 130 may be implemented in different sizes. In this instance, L1I cache 121 and L1D cache 123 are each 32KB, and L2 cache 130 is 1024KB. In instance processor 100, L1I cache 121, L1D cache 123, and L2 combined instruction / data cache 130 are formed on a single integrated circuit. This single integrated circuit optionally includes other circuitry.

[0058] Processing unit core 110 fetches instructions from L1I cache 121, as controlled by instruction fetch unit 111. Instruction fetch unit 111 determines the next instruction to be executed and invokes a set of such instructions of fetch packet size. The nature and size of the fetch packet are further detailed below. If the instruction is stored in L1I cache 121, the instruction is fetched directly from L1I cache 121 after a cache hit. After a cache miss occurs, when the specified instruction is not stored in L1I cache 121, the instruction is searched in L2 combined cache 130. In this example, the size of the cache line in L1I cache 121 is equal to the size of the fetch packet, which is 512 bits. The memory location of these instructions is either a hit or a miss in L2 combined cache 130. A hit is serviced by L2 combined cache 130. A miss is serviced by a higher-level cache (not specified) or by main memory (not specified). In this example, the requested instructions are supplied to both the L1I cache 121 and the processing unit core 110 to accelerate their use.

[0059] In this example, the processing unit core 110 contains multiple functional units to execute instruction-specified data processing tasks. The instruction dispatch unit 112 determines the target functional unit for each fetched instruction. In this example, the processing unit 110 operates as a Very Long Instruction Word (VLIW) processor capable of simultaneously operating on multiple instructions within a corresponding functional unit. The compiler organizes the instructions into an execution packet for execution. The instruction dispatch unit 112 directs each instruction to its target functional unit. The functional unit assigned to an instruction is entirely specified by the instructions generated by the compiler. The hardware of the processing unit core 110 does not participate in functional unit assignment. In this example, the instruction dispatch unit 112 operates on several instructions in parallel. The number of such parallel instructions is set by the size of the execution packet, as described further herein.

[0060] Part of the dispatching task of instruction dispatch unit 112 is to determine whether an instruction is executed on a functional unit on the scalar data path side A 115 or the vector data path side B 116. An instruction bit, referred to as the s-bit, within each instruction determines which data path the instruction controls. This is described further herein.

[0061] Instruction decoding unit 113 decodes each instruction in the current execution packet. Decoding includes identifying the functional unit executing the instruction, identifying the registers used to supply data for the corresponding data processing operation from a possible register file, and identifying the register destination of the result of the corresponding data processing operation. As further described below, an instruction may contain a constant field instead of a register number operand field. The result of this decoding is a signal used to control the target functional unit to perform the data processing operation specified by the corresponding instruction on the specified data.

[0062] The processing unit core 110 includes a control register 114. The control register 114 stores information for controlling the functional units in the scalar data path side A 115 and the vector data path side B 116. This information may include mode information or the like.

[0063] The decoded instruction from instruction decoder 113 and the information stored in control register 114 are supplied to scalar data path side A 115 and vector data path side B 116. Therefore, the functional units within scalar data path side A 115 and vector data path side B 116 perform instruction-specified data processing operations on the instruction-specified data and store the results in one or more instruction-specified data registers. Each of scalar data path side A 115 and vector data path side B 116 contains multiple functional units operating in parallel. These are described below in conjunction with... Figure 2 Further description. A data path 117 exists between scalar data path side A 115 and vector data path side B 116, thereby allowing data exchange.

[0064] Processing unit core 110 includes additional non-instruction-based modules. Simulation unit 118 allows the determination of the machine state of processing unit core 110 in response to instructions. This capability can be used for algorithm development. Interrupt / Exception unit 119 enables processing unit core 110 to respond to external asynchronous events (interrupts) and attempts to perform improper operations (exceptions).

[0065] Processor 100 includes a streaming engine 125. Streaming engine 125 supplies two data streams from predetermined addresses cached in L2 combined cache 130 to the register file of the vector data path side B of processing unit core 110. This provides controlled data movement directly from memory (such as cached in L2 combined cache 130) to function unit operand inputs. This is further described herein.

[0066] Figure 1 This describes the example data width of the buses between the various components. L1I cache 121 supplies instructions to instruction fetch unit 111 via bus 141. In this example, bus 141 is a 512-bit bus. Bus 141 is unidirectional from L1I cache 121 to processing unit 110. L2 combined cache 130 supplies instructions to L1I cache 121 via bus 142. In this example, bus 142 is a 512-bit bus. Bus 142 is unidirectional from L2 combined cache 130 to L1I cache 121.

[0067] L1D cache 123 exchanges data with the register file in scalar data path A 115 via bus 143. In this example, bus 143 is a 64-bit bus. L1D cache 123 exchanges data with the register file in vector data path B 116 via bus 144. In this example, bus 144 is a 512-bit bus. Buses 143 and 144 are described as bidirectional, thereby supporting both data reading and data writing to the processing unit core 110. L1D cache 123 exchanges data with L2 combined cache 130 via bus 145. In this example, bus 145 is a 512-bit bus. Bus 145 is described as bidirectional, thereby supporting cache services for both data reading and data writing to the processing unit core 110.

[0068] After a cache hit (if the requested data is stored in L1D cache 123), the processor data request is fetched directly from L1D cache 123. After a cache miss (the specified data is not stored in L1D cache 123), the data is searched in L2 combined cache 130. The memory location of the requested data is either a hit or a miss in L2 combined cache 130. A hit is served by L2 combined cache 130. A miss is served by another cache level (not specified) or by main memory (not specified). The requested data can be supplied to both L1D cache 123 and processing unit core 110 simultaneously for accelerated use.

[0069] L2 combinational cache 130 supplies data of the first data stream to streaming engine 125 via bus 146. In this example, bus 146 is a 512-bit bus. Streaming engine 125 supplies data of the first data stream to the functional unit of vector data path side B 116 via bus 147. In this example, bus 147 is a 512-bit bus. L2 combinational cache 130 supplies data of the second data stream to streaming engine 125 via bus 148. In this example, bus 148 is a 512-bit bus. Streaming engine 125 supplies this second data stream to the functional unit of vector data path side B 116 via bus 149. In this example, bus 149 is a 512-bit bus. In this example, buses 146, 147, 148, and 149 are described as unidirectional from L2 combinational cache 130 to streaming engine 125 and to vector data path side B 116.

[0070] After a cache hit (if the requested data is stored in L2 combined cache 130), the streaming engine data request is retrieved directly from L2 combined cache 130. After a cache miss (the specified data is not stored in L2 combined cache 130), data is searched from another level cache (not specified) or from main memory (not specified). In some instances, it is technically feasible for L1D cache 123 to cache data not stored in L2 combined cache 130. If this is supported, then after a streaming engine data request that is missing from L2 combined cache 130, L2 combined cache 130 probes for the streaming engine request data from L1D cache 123. If L1D cache 123 stores the data, then the probe response contains the data subsequently supplied to service the streaming engine request. If L1D cache 123 does not store the data, then the probe response indicates that L2 combined cache 130 is serving the streaming engine request from another level cache (not specified) or from main memory (not specified).

[0071] In this example, both L1D cache 123 and L2 combined cache 130 can be configured as selected amounts of cache or direct addressable memory as described in U.S. Patent No. 6,606,686 entitled "Unified Memory System Architecture Including Cached and Directly Addressable Static Random Access Memory," which is incorporated herein by reference.

[0072] In this example, processor 100 is fabricated on an integrated chip (IC) mounted on a ball grid array (BGA) substrate. The BGA substrate and the IC die together may be referred to as a "BGA package," "IC package," "integrated circuit," "IC," "chip," "microelectronic device," or similar terms. The BGA package may contain an encapsulating material that covers and protects the IC die from damage. In another example, other types of known or later-developed packaging technologies may be used with processor 100.

[0073] Figure 2 Further details regarding the functional units and register files within scalar data path A 115 and vector data path B 116 are provided. Scalar data path A 115 includes L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 226. Scalar data path A 115 also includes a global scalar register file 211, L1 / S1 local register file 212, M1 / N1 local register file 213, and D1 / D2 local register file 214. Vector data path B 116 includes L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245, and P unit 246. Vector data path B 116 includes a global vector register file 231, L2 / S2 local register file 232, M2 / N2 / C local register file 233, and assertion register file 234. Which functional units can read from or write to which register files is described in more detail in this article.

[0074] The scalar data path A 115 contains L1 unit 221. L1 unit 221 typically accepts two 64-bit operands and produces a 64-bit result. Each operand is retrieved from the instruction-specified register in either the global scalar register file 211 or the L1 / S1 local register file 212. L1 unit 221 performs the following instruction-selected operations: 64-bit addition / subtraction; 32-bit minimum / maximum operations; 8-bit Single Instruction Multiple Data (SIMD) instructions, such as summation of absolute values, minimum and maximum value determination; looping minimum / maximum operations; and various move operations between register files. The result is written to the instruction-specified register in the global scalar register file 211, the L1 / S1 local register file 212, the M1 / N1 local register file 213, or the D1 / D2 local register file 214.

[0075] The scalar data path side A 115 contains S1 unit 222. S1 unit 222 typically accepts two 64-bit operands and produces a 64-bit result. Each operand is called from an instruction-specified register in either the global scalar register file 211 or the L1 / S1 local register file 212. In this example, S1 unit 222 performs the same type of operation as L1 unit 221. In another example, there may be slight variations between the data processing operations supported by L1 unit 221 and S1 unit 222. The result is written to an instruction-specified register in either the global scalar register file 211, the L1 / S1 local register file 212, the M1 / N1 local register file 213, or the D1 / D2 local register file 214.

[0076] The scalar data path A 115 contains M1 unit 223. M1 unit 223 typically accepts two 64-bit operands and produces a 64-bit result. Each operand is called from an instruction-specified register in either the global scalar register file 211 or the M1 / N1 local register file 213. In this example, M1 unit 223 performs the following instruction-selected operations: 8-bit multiplication; complex dot product; 32-bit bit counting; complex conjugate multiplication; and bitwise logical operations, shift operations, addition, and subtraction. The result is written to an instruction-specified register in either the global scalar register file 211, the L1 / S1 local register file 212, the M1 / N1 local register file 213, or the D1 / D2 local register file 214.

[0077] On the scalar data path side A 115, there is an N1 unit 224. N1 unit 224 typically accepts two 64-bit operands and produces a 64-bit result. Each operand is called from an instruction-specified register in either the global scalar register file 211 or the M1 / N1 local register file 213. In this example, N1 unit 224 performs the same type of operation as M1 unit 223. There are also dual operations (called double-issue instructions) that utilize both M1 unit 223 and N1 unit 224 simultaneously. The result is written to an instruction-specified register in either the global scalar register file 211, the L1 / S1 local register file 212, the M1 / N1 local register file 213, or the D1 / D2 local register file 214.

[0078] The scalar data path A 115 contains D1 unit 225 and D2 unit 226. D1 unit 225 and D2 unit 226 typically each accept two 64-bit operands and each produce a 64-bit result. D1 unit 225 and D2 unit 226 typically perform address calculations and corresponding load and store operations. D1 unit 225 is used for 64-bit scalar load and store. D2 unit 226 is used for 512-bit vector load and store. In this example, D1 unit 225 and D2 unit 226 also perform: swapping, packing and unpacking payloads and storing data; 64-bit SIMD arithmetic operations; and 64-bit bitwise logical operations. The D1 / D2 local register file 214 stores the base address and offset address used in the address calculations for the corresponding load and store operations. The two operands are each called from the instruction-specified register in the global scalar register file 211 or the D1 / D2 local register file 214. The calculated results are written to the instruction-specified registers in the global scalar register file 211, the L1 / S1 local register file 212, the M1 / N1 local register file 213, or the D1 / D2 local register file 214.

[0079] The vector data path side B 116 contains L2 unit 241. L2 unit 241 typically accepts two 512-bit operands and produces a single 512-bit result. Each operand is called from an instruction-specified register in the global vector register file 231, the L2 / S2 local register file 232, or the assertion register file 234. In this example, except for wider 512-bit data, L2 unit 241 executes instructions similar to those in L1 unit 221. The result can be written to an instruction-specified register in the global vector register file 231, the L2 / S2 local register file 232, the M2 / N2 / C local register file 233, or the assertion register file 234.

[0080] The vector data path side B 116 contains S2 unit 242. S2 unit 242 typically accepts two 512-bit operands and produces a single 512-bit result. Each operand is retrieved from an instruction-specified register in the global vector register file 231, the L2 / S2 local register file 232, or the assertion register file 234. In this example, S2 unit 242 executes instructions similar to those in S1 unit 222. The result is written to an instruction-specified register in the global vector register file 231, the L2 / S2 local register file 232, the M2 / N2 / C local register file 233, or the assertion register file 234.

[0081] The vector data path side B 116 contains M2 unit 243. M2 unit 243 typically accepts two 512-bit operands and produces a single 512-bit result. Each operand is called from an instruction-specified register in either the global vector register file 231 or the M2 / N2 / C local register file 233. In this example, except for wider 512-bit data, M2 unit 243 executes instructions similar to those in M1 unit 223. The result is written to the instruction-specified register in either the global vector register file 231, the L2 / S2 local register file 232, or the M2 / N2 / C local register file 233.

[0082] The vector data path B 116 contains N2 unit 244. N2 unit 244 typically accepts two 512-bit operands and produces a single 512-bit result. Each operand is called from the instruction specifying register in either the global vector register file 231 or the M2 / N2 / C local register file 233. In this example, N2 unit 244 performs the same type of operation as M2 unit 243. There are also dual operations (called double-issued instructions) that utilize both M2 unit 243 and N2 unit 244 simultaneously. The result is written to the instruction specifying register in either the global vector register file 231, the L2 / S2 local register file 232, or the M2 / N2 / C local register file 233.

[0083] The vector data path side B 116 contains a related (C) unit 245. C unit 245 typically accepts two 512-bit operands and produces a 512-bit result. Each operand is called from an instruction-specified register in either the global vector register file 231 or the M2 / N2 / C local register file 233. In this example, C unit 245 executes "rake" and "search" instructions for WCDMA (Wideband Code Division Multiple Access) encoding / decoding. In this example, C unit 245 can execute up to 512 cycles per clock cycle for 2-bit PN (pseudo-random number) and 8-bit I / Q (complex number), 8-bit and 16-bit absolute difference sum (SAD) calculations, up to 512 SAD calculations per clock cycle, lateral addition, lateral minimum / maximum instructions, and vector permutation instructions. C unit 245 also contains four vector control registers (CUCR0 to CUCR3) for controlling certain operations of C unit 245 instructions. Control registers CUCR0 to CUCR3 are used as operands in some C unit 245 operations. In some instances, control registers CUCR0 through CUCR3 are used to control the general permutation instruction (VPERM) and serve as masks for SIMD multiple dot product operations (DOTPM) and SIMD multiple absolute difference sums (SAD) operations. In other instances, control register CUCR0 is used to store polynomials for Galois field multiplication operations (GFMPY), and control register CUCR1 is used to store Galois field polynomial generator functions.

[0084] The vector data path side B 116 contains P unit 246. Vector assertion (P) unit 246 performs basic logical operations on registers in the local assertion register file 234. P unit 246 can directly read from and write to the assertion register file 234. Logical operations include single-register unary operations, such as: NEG (invert), which inverts each bit of a single register; BITCNT (bit count), which returns the count of the number of bits in a single register with a predetermined numeric state (1 or 0); RMBD (rightmost bit check), which returns the number of bit positions from the least significant bit position (rightmost) to the first bit position with a predetermined numeric state (1 or 0); DECIMATE, which selects the Nth (1, 2, 4, etc.) bit for each instruction to be output; and EXPAND, which copies each bit of the instruction N (2, 4, etc.) times. Logical operations also include: two-register binary operations, such as: AND, which is a bitwise AND of the data in two registers; NAND, which is a bitwise AND and inversion of the data in two registers; OR, which is a bitwise OR of the data in two registers; NOR, which is a bitwise OR and inversion of the data in two registers; and XOR, which is a mutually exclusive OR of the data in two registers. Logical operations include transferring data from the assertion registers in assertion register file 234 to another specified assertion register or a specified data register in global vector register file 231. One use of unit P 246 is to manipulate the results of SIMD vector comparisons to control another SIMD vector operation. The BITCNT instruction can be used to count the number of 1s in the assertion registers to determine the number of valid data elements from the assertion registers.

[0085] Figure 3 Description of Global Scalar Register File 211. There are 16 independent 64-bit wide scalar registers, labeled A0 through A15. Each register in Global Scalar Register File 211 can be read or written as 64-bit scalar data. All scalar data path side A115 functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 226) can be read or written to Global Scalar Register File 211. Global Scalar Register File 211 can be read as 32 bits or 64 bits and written as 64 bits. Instruction execution determines the size of the data to be read. Vector data path side B116 functional units (L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245, and P unit 246) can be read from Global Scalar Register File 211 via cross path 117, subject to the constraints described below.

[0086] Figure 4Description of D1 / D2 local register file 214. There are 16 independent 64-bit wide scalar registers labeled D0 to D15. Each register in D1 / D2 local register file 214 is read or written as 64-bit scalar data. All scalar data path-side A115 functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 226) can be written to global scalar register file 211. Only D1 unit 225 and D2 unit 226 can be read from D1 / D2 local scalar register file 214. Data stored in D1 / D2 local scalar register file 214 may contain base and offset addresses used in address calculations.

[0087] Figure 5 The L1 / S1 local register file 212 is described. In this example, the L1 / S1 local register file 212 contains eight independent 64-bit wide scalar registers labeled AL0 to AL7. In this example, the instruction encoding allows the L1 / S1 local register file 212 to contain up to 16 registers. In this example, eight registers are implemented to reduce circuit size and complexity. Each register in the L1 / S1 local register file 212 can be read or written as 64-bit scalar data. All scalar data path-side A115 functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 226) can be written to the L1 / S1 local scalar register file 212. L1 unit 221 and S1 unit 222 can be read from the L1 / S1 local scalar register file 212.

[0088] Figure 6 The M1 / N1 local register file 213 is described. In this example, eight independent 64-bit wide scalar registers, labeled AM0 to AM7, are implemented. In this example, the instruction encoding allows the M1 / N1 local register file 213 to contain up to 16 registers. In this example, eight registers are implemented to reduce circuit size and complexity. Each register in the M1 / N1 local register file 213 can be read or written as 64-bit scalar data. All scalar data path-side A115 functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 226) can be written to the M1 / N1 local scalar register file 213. M1 unit 223 and N1 unit 224 can be read from the M1 / N1 local scalar register file 213.

[0089] Figure 7Description of Global Vector Register File 231. There are 16 independent 512-bit wide vector registers. Each register in Global Vector Register File 231 can be read or written as 64-bit scalar data labeled B0 to B15. Each register in Global Vector Register File 231 can be read or written as 512-bit vector data labeled VB0 to VB15. The instruction type determines the data size. All vector data path side B 116 function units (L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245, and P unit 246) can be read or written to Global Vector Register File 231. Scalar data path side A 115 function units (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 226) can be read from Global Vector Register File 231 via cross path 117 under the constraints described below.

[0090] Figure 8 The assertion (P) local register file 234 is described. There are eight independent 64-bit wide registers, labeled P0 through P7. Each register in the P local register file 234 can be read from or written as 64-bit scalar data. Functional units B116 on the vector data path side (L2 unit 241, S2 unit 242, C unit 244, and P unit 246) can be written to the P local register file 234. L2 unit 241, S2 unit 242, and P unit 246 can be read from the P local scalar register file 234. One use of the P local register file 234 is to: write a 1-bit SIMD vector comparison result from L2 unit 241, S2 unit 242, or C unit 244; manipulate the SIMD vector comparison result via P unit 246; and use the manipulated result to control another SIMD vector operation.

[0091] Figure 9Description of L2 / S2 local register file 232. In this example, eight independent 512-bit wide vector registers are implemented. In this example, the instruction encoding allows L2 / S2 local register file 232 to contain up to 16 registers. In this example, eight registers are implemented to reduce circuit size and complexity. Each register in L2 / S2 local vector register file 232 can be read or written as 64-bit scalar data labeled BL0 to BL7. Each register in L2 / S2 local vector register file 232 can be read or written as 512-bit vector data labeled VBL0 to VBL7. The instruction type determines the data size. All vector data path-side B 116 functional units (L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245, and P unit 246) can be written to L2 / S2 local vector register file 232. L2 unit 241 and S2 unit 242 can be read from L2 / S2 local vector register file 232.

[0092] Figure 10 Description of M2 / N2 / C local register file 233. In this example, eight independent 512-bit wide vector registers are implemented. In this example, the instruction encoding allows M2 / N2 / C local register file 233 to contain up to 16 registers. In this example, eight registers are implemented to reduce circuit size and complexity. Each register in M2 / N2 / C local vector register file 233 can be read or written as 64-bit scalar data labeled BM0 to BM7. Each register in M2 / N2 / C local vector register file 233 can be read or written as 512-bit vector data labeled VBM0 to VBM7. All vector data path-side B 116 functional units (L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245, and P unit 246) can be written to M2 / N2 / C local vector register file 233. Units M2 233, N2 244, and C 245 can be read from the M2 / N2 / C local vector register file 233.

[0093] Providing a global register file accessible to all functional units on one side and a local register file accessible to some functional units on one side is a design choice. In another instance, different accessibility can be provided, for example, by using a register file of a type corresponding to the global register file described herein.

[0094] Cross path 117 permits limited data exchange between scalar data path side A 115 and vector data path side B 116. During each operation cycle, a 64-bit data word can be called from global scalar register file A 211 to be used as an operand by one or more functional units on vector data path side B 116, and a 64-bit data word can be called from global vector register file 231 to be used as an operand by one or more functional units on scalar data path side A 115. Any functional unit on scalar data path side A 115 (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 226) can read a 64-bit operand from global vector register file 231. This 64-bit operand is the least significant bit of the 512-bit data in the accessed registers of global vector register file 231. Multiple scalar data path side A 115 functional units can use the same 64-bit interleaved path data as the operands during the same operation cycle. However, within a single operation cycle, a single 64-bit operand is transferred from the vector data path side B 116 to the scalar data path side A 115. Any vector data path side B 116 functional unit (L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245, and P unit 246) can read the 64-bit operand from the global scalar register file 211. If the corresponding instruction is a scalar instruction, the interleaved path operand data is treated as a 64-bit operand. If the corresponding instruction is a vector instruction, the 448 high-order bits of the operand are padded with zeros. Multiple vector data path side B 116 functional units can use the same 64-bit interleaved path data as the operands during the same operation cycle. In one instance, within a single operation loop, a single 64-bit operand is transferred from the scalar data path side A 115 to the vector data path side B 116.

[0095] Streaming Engine 125 ( Figure 1Data is transmitted under certain constraints. Streaming engine 125 controls two data streams. Each stream contains a sequence of elements of a specific type. A program operating on the stream sequentially reads the data, operating on each element in turn. Each stream has the following basic properties: the stream data has a well-defined start and end in time; the stream data has a fixed element size and type throughout the stream; and the stream data has a fixed element sequence. Once a stream is opened, streaming engine 125 performs the following operations: calculates the address; retrieves the defined data type from L2 unified cache 130 (which requires cache service from higher-level memory, e.g., in case of a cache miss in L2); performs data type manipulations, such as zero extension, sign extension, data element sorting / swapping, such as matrix transpose; and delivers the data directly to the programmed data register file within processor core 110. Therefore, streaming engine 125 is beneficial for real-time digital filtering of well-formed data. Streaming engine 125 frees the corresponding processor from these memory fetching tasks, enabling other processing functions.

[0096] The streaming engine 125 provides several benefits. For example, the streaming engine 125 allows multidimensional memory access, increases the bandwidth available for functional units, minimizes the number of cache miss stops (because the stream buffer bypasses the L1D cache 123), and reduces the number of scalar operands required to maintain loops. The streaming engine 125 also manages address pointers and disposes of address generation, releasing address generation instruction slots and D1 unit 225 and D2 unit 226 for other calculations.

[0097] Processor Core 110 ( Figure 1 Operations occur on the instruction pipeline. Instructions are fetched from fixed-length instruction packets, as described further below. All instructions require the same number of pipeline stages for fetching and decoding, but a different number of execution stages.

[0098] Figure 11 The following pipeline phases are described: program fetch phase 1110, dispatch and decode phase 1120, and execution phase 1130. Program fetch phase 1110 contains three phases for all instructions. Dispatch and decode phase 1120 contains three phases for all instructions. Execution phase 1130 contains one to four phases depending on the instruction.

[0099] The fetch phase 1110 includes a program address generation (PG) phase 1111, a program access (PA) phase 1112, and a program reception (PR) phase 1113. During the program address generation phase 1111, a program address is generated in the processor, and a read request is sent to the memory controller of the L1I cache. During the program access phase 1112, the L1I cache processes the request, accesses the data in its memory, and sends the fetch packet to the processor boundary. During the program reception phase 1113, the processor registers the fetch packet.

[0100] The instruction is retrieved in a retrieval packet containing 16 32-bit wide words. Figure 12 This describes 16 instructions, 1201 to 1216, in a single fetch packet. The fetch packet is aligned on 512-bit (16-word) boundaries. This example uses a fixed 32-bit instruction length to implement decoder alignment. Properly aligned instruction fetches can load multiple instructions into a parallel instruction decoder. This properly aligned instruction fetch can be achieved through predetermined instruction alignment by storing fetch packets aligned on 512-bit boundaries in memory with fixed instruction packet fetches. In contrast, variable-length instructions require an initial step of locating each instruction boundary before decoding. Fixed-length instruction settings generally allow for a more regular layout of instruction fields to simplify the construction of each decoder, which is advantageous for wide-issue VLIW processors.

[0101] The execution portion of an individual instruction is controlled by a p-bit within each instruction. In this example, the p-bit is bit 0 of a 32-bit slot. The p-bit determines whether the instruction is executed in parallel with the next instruction. In this example, instructions are scanned from lower addresses to higher addresses. If the p-bit of an instruction is 1, then the next instruction (at a higher memory address) is executed in parallel with the instruction (within the same loop as the instruction). If the p-bit of an instruction is 0, then the next instruction is executed within the loop following the instruction.

[0102] Processor Core 110 ( Figure 1 ) and L1I cache 121 pipeline ( Figure 1 They are decoupled from each other. The return of a fetched packet from the L1I cache may take a different number of clock cycles depending on the external environment (e.g., whether there is a hit in the L1I cache 121 or the L2 combined cache 130). Therefore, the program access phase 1112 may take several clock cycles instead of a single clock cycle as other phases do.

[0103] Instructions are executed in parallel to form execution packets. In this example, an execution packet may contain up to 16 32-bit wide slots for 16 instructions. No two instructions in an execution packet can use the same functional unit. Slots are one of the following five types: 1) self-contained instructions that execute on one of the functional units of processor core 110 (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, D2 unit 226, L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245, and P unit 246); 2) unitless instructions, such as a NOP (no operation) instruction or multiple NOP instructions; 3) branch instructions; 4) constant field extensions; 5) condition code extensions. Some of these slot types are further described herein.

[0104] Distribution and decoding phase 1120 ( Figure 11 The process includes an instruction dispatch to the appropriate execution unit (DS) phase 1121, an instruction pre-decoding (DC1) phase 1122, and an instruction decoding and operand fetch (DC2) phase 1123. During the instruction dispatch to the appropriate execution unit phase 1121, the fetched packet is divided into execution packets and assigned to the appropriate functional unit. During the instruction pre-decoding phase 1122, the source register, destination register, and associated paths are decoded for instruction execution within the functional unit. During the instruction decoding and operand fetch phase 1123, more detailed unit decoding is performed, and operands are read from the register file.

[0105] Execution phase 1130 comprises execution (E1 to E5) phases 1131 to 1135. Different types of instructions require different numbers of such phases to complete execution. The execution phases of the pipeline play a crucial role in understanding the device state at processor loop boundaries.

[0106] During phase E1 (1131), the conditions of the instruction are evaluated and the operands are operated on. For example... Figure 11 The description states that E1 stage 1131 can receive operands from either stream buffer 1141 or the register file, schematically shown as 1142. For load and store instructions, an execution address is generated, and the address modification is written to the register file. For branch instructions, this affects the branch fetch packet in the PG stage. (The text then repeats itself, so the translation stops.) Figure 11 The diagram illustrates that load and store instructions access memory, schematically shown here as memory 1151. For single-loop instructions, the result is written to the destination register file when any condition of the instruction is evaluated as true. If the condition is evaluated as false, the instruction either does not write any result or has no pipeline operation after E1 stage 1131.

[0107] During E2 phase 1132, the load instruction sends the address to memory. The store instruction sends the address and data to memory. If saturation occurs, the single-loop instruction that saturates the result sets the SAT bit in the control status register (CSR). For the two-loop instruction, the result is written to the destination register file.

[0108] During E3 phase 1133, data memory access is performed. If saturation occurs, any multiplication instructions that saturate the result set the SAT bit in the control status register (CSR). For 3-loop instructions, the result is written to the destination register file.

[0109] During E4 stage 1134, load instructions bring data to the processor boundary. For the 4-loop instruction, the result is written to the destination register file.

[0110] During E5 stage 1135, the load instruction writes data into the register, such as... Figure 11 The diagram illustrates the process of inputting from memory 1151 to E5 stage 1135.

[0111] Figure 13 This example illustrates an instance of instruction code 1300 for the functional unit instructions used in this instance. Each instruction contains 32 bits and controls the operation of one of the individual controllable functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, D2 unit 226, L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, C unit 245, and P unit 246).

[0112] The creg field 1301 (bits 29 to 31) and z bit 1302 (bit 28) are optional fields used in conditional instructions. These bits are used in conditional instructions to identify the assertion register and the condition. z bit 1302 (bit 28) indicates whether the assertion is based on zero or non-zero in the assertion register. If z=1, the test is for zero. If z=0, the test is for non-zero. The case where creg=0 and z=0 is considered true to allow unconditional instructions to execute. The creg field 1301 and z field 1302 are encoded in the instructions, as shown in Table 1.

[0113] Table 1

[0114]

[0115] Execution of conditional instructions is conditional upon a value stored in a designated data register. The data register is located in the global scalar register file 211 of all functional units. Note: "z" in the z-bit column refers to the zero / non-zero comparison selection mentioned above, and "x" is an irrelevant state. This encoding designates a subset of the 16 global registers as assertion registers, which hold the bits in the instruction encoding. Note: Unconditional instructions do not have optional bits. For unconditional instructions, bits (28 to 31) in fields 1301 and 1302 are used as additional opcode bits.

[0116] The dst field 1303 (bits 23 to 27) specifies the register in the corresponding register file as the destination of the instruction result.

[0117] The src2 / cst field 1304 (bits 18 to 22) has several meanings depending on the instruction opcode fields (bits 3 to 12 for all instructions and bits 28 to 31 for unconditional instructions). One meaning is to designate a register in the corresponding register file as the second operand. Another meaning is an immediate constant. Depending on the instruction type, field 1304 is treated as an unsigned integer extended to the specified data length plus zero, or as a signed integer extended to the specified data length plus the sign.

[0118] The scr1 field 1305 (bits 13 to 17) designates a register in the corresponding register file as the first source operand.

[0119] The opcode field 1306 (bits 3 to 12) for all instructions (and bits 28 to 31 for unconditional instructions) specifies the instruction type and indicates the appropriate instruction options, including explicit specification of the functional unit used and the operation performed. A detailed explanation of the opcodes is beyond the scope of this description, except for the instruction options described below.

[0120] Bit 1307 (bit 2) is used for immediate constant instructions, where the constant is expandable. If e=1, the immediate constant is expanded as described below. If e=0, the immediate constant is not expanded, and the immediate constant is specified by field 1304 (bits 18 to 22) of src2 / cst. Note: Bit 1307 is used for some instructions. Therefore, bit 1307 can be omitted from some instructions under appropriate encoding, and this bit can be used as an additional opcode bit.

[0121] Bit 1308 (bit 1) indicates either scalar data path side A 115 or vector data path side B 116. If s=0, then scalar data path side A 115 is selected, which limits the functional unit to... Figure 2The description includes L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 226, along with their corresponding register files. Similarly, s=1 selects the vector data path side B 116, which limits the functional unit to... Figure 2 The L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, P unit 246 and their corresponding register files are described in the document.

[0122] Bit p1309 (bit 0) marks the execution packet. Bit p determines whether an instruction is executed concurrently with subsequent instructions. Bit p scans from lower to higher addresses. For the current instruction, if p=1, the next instruction is executed concurrently with the current instruction. For the current instruction, if p=0, the next instruction is executed in the loop following the current instruction. All instructions executed concurrently constitute an execution packet. An execution packet can contain up to 16 instructions. Each instruction in an execution packet uses a different functional unit.

[0123] There are two distinct condition code extension slots. Each execution package may contain each of these unique 32-bit condition code extension slots, which contains a 4-bit creg / z field for instructions within the same execution package. Figure 14 This describes the encoding used for condition code extension slot 0, and Figure 15 This describes the encoding used for condition code extension slot 1.

[0124] Figure 14This describes the encoding used for the 32-bit condition code extension slot 0. Field 1401 (bits 28 to 31) specifies the 4 creg / z bits assigned to the L1 unit 221 instruction within the same execution package. Field 1402 (bits 27 to 24) specifies the 4 creg / z bits assigned to the L2 unit 241 instruction within the same execution package. Field 1403 (bits 20 to 23) specifies the 4 creg / z bits assigned to the S1 unit 222 instruction within the same execution package. Field 1404 (bits 16 to 19) specifies the 4 creg / z bits assigned to the S2 unit 242 instruction within the same execution package. Field 1405 (bits 12 to 15) specifies the 4 creg / z bits assigned to the D1 unit 225 instruction within the same execution package. Field 1406 (bits 8 to 11) specifies the 4 creg / z bits assigned to the D2 unit 226 instruction within the same execution package. Field 1407 (bits 6 and 7) is unused / reserved. Field 1408 (bits 0 to 5) is encoded as a unique set of bits (CCEX0) to identify condition code extension slot 0. Once the unique ID of condition code extension slot 0 is detected, the corresponding creg / z bit is used to control the conditional execution of any L1 unit 221, L2 unit 241, S1 unit 222, S2 unit 242, D1 unit 225, and D2 unit 226 instructions in the same execution package. The creg / z bit is interpreted as shown in Table 1. If the corresponding instruction is conditional (including the creg / z bit), then the corresponding bit in condition code extension slot 0 overrides the condition code bit in the instruction. Setting the creg / z bit to "0000" makes the instruction unconditional. Therefore, a properly encoded condition code extension slot 0 can make some corresponding instructions conditional and others unconditional.

[0125] Figure 15This describes the encoding used for the 32-bit condition code extension slot 1. Field 1501 (bits 28 to 31) specifies the 4 creg / z bits assigned to the M1 unit 223 instruction in the same execution package. Field 1502 (bits 27 to 24) specifies the 4 creg / z bits assigned to the M2 unit 243 instruction in the same execution package. Field 1503 (bits 19 to 23) specifies the 4 creg / z bits assigned to the C unit 245 instruction in the same execution package. Field 1504 (bits 16 to 19) specifies the 4 creg / z bits assigned to the N1 unit 224 instruction in the same execution package. Field 1505 (bits 12 to 15) specifies the 4 creg / z bits assigned to the N2 unit 244 instruction in the same execution package. Field 1506 (bits 6 and 11) is unused / reserved. Field 1507 (bits 0 to 5) is encoded as a unique set of bits (CCEX1) to identify condition code extension slot 1. Once the unique ID of condition code extension slot 1 is detected, the corresponding creg / z bits are used to control the conditional execution of any instructions in M1 unit 223, M2 unit 243, C unit 245, N1 unit 224, and N2 unit 244 within the same execution package. These creg / z bits are interpreted as shown in Table 1. If the corresponding instruction is conditional (including the creg / z bits), then the corresponding bits in condition code extension slot 1 override the condition code bits in the instruction. Setting the creg / z bits to "0000" makes the instruction unconditional. Therefore, a properly encoded condition code extension slot 1 can make some instructions conditional and others unconditional.

[0126] Both condition code extension slot 0 and condition code extension slot 1 can contain p bits as described above. Figure 13 The execution package is defined as described. In this example, as... Figure 14 and 15 The description states that code extension slot 0 and condition code extension slot 1 have bits 0 (p bits) encoded as 1. Therefore, condition code extension slot 0 and condition code extension slot 1 may not be in the last instruction slot of the execution package.

[0127] There are two distinct 32-bit constant extension slots. Each execution packet may contain one of these unique constant extension slots, which contains 27 bits as high-order bits concatenated with the 5-bit constant field 1305 to form a 32-bit constant. As mentioned in the instruction encoding description above, some instructions define the src2 / cst field 1304 as a constant rather than a source register identifier. At least some of these instructions may use a constant extension slot to extend the constant to 32 bits.

[0128] Figure 16 This describes the fields for constant expansion slot 0. Each execution package may contain one example of constant expansion slot 0 and one example of constant expansion slot 1. Figure 16The constant extension slot 0 1600 contains two fields. Field 1601 (bits 5 to 31) constitutes the 27 most significant bits of an extended 32-bit constant containing the target instruction scr2 / cst field 1304 as the 5 least significant bits. Field 1602 (bits 0 to 4) is encoded as a unique set of bits (CSTX0) to identify constant extension slot 0. In this example, constant extension slot 0 1600 can be used to extend the constant of one of the following instructions in the same execution package: L1 unit 221 instruction, D1 unit 225 instruction, S2 unit 242 instruction, D2 unit 226 instruction, M2 unit 243 instruction, N2 unit 244 instruction, branch instruction, or C unit 245 instruction. Constant extension slot 1 is similar to constant extension slot 0, except that bits 0 to 4 are encoded as a unique set of bits (CSTX1) to identify constant extension slot 1. In this example, constant extension slot 1 can be used to extend the constant of one of the following instructions in the same execution package: L2 unit 241 instruction, D2 unit 226 instruction, S1 unit 222 instruction, D1 unit 225 instruction, M1 unit 223 instruction, or N1 unit 224 instruction.

[0129] Constant extension slots 0 and 1 are used as follows. The target instruction is of a type that allows constant specifications. In this example, extension is implemented by replacing an input operand register specification field with the least significant bit of the constant, as described above regarding the scr2 / cst field 1304. The instruction decoder 113 determines this from the instruction opcode bits (called the immediate field). The target instruction also contains a constant extension bit (e bit 1307), which is dedicated to signaling whether the specified constant is unextended (constant extension bit = 0) or extended (constant extension bit = 1). If the instruction decoder 113 detects constant extension slot 0 or constant extension slot 1, then the instruction decoder 113 further examines other instructions within the execution packet corresponding to the detected constant extension slot. If a corresponding instruction has a constant extension bit (e bit 1307) equal to 1, then constant extension is performed.

[0130] Figure 17 This is a partial block diagram 1700 illustrating the extension of the constant. Figure 17 Assuming instruction decoder 113 ( Figure 1A constant extension slot and its corresponding instruction are detected in the same execution package. Instruction decoder 113 supplies 27 extension bits from the constant extension slot (bit field 1601) and 5 constant bits (bit field 1305) from the corresponding instruction to serializer 1701. Serializer 1701 forms a single 32-bit word from these two parts. In this example, the 27 extension bits (bit field 1601) from the constant extension slot are the most significant bits, and the 5 constant bits (bit field 1305) are the least significant bits. The combined 32-bit word is supplied to one input of multiplexer 1702. The 5 constant bits from the corresponding instruction field 1305 supply a second input to multiplexer 1702. The selection of multiplexer 1702 is controlled by the state of the constant extension bits. If the constant extension bit (e bit 1307) is 1 (extended), then multiplexer 1702 selects the serialized 32-bit input. If the constant extension bits are 0 (unextended), then multiplexer 1702 selects 5 constant bits from the corresponding instruction field 1305. The output of multiplexer 1702 supplies the input of symbol extension unit 1703.

[0131] The symbol extension unit 1703 forms the final operand value from the input from the multiplexer 1703. The symbol extension unit 1703 receives control input scalar / vector and data size. The scalar / vector input indicates whether the corresponding instruction is a scalar instruction or a vector instruction. The functional units (L1 unit 221, S1 unit 222, M1 unit 223, N1 unit 224, D1 unit 225, and D2 unit 226) on the data path side A 115 execute scalar instructions. Any instruction involving one of these functional units is a scalar instruction. The functional units (L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, and C unit 245) on the data path side can execute either scalar or vector instructions. The instruction decoder 113 determines whether the instruction is a scalar or vector instruction from the opcode bits. The P unit 246 can execute scalar instructions. The data size can be 8 bits (byte B), 16 bits (half-word H), 32 bits (word W), or 64 bits (double word D).

[0132] Table 2 lists the operations of the symbol extension unit 1703 for various options.

[0133] Table 2

[0134]

[0135] Both constant expansion slot 0 and constant expansion slot 1 can contain p bits as described above. Figure 13 The execution package is defined as described. In this example, as with the condition code extension slots, constant extension slot 0 and constant extension slot 1 have bit 0 (p bits) of the encoded bit 1. Therefore, neither constant extension slot 0 nor constant extension slot 1 needs to be in the last instruction slot of the execution package.

[0136] The execution package may contain constant extension slot 0 or 1 and more than one corresponding instruction marked as constant extension (e bit = 1). For this event, for constant extension slot 0, more than one of the following instructions in the execution package—L1 unit 221, D1 unit 225, S2 unit 242, D2 unit 226, M2 unit 243, or N2 unit 244—may have e bit 1. For this event, for constant extension slot 1, more than one of the following instructions in the execution package—L2 unit 241, D2 unit 226, S1 unit 222, D1 unit 225, M1 unit 223, or N1 unit 224—may have e bit 1. In one instance, instruction decoder 113 determines that this event is an invalid operation and is not supported. Alternatively, the combination can be supported by the extension bits of the constant extension slot applied to each corresponding function unit instruction marked as constant extension.

[0137] Units L1 221, S1 222, L2 241, S2 242, and C 245 typically operate in Single Instruction Multiple Data (SIMD) mode. In this SIMD mode, the same instruction is applied to data packaged in two operands. Each operand holds multiple data elements placed in a predetermined slot. SIMD operations are implemented through carry control at data boundaries. This carry control enables operations on different data widths.

[0138] Figure 18The carry control logic is explained below. The AND gate 1801 receives a carry output of bit N within an operand-wide arithmetic logic unit (A115 for 64 bits on the scalar data path and B116 for 512 bits on the vector data path). The AND gate 1801 also receives carry control signals, as further described below. The output of the AND gate 1801 is supplied to the carry input of bit N+1 of the operand-wide arithmetic logic unit. For example, the AND gate of the AND gate 1801 is positioned between each pair of bits at possible data boundaries. For instance, for 8 bits of data, this AND gate would be between bits 7 and 8, bits 15 and 16, bits 23 and 24, etc. Each AND gate receives a corresponding carry control signal. If the data size is the minimum size, each carry control signal is 0, effectively preventing carry transmission between adjacent bits. If the selected data size requires two arithmetic logic unit segments, then the corresponding carry control signal is 1. Table 3 below shows example carry control signals for the case of a 512-bit wide operand used by the B116 functional unit on the vector data path side, which can be divided into segments of 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, or 256 bits. In Table 3, 32 high-order bits control the high-order carry (bits 128 to 511) and 32 low-order bits control the low-order carry (bits 0 to 127). There is no need to control the carry output of the most significant bit, therefore only 63 carry control signals are required.

[0139] Table 3

[0140]

[0141] For integer powers of 2 (2) N Operations involving data sizes of 2 are very common. However, carry control techniques are not limited to powers of 2, but can be applied to other data sizes and operand widths.

[0142] In this example, at least L2 unit 241 and S2 unit 242 employ two types of SIMD instructions using registers in the assertion register file 234. In this example, the SIMD vector assertion instruction operates on a specified data size. The data size includes byte (8-bit) data, half-word (16-bit) data, word (32-bit) data, double-word (64-bit) data, quad-word (128-bit) data, and half-vector (256-bit) data. In the first of these instruction types, the functional unit (L unit 241 or S unit 242) performs a SIMD comparison of packed data in two general-purpose data registers and supplies the result to the assertion data register. The instruction specifies the data size, two general-purpose data register operands, and the destination assertion register. In this example, each assertion data register contains one bit corresponding to each minimum data size portion of the general-purpose data register. In this current example, the general-purpose data register is 512 bits (64 bytes), and the assertion data register is 64 bits (8 bytes). Each bit of the assertion data register corresponds to 8 bits of the general-purpose data register. A comparison is performed on a specified data size (8, 16, 32, 64, 128, or 256 bits). If the comparison is true, the function supplyes 1s to all assertion register bits corresponding to the portion of the data size. If the comparison is false, the function supplyes 0s to the assertion register bits corresponding to the portion of the data size. In this example, the enabled comparison operations include less than, greater than, and equal to.

[0143] In the second instruction type, the functional unit (L2 unit 241 or S2 unit 242) performs a first SIMD operation or a second SIMD operation on the packed data in the general-purpose data register individually based on the data state in the assertion data register. The instruction specifies the data size, one or two general-purpose data register operands, controls the assertion register, and the destination of the general-purpose data register. For example, the functional unit may select the first data element of the first operand or the second data element of the second operand to be stored in the destination register, depending on the 1 / 0 state of the corresponding bit in the assertion data register for each data size portion of two vector operands. In another instance, the data element of a single vector operand may be saved to memory or not saved, depending on the data of the corresponding bit in the assertion register.

[0144] The operation of P unit 245 allows for various composite vector SIMD operations based on more than one vector comparison. For example, range determination can be performed using two comparisons. In a SIMD operation, a candidate vector is compared with a vector reference having the minimum value of the range packed in a data register. The greater than result is a scalar data with bits corresponding to the SIMD data width, set to 0 or 1 depending on the SIMD comparison, and stored in an assertion data register. Performing another SIMD comparison of the candidate vector with another reference vector having the maximum value of the range packed in a different data register produces another scalar with a less than result stored in another assertion register. Then, P unit performs an AND operation on the two assertion registers. The AND result indicates whether each SIMD data portion of the candidate vector is within or outside the range. The P unit BITCNT instruction for the AND result can produce a count of data elements within the comparison range. The P unit NEG function can be used to transform various expressions, such as converting a less than comparison result to a greater than or equal comparison result, a greater than comparison result to a less than or equal comparison result, or an equal comparison result to a not equal comparison result.

[0145] Streaming engine

[0146] Figure 19 yes Figure 1 A conceptual diagram of the streaming engine 125 of the instance processor 100. Figure 19 This description illustrates the processing of a single stream from two streams controlled by streaming engine 125. Streaming engine 1900 includes a stream address generator 1901. Stream address generator 1901 sequentially generates addresses of the elements of the stream and supplies these element addresses to system memory 1910. Memory 1910 retrieves data stored at the element addresses (data elements) and supplies these data elements to a data FIFO buffer 1902. The data FIFO buffer 1902 provides buffering between memory 1910 and processor 1920. Data formatter 1903 receives data elements from data FIFO memory 1902 and provides data formatting according to the stream definition. This process is described in more detail herein. Streaming engine 1900 supplies formatted data elements from data formatter 1903 to processor 1920. The program executing on processor 1920 consumes the data and produces output.

[0147] Stream elements can reside in system memory. Memory does not impose a specific structure on a stream. The program defines a stream and thereby imposes a structure by specifying stream attributes such as the address of the first element of the stream, the size and type of the elements in the stream, the formatting of the data in the stream, and the address sequence associated with the stream.

[0148] The streaming engine defines the address sequence of stream elements based on pointers traversing memory. Multiple nested loops control the paths traversed by the pointers. The iteration count of each loop level indicates the number of times that level is repeated. The dimension gives the distance between the pointer positions within each loop level.

[0149] In the basic forward flow, the innermost loop consumes physically contiguous elements from memory because the implicit dimension of the innermost loop is one element. The pointer moves from element to element in continuously increasing order. At each level outside the inner loop, the loop moves the pointer to a new position based on the dimension size of the loop level.

[0150] This form of addressing allows a program to specify a regular path through memory using a small number of parameters. Table 4 lists the addressing parameters for basic streams.

[0151] Table 4

[0152]

[0153] In this example, the range of ELEM_BYTES is from 1 to 64 bytes, as shown in Table 5.

[0154] Table 5

[0155]

[0156] The definition above maps consecutive elements of a stream to incrementing addresses in memory, which is suitable for many algorithms. Some algorithms are better served by reading elements in descending memory address order or by reverse stream addressing. For example, discrete convolution computes the vector dot product, as illustrated by expression (1).

[0157] (1)

[0158] In expression [1], f[] and g[] represent arrays in memory. For each output, the algorithm reads f[] in the forward direction and g[] in the reverse direction. The practical filter limits the exponent range of [x] and [tx] to a finite number of elements. To support this mode, the streaming engine supports reading elements in descending address order.

[0159] Matrix multiplication presents a unique problem for streaming engines. Each element in the matrix product is the vector dot product between a row from the first matrix and a column from the second matrix. Programs can store matrices in either row-major or column-major order. Row-major order stores all elements of a single row contiguously in memory. Column-major order stores all elements of a single column contiguously in memory. Matrices can be stored in the same order as the language's default array order. Therefore, only one of the two matrices in matrix multiplication maps to the streaming engine's 2D stream definition. In practice, this involves stepping through columns of one array and rows of another. Streaming engines support implicit matrix transpose with transposed streams. Transposed streams avoid the cost of explicitly transforming data in memory. Instead of accessing data in strictly consecutive element order, the streaming engine efficiently interchanges the two inner loop dimensions of the traversal order, thus accessing elements along the second dimension of the continuous vector path.

[0160] This algorithm is effective but cannot be implemented for small element sizes. Some algorithms are suitable for matrix chunks, which consist of multiple columns and rows together. Therefore, the streaming engine defines a separate transpose granularity. The hardware imposes a minimum granularity. The transpose granularity must be at least as large as the element size. The transpose granularity causes the streaming engine to fetch one or more consecutive elements from dimension 0 before moving along dimension 1. When the granularity is equal to the element size, a single column is fetched from the row-major array. Otherwise, the granularity specifies fetching two, four, or more columns from the row-major array at a time. This also applies to column-major layouts by swapping rows and columns in this description. The GRANULE parameter indicates the transpose granularity (in bytes).

[0161] Another common matrix multiplication technique swaps the innermost two loops of the matrix multiplication. The resulting inner loop no longer reads columns of one matrix while simultaneously reading rows across the other. For example, the algorithm can promote an item outside the inner loop, replacing it with a scalar value. The innermost loop can be implemented with a single scalar using vector multiplication followed by vector addition. Alternatively, the scalar value can be copied across the length of the vector and used with vector multiplication. The streaming engine in this example directly supports the latter case with element-copying mode and the associated use case. In this mode, the streaming engine reads particles smaller than the full vector size and copies the particles to fill the next vector output.

[0162] The streaming engine treats each complex number as a single element having two sub-elements that give the real and imaginary parts (Cartesian coordinates) or the magnitude and angle (polar coordinates) of the complex number. Not all programs or peripherals agree on the order in which these sub-elements should appear in memory. Therefore, the streaming engine provides the ability to swap two sub-elements of a complex number at no cost. This allows swapping half of an element without decoding its contents and can be used to swap sub-element pairs of any type, not just complex numbers.

[0163] Algorithms typically prefer to operate at high precision, but high-precision values ​​require more storage and bandwidth compared to lower-precision values. Generally, a program stores data in memory at low precision, promotes the value to higher precision for computation, and then demotes the value to lower precision for storage. Streaming engines directly support this type of operation by allowing algorithms to specify a first-level type promotion. In this example, each sub-element can be promoted to a larger type size using an integer with sign or zero extension. In some instances, streaming engines support floating-point promotion, thus promoting 16-bit and 32-bit floating-point values ​​to 32-bit and 64-bit formats respectively.

[0164] Although the streaming engine defines a stream as a sequence of discrete data elements, the processing unit core 110 consumes elements consecutively packed in a vector. A vector is similar to a stream because it contains multiple homogeneous elements with some implicit sequence. Because the streaming engine reads a stream, but the processing unit core 110 consumes a vector, the streaming engine maps the stream to a vector in a consistent manner.

[0165] The vector contains channels of equal size, each containing child elements. Processing unit core 110 designates the rightmost channel of the vector as channel 0, regardless of the current byte order mode. The number of channels increases from right to left. The actual number of channels within a vector depends on the length of the vector and the size of the child elements.

[0166] Figure 20 This describes the sequence of formatting operations performed by formatter 1903. Formatter 1903 comprises three sections: input section 2010, formatting section 2020, and output section 2030. Input section 2010 receives data retrieved from system memory 1910 accessed by stream address generator 1901. The data can be retrieved via linear access stream 2011 or via transposed access stream 2012.

[0167] Formatting section 2020 contains various formatting blocks. The formatting performed by these blocks within formatter 1903 is further described below. Complex exchange block 2021 optionally exchanges two sub-elements forming a complex number element. Type promotion block 2022 optionally promotes each data element to a larger data size. Promotion includes zero extension for unsigned integers and sign extension for signed integers. Extraction block 2023 optionally extracts data elements. In this example, extraction could be a 2:1 ratio to every other data element or a 4:1 ratio to every fourth data element. Element copying block 2024 optionally copies individual data elements. In this example, data element copying is a power of 2 (2^2 / 2^2). N(where N is an integer), which includes multipliers of 2, 4, 8, 16, 32, and 64. In this example, data replication can be extended across multiple destination vectors. The Vector Length Masking / Group Replication Block 2025 has two main functions. The independently specified vector length VECLEN controls the data elements supplied to each output data vector. When group replication is off, extra channels in the output data vector are zero-padded and marked as invalid. When group replication is on, input data elements of the specified vector length are replicated to fill the output data vector.

[0168] Output section 2030 holds data for output to the corresponding functional unit. Registers and buffers 2031 for the processor store data from the processing unit core 110. Figure 1 The functional unit is used as a formatted vector for the operand data.

[0169] Figure 21 This illustrates an example of channel allocation in a vector. Vector 2100 is divided into eight 64-bit channels (8 × 64 bits = 512 bits, the vector length). Channel 0 contains bits 0 to 63, channel 1 contains bits 64 to 127, channel 2 contains bits 128 to 191, channel 3 contains bits 192 to 255, channel 4 contains bits 256 to 319, channel 5 contains bits 320 to 383, channel 6 contains bits 384 to 447, and channel 7 contains bits 448 to 511.

[0170] Figure 22 Here is another example illustrating channel allocation in a vector. Vector 2200 is divided into 16 32-bit channels (16 × 32 bits = 512 bits, the vector length). Channel 0 contains bits 0 to 31, channel 1 contains bits 32 to 63, channel 2 contains bits 64 to 95, channel 3 contains bits 96 to 127, channel 4 contains bits 128 to 159, channel 5 contains bits 160 to 191, channel 6 contains bits 192 to 223, channel 7 contains bits 224 to 255, channel 8 contains bits 256 to 287, channel 9 contains bits 288 to 319, channel 10 contains bits 320 to 351, channel 11 contains bits 352 to 383, channel 12 contains bits 384 to 415, channel 13 contains bits 416 to 447, channel 14 contains bits 448 to 479, and channel 15 contains bits 480 to 511.

[0171] The streaming engine directly maps the innermost stream dimension to vector channels. It maps earlier elements within the innermost stream dimension to lower channel numbers and later elements to higher channel numbers, regardless of whether the stream is progressed in ascending or descending address order. Regardless of the stream's defined order, the streaming engine stores elements in the vector in ascending channel order. For non-complex data, the streaming engine places the first element in processing unit core 110 (…). Figure 1For the vector obtained, in channel 0, the second element is placed in channel 1, and so on. For complex data, the streaming engine places the first element in channels 0 and 1, the second element in channels 2 and 3, and so on. Sub-elements within an element maintain the same relative order, regardless of the flow direction. For unswapped complex elements, the sub-elements with the lower address in each pair are placed in even-numbered channels, and the sub-elements with the higher address in each pair are placed in odd-numbered channels. For swapped complex elements, the placement is reversed.

[0172] The streaming engine fills each vector with as many elements as possible from the innermost stream dimension as possible from the processing unit cores 110. If the innermost dimension is not a multiple of the vector length, the streaming engine pads the dimension with zeros until it becomes a multiple of the vector length. As noted below, the streaming engine also marks channels as invalid. Therefore, for higher-dimensional streams, the first element from each iteration of the outer dimension arrives at channel 0 of the vector. The streaming engine maps the innermost dimension to consecutive channels in the vector. For transposed streams, the innermost dimension contains a group of sub-elements along dimension 1, not dimension 0, because the transpose swaps these two dimensions.

[0173] Compared to one-dimensional streams, two-dimensional (2D) streams exhibit greater variation. A basic 2D stream extracts smaller rectangles from larger ones. A transposed 2D stream reads rectangles column-by-column rather than row-by-row. A cyclic stream (where the second dimension overlaps with the first dimension) performs repeated cyclic FIR filter taps on finite impulse response (FIR) filter samples provided by a sliding window of input samples.

[0174] Figure 23 This describes the memory region that can be accessed using a basic two-dimensional stream. The two inner dimensions, represented by ELEM_BYTES, ICNT0, DIM1, and ICNT1 (refer to Table 4), provide sufficient flexibility to describe the extraction of a smaller rectangle 2320 with dimensions 2321 and 2322 from a larger rectangle 2310 with dimensions 2311 and 2312. In this example, rectangle 2320 is a 9x13 rectangle with 64-bit values, and rectangle 2310 is a larger 11x19 rectangle. The following stream parameters define this stream: ICNT0 = 9, ELEM_BYTES = 8, ICNT1 = 13, and DIM1 = 88 (11x8).

[0175] Therefore, the iteration count in 0-dimensional 2321 is 9, and the iteration count in 1-dimensional 2322 is 13. Note: ELEM_BYTES scales the innermost dimension. The first dimension has ICNT0 elements of size ELEM_BYTES. The stream address generator does not scale the outer dimensions. Therefore, DIM1 = 88, which is 11 elements scaled by 8 bytes per element.

[0176] Figure 24 illustrate Figure 23 The order of elements within the instance stream. The streaming engine retrieves the elements of the stream in the order described in sequence 2400. The first 9 elements come from the first row of rectangle 2320, skipping from left to right from row 1 to 8. Elements 10 to 24 come from the second row, and so on. When the stream moves from the 9th element to the 10th element... Figure 24 When jumping to position 9 in the loop, the streaming engine calculates the new position based on the pointer's position at the beginning of the inner loop, rather than at the end of the first dimension. Therefore, DIM1 is independent of ELEM_BYTES and ICNT0. DIM1 represents the distance between the first bytes of each consecutive line.

[0177] The transposed stream is accessed along dimension 1 preceding dimension 0. The following examples illustrate transposed streams with different transpose granularities. Figure 25 This describes how to extract a smaller rectangle 2520 (12×8) with dimensions 2521 and 2522 from a larger rectangle 2510 (14×13) with dimensions 2511 and 2512. Figure 25 In this context, ELEM_BYTES equals 2.

[0178] Figure 26 Explain how the streaming engine obtains data at a transpose granularity of 4 bytes. Figure 25 The stream of instances. The get pattern 2600 retrieves pairs of elements from each row (because granularity 4 is twice that of ELEM_BYTES 2), but otherwise moves down the columns. Once the streaming engine reaches the bottom of a pair of columns, it repeats the pattern for the next pair of columns.

[0179] Figure 27 Explain how the streaming engine obtains data at an 8-byte transpose granularity. Figure 25 The stream of instances. The overall structure remains the same. The streaming engine fetches 4 elements from each row before moving to the next row in the column (because granularity 8 is four times that of ELEM_BYTES 2), as shown in fetch mode 2700.

[0180] The stream examined so far reads each element from memory only once. A stream can read a given element from memory multiple times, effectively cycling through portions of memory. FIR filters exhibit two common cycling patterns: rereading the same filter taps for each output and reading input samples from a sliding window. Two consecutive outputs require inputs from two overlapping windows.

[0181] Figure 28 illustrate Figure 1Details of the streaming engine 125. The streaming engine 125 contains three main sections: Stream 0 engine 2810, Stream 1 engine 2820, and a shared L2 interface 2830. Stream 0 engine 2810 and Stream 1 engine 2820 both contain the same hardware for parallel operation. Stream 0 engine 2810 and Stream 1 engine 2820 share the L2 interface 2830. Stream 0 engine 2810 and Stream 1 engine 2820 each feed data to the processing unit core 110 at a rate of up to 512 bits / cycle per loop. Figure 1 This provides data, which is achieved through a dedicated stream path and a shared dual L2 interface.

[0182] Each stream engine 125 includes a corresponding dedicated 6D stream address generator 2811 / 2821, which generates a new misaligned request per cycle. As further described herein, the address generator 2811 / 2821 outputs a 512-bit aligned address that overlaps with the elements in the sequence defined by the stream parameters.

[0183] Each address generator 2811 / 2821 is connected to a corresponding dedicated microtable backstop buffer (μTLB) 2812 / 2822. Each μTLB 2812 / 2822 translates a single 48-bit virtual address into a 44-bit physical address per cycle. Each μTLB 2812 / 2822 has 8 entries, thus covering a minimum of 32kB with a 4kB page or a maximum of 16MB with a 2MB page. Each address generator 2811 / 2821 generates 2 addresses per cycle. Each μTLB 2812 / 2822 translates only 1 address per cycle. To maintain throughput, the streaming engine 125 operates under the assumption that most stream references are within the same 4kB page. Therefore, address translation does not modify bits 0 through 11 of the address. If aout0 and aout1 are in the same 4 kB page (aout0[47:12] is the same as aout1[47:12]), then μTLB 2812 / 2822 will only translate aout0 and will use the translation for the high bits of both addresses.

[0184] The translated addresses are queued in the corresponding command queues 2813 / 2823. These addresses are aligned with information from the corresponding memory allocation and trace blocks 2814 / 2824. The streaming engine 125 does not explicitly manage μTLBs 2812 / 2822. The system memory management unit (MMU) invalidates the μTLBs as needed during context switching.

[0185] Storage allocation and tracking of the 2814 / 2824 management stream's internal storage enables data reuse detection and tracks the lifetime of each data item. Each block accepts two virtual addresses per cycle and binds these addresses to slots in the internal storage device. Data storage is organized as an array of slots. The streaming engine maintains the following metadata to track the content and lifetime of data in each slot: the 49-bit virtual address associated with the slot, a valid bit indicating a valid address, a ready bit indicating that data has arrived at the address, a valid bit indicating whether any references exceed the validity of this data, and the last reference value indicating the most recent reference in the reference queue for this slot. Storage allocation and tracking are further described herein.

[0186] The corresponding reference queues 2815 / 2825 store reference sequences generated by the corresponding address generators 2811 / 2821. These reference sequences enable the data formatting network to present data to the processing unit core 110 in the correct order. Each entry in the corresponding reference queues 2815 / 2825 contains information required to read data from the data storage and align it with the data used by the processing unit core 110. The corresponding reference queues 2815 / 2825 store the information listed in Table 6 in each slot.

[0187] Table 6

[0188]

[0189] As address generators 2811 / 2821 generate a new address, memory allocation and tracing 2814 / 2824 inserts a reference into reference queues 2815 / 2825. When data becomes available and there is space in the stream header register, memory allocation and tracing 2814 / 2824 removes the reference from reference queues 2815 / 2825. As memory allocation and tracing 2814 / 2824 removes the slot reference from reference queues 2815 / 2825 and formats the data, it checks the last reference of the corresponding slot. Memory allocation and tracing 2814 / 2824 compares the last reference of the removed pointer and the last reference of the slot in reference queues 2815 / 2825. If the pointer and the last reference of the record match, then memory allocation and tracing 2814 / 2824 marks the slot as invalid once the data is no longer needed.

[0190] The streaming engine 125 has corresponding data storage devices 2816 / 2826 for the selected number of elements. Deep buffering allows the streaming engine to acquire the stream early, thus hiding memory system latency. Each data storage device 2816 / 2826 accommodates two simultaneous read operations and two simultaneous write operations per cycle and is therefore referred to as a two-read-two-write (2r2w) data storage device. In other instances, the buffer size may vary. In this instance, the streaming engine 125 dedicates 32 slots to each stream, with each slot marked by a virtual address. Each slot holds 64 bytes of data in eight 8-byte banks.

[0191] Implementation Reference for Data Storage Devices 2816 / 2826 and Corresponding Storage Allocation / Tracking Logic 2814 / 2824 and Reference Queues 2815 / 2825 Figure 19 The data described is FIFO 1902.

[0192] The corresponding butterfly network 2817 / 2827 includes implementation formatter 1903 ( Figure 19 , Figure 20 The seven-stage butterfly network (2817 / 2827) receives 128 bytes of input and produces 64 bytes of output. The first stage of the butterfly is actually a half-stage that collects bytes from two slots with mismatched data and merges the collected bytes into a single rotated 64-byte array. The remaining six stages form the standard butterfly network. The corresponding butterfly network (2817 / 2827) performs the following operations: rotates the next element down to byte channel 0; promotes the data type by powers of 2 if requested; swaps the real and imaginary parts of a complex number if requested; and converts the big-endian to little-endian if the processing unit core 110 is currently in big-endian mode. The user specifies the element size, type promotion, and real / imaginary part swapping as parameters of the stream.

[0193] The streaming engine 125 attempts to acquire and format data before the processing unit core 110 requests it to maintain full throughput. The corresponding stream head registers 2818 / 2828 provide minimal buffering, keeping the process fully pipelined. The corresponding stream head registers 2818 / 2828 are not directly visible in the architecture. Each stream also has a corresponding stream valid register 2819 / 2829. The valid registers 2819 / 2829 indicate which elements in the corresponding stream head registers 2818 / 2828 are valid. The outputs of the stream head registers 2818 / 2828 and the valid registers 2819 / 2829 are provided to the processing unit core 110 via buses 2840 / 2841.

[0194] The two streams 2810 / 2820 share a pair of independent L2 interfaces 2830: L2 interface A (IFA) 2833 and L2 interface B (IFB) 2834. Each L2 interface provides 512 bits / cycle throughput directly to the L2 controller 130 via the corresponding bus 147 / 149 for an aggregate bandwidth of 1024 bits / cycle. Figure 1The L2 interface uses a credit-based multicore bus architecture (MBA) protocol. The MBA protocol is described in more detail in U.S. Patent 9,904,645, entitled "Multicore Bus Architecture with Non-Blocking High Performance Transaction Credit System," which is incorporated herein by reference. The L2 controller assigns command credit pooling to each interface. The pooling has sufficient credits so that each interface can send enough requests to read L2 RAM, L2 cache, and multicore shared memory controller (MSMC) memory to achieve full read-back bandwidth, as described in more detail herein.

[0195] To maximize performance, in this example, the two streams can use two L2 interfaces, allowing a single stream to send peak command rates of two requests per cycle. Each interface prioritizes one stream over the other, but this preference changes dynamically based on requests. IFA 2833 and IFB 2834 prioritize opposite streams; when IFA 2833 prioritizes stream 0, IFB 2834 prioritizes stream 1, and vice versa.

[0196] Each corresponding arbitrator 2831 / 2832 preceding each corresponding interface 2833 / 2834 applies the following basic protocol on each cycle with available credit. Arbitrator 2831 / 2832 checks if the preferred flow has a command ready to be sent. If so, arbitrator 2831 / 2832 selects the command. Arbitrator 2831 / 2832 then checks whether the alternative flow has at least two requests ready to be sent or has one command and no credit. If so, arbitrator 2831 / 2832 extracts the command from the alternative flow. If either interface issues a command, the opinions of the preferred and alternative flows are exchanged for the next request. Using this algorithm, the two interfaces dispatch requests as quickly as possible while preserving fairness between the two flows. The first rule ensures that each flow can send requests on each cycle with available credit. The second rule provides a mechanism for a flow to borrow the interface of the other flow when the second interface is idle. The third rule distributes the bandwidth requirements of each flow across the two interfaces, thereby ensuring that the two interfaces do not become bottlenecks.

[0197] The corresponding coarse-grained rotators 2835 / 2836 enable the streaming engine 125 to support a transposed matrix addressing mode. In this mode, the streaming engine 125 swaps the two innermost dimensions of a multidimensional loop to access the array column-by-column rather than row-by-row. The corresponding rotators 2835 / 2836 are not architecturally visible.

[0198] Figure 29This section describes the example stream template register 2900. The stream definition template provides the full structure of a stream containing data. Iteration counts and dimensions provide most of the structure, while various flags provide the remaining details. In this example, a single stream template 2900 is defined for all streams containing data. All stream types supported by the streaming engine are covered by template 2900. The streaming engine supports six levels of nested loops for addressing elements within the stream. Most fields in the stream template 2900 directly map to parameters in the algorithm. The number above the fields is the number of bits within a 256-bit vector. Table 7 shows the stream field definitions for the stream template.

[0199] Table 7

[0200]

[0201] Loop 0 is the innermost loop, and loop 5 is the outermost loop. In the current instance, DIM0 is equal to ELEM_BYTES, which defines physically continuous data. Therefore, stream template register 2900 does not define DIM0. Streaming engine 125 interprets the iteration count as an unsigned integer and the dimension as an unscaled signed integer. An iteration count of 0 at any level (ICNT0, ICNT1, ICNT2, ICNT3, ICNT4, or ICNT5) indicates an empty stream. Each iteration count must be at least 1 to define a valid stream. The template above specifies the element type, the length of the stream, and the dimension. Stream instructions specify the start address individually, for example, by specifying a scalar register in file 211 that stores the start address. Therefore, a program can open multiple streams using the same template but different registers storing the start address.

[0202] Figure 30 illustrate Figure 29 The example shown is a subfield definition of flag field 2921. Figure 30 As shown, the flag field 2911 is 6 bytes or 48 bits. Figure 30 Displays the number of bits in the fields. Table 8 shows the definitions of these fields.

[0203] Table 8

[0204]

[0205] The ELTYPE field 3001 defines the data type of the elements in the stream. The 4-bit encoding of the ELTYPE field 3001 is defined as shown in Table 9.

[0206] Table 9

[0207]

[0208] The real / complex number type determines whether the streaming engine treats each element as either a real number or two parts of a complex number (real / imaginary part or magnitude / angle) and also specifies whether to swap the two parts of a complex number. Complex numbers have a total element size that is twice the size of their child elements. Otherwise, the child element size is equal to the total element size.

[0209] The child element size determines the type and vector channel width used for type promotion. For example, when a streaming request is type promoted, a 16-bit child element is promoted to a 32-bit or 64-bit child element. The vector channel width is determined in processing unit core 110 ( Figure 1 This is important when operating in big-endian mode because the core 110 arranges vectors in little-endian order.

[0210] The total element size specifies the smallest granularity of the stream, determining the number of bytes the stream acquires in each iteration of the innermost loop. The stream reads all elements in ascending or descending order. Therefore, the innermost dimension of the stream spans ICNT0 × total element size bytes.

[0211] The TRANSPOSE field 3002 determines whether the streaming engine accesses the stream in transposed order. Transposed order swaps the two internal addressing levels. The TRANSPOSE field 3002 also indicates the granularity used for transposing the stream. The encoding of the three bits of the TRANSPOSE field 3002 is defined as shown in Table 10 for normal 2D operation.

[0212] Table 10

[0213]

[0214] The streaming engine 125 can transpose data elements with different granularities of element size, allowing the program to retrieve multiple columns of elements from each row. The transpose granularity cannot be smaller than the element size. The TRANSPOSE field 3002 and the DIMFMT field 3009 interact as further described below.

[0215] The PROMOTE field 3003 controls whether the streaming engine promotes child elements in the stream and the promotion type. When enabled, the streaming engine 125 promotes the type by powers of 2. The encoding of the three bits of the PROMOTE field 3003 is defined as shown in Table 11.

[0216] Table 11

[0217]

[0218] When PROMOTE is 000 corresponding to 1x promotion, each child element remains unchanged and occupies a vector channel with a width equal to that specified by ELTYPE. When PROMOTE is 001 corresponding to 2x promotion and zero extension, each child element is treated as an unsigned integer and zero-extended to a vector channel twice the width specified by ELTYPE. 2x promotion has no effect on initial child element sizes of 64 bits. When PROMOTE is 010 corresponding to 4x promotion and zero extension, each child element is treated as an unsigned integer and zero-extended to a vector channel four times the width specified by ELTYPE. 4x promotion has no effect on initial child element sizes of 32 or 64 bits. When PROMOTE is 011 corresponding to 8x promotion and zero extension, each child element is treated as an unsigned integer and zero-extended to a vector channel eight times the width specified by ELTYPE. 8x promotion has no effect on initial child element sizes of 16, 32, or 64 bits. When PROMOTE is 101, corresponding to 2x promotion and sign extension, each child element is treated as a signed integer with a sign extension to a vector channel twice the width specified by ELTYPE. 2x promotion has no effect on initial child element sizes of 64 bits. When PROMOTE is 110, corresponding to 4x promotion and sign extension, each child element is treated as a signed integer with a sign extension to a vector channel four times the width specified by ELTYPE. 4x promotion has no effect on initial child element sizes of 32 or 64 bits. When PROMOTE is 111, corresponding to 8x promotion and zero extension, each child element is treated as a signed integer with a sign extension to a vector channel eight times the width specified by ELTYPE. 8x promotion has no effect on initial child element sizes of 16, 32, or 64 bits.

[0219] The VECLEN field 3004 defines the stream vector length (in bytes) of the stream. The streaming engine 125 divides the stream into groups of elements of length VECLEN bytes. The encoding of the 3 bits of the VECLEN field 3004 is defined as shown in Table 12.

[0220] Table 12

[0221]

[0222] VECLEN cannot be less than the product of the element size (in bytes) and the number of copies. As shown in Table 12, the maximum VECLEN of 64 bytes is equal to the preferred vector size of B 116 on the vector data path side. When VECLEN is shorter than the native vector width of the processing unit core 110, the streaming engine 125 fills in the additional channels provided to the vector to the processing unit core 110. The GRDUP field 3006 determines the filling type. The VECLEN field 3004 interacts with the ELDUP field 3005 and the GRDUP field 3006 as detailed below.

[0223] The ELDUP field 3005 specifies the number of times each element is copied. The element size multiplied by the number of element copies cannot exceed 64 bytes. The 3-bit encoding of the ELDUP field 3005 is defined as shown in Table 13.

[0224] Table 13

[0225]

[0226] The ELDUP field 3005 interacts with the VECLEN field 3004 and the GRDUP field 3006 as detailed below. The relationship between the permitted element size, the number of element copies, and the destination vector length requires that copied elements overflowing the first destination register fill an integer number of destination registers after copying is complete. Data in the additional destination registers is ultimately supplied to the corresponding stream header registers 2818 / 2828. After the first data element is copied, the next data element is rotated down to the least significant bit of the source register 3100 that discarded the first data element. The process is then repeated for the new data element.

[0227] GRDUP bit 3006 determines whether group replication is enabled. If GRDUP bit 3006 is 0, group replication is disabled. If GRDUP bit 3006 is 1, group replication is enabled. When enabled by GRDUP bit 3006, streaming engine 125 replicates a group element to fill the vector width. VECLEN field 3004 defines the length of the group to be replicated. When VECLEN field 3004 is less than the vector length of processing unit core 110 and GRDUP bit 3006 enables group replication, streaming engine 125 fills additional channels with an additional copy of the stream vector (see...). Figure 21 and 22 Because the stream vector length and the vector length of the processing unit core 110 are integer powers of 2, group replication produces an integer number of copies. Note that GRDUP and VECLEN do not specify the number of copies. The number of copies performed is based on the ratio of VECLEN to the original vector length, which in this instance is 64 bytes / 512 bits.

[0228] The GRDUP field 3006 specifies how the streaming engine 125 pads the bit stream vector to the vector length of the processing unit core 110 after the VECLEN length. When the GRDUP bit 3006 is 0, the streaming engine 125 fills the extra channels with zeros and marks the extra vector channels as invalid. When the GRDUP bit 3006 is 1, the streaming engine 125 fills the extra channels with a copy of the element group in each stream vector. Setting the GRDUP bit 3006 to 1 is invalid when VECLEN is set to the native vector width of the processing unit core 110. VECLEN must be at least as large as the product of ELEM_BYTES and the element copy number ELDUP. Therefore, an element or a number of copies of an element cannot be separated using VECLEN.

[0229] Group copy operation up to the destination vector size. Group copy does not change the data supplied when the product of the element size ELEM_BYTES and the number of element copies ELDUP is equal to or greater than the destination vector width. Under such conditions, the state of GRDUP bit 3006 and VECLEN field 3004 has no effect on the supplied data.

[0230] The following set of examples illustrates the interaction between VECLEN and GRDUP. Each example demonstrates how the streaming engine maps streams to vectors across different stream vector lengths and sizes on the B116 vector data path side. This example's stream contains 29 elements (E0 to E28) of 64 bits / 8 bytes. The stream can be a linear stream of 29 elements or an inner loop of 29 elements. The table illustrates an 8-byte channel, for example... Figure 21 The vectors described are then stored in the corresponding stream head registers 2818 / 2828.

[0231] Table 14 illustrates how instance streams are mapped to bits within a 64-byte processor vector when VECLEN is 64 bytes.

[0232] Table 14

[0233]

[0234] As shown in Table 14, the flow expands across 4 vectors. As previously described, channels within the 4 vectors that expand beyond the flow are zero-padded. When the VECLEN has a size equal to the original vector length, the GRDUP value is irrelevant because no copying occurs with this VECLEN.

[0235] Table 15 shows the same parameters as those shown in Table 14, except that VECLEN has 32 bytes. Group replication is disabled (GRDUP=0).

[0236] Table 15

[0237]

[0238] The 29 elements of the stream are distributed across channels 0 to 3 in eight vectors. The additional channels 4 to 7 in vectors 1 to 7 are zero-filled. In vector 8, channel 1 has the stream element (E28) and the other channels are zero-filled.

[0239] Table 16 shows the same parameters as those shown in Table 14, except that VECLEN has 16 bytes. Group replication is disabled (GRDUP=0).

[0240] Table 16

[0241]

[0242] The 29 elements of the stream are distributed across channels 0 and 1 in 15 vectors. The additional channels 2 to 7 in vectors 1 to 14 are zero-filled. In vector 15, channel 1 has the stream element (E28) and the other channels are zero-filled.

[0243] Table 17 shows the same parameters as those shown in Table 14, except that VECLEN has 8 bytes. Group replication is disabled (GRDUP=0).

[0244] Table 17

[0245]

[0246] The 29 elements of the stream appear in channel 0 of the 29 vectors. The additional channels 1 to 7 of the vectors 1 to 29 are filled with zeros.

[0247] Table 18 shows the same parameters as those shown in Table 15, except that VECLEN has 32 bytes and group replication is enabled (GRDUP=1).

[0248] Table 18

[0249]

[0250] The 29 elements of the stream are distributed across channels 0 through 7 in 8 vectors. Each vector 1 through 7 contains 4 copied elements. The reason for the number of copies (2) is that VECLEN (32 bytes) is half the length of the original vector of 64 bytes. In vector 8, channel 0 has the stream element (E28) and channels 1 through 3 are zero-padded. Channels 4 through 7 of vector 8 replicate this pattern.

[0251] Table 19 shows the same parameters as those shown in Table 16, except that VECLEN has 16 bytes. Group replication is enabled (GRDUP=1).

[0252] Table 19

[0253]

[0254] The 29 elements of the stream are distributed across channels 0 through 7 in 15 vectors. Each vector 1 through 7 contains two elements copied four times. The reason for the number of copies (4) is that VECLEN (16 bytes) is 1 / 4 of the native vector length of 64 bytes. In vector 15, channel 0 has the stream element (E28) and channel 1 is zero-padded. This pattern is repeated in channels 2 and 3, channels 4 and 5, and channels 6 and 7 of vector 15.

[0255] Table 20 shows the same parameters as those shown in Table 17, except that VECLEN has 8 bytes. Group replication is enabled (GRDUP=1).

[0256] Table 20

[0257]

[0258] All 29 elements of the stream appear in channels 0 through 7 of the 29 vectors. Each vector contains one element copied eight times. The reason for the number of copies (8) is that VECLEN (8 bytes) is 1 / 8 of the original vector length of 64 bytes. Therefore, each channel is the same in vectors 1 through 29.

[0259] Figure 31 The description includes Figure 19 Vector length masking / group copying block 2025 within formatter block 1903 (see Formatter Block 1903) Figure 20 An example of ). Input register 3100 from Figure 20 The element copy block 2024 shown receives vector input. Input register 3100 contains 64 bytes arranged in 64 one-byte blocks, byte0 to byte63. Note that the length of each byte from byte0 to byte63 is equal to the minimum ELEM_BYTES. A set of multiplexers 3101 to 3163 couples the input bytes from source register 3100 to output register 3170. Each corresponding multiplexer 3101 to 3163 feeds the input to the corresponding byte1 to byte63 of output register 3170. Not all input bytes byte0 to byte63 of input register 3100 are coupled to each multiplexer 3101 to 3163. Note that no multiplexer feeds byte0 of output register 3170. In this example, byte0 of output register 3170 is fed by byte0 of input register 3100.

[0260] Multiplexers 3101 to 3163 are controlled by multiplexer control encoder 3180. Multiplexer control encoder 3180 receives the ELEM_BYTES, VECLEN, and GRDUP input signals and generates corresponding control signals for multiplexers 3101 to 3163. ELEM_BYTES and ELDUP are supplied to multiplexer control encoder 3180 to check and ensure that VECLEN is at least as large as the product of ELEM_BYTES and ELDUP. In operation, multiplexer control encoder 3180 controls multiplexers 3101 to 3163 to transfer the least significant bit of VECLEN from input register 3100 to output register 3170. If GRDUP=0 indicates that group copying is disabled, then multiplexer control encoder 3180 controls the remaining multiplexers 3101 to 3163 to transfer zeros to all bits of the remaining most significant bit in output register 3170. If GRDUP=1 indicates that group copying is enabled, then the multiplexer control encoder 3180 controls the remaining multiplexers 3101 to 3163 to copy the number of least significant VECLEN bits from the input register 3100 to the most significant channel of the output register 3170. This control is similar to the element copying control described above and fills the output register 3170 with the first vector. For the next vector, the data in the input register 3100 is rotated down by VECLEN, thus discarding the least significant bits of the previous VECLEN. Formatter 1903 ( Figure 19 The data movement rate in the processing unit core 110 is achieved through the processing unit core 110. Figure 1 The rate at which data is consumed is set via the stream read and advance instructions described below. Group copy formatting repeats whenever the stream contains additional data elements.

[0261] Element copy (ELDUP) and group copy (GRDUP) are independent. Note that these features have independent specifications and parameter settings. Therefore, element copy and group copy can be used together or separately. Due to these specifications, element copy allows overflow to the next vector, while group copy does not.

[0262] Refer again Figure 30 The DECIM field 3007 controls the extraction of elements from the corresponding stream. The streaming engine 125 removes data elements from the stream after storing them in the corresponding stream header registers 2818 / 2828 to present them to the requesting functional unit. Extraction removes all data elements, not just child elements. The DECIM field 3007 is defined as listed in Table 21.

[0263] Table 21

[0264]

[0265] If DECIM field 3007 equals 00, no decimation occurs. The data element is passed unchanged to the corresponding stream head register 2818 / 2828. If DECIM field 3007 equals 01, a 2:1 decimation occurs. The streaming engine 125 removes odd-numbered elements from the data stream after storing them in stream head registers 2818 / 2828. Restrictions in the formatted network require 2:1 decimation when the data is boosted at least twice (PROMOTE cannot be 000), ICNT0 must be a multiple of 2, and the total vector length (VECLEN) must be large enough to maintain a single boosted copy element. For transposed streams (TRAMSPOSE ≠ 0), the transpose granules must be at least twice the element size (in bytes) before boosting. If DECIM field 3007 equals 10, a 4:1 decimation occurs. The streaming engine 125 retains each fourth data element, thus removing three elements from the data stream after they are stored in the stream head registers 2818 / 2828. Constraints in the formatted network require a 4:1 decimation when the data is boosted at least 4 times (PROMOTE cannot be 000, 001, or 101), ICNT0 must be a multiple of 4, and the total vector length (VECLEN) must be large enough to maintain a single boosted copy element. For transposed streams (TRANSPOSE ≠ 0), in one instance, columns are decimated but rows are not. Therefore, in such cases, the transposed particles must be at least twice the element size (in bytes) before boosting for 2:1 decimation (GRANULE ≥ 2 × ELEM_BYTES) and at least four times the element size (in bytes) before boosting for 4:1 decimation (GRANULE ≥ 4 × ELEM_BYTES).

[0266] The THROTTLE field 3008 controls the urgency with which the streaming engine acquires data before processing unit core 110. The two bits of this field are encoded as follows: Figure 22 The definition is shown in the image.

[0267] Table 22

[0268]

[0269] `THROTTLE` does not change the meaning of the stream and is only used as a hint. The streaming engine can ignore this field. Programs should not rely on specified throttling behavior to guarantee program correctness, as the architecture does not specify precise throttling behavior. `THROTTLE` allows programmers to provide hints to the hardware about program behavior. By default, the streaming engine attempts to hide as much latency as possible before processing unit core 110 (equivalent to `THROTTLE = 11`) while providing full stream throughput to processing unit core 110. While some applications require this level of throughput, this throughput can lead to undesirable system-level behavior in other applications. For example, the streaming engine discards all fetched data across context switches. Therefore, in systems with a large number of context switches, aggressively fetching in advance can result in wasted bandwidth.

[0270] The DIMFMT field 3009 defines which of the cycle dimension fields DIM1 2911, DIM2 2912, DIM3 2913, DIM4 2914, and DIM5 2915 and the addressing mode fields AM0 3013, AM1 3014, AM2 3015, AM3 3016, AM4 3017, and AM5 3018 (part of the FLAGS field 2921) of the DIMFMT field 3009 are valid for a specific stream. Table 23 lists the valid cycles for each value of the DIMFMT field 3009. Each valid cycle count must be at least 1, and the outer valid cycle count must be greater than 1.

[0271] Table 23

[0272]

[0273] DIR bit 3010 determines the fetch direction of the inner loop (loop 0). If DIR bit 3010 is 0, loop 0 fetches in the forward direction toward the incrementing address. If DIR bit 3010 is 1, loop 0 fetches in the backward direction toward the decrementing address. The fetch direction of other loops is determined by the signs of the corresponding loop dimensions DIM1, DIM2, DIM3, DIM4, and DIM5.

[0274] CBK0 field 3011 and CBK1 field 3012 control the circular block size after circular addressing is selected. The method for determining the circular block size is described in this document.

[0275] Fields AM0 (3013), AM1 (3014), AM2 (3015), AM3 (3016), AM4 (3017), and AM5 (3018) control the addressing mode of the corresponding cycle, thus allowing the addressing mode to be independently specified for each cycle. Each of fields AM0 (3013), AM1 (3014), AM2 (3015), AM3 (3016), AM4 (3017), and AM5 (3018) is 3 bits and is decoded as listed in Table 24.

[0276] Table 24

[0277]

[0278] In linear addressing, the address advances based on whether address arithmetic is forward or backward. In circular addressing, the address remains within a defined address block. After reaching the end of the circular address block, the address wraps back to the beginning of the block. Circular addressing blocks are limited to 2 bytes. N A number of addresses, where N is an integer. Circular address arithmetic is performed by cutting the carry chain between bits and disallowing changes to the selected number of most significant bits. Therefore, arithmetic beyond the end of the circular block only changes the least significant bits. The block size is set as listed in Table 25.

[0279] Table 25

[0280]

[0281] In this example, the circular block size is set by the number encoded by CBK0 (first circular address mode 01) or by the number encoded by CBK0+CBK1+1 (second circular address mode 10). For example, in the first circular address mode, the circular address block size can range from 512 bytes to 16M bytes. For the second circular address mode, the circular address block size can range from 1K bytes to 64G bytes. Therefore, the encoded block size is 2. (B+9) The number of bytes is B, where B is the number of encoded blocks, which is CBK0 for the first block size (AMx of 01) and CBK0+CBK1+1 for the second block size (AMx of 10).

[0282] Processing unit 110 ( Figure 1 The streaming engine 125 () uses a small number of instructions and dedicated registers to... Figure 28The stream is exposed to the program. The program begins and ends the stream with SEOPEN and SECLOSE. SEOPEN opens a new stream and the stream remains open until explicitly terminated by SECLOSE or replaced by a new stream with SEOPEN. The SEOPEN instruction specifies the stream number that indicates stream 0 or stream 1 to be opened. The SEOPEN instruction specifies the data register that stores the starting address of the stream. The SEOPEN instruction also specifies the stream template register that stores the stream template, as described above. The arguments of the SEOPEN instruction are listed in Table 26.

[0283] Table 26

[0284]

[0285] In this example, the stream start address register is a general-purpose scalar register file 211 ( Figure 2 The SEOPEN instruction can be encoded via instance instruction 1300 (in the registers). Figure 13 )scr1 field 1305 ( Figure 13 The SEOPEN instruction specifies the stream start address register. The SEOPEN instruction specifies stream 0 or stream 1 in the opcode. In this example, the stream template register is a vector register in the general-purpose vector register file 221. The SEOPEN instruction can be accessed via the scr2 / cst field 1304 ( Figure 13 Specify the stream template register. If the specified stream is valid, the SEOPEN instruction closes the previous stream and replaces it with the specified stream.

[0286] SECLOSE explicitly marks a stream as invalid, thus flushing any salient activity. Further reference: stream triggering exceptions. SECLOSE also allows the program to prematurely terminate one or two streams.

[0287] The SESAVE command saves the state of a specified stream by capturing enough state information to restart the stream in the future. The SERSTR command restores a previously saved stream. The SESAVE command saves stream metadata but not any stream data. A stream responds to the SERSTR command by re-acquiring its stream data.

[0288] Each stream can be in one of three states after a reset: invalid, valid, or frozen. Two streams begin in the invalid state. Opening a stream moves it to the valid state. Closing a stream returns it to the invalid state. Without interruptions or exceptions, streams typically do not undergo any other state transitions. To account for interruptions, the streaming engine adds a third state: frozen. The frozen state indicates a valid stream that has been interrupted.

[0289] In this example, four bits (two bits per stream) define the state of the two streams. One bit per stream resides within the streaming engine, and the other resides within processor core 110. The streaming engine internally tracks whether each stream maintains the parameter set associated with the valid stream. This bit distinguishes between invalid and valid streams. Processor core 110 uses dedicated bits in the Task Status Register (TSR) to track the state of each stream individually: TSR.SE0 for stream 0 and TSR.SE1 for stream 1. These bits distinguish between valid and invalid streams.

[0290] Opening a stream moves it to a valid state. Closing a stream moves it to an invalid state. If a program opens a new stream on a frozen stream, the new stream replaces the old stream, and the streaming engine discards the contents of the previous stream. The streaming engine supports opening new streams on currently valid streams. The streaming engine discards the contents of the previous stream, flushes the pipeline, and begins fetching data from the newly opened stream. Once the data is returned, the data is asserted to the processor. If a program closes a stream that has already been closed, nothing happens. If a program closes an open or frozen stream, the streaming engine discards all state associated with the stream, clears the internal stream valid bits, and clears the counter, tag, and address registers. Closing a stream serves two purposes. Closing a valid stream allows the program to explicitly declare the stream and no longer need the resources associated with it. Closing a frozen stream also allows context switching codes to clear the state of the frozen stream, making it invisible to other tasks.

[0291] As noted above, some data within stream holding registers 2818 or 2828 may be invalid. This can occur at the end of an inner loop when the number of stream elements is less than the size of the corresponding stream holding register 2818 / 2828, or at the end of an inner loop when the number of remaining stream elements is less than the number of channels defined by VECLEN. Outside of the inner loop's end time, if VECLEN is less than the width of stream holding registers 2818 / 2828 and GRDUP is disabled, then channels in stream holding registers 2818 / 2828 that exceed VECLEN are invalid.

[0292] Refer again Figure 28In this example, the streaming engine 125 further includes valid registers 2819 and 2829. Valid register 2819 indicates a valid channel in the stream header register 2818. Valid register 2829 indicates a valid channel in the stream header register 2828. Each valid register 2819 / 2829 contains one bit for each minimum ELEM_BYTES channel within the corresponding stream header register 2818 / 2828. In this example, the minimum ELEM_BYTES is one byte. The preferred data path width of the processor 100 and the data length of the stream header registers 2818 / 2828 are 64 bytes (512 bits). Therefore, the valid registers 2819 / 2829 have a data width of 64 bits. Each bit in the valid registers 2819 / 2829 indicates whether the corresponding byte in the stream header registers 2818 / 2828 is valid. In this example, 0 indicates that the corresponding byte in the stream header register is invalid, and 1 indicates that the corresponding byte is valid.

[0293] In this example, after reading the corresponding data from the stream header registers 2818 / 2828 and transferring the data to the requesting function unit, invalid / valid data in the corresponding valid registers 2819 / 2829 is automatically transferred to the assertion register file 234 corresponding to the specific stream. Figure 2 The data registers within the assertion register. In this example, the valid data for stream 0 is stored in assertion register P0, and the valid data for stream 1 is stored in assertion register P1.

[0294] The valid data stored in assertion register 234 can be used in various ways. The functional unit can combine vector stream data with another set of vectors and then use the valid data indicator as a mask to store the combined data in memory, thus allowing the same process to be used at the end of loop data, as in the case where all channels are valid, thereby avoiding the storage of invalid data. The valid indicator stored in assertion register file 234 can be used as a mask or operand in other processes. P unit 246 ( Figure 2 It can have an instruction (BITCNT) that counts the number of 1s in the assertion register, which can be used to determine the count of valid data elements from the assertion register.

[0295] Figure 32 This description is used to generate the data stored in valid register 2819 ( Figure 28 Example hardware 3200 with valid / invalid indication in ) Figure 32 This describes the hardware used for stream 0; stream 1 contains the corresponding hardware. Hardware 3200 operates whenever data is in stream header register 2818 ( Figure 28A valid word is generated during the update. The first input ELTYPE is supplied to the decoder 3201. The decoder 3201 produces an output TOTAL ELEMENT SIZE corresponding to the minimum data size based on the element size ELEM_BYTES and whether the element is real or complex. The meaning of the various encodings of ELTYPE is shown in Table 9. Table 27 shows example outputs of the decoder 3201 for various ELTYPE encodings (in bytes). Note that Table 9 lists bits and Table 27 lists bytes. As shown in Table 27, TOTAL ELEMENT SIZE is 1, 2, 4, or 8 bytes when the element is real and 2, 4, 8, or 16 bytes when the element is complex.

[0296] Table 27

[0297]

[0298] The second input, PROMOTE, is supplied to decoder 3202. Decoder 3202 produces an output boost factor corresponding to the PROMOTE input. The meaning of the various PROMOTE encodings is shown in Table 28, which displays example outputs (in bytes) of decoder 3202 for various PROMOTE encodings. Differences in extension types (zero extension or sign extension) are irrelevant to decoder 3202.

[0299] Table 28

[0300]

[0301] The outputs of decoders 3201 and 3202 are supplied to multiplier 3203. The product generated by multiplier 3203 corresponds to the channel size and boost factor. Since the boost factor is an integer power of 2 (2^3 + ... N Therefore, multiplication can be achieved by shifting the corresponding TOTAL ELEMENT SIZE, for example, no shift for boosting factor 1, 1 shift for boosting factor 2, 2 shifts for boosting factor 4, and 3 shifts for boosting factor 8.

[0302] The NUMBER OF LANES unit 3204 receives the vector length VECLEN and LANE SIZE and generates NUMBER OF LANES. Table 29 shows an example of decoding the number of channels and the vector length VECLEN for each channel size (in bytes).

[0303] Table 29

[0304]

[0305] As previously stated, VECLEN must be greater than or equal to the product of the element size and the number of copies. As shown in Table 29, VECLEN must also be greater than or equal to the product of the element size and the promotion factor. This means that VECLEN must be large enough to ensure that the element does not overlap with the type-promoted block 2022 (...). Figure 20 The resulting extended separation. The cells marked with "-" below the diagonal in Table 29 indicate unapproved parameter combinations.

[0306] The NUMBER OF LANES output of unit 3204 serves as one input to the LANE / REMAINING ELEMENTS CONTROL WORD unit 3211. A second input comes from multiplexer 3212. Multiplexer 3212 receives a loop 0 input and a loop 1 input. The loop 0 and loop 1 inputs represent the number of remaining elements in the current iteration of the corresponding loop.

[0307] Figure 33 illustrate Figure 28 The diagram shows a partial schematic of the address generator 2811. The address generator 2811 forms the address used to obtain the next element in the defined stream of the corresponding streaming engine. The start address register 3301 stores the start address of the data stream. As described above, in this example, the start address register 3301 is a scalar register in the global scalar register file 211 specified by the SEOPEN instruction that opens the corresponding stream. The start address can be copied from the specified scalar register and stored locally at the corresponding address generator 2811 / 2821 via control logic contained within the address generator 2811. The first loop of the stream uses a loop 0 counter register 3311, an adder 3312, a multiplier 3313, and a comparator 3314. The loop 0 counter register 3311 stores a working copy of the iteration count of the first loop (loop 0). For each iteration of loop 0, the adder 3312 increments the loop count by 1 when triggered by the next address signal, and the count is stored back in the loop 0 counter register 3311. Multiplier 3313 multiplies the current loop count by the quantity ELEM_BYTES. ELEM_BYTES is the size (in bytes) of each data element in loop 0. Loop 0 iterates through physically contiguous data elements in memory with an iteration step size ELEM_BYTES.

[0308] Comparator 3314 compares the count stored in loop 0 counter register 3311 (after being incremented by adder 3313) with the count from the corresponding stream template register 2900. Figure 29 ICNT0 2901 ( Figure 29The value of adder 3312 equals the value of ICNT0 2901 in stream template register 2900. The iteration of loop 0 is complete when the output of adder 3312 equals the value of ICNT0 2901 in stream template register 2900. Comparator 3314 generates a valid loop 0 end signal. Loop 0 counter register 3311 is reset to 0 and the iteration of the next higher loop (loop 1 in this case) is triggered.

[0309] The circuits with higher cycles (cycle 1, cycle 2, cycle 3, cycle 4, and cycle 5) are similar to Figure 33 The circuit described in the diagram. Each cycle includes a corresponding working cycle counter register, an adder, a multiplier, and a comparator. The adder for each cycle is triggered by the cycle end signal of the previous cycle. The second input of each multiplier is from the corresponding dimensions DIM1, DIM2, DIM3, DIM4, and DIM5 of the corresponding stream template. The comparator for each cycle compares the working cycle register count with the corresponding iteration values ​​ICNT1, ICNT2, ICNT3, ICNT4, and ICNT5 of the corresponding stream template register 2900. The cycle end signal generates the iteration for the next higher-order cycle. The cycle end signal from cycle 5 terminates the stream.

[0310] Figure 33 The generation of the loop 0 count is also explained. The loop 0 count is equal to the updated data stored in the corresponding working count register 3311. The loop 0 count is updated based on each change in the working loop 0 count register 3311. The loop counts for higher loops (loop 1, loop 2, loop 3, loop 4, and loop 5) are generated similarly.

[0311] Figure 33 The generation of the loop 0 address is also explained. The loop 0 address is equal to the data output from multiplier 3313. The loop 0 address is updated based on each change in the working loop 0 counter register 3311. Similar circuitry for loops 1, 2, 3, 4, and 5 generates the corresponding loop addresses. In this example, the loop 0 counter register 3311 and the other loop counter registers are implemented as incrementing counter registers. In another example, the initialization and comparison operations are decrementing counter circuitry.

[0312] Refer again Figure 32 For example, the cyclic decrementing count value of loop 0 / is given by expression (2).

[0313] (2)

[0314] Therefore, the cyclic decrement count is the initial iteration count specified in the stream template register and, as shown below... Figure 33 The text explains the difference between the cyclically incrementing counts generated in that way.

[0315] LANE / REMAINING ELEMENTS CONTROL WORD Unit 3211 ( Figure 32 The control word 3213 is generated based on the number of channels from the NUMBER OFLANES unit 3204 and the cyclic decrementing count selected by the multiplexer 3212. The control input to the multiplexer 3212 is from... Figure 30 The TRANSPOSE signal in field 3002. If TRANSPOSE is disabled (“000”), then multiplexer 3212 selects loop 0 decrementing count Loop0 / . For all other valid values ​​of TRANSPOSE (“001”, “010”, “011”, “100”, “101”, and “110”), multiplexer 3212 selects loop 1 decrementing count Loop1 / . The streaming engine maps the innermost dimension to consecutive channels in the vector. For a normal stream, it is Loop0. For a transposed stream, it is Loop1 because transposition swaps two dimensions.

[0316] The LANE / REMAINING ELEMENTS CONTROL WORD unit 3211 generates control word 3213 as follows. Control word 3213 has a number of bits equal to the number of channels from unit 3204. If the remaining element count of the selected loop is greater than or equal to the number of channels, then all channels are active. In this case, control word 3213 is all 1s to indicate that all channels within the vector length VECLEN are active. If the remaining element count of the selected loop is non-zero and less than the number of channels, then some channels are active and some channels are inactive. Based on the above... Figure 21 and 22 The described channel allocation assigns stream elements starting with the least significant channel. In these cases, control word 3213 contains a number of least significant bits set to 1 equal to the number of selected cyclic decrement counts. All other bits of control word 3213 are set to zero. Figure 32 In the example described, the number of channels is equal to 8 and there are 5 significant (1) least significant bits after 3 invalid (0) most significant bits, which corresponds to a loop with 5 elements remaining in the final iteration.

[0317] The control word extension unit 3214 is a magnitude extension of the control word 3213 based on the LANE SIZE. The extended control word contains one bit for each minimum channel. In this example, the minimum stream element size, and therefore the minimum channel size, is one byte (8 bits). In this example, the size of holding registers 2818 / 2828 is equal to the vector size of 64 bytes (512 bits). Therefore, the extended control word has 64 bits, with one bit for each byte of the stream holding registers 2818 / 2828. This extended control word fills the corresponding valid registers 2819 and 2829 (…). Figure 28 The least significant bit of ).

[0318] The description is complete when VECLEN equals the vector length. The extended control word contains bits for all positions within the corresponding valid registers 2819 / 2829. When VECLEN is not equal to the vector length, some additional considerations apply. When VECLEN is not equal to the vector length, the extended control word does not have enough bits to fill the corresponding valid registers 2819 / 2829. For example... Figure 32 The description states that the least significant bit of the corresponding valid registers 2819 / 2829 is filled with an extended control word, thereby providing valid / invalid bits for channels within the VECLEN width. An additional mechanism is provided for channels exceeding the VECLEN width up to the data width of the stream header register 2818.

[0319] Still referencing Figure 32 This describes how multiplexer 3215 and group copy unit 3216 provide the necessary additional valid / invalid bits. Referring to the description of VECLEN, if group copy is not enabled (GRDUP=0), the additional channels are invalid. The first input of multiplexer 3215 is an INVALID 0 signal containing a number of bits equal to VECLEN. Multiplexer 3215 selects this input when GRDUP=0. Group copy unit 3216 copies this input to all additional channels of stream head register 2818. Therefore, the most significant bit of valid register 2819 is set to zero, indicating that the corresponding byte in stream head register 2818 is invalid. This occurs in vectors 1 to 8 of the examples shown in Table 15, vectors 1 to 15 of the examples shown in Table 16, and vectors 1 to 29 of the examples shown in Table 17.

[0320] In another example, the multiplexer 3215 and the group copy block 3216 use a similar... Figure 31 The group replication logic described in the documentation is replaced with the group replication logic in 2025.

[0321] As previously described, if group replication is enabled (GRDUP=1), then the stream head register 2818 ( Figure 28The additional channels of the multiplexer 3215 are filled with copies of the least significant bits. The second input of the multiplexer 3215 is the expanded control word from the control word expansion unit 3214. The multiplexer 3215 selects this input when GRDUP=1. The group copy unit 3216 copies this input to all the additional channels of the stream head register 2818.

[0322] There are two possible outcomes. In one outcome, in most cases, all channels within the VECLEN are valid and all bits from the control word extension unit 3214 are 1. This occurs in vectors 1 to 7 of the group replication examples shown in Table 18 and vectors 1 to 14 of the group replication examples shown in Table 19. Under these conditions, all bits of the extended control word from the control word extension unit 3214 are 1 and all channels of the stream header register 2818 are valid. Therefore, the group replication unit 3216 fills all additional channels with 1s. In another outcome, the number of remaining stream data elements is less than the number of channels within the VECLEN. This occurs in vector 8 of the group replication examples shown in Table 18 and vector 15 of the group replication examples shown in Table 19. Under these conditions, some channels within the VECLEN are valid and some are invalid. The group replication unit 3216 fills the additional channels with bits having the same pattern as the extended control word bits. In either case, the additional channels are filled with bits corresponding to the extended control bits.

[0323] Still referencing Figure 32 This indicates the boundary 3217 between the least significant bit and the most significant bit. The location of this boundary is set by the size of VECLEN relative to the size of the stream head register 2818.

[0324] Figure 34 This is a partial schematic diagram 3400 illustrating the encoding of the stream input operands described above. Figure 34 This describes the instruction decoder 113 that decodes the src1 field 1305 of an instruction to control the corresponding src1 input of the functional unit 3420 (see [link]). Figure 1 This is part of the instruction set src2 / cst field 1304 used to control the function unit 3420. Additionally, this circuitry is copied for each instruction within an execution packet that can use stream data as operands, and is dispatched simultaneously.

[0325] The instruction decoder 113 receives bits 13 to 17 of the src1 field 1305 of the instruction. The opcode field (bits 3 to 12 for all instructions and bits 28 to 31 for unconditional instructions) explicitly specifies the corresponding functional unit 3420 and the function to be executed. In this example, functional unit 3420 can be L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, or C unit 245. Figure 34The relevant part of the instruction decoder 113, as described, decodes the src1 bit field 1305. Sub-decoder 3411 determines whether the src1 bit field 1305 is within the range of 00000 to 01111. If so, sub-decoder 3411 supplies the corresponding register number to the global vector register file 231. In this example, the register number is the four least significant bits of the src1 bit field 1305. The global vector register file 231 retrieves the data corresponding to the register number stored in the registers and supplies the data to the src1 input of functional unit 3420.

[0326] Sub-decoder 3412 determines whether the src1 bit field 1305 is within the range of 10000 to 10111. If so, sub-decoder 3412 supplies the corresponding register number to the corresponding local vector register file. If the instruction is directed to L2 unit 241 or S2 unit 242, the corresponding local vector register file is local vector register file 232. If the instruction is directed to M2 unit 243, N2 unit 244, or C unit 245, the corresponding local vector register file is local vector register file 233. In this example, the register number is the three least significant bits of the src1 bit field 1305. The corresponding local vector register files 232 / 233 retrieve the data corresponding to the register number stored in the registers and supply the data to the src1 input of function unit 3420.

[0327] Sub-decoder 3413 determines whether the src1 bit field 1305 is 11100. If so, sub-decoder 3413 supplies the stream 0 read signal to the streaming engine 125. Then, the streaming engine 125 supplies the stream 0 data stored in the holding register 2818 to the src1 input of the functional unit 3420.

[0328] Sub-decoder 3414 determines whether the src1 bit field 1305 is 11101. If so, sub-decoder 3414 supplies a stream 0 read signal to streaming engine 125. Stream engine 125 then supplies the stream 0 data stored in holding register 2818 to the src1 input of functional unit 3420. Sub-decoder 3414 also supplies a push signal to stream 0. As previously described, streaming engine 125 pushes the data to store the next sequence vector of stream 0 data elements in holding register 2818.

[0329] The sub-decoder 3413 or sub-decoder 3414 supplies a stream 0 read signal to the streaming engine 125, triggering another data move. Following this stream 0 read signal, the streaming engine 125 supplies the data stored in the valid register 2819 to the assertion register file 234 for storage. In this example, this is a predetermined data register within the assertion register file 234. In this example, data register P0 corresponds to stream 0.

[0330] Sub-decoder 3415 determines whether the src1 bit field 1305 is 11110. If so, sub-decoder 3415 supplies the stream 1 read signal to the streaming engine 125. Then, the streaming engine 125 supplies the stream 1 data stored in the holding register 2828 to the src1 input of the functional unit 3420.

[0331] Sub-decoder 3416 determines whether the src1 bit field 1305 is 11111. If so, sub-decoder 3416 supplies a stream 1 read signal to streaming engine 125. Stream engine 125 then supplies the stream 1 data stored in holding register 2828 to the src1 input of functional unit 3420. Sub-decoder 3414 also supplies a push signal to stream 1. As previously described, streaming engine 125 pushes the data to the next sequence vector of stream 1 data elements and stores it in holding register 2828.

[0332] The sub-decoder 3415 or sub-decoder 3416 supplies a Stream 1 read signal to the streaming engine 125, triggering another data move. Following this Stream 1 read signal, the streaming engine 125 supplies the data stored in the valid register 2829 to the assertion register file 234 for storage. In this example, this is a predetermined data register within the assertion register file 234. In this example, data register P1 corresponds to Stream 1.

[0333] Similar circuitry is used to select the data supplied to the scr2 input of functional unit 3420 in response to the bit encoding of src2 / cst field 1304. The src2 input of functional unit 3420 can be supplied as a constant input in the manner described above. If instruction decoder 113 generates a read signal for stream 0 from scr1 field 1305 or scr2 / cst field 1304, then streaming engine 125 supplies the data stored in valid register 2819 to assert register P0 of assert register file 234 for storage. If instruction decoder 113 generates a read signal for stream 1 from scr1 field 1305 or scr2 / cst field 1304, then streaming engine 125 supplies the data stored in valid register 2829 to assert register P1 of assert register file 234 for storage.

[0334] The exact number of instruction bits dedicated to operand specifications, as well as the number of data registers and streams, are design choices. Specifically, specifying a single global vector register file and omitting local vector register files is feasible. This example uses bit encoding of the input operand select field to specify stream reading and another bit encoding to specify stream reading and advance the stream.

[0335] Figure 34 The process described herein automatically transfers valid data to the assertion register file 234 each time the stream data is read. The transferred valid data can then be used by unit P 246 for further metadata calculations. The transferred valid data can also be used as an operand for a mask or other operations by one or more of the vector data path side B 116 functional units, including units L2 241, S2 242, M2 243, N2 244, and C 245. Numerous feasible complex logic operations exist that utilize this stream valid data.

[0336] Figure 35 This is a partial schematic diagram 3500 illustrating another instance of configuration for selecting operand sources. In this instance, the corresponding stream valid registers 2819 / 2829 do not need to be automatically loaded into predetermined registers in the assertion register file 234. Instead, explicit instructions in P unit 246 are used to move data. Figure 35 This describes the instruction decoder 113 that decodes the src1 field 1305 of an instruction to control the corresponding src1 input of P unit 246 (see [link]). Figure 1 This is part of the instruction used to control P unit 246, specifically the src2 / cst field 1304. Figure 13 ).

[0337] The instruction decoder 113 receives bits 13 to 17 of the src1 field 1305 of the instruction. The opcode field (bits 3 to 12 for all instructions and bits 28 to 31 for unconditional instructions) explicitly specifies the P unit 246 and the function to be executed. Figure 35 The relevant part of the instruction decoder 113, as described, decodes the src1 bit field 1305. The sub-decoder 3511 determines whether the src1 bit field 1305 is within the range 00000 to 01111. If so, the sub-decoder 3511 supplies the corresponding register number to the global vector register file 231. In this example, the register number is the four least significant bits of the src1 bit field 1305. The global vector register file 231 retrieves the data corresponding to the register number stored in the registers and supplies the data to the src1 input of the P unit 246.

[0338] Sub-decoder 3512 determines whether the src1 bit field 1305 is within the range 10000 to 10111. If so, sub-decoder 3512 supplies the decoded register number to assert register file 234. In this example, the register number is the three least significant bits of the src1 bit field 1305. Assertion register file 234 retrieves the data corresponding to the register number stored in the register and supplies the data to the src1 input of assertion unit 246.

[0339] Sub-decoder 3513 determines whether the src1 bit field 1305 is 11100. If so, sub-decoder 3513 supplies the stream 0 valid read signal to the streaming engine 125. Then, the streaming engine 125 supplies the valid data stored in the valid register 2819 to the src1 input of P unit 246.

[0340] Sub-decoder 3514 determines whether the src1 bit field 1305 is 11101. If so, sub-decoder 3514 supplies the stream 1 valid read signal to the streaming engine 125. Then, the streaming engine 125 supplies the valid stream 1 data stored in the valid register 2829 to the src1 input of P unit 246.

[0341] The P-unit 246 instruction that uses stream valid registers 2819 / 2829 as operands can be any of the previously described P-unit instructions, such as NEG, BITCNT, RMBD, DECIMATE, EXPAND, AND, NAND, OR, NOR, and XOR.

[0342] The special instructions mentioned above are limited to unit P246. Therefore, Figure 34 and 35 The operations outlined herein can be used together. If the functional unit specified by the instruction is L2 unit 241, S2 unit 242, M2 unit 243, N2 unit 244, or C unit 245, then the src1 field 1305 is as follows: Figure 34 The summary is as follows. If the functional unit specified by the instruction is unit P246, then the src1 field 1305 is as per [the relevant information]. Figure 35 The summary is interpreted as described above. Alternatively, the stream valid register is automatically saved to... Figure 34 The predefined assertion register described herein may be implemented in one instance and not in another.

[0343] Matrix multiplication accelerator

[0344] Figure 36This is a block diagram of a system 3600 including a processing unit core 110, a streaming engine 125, a system memory 130, and a matrix multiplication accelerator (MMA) 3640. The MMA 3640 is a tightly coupled matrix multiplication accelerator unit that is a third type of functional unit within the processing unit core 110. The other two are scalar data paths 115 (…). Figure 1 ) and vector data path 116 ( Figure 1 The MMA 3640 enables System 3600 to meet the requirements of large-scale image and data processing applications. The MMA 3640 supports the high computational performance requirements of matrix multiplication. With the support of the streaming engine 125, processing unit core 110, and L2 memory 130, the MMA3640 efficiently computes a large number of MACs (hybrids) required for various vision algorithms, dense linear algebra, FFT operations, and advanced applications including convolutional neural networks (CNNs), structure-of-motion (SFM) mapping, and radar, without increasing the memory bandwidth in the processing unit core 110.

[0345] Generally, the MMA 3640 supports matrix multiplication of two matrices. As shown in expression (3), when matrix A is an [n x m] matrix and matrix B is an [m x p] matrix, the matrix multiplication outputs matrix C.

[0346] * (3)

[0347] Each i,j element is given by multiplying element Aik (row i across A) by element Bkj (column j along B), where k = 1, 2, ..., m.

[0348] Figure 37 More detailed descriptions of instance matrices A, B, and the resulting matrix C. Each element of the result C... ij It is the sum of the products of the elements from the rows of matrix A and the elements from the columns of matrix B, as defined by expression (3).

[0349] Return to reference Figure 36 The MMA includes an A[.] buffer 3641 for holding matrix A, a B[.] buffer 3642 for holding matrix B, and a C[.] buffer 3643 for collecting the resulting elements of matrix C. The MMA 3640 includes an array of individual multipliers and a set of accumulators (as indicated at 3644) to allow computation of entire rows of matrix C within one execution loop of the MMA 3640. In this example, the MMA 3640 is configured to handle 32x32 16-bit fixed / floating-point matrix multiplication and produce a 32x32 16-bit product matrix in 32 loops.

[0350] In this example, the MMA 3640 is also equipped to multiply two 64x64 8-bit matrices by dividing each into four 32x32 submatrices, multiplying the combinations of the submatrices, and then combining the results to produce a final 64x64 8-bit matrix. Other examples can be implemented to support larger or smaller matrices with greater or less precision than 8 or 16 bits.

[0351] For the MMA 3640 to operate correctly on matrices smaller than its native size (such as the 32x32 instance mentioned above), all elements of each input matrix A and B must contain valid data. Unused elements should be set to zero or some other agreed-upon null value. Streaming engine 125 includes support for making unused elements of an array null during stream access without performing additional memory accesses, as described in more detail below. In this example, streaming engine 125 includes support for inserting zeros or selected values ​​(such as maximum or minimum values) during stream accesses without performing memory accesses to provide the selected values.

[0352] MMA's streaming engine supports

[0353] Still referencing Figure 36 The streaming engine 125 interfaces with the L2 memory 130 via a unified memory controller (UMC) 3601. The UMC 3601 coordinates access to the L2 memory 130 through various functional units within the system 3600. The data routing unit 3602 interfaces with the UMC 3601 to route dates to the system memory 130. Stream header registers 2818 / 2828 (… Figure 28 ) and valid registers 2819 / 2829 ( Figure 28 The output of the product matrix C[.] is provided to the streaming engine interface 3611 via the corresponding bus 2840 / 2841 and thus to the logic unit in the processing unit core 110. In this example, the S unit 242 can be programmed to pass one stream from SE 125 to the MMA A[.] buffer 3641 and another stream from SE 125 to the B[.] buffer 3642. The output of the product matrix C[.] can be directed to the L2 memory 130 via UMC 3601 or to the destination register in the register file 233 via the destination multiplexer shown in C unit 245.

[0354] Figure 38 yes Figure 28A more detailed block diagram of a portion of the streaming engine 125. Linear streaming is suitable for a large number of algorithms, but not all. For example, matrix multiplication presents a unique problem for streaming engines because each element in the matrix product is the result of a vector dot product between a row from the first matrix and a column from the second matrix. Programs can store matrices entirely in row-major or column-major order. Row-major order stores all elements of a single row contiguously in memory. C and C++ programs typically store arrays in row-major order. Column-major order stores all elements of a single column contiguously in memory. FORTRAN programs typically store arrays in column-major order. Depending on the programming language, matrices can be stored in the same order as the language's default array order.

[0355] Therefore, only one of the two matrices in matrix multiplication maps to the 2D stream definition of a streaming engine. This problem is not unique to streaming engines. In fact, the access pattern of matrix multiplication is not suitable for most general-purpose memory hierarchies. Some software libraries solve this problem by directly transposing one of the two matrices, allowing them to be accessed row-wise (or column-wise) during multiplication.

[0356] Transpose mode

[0357] Using a streaming engine, programs don't need to go to extremes. The streaming engine supports implicit matrix transposition, similar to the concept of transposed streams. Transposed streams avoid the cost of explicitly transforming data in memory. Instead of accessing data in strictly consecutive element order, the streaming engine efficiently swaps the two inner loop dimensions according to its traversal order, thus retrieving elements along the second dimension of the continuous vector channel.

[0358] The transpose mode swaps the two innermost loop levels. Therefore, in transpose mode, the two innermost loops, ICNT0 and ICNT1, are swapped. ICNT1 determines the number of rows in each column. Columns are defined as GRANULE sizes. ICNT0 is the second dimension in the transpose stream and defines the horizontal width (which may or may not be a multiple of the GRANULE). In this instance streaming engine, the maximum row height ICNT1 must be at least 1 and less than or equal to 16. There are no constraints on ICNT0 in the transpose. However, if ICNT0 is not a multiple of the GRANULE size, the streaming engine will pad with zeros in each missing element of the GRANULE.

[0359] The coarse rotator 2835 and data storage device 2816 of the flow engine 2810 are in Figure 38 The description continues; however, the coarse rotator 2836 of the Stream 1 engine 2820 is similar to and operates in a similar manner to the data storage device 2826. The transpose mode is provided by the Stream 0 engine 2810 ( Figure 28 The transpose mode is executed using the resources of the coarse rotator 2835, data storage unit 2816, and butterfly network 2817. The transpose mode is performed by the stream 1 engine 2820. Figure 28 Using coarse rotator 2836 ( Figure 28 ), data storage unit 2826 ( Figure 28 ) and butterfly network 2827 ( Figure 28 ) resource execution.

[0360] In this example, the data storage device 2816 is organized as a register file 3810 with 32 slots and 64 bytes (512 bits). Other examples can provide larger or smaller amounts of storage without changing the semantics of the stream. The data storage device 2816 is organized as eight independent memory banks, each 8 bytes (64 bits) wide. Each memory bank includes two write ports and two read ports. In this example, each memory bank also includes two bits / lines for parity protection.

[0361] In transpose mode, the SE organizes internal storage into sector fragments, and the number of sector fragments depends on the current vertical count (ICNT1) setting. This allows the SE to retrieve as many rows and columns as possible and organize and rotate the data returned from L2 into sectors. This allows the SE to use both read and write ports per memory bank when reading and writing data in transpose mode, so that data can be rotated and sorted according to its sectors.

[0362] In this example, the coarse rotator 2835 comprises a set of 16 multiplexers represented by multiplexers 3806, 3807, and 3808. For example, each multiplexer of multiplexer 3806 has 16 inputs, each 4 bytes (32 bits) wide, concatenated to receive all 512 bits provided on bus 3802 by L2 interface 2833. The 4-byte output of each multiplexer is coupled to provide data to half of the corresponding memory bank of register file 3810. Each memory bank of register file 3810 is coupled to receive data in parallel from two multiplexers (e.g., 3806, 3807), such that data received from the L2 interface via bus 3802 can be manipulated in 4-byte elements.

[0363] Reference queue 2815 receives storage allocations and tracks data from storage and allocation logic 2814. Figure 28 Metadata. Because each 512-bit row of data is transmitted from L2 via the L2 interface 2833 ( Figure 28Upon receiving the data, control logic 3804 generates control signals to independently control each of the 16-way multiplexers 3806, 3807, and 3808, such that any 4-byte data element from a received 512-bit data line can be stored on either side of each of the eight memory banks in the selected slot of register file 3810 based on metadata provided by reference queue 2815. Coarse rotator 2835 allows each 512-bit line of data to be rotated, shifted, truncated, or copied by stream 0 engine 2810, as described in more detail above. Furthermore, matrix transpose can be performed by stream 0 engine 2810 using coarse rotator 2835, as described in more detail below.

[0364] Alignment networks 3820 and 3821 are each similar to coarse rotator 2835. In this example, alignment network 3820 contains a set of 16 multiplexers represented by multiplexers 3822 and 3823. For example, each multiplexer of multiplexer 3822 has 32 inputs, each 4 bytes (32 bits) wide, concatenated to receive all 512 bits provided by each of the two read ports of register file 3810. For example, the 4-byte output of each multiplexer of multiplexer 3822 is coupled to provide data to the corresponding input of butterfly network 2817. In this way, multiplexers 3822 and 3823 can select 16 4-byte data elements from register file 3810 to form a 64-byte data line provided to butterfly network 2817.

[0365] Similarly, in this example, alignment network 3821 comprises a set of 16 multiplexers represented by multiplexers 3824 and 3825. For example, each multiplexer of multiplexer 3824 has 32 inputs, each 4 bytes (32 bits) wide, concatenated to receive all 512 bits provided by each of the two read ports of register file 3810. For example, the 4-byte output of each multiplexer of multiplexer 3824 is coupled to provide data to the corresponding input of butterfly network 2817. In this way, multiplexers 3824 and 3825 can select 16 4-byte data elements from register file 3810 to form a 64-byte data line provided to butterfly network 2817.

[0366] Control logic 3814 generates control signals to independently control each of the 16-way multiplexers 3822 and 3823 in alignment network 3820 and each of the 16-way multiplexers 3824 and 3825 in alignment network 3821, such that any 4-byte data element retrieved from register file 3810 can be aligned to any 4-byte position within two 64-byte output lines provided to butterfly network 2817 based on metadata provided by reference queue 2815.

[0367] The butterfly network 2817 is controlled by the streaming engine 2810 to further format the data retrieved from the data storage device 2816 before sending the formatted data to the processing unit core 110, such as regarding Figure 20 , Figure 28 More detailed description. The butterfly network 2817 contains multiple levels of cross-coupled multiplexer nodes that perform data formatting.

[0368] In this example, the control logic 3804 for the coarse rotator 2835 and the control logic 3814 for the data storage device 2816 are implemented as asynchronous Boolean logic capable of generating control signals for each of the multiplexer and register file 3810 in parallel based on the content of metadata provided by the reference queue 2815.

[0369] In transpose mode, SE0 2810 organizes internal storage device 2816 into sector blocks, and the number of sector blocks depends on what the current vertical count (ICNT1) is set to. This allows SE0 2810 to acquire as many rows and columns as possible and from L2 130 ( Figure 1 The returned data is organized and rotated into sectors of data storage device 2816. In this example, register file 3810 contains 32 lines x 64 bytes and is organized into 8 independent 4-port banks. This allows SE0 2810 to use both read and write ports per bank when reading and writing data in transposed mode, as the data is rotated and sorted according to its sectors.

[0370] The coarse rotator is controlled by logic 3804 based on reference queue 2815 ( Figure 28 , Figure 38 The metadata queue is controlled by the storage allocation tracking logic 2814. Figure 28 The flow parameters are derived from matrix 3700.

[0371] Insert the empty vector into the stream.

[0372] refer to Figure 38 Control logic 3814 can control each of the multiplexers on alignment networks 3820 and 3821 to force null values ​​onto selected elements of the stream vector based on metadata provided by reference queue 2815. In this way, the entire stream vector can be set to null without fetching any data from system memory. In one instance, null is "0". In another instance, a predefined value or pattern can be used to represent null.

[0373] In another instance, control logic 3814 can control each of the multiplexers on alignment networks 3820, 3821 to force a selected value onto a selected element of the stream vector based on metadata provided by reference queue 2815. In this way, the entire stream vector can be set to the selected value (e.g., minimum or maximum value) without fetching any data from system memory.

[0374] In this example, the address generator 2811 / 2821 (see...) Figure 28 The control logic in the process performs counting tracking for the six-level nested iteration. As will be described in more detail below, it can detect various conditions associated with one or more of the six-level nested iterations and is used to signal that a null value or selected value will be inserted into the vector stream (without needing to fetch the data of the null value or selected value vector from memory).

[0375] The metadata generated by the address generator is pushed to the stream reference queue 2815 / 2825 (see Figure 28 In the backend, when the processing unit core 110 ( Figure 1 When performing a read, this metadata is popped and sent to control logic 3814 in alignment network 2820 / 2821 via a channel specified as empty or set to the selected value, without reading data from system memory. The metadata keeps track of the decremented dimension (DECDIM) and dimension width (DECDIM_WIDTH) and is fed to the backend logic, as described in more detail below.

[0376] Data with Mining Support

[0377] For example, complex signal processing applications such as various vision algorithms, dense linear algebra, FFT operations, and advanced applications including convolutional neural networks (CNNs), structured motion reconstruction (SFM), and radar require digital operations on complex multidimensional data structures. The boundaries of these data structures need to be cleaned to prevent outlier data from being included in calculations performed at those boundaries. Typically, this requires the programmer to allocate memory space at the boundaries of the data structure, which can be preset to known values, such as zero, the maximum, or the minimum data value. Then, additional processing loops are used to access and set the boundary data values.

[0378] Example Streaming Engine 125 ( Figure 1The streaming engine 125 includes a DECDIM_WIDTH parameter that allows programmers to specify a decrement dimension (DECDIM_WIDTH) for one or more of the six nested loops supported by the streaming engine 125, which differs from the dimension defined by the corresponding loop dimension parameter. In this way, the streaming engine 125 can be programmed to insert appropriate known values ​​into the data stream as it is retrieved from system memory. Furthermore, the streaming engine 125 can be programmed to insert empty or known value stream vectors corresponding to the boundary regions of data structures without accessing the system memory of these boundary region stream vectors, thereby reducing processing loops and time.

[0379] In this example, the DECDIM feature uses two sets of flags: a first decrement flag (DECDIMx) (where "x" is either "1" or "2" in this example) and a second decrement flag (DECDIMxSD), along with a corresponding width parameter (DECDIMx_WIDTH). This allows the programmer to define four mask values ​​on the selected dimension of a nested loop to mask portions of the streaming data or to zero out portions of the streaming data. Table 30 defines a set of instance DECDIM flags applied to the streaming engine 125. In this example, three bits are used to define the DECDIMx flag, while only two bits are used to define the DECDIMxSD flag. In this example, the DECDIMxSD flag is therefore limited to DIM1 through DIM3. Various combinations of DECDIM settings are supported, as described in more detail below.

[0380] Table 30 DECDIM Flags for Control Data Mining Operations

[0381]

[0382] refer to Figure 29 The two DECDIMx_WIDTH fields 2931 and 2932 are defined in the stream template register 2900. Next, DECDIM1_WIDTH 2931 and the DECDIM1 flag 3019 contained in the flag field 2921 of the stream template register 2900 are... Figure 30 ) and DECDIM1SD Flagship 3020 ( Figure 30 Used together with DECDIM2_WIDTH 2932 and DECDIM2 flag 3019. Figure 30 ) and DECDIM2SD flagship 3020 ( Figure 30 Use them together.

[0383] The DECDIMx flag field and DECDIMx_WIDTH support data-band mining features, allowing programmers to use the DECDIM_WIDTH count to define the "total actual width" size of the image to provide this maximum actual width. In this mode, when the address generator 2811 / 2821 ( Figure 28 When entering a selected dimension (DIM1 to DIM5), DECDIM_WIDTH decreases according to the cycle of dimensions, as shown in Table 30. For example, if DECDIM=010b, then whenever the cycle enters dimension 2, the current value of DECDIM_WIDTH decreases according to the DIM2 value. When DECDIM_WIDTH becomes less than ICNT0 (the "patch width" in the DECDIM pattern), SE 125 will fill unused elements with zero data. This is called "data band mining".

[0384] When the selected DECDIM dimension cycle count (ICNT) expires, the DECDIM_WIDTH count value will simply be reloaded to the programmed value. In the example above, when dimension 2 (i.e., ICNT2) expires, the width will be reloaded, and dimension 3 will be moved on. In other words, the width will be reloaded when moving on to any dimension higher than the selected DECDIM dimension. Therefore, the width can be programmed to be less than the cycle iteration count of the selected dimension, which can cause the width count to underflow. This is described further below.

[0385] As mentioned above, a second DECDIMxSD flag also exists. This second flag allows the "second decrement" counter mask at the top of its corresponding DECDIMx flag to be used for the corresponding DECDIM_WIDTH. In other words, when entering or exiting the selected dimension, the corresponding DECDIM_WIDTH is decremented or reloaded using the settings of both the corresponding DECDIMx and DECDIMxSD.

[0386] In this example, the DECDIM selection can be applied to any DIM1 through DIM5 setting, operating in linear mode. However, in transpose mode, DIM1 selection is not supported, and only DIM2 through DIM5 are supported. In this example, the selected DIMx value must be unsigned (i.e., DIMx bit 31 is zero). All other dimensions can be unsigned or signed.

[0387] For example, if DECDIM=010b, then DIM2 must be an unsigned value. DIM1, DIM3, DIM4, and DIM5 can be unsigned or signed. In instance hardware, the innermost loop count is scaled proportionally to the total number of bytes. In other words, the innermost loop ICNT value is equal to (ICNT0 x ELEM_BYTES), and in DECDIM mode, the DECDIM_WIDTH count is (DECDIM_WIDTH x ELEM_BYTES). During processing unit core 110 fetches, the innermost loop count expires when all elements have been consumed before any data formatting (element copying, promotion, decimation) is considered. Therefore, with data formatting enabled, multiple processing unit core 110 fetches can occur before the innermost loop count expires.

[0388] In one instance, if the loop iteration count parameter for the selected DECDIM (i.e., ICNT1 to ICNT5) is incorrectly programmed, causing the DECDIM_WIDTH count to underflow as it decreases with the selected DIMx value, the DECDIM_WIDTH count will saturate at zero. When saturation occurs, the streaming engine 125 will keep the DECDIM_WIDTH count at zero, and the data phase after the processing unit core 110 will also be zero.

[0389] The following table and Figures 39 to 42 Each instance illustrates how this feature operates. Figure 39 Describe what the "normal" streaming mode looks like. Figures 40 to 42 This section provides several examples illustrating what data with mining patterns might look like. The DECDIM_WIDTH values ​​in the table represent the count for the entire frame, which decreases with the selected DIMx value as it enters each DECDIM dimension. Note that the examples show data moving from left to right, so LSByte is the leftmost. Furthermore, Figures 39 to 42 No 'data formatting' is enabled. If data formatting were enabled, then additional processing unit core 110 would be required to acquire each of the 'piece' widths, i.e., the innermost ICNT0.

[0390] SE 125 will not transmit any μTLB2812 / 2822 data in regions that are completely masked and saturated (DECDIM_WIDTH). Figure 28 ) or L2 130 ( Figure 1Therefore, even if the streaming addressing mode covers these regions, the programmer does not need to allocate these regions in memory. The SE 125 does this by keeping track of the overall DECDIM_WIDTH mask count and applying the DECDIMx and DECDIMxSD settings, and by blocking all subsequent L2 fetches when this count saturates to zero or ends at a 64-byte line boundary. By doing so, the SE 125 suppresses any errors or faults associated with subsequent unallocated regions because no μTLB 2812 / 2822 or L2 130 fetch is sent.

[0391] Transposed band mining is similar to linear band mining, except that the data is moved in transposed mode. The DECDIM flag provides the dimension when the DECDIM_WIDTH count decreases. This allows band masking to be applied to the remaining elements in the GRANULE. In both DECDIM mode and transposed mode, the setting DECDIM=DIM1 is not supported, as mentioned in the previous table. Note that for normal transposed mode, gap data occurs and is filled with zeros when ICNT0 is not a multiple of the GRANULE. Similarly, gap data occurs in DECDIM transposed mode, and band masking applied using DECDIM_WIDTH occurs when ICNT0 is not a multiple of the GRANULE.

[0392] Figure 39 , 40 Sections 41, 42, and 43 describe several instance linear stream deliveries that can be executed by instance streaming engine 125. These instances are by no means an exhaustive list of individual configurations of stream deliveries that can be executed by the six-dimensional nested loop control logic contained within streaming engine 125.

[0393] Figure 39 This illustrates an example of normal mode streaming 3900, where the DECDIM flag is set to "000b", as shown in Table 30. Table 31 lists the entries placed in stream template register 2900 ( Figure 29 The stream parameters are defined in the system memory 130. In this example, VECLEN = 16 elements, and each element is 1 byte. Therefore, each stream vector contains 16 bytes of array element data. The inner loop is specified by ICNT0 = 56 bytes, so the streaming engine 125 will access the system memory 130. Figure 1 The fetching process continues until 56 bytes of array data have been retrieved. In this case, fetch 1, indicated at 3921, actually retrieves 64 bytes of data from system memory 130, but only 16 bytes of data starting at address 0x0 are used to form the first stream vector. This data is then provided to processing unit core 110. Figure 1 The remaining part of the vector is obtained by the alignment network 3820 / 3821 ( Figure 39Zero padding is used, as described above. Similarly, fetch 2 3922, fetch 3 3923, and fetch 4 3924 access system memory 130 to form three additional stream vectors provided to processing unit core 110. However, ICNT0 expires at 56 elements, so only eight 1-byte elements are included in the fourth stream vector. Streaming engine 125 masks this by using alignment network zero padding to indicate the remaining bytes at 3925, as described above.

[0394] After ICNT0 expires, the next loop level, DIM1, is entered, as indicated at address 3911. In this example, DIM1 specifies a distance of 128 bytes; therefore, 53926 is fetched starting at address 0x128, and the inner loop is repeated to produce four stream vectors loaded with 56 bytes of element data. This is repeated seven times because ICNT1 = 7 in this example.

[0395] After ICNT1 expires, the next cycle level, DIM2, is entered, as indicated at 3912. In this case, DIM2 specifies a distance of 80 bytes; therefore, 293927 is obtained starting at address 0x080, and the inner loop is repeated to produce four stream vectors loaded with 56 bytes of element data. Similar to 3925, 8 bytes of element data are masked, as indicated at 3928. This pattern is repeated seven times because ICNT1 = 7 in this instance.

[0396] Table 31 – Examples of Linear Flow in Normal Mode

[0397]

[0398] Figure 40 This illustrates an example of linear streaming data with a 4000-band mining transfer, where the DECDIM flag is set to "010b" as shown in Table 30 to associate DECDIM_WIDTH with DIM2. Table 32 lists the entries placed in streaming template register 2900 ( Figure 29 The stream parameters are defined in the system memory 130. In this example, VECLEN = 16 elements, and each element is 1 byte. Therefore, each stream vector contains 16 bytes of array element data. The inner loop is specified by ICNT0 = 16 bytes, so the streaming engine 125 will access the system memory 130. Figure 1 The fetch is performed to fetch 16 bytes of array data in each iteration of the inner loop. In this case, fetch 1 indicated at 4021 actually retrieves 64 bytes of data from system memory 130, but only 16 bytes of data starting at address 0x0 are used to form the first stream vector in response to ICNT0 indicated at 4010. This data is then provided to processing unit core 110. Figure 1 The remaining part of the vector is obtained by the alignment network 3820 / 3821 ( Figure 39 Zero padding, as described above.

[0399] After ICNT0 completes its first fetch of system memory, it proceeds to the next cycle level, DIM1, as indicated at address 4011. In this example, DIM1 specifies a distance of 128 bytes; therefore, fetch 2 4022 begins at address 0x128. ICNT1 = 7, so this cycle repeats seven times, causing fetch 2 4022 to access system memory 130 until fetch 7 4023, forming six additional flow vectors provided to processing unit core 110.

[0400] After ICNT1 expires, the next cycle level DIM2 is entered, as indicated at 4012. In this case, DIM2 specifies a distance of 80 bytes; therefore, fetch 8 4024 begins at address 0x080 and the inner loop repeats to produce a single stream vector loaded with 16 bytes of element data. This loop repeats until fetch 14 4025, as ICNT1 expires again and the next cycle level DIM2 is entered again.

[0401] After ICNT1 expires again, the next cycle level DIM2 is entered, as indicated at 4026. DIM2 again specifies a distance of 80 bytes; therefore, fetch 15 4026 begins at address 0x160 and the inner loop repeats to produce a single stream vector loaded with 16 bytes of element data. This loop repeats until fetch 21 4027, as ICNT1 expires again and the next cycle level DIM2 is entered again.

[0402] After ICNT1 expires again, the next loop level DIM2 is entered, as indicated at 4026. DIM2 again specifies a distance of 80 bytes; therefore, 22 4028 is obtained starting at address 0x240 and the inner loop is repeated to produce a single stream vector loaded with 16 bytes of element data. However, in this case, the DECDIM_WIDTH parameter 4014 is set to 248 elements, which is still 248 bytes in this instance. Therefore, the array data elements in the boundary region indicated at 4029 will not be used in signal processing applications that consume this stream. By address generator 2811 / 2821 ( Figure 28 ) are generated and stored in reference queue 2815 / 2825 ( Figure 28 The metadata in ) is controlled by control logic 3814 ( Figure 38 ) used to control the alignment network 2820 / 2821 ( Figure 38 This is to mask the data in boundary region 4029. In this example, the data in each flow vector involving boundary region 4029 is masked in response to the data stored in flow template register 2900. Figure 29The DECDIM_WIDTH parameter and the DECDIM flag in the () are set to the value "0".

[0403] When ICNT1 expires again, this loop continues until 284029 is obtained. In this instance, ICNT also expires and the stream completes after these four repetitions.

[0404] Table 32 – Linear Flow Examples, Data Band Mining, DECDIM on DIM2

[0405]

[0406] Figure 41 This illustrates an example of linear streaming data band mining transmission 4100, where the DECDIM flag is set to "010b" as shown in Table 30 to associate DECDIM_WIDTH with DIM2. Table 33 lists the entries placed in streaming template register 2900 ( Figure 29 The stream parameters in ) are similar to this example. Figure 40 The example is shown in Table 33, except that ICNT2 is set to 5. The DIM1 and DIM2 level loops execute similarly from fetch 1 4121 to fetch 28 4128. Data in boundary region 4029 responds to DECDIM_WIDTH 4012 and... Figure 40 The same method of occlusion as described in the text.

[0407] However, in this example, ICNT2 = 5. Therefore, the streaming engine 125 is prepared to generate an address in another repetition of the DIM2 loop. However, DECDIM_WIDTH 4112 has now underflowed, indicating that the remaining stream vector of the boundary region 4130 will be set to a null vector. The underflow condition is called "saturation". In this example, the null vector is set to "0". When the DECDIM_WIDTH count is saturated, there is no need to access the array element data of the system memory because the stream vector is set to null. The control logic within the address generator 2811 / 2821 suppresses address generation in response to DECDIM_WIDTH saturation, so that μTLB2812 / 2822 is not accessed. Figure 28 And does not access system memory 130 ( Figure 1 This generates an empty vector. However, the metadata is formed by control logic 3814 and stored in reference queues 2815 / 2825. Then, control logic 3814 uses the metadata to create an empty vector using alignment network 2820 / 2821.

[0408] In this way, the array element stream is provided to the processing unit core 110 in response to the stream parameter, but the null vector avoids the loop of accessing system memory and μTLB.

[0409] Table 33 – Linear Flow Examples, Data Band Mining, DECDIM on DIM2 with DECDIM_WIDTH Saturation

[0410]

[0411] Figure 42 This illustrates an example of a linear streaming data band-mining transfer 4200, where the DECDIM flag is set to "001b" as shown in Table 30 to associate DECDIM_WIDTH with DIM1. Table 34 lists the entries placed in streaming template register 2900 ( Figure 29 The stream parameters in ) are similar to this example. Figure 40 The instance is simply that the DECDIM_WIDTH count is linked to DIM1 instead of DIM2. The DIM1 level loop executes in a similar manner from fetching 1 4221 to fetching 3 4222.

[0412] At position 44223, the DECDIM_WIDTH count reaches zero and the data in the boundary region 4224 is masked using alignment network 3820 / 3821, as described in more detail above.

[0413] In the next iteration, the DECDIM_WIDTH count saturates and access to system memory and μTLB is suppressed, as described in more detail above. In this way, stream vectors 4225, 4226, and 4227 are formed by the streaming engine 125 without accessing system memory or μTLB.

[0414] After ICNT1 expires, the next cycle level, DIM2, is entered, as indicated at 4212. In this case, DIM2 specifies a distance of 80 bytes; therefore, fetch 84228 begins at address 0x080 and the inner loop repeats to produce a single stream vector loaded with 16 bytes of element data. This loop continues until fetch 114029, where a portion of the stream vector is masked in response to DECDIM_WIDTH count 4214, and the remaining 3 empty stream vectors are formed by SE 125 without accessing system memory.

[0415] In this example, ICNT2=4, so there is no need for the same system access and null vector formation of this cycle to be repeated twice, as indicated at 4230.

[0416] Table 34 – Linear Flow Examples, where DECDIM_WIDTH saturates on DIM1.

[0417]

[0418] Figure 43This illustrates an example of a linear streaming data band-mining transfer 4300, where the DECDIM1 flag is set to "001b" as shown in Table 30 to associate DECDIM1_WIDTH with DIM1, and the DECDIM2 flag is set to "010b" as shown in Table 30 to associate DECDIM2_WIDTH with DIM2. Table 35 lists the data placed in the streaming template register 2900 ( Figure 29 The stream parameters in ) are similar to this example. Figure 40 The instance stream 4000 simply adds a second set of DECDIMx flags and the DECDIMx_WIDTH parameter.

[0419] The DIM1 level loop executes in a manner similar to stream 4000 until DECDIM1_WIDTH 4312 saturates. In this example, ICNT0 4310 = 16 and ICNT1 = 7. As described in more detail above, once the DECDIM1_WIDTH 4312 count is saturated, the loop requires no additional system memory access. In this case, empty stream vectors 4320 and 4321 are formed by the streaming engine 125 without accessing system memory 130.

[0420] After ICNT1 expires, the next cycle level DIM2 is entered, as indicated at 4311. The DECDIM1_WIDTH counter is reloaded with the count value from the stream template register 2900. In this case, DIM2 specifies a distance of 80 bytes; therefore, 84322 is retrieved starting at address 0x080 and the inner loop is repeated to generate a single stream vector loaded with 16 bytes of element data. This loop repeats until the DECDIM1_WIDTH 4312 count saturates again, after which the loop requires no further system memory access. In this case, empty stream vectors 4323 and 4324 are formed by the streaming engine 125 without accessing system memory 130.

[0421] In this example, ICNT2=4, so there is no need to repeat the same cycle of system access and null vector formation twice.

[0422] However, in this case, the DECDIM2_WIDTH parameter 4313 is set to 248 elements, which in this instance is still 248 bytes. Therefore, the array data elements in the boundary region indicated at 4325 will not be used in signal processing applications that consume this stream. (This is from address generators 2811 / 2821...) Figure 28 ) are generated and stored in reference queue 2815 / 2825 ( Figure 28 The metadata in ) is controlled by control logic 3814 ( Figure 38 ) used to control the alignment network 2820 / 2821 ( Figure 38This is to mask the data in boundary region 4325. In this example, the data in each flow vector involving boundary region 4325 is masked in response to the data stored in flow template register 2900. Figure 29 The DECDIM2_WIDTH parameter and the DECDIM2 flag in the ) are set to the value "0".

[0423] This loop repeats until the DECDIM1_WIDTH 4312 count saturates again, after which the loop requires no further system memory access. In this case, the empty stream vectors 4326 and 4327 are formed by the streaming engine 125 without accessing system memory 130.

[0424] In this example, ICNT3=2 and DIM3=1500, so the entire sequence is repeated again starting at address 0x1500, as indicated at 4341.

[0425] Table 35 – Examples of Linear Flow, DECDIM1 on DIM1 and DECDIM2 on DIM2

[0426]

[0427] Figure 44A , 44B Together, we explain how the submatrix is ​​passed through Figure 28 The streaming engine is used to amplify empty vectors for matrix multiplication. Table 36 lists the vectors placed in the stream template register 2900 ( Figure 29 The stream parameters are defined in this example. In this instance, VECLEN = 16 elements, and each element is 4 bytes (32 bits). Therefore, each stream vector contains 64 bytes of array element data to match MMA 3640 ( Figure 36 The width of the loop is specified by ICNT0 = 16 elements (64 bytes), therefore the streaming engine 125 will access the system memory 130 ( Figure 1 This is used to retrieve 64 bytes of array data in each iteration of the inner loop.

[0428] In this example, matrix 4400 is located in L2 system memory 130 and has 19 columns of 32-bit elements and 18 rows of 32-bit elements, where each element is in Figure 44A , 44B The symbol is indicated by "d". Since this particular instance of the MMA 3640 can only handle 16x16 arrays of 32-bit elements, matrix 4400 is subdivided into four submatrices: 4401, 4402, 4403, and 4404.

[0429] Using the stream parameters listed in Table 36, the streaming engine 125 first obtains a subarray 4401 and provides it to the MMA 3640. Next, the streaming engine 125 obtains a subarray 4402. Subarray 4402 contains the boundary region 4405, where all data elements provided to the MMA 3640 need to be set to zero for the matrix multiplication operations performed by the MMA 3640 to operate correctly. As described in more detail above, the streaming engine 125 uses DECDIM1_WIDTH 4410 to define the range of matrix 440, which in this instance is 19 elements.

[0430] In this case, the DECDIM1_WIDTH parameter 4410 is set to 19 elements, which is 76 bytes in this instance. Therefore, the array data elements in the boundary region indicated at 4405 will not be used in signal matrix multiplication applications that consume this stream. (This is from address generator 2811 / 2821) Figure 28 ) are generated and stored in reference queue 2815 / 2825 ( Figure 28 The metadata in ) is controlled by control logic 3814 ( Figure 38 ) used to control the alignment network 2820 / 2821 ( Figure 38 This is to mask the data in boundary region 4405. In this example, the data in each flow vector involving boundary region 4405 is masked in response to the data stored in flow template register 2900. Figure 29 The DECDIM1_WIDTH parameter and the DECDIM1 flag in the ) are set to the value "0".

[0431] Next, the streaming engine 125 begins acquiring subarray 4403. At acquisition 4412, the DECDIM2_WIDTH count reaches zero and saturates, and access to system memory and μTLB is suppressed, as described in more detail above. In this way, the stream vector of boundary region 4406 is formed by the streaming engine 125 without accessing system memory or μTLB.

[0432] Next, the streaming engine 125 begins acquiring subarray 4404. Acquisitions of system memory are performed at 4413 and 4414, and boundary region 4405 is again masked by alignment networks 2820 / 2821 in response to DECDEM1_WIDTH saturation, as described above. In the next loop, the DECDIM2_WIDTH count reaches zero and saturates, and acquisitions of system memory and μTLB are suppressed, as described in more detail above. In this way, the stream vector of boundary region 4406 in subarray 4404 is formed by the streaming engine 125 without accessing system memory or μTLB.

[0433] Table 36 – Examples of Linear Flow, DECDIM1 on DIM3 and DECDIM2 on DIM2

[0434]

[0435] Empty flow vectors by count

[0436] In the examples described above, several ways of filling stream vectors with constant values ​​and several ways of forming empty stream vectors without accessing system memory are described, based on a specified value of the DECDIMx_WIDTh count. In another example, specifying the number of empty vectors inserted into the stream by the streaming engine can be useful.

[0437] Creating Toeplitz matrices for convolutional neural networks (CNNs) in real time requires feeding zero or constant values ​​into a matrix multiplication accelerator after the final feature maps in the CNN layers. In linear algebra, each of these Toeplitz matrices, ordered diagonally from left to right, is a constant. (See also: Regarding...) Figure 36 The description states that MMA 3640 has a fixed number of rows that need to be padded before the initial matrix multiplication. Rows remaining during the creation of the Toplitz matrix cannot have residual values; otherwise, the result will be incorrect.

[0438] Assume that all unused data elements are set to zero or a selected null value using software that requires memory allocation and instruction execution to write the null value.

[0439] In this example, the Loop End Zero (LEZR) feature is implemented in the streaming engine 125 ( Figure 28 In the hardware, when the streaming engine dimension of the last feature map expires, several blank lines are fed to the streaming engine without reading data from memory. In this example, the streaming engine 125 sends the entire 64-byte stream vector on each fetch via the processing unit core 110, thus avoiding any data formatting, scaling, and playback during normal dimension loops. The data is then provided to the matrix multiplication accelerator 3640. The matrix fed to the matrix accelerator after the last feature map of the CNN network expires is complete and ready for block matrix multiplication. This requires no software overhead and improves performance, helping to reduce software code size and complexity.

[0440] In this example, the 8-bit LEZR_CNT (LEZR count) field 2933 is provided in stream template 2900. Figure 29 This allows up to 255 empty vectors to be inserted into the stream by the streaming engine 125 without accessing system memory. The 3-bit LEZR flag 3023 is included in the flag field 2921 of the stream template 2900. Instance definitions of the LEZR flag are included in Table 37.

[0441] Table 37 LEZR Flags for Control Data with Mining Operations

[0442]

[0443] refer to Figure 38 Control logic 3814 can control each of the multiplexers on alignment networks 3820 and 3821 to force null values ​​onto selected elements of the stream vector based on metadata provided by reference queue 2815. In this way, the entire stream vector can be set to null without fetching any data from system memory. In one instance, null is "0". In another instance, a predefined value or pattern can be used to represent null.

[0444] In another instance, control logic 3814 can control each of the multiplexers on alignment networks 3820, 3821 to force a selected value onto a selected element of the stream vector based on metadata provided by reference queue 2815. In this way, the entire stream vector can be set to the selected value (e.g., minimum or maximum value) without fetching any data from system memory.

[0445] In this example, the address generator 2811 / 2821 (see...) Figure 28 The control logic in ) performs count tracking for the six-level nested iteration. When the cycle count of the dimension specified by LEZR expires, it signals that a null or selected value will be inserted into the vector stream without retrieving data from memory for the null or selected value vectors of the stream vectors specified by the LEZR field in the stream template 2900.

[0446] Metadata generated by the address generator is pushed to stream reference queue 2815 / 2825 (see Figure 28 In the backend, when the processing unit core 110 ( Figure 1 When performing a read operation, this metadata is popped and sent to the control logic 3814 in the alignment network 2820 / 2821 via a channel specified as empty or set to the selected value, without having to read the data from the system memory.

[0447] Figure 45 This demonstrates the use of the LEZR function to add null vectors to the stream. In this example, the multiple feature maps indicated at 4501 are stored in system memory L2 130 (…). Figure 1The stream template is created by the software application, where ICNT1 defines the number of filter rows (Fr) in the feature map, ICNT2 defines the number of filter columns (Fc) in the feature map, and ICNT3 defines the number of input feature maps (Ni). ICNT0 defines the number of elements of feature map 4501 obtained by the streaming engine from a row. In this example, ICNT0 will be set to 64 for 8-bit elements and to 32 for 16-bit elements. The streaming engine 125 obtains feature maps according to the multidimensional stream template (as described in more detail above) to form stream 4500, as indicated at 4502. Stream 4500 is provided to MMA 3640 as a B matrix, as described in more detail above.

[0448] In this example, the MMA width can be chosen as 16, 32, or 64 elements depending on the size of the elements, as per the example. Figure 36 Description. Ni * Fr * Fc forms the rows of the B matrix. This value is not necessarily a multiple of the selected MMA width. Therefore, an additional number of empty vectors are formed by specifying LEZR_CNT in the stream template, where LEZR_CNT is defined by expression (4).

[0449] (4)

[0450] Fill value

[0451] In some applications of pooling operations, such as max, min, and average pooling in convolutional neural networks, the end of a sequence of image pixels needs to be a maximum or minimum value. Similarly, analysis tools for various deep learning neural networks, such as TensorFlow and Caffe networks, require arbitrary zero, maximum, or minimum padding for pooling operations.

[0452] In this example, the 3-digit padding value (PADVAL) flag 3023 is included in the flag field 2921 of the flow template 2900. The instance definition of the PADVAL flag is included in Table 38. The padding size is applied across the entire element, as defined by flow template 2900 ( Figure 29 The ELTYPE flag 3001 in ) Figure 30 The default value of zero is defined in this example. It is selected by setting the PADVAL flag 3023 to 000b "Unsigned Minimum".

[0453] Table 38 PADVAL Flags for Control Data with Mining Operations

[0454]

[0455] In one instance, the flow vector is formed using the specified padding value as the last element of the flow vector. In another instance, another flag field (e.g., LEZR flag 3023, Table 37) can be used to specify the selected dimension on which the padding value is included.

[0456] refer to Figure 38 Control logic 3814 can control each of the multiplexers on alignment networks 3820 and 3821 to force null values ​​onto selected elements of the stream vector based on metadata provided by reference queue 2815. In this way, the entire stream vector can be set to a specified null value without fetching any data from system memory. In one instance, null is the unsigned maximum value. In another instance, other predefined values ​​or patterns can be used to represent null values.

[0457] In this example, control logic 3814 can control each of the multiplexers on alignment networks 3820 and 3821 to force a selected value onto a selected element of the flow vector based on metadata provided by reference queue 2815. In this way, the entire flow vector can be set to the selected value (e.g., minimum or maximum value) without fetching any data from system memory.

[0458] In this example, the address generator 2811 / 2821 (see...) Figure 28 The control logic in the stream performs count tracking for the six-level nested iteration. When the loop count at the end of the stream expires, it signals that a null or selected value will be inserted into the vector stream without retrieving the data of the null or selected value vector from memory, as defined by the PADVAL flag 3024 in the stream template 2900.

[0459] Metadata generated by the address generator is pushed to stream reference queue 2815 / 2825 (see Figure 28 In the backend, when the processing unit core 110 ( Figure 1 When performing a read operation, this metadata is popped and sent to the control logic 3814 in the alignment network 2820 / 2821 via a channel specified as empty or set to the selected value, without having to read the data from the system memory.

[0460] To achieve this, the permutation operation needs to be performed in the software code. In the example simulation, the hardware-based PADVAL solution outperforms the software solution for images with a width > 64 by 4 times and outperforms the software solution for images with a width < 64 by 2 times; furthermore, the software solution requires three different loops.

[0461] Figure 46 Explanation by, for example Figure 28The streaming engine 125 inserts empty or predefined data vectors to form a stream. In this example, at 4600, the stream parameters are stored in the stream template register within the streaming engine. Figure 29 To open a stream on the streaming engine, use (2900). Stream parameters include the array element size, the number of elements in each vector of the stream, the number of vectors in each dimension of the stream, and the width indicator for the selected dimension of the array, such as the DECDIM flag and DECDIM_COUNT, see Table 30.

[0462] In step 4601, an address stream is generated based on the stream parameters stored in the stream template. Metadata indicating loop count, loop end, remaining width count, etc., is stored.

[0463] In 4602, the streaming engine uses an address sequence generated by the address generator to retrieve a row of matrix data from the system memory.

[0464] At 4603, check if the width count parameter is associated with the current dimension of the multidimensional flow. If not, then form a normal flow vector at 4605.

[0465] If the width count parameter is associated with the current dimension, then at 4604, a check is performed to determine if the width count has been exhausted. If it has not been exhausted, then a normal flow vector is formed at 4605.

[0466] If the width count is exhausted, then at 4606, a check is performed to determine if the width count is saturated. If it is not saturated, then at 4607, a flow vector is formed by masking a portion of the vector exceeding the width count. In some instances, zero values ​​can be inserted into the mask. In another instance, predefined values ​​can be inserted by masks such as minimum, maximum, etc.

[0467] If the width count reaches saturation, a null vector is formed at 4608, eliminating the need to access system memory. In one instance, the null vector can be set to all zeros. In another instance, the null vector can be set to a predetermined padding value, such as a minimum or maximum value. The predetermined padding value can be, for example, specified by the PADVAL flag 3023 in the stream template. Figure 30 The flag field is specified.

[0468] At 4609, if the current dimension is not yet complete, the loop repeats at 4602. If the current dimension is complete, the width count is decremented at 4610 as it is associated with the current dimension. Otherwise, the width count is not decremented.

[0469] At 4611, check to determine if the flow is complete. If not, the process repeats to retrieve more matrix data from the system and form the flow vector.

[0470] In 4612, once the entire matrix has been accessed from memory, the data stream is closed.

[0471] Figure 47 Explanation by, for example Figure 28 The streaming engine 125 inserts empty or predefined data vectors to form a stream. In this example, at 4700, the stream parameters are stored in the stream template register (2900) within the streaming engine. Figure 29 To open a stream on a streaming engine, use the array `<array>`. Stream parameters include the array size, the number of elements in each vector of the stream, the number of vectors in each dimension of the stream, and the loop end count (e.g., loop end count zero 2933). Figure 29 The flags that associate the zero count at the end of the loop with the selected dimension of the multidimensional loop are described in Table 37.

[0472] In step 4701, an address stream is generated based on the stream parameters stored in the stream template. Metadata indicating loop count, loop end, remaining width count, etc., is stored.

[0473] In 4702, the streaming engine uses an address sequence generated by the address generator to retrieve a row of matrix data from the system memory.

[0474] In 4703, data elements obtained from system memory are used to form a normal flow vector.

[0475] At 4704, if the current dimension is not yet complete, the loop repeats at 4602. If the current dimension is complete...

[0476] At 4705, a check determines whether the null count is applied to the current dimension. If so, the streaming engine forms n null vectors equal to the null count values ​​in the streaming template. In this example, the data in the null vectors is set to zero. In another example, pre-selected null values ​​can be used to form null vectors such as minimum, maximum, etc.

[0477] At 4707, check to determine if the flow is complete. If not, the process repeats in 4708 for the next loop to retrieve more matrix data from the system and form the flow vector.

[0478] In 4709, once the entire matrix has been accessed from memory and a specified empty vector has been formed without accessing data from system memory, the data stream is closed.

[0479] Figure 48 This example illustrates a multiprocessor system. In this example, the SoC 4800 includes 100 processors ( Figure 1(Referring to "Processor A") and combined with a second processor 4811 (referring to "Processor B"). Each processor is coupled to a block of shared Level 3 (L3) memory 4850 via bus 4851. Processor B contains a block of unshared Level 2 memory 4812. A Direct Memory Access (DMA) engine 4860 can be programmed to transfer blocks of data / instructions from L3 memory to L2 memory 130 or L2 memory 4812 using known or later-developed DMA technologies. Various types of peripheral devices 4862 are also coupled to memory bus 4851, such as wireless and / or wired communication controllers.

[0480] In this example, processor A, processor B, and L3 memory 4850 are all contained within SoC 4800, which can be encapsulated using known or later-developed packaging technologies to form a package that can be mounted on a substrate such as a printed circuit board (PCB). For example, SoC 4800 can be encapsulated in a ball grid array (BGA) package. In this example, external memory interface (EMI) 4852 allows additional external bulk memory 4854 to be accessed by processor A and / or processor B.

[0481] In this example, processor B is an ARM® processor used for scalar processing and control functions. In other examples, various types of known or later-developed processors can be combined with DSP 100. Although two processors are described in this example, in another example, multiple copies of DSP 100 and / or multiple copies of processor B can be included within the SoC and use techniques for forming masked and null vectors, without accessing the system memory provided by streaming engine 125, as described in more detail herein.

[0482] Other examples

[0483] In the described instance, the streaming engine comprises two tightly coupled streaming engines capable of managing two data streams simultaneously. In another instance, the streaming engine can manage only a single stream, while in still others, it can handle more than two streams. In each case, for each stream, the streaming engine includes an address generation phase, a data formatting phase, and some storage for the formatted data awaiting consumption by the processor.

[0484] In the described instance, addresses are derived from algorithms that may involve multidimensional loops, with each dimension maintaining an iteration count. In one instance, the streaming engine supports six levels of nested iterations. Other instances support more or fewer levels of iteration.

[0485] In the described instance, one-dimensional zero-padding of the flow vector is provided. In another instance, two-dimensional zero-padding of the flow vector is provided. In yet another instance, zero-padding of more than two dimensions can be provided.

[0486] In the described example, a complex DSP processor with multiple functional units and dual data paths is depicted. In another example, a simpler DSP coupled to a stream processor can be used. In yet another example, other types of known or later-developed processors, such as Reduced Instruction Set Computers (RISC), microprocessors, etc., can be coupled to a stream processor.

[0487] In the described example, MMA supports 32x32 16-bit matrix multiplication and 64x64 8-bit matrix multiplication, and the streaming engine is configured to provide 64-byte stream vectors. In another example, MMA can be configured to support larger or smaller matrix sizes. The associated streaming engine can be configured to provide stream vectors with sizes greater than or less than 64 bytes.

[0488] In the described example, the processor consuming the data stream and the streaming engine retrieving the data stream from system memory are both contained within a single integrated circuit (IC) as a system-on-a-chip. In another example, the processor consuming the data stream may be packaged in a first IC, and the streaming engine may be packaged in a second separate IC coupled to the first IC via a known or later-developed communication channel or bus.

[0489] In this description, the term "coupled" and its derivatives mean indirect, direct, optical, and / or radio connection. Therefore, if a first device is coupled to a second device, the connection can be a direct electrical connection, an indirect electrical connection via other devices and connections, an optical electrical connection, and / or a radio connection.

[0490] Modifications may be made to the described examples, and other examples may be made within the scope of the claims.

Claims

1. An apparatus comprising: Processing unit core; Memory; A register configured to store a template that specifies the following for a set of data: The data width of the set of data along the first dimension; The amount of data retrieved from the memory to form the first portion of the first dimension of the set of data; as well as The fill value used for the second part of the first dimension of the set of data; as well as A stream circuit, coupled between the cores of the processing unit, is configured to perform the following operations based on a stream open instruction associated with the template: The set of data was generated in the following manner: Retrieve the data of the first portion from the memory; and The second portion of the set of data is generated based on the imputed value without accessing the memory; as well as The set of data is provided to the core of the processing unit.

2. The apparatus of claim 1, wherein the template includes a field specifying whether the fill value is a minimum representable value or a maximum representable value.

3. The apparatus of claim 1, wherein the template includes a field specifying whether the padding value is an unsigned minimum representable value, an unsigned maximum representable value, a signed minimum representable value, or a signed maximum representable value.

4. The apparatus of claim 1, wherein the template specifies the amount of data to be retrieved in the first dimension by specifying a count of iterations in a loop in the first dimension.

5. The apparatus of claim 1, wherein the template specifies the amount of data to be retrieved in the first dimension by specifying the number of data elements to be retrieved and the size of each of the data elements.

6. The apparatus of claim 1, wherein the template specifies the first dimension from a plurality of dimensions of the set of data.

7. The apparatus of claim 1, wherein the flow circuit comprises an alignment circuit, the alignment circuit comprising: A register configured to store the first portion of the set of data; A set of multiplexers coupled to the register; as well as Control logic coupled to the set of multiplexers and configured such that the set of multiplexers generates the set of data by providing the data from the first portion of the register and generating the second portion of the set of data based on the padding value.

8. The apparatus of claim 7, wherein the flow circuit comprises: A memory interface coupled to the memory and configured to retrieve the data of the first portion of the set of data; and A butterfly network coupled between the memory interface and the register and configured to reorder the data of the first portion of the set of data before storing the data of the first portion into the register.

9. The apparatus according to claim 1, wherein: The flow circuit includes an address generator configured to generate a set of addresses based on the template; and The flow circuit is configured to retrieve the data of the first portion from the memory using the set of addresses.

10. The device of claim 1, wherein the memory is a level two (L2) cache.

11. The device of claim 10, further comprising a level one (LI) cache coupled between the processing unit core and the L2 cache, wherein the stream circuit is coupled between the processing unit core and the L2 cache in parallel with the LI cache.

12. A method comprising: receiving a stream instruction by a circuit device; and based on the stream instruction: receiving a template from a register, the template specifying for a set of data: a data width of the set of data along a first dimension; retrieving from the memory a subset of the set of data to form a first portion of the set of data in the first dimension; and a padding value for a second portion of the set of data in the first dimension; and retrieving from the memory the subset of the set of data; generating the second portion of the set of data based on the padding value without accessing the memory; and providing the set of data to a processing unit core.

13. The method of claim 12, wherein the template includes a field specifying whether the padding value is a minimum representable value or a maximum representable value.

14. The method of claim 13, wherein the field specifies whether the padding value is signed or unsigned.

15. The method of claim 12, wherein the template specifies the subset of the set of data retrieved from the memory by specifying a count of iterations of a loop in the first dimension.

16. The method of claim 12, wherein the template specifies the subset of the set of data to retrieve from the memory by specifying a number of data elements to retrieve and a size of each of the data elements.

17. The method of claim 12, wherein the template specifies the first dimension from a plurality of dimensions of the set of data.

18. The method of claim 12, further comprising reordering the subset of the set of data prior to providing the set of data.

19. The method of claim 12, wherein the memory is a level two (L2) cache.

20. The method of claim 12, wherein the stream instruction specifies the template by specifying the register from a set of registers.

21. A method of operating a stream engine in a computer system, the method comprising: receiving, by the stream engine, stream parameters into control logic of the stream engine to define a multi-dimensional array, wherein the stream parameters include width indicators for dimensions in the multi-dimensional array, wherein the width indicators include a width count; generating, by an address generator, an address stream based on the stream parameters; retrieving, by the stream engine, a row of matrix data from a memory based on the address stream; and forming, by the stream engine, a stream vector in response to a state of the width count.

22. The method of claim 21, wherein: ​ ​ The state of the width count is exhausted.

23. The method of claim 22, wherein: The state of the width count is not saturated.

24. The method of claim 23, wherein: The stream vector is formed by masking out a portion of the stream vector that exceeds the width count.

25. The method of claim 24, wherein: A zero value is inserted into the portion of the stream vector that is masked out.

26. The method of claim 24, wherein: A predefined value is inserted into the portion of the stream vector that is masked out.

27. The method of claim 26, wherein: The predefined value is one of a minimum value and a maximum value.

28. The method of claim 22, wherein: The state of the width count is saturated.

29. The method of claim 28, wherein: The stream vector is a null vector formed without accessing the memory.

30. The method of claim 29, wherein: The null vector is all zeros.

31. The method of claim 29, wherein: The null vector is a predetermined value.

32. The method of claim 31, wherein: The predetermined value is selected from one of a minimum value and a maximum value.

33. The method of claim 31, wherein: The predetermined value is specified by a flag value in the stream parameters.

34. The method of claim 31, further comprising: Decrementing the width count.

35. The method of claim 34, further comprising: Responsive to matrix data still to be retrieved, retrieving, by the stream engine, a next row of the matrix data.

Images