Stream processing-oriented multi-level cache collaborative CGRA processor architecture

Abstract

The invention relates to the technical field of computer system structures, and discloses a multi-level cache collaborative CGRA processor architecture for stream processing, which comprises a global controller, a software management memory subsystem and a processing element array, and is characterized in that the global controller is connected with the software management memory subsystem and the processing element array through a special bus; and the global controller is used for analyzing the instruction to generate a control signal, and cooperatively scheduling the external storage device, the special cache library and the private storage of the processing element to construct a three-level cache collaborative system. An intermediate result is written into a special cache library through a stream write-back channel so as to reduce access to an external memory device, and an asynchronous execution protocol is executed by utilizing a double-layer configuration storage space in a processing element micro-architecture so as to realize parallel configuration loading and task execution. A configurable data caching mechanism or a roundabout routing path is configured by using a static delay planning method to solve the problem of data synchronization, so that the flow processing calculation energy efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of computer architecture technology, specifically to a multi-level cache collaborative CGRA processor architecture for stream processing. Background Technology

[0002] Coarse-grained reconfigurable architecture (CGRA) is a spatial computing architecture designed for specific domain applications. CGRA achieves parallel computing through interconnected arrays of processing units. When processing tasks such as Fast Fourier Transform or Finite Impulse Response Filtering streaming computation, data flows continuously between processing units. However, existing coarse-grained reconfigurable architectures have limitations in their storage hierarchy design. Intermediate computation results generated by processing units cannot reside in on-chip memory and need to be written to external storage devices. Subsequent computation tasks then need to read the intermediate computation results from external storage devices again. This repeated access to external storage devices consumes data transmission bandwidth and increases data transfer power consumption.

[0003] Coarse-grained reconfigurable architectures need to switch hardware configurations according to the algorithm stage during operation. Existing coarse-grained reconfigurable architectures usually start loading the configuration data for the next stage after the current task is completed. The configuration data loading process consumes the runtime of computing resources. Computing resources cannot perform operations during configuration loading. The serial configuration and execution mode leads to reduced hardware utilization.

[0004] In a spatially distributed array of processing units, data is transmitted through an on-chip network. The different physical lengths of different data paths cause deviations in the time when the data arrives at the target processing unit. Multi-operand operation nodes require input data to arrive in the same clock cycle. To solve the data synchronization problem, existing technologies introduce dynamic scheduling mechanisms or handshake protocols at the hardware level. Dynamic scheduling mechanisms increase the area overhead of hardware logic and increase design complexity. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a multi-level cache collaborative CGRA processor architecture for stream processing. This architecture solves the problems faced by existing coarse-grained reconfigurable architectures when processing streaming tasks, such as limited data transmission bandwidth, high configuration context switching overhead, and difficulties in data synchronization between multiple nodes, resulting in low utilization of computing units and insufficient overall energy efficiency.

[0006] To achieve the above objectives, the present invention provides a multi-level cache-coordinated CGRA processor architecture for stream processing, including a global controller, a software-managed memory subsystem, and an array of processing elements.

[0007] The global controller physically connects the software-managed memory subsystem and the processing element array via a dedicated bus. The global controller parses binary instruction packets to generate control signals and configuration parameters for the target module. The software-managed memory subsystem responds to the control signals from the global controller. The software-managed memory subsystem uses a built-in DMA engine to establish a data transfer path between external storage devices and a dedicated cache library. The software-managed memory subsystem configures the dedicated cache library as a shared on-chip storage bank. The processing element array consists of multiple interconnected processing elements. The processing element array configures its internal processing element private storage, connection units, and functional units according to configuration parameters. The global controller coordinates the scheduling of external storage devices, the shared on-chip storage bank, and the processing element private storage through control signals. The global controller constructs a three-level cache coordination system to execute software-defined data prefetching and transformation strategies.

[0008] Furthermore, the global controller includes an instruction memory buffer, an instruction decoder unit, and an instruction transmitter. The instruction memory buffer transmits buffered instructions to the instruction decoder unit via a dedicated bus. The instruction decoder unit uses hardware decoding logic to parse the target module identifier field in the instruction. The instruction decoder unit identifies and processes stream node operations, including data distribution, preprocessing, timing alignment, and computation operations. The instruction transmitter is connected to the instruction decoder unit. The instruction transmitter implements dynamic routing based on the target module identifier field. The instruction transmitter distributes the decoded control signals and configuration parameters to the software-managed memory subsystem or processing element array.

[0009] Furthermore, the software-managed memory subsystem integrates a write-back channel, which establishes a direct data feedback path between the data loading and storage unit and the dedicated cache library. The write-back channel is used to directly write the intermediate results generated by the processing element array into the dedicated cache library. The software-managed memory subsystem uses the write-back channel to keep the intermediate results resident in the dedicated cache library. The software-managed memory subsystem transmits the intermediate results as input data for subsequent computing tasks in a closed loop without accessing external storage devices.

[0010] Furthermore, the data loading and storage unit employs hardware-accelerated logic. The data loading and storage unit is used to perform data conversion between the dedicated cache library and the processing element array. The data loading and storage unit responds to control signals and performs continuous access mode, affine access mode that calculates the target access address based on base address parameters and index parameters, or bit-reversed access mode that reverses the binary bits of the target access address.

[0011] Furthermore, the dedicated cache library consists of multiple independent 64-bit wide cache libraries. Each of these cache libraries is organized in a circular buffer FIFO manner. The circular buffer FIFO manner supports continuous access to stream processing data and allows the global controller to calculate and manage stream address information through pointers. The dedicated cache libraries provide data buffer space for the data stream distribution and integration unit, as well as the data loading and storage unit in the software-managed memory subsystem.

[0012] Furthermore, the microarchitecture of the processing element integrates a dual-layer configuration storage space, which consists of a background shadow configuration layer and a foreground active configuration layer. During the execution of the current stream processing task by the processing element array, the global controller writes configuration parameters defining the next stage operation type and routing path to the background shadow configuration layer in parallel. In response to the context switching signal, the processing element array maps the contents of the background shadow configuration layer to the foreground active configuration layer, thereby enabling the processing element array to switch hardware contexts in real time.

[0013] Furthermore, the global controller and the processing element array are configured to execute an asynchronous execution protocol to control configuration loading and task execution. While the functional unit performs arithmetic logic operations, the global controller transmits configuration parameters to the background shadow configuration layer. After the transmission is completed, the global controller sets the configuration ready status register. When the stream consumption counter in the software-managed memory subsystem reaches the preset task length and the configuration ready status register is set, the global controller issues a context switching signal. The context switching signal triggers the processing element array to start a new computing task.

[0014] Furthermore, the global controller uses configuration parameters generated by the static delay planning method to configure the processing element array to solve the data synchronization problem. The static delay planning method satisfies the data synchronization constraint: for any two-operand operation node, the data arrival time of the first predecessor node is equal to the data arrival time of the second predecessor node. The data arrival time is calculated based on the sum of the absolute clock cycle of the predecessor node's execution start, the number of hardware pipeline delay cycles of the predecessor node, the number of on-chip network physical transmission delay cycles, and the number of inserted static compensation delay cycles. The global controller generates configuration parameters based on the calculated number of static compensation delay cycles. The global controller uses the hardware resources of the processing elements to eliminate the time difference in data arrival.

[0015] Furthermore, the functional unit input port of the processing element integrates a configurable data caching mechanism, and the connection unit of the processing element supports multi-hop data transmission. When the number of static compensation delay cycles is less than or equal to the depth threshold of the configurable data caching mechanism, the configuration parameters configure the configurable data caching mechanism to delay the input operands. When the number of static compensation delay cycles is greater than the depth threshold, the configuration parameters configure the connection unit to construct a detour routing path. The connection unit accumulates physical transmission delay by increasing the number of data transmission hops in the programmable on-chip network. Intermediate processing elements on the detour routing path are configured in routing pass-through mode.

[0016] Furthermore, the processing element includes a functional unit responsible for arithmetic and logic operations and a connection unit responsible for data interaction. The connection unit serves as a routing node of the programmable on-chip network. The connection unit is configured with four physical communication ports in the north, south, east, and west directions, as well as a local communication port. The connection unit integrates routing multiplexer logic. The connection unit defines the mapping relationship between input channels and output channels according to the configuration parameters maintained by the foreground activity configuration layer. The connection unit implements single-hop data transmission, multi-hop data relay, and broadcast transmission modes.

[0017] This invention provides a multi-level cache-coordinated CGRA processor architecture for stream processing, which has the following advantages:

[0018] 1. This invention constructs a three-level cache coordination system by coordinating the scheduling of external storage devices, a dedicated cache library in the software-managed memory subsystem, and the private storage of processing elements through a global controller. The software-managed memory subsystem uses a write-back channel to directly write the intermediate results generated by the processing element array into the dedicated cache library, and uses the intermediate results as input for subsequent tasks without accessing external storage devices. The three-level cache coordination system reduces the frequency of processor architecture access to external storage devices and reduces the power consumption generated by data transfer between the chip and external storage devices.

[0019] 2. This invention integrates a dual-layer configuration storage space consisting of a background shadow configuration layer and a foreground active configuration layer within the microarchitecture of the processing element, and adopts an asynchronous execution protocol. While the processing element array is executing the current computing task, the global controller preloads the configuration parameters for the next stage into the background shadow configuration layer. The processing element array responds to the context switching signal and maps the content of the background shadow configuration layer to the foreground active configuration layer within a single clock cycle. The dual-layer configuration storage space and the asynchronous execution protocol achieve time overlap between the execution of computing tasks and the loading of configuration parameters, eliminating pipeline pauses caused by hardware refactoring.

[0020] 3. This invention uses configuration parameters generated by the static delay planning method to configure the processing element to solve the data synchronization problem. The global controller delays the input data through the configurable data caching mechanism in the microarchitecture of the processing element according to the static compensation delay cycle number, or increases the transmission hop number by constructing a detour path through the connection unit. The static delay planning method, together with hardware resources, ensures that the input data of the dual-operand operation node is aligned at the clock cycle time, avoiding the hardware area overhead caused by the use of dynamic scheduling logic. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall structure of the present invention;

[0022] Figure 2 This is a schematic diagram of the microarchitecture of the processing element and the hardware pipeline structure of the present invention;

[0023] Figure 3 This is a schematic diagram illustrating the working principle of the asynchronous configuration-execution model and the two-layer configuration storage space of the present invention;

[0024] Figure 4 This is a flowchart illustrating the static delay planning method and hardware coordination mechanism of the present invention.

[0025] Figure 5 This is a comparison chart of clock cycle consumption during the execution phase of the present invention;

[0026] Figure 6 This is a bar chart showing the performance speedup ratio of the present invention;

[0027] Figure 7 This is a logarithmic coordinate graph showing the relationship between the number of nodes in the data flow graph and the compilation mapping time in this invention.

[0028] Among them, 1. Global controller; 2. Software-managed memory subsystem; 3. Processing element array. Detailed Implementation

[0029] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] See attached document Figure 1 This invention provides a multi-level cache-coordinated CGRA processor architecture for stream processing. It includes a global controller 1, a software-managed memory subsystem 2, and a processing element array 3. The global controller 1, the software-managed memory subsystem 2, and the processing element array 3 work together to accelerate stream processing tasks.

[0031] Global Controller 1 is the core scheduling unit of the multi-level cache-coordinated CGRA processor architecture for stream processing. Global Controller 1 is responsible for coordinating the operational workflow and ensuring the execution of stream processing tasks. Global Controller 1 unifies the management of instruction and data streams within the control unit. Global Controller 1 includes an instruction memory buffer, an instruction decoder unit, and an instruction issuer.

[0032] The instruction memory buffer is used to buffer binary instruction packets generated by the software stack. It supports 64-bit wide instruction transfer and can process multiple instructions simultaneously. The instruction memory buffer is connected to the instruction decoder unit via a dedicated bus. The instruction decoder unit employs dedicated hardware decoding logic. The instruction decoder unit is used to identify and process stream node operations. Stream node operations include data distribution, data loading, preprocessing, timing alignment, interconnection, computation, constant configuration, data storage, and data output. The instruction decoder unit is connected to the instruction transmitter. The instruction transmitter supports precise scheduling of nine types of target modules. The instruction transmitter implements dynamic routing based on the target module identifier field.

[0033] The software-managed memory subsystem 2 is a programmable abstraction layer. It standardizes memory access patterns through software definition. The software-managed memory subsystem 2 includes a data stream distribution and integration unit, a dedicated cache library, a data loading and storage unit, and a stream write-back channel.

[0034] The data stream distribution and integration unit is equipped with an independent DMA (Direct Memory Access) engine. The DMA engine incorporates stream production and consumption counters, which track the progress of the data stream. A dedicated cache library consists of multiple independent 64-bit wide cache libraries. These multiple independent 64-bit wide cache libraries form a configurable stream processing data storage area. Each of these cache libraries is organized in a circular buffer FIFO manner. The data loading and storage unit employs hardware acceleration to support real-time data transformation. The data loading and storage unit implements data loading and storage operations. It supports sequential access mode, affine access mode, and bit-reversed access mode. The stream write-back channel supports data reuse and iterative processing. The stream write-back channel is used to implement the cyclic return of intermediate results.

[0035] Processing element array 3 consists of a two-dimensional array of processing elements interconnected via a programmable on-chip network. The processing elements in processing element array 3 are interconnected via a programmable on-chip network. Processing element array 3 is used to perform computational operations for stream processing tasks. Software-managed memory subsystem 2 is connected to external storage devices, including DDR (Synchronous Dynamic Random Access Memory). Software-managed memory subsystem 2 is used to transfer data between the external storage devices and processing element array 3. Global controller 1 is connected to software-managed memory subsystem 2. Global controller 1 is connected to processing element array 3. Global controller 1 is used to send control signals and configuration parameters to software-managed memory subsystem 2 and processing element array 3.

[0036] Global controller 1 consists of an instruction memory buffer, an instruction decoder unit, and an instruction issuer. The instruction memory buffer is connected to the instruction decoder unit via a dedicated bus. The instruction decoder unit is connected to the instruction issuer.

[0037] The instruction memory buffer is used to buffer binary instruction packets. These binary instruction packets are generated by the software stack. The instruction memory buffer supports 64-bit wide instruction transfers. It can process multiple instructions simultaneously. Instructions are transferred to the instruction decoder unit via a dedicated bus.

[0038] The instruction decoder unit employs dedicated hardware decoding logic. It performs hardware parsing of each instruction. The instruction decoder unit parses the target module identifier field and instruction type field in the instruction to determine the operation type. The instruction decoder unit identifies stream node operations. It processes stream node operations, which include data distribution operations, data loading operations, preprocessing operations, timing alignment operations, interconnect operations, computation operations, constant configuration operations, data storage operations, and data output operations.

[0039] The command transmitter receives decoded parameters from the command decoder unit. Based on the target module identifier field, the command transmitter distributes the decoded parameters to the target modules. The command transmitter supports precise scheduling of nine types of target modules. Dynamic routing is implemented based on the target module identifier field.

[0040] Global Controller 1 maintains the state machine of the stream processing tasks. Global Controller 1 monitors the progress of the data stream in real time. Global Controller 1 monitors memory access patterns in real time. Global Controller 1 coordinates the operation workflow based on the progress of the data stream and memory access patterns. Global Controller 1 ensures the execution of the stream processing tasks.

[0041] The three-level cache architecture consists of external storage devices, a shared on-chip memory bank, and processor-specific private storage. External storage devices include DDR memory. The shared on-chip memory bank comprises a dedicated cache library within the software-managed memory subsystem 2. Processor-specific private storage is integrated within the processor microarchitecture of the processor array 3.

[0042] The data stream distribution and integration unit establishes a data transfer path between the external storage device and the shared on-chip memory bank. The unit reads the data stream from the external storage device via a DMA engine. It then writes the data stream to a dedicated cache library. This dedicated cache library consists of multiple independent 64-bit wide cache libraries. The dedicated cache library serves as a temporary storage for stream processing data within the shared on-chip memory bank.

[0043] The data loading and storage unit establishes a data transfer path between the shared on-chip storage bank and the processing element's private storage. The data loading and storage unit reads data from a dedicated cache. The data loading and storage unit loads the data into the processing element's private storage. The data loading and storage unit accesses the data according to the requirements of the stream processing task.

[0044] Global Controller 1 configures the three-level cache coordination system via instructions. Global Controller 1 manages the data flow between external storage devices, the shared on-chip storage bank, and the processor's private storage. The three-level cache coordination system executes a software-defined data prefetching strategy. The three-level cache coordination system executes a software-defined data translation strategy. The three-level cache coordination system preloads data between external storage devices, the shared on-chip storage bank, and the processor's private storage. The three-level cache coordination system reduces the frequency of access to external storage devices.

[0045] The data stream distribution and integration unit is equipped with an independent DMA engine. This independent DMA engine has a built-in stream production counter. Both the stream production and consumption counters precisely track the progress of the data stream.

[0046] The data stream distribution and integration unit performs data loading operations. During the data loading phase, the data stream distribution and integration unit retrieves data from off-chip memory according to the streaming production requirements. The data stream distribution and integration unit transfers data to a dedicated cache library via an independent DMA engine.

[0047] The data stream distribution and integration unit performs data output operations. During the data output phase, the data stream distribution and integration unit submits the processed data to the output port. The data stream distribution and integration unit then makes the processed data available for subsequent processing.

[0048] The dedicated cache library contains multiple independent 64-bit wide cache libraries. These multiple independent 64-bit wide cache libraries form a configurable stream processing data storage area. Each of the multiple independent 64-bit wide cache libraries is organized in a circular buffer FIFO manner.

[0049] The circular buffer FIFO organization supports continuous access to stream data. The circular buffer FIFO organization allows the global controller 1 to calculate and coordinate the address information of the compiled stream via pointers. The circular buffer FIFO organization simplifies the software management complexity of stream processing. A dedicated cache library provides data buffering space for data stream distribution and integration units. A dedicated cache library provides data buffering space for data loading and storage units.

[0050] The data loading and storage unit employs hardware-accelerated logic to support real-time data transformation. The data loading and storage unit performs data loading operations. The data loading and storage unit establishes a data transmission channel between a dedicated cache library and the processing element array 3.

[0051] The data loading and storage unit supports sequential access mode. In sequential access mode, the data loading and storage unit accesses the dedicated cache library in linear order of memory addresses. The data loading and storage unit also supports affine access mode. In affine access mode, the data loading and storage unit calculates the target access address based on the base address parameter, index parameter, and step size parameter. The data loading and storage unit also supports bit-reversal access mode. In bit-reversal access mode, the data loading and storage unit reverses the binary bits of the target access address. Bit-reversal access mode supports the data rearrangement required by the Fast Fourier Transform algorithm.

[0052] The data loading and storage unit accesses data according to the requirements of the stream processing task. The data loading and storage unit loads data read from the dedicated cache into the processing element array 3. The data loading and storage unit stores data from the processing element array 3 into the dedicated cache.

[0053] The write-back stream is integrated into the software-managed memory subsystem 2. It supports the cyclical return of intermediate results. The write-back stream establishes a direct data feedback path between the data loading and storage units and the dedicated cache library.

[0054] The write-back stream allows the data loading and storage unit to directly write intermediate results generated by the processing element array 3 into a dedicated cache. The write-back stream does not access external storage devices while transmitting intermediate results. It maintains the residency of intermediate results in the dedicated cache. The write-back stream supports using intermediate results as input data for subsequent computation tasks. It reduces data movement overhead by transmitting data in a closed loop within the chip. The write-back stream reduces the bandwidth consumption of external storage devices by streaming processing tasks.

[0055] See attached document Figure 2 The processing element adopts a dual-input, dual-output hardware pipeline structure. The processing element consists of functional units and connection units.

[0056] The connection unit is responsible for handling data interaction between the device and the programmable on-chip network (SoC). The connection unit receives the input data stream from the SoC. The connection unit distributes the input data stream to the functional units. The functional units are the execution core of the processing device. The functional units are responsible for performing arithmetic and logical operations. The functional units receive operands from the connection units. The functional units perform calculations on the operands to generate results. The functional units return the results to the connection units. The connection units output the results to the SoC.

[0057] The processing element supports the pipelined execution of stream processing tasks through a hardware pipelined architecture. The behavior of the processing element in each clock cycle is controlled by a configuration word. The configuration word defines the operation type of the functional unit. The configuration word defines the routing path for the connection units.

[0058] A functional unit is the hardware carrier that enables processing elements to perform arithmetic operations. Functional units integrate complex FP32 (32-bit floating-point) computing units. Functional units also integrate scalar FP32 computing units. Functional units employ a resource reuse architecture. Functional units control the operating mode of computing resources based on configuration words.

[0059] In complex number operation mode, the functional unit calls the complex FP32 computation unit. The functional unit performs a single 64-bit complex floating-point operation. In scalar operation mode, the functional unit calls the scalar FP32 computation unit. The functional unit performs two 32-bit scalar floating-point operations in parallel. The functional unit improves hardware utilization through a resource reuse architecture.

[0060] The functional unit incorporates a configurable data caching mechanism. This configurable data caching mechanism is located at the input port of the computing resource. It consists of a multi-level register chain or a small FIFO. The configurable data caching mechanism is used to temporarily store operands within the processing element. It adjusts the operand dwell time according to configuration parameters. It ensures that the two operands involved in two-operand operations are precisely aligned over clock cycles. It maintains data continuity for stream processing tasks.

[0061] The processing element array 3 constructs a programmable on-chip network based on a two-dimensional mesh topology. Connectivity units are integrated within the microarchitecture of each processing element. These connectivity units serve as independent routing nodes for the programmable on-chip network. The programmable on-chip network employs an unbuffered design to reduce data movement overhead.

[0062] The connection unit is configured with four physical communication ports: a north port, a south port, an east port, and a west port. Each physical communication port is equipped with two 64-bit physical channels. These two 64-bit physical channels support parallel data transmission. The two 64-bit physical channels can be configured to transmit a single 64-bit complex FP32 data value or two scalar FP32 data values. Each physical communication port includes an input channel and an output channel. The connection unit is also configured with a local communication port. The local communication port establishes a data path between the connection unit and the functional unit. The connection unit distributes operands to the functional unit through the local communication port. The connection unit also receives the calculation results output by the functional unit through the local communication port.

[0063] The connection unit integrates routing multiplexer logic. This logic is controlled by a configuration word. The configuration word defines the mapping between input and output channels in each clock cycle. The connection unit establishes data transmission paths within the processing element array 3 based on the configuration word. The connection unit supports single-hop data transmission between adjacent processing elements. It also supports multi-hop data relay between non-adjacent processing elements. Furthermore, the connection unit supports a broadcast transmission mode that simultaneously sends the same data to multiple output channels. The programmable on-chip network utilizes distributed connection units to implement the spatial data flow graph mapping required for stream processing tasks.

[0064] See attached document Figure 3 To maximize hardware utilization, this invention constructs an asynchronous configuration-execution model. This model is built upon a two-tier configuration storage space integrated within the microarchitecture of the processing component. The two-tier configuration storage space consists of a background shadow configuration layer and a foreground active configuration layer.

[0065] Global Controller 1 writes configuration words to the background shadow configuration layer via a dedicated bus. Global Controller 1 writes configuration words during the execution of the current stream processing task by the processing element. The configuration word defines the types of arithmetic and logical operations required for the next stage of the stream processing task. The configuration word also defines the data routing path required for the next stage of the stream processing task.

[0066] The functional unit reads the control signals held by the foreground activity configuration layer. The functional unit performs arithmetic and logical operations based on the control signals held by the foreground activity configuration layer. The connection unit reads the control signals held by the foreground activity configuration layer. The connection unit performs data routing based on the control signals held by the foreground activity configuration layer. The configuration word writing action and the functional unit's calculation action are physically independent. The configuration word writing action and the connection unit's routing action are performed in parallel timing.

[0067] The configuration-execution asynchronous model supports instantaneous switching of hardware contexts. When the current streaming task completes, the processing element microarchitecture receives an update trigger signal. Within a single clock cycle, the processing element microarchitecture maps the contents of the background shadow configuration layer to the foreground active configuration layer. The configuration-execution asynchronous model eliminates the time overhead associated with hardware reconfiguration. The configuration-execution asynchronous model maintains continuous throughput for the data stream from the processing element array 3.

[0068] By employing a configuration-execution asynchronous model, this invention establishes an asynchronous execution protocol to define the interaction timing between the global controller 1 and the processing element array 3. The asynchronous execution protocol parallelizes the write operations of configuration words and the execution operations of computational tasks.

[0069] Global Controller 1 transmits the configuration word to the background shadow configuration layer. Global Controller 1 transmits the configuration word while the functional unit performs arithmetic and logical operations. The processing element stores the configuration word in the background shadow configuration layer. The processing element sets the configuration ready status register. The configuration ready status register indicates that the background shadow configuration layer has completed data loading.

[0070] Global Controller 1 reads the stream consumption counter from the data stream distribution and integration unit. Global Controller 1 compares the stream consumption counter with the preset task length. When the stream consumption counter equals the preset task length, Global Controller 1 confirms that the current computation task is complete. Global Controller 1 then queries the configuration ready status register.

[0071] Upon confirming the completion of the current computation task and setting the configuration ready state register, Global Controller 1 issues a context switch signal. The processing element responds to the context switch signal. The processing element maps the contents of the background shadow configuration layer to the foreground active configuration layer. The processing element resets the configuration ready state register. The processing element starts a new computation task based on the updated foreground active configuration layer. The asynchronous execution protocol eliminates idle waiting periods caused by hardware circuit reconfiguration.

[0072] See attached document Figure 4 To address the data synchronization problem, this invention employs a static delay planning method. This method is applied during the compiler's mapping of the data flow graph to the processing element array 3. The data flow graph contains multiple operation nodes. It also contains data dependency edges connecting these operation nodes. The compiler maps the operation nodes to specific processing elements. Finally, the compiler maps the data dependency edges to physical routing paths in the on-chip network.

[0073] Due to the varying lengths of the physical routing paths within the processing element array 3, the data generated by the source operation node arrives at the target operation node at inconsistent times. The functional unit's execution of two-operand operations requires both input operands to arrive at the input port within the same clock cycle. The static delay planning method eliminates this time difference by inserting compensating delays into the data transmission path.

[0074] For any two-operand operand node with two predecessor nodes, the static delay programming method must satisfy the data synchronization constraint:

[0075] ;

[0076] in: Defined as the first predecessor node Data arrives at the node The clock cycle time. Defined as the second predecessor node Data arrives at the node The clock cycle time; Defined as the current operating node; Defined as the current operation node The first predecessor node; Defined as the current operation node The second precursor node.

[0077] The arrival time of data in a clock cycle is calculated using the following formula:

[0078] ;

[0079] in: Represents a node The predecessor node, with a value of or ; Indicates the predecessor node The absolute clock cycle at which an operation begins to be executed on the processing element; Indicates the predecessor node The number of hardware pipeline delay cycles consumed in performing arithmetic and logical operations; This indicates that the data originates from the mapping predecessor node. The source processing element is transmitted to the mapping node. The number of on-chip network physical transmission delay cycles required by the target processing element; This indicates that the compiler is linking the predecessor node. With nodes The number of static compensation delay cycles to be inserted on the data path.

[0080] The goal of static delayed planning is to solve for each data dependency edge. Value. The static delay planning method is physically implemented by configuring the routing paths of the connection units in the processing element or the input buffers of the functional units. .

[0081] The delay planning algorithm is executed by the compiler. The delay planning algorithm first performs initialization, loading the inherent internal delay for each input / output port pair of each processing element, and initializing the cumulative delay table.

[0082] The algorithm performs a recursive calculation of the maximum cumulative latency. For each input port of each processing element, a reverse depth-first search process is performed. This reverse depth-first search process starts from the input port and backtracks along the data flow graph to the data input endpoint. During the backtracking process, the internal latency values ​​of the traversed paths are accumulated, and the maximum cumulative latency of all paths is taken at the convergence point. This maximum value is recorded as the cumulative latency of the input port. This step ensures that the slowest path from all data sources to the port is identified.

[0083] The algorithm calculates and configures the delay compensation value. For each processing element, it iterates through each pair of input ports that need to work together. The cumulative delay difference between the two ports is calculated. Based on the cumulative delay difference, a configuration command is issued to the hardware-configurable delay unit corresponding to the data path with the smaller delay. The configuration command causes the hardware-configurable delay unit to increase its delay by the cumulative delay difference by one cycle. This operation physically aligns the arrival times of the two data paths.

[0084] Once all input port pairs of all processing elements have been processed, the processor architecture has completed the timing optimization configuration for the current computation graph.

[0085] The compiler updates the execution start time of the operation node. The execution start time of the operation node is equal to the moment when the data synchronization constraint is satisfied. The compiler maps the calculated number of compensated latency cycles to hardware resources. When the number of compensated latency cycles is less than the depth threshold of the configurable data caching mechanism within the functional unit, the compiler configures the configurable data caching mechanism to absorb the time difference. When the number of compensated latency cycles is greater than the depth threshold of the configurable data caching mechanism, the compiler configures the routing path of the connection unit to generate additional transmission latency using the transmission hop count.

[0086] To implement the compensated latency cycle number calculated by the compiler at the physical level, this invention employs a hardware coordination mechanism. This hardware coordination mechanism relies on configurable components within the processing element's microarchitecture.

[0087] The input ports of the functional unit integrate a configurable data buffer mechanism. This configurable data buffer mechanism consists of shift registers or a first-in, first-out queue. When the number of compensated delay cycles is less than or equal to the maximum depth of the configurable data buffer mechanism, the compiler directly configures the delay parameters of the configurable data buffer mechanism. The configurable data buffer mechanism delays input operands within the processing element. The configurable data buffer mechanism ensures that two input operands enter the arithmetic logic unit of the functional unit within the same clock cycle.

[0088] The connection unit supports multi-hop data transmission within the processing element array 3. When the number of compensation delay cycles exceeds the maximum depth of the configurable data buffer mechanism, the compiler utilizes the connection unit to construct a detour routing path. This detour routing path artificially increases the number of hops in the on-chip network. The connection unit consumes a fixed number of clock cycles in each data forwarding operation. By increasing the number of transmission hops, the connection unit accumulates physical transmission delay to meet the compensation delay cycle requirement.

[0089] Intermediate processing elements on the detour routing path are configured in route pass-through mode. In route pass-through mode, the functional units of the intermediate processing element are disabled. Only the connection units of the intermediate processing element are activated for data relay. The hardware coordination mechanism, by combining a configurable data caching mechanism with the detour routing path, strictly enforces the synchronization constraints generated by the static delay planning method at the physical layer.

[0090] The entire stream processing task consists of a compilation-mapping phase, a configuration loading phase, and a hardware execution phase. In the compilation-mapping phase, the compiler reads the data flow graph of the stream processing application. The compiler maps the operation nodes in the data flow graph to physical processing elements in the processing element array 3. The compiler maps the data dependency edges in the data flow graph to physical routing paths in the programmable on-chip network. The compiler runs a static delay planning method. The compiler calculates the number of compensation delay cycles required for each data dependency edge. The compiler generates a binary configuration bitstream based on the mapping results and the number of compensation delay cycles. The binary configuration bitstream contains the operation type control words for the functional units. The binary configuration bitstream contains the routing multiplexer control words for the connection units. The binary configuration bitstream contains delay parameters for a configurable data buffering mechanism.

[0091] During the configuration loading phase, Global Controller 1 writes the binary configuration bitstream to the background shadow configuration layer inside the processing element via a dedicated configuration bus. Global Controller 1 performs the write operation in parallel while the processing element array 3 executes the current stream processing task. Global Controller 1 monitors the progress of the configuration data write. After completing the data loading in the background shadow configuration layer, Global Controller 1 sets the configuration ready status register.

[0092] During the hardware execution phase, Global Controller 1 issues a context switch signal. The processing element updates the contents of the background shadow configuration layer to the foreground active configuration layer. The data stream distribution and integration unit reads the source data stream from the shared on-chip storage bank. The data stream distribution and integration unit injects the source data stream into the programmable on-chip network. The connectivity unit forwards data packets according to the routing path defined by the foreground active configuration layer. The functional unit aligns the arrival times of input operands using a configurable data caching mechanism. The functional unit performs arithmetic and logical operations to generate calculation results. The processing element transmits the calculation results to the next-level node. The data stream distribution and integration unit collects the final results output by the processing element array 3. The data stream distribution and integration unit writes the final results back to the shared on-chip storage bank.

[0093] The prototype system of this invention is implemented based on an FPGA (Field Programmable Gate Array) platform. The processing element array 3 consists of 64 independent processing elements. The processing element array 3 adopts an 8-row, 8-column two-dimensional grid topology.

[0094] Each processing element integrates a scalar FP32 computing unit. The scalar FP32 computing unit is compatible with 64-bit complex floating-point arithmetic. The on-chip network employs an unbuffered design, supporting single-cycle, single-hop data transmission.

[0095] The dedicated cache library is divided into multiple independent storage entities for parallel access. The depth of the background shadow configuration layer and the depth of the foreground active configuration layer within the processing element are configurable. The adjustable latency range of the configurable data caching mechanism meets the timing alignment requirements of stream processing tasks.

[0096] The FPGA-based prototype system achieves an average performance of 29.73 GOPS (billion operations per second). This prototype system significantly reduces the frequency of off-chip memory accesses through a multi-level cache coordination mechanism.

[0097] This invention selects existing technology as a performance comparison benchmark. Under the condition of executing a stream processing task, the multi-level cache collaborative CGRA processor architecture records the first execution time. Existing technology records the second execution time.

[0098] According to actual test data, compared with the existing technology, the present invention improves the performance by 1.55 times on FFT (Fast Fourier Transform) workload and by 1.62 times on FIR (Finite Impulse Response) workload.

[0099] The configuration-execution asynchronous model reduces overall task latency by overlapping configuration loading time and computation execution time. Static latency planning methods reduce instruction decoding power consumption by eliminating dynamic scheduling overhead. The multi-level cache collaborative CGRA processor architecture leverages near-memory computing characteristics to reduce data transport power consumption.

[0100] The multi-level cache-coordinated CGRA processor architecture supports parameterized configuration at the 3-processor element array scale. The 3-processor element array employs a two-dimensional mesh topology. This two-dimensional mesh topology allows for scaling the parallel computing capabilities of the multi-level cache-coordinated CGRA processor architecture by increasing the number of rows and columns.

[0101] The theoretical peak computational performance of the multi-level cache-coordinated CGRA processor architecture is calculated using the following formula:

[0102] ;

[0103] in: This represents the theoretical peak computational performance of the multi-level cache collaborative CGRA processor architecture; This indicates the number of rows of processing elements contained in processing element array 3; This indicates the number of columns of processing elements contained in processing element array 3; This indicates the operating clock frequency of the processing element array 3; It represents the number of floating-point operations that a functional unit in a single processing element can complete in a single clock cycle.

[0104] The compiler runs a static deferred planning method on the host computer. This method maps the data flow graph to the processing element array 3. The time overhead of the compilation mapping process depends primarily on the node size of the data flow graph. The compilation mapping time exhibits a polynomial growth relationship with the number of operation nodes in the data flow graph, reducing compilation time to the microsecond level, an order of magnitude faster than traditional methods.

[0105] Compilation mapping time is approximately calculated using the following formula:

[0106] ;

[0107] The definitions of each symbol in the formula are as follows: This represents the compilation mapping time required for the compiler to complete the data flow graph mapping and generate the binary configuration bitstream; This represents a baseline time constant related to the computing performance of the host computer. This represents the total number of operation nodes contained in the data flow graph to be mapped; This represents the time complexity index of the static delayed planning method.

[0108] The static delay planning method solves for the compensation delay through topology sorting and deterministic computation steps. It avoids the iterative convergence process present in traditional place-and-route algorithms. As the size of the processing element array 3 increases, the multi-level cache, in conjunction with the CGRA processor architecture, ensures that compilation efficiency does not degrade exponentially.

[0109] Furthermore, the multi-level cache collaborative CGRA processor architecture possesses software-transparent elastic scalability. Through unified hardware instruction set construction rules and a co-built software stack design, the multi-level cache collaborative CGRA processor architecture achieves software transparency during large-scale expansion. The software stack achieves unified management of hardware of different scales through an abstraction layer. Developers can fully utilize the expanded processing element array 3 resources by recompiling and mapping without modifying the application code.

[0110] To aid in understanding the specific working mechanism of the multi-level cache collaborative CGRA processor architecture for stream processing, a specific application example based on a 1024-point complex fast Fourier transform (FFT) stream processing task is given below, and the beneficial effects of the present invention are verified by combining experimental data.

[0111] See attached document Figure 5 This invention provides a bar chart comparing the clock cycle consumption of a 1024-point complex fast Fourier transform stream processing task on a multi-level cache collaborative CGRA processor architecture during the execution phase.

[0112] The horizontal axis of this bar chart represents the four key execution stages of a stream processing task: configuration loading, data loading and preprocessing, computation execution and interconnection, and data write-back. The vertical axis represents the number of clock cycles consumed in each stage. The chart compares the traditional synchronous execution architecture with the multi-level cache collaborative CGRA processor architecture of this invention.

[0113] During the configuration loading phase, the bar chart shows that the clock cycles consumed by the architecture of this invention are lower than those of traditional architectures, even approaching zero. This is because the present invention adopts an asynchronous configuration-execution model. The global controller 1 utilizes the idle time gap of the previous task execution to pre-write the binary configuration bit stream into the background shadow configuration layer inside the processing element via a dedicated configuration bus. When a task switch occurs, the processing element updates the data of the background shadow configuration layer to the foreground active configuration layer within a single clock cycle, thereby physically masking the configuration data write time and eliminating the idle waiting period caused by reconfiguring hardware circuits.

[0114] During the data loading and preprocessing stages, the column height of the architecture of this invention is significantly reduced. This is thanks to the fact that the data loading and storage unit in the software-managed memory subsystem 2 supports bit-reversed access mode. The data loading and storage unit directly rearranges the data in a dedicated cache library, which is organized in a circular buffer FIFO manner. Combined with the DMA engine of the data stream distribution and integration unit, it achieves high-efficiency data throughput and avoids the additional data handling overhead at the software level in traditional architectures.

[0115] During the computation execution and interconnection phases, the architecture of this invention exhibits lower clock cycle usage. This is because the compiler runs a static delay planning method during the mapping phase. The static delay planning method accurately calculates the number of compensation delay cycles required for each data dependency edge in the data flow graph and eliminates time differences by configuring the configurable data caching mechanism of the functional units or the routing path of the connection units. This ensures that the complex FP32 computing units in the processing element array 3 can operate at full capacity without dynamically waiting for data alignment, thereby maximizing hardware utilization.

[0116] During the data write-back phase, the architecture of this invention utilizes a streaming write-back channel to establish a direct data feedback path between the data loading and storage unit and the dedicated cache library. For intermediate frequency domain data generated by the FFT algorithm, the streaming write-back channel directly stores it in the dedicated cache library for use in the next stage of butterfly operations, unlike traditional architectures that frequently access external storage devices (such as DDR). This closed-loop transmission of on-chip data reduces bandwidth consumption on external storage devices, thereby shortening the time consumed in the data write-back phase.

[0117] See attached document Figure 6 This invention provides performance comparison experimental results between the multi-level cache collaborative CGRA processor architecture and existing technologies.

[0118] The experimental environment utilizes an FPGA-based prototype system. The prototype system operates at a 1.0 GHz clock frequency. Existing technologies employ general-purpose digital signal processors with the same number of single-precision floating-point units. The experiment uses Fast Fourier Transform (FFT) and Finite Impulse Response (FIR) filters as benchmark procedures.

[0119] When performing a 1024-point complex Fast Fourier Transform task, the multi-level cache-coordinated CGRA processor architecture eliminates the overhead of dynamic instruction scheduling by utilizing a static latency planning method. The multi-level cache-coordinated CGRA processor architecture also reduces external memory access latency by utilizing a streaming write-back channel. Experimental data shows that the execution time of the multi-level cache-coordinated CGRA processor architecture is lower than that of existing technologies. According to the formula:

[0120] ;

[0121] in: This indicates the performance speedup of the multi-level cache collaborative CGRA processor architecture relative to existing technologies; This represents the absolute time consumed by existing technologies in executing stream processing tasks; This represents the absolute time consumed by the multi-level cache-coordinated CGRA processor architecture to execute stream processing tasks.

[0122] The multi-level cache-coordinated CGRA processor architecture achieves a 1.55x performance speedup on the Fast Fourier Transform task.

[0123] When performing a 64-tap finite impulse response (FIR) filter task, the multi-level cache-coordinated CGRA processor architecture utilizes an asynchronous configuration-execution model to mask configuration loading time. Experimental data show that the multi-level cache-coordinated CGRA processor architecture achieves a 1.62x performance speedup on the FIR filter task.

[0124] See attached document Figure 7 This invention provides compilation efficiency verification of a multi-level cache collaborative CGRA processor architecture at different scales. The horizontal axis in the figure represents the total number of operation nodes, and the vertical axis represents the compilation mapping time. The figure shows the measured compilation time and the polynomial fitting curve.

[0125] The experiment measured the compiler's compilation mapping time on the host computer by adjusting the total number of operation nodes in the data flow graph. The results show that as the total number of operation nodes increases from 100 to 10000, the measured compilation time exhibits a polynomial growth rather than an exponential growth. The polynomial fitting curve conforms to the formula... , where the index It is approximately 1.2.

[0126] Experimental data demonstrates that the deterministic computation steps employed by the static delayed planning method avoid iterative convergence. When the processing element array 3 scales up, the compiler maintains a microsecond-level compilation speed. The multi-level cache-coordinated CGRA processor architecture exhibits excellent scalability. After scaling up the processing element array 3, developers only need to recompile to deploy the application to the new hardware platform, verifying the software transparency of the multi-level cache-coordinated CGRA processor architecture during scaling.

Claims

1. A multi-level cache-cooperative CGRA processor architecture for stream processing, characterized in that, include: The global controller (1) is physically connected to the software-managed memory subsystem (2) and the processing element array (3) via a dedicated bus, and is used to parse binary instruction packets to generate control signals and configuration parameters for the target module; The software-managed memory subsystem (2) responds to the control signal from the global controller (1), establishes a data transmission path between the external storage device and the dedicated cache library using the built-in DMA engine, and configures the dedicated cache library as a shared on-chip memory. The processing element array (3) is composed of multiple interconnected processing elements, and the private storage, connection unit and functional unit inside the microarchitecture of the processing elements are configured according to the configuration parameters; The global controller (1) coordinates the scheduling of the external storage device, the shared on-chip storage bank and the private storage of the processing element through the control signal to construct a three-level cache coordination system to execute the software-defined data prefetching and conversion strategy.

2. The multi-level cache-coordinated CGRA processor architecture for stream processing according to claim 1, characterized in that, The global controller (1) includes an instruction memory buffer, an instruction decoder unit, and an instruction transmitter; The instruction memory buffer transmits the buffered instructions to the instruction decoder unit via a dedicated bus; The instruction decoder unit uses hardware decoding logic to parse the target module identifier field in the instruction, and identifies and processes stream node operations including data distribution, preprocessing, timing alignment and computation operations. The instruction transmitter is connected to the instruction decoder unit and achieves dynamic routing through the target module identifier field, distributing the decoded control signal and the configuration parameters to the software-managed memory subsystem (2) or the processing element array (3).

3. The multi-level cache-coordinated CGRA processor architecture for stream processing according to claim 1, characterized in that, The software-managed memory subsystem (2) integrates a stream write-back channel; The write-back channel establishes a direct data feedback path between the data loading and storage unit and the dedicated cache library, and is used to directly write the intermediate results generated by the processing element array (3) into the dedicated cache library; The software-managed memory subsystem (2) uses the write-back channel to keep the intermediate results resident in the dedicated cache library and transmits the intermediate results as input data for subsequent computing tasks in a closed loop without accessing the external storage device.

4. The multi-level cache-coordinated CGRA processor architecture for stream processing according to claim 3, characterized in that, The data loading and storage unit employs hardware-accelerated logic to perform data conversion between the dedicated cache library and the processing element array (3); The data loading and storage unit responds to the control signal to execute a continuous access mode, an affine access mode that calculates the target access address based on the base address parameter and the index parameter, or a bit-reversal access mode that reverses the binary bits of the target access address.

5. The multi-level cache-cooperative CGRA processor architecture for stream processing according to claim 1, characterized in that, The dedicated cache library consists of multiple independent 64-bit wide cache libraries, and each of the multiple independent 64-bit wide cache libraries is organized in a circular buffer FIFO manner. The circular buffer FIFO method supports continuous access to stream processing data, allowing the global controller (1) to calculate and manage stream address information through pointers, and to provide data buffer space for the data stream distribution and integration unit and the data loading and storage unit in the software-managed memory subsystem (2).

6. The multi-level cache-cooperative CGRA processor architecture for stream processing according to claim 1, characterized in that, The microarchitecture of the processing element integrates a dual-layer configuration storage space, which consists of a background shadow configuration layer and a foreground activity configuration layer. During the execution of the current stream processing task by the processing element array (3), the global controller (1) writes the configuration parameters defining the next stage operation type and routing path to the background shadow configuration layer in parallel. The processing element array (3) responds to the context switching signal and maps the content of the background shadow configuration layer to the foreground activity configuration layer, thereby realizing the instant switching of the hardware context.

7. The multi-level cache-coordinated CGRA processor architecture for stream processing according to claim 6, characterized in that, The global controller (1) and the processing element array (3) are used to perform asynchronous execution protocol control configuration loading and task execution: The global controller (1) transmits the configuration parameters to the background shadow configuration layer while the functional unit performs arithmetic logic operations, and sets the configuration ready state register after the transmission is completed. When the stream consumption counter in the software-managed memory subsystem (2) reaches the preset task length and the configuration ready state register is set, the global controller (1) issues the context switching signal to trigger the processing element array (3) to start a new computing task.

8. The multi-level cache-coordinated CGRA processor architecture for stream processing according to claim 6, characterized in that, The global controller (1) uses configuration parameters generated by the static delay planning method to configure the processing element array (3) to solve the data synchronization problem. The static delay planning method satisfies the data synchronization constraints: For any two-operand operation node, the arrival time of the data in the first predecessor node is equal to the arrival time of the data in the second predecessor node; The data arrival time is calculated by summing the absolute clock cycle of the predecessor node's execution start, the number of predecessor node hardware pipeline delay cycles, the number of on-chip network physical transmission delay cycles, and the number of inserted static compensation delay cycles. The global controller (1) generates the configuration parameters based on the calculated number of static compensation delay cycles and uses the hardware resources of the processing element to eliminate the time difference in data arrival.

9. The multi-level cache-coordinated CGRA processor architecture for stream processing according to claim 8, characterized in that, The input port of the functional unit of the processing element integrates a configurable data caching mechanism, and the connection unit of the processing element supports multi-hop data transmission. When the number of static compensation delay cycles is less than or equal to the depth threshold of the configurable data caching mechanism, the configuration parameter configures the configurable data caching mechanism to delay the number of input operations; When the number of static compensation delay cycles is greater than the depth threshold, the configuration parameters configure the connection unit to construct a detour routing path, using the increased number of data transmission hops in the programmable on-chip network to accumulate physical transmission delay. At this time, the intermediate processing element on the detour routing path is configured in routing pass-through mode.

10. The multi-level cache-coordinated CGRA processor architecture for stream processing according to claim 6, characterized in that, The processing element includes a functional unit responsible for arithmetic and logical operations and a connection unit responsible for data interaction. The connection unit, as a routing node of the programmable on-chip network, is configured with four physical communication ports in the north, south, east and west directions, as well as a local communication port. The connection unit integrates routing multiplexer logic and defines the mapping relationship between input channels and output channels according to the configuration parameters maintained by the front-end activity configuration layer, thereby realizing single-hop data transmission, multi-hop data relay and broadcast transmission modes.

Images