A multi-core heterogeneous processor on-chip staging dynamic dispatch manager

Abstract

The application provides an on-chip temporary storage dynamic scheduling manager of a multi-core heterogeneous processor, which comprises an on-chip temporary storage dynamic scheduling unit and an on-chip temporary storage network; the on-chip temporary storage dynamic scheduling unit is a control unit arranged in a main controller of the multi-core processor, and comprises a scheduler and a mapping unit; the scheduler is used for maintaining a storage node state table and scheduling on-chip temporary storage mechanism data of a task according to the storage node state table; the mapping unit is used for calculating coordinates of a functional node for executing a subsequent task on the network; the on-chip temporary storage network comprises storage nodes in all functional nodes on the multi-core processor, and the storage nodes form a 2D-mesh topology network. The on-chip temporary storage dynamic scheduling manager of the application can effectively reduce the number of times of writing back the calculation task result data to a DDR external memory, greatly saves the memory access overhead, and greatly improves the coarse-grained task calculation performance.

Description

Technical Field

[0001] This invention relates to the field of processor technology, and more specifically to an on-chip temporary storage dynamic scheduling manager for multi-core heterogeneous processors. Background Technology

[0002] As the integration level of integrated circuits continues to increase and integrated circuits transform into integrated systems, higher requirements are placed on the communication efficiency between processor cores. The scalability, parallel access, and global clock synchronization of traditional shared memory structures have become bottlenecks restricting their development. In contrast, distributed on-chip storage networks have good address space, scalability, and parallel processing capabilities, effectively alleviating the power consumption and area problems of wide topology comb networks, and are gradually becoming a preferred high-efficiency on-chip data storage method in the industry.

[0003] Furthermore, the memory wall bottleneck in multi-core processors is becoming increasingly apparent as processor clock speeds increase. The increased latency of off-chip memory access severely restricts processor performance. For example, the current mainstream server-grade memory standard DDR4-3200 has a standard read / write bandwidth of approximately 50GB / s, but in actual use, the latency caused by numerous out-of-order reads reduces this bandwidth. In addition, with the upgrading of memory products, major memory chip manufacturers such as Samsung and Micron are gradually increasing the prefetch width of the DDR interface to improve user clock frequencies. This also leads to increased latency within the DDR controller. For example, in a DDR4-2667 read / write test using an Intel Core i7-7800x processor (Gigabyte x299 motherboard), the memory latency reached as high as 80ns. However, the 3.5GHz i7-7800x processor has a single clock cycle of only 0.3ns. If a cache hit fails, the processor may have to wait hundreds of clock cycles to access memory data. This situation significantly restricts processor performance. Summary of the Invention

[0004] To overcome the high off-chip access latency problem of DDR, this invention proposes an on-chip temporary storage dynamic scheduling manager for multi-core heterogeneous processors.

[0005] This invention aims to provide an on-chip temporary storage dynamic scheduling manager for a multi-core heterogeneous processor. The on-chip temporary storage dynamic scheduling manager is disposed on the multi-core processor. The on-chip temporary storage dynamic scheduling manager includes an on-chip temporary storage dynamic scheduling unit and an on-chip temporary storage network.

[0006] The on-chip temporary storage dynamic scheduling unit is deployed within the main controller of the multi-core processor and works as a coprocessor unit of the main controller. The on-chip temporary storage dynamic scheduling unit includes a scheduler and a mapping unit.

[0007] The scheduler is used to maintain the storage node status table and schedule the data generated by the task according to the storage node status table using the on-chip temporary storage mechanism.

[0008] The mapping unit is used to calculate the coordinates of the functional node that performs the subsequent task on the network;

[0009] The on-chip temporary storage network includes storage nodes among all functional nodes on the multi-core processor, and the storage nodes form a 2D-mesh topology network.

[0010] Furthermore, the storage node status table maintains the current computation task information when on-chip temporary storage is enabled;

[0011] The task information includes the task ID, working mode, the amount of data that needs to be temporarily stored for this task, the coordinates of the newly occupied temporary storage nodes for this task, and the number of capacity segments already occupied.

[0012] The task ID is the input of the scheduler to the storage node status table. The capacity segment contained in a storage node corresponds to only one storage node coordinate, but may correspond to multiple task IDs.

[0013] The operating mode is the scheduler's input to the storage node's status table, used to determine whether to perform a release or occupation operation on the status table.

[0014] The amount of data that needs to be temporarily stored for this task is the scheduler's input to the storage node status table, used to determine which capacity segment should be used for the current temporary storage.

[0015] The coordinates of the temporary storage node to be released in this task are the output of the storage node status table to the scheduler, which is used to notify the scheduler which storage node to retrieve the temporary data from.

[0016] The number of occupied capacity segments records the number of occupied capacity segments of various sizes, which the scheduler uses to determine whether there are enough storage segments to enable the on-chip temporary storage mechanism.

[0017] Furthermore, the scheduler can determine whether to activate the on-chip temporary storage mechanism based on the information provided by the resource status table and the current task information;

[0018] The scheduler initiates the on-chip temporary storage mechanism when the following conditions are met simultaneously:

[0019] The data stream size of the task is less than or equal to the remaining space size of a certain capacity segment;

[0020] The data generated by the task needs to be received by the successor node within a predetermined time.

[0021] Furthermore, the scheduler can determine whether a new storage node needs to be created;

[0022] Based on the current data volume generated by the calculation, the data is divided into capacity segments. It is determined whether any storage nodes have already been divided into nodes of the corresponding type. If there are nodes of the corresponding type, it is determined whether the node has a free capacity segment. If there is a free segment, the data of this calculation is stored in the free capacity segment. If no storage node has been divided into nodes of the corresponding type or the nodes of the corresponding type do not have a free capacity segment, the node that generated the data of this calculation is divided into a capacity segment of the corresponding capacity.

[0023] Furthermore, the mapping unit assigns a relatively nearest functional node to a task that requires data located on an on-chip storage node as source data based on the minimum Manhattan distance;

[0024] The minimum Manhattan distance is expressed as:

[0025] d(i,j)=|X1-X2|+|Y1-Y2|

[0026] Where node i has coordinates (X1, Y1) and node j has coordinates (X2, Y2).

[0027] The beneficial effects of this invention include:

[0028] The dynamic scheduling manager provided by this invention can make dynamic on-chip temporary storage decisions by fully combining information such as on-chip computing resource usage and current computing task characteristics. This allows programmers to save a lot of task path planning time even without a full understanding of the system layout. While partially avoiding high-latency off-chip memory access operations, it also lowers the threshold for developers to use multi-core heterogeneous computing systems for computing.

[0029] The dynamic scheduling manager of this invention fully considers the scheduling of functional nodes for subsequent tasks after the release of temporary resources. By using the nearest neighbor algorithm to calculate in real time the node most suitable for undertaking the subsequent computation, the network congestion risk caused by long data paths is greatly avoided, while reducing the time cost of data transfer within the on-chip network.

[0030] The dynamic scheduling manager of this invention employs a method of dynamically dividing and releasing capacity segments. This avoids resource conflicts caused by fixed mapping of capacity segments to storage nodes, and improves the flexibility of storage nodes in managing data of different capacities.

[0031] The main purpose of the dynamic scheduling manager of this invention is to perform secure on-chip temporary storage management. The improvement in computing speed is due to the fact that secure on-chip temporary storage management allows us to avoid forcing all tasks to be written back to off-chip memory after completion, thereby indirectly improving the running speed.

[0032] In the dynamic scheduling manager of this invention, functional nodes determine whether data can be transferred or given to subsequent nodes within a threshold time. Through this threshold judgment mechanism, a significant decrease in storage node occupancy and an increase in the utilization rate of corresponding functional nodes can be observed. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the structure of the on-chip temporary storage scheduling unit in the on-chip temporary storage dynamic scheduling manager of a multi-core heterogeneous processor provided in an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of the internal structure of the on-chip temporary storage scheduling unit in an on-chip temporary storage dynamic scheduling manager for a multi-core heterogeneous processor provided in an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of the on-chip data network in the on-chip temporary storage dynamic scheduling manager of a multi-core heterogeneous processor provided in an embodiment of the present invention;

[0036] Figure 4 This is a schematic diagram of the on-chip temporary storage occupancy decision mechanism in the on-chip temporary storage dynamic scheduling manager of a multi-core heterogeneous processor provided in an embodiment of the present invention. Detailed Implementation

[0037] The technical solution of the present invention will be described in more detail below with reference to the accompanying drawings. The present invention includes, but is not limited to, the following embodiments.

[0038] As attached Figure 1-2 As shown, the present invention provides an on-chip temporary storage dynamic scheduling manager for a multi-core heterogeneous processor. The on-chip temporary storage dynamic scheduling manager is set on the multi-core processor and includes an on-chip temporary storage dynamic scheduling unit and an on-chip temporary storage network.

[0039] The on-chip temporary storage dynamic scheduling unit is a control unit that integrates functions such as summarizing storage resource status, actively occupying on-chip temporary storage resources, automatically releasing on-chip temporary storage resources, and optimizing the scheduling of subsequent computing units. It is deployed within the main controller of a multi-core processor, acting as a coprocessor unit for the main controller. The node status information of the on-chip temporary storage dynamic scheduling unit comes from the node status RAM (stat_ram) inside the multi-core processor's main controller. This node status RAM collects status information from all nodes in the state network, which can be used to determine whether any node is in an idle or busy state.

[0040] The on-chip temporary dynamic scheduling unit (PDSU) provides the basis for decision-making logic and mapping units by reading node state RAM. Additionally, the control words input to the PDSU come from the execution unit (EXU) of the main controller. Under program control, the EXU accesses the PDSU and writes control words during the task write-back and task launch phases. The PDSU outputs primarily the coordinates of the temporary node and the coordinates of the successor node. The coordinates of the temporary node are sent through the main controller's configuration network (cfg_net) interface during the task write-back phase, and the coordinates of the successor node are sent through the configuration network during the subsequent task launch phase.

[0041] The on-chip temporary dynamic scheduling unit includes a scheduler and a mapping unit.

[0042] The scheduler maintains the storage node state table and schedules task-generated data using an on-chip temporary storage mechanism based on the storage node state table. The scheduler includes a resource state table and an adjudication logic module. The resource state table is connected to the execution unit in the multi-core processor's main controller. The execution unit is a simple CPU core with a five-stage pipeline architecture, capable of generating control words with memory address mappings to operate on the resource state table. The resource state table obtains the control words through the execution unit. The control words mainly include the opcode (funct) for the resource state table, the task ID (task_id), the input temporary storage node (temp_node_pos_i), and other auxiliary signals.

[0043] The adjudication logic module is connected to the resource status table and the status network interface of the multi-core processor main controller. The adjudication logic module obtains the node status through the status network interface. The node status information comes from the feedback of the multi-core processor status network and is mainly used by the adjudication logic module to determine whether to enable on-chip temporary storage.

[0044] The mapping unit is connected to the state network interface and the adjudication logic module respectively, and is used to calculate the coordinates of the functional node that will execute the subsequent task on the network based on the node status.

[0045] The on-chip scratchpad network comprises the cache spaces of all functional nodes on the heterogeneous multi-core processor. The 2D-mesh topology formed by these cache spaces constitutes the on-chip scratchpad distributed network. The storage space within each functional node is also called a storage node, and this storage space is RAM generated by Verilog HDL. The cache spaces can be accessed by a unified memory access protocol supported by the on-chip data network; that is, they can serve not only as buffers for computational data on the corresponding heterogeneous functional nodes but also as a homogeneous storage node network for unified management by the scheduler.

[0046] As attached Figure 3As shown, in one embodiment, the functional nodes are all deployed on a grid-like network, which comprises three layers: a data network, a configuration network, and a status network. The data network is composed of interconnected routing nodes. A routing node can receive input from any five directions (east, south, west, north, and local) and forward it to any of the remaining four directions, with the forwarding coordinates depending on the destination node coordinates indicated in the header of the data stream.

[0047] One of the characteristics of on-chip temporary storage networks is the homogeneity of their storage nodes. Therefore, several storage nodes can be merged; in one embodiment, they can be combined into a large-capacity segment with a capacity of up to 16K*64bits, or a portion of the space of a storage node can be allocated into a small-capacity segment of 256*64bits.

[0048] The storage node status table maintains current computation task information when on-chip temporary storage is enabled. It's worth noting that networks with different computing resources can add or delete some task information based on their own needs. Task information includes a task ID, which is the scheduler's input to the storage node status table and serves as a tag used internally by the scheduler to manage the computation process. Each task tag uniquely corresponds to a specific storage node's occupancy and release cycle. It's important to note that in the resource status table, for a given storage node, its capacity segment corresponds to only one storage node coordinate, but may correspond to multiple task IDs because the computation results of multiple tasks may be stored in the same node. The operating mode is the scheduler's input to the storage node status table, used to determine whether to perform a release or occupancy operation on the status table. The size of the data that this task needs to temporarily store is also included. The scheduler's input to the storage node status table determines which capacity segment should be used for temporary storage. The coordinates of the newly occupied temporary storage node in this task are the scheduler's input to the storage node status table, used to store the calculated temporary storage node in the status table. The coordinates of the temporary storage node to be released in this task are the storage node status table's output to the scheduler, used to notify the scheduler from which storage node to retrieve the temporary data. The number of occupied capacity segments, including the number of occupied 16K, 8K, 4K, 2K, 1K, 0.5K, and 0.25K capacity segments, is the storage node status table's output to the scheduler, used by the scheduler to determine whether there are enough storage segments to enable the on-chip temporary storage mechanism.

[0049] During the idle node search phase, the scheduler determines, based on its internal storage node status table, whether the output data stream of a functional unit should be written back to off-chip memory (which has lower memory access efficiency) or to the on-chip temporary storage network (which has lower access latency). If it decides to write to the on-chip temporary storage network, the scheduler will, depending on the situation, instruct the functional node to store the calculation result locally or transfer the calculation result to the internal storage of other functional units.

[0050] During the write-back management phase, if on-chip temporary storage was previously enabled, the scheduler notifies the mapping unit to generate the coordinates of the functional nodes that will execute subsequent computation tasks. In order to output the temporary storage data to the subsequent functional units as soon as possible to reduce on-chip network congestion, the mapping unit uses the Nearest Neighbor Algorithm to select the coordinates of the subsequent functional nodes. The distance specified by the Nearest Neighbor Algorithm is based on the Manhattan Distance.

[0051] When a task needs to use data located on an on-chip storage node as source data, the mapping unit will assign a relatively nearest functional node to the task. The selection of this node is based on the minimum Manhattan distance, which is calculated in a grid-like topology data network as follows: where node i has coordinates (X1, Y1), and node j has coordinates (X2, Y2):

[0052] d(i,j)=|X1-X2|+|Y1-Y2|

[0053] The mapping unit calculates the coordinates of all idle functional nodes within the on-chip network capable of executing the current task, and uses these coordinates as the destination coordinates for temporary data storage. This design ensures that data released by storage nodes reaches subsequent task functional nodes as quickly as possible, effectively reducing the likelihood of network congestion.

[0054] As attached Figure 4 As shown, the scheduler can make a series of decisions regarding the on-chip temporary storage mechanism based on the information provided by the resource status table and the current task information.

[0055] First, the scheduler should determine whether the on-chip temporary storage mechanism needs to be activated.

[0056] The on-chip temporary storage mechanism requires the following two conditions to be met simultaneously:

[0057] a. The data stream size of the task is less than or equal to the remaining space size of a certain capacity segment;

[0058] b. The data generated by this task needs to be received by the successor node within the predetermined time.

[0059] Condition a is determined by monitoring the status table of the storage nodes inside the scheduler.

[0060] Condition b sets a predetermined time to prevent temporary data from remaining on the storage node for too long, thus preventing the corresponding functional node from performing calculations. To address this, the scheduler implements several timers to manage the current task. When a timer expires and the number of functional nodes of that type is insufficient, the on-chip temporary data should be written back to off-chip storage to prevent the storage node from being occupied and causing prolonged blocking of the functional node.

[0061] Secondly, the scheduler needs to determine whether a new storage node needs to be created;

[0062] If the on-chip temporary storage mechanism is activated, the scheduler needs to decide whether to allocate a new storage node for temporary storage or to transfer the computational data to the remaining capacity segments within an already allocated storage node. The scheduler employs a method that dynamically divides storage nodes into different capacity segments while dynamically releasing occupied capacity segments, achieving unified management of storage nodes. For example, if the on-chip network specifies a maximum of two 8K capacity segments and a maximum of four 4K capacity segments globally, then clearly multiple partitioning methods can be generated.

[0063] The decision-making principles for this partitioning are described in the following step-by-step order:

[0064] During system initialization, the possible capacity fragment sizes for storage nodes are defined. The maximum allowed capacity fragment size is denoted as N. max The next largest capacity segment is denoted as N. max-1 And so on, with the smallest capacity segment size denoted as N0. If a storage node is called N... i If the type is (i∈[0,max]), it means that the size of each capacity segment within this storage node is N. i .

[0065] a. The on-chip temporary storage occupancy mechanism is activated, proceeding to step b;

[0066] b. Divide the data into capacity segments based on the current data volume generated by the calculation. If the data volume generated by the current node is between N... i With N i+1 Then proceed to step c.

[0067] c. Determine if any storage nodes have already been partitioned into N. i If the type exists, proceed to step d; otherwise, proceed to step e.

[0068] d. Determine if the node has any free capacity segments. If so, store the data calculated in this operation into the free capacity segment and execute step f. If not, execute step e.

[0069] e. Divide the node that generated the calculation data into Ni-type capacity segments and execute step f;

[0070] f. The on-chip temporary storage mechanism is in effect; exit.

[0071] In one embodiment, taking the dynamic partitioning of two 8K capacity segments as an example, task A stores its generated 8K data in the storage node Node1 corresponding to the functional node executing task A. This causes Node1 to be marked by the scheduler as a storage area containing two 8K capacity segments, which is equivalent to allocating Node1 as a new storage node. Simultaneously, task B drives another functional node to generate 6K of computational data. Since its length is between a 4K and an 8K capacity segment, the computation result of task B is forwarded to the remaining 8K capacity segment of Node1. At the same time, the scheduler will mark the successor task B as a task seeking source data from the second 8K capacity segment of Node1, i.e., perform a source data redirection operation. If these two 8K capacity segments are not yet released, other tasks generating data of the corresponding size will have to write back to off-chip storage. Other capacity partitioning schemes follow the same principles.

[0072] In one embodiment, the process of occupying and releasing a task is described in sequence. The actions performed by the storage node status table can be described as the following code (working mode funct is 1 for occupying, 2 for releasing, and other values ​​are invalid):

[0073] Resource status table occupancy process:

[0074] a. If the current working mode funct is 1 and the data size is 8K, proceed to step b; otherwise, this step is invalid.

[0075] b. Determine if the value of the temp_8K_cnt register, which indicates the number of currently occupied 8K capacity segments, has reached its upper limit. If it has, proceed to step f; otherwise, proceed to step c.

[0076] c. If the resource status table has already recorded the node divided into 8K capacity segments, proceed to step d. Otherwise, store the coordinates of the newly occupied temporary node (Temp_node_pos_i) in the temporary coordinate register group, and store the task ID (Task_id) in the 8K type task ID register group in the resource status table. Proceed to step e;

[0077] d. Store the task ID (Task_id) into the task ID register group that manages the "recorded node". Since the coordinates of the "recorded node" have already been saved, the temporary node coordinates (Temp_node_pos_i) newly occupied by this task are not needed for this task. Then execute step e.

[0078] e. Increment the number of occupied 8K capacity segments in the temp_8K_cnt register by 1, and then proceed to step f;

[0079] f. Exit the current operation on the resource status table.

[0080] Resource status table release process:

[0081] a. If the current working mode funct is 2, proceed to step b; otherwise, this step is invalid.

[0082] b. Based on the input task ID, output the coordinates of the temporary node to be released by this task to the temporary node coordinate release port (Src_temp_pos_o), and execute step c;

[0083] c. Exit the current operation on the resource status table.

[0084] The on-chip temporary storage dynamic scheduling unit is characterized by its internal scheduler being able to make a series of decisions regarding the on-chip temporary storage mechanism based on information provided by the resource status table and current task information.

[0085] In one embodiment, a two-dimensional MESH 15*9 on-chip network structure is implemented on the Xilinx development board Vertex6240t using Verilog HDL, and its hardware resource consumption is shown in Table 1.

[0086] Resource types LUTs FF Pairs Registers Resource consumption 16733 12437 8599

[0087] Table 1

[0088] This invention is not limited to the specific embodiments described above. Those skilled in the art can implement this invention using various other specific embodiments based on the disclosed content of the embodiments and accompanying drawings. Therefore, any design that adopts the design structure and concept of this invention and makes some simple changes or modifications falls within the protection scope of this invention.

Claims

1. An on-chip temporary storage dynamic scheduling manager for a multi-core heterogeneous processor, wherein the on-chip temporary storage dynamic scheduling manager is disposed on the multi-core processor, characterized in that, The on-chip temporary storage dynamic scheduling manager includes an on-chip temporary storage dynamic scheduling unit and an on-chip temporary storage network; The on-chip temporary storage dynamic scheduling unit is deployed within the main controller of the multi-core processor and works as a coprocessor unit of the main controller. The on-chip temporary storage dynamic scheduling unit includes a scheduler and a mapping unit. The scheduler is used to maintain the storage node status table and schedule the data generated by the task according to the storage node status table using the on-chip temporary storage mechanism. The mapping unit is used to calculate the coordinates of the functional node that performs the subsequent task on the network; The mapping unit assigns a relatively nearest functional node to a task that needs to use data located on an on-chip storage node as source data based on the minimum Manhattan distance. The minimum Manhattan distance is expressed as: d(i,j)=|X1-X2|+|Y1-Y2| Where node i has coordinates (X1, Y1) and node j has coordinates (X2, Y2); The on-chip temporary storage network includes storage nodes among all functional nodes on the multi-core processor, and the storage nodes form a 2D-mesh topology network.

2. The on-chip temporary storage dynamic scheduling manager for a multi-core heterogeneous processor according to claim 1, characterized in that, The storage node status table maintains the task information of the current computation when on-chip temporary storage is enabled; The task information includes the task ID, working mode, the amount of data that needs to be temporarily stored for this task, the coordinates of the newly occupied temporary storage nodes for this task, and the number of capacity segments already occupied. The task ID is the input of the scheduler to the storage node status table. The capacity segment contained in a storage node corresponds to only one storage node coordinate, but may correspond to multiple task IDs. The operating mode is the scheduler's input to the storage node's status table, used to determine whether to perform a release or occupation operation on the status table. The amount of data that needs to be temporarily stored for this task is the scheduler's input to the storage node status table, used to determine which capacity segment should be used for the current temporary storage. The coordinates of the temporary storage node to be released in this task are the output of the storage node status table to the scheduler, which is used to notify the scheduler which storage node to retrieve the temporary data from. The number of occupied capacity segments records the number of occupied capacity segments of various sizes, which the scheduler uses to determine whether there are enough storage segments to enable the on-chip temporary storage mechanism.

3. The on-chip temporary storage dynamic scheduling manager for a multi-core heterogeneous processor according to claim 1, characterized in that, The scheduler can determine whether to activate the on-chip temporary storage mechanism based on the information provided by the resource status table and the current task information. The execution unit is a simple CPU core that can generate control words that map memory addresses to operate on the resource status table; The resource status table obtains the control word through the execution unit. The control word mainly includes the opcode for the resource status table, the task ID, and the input temporary storage node. The scheduler initiates the on-chip temporary storage mechanism when the following conditions are met simultaneously: The data stream size of the task is less than or equal to the remaining space size of a certain capacity segment; The data generated by the task needs to be received by the successor node within a predetermined time.

4. The on-chip temporary storage dynamic scheduling manager for a multi-core heterogeneous processor according to claim 3, characterized in that, The scheduler can determine whether a new storage node needs to be created. Based on the current data volume generated by the calculation, the data is divided into capacity segments. It is determined whether any storage nodes have been divided into nodes of the corresponding type. If there are nodes of the corresponding type, it is determined whether the node has a free capacity segment. If there is a free segment, the data calculated in this calculation is stored in the free capacity segment. If no storage node has been allocated to the corresponding type or if no free capacity segment exists for the corresponding type of node, then the node that generated the data for this calculation will be allocated to the corresponding capacity segment.

Images