Warp-aware memory controller

Abstract

The present invention discloses a memory controller based on perceptual thread bundles, belonging to the technical field of GPGPU memory control and scheduling. The technical problem to be solved by the present invention is how to realize that multiple thread requests in the same thread bundle can be processed simultaneously, and to avoid thread divergence delays that may be caused by cross scheduling between different thread bundles. The adopted technical scheme is: including a transaction scheduler, a thread bundle sorting module, a bank scoring module and a command scheduler; the bank scoring module is used to calculate the score of a thread bundle based on the type of request and the number of requests in the bank command queue; the thread bundle sorting module is used to optimize the memory access request of the thread bundle, and sort the thread bundle according to the attributes of the thread bundle and the score of the thread bundle; the transaction scheduler is used to merge the thread requests in the same thread bundle, and put the thread bundle request with the lowest score into the command queue in priority according to the score of the thread bundle.

Description

Technical Field

[0001] The invention relates to the technical field of GPGPU memory control and scheduling, in particular to a memory controller based on perceiving thread warps. Background Art

[0002] Memory merging technology means that requests from the same warp should be merged into cache line-sized requests based on the target address as much as possible, which helps to eliminate repeated access to the same cache line and reduce bandwidth requirements. However, if the data accessed by the threads in the warp is inconsistent in space, the merging technology will not work.

[0003] Requests from different streaming multiprocessors (SMs) are searched in the first and second level caches and then reach the memory controller through the cross-interconnect network; therefore, memory accesses between different thread warps are mixed together in the queue; the transaction scheduler transmits each independent thread request to the cache, and the caches with different occupancy may cause threads in the same thread warp to be processed at different time points, which may cause the thread warp to be blocked waiting for the memory request to complete, affecting the overall performance. In GPU and other parallel processing architectures, reducing latency divergence is the key to improving efficiency and performance.

[0004] In recent years, various parallel algorithms have been developed for GPGPU implementation in the fields of high-performance computing and finance. Compared with traditional GPGPU applications, memory access in these programs is usually data-dependent and often has less spatial locality. This obvious irregularity in memory access creates a large performance bottleneck. In existing GPGPU memory addressing technology, due to the reordering and cross-issuance between thread warps, the latency of different thread requests in the thread warp varies greatly. Especially in the SIMT architecture, the thread warp must wait for all thread requests in the memory to return before it can continue to execute. This latency difference can cause significant performance loss when running irregular GPGPU applications.

[0005] Therefore, how to ensure that multiple thread requests in the same thread warp can be processed simultaneously and avoid thread divergence delays that may be caused by cross-scheduling between different thread warps is a technical problem that needs to be solved urgently. Summary of the invention

[0006] The technical task of the present invention is to provide a memory controller based on thread warp perception to solve the problem of how to enable multiple thread requests in the same thread warp to be processed simultaneously and avoid thread divergence delays that may be caused by cross-scheduling between different thread warps.

[0007] The technical task of the present invention is achieved in the following manner: a memory controller based on perceptual warps, the memory controller comprising a transaction scheduler, a warp sorting module, a bank scoring module and a command scheduler;

[0008] The Bank scoring module is used to calculate the score of a thread warp based on the type of request and the number of requests in the Bank command queue; the score of a thread warp refers to the sum of the request scores of all threads in the thread warp; the request type is the number of row hits or the number of row misses;

[0009] The warp sorting module is a memory management mechanism used to optimize the memory access requests of warps, improve memory access efficiency and reduce latency, and sort warps according to the attributes of warps and the scores of warps;

[0010] The transaction scheduler is used to merge thread requests in the same warp and put the warp request with the lowest score into the command queue first according to the score of the warp; the warp request with the lowest score is the warp request with the shortest expected execution time;

[0011] The command scheduler is used to detect different bank buffers in the command queue in turn according to the polling scheduling strategy. If the corresponding bank buffer is empty, it will be skipped; if the corresponding bank buffer has a DRAM instruction to be sent, the instruction will be read and sent to DRAM (Dynamic Random Access Memory).

[0012] Preferably, the warp sorting module includes SM_id (stream multiprocessor identifier), Warp_id (warp identifier), Score (score value) and Row_id (row identifier);

[0013] Wherein, SM_id represents the unique identifier of the streaming multiprocessor assigned to each thread warp; in a multiprocessor system, each streaming multiprocessor (Streaming Multiprocessor, referred to as SM) executes multiple thread warps, and SM_id is used to distinguish the streaming multiprocessors;

[0014] Warp_id represents the unique identifier assigned to the thread warp in each streaming multiprocessor, which is used to distinguish different thread warps in the same streaming multiprocessor;

[0015] Score is used to indicate whether all warp requests have been sent to the memory controller. When the warp sorting module receives the marking signal of the last request in the warp request, it writes the value 0 into the Score field.

[0016] Row_id represents the identifier of the memory row associated with the thread warp request; in a memory system, data is usually organized by rows, and Row_id helps to identify and track access requests to a specific memory row.

[0017] Preferably, the warp sorting module includes 128 entries, each entry corresponding to a unique<SM_id,Warp_id> The combination is used to distinguish different warps and track the access addresses of threads in the corresponding warps.

[0018] Preferably, in the Bank scoring module, an independent entry is allocated to each Bank to record the thread bundles and their memory requests that have not yet been processed on the corresponding Bank. Each thread bundle to be processed has a corresponding Bank score, and the Bank score is used to reflect the expected delay required to complete all thread requests of the corresponding thread bundle; each time the read-write queue receives a new request or the transaction scheduler extracts a thread bundle, the Bank score will be updated.

[0019] Preferably, the transaction scheduler reads and updates the Bank score in each cycle to ensure that the latest results sorted by score are obtained, and the Bank with the highest score indicates that the corresponding thread warp request requires the longest completion time;

[0020] When it is confirmed that all thread requests of a warp have been completely transmitted to the read and write queues of the global memory controller, the transaction scheduler selects the warp with the lowest score and sends the corresponding warp to the command queue.

[0021] Preferably, the transaction scheduler is used to verify whether the transmission of the thread warp is complete, specifically: marking the last request sent by the current thread warp to the memory controller; when the thread warp sorting module receives the last request of the thread warp, updating the score value of the thread warp to ensure the integrity and accuracy of the thread warp transmission.

[0022] Preferably, the scheduling mechanism of the transaction scheduler is as follows:

[0023] Each thread warp is assigned a Bank score, which reflects the time required to process the corresponding thread warp. The lower the Bank score, the higher the priority.

[0024] If the request of the current thread bundle has been sent to the command queue, the transaction scheduler starts looking for the next thread bundle to be scheduled; at this time, the transaction scheduler selects the thread bundle with the lowest score, that is, the task with the shortest estimated completion time;

[0025] If there are warps with the same score, the transaction scheduler will give priority to the warp with the highest row hit count to reduce DRAM power consumption.

[0026] More preferably, the score of a warp is calculated based on the type of request and the number of requests in the bank command queue as follows:

[0027] The score of a corresponding request is determined based on whether the request is hit or miss in the memory bank at the time of final scheduling, that is, whether the last scheduled request has a matching row address with the current request, as follows:

[0028] If the request is successful, the corresponding thread warp's request gets 1 point;

[0029] If the request is not hit, the request of the corresponding thread warp gets 3 points;

[0030] After assigning a score based on DRAM array access latency to the warp's request, the score for queuing latency is added, which is calculated by accumulating the scores of all pending requests in the corresponding Bank command queue;

[0031] Whenever a new request is added or an existing request is scheduled into the Bank command queue, the score of the corresponding warp is updated.

[0032] Preferably, the Bank scoring module includes Bank_N, Warp_N, Row_N and Bank_N score;

[0033] Where, Bank_N represents different banks in the memory. N in Bank_N is a variable, which takes any number from 0 to N, indicating the Nth Bank in the system. Bank is a basic unit in the memory for storing data.

[0034] Warp_N represents the unique identifier of the warp associated with each bank. The N in Warp_N represents a different warp number, which is used to distinguish different warps.

[0035] Row_N represents the memory access request of different rows in the Bank; N in row_N is a variable, starting from 1 and going up to N, representing the Nth row in the Bank, each row storing a corresponding amount of data;

[0036] The Bank_N score indicates the score associated with a specific bank. The score associated with a bank reflects the expected latency required to process all warp requests on the corresponding memory bank. The lower the score, the faster the requests on the corresponding memory bank are completed.

[0037] Preferably, the working process of the memory control is as follows:

[0038] (1) Request reception and request queuing: The memory controller receives memory access requests from different thread warps and sends read requests and write requests to the corresponding read request queue and write request queue;

[0039] (2) Thread bundle sorting: The thread bundle sorting module sorts the thread bundles according to their attributes and scoring results;

[0040] (3) Scoring and selection: The transaction scheduler selects the thread bundle with the lowest score based on the score provided by the Bank scoring module and sends it to the command queue to wait for execution;

[0041] (4) Command Scheduling: The command scheduler extracts memory access commands from the bank buffer queue and sends them to DRAM;

[0042] (5) Data transmission: DRAM executes memory access commands to read or write data;

[0043] (6) Result return: After the execution is completed, the execution result is returned to the thread bundle that initiated the request;

[0044] (7) Status update: The memory controller updates the memory status information, which includes changes to memory data and updates to queue status;

[0045] (8) Feedback loop: Adjust thread warp order and command scheduling based on execution results and memory status.

[0046] The memory controller based on warp awareness of the present invention has the following advantages:

[0047] (i) The working process of the memory controller of the present invention is a dynamic cycle. The memory controller continuously receives requests from new thread warps, schedules and executes according to the requests from the new thread warps, and adjusts corresponding operations according to the memory status and feedback information to achieve optimal memory access performance;

[0048] (ii) The present invention merges thread requests in the same thread warp and introduces a new thread warp sorting module and a bank scoring module. The transaction scheduler will prioritize the thread warp request with the lowest score, i.e., the thread warp request with the shortest expected execution time, into the command queue according to the scores provided by the thread warp sorting module and the bank scoring module, so that multiple thread requests in the same thread warp can be processed simultaneously, thereby avoiding thread divergence delays that may be caused by cross-scheduling between different thread warps and reducing adverse effects on performance;

[0049] (III) The Bank scoring module of the present invention includes Bank_N, thread warp_N, row_N and Bank_N score, which can score and sort the thread warp requests on each storage body, thereby helping the transaction scheduler to select the most appropriate thread warp for memory access, so as to optimize memory access efficiency and reduce latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The present invention is further described below in conjunction with the accompanying drawings.

[0051] Attached Figure 1 It is a schematic diagram of the structure of the memory controller based on the awareness thread warp;

[0052] Attached Figure 2 It is a structural diagram of the thread warp sorting module;

[0053] Attached Figure 3 This is a structural diagram of the Bank scoring module. DETAILED DESCRIPTION

[0054] The warp-aware memory controller of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] Example: As shown in the attached Figure 1 As shown, this embodiment provides a memory controller based on perceptual warps, and the memory controller includes a transaction scheduler, a warp sorting module, a bank scoring module and a command scheduler;

[0056] The Bank scoring module is used to calculate the score of a thread warp based on the type of request and the number of requests in the Bank command queue; the score of a thread warp refers to the sum of the request scores of all threads in the thread warp; the request type is the number of row hits or the number of row misses;

[0057] The warp sorting module is a memory management mechanism used to optimize the memory access requests of warps, improve memory access efficiency and reduce latency, and sort warps according to the attributes of warps and the scores of warps;

[0058] The transaction scheduler is used to merge thread requests in the same warp and put the warp request with the lowest score into the command queue first according to the score of the warp; the warp request with the lowest score is the warp request with the shortest expected execution time;

[0059] The command scheduler is used to detect different bank buffers in the command queue in turn according to the polling scheduling strategy. If the corresponding bank buffer is empty, it will be skipped; if the corresponding bank buffer has a DRAM instruction to be sent, the instruction will be read and sent to DRAM (Dynamic Random Access Memory).

[0060] As attached Figure 2 As shown, the warp sorting module in this embodiment includes SM_id (stream multiprocessor identifier), Warp_id (warp identifier), Score (score value) and Row_id (row identifier);

[0061] Wherein, SM_id represents the unique identifier of the streaming multiprocessor assigned to each thread warp; in a multiprocessor system, each streaming multiprocessor (Streaming Multiprocessor, referred to as SM) executes multiple thread warps, and SM_id is used to distinguish the streaming multiprocessors;

[0062] Warp_id represents the unique identifier assigned to the thread warp in each streaming multiprocessor, which is used to distinguish different thread warps in the same streaming multiprocessor;

[0063] Score is used to indicate whether all warp requests have been sent to the memory controller. When the warp sorting module receives the marking signal of the last request in the warp request, it writes the value 0 into the Score field.

[0064] Row_id represents the identifier of the memory row associated with the thread warp request; in a memory system, data is usually organized by rows, and Row_id helps to identify and track access requests to a specific memory row.

[0065] The thread warp sorting module in this embodiment includes 128 entries, each of which corresponds to a unique<SM_id,Warp_id> The combination is used to distinguish different warps and track the access addresses of threads in the corresponding warps.

[0066] In this embodiment, in the Bank scoring module, an independent entry is allocated to each Bank to record the thread bundles and their memory requests that have not yet been processed on the corresponding Bank. Each thread bundle to be processed has a corresponding Bank score, and the Bank score is used to reflect the expected delay required to complete all thread requests of the corresponding thread bundle; each time the read-write queue receives a new request or the transaction scheduler extracts a thread bundle, the Bank score will be updated.

[0067] The transaction scheduler in this embodiment reads and updates the Bank score in each cycle to ensure that the latest results sorted by score are obtained. The Bank with the highest score indicates that the corresponding thread warp request requires the longest completion time.

[0068] When it is confirmed that all thread requests of a warp have been completely transmitted to the read and write queues of the global memory controller, the transaction scheduler selects the warp with the lowest score and sends the corresponding warp to the command queue.

[0069] The transaction scheduler in this embodiment is used to verify whether the transmission of the thread warp is complete, specifically: marking the last request sent by the current thread warp to the memory controller; when the thread warp sorting module receives the last request of the thread warp, updating the score value of the thread warp to ensure the integrity and accuracy of the thread warp transmission.

[0070] The scheduling mechanism of the transaction scheduler in this embodiment is as follows:

[0071] Each thread warp is assigned a Bank score, which reflects the time required to process the corresponding thread warp. The lower the Bank score, the higher the priority.

[0072] If the request of the current thread bundle has been sent to the command queue, the transaction scheduler starts looking for the next thread bundle to be scheduled; at this time, the transaction scheduler selects the thread bundle with the lowest score, that is, the task with the shortest estimated completion time;

[0073] If there are warps with the same score, the transaction scheduler will give priority to the warp with the highest row hit count to reduce DRAM power consumption.

[0074] In this embodiment, the score of a warp is calculated based on the request type and the number of requests in the Bank command queue as follows:

[0075] The score of a corresponding request is determined based on whether the request is hit or miss in the memory bank at the time of final scheduling, that is, whether the last scheduled request has a matching row address with the current request, as follows:

[0076] If the request is successful, the corresponding thread warp's request gets 1 point;

[0077] If the request is not hit, the request of the corresponding thread warp gets 3 points;

[0078] After assigning a score based on DRAM array access latency to the warp's request, the score for queuing latency is added, which is calculated by accumulating the scores of all pending requests in the corresponding Bank command queue;

[0079] Whenever a new request is added or an existing request is scheduled into the Bank command queue, the score of the corresponding warp is updated.

[0080] As attached Figure 3 As shown, the Bank scoring module in this embodiment includes Bank_N, Warp_N, Row_N and Bank_N score;

[0081] Where, Bank_N represents different banks in the memory. N in Bank_N is a variable, which takes any number from 0 to N, indicating the Nth Bank in the system. Bank is a basic unit in the memory for storing data.

[0082] Warp_N represents the unique identifier of the warp associated with each bank. The N in Warp_N represents a different warp number, which is used to distinguish different warps.

[0083] Row_N represents the memory access request of different rows in the Bank; N in row_N is a variable, starting from 1 and going up to N, representing the Nth row in the Bank, each row storing a corresponding amount of data;

[0084] The Bank_N score indicates the score associated with a specific bank. The score associated with a bank reflects the expected latency required to process all warp requests on the corresponding memory bank. The lower the score, the faster the requests on the corresponding memory bank are completed.

[0085] The working process of memory control in this embodiment is as follows:

[0086] (1) Request reception and request queuing: The memory controller receives memory access requests from different thread warps and sends read requests and write requests to the corresponding read request queue and write request queue;

[0087] (2) Thread bundle sorting: The thread bundle sorting module sorts the thread bundles according to their attributes and scoring results;

[0088] (3) Scoring and selection: The transaction scheduler selects the thread bundle with the lowest score based on the score provided by the Bank scoring module and sends it to the command queue to wait for execution;

[0089] (4) Command Scheduling: The command scheduler extracts memory access commands from the bank buffer queue and sends them to DRAM;

[0090] (5) Data transmission: DRAM executes memory access commands to read or write data;

[0091] (6) Result return: After the execution is completed, the execution result is returned to the thread bundle that initiated the request;

[0092] (7) Status update: The memory controller updates the memory status information, which includes changes to memory data and updates to queue status;

[0093] (8) Feedback loop: Adjust thread warp order and command scheduling based on execution results and memory status.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A memory controller based on thread warp awareness, characterized in that: The memory controller includes a transaction scheduler, a thread warp sorting module, a bank scoring module and a command scheduler; The Bank scoring module is used to calculate the score of a warp based on the request type and the number of requests in the Bank command queue; the score of a warp refers to the sum of the request scores of all threads in the warp; the request type is the number of row hits or the number of row misses; The warp sorting module is used to optimize the memory access requests of warps and sort the warps according to the attributes of the warps and the scores of the warps; The transaction scheduler is used to merge thread requests in the same warp and put the warp request with the lowest score into the command queue first according to the score of the warp; the warp request with the lowest score is the warp request with the shortest expected execution time; The command scheduler is used to detect different bank buffers in the command queue in turn according to the polling scheduling strategy. If the corresponding bank buffer is empty, it will be skipped; if the corresponding bank buffer has a DRAM instruction to be sent, the instruction will be read and sent to DRAM; In the bank scoring module, an independent entry is allocated to each bank to record the thread warps and their memory requests that have not been processed on the corresponding bank. Each thread warp to be processed has a corresponding bank score, which is used to reflect the expected delay required to complete all thread requests of the corresponding thread warp. Whenever the read / write queue receives a new request or the transaction scheduler extracts a thread warp, the bank score will be updated. The score of a warp is calculated based on the type of request and the number of requests in the bank command queue as follows: The score of a corresponding request is determined based on whether the request is hit or miss in the memory bank at the time of final scheduling, that is, whether the last scheduled request has a matching row address with the current request, as follows: If the request is successful, the corresponding thread warp's request gets 1 point; If the request is not hit, the request of the corresponding thread warp gets 3 points; After assigning a score based on DRAM array access latency to the warp's request, the score for queuing latency is added, which is calculated by accumulating the scores of all pending requests in the corresponding Bank command queue; Whenever a new request is added or an existing request is scheduled into the Bank command queue, the score of the corresponding warp is updated.

2. The warp-aware memory controller according to claim 1, characterized in that: The thread warp sorting module includes SM_id, Warp_id, Score and Row_id; Wherein, SM_id represents a unique identifier of a stream multiprocessor assigned to each thread warp; in a multiprocessor system, each stream multiprocessor executes multiple thread warps, and SM_id is used to distinguish the stream multiprocessors; Warp_id represents the unique identifier assigned to the thread warp in each streaming multiprocessor, which is used to distinguish different thread warps in the same streaming multiprocessor; Score is used to indicate whether all warp requests have been sent to the memory controller. When the warp sorting module receives the marking signal of the last request in the warp request, it writes the value 0 into the Score field. Row_id represents the identifier of the memory row associated with the warp request; Row_id helps to identify and track access requests to a specific memory row.

3. The warp-aware memory controller according to claim 1 or 2, characterized in that: The thread warp sorting module includes 128 entries, each of which corresponds to a<SM_id,Warp_id> The combination is used to distinguish different warps and track the access addresses of threads in the corresponding warps.

4. The warp-aware memory controller according to claim 1, wherein: The transaction scheduler reads and updates the Bank score in each cycle to ensure that the latest results sorted by score are obtained. The Bank with the highest score indicates that the corresponding thread warp request requires the longest completion time; When it is confirmed that all thread requests of a warp have been completely transmitted to the read and write queues of the memory controller, the transaction scheduler selects the warp with the lowest score and sends the corresponding warp to the command queue.

5. The warp-aware memory controller according to claim 4, characterized in that: The transaction scheduler is used to verify whether the transmission of the thread warp is complete, specifically: marking the last request sent by the current thread warp to the memory controller; when the thread warp sorting module receives the last request of the thread warp, updating the score value of the thread warp.

6. The warp-aware memory controller according to claim 1, wherein: The scheduling mechanism of the transaction scheduler is as follows: Each thread warp is assigned a Bank score, which reflects the time required to process the corresponding thread warp. The lower the Bank score, the higher the priority. If the request of the current thread bundle has been sent to the command queue, the transaction scheduler starts looking for the next thread bundle to be scheduled; at this time, the transaction scheduler selects the thread bundle with the lowest score, that is, the task with the shortest estimated completion time; If there are warps with the same score, the transaction scheduler will give priority to the warp with the highest row hit count to reduce DRAM power consumption.

7. The warp-aware memory controller according to claim 1, wherein: The Bank scoring module includes Bank_n, Warp_n, Row_n, and Bank_n score; Among them, Bank_n represents different banks in the memory; n in Bank_n is a variable, which takes any number from 0 to N, indicating the nth bank in the system; Bank is a basic unit in the memory for storing data; Thread bundle_n represents the unique identifier of the thread bundle associated with each bank; the n in thread bundle_n represents a different thread bundle number, which is used to distinguish different thread bundles; Row_n represents the memory access request of different rows in the Bank; n in row_n is a variable, starting from 1 and going up to N, indicating the nth row in the Bank, each row stores a corresponding amount of data; The Bank_n score indicates the score associated with a bank, which reflects the expected latency required to process all warp requests on the corresponding memory bank; a lower score indicates that requests on the corresponding memory bank are completed faster.

8. The warp-aware memory controller according to claim 1, wherein: The working process of this memory control is as follows: (1) Request reception and request queuing: The memory controller receives memory access requests from different thread warps and sends read requests and write requests to the corresponding read request queue and write request queue; (2) Thread bundle sorting: The thread bundle sorting module sorts the thread bundles according to their attributes and scoring results; (3) Scoring and selection: The transaction scheduler selects the thread bundle with the lowest score based on the score provided by the Bank scoring module and sends it to the command queue to wait for execution; (4) Command Scheduling: The command scheduler extracts memory access commands from the bank buffer queue and sends them to DRAM; (5) Data transmission: DRAM executes memory access commands to read or write data; (6) Result return: After the execution is completed, the execution result is returned to the thread bundle that initiated the request; (7) Status update: The memory controller updates the memory status information, which includes changes to memory data and updates to queue status; (8) Feedback loop: Adjust thread warp order and command scheduling based on execution results and memory status.

Images