Data access processing method and apparatus for GPU, and storage medium
By designing the bridge module and scheduling module inside the GPU, and optimizing the data access processing method, the problem of cross-partition data access delay between GPU modules is solved, system performance is improved, and order-saving operations are implemented.
Patent Information
- Application Number
- PCT/CN2024/094534
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-26
- Filing Date
- 2024-05-21
- Publication Date
- 2024-12-05
AI Technical Summary
The delay in accessing data across partitions between GPUs is too long, resulting in a degradation of system performance. The prior art does not provide an effective processing method.
A GPU data access processing method and device is designed, including a bridge module and a scheduling module, which outputs access requests through the requesting unit, performs hit tests, and reads data directly from the cache line in the hit situation, adopts lookup tables and statuses. Queue optimizes data access order and reduces delay.
It shortens the data delay of accessing scalar registers across partitions, improves GPU system performance, and supports order-saving operations, reducing resource consumption on the requesting side.
Smart Images

Figure CN2024094534_05122024_PF_FP_ABST
Abstract
Description
GPU data access processing method, device and storage medium
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This disclosure claims priority to Chinese patent application number 202310602802.9, filed with the China Patent Office on May 26, 2023, entitled “GPU Data Access Processing Method, Device and Storage Medium,” the entire contents of which are incorporated by reference into this disclosure. Technical Field
[0003] The present application relates to the field of computer technology, and in particular to a method, device, and storage medium for processing data access to a GPU. Background Art
[0004] As the application of Graphics Processing Unit (GPU) becomes more and more extensive, the performance requirements for GPU are also getting higher and higher.
[0005] GPUs typically consist of multiple modules. When these modules access data across partitions, if the distance between them is large, they may not meet certain timing requirements. Currently, to meet timing requirements for inter-module communication, flip-flops or first-input, first-out (FIFO) memories can be added between modules.
[0006] However, the above approach will result in a long delay between the time when the cross-partition modules send access requests and when they receive return data, which in turn leads to reduced system performance. Based on the above problem, a reasonable and effective solution has not yet been provided in the relevant art.
[0007] Summary of the Invention
[0008] In view of this, a data access processing method, device and storage medium for a GPU are proposed.
[0009] In a first aspect, an embodiment of the present application provides a data access processing device for a GPU, the device comprising: a bridge module and a scheduling module, the bridge module comprising a request unit and a scheduling multiplexing unit, the scheduling module comprising a scalar register (slot register, SRS);
[0010] The bridging module is configured to output a first access request through the request unit, where the first access request is used to indicate a data address of the SRS of the scheduling module across partitions to be accessed this time;
[0011] The bridging module is further configured to perform a hit test on the first access request through the scheduling multiplexing unit;
[0012] The bridge module is further configured to read and return data corresponding to the data address from the cache line of the scheduling module when the result of the hit test is a first test result, wherein the first test result indicates that the data address accessed this time is hit and the data address accessed historically is hit.
[0013] In a possible implementation, the bridging module is further configured to:
[0014] According to the data address of the current access, obtaining the index of the corresponding first cache line from the lookup table address queue;
[0015] Retrieving corresponding data from a lookup table data queue according to an index of the first cache line;
[0016] The acquired data is subjected to a beat operation and then returned, wherein the beat operation is used to indicate that the data is output after being delayed for k clock cycles, where k is a positive integer.
[0017] In another possible implementation, the bridging module is further configured to:
[0018] When the result of the hit test is the second test result, the data address of the current access is stored in a status queue, wherein the status queue is used to indicate that corresponding data is returned in sequence according to the access order;
[0019] The second test result indicates that the data address of the current access is hit and the data address of the historical access is not hit, or the data address of the current access is not hit.
[0020] In another possible implementation, the depth of the status queue is set according to a data cycle from outputting the first access request to returning data.
[0021] In another possible implementation, the status queue is used to store a missed data address and a second data address that is hit when the first data address is missed, where the first data address is a data address accessed before the second data address is accessed.
[0022] In another possible implementation, the update condition of the status queue includes:
[0023] When the earliest data address in the status queue is the same as the return address in the return address queue, the data in the status queue is updated.
[0024] In another possible implementation, the bridge module further includes an arbitrator, and the bridge module is further configured to:
[0025] determining an unlocked second cache line in a lock tag queue, wherein the lock tag queue indicates whether each cache line in the lookup table address queue is locked;
[0026] Updating the data address of the current access in the second cache line corresponding to the lookup table data queue, and updating the valid flag corresponding to the second cache line in the lookup table valid queue to the first flag, where the first flag is used to indicate that the data is invalid;
[0027] The first access request is sent to the arbitrator, and the arbitrator is used to select multiple cross-partition access requests.
[0028] In another possible implementation, the bridge module further includes a first-in-first-out (FIFO) memory, and the bridge module is further configured to:
[0029] A second access request is hit by the arbiter, where the second access request is one of the multiple cross-partition access requests;
[0030] sending the data address indicated by the second access request to the SRS in the scheduling module through the FIFO memory;
[0031] According to the data address of the returned data returned by the SRS, obtaining the index of the corresponding third cache line from the lookup table address queue;
[0032] According to the index of the third cache line, the returned data is updated in the third cache line of the lookup table data queue, and the valid flag corresponding to the third cache line in the lookup table valid queue is updated to a second flag, where the second flag is used to indicate that the data is valid.
[0033] In another possible implementation, the bridge module further includes a comparison unit, and the bridge module is further configured to:
[0034] Detecting the status queue, the lookup table address queue, and the lookup table valid queue by the comparison unit;
[0035] When the earliest accessed data address in the status queue is the same as the data address in the lookup table address queue, and the corresponding valid flag indicates that the data in the fourth cache line is valid, the data is read from the fourth cache line and returned.
[0036] In another possible implementation manner, the device is implemented by a hardware circuit.
[0037] In a second aspect, an embodiment of the present application provides a data access processing method for a GPU, the method comprising:
[0038] Outputting a first access request, where the first access request is used to indicate a data address of a cross-partition scalar register SRS to be accessed this time;
[0039] performing a hit test on the first access request;
[0040] When the result of the hit test is a first test result, data corresponding to the data address is read from the cache line and returned, and the first test result indicates that the data address accessed this time is a hit and the data address accessed previously is a hit.
[0041] In a possible implementation, when the result of the hit test is the first test result, reading and returning the data corresponding to the data address from the cache line includes:
[0042] According to the data address of the current access, obtaining the index of the corresponding first cache line from the lookup table address queue;
[0043] Retrieving corresponding data from a lookup table data queue according to an index of the first cache line;
[0044] The acquired data is subjected to a beat operation and then returned, wherein the beat operation is used to indicate that the data is output after being delayed for k clock cycles, where k is a positive integer.
[0045] In another possible implementation, the method further includes:
[0046] When the result of the hit test is the second test result, the data address of the current access is stored in a status queue, wherein the status queue is used to indicate that corresponding data is returned in sequence according to the access order;
[0047] The second test result indicates that the data address of the current access is hit and the data address of the historical access is not hit, or the data address of the current access is not hit.
[0048] In another possible implementation, the depth of the status queue is set according to a data cycle from outputting the first access request to returning data.
[0049] In another possible implementation, the status queue is used to store a missed data address and a second data address that is hit when the first data address is missed, where the first data address is a data address accessed before the second data address is accessed.
[0050] In another possible implementation, the update condition of the status queue includes:
[0051] When the earliest data address in the status queue is the same as the return address in the return address queue, the data in the status queue is updated.
[0052] In another possible implementation, when the result of the hit test is the second test result, after storing the data address of the current access in the status queue, the method further includes:
[0053] determining an unlocked second cache line in a lock tag queue, wherein the lock tag queue indicates whether each cache line in the lookup table address queue is locked;
[0054] Updating the data address of the current access in the second cache line corresponding to the lookup table data queue, and updating the valid flag corresponding to the second cache line in the lookup table valid queue to the first flag, where the first flag is used to indicate that the data is invalid;
[0055] The first access request is sent to an arbitrator, where the arbitrator is used to select multiple cross-partition access requests.
[0056] In another possible implementation, the method further includes:
[0057] A second access request is hit by the arbiter, where the second access request is one of the multiple cross-partition access requests;
[0058] sending the data address indicated by the second access request to the SRS;
[0059] According to the data address of the returned data returned by the SRS, obtaining the index of the corresponding third cache line from the lookup table address queue;
[0060] According to the index of the third cache line, the returned data is updated in the third cache line of the lookup table data queue, and the valid flag corresponding to the third cache line in the lookup table valid queue is updated to a second flag, where the second flag is used to indicate that the data is valid.
[0061] In another possible implementation, the method further includes:
[0062] Detecting the status queue, the lookup table address queue, and the lookup table valid queue;
[0063] When the earliest accessed data address in the status queue is the same as the data address in the lookup table address queue, and the corresponding valid flag indicates that the data in the fourth cache line is valid, the data is read from the fourth cache line and returned.
[0064] In another possible implementation manner, the method is implemented through a hardware circuit.
[0065] In a third aspect, an embodiment of the present application provides a data access processing device for a GPU, the device comprising:
[0066] processor;
[0067] a memory for storing processor-executable instructions;
[0068] The processor is configured to implement the method provided by the first aspect or any possible implementation manner of the first aspect when executing the instruction.
[0069] In a fourth aspect, an embodiment of the present application provides a non-volatile computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the method provided by the first aspect or any possible implementation of the first aspect.
[0070] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a computer device, the processor in the computer device executes the method provided in the first aspect or any possible implementation of the first aspect.
[0071] To summarize, the embodiment of the present application outputs a first access request, which is used to indicate the data address of the cross-partition SRS being accessed this time; performs a hit test on the first access request; when the result of the hit test is the first test result, reads and returns the data corresponding to the data address from the cache line, and the first test result indicates that the data address of the current access is hit and the data address of the historical access is hit; that is, a hardware circuit for a read cache (English: readcache) inside the GPU is provided, so that when the data address of the current access is hit and the data address of the historical access is hit, the data can be directly read and returned from the cache line, which greatly shortens the delay in accessing the data of the SRS across partitions, and can improve the system performance of the GPU when accessing data across partitions. BRIEF DESCRIPTION OF THE DRAWINGS
[0072] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the application and, together with the description, serve to explain the principles of the application.
[0073] FIG1 shows a schematic structural diagram of a GPU provided by an exemplary embodiment of the present application.
[0074] FIG2 shows a flowchart of a data access processing method for a GPU provided by an exemplary embodiment of the present application.
[0075] FIG3 shows a schematic structural diagram of a status queue provided by an exemplary embodiment of the present application.
[0076] FIG4 shows a schematic structural diagram of a lock mark queue provided by an exemplary embodiment of the present application.
[0077] FIG5 shows a schematic diagram of the structure of a lookup table address queue provided by an exemplary embodiment of the present application.
[0078] FIG6 shows a schematic structural diagram of a lookup table data queue provided by an exemplary embodiment of the present application.
[0079] FIG7 shows a schematic diagram of the structure of a valid queue of a lookup table provided by an exemplary embodiment of the present application.
[0080] FIG8 shows a flowchart of a data access processing method for a GPU provided by another exemplary embodiment of the present application.
[0081] FIG9 shows a schematic structural diagram of a data access processing device for a GPU provided by an embodiment of the present application. DETAILED DESCRIPTION
[0082] Various exemplary embodiments, features, and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.
[0083] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
[0084] In addition, numerous specific details are provided in the detailed description below to better illustrate the present application. Those skilled in the art will appreciate that the present application can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present application.
[0085] The embodiments of the present application provide a hardware circuit for reading cache data within the GPU. When the data address of the current access and the data address of the previous access are hit, the data can be directly read and returned from the cache line. This greatly reduces the latency of accessing SRS data across partitions, and can improve the system performance of the GPU when accessing data across partitions. In addition, the hardware circuit provided by the embodiments of the present application supports order-preserving operations, reducing the resource consumption of the requesting end to perform order-preserving functions.
[0086] First, the application scenarios involved in this application are introduced.
[0087] The present invention provides a data access processing method for a GPU, executed by a GPU. The GPU can utilize various interfaces and circuits to connect various components of a computer device. The GPU executes or runs software programs and / or modules stored in memory and accesses data stored in memory to perform various functions of the computer device and process data.
[0088] The GPU provided in an embodiment of the present application includes a hardware circuit for a read cache. The GPU comprises multiple different modules, including at least one bridge module and at least one scheduling module. The bridge module is configured to receive access requests to the scheduling module and receive data returned by the scheduling module. The scheduling module includes an SRS, which is used to store data.
[0089] Please refer to FIG1 , which shows a schematic structural diagram of a GPU provided by an exemplary embodiment of the present application.
[0090] The GPU's request side includes a bridge module 10, which contains five request units, or five master units 11. The GPU's return side includes four scheduling modules. For example, the five master units 11 include master0, master1, master2, master3, and master4. The four scheduling modules include scheduling module 0, scheduling module 1, scheduling module 2, and scheduling module 3.
[0091] Each scheduling module includes an SRS. The SRSs in the four scheduling modules store the same data. Setting four identical SRSs in the scheduling module within the GPU is intended to prevent concentrated access to a single SRS, which could lead to system performance degradation. It should be noted that the present embodiment does not limit the number of these modules.
[0092] Optionally, the bridge module 10 includes but is not limited to the following modules: a master unit 11 , an arbiter 12 (arb unit), a FIFO memory 13 and a comparison unit 14 .
[0093] Optionally, interfaces for accessing the SRS include but are not limited to the following interfaces: Srs_valid interface, Srs_enable interface, and Srs_rd_addr interface. The Srs_valid interface is used to receive a valid identifier corresponding to an access request, the Srs_enable interface is used to receive an enable identifier corresponding to an access request, and the Srs_rd_addr interface is used to receive a data address corresponding to an access request.
[0094] Optionally, interfaces for SRS return data include but are not limited to the following interfaces: Srs_rtn_valid interface and Srs_rtn_data interface. The Srs_rtn_valid interface is used to receive a valid identifier of SRS return data, and the Srs_rtn_data interface is used to receive SRS return data.
[0095] For ease of explanation, the figure schematically shows only the data access processing flow of one master unit 11. The master unit 11 may be a master0 unit, a master1 unit, a master2 unit, a master3 unit, or a master4 unit.
[0096] The bridge module 10 is used to output a read request, ie, a first access request, when receiving an SRS access request through the master unit 11. The first access request is used to indicate the data address of the cross-partition SRS to be accessed this time.
[0097] The bridge module 10 is further configured to perform a hit test on the first access request.
[0098] The bridge module 10 is further configured to read and return data corresponding to the data address from the cache line when the result of the hit test is a first test result, the first test result indicating that the data address accessed this time is a hit and the data address accessed previously is a hit.
[0099] Optionally, the bridge module 10 is further used to obtain the index of the corresponding first cache line from the lookup table address queue according to the data address of this access; obtain the corresponding data from the lookup table data queue according to the index of the first cache line; and perform a beat operation on the obtained data and return it.
[0100] Optionally, the bridge module 10 is also used to store the data address of this access in a status queue when the result of the hit test is a second test result, and the status queue is used to indicate that the corresponding data is returned in sequence according to the access order; wherein the second test result indicates that the data address of this access is hit and the data address of the historical access is not hit, or the data address of this access is not hit.
[0101] Optionally, the bridge module 10 is also used to determine an unlocked second cache line in the lock tag queue, which indicates whether each cache line in the lookup table address queue is locked; update the data address of this access in the second cache line corresponding to the lookup table data queue, and update the valid flag corresponding to the second cache line in the lookup table valid queue to a first flag, which is used to indicate that the data is invalid; send the first access request to the arbitrator 12, and the arbitrator 12 is used to select multiple cross-partition access requests.
[0102] Optionally, the bridge module 10 is also used to hit the second access request through the arbiter 12, and the second access request is one of the access requests across multiple partitions; the data address indicated by the second access request is sent to the FIFO memory 13, and the FIFO memory 13 is used to cut off the timing for cross-partition data access. The output parameter of the FIFO memory 13 is still the data address indicated by the second access request. The output parameter of the FIFO memory 13 also needs to be beat on the scheduling module side (for example, sent to a D flip-flop, and the D flip-flop is used to perform a beat operation), and the corresponding second access request is sent to the SRS for storage. When the return data returned by the SRS is beaten (for example, 2 beat operations), the index of the corresponding third cache line is obtained from the lookup table address queue; according to the index of the third cache line, the return data is updated in the third cache line of the lookup table data queue, and the valid flag corresponding to the third cache line in the lookup table valid queue is updated to the second flag, and the second flag is used to indicate that the data is valid, completing the operation of storing the return data in the queue.
[0103] Among them, the beat operation is used to indicate that a certain parameter is output after being delayed for several clock cycles. That is to say, the output parameters of the above-mentioned FIFO memory 13 are subjected to a beat operation on the scheduling module side, including: delaying the output parameters of the FIFO memory 13 by n clock cycles before output, where n is a positive integer. For example, n can be 1 or 2 or 3. The embodiment of the present application does not limit this. The return data returned by the above-mentioned SRS is subjected to a beat operation, including: delaying the return data returned by the SRS by m clock cycles before output, where m is a positive integer. For example, m can be 1 or 2 or 3. The embodiment of the present application does not limit this.
[0104] It should be noted that the data requested in the embodiments of the present application may include at least one of image representation data, image color data (such as RGB data), and image position data (such as XYZ coordinate data). The data requested in the embodiments of the present application may also include other types of data, which are not limited in the embodiments of the present application.
[0105] Another point that needs to be explained is that the embodiment of the present application does not limit the number of request units in the bridge module and the number of scheduling modules. That is, the request unit in the bridge module can be one or more, and the scheduling module can be one or more. For the sake of convenience, the embodiment of the present application only schematically illustrates the example of 5 request units and 4 scheduling modules in the bridge module.
[0106] Optionally, the bridge module 10 is also used to detect the status queue, the lookup table address queue and the lookup table valid queue through the comparison unit 14; when the earliest accessed data address in the status queue is the same as the address in the lookup table address queue, and the corresponding valid flag indicates that the data in the fourth cache line is valid, the data is read from the fourth cache line and returned.
[0107] Below, several exemplary embodiments are used to introduce the GPU data access processing method provided in the embodiments of the present application.
[0108] Please refer to Figure 2, which shows a flow chart of a data access processing method for a GPU provided by an exemplary embodiment of the present application. This embodiment uses the method as an example for the GPU shown in Figure 1. The method includes the following steps.
[0109] Step 201: Output a first access request, where the first access request is used to indicate the data address of the cross-partition SRS to be accessed this time.
[0110] When a read request is output by the master unit and at least one second flag is detected among the multiple lock flags in the lock flag queue, step 202 is executed. The lock flag is used to indicate whether the corresponding cache line is locked. When the lock flag is the first flag, it indicates that the cache line is locked, and when the lock flag is the second flag, it indicates that the cache line is unlocked. For example, the first flag is 1 and the second flag is 0. This embodiment of the present application is not limited to this.
[0111] Step 202: Perform a hit test on the first access request.
[0112] Optionally, a hit test is performed on the first access request to obtain a hit test result, where the hit test result includes a first test result or a second test result. The first test result indicates that the data address of the current access is a hit and the data address of the previous access is a hit. The second test result indicates that the data address of the current access is a hit and the data address of the previous access is a miss, or that the data address of the current access is a miss.
[0113] Optionally, the historically accessed data address is the data address of the SRS indicated by an access request received before the first access request is received. Illustratively, the access request received before the first access request is any access request received within a target time period before the first access request is received, where the target time period is a default setting or a custom setting. This is not limited in the embodiments of the present application.
[0114] Schematically, the result of the data address of this access being hit and the data address of the historical access being hit is also called a hit result, the result of the data address of this access being hit and the data address of the historical access being missed is also called a hit on miss result, and the result of the data address of this access being missed is also called a miss result, that is, the first test result is a hit result, and the miss result can be a miss result or a hit on miss result.
[0115] When the result of the hit test is the first test result, it means that the data address of this access is hit and the data address of the historical access is hit, and step 203 is executed; when the result of the hit test is the second test result, it means that the data address of this access is hit and the data address of the historical access is not hit, or the data address of this access is not hit, and step 204 is executed.
[0116] Step 203 : When the result of the hit test is a first test result, data corresponding to the data address is read from the cache line and returned. The first test result indicates that the data address accessed this time is a hit and the data address accessed previously is a hit.
[0117] Optionally, based on the data address being accessed, the index of the corresponding first cache line is obtained from the lookup table address queue; based on the index of the first cache line, the corresponding data is obtained from the lookup table data queue; the obtained data is subjected to a beat operation and returned to the request unit. The beat operation is used to indicate that the data is delayed for k clock cycles before being output, where k is a positive integer, and a read access request is completed at this time. For example, k can be 1, 2, or 3. This is not limited in the present embodiment.
[0118] Step 204, when the result of the hit test is the second test result, the data address of this access is stored in the status queue, and the first access request is sent to the arbitrator. The second test result indicates that the data address of this access is hit and the data address of the historical access is not hit, or the data address of this access is not hit.
[0119] The status queue is used to indicate that corresponding data is returned in sequence according to the access order. Optionally, the status queue is used to store the missed data address and the second data address that is hit when the first data address is missed, where the first data address is the data address accessed before the second data address is accessed.
[0120] Optionally, when the result of the hit test is the second test result, the data address of this access is stored in the status queue, and the first access request is sent to the arbitrator, including: determining the unlocked second cache line in the lock tag queue, the lock tag queue indicates whether each cache line in the lookup table address queue is locked; updating the data address of this access in the second cache line corresponding to the lookup table data queue, and updating the valid flag corresponding to the second cache line in the lookup table valid queue to the first flag, the first flag is used to indicate that the data is invalid; sending the first access request to the arbitrator, the arbitrator is used to select multiple cross-partition access requests.
[0121] Optionally, the update condition of the status queue includes: when the earliest data address in the status queue is the same as the return address in the return address queue, updating the data in the status queue.
[0122] The status queue is used to store multiple data addresses to be accessed in the order of access. Optionally, the depth of the status queue is set based on the data cycle from the output of the first access request to the return of the data. For example, the depth of the status queue is set to a positive integer greater than the data cycle. This is not limited in the present embodiment. In an illustrative example, the structural diagram of the status queue is shown in Figure 3. Each master0 unit corresponds to four positions, each for storing an 11-bit data address. The access of the master0 unit is placed in positions 0 to 3 of the status queue. Position 0 is used to store the earliest accessed data address, and positions 1, 2, and 3 are used to store sequentially accessed data addresses. When the data address stored in position 0 matches the data address in the lookup table address queue and the corresponding cache line in the lookup table valid queue indicates that the data is valid, it indicates that the earliest accessed data address has waited for the return data. At this time, the corresponding returned data is returned, and the data in positions 1 to 3 is moved down to positions 0 to 2. The method provided in the present embodiment can ensure that the order of data return follows the access order, meeting the order preservation requirement. Similarly, the operations of the master1, master2, master3, and master4 units can be analogous to those of the master0 unit and will not be further described here.
[0123] In an illustrative example, a structural diagram of a lock tag queue is shown in FIG4 , where the lock tag queue includes lock tags corresponding to multiple cache lines, each lock tag being 1 bit of information. When the lock tag is 1, it indicates that the cache line is locked, and when the lock tag is 0, it indicates that the cache line is unlocked. The embodiment of the present application does not limit this. The lock tag queue is used to perform a lock detection mechanism when the result of the hit detection is the second test result, that is, to compare the data address in the status queue with the lookup table address queue. When the data address of the corresponding cache line in the lookup table address queue is the same as the data address in the status queue, the lock tag of the corresponding cache line is pulled high, indicating that the cache line is locked. When there is a new access request subsequently, the accessed data address will not be updated in the corresponding lookup table address queue. When there is no identical address in the status queue and the lookup table address queue, the corresponding lock tag is pulled low, indicating that the cache line is unlocked.
[0124] The arbitrator is used to select multiple cross-partition access requests, and the selection may include round-robin selection or custom selection, and the custom selection may be a custom selection based on an application scenario. This embodiment of the present application is not limited to this.
[0125] Step 205: The second access request is hit by the arbiter, the data address indicated by the second access request is sent to the SRS in the scheduling module, and the queue information is updated according to the return information of the SRS.
[0126] Optionally, a second access request is hit through custom selection or polling by the arbiter, and the second access request is one of multiple cross-partition access requests; the data address indicated by the second access request is sent to the SRS in the scheduling module; according to the data address of the return data returned by the SRS, the index of the corresponding third cache line is obtained from the lookup table address queue; according to the index of the third cache line, the return data is updated in the third cache line of the lookup table data queue, and the valid flag corresponding to the third cache line in the lookup table valid queue is updated to the second flag, and the second flag is used to indicate that the data is valid.
[0127] Optionally, when the arbiter selects a corresponding second access request, and the second access request is one of multiple cross-partition access requests, the data address indicated by the second access request is input into the FIFO memory, and the FIFO memory is used to cut off the timing for cross-partition data access.
[0128] Optionally, the output parameter of the FIFO memory, i.e., the data address indicated by the second access request, is subjected to a beat operation (for example, input into a D flip-flop, which is used to perform a beat operation), and the corresponding second access request is sent to the SRS for storage.
[0129] Optionally, the return data returned by the SRS carries the data address of this access. After the return data returned by the SRS undergoes a beat operation (such as two beat operations), the queue information is updated according to the return information of the SRS, including: obtaining the index of the corresponding third cache row from the lookup table address queue; according to the index of the third cache row, updating the return data in the third cache row of the lookup table data queue, and updating the valid flag corresponding to the third cache row in the lookup table valid queue to the second flag, the second flag is used to indicate that the data is valid, and completing the operation of storing the return data in the queue.
[0130] In an illustrative example, the structure of the lookup table address queue is shown in Figure 5, where each row stores an 11-bit data address. The lookup table address queue is used to store the data address of the lookup table data stored in the SRS. After outputting the first access request, if a lock flag is detected for a cache line indicating that the cache line is unlocked, the accessed address is updated in the corresponding lookup table address queue and status queue.
[0131] In an illustrative example, the structure of the lookup table data queue is shown in Figure 6, where each row stores 30 bits of return data. The lookup table data queue is used to store the return data returned by the SRS. When the data address of the return data matches the data address in the lookup table address queue, the lookup table data queue for the corresponding cache line is updated.
[0132] In an illustrative example, the structure of the lookup table valid queue is shown in FIG7 . Each row stores a 1-bit valid flag. When the valid flag is 1, it indicates that the data is valid, and when the lock flag is 0, it indicates that the data is invalid. The structure of the lookup table valid queue is the same as that of the lock flag queue. When the data address of the return data returned by the SRS is present, the data address of the returned data is compared with the data address in the lookup table address queue. If they are the same, the return data is updated in the lookup table data queue, and the valid flag of the corresponding cache line in the lookup table valid queue is updated to 1.
[0133] Step 206 : The status queue, the lookup table address queue, and the lookup table valid queue are checked by a comparison unit. When a read condition is met, the data is read from the cache line and returned.
[0134] Optionally, a comparison unit is used to check the status queue, the lookup table address queue, and the lookup table valid queue; when a read condition is met, the data is read from the cache line and returned; when the read condition is not met, the data is waited for to be returned until the read condition is met. The read condition is that the address of the earliest accessed data in the status queue is the same as the address in the lookup table address queue, and the corresponding valid flag indicates that the data in the fourth cache line is valid.
[0135] Optionally, when the read condition is met, that is, the earliest accessed data address in the status queue is the same as the address in the lookup table address queue, and the corresponding valid flag indicates that the data in the fourth cache line is valid, the return data is read from the fourth cache line and returned to the corresponding request unit.
[0136] It should be noted that during the operation of the GPU system, there is a situation where the result of the hit test of the first access request is a miss result, while the result of the hit test of the second access request is a hit result. This situation can be called a hit on miss result. At this time, the data address of the second access will be updated in the status queue until the data returned by the first miss result is updated in the lookup table data queue and the lookup table valid queue and compared by the comparison unit, and then returned to the corresponding request unit, and a shift operation is performed on the status queue, that is, the status queue will be moved downward as a whole. At this time, the data address of the second hit is processed, thereby meeting the order preservation requirement. Among them, the order preservation operation provided in the embodiment of the present application indicates that the result of the hit test corresponding to this access request is a hit result, and the corresponding data cannot be returned directly. It is necessary to wait until the access request corresponding to the earliest miss result is replied before the data can be returned in order.
[0137] To sum up, when the data address of the current access is hit and the data address of the historical access is hit, the data can be directly read and returned from the cache line, so that for the cross-partition hardware structure, there is no read cache structure in the related technology, and a single access is 6 cycles; and the hardware circuit including the read cache structure provided by the embodiment of the present application requires 7 cycles for a single access when the data address of the cross-partition access does not hit, and the duration of a single access is shortened to 1 cycle when the data address of the cross-partition access hits, which greatly shortens the delay of accessing SRS data across partitions as a whole, and improves the system performance of the GPU.
[0138] In addition, in the design of the status queue, by setting miss results and hit-on-miss results, both scenarios are written into the status queue, and a mechanism is set up for the status queue to prioritize the earliest accessed data address each time, ensuring that data can be returned in order, thereby reducing the resource consumption of the request end to perform the order-preserving function.
[0139] Please refer to Figure 8, which shows a flow chart of a data access processing method for a GPU provided by another exemplary embodiment of the present application. This embodiment uses the method as an example for the GPU shown in Figure 1. The method includes the following steps.
[0140] Step 801: Output a first access request, where the first access request is used to indicate the data address of the cross-partition scalar register SRS to be accessed this time.
[0141] Step 802: Perform a hit test on the first access request.
[0142] Step 803 : When the result of the hit test is a first test result, data corresponding to the data address is read from the cache line and returned. The first test result indicates that the data address accessed this time is a hit and the data address accessed previously is a hit.
[0143] It should be noted that the relevant details of each step in the embodiments of the present application can be referred to the relevant description in the above embodiments and will not be repeated here.
[0144] Please refer to Figure 9, which shows a schematic diagram of the structure of a data access processing device for a GPU provided by one embodiment of the present application. This device can be implemented as all or part of the GPU in Figure 1 through dedicated hardware circuits, or a combination of software and hardware. The device includes: a bridge module 910 and a scheduling module 920. The bridge module 910 includes a request unit and a scheduling multiplexing unit, and the scheduling module 920 includes an SRS;
[0145] The bridging module 910 is configured to output a first access request through a request unit, where the first access request is used to indicate a data address of the SRS of the cross-partition scheduling module 920 to be accessed this time;
[0146] The bridging module 910 is further configured to perform a hit test on the first access request through the scheduling multiplexing unit;
[0147] The bridge module 910 is also used to read and return data corresponding to the data address from the cache line of the scheduling module 920 when the result of the hit test is a first test result, and the first test result indicates that the data address of this access is hit and the data address of the historical access is hit.
[0148] In a possible implementation, the bridging module 910 is further configured to:
[0149] According to the data address of this access, the index of the corresponding first cache line is obtained from the lookup table address queue;
[0150] Obtain corresponding data from the lookup table data queue according to the index of the first cache line;
[0151] The acquired data is then returned after being tapped. The tapping operation is used to indicate that the data is output after being delayed by k clock cycles, where k is a positive integer.
[0152] In another possible implementation, the bridging module 910 is further configured to:
[0153] When the result of the hit test is the second test result, the data address of this access is stored in the status queue, and the status queue is used to indicate that the corresponding data is returned in sequence according to the access order;
[0154] The second test result indicates that the data address of the current access is hit and the data address of the historical access is not hit, or the data address of the current access is not hit.
[0155] In another possible implementation, the depth of the status queue is set according to a data cycle from outputting the first access request to returning data.
[0156] In another possible implementation, the status queue is used to store a missed data address and a second data address that is hit when the first data address is missed, where the first data address is a data address accessed before the second data address is accessed.
[0157] In another possible implementation, the update conditions of the status queue include:
[0158] When the earliest data address in the status queue is the same as the return address in the return address queue, the data in the status queue is updated.
[0159] In another possible implementation, the bridging module 910 further includes an arbitrator, and the bridging module 910 is further configured to:
[0160] determining an unlocked second cache line in a lock tag queue, the lock tag queue indicating whether each cache line in the lookup table address queue is locked;
[0161] The data address of the current access is updated in the second cache line corresponding to the data queue of the lookup table, and the valid flag corresponding to the second cache line in the valid queue of the lookup table is updated to the first flag, where the first flag is used to indicate that the data is invalid;
[0162] The first access request is sent to an arbitrator, which is used to select multiple cross-partition access requests.
[0163] In another possible implementation, the bridge module 910 further includes a first-in-first-out (FIFO) memory, and the bridge module 910 is further configured to:
[0164] A second access request is hit by the arbiter, where the second access request is one of the multiple cross-partition access requests;
[0165] Sending the data address indicated by the second access request to the SRS in the scheduling module 920 through the FIFO memory;
[0166] According to the data address of the returned data returned by the SRS, the index of the corresponding third cache line is obtained from the lookup table address queue;
[0167] According to the index of the third cache line, the returned data is updated in the third cache line of the lookup table data queue, and the valid flag corresponding to the third cache line in the lookup table valid queue is updated to a second flag, which is used to indicate that the data is valid.
[0168] In another possible implementation, the bridging module 910 further includes a comparison unit, and the bridging module 910 is further configured to:
[0169] The status queue, the lookup table address queue and the lookup table valid queue are detected by the comparison unit;
[0170] When the earliest accessed data address in the status queue is the same as the data address in the lookup table address queue, and the corresponding valid flag indicates that the data in the fourth cache line is valid, the data is read from the fourth cache line and returned.
[0171] In another possible implementation manner, the device is implemented by a hardware circuit.
[0172] It should be noted that the apparatus provided in the above embodiments, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.
[0173] An embodiment of the present application provides a data access processing device for a GPU, the device comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the methods executed by the GPU in the above-mentioned various method embodiments when executing the instructions.
[0174] An embodiment of the present application provides a non-volatile computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the methods executed by the GPU in the above-mentioned various method embodiments are implemented.
[0175] An embodiment of the present application provides a computer program product, which includes computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a computer device, the processor in the computer device executes the method performed by the GPU in each of the above-mentioned method embodiments.
[0176] The present application may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present application.
[0177] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.
[0178] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.
[0179] The computer program instructions for performing the operation of the present application can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data or source code or object code written in any combination of one or more programming languages, wherein the programming language includes object-oriented programming languages such as Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. Computer-readable program instructions can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or executed completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer by any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (such as by using an Internet service provider to connect to the Internet). In certain embodiments, by utilizing the state information of computer-readable program instructions to personalize electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs) or programmable logic arrays (PLAs), the electronic circuits can execute computer-readable program instructions, thereby realizing various aspects of the present application.
[0180] Various aspects of the present application are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.
[0181] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.
[0182] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.
[0183] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the system, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a special hardware-based system that performs the function or action of the specification, or can be implemented by a combination of special hardware and computer instructions.
[0184] The embodiments of the present application have been described above. The above description is illustrative and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to the technology in the market, or to enable other persons skilled in the art to understand the embodiments disclosed herein.
Claims
1. A data access processing device for a GPU, characterized in that: The device comprises: a bridge module and a scheduling module, the bridge module comprises a request unit and a scheduling multiplexing unit, and the scheduling module comprises a scalar register SRS; The bridging module is used to output a first access request through the request unit, where the first access request is used to indicate a data address of the SRS of the scheduling module across partitions that is accessed this time; The bridging module is further used to perform a hit test on the first access request through the scheduling multiplexing unit; The bridge module is also used to read and return data corresponding to the data address from the cache line of the scheduling module when the result of the hit test is a first test result, and the first test result indicates that the data address accessed this time hits and the data address accessed historically hits.
2. The device according to claim 1, characterized in that The bridge module is also used for: According to the data address accessed this time, obtaining the index of the corresponding first cache line from the lookup table address queue; Obtaining corresponding data from a lookup table data queue according to an index of the first cache line; The acquired data is subjected to a beat operation and then returned, wherein the beat operation is used to indicate that the data is output after being delayed by k clock cycles, where k is a positive integer.
3. The device according to claim 1, characterized in that The bridge module is also used for: When the result of the hit test is the second test result, the data address accessed this time is stored in a status queue, wherein the status queue is used to indicate that corresponding data is returned in sequence according to the access order; The second test result indicates that the data address of the current access is hit and the data address of the historical access is not hit, or the data address of the current access is not hit.
4. The device according to claim 3, characterized in that The depth of the status queue is set according to a data cycle from outputting the first access request to returning data.
5. The device according to claim 3, characterized in that The status queue is used to store a missed data address and a hit second data address when a first data address is missed, wherein the first data address is a data address accessed before the second data address is accessed.
6. The device according to claim 3, characterized in that The update conditions of the status queue include: When the earliest data address in the status queue is the same as the return address in the return address queue, the data in the status queue is updated.
7. The device according to claim 3, characterized in that The bridge module further includes an arbitrator, and the bridge module is further used for: determining an unlocked second cache line in a lock tag queue, wherein the lock tag queue indicates whether each cache line in the lookup table address queue is locked; The data address of the current access is updated in the second cache line corresponding to the data queue of the lookup table, and the valid flag corresponding to the second cache line in the valid queue of the lookup table is updated to the first flag, where the first flag is used to indicate that the data is invalid; The first access request is sent to the arbitrator, and the arbitrator is used to select multiple cross-partition access requests.
8. The device according to claim 7, characterized in that The bridge module also includes a first-in-first-out FIFO memory, and the bridge module is further used for: A second access request is hit by the arbitrator, where the second access request is one of the multiple cross-partition access requests; Sending the data address indicated by the second access request to the SRS in the scheduling module through the FIFO memory; According to the data address of the returned data returned by the SRS, obtaining the index of the corresponding third cache line from the lookup table address queue; According to the index of the third cache line, the returned data is updated in the third cache line of the lookup table data queue, and the valid flag corresponding to the third cache line in the lookup table valid queue is updated to a second flag for indicating that the data is valid.
9. The device according to claim 8, characterized in that The bridge module further comprises a comparison unit, and the bridge module is further used for: Detecting the state queue, the lookup table address queue and the lookup table valid queue by the comparison unit; When the earliest accessed data address in the status queue is the same as the data address in the lookup table address queue, and the corresponding valid flag indicates that the data in the fourth cache line is valid, the data is read from the fourth cache line and returned.
10. The device according to any one of claims 1 to 9, characterized in that: The device is implemented by a hardware circuit.
11. A GPU data access processing method, characterized in that: The method comprises: Output a first access request, where the first access request is used to indicate the cross-partition scalar register to be accessed this time Data address of SRS; Performing a hit test on the first access request; When the result of the hit test is a first test result, data corresponding to the data address is read from the cache line and returned, and the first test result indicates that the data address accessed this time is a hit and the data address accessed historically is a hit.
12. The method according to claim 11, characterized in that When the result of the hit test is the first test result, reading and returning the data corresponding to the data address from the cache line includes: According to the data address accessed this time, obtaining the index of the corresponding first cache line from the lookup table address queue; Obtaining corresponding data from a lookup table data queue according to an index of the first cache line; The acquired data is subjected to a beat operation and then returned, wherein the beat operation is used to indicate that the data is output after being delayed by k clock cycles, where k is a positive integer.
13. The method according to claim 11, characterized in that The method further comprises: When the result of the hit test is the second test result, the data address accessed this time is stored in a status queue, and the status queue is used to indicate that corresponding data is returned in sequence according to the access order; The second test result indicates that the data address of the current access is hit and the data address of the historical access is not hit, or the data address of the current access is not hit.
14. The method according to claim 13, characterized in that The depth of the status queue is set according to a data cycle from outputting the first access request to returning data.
15. The method according to claim 13, characterized in that The status queue is used to store a missed data address and a hit second data address when a first data address is missed, wherein the first data address is a data address accessed before the second data address is accessed.
16. The method according to claim 13, characterized in that The update conditions of the status queue include: When the earliest data address in the status queue is the same as the return address in the return address queue, the data in the status queue is updated.
17. The method according to claim 13, characterized in that When the result of the hit test is the second test result, after storing the data address of the current access in the status queue, the method further includes: determining an unlocked second cache line in a lock tag queue, wherein the lock tag queue indicates whether each cache line in the lookup table address queue is locked; The data address of this access is updated in the second cache line corresponding to the lookup table data queue, and The valid flag corresponding to the second cache line in the valid queue of the lookup table is updated to a first flag, where the first flag is used to indicate that the data is invalid; The first access request is sent to an arbitrator, where the arbitrator is used to select multiple cross-partition access requests.
18. The method according to claim 17, characterized in that The method further comprises: A second access request is hit by the arbitrator, where the second access request is one of the multiple cross-partition access requests; sending the data address indicated by the second access request to the SRS; According to the data address of the returned data returned by the SRS, obtaining the index of the corresponding third cache line from the lookup table address queue; According to the index of the third cache line, the returned data is updated in the third cache line of the lookup table data queue, and the valid flag corresponding to the third cache line in the lookup table valid queue is updated to a second flag for indicating that the data is valid.
19. The method according to claim 18, characterized in that The method further comprises: Detecting the status queue, the lookup table address queue, and the lookup table valid queue; When the earliest accessed data address in the status queue is the same as the data address in the lookup table address queue, and the corresponding valid flag indicates that the data in the fourth cache line is valid, the data is read from the fourth cache line and returned.
20. The method according to any one of claims 11 to 19, characterized in that: The method is implemented by a hardware circuit.
21. A data access processing device for a GPU, characterized in that: The device comprises: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method described in any one of claims 11-20 when executing the instructions.
22. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 11 to 20 is implemented.
23. A computer program product, comprising a computer readable code, or a non-volatile computer readable storage medium carrying the computer readable code, characterized in that: When the computer readable code is executed in a computer device, a processor in the computer device executes the method according to any one of claims 11 to 20.
Citation Information
Patent Citations
Constant Cache capable of supporting pipeline operation
CN108132893A
Cross-regional multi-level cache data processing system, method and device and storage medium
CN114385678A
Data access method, electronic equipment and storage medium
CN115269454A
Data prefetching method and data prefetching device
CN115495394A
Data access processing method and device of GPU (Graphics Processing Unit) and storage medium
CN117132446A
Cited By
Instruction fetching method, device and equipment based on multi-level branch prediction and storage medium
CN121387368A
Cache controller verification method and device, computer equipment and readable storage medium
CN121523955A
