Instruction scheduling method, instruction scheduling device, processor and storage medium

Abstract

An instruction scheduling method, an instruction scheduling device, a processor, and a storage medium. The instruction scheduling method includes: selecting a first instruction fetch request for a first instruction address initiated by a first thread bundle and performing an instruction fetch operation on the first instruction address; receiving first instruction data corresponding to the first instruction fetch request returned from the first instruction address; and in response to a second instruction fetch request for the first instruction address initiated by a second thread bundle, broadcasting the first instruction data to the write address of the instruction data access area of ​​the first thread bundle and the write address of the instruction data access area of ​​the second thread bundle in a first clock cycle. The instruction scheduling method can reduce the access of a computing unit to a cache system such as an instruction cache or other caches at several levels due to instruction fetch operations, reduce the access bandwidth of the instruction cache or other caches at several levels, and thereby reduce the access bandwidth of the cache system such as a data cache or other caches at several levels for data required to execute instructions.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to an instruction scheduling method, an instruction scheduling device, a processor, and a storage medium. Background Art

[0002] A general-purpose graphics processing unit (GPGPU) is a type of GPU that is more inclined towards general-purpose computing rather than graphics rendering. It also has a large number of independently running computing units inside, such as the Streaming Multiprocessor (SM). Therefore, the degree of parallelism of GPGPU is very high.

[0003] Figure 1 A schematic diagram of the structure of a general-purpose graphics processing unit (GPGPU) is shown. In parallel computing, computing tasks are generally executed by multiple threads, with multiple threads (thread warps) sharing a common instruction stream. Before execution in a GPGPU (also known as a parallel computing processor), these threads are divided into multiple thread blocks (thread blocks) by a thread block dispatcher. Different computing tasks correspond to different thread blocks, and the kernel (GPGPU executable program code) executed by the thread blocks corresponding to the same computing task is the same, differing only in the data operated on by each thread in the same thread block. The thread blocks are then distributed to various compute units (CUs) (e.g., streaming multiprocessors (SMs)) via a thread block dispatcher. Multiple thread blocks can execute in the same CU or in different CUs. All threads in a thread block must be assigned to the same CU for execution. Furthermore, thread blocks are split into minimum execution thread warps (or simply thread warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. When multiple thread blocks are executed in the same compute unit, the thread warps in the compute unit can come from the same thread block or different thread blocks, and all threads in the same thread warp can be executed in a SIMD (Single Instruction Multiple Data) manner.

[0004] For example, Figure 1Each computing unit shown includes an instruction scheduler (also known as a warp scheduling / dispatching module, or warp scheduler) and multiple computing cores. The instruction scheduler may include an instruction data access area, such as random access memory (RAM) for temporarily storing instruction data, and each computing core includes a register file. The instruction cache and data cache corresponding to each computing unit may be L1 caches outside the computing unit. For example, a small number of computing units may share the same instruction cache and data cache, and the instruction caches and data caches of different computing units may also share the next-level cache. The instruction scheduler performs a series of functions such as instruction fetching, decoding, scheduling, and dispatching on the warps running on the computing unit, so that the multiple computing cores (e.g., stream processors (SPs)) of the computing unit can execute the warps. For example, each computing core includes an arithmetic logic unit (ALU), a floating-point unit, etc. Depending on the number of computing cores in the computing unit, multiple warps in a thread block can be executed simultaneously or in a time-sharing manner. Multiple threads in each warp will execute the same instruction, and the results obtained after the instruction is executed are updated to the registers corresponding to each warp.

[0005] For example, Figure 2 As shown, the GPGPU pipeline operation consists of five stages: instruction fetch (IF), instruction dispatch / decode (ID), execute (EX), memory access (MEM), and write back (WB, specifically updating the result of instruction execution to the register). Each thread warp has a program counter (PC) that records the address of the next instruction to be executed in the thread warp, i.e., the instruction fetch address. The value in the program counter indicates the location of the next instruction in main memory. When an instruction is fetched, the value in the program counter automatically increments. After the instruction is executed and the instruction data is written back, the computer retrieves the next instruction address from the program counter. Instruction fetch is the first stage in the pipeline. The fetched instruction data is input to subsequent stages for processing, thus realizing the operation of the entire computing unit.

[0006] For example, Figure 3As shown, when a GPGPU includes compute units 0 to N, the instruction dispatcher in each compute unit sends a thread warp's instruction fetch request (including the instruction address) to the instruction cache. If the instruction data corresponding to the instruction address exists in the instruction cache, there is no need to access the shared cache. If the instruction data corresponding to the instruction address does not exist in the instruction cache, the shared cache is accessed or further sent to the unified cache for instruction fetch operations. The data required for instruction execution is first obtained from the data cache. If the data does not exist in the data cache, the shared cache is accessed or further sent to the unified cache. Therefore, the instruction fetch and data fetch operations of the thread warp will cause bandwidth competition when accessing the shared cache. Summary of the Invention

[0007] At least one embodiment of the present disclosure provides an instruction scheduling method, comprising: selecting a first instruction fetch request for a first instruction address initiated by a first thread warp, and performing an instruction fetch operation on the first instruction address; receiving first instruction data corresponding to the first instruction fetch request returned from the first instruction address; and in response to a second instruction fetch request for fetching the first instruction address initiated by a second thread warp, broadcasting the first instruction data in a first clock cycle to a write address of an instruction data access area of ​​the first thread warp and a write address of an instruction data access area of ​​the second thread warp.

[0008] For example, the instruction scheduling method provided in some embodiments of the present disclosure further includes:

[0009] The second instruction fetch request is canceled.

[0010] For example, in the instruction scheduling method provided by some embodiments of the present disclosure, when the second instruction fetch request is a candidate and is selected in the first clock cycle, the instruction fetch operation corresponding to the second instruction fetch request is canceled.

[0011] For example, in the instruction scheduling method provided in some embodiments of the present disclosure, the second instruction fetch request is a candidate in the first clock cycle and is ignored.

[0012] For example, in the instruction scheduling method provided in some embodiments of the present disclosure, the first instruction fetch request and the second instruction fetch request are candidates in the second clock cycle, the first instruction fetch request is selected, and the second instruction fetch request is ignored, wherein the first clock cycle is located after the second clock cycle.

[0013] For example, in the instruction scheduling method provided by some embodiments of the present disclosure, the second instruction fetch request is ignored in one or more intermediate operation cycles between the second clock cycle and the first clock cycle.

[0014] For example, in the instruction scheduling method provided in some embodiments of the present disclosure, in the second clock cycle, the third instruction fetch request initiated by the third thread warp to fetch the first instruction address is a candidate but is ignored; in the third clock cycle between the second clock cycle and the first clock cycle, the third instruction fetch request is a candidate and is selected to perform an instruction fetch operation on the first instruction address.

[0015] For example, in the instruction scheduling method provided by some embodiments of the present disclosure, the first thread warp and the second thread warp belong to the same thread block or to different thread blocks.

[0016] For example, the instruction scheduling method provided in some embodiments of the present disclosure further includes:

[0017] Recording attribute information of each instruction fetch request, wherein the attribute information includes the thread block number of the thread block to which the thread warp initiating the instruction fetch request belongs and the thread warp number of the thread warp initiating the instruction fetch request in the thread block to which it belongs.

[0018] For example, the instruction scheduling method provided by some embodiments of the present disclosure also includes: in response to returning the first instruction data, obtaining the thread block number corresponding to the first thread warp and the thread warp number corresponding to the first thread warp according to the attribute information of the first instruction fetch request; determining the second instruction fetch request based on the thread block number corresponding to the first thread warp and the thread warp number corresponding to the first thread warp and the thread block number and thread warp number included in the attribute information of each of the multiple candidate instruction fetch requests.

[0019] For example, in the instruction scheduling method provided in some embodiments of the present disclosure, the second instruction fetch request is determined based on the thread block number corresponding to the first thread warp and the thread warp number corresponding to the first thread warp, as well as the thread block number and thread warp number included in the attribute information of each of the multiple candidate instruction fetch requests, including: generating a broadcast mask, wherein the broadcast mask includes information bits corresponding to the second instruction fetch request; wherein the first instruction data is broadcast and sent to the write address of the instruction data access area of ​​the second thread warp in the first clock cycle using the broadcast mask.

[0020] For example, in the instruction scheduling method provided in some embodiments of the present disclosure, the broadcast mask also includes an information bit corresponding to the first instruction fetch request, and the broadcast mask is used to broadcast the first instruction data to the write address of the instruction data access area of ​​the first thread bundle and the write address of the instruction data access area of ​​the second thread bundle in the first clock cycle.

[0021] For example, in the instruction scheduling method provided by some embodiments of the present disclosure, the attribute information of each instruction fetch request also includes the write address of the instruction data access area, and the write address of the instruction data access area of ​​the first instruction fetch request is obtained through the attribute information of the first instruction fetch request; the write address of the instruction data access area of ​​the second instruction fetch request is obtained through the attribute information of the second instruction fetch request.

[0022] For example, in the instruction scheduling method provided by some embodiments of the present disclosure, the attribute information of each instruction fetch request also includes a program counter, and the first instruction fetch address corresponding to the first instruction fetch request is obtained through the program counter of the attribute information of the first instruction fetch request.

[0023] For example, in the instruction scheduling method provided by some embodiments of the present disclosure, the attribute information of each instruction fetch request also includes the status of the instruction data access area, and the status of the instruction data access area includes whether the instruction data access area of ​​the thread warp that initiates the instruction fetch request is not full.

[0024] At least some embodiments of the present disclosure also provide an instruction scheduling device, comprising: an instruction fetch arbitration unit, configured to select a first instruction fetch request for a first instruction address initiated by a first thread bundle, and perform an instruction fetch operation on the first instruction address; an instruction preprocessing unit, configured to receive first instruction data corresponding to the first instruction fetch request returned from the first instruction address; and in response to a second instruction fetch request for fetching the first instruction address initiated by a second thread bundle, broadcast the first instruction data to the write address of the instruction data access area of ​​the first thread bundle and the write address of the instruction data access area of ​​the second thread bundle in a first clock cycle.

[0025] For example, in the instruction scheduling apparatus provided in some embodiments of the present disclosure, the instruction preprocessing unit is further configured to cancel the second instruction fetch request.

[0026] For example, in the instruction scheduling device provided in some embodiments of the present disclosure, the instruction preprocessing unit is further configured to cancel the instruction fetch operation corresponding to the second instruction fetch request when the second instruction fetch request is a candidate and is selected in the first clock cycle.

[0027] For example, in the instruction scheduling apparatus provided in some embodiments of the present disclosure, the instruction preprocessing unit is further configured to ignore the second instruction fetch request when the second instruction fetch request is a candidate in the first clock cycle.

[0028] For example, in the instruction scheduling device provided in some embodiments of the present disclosure, the instruction preprocessing unit is further configured to select the first instruction fetch request and ignore the second instruction fetch request when the first instruction fetch request and the second instruction fetch request are candidates in a second clock cycle, wherein the first clock cycle is located after the second clock cycle.

[0029] For example, in the instruction scheduling device provided in some embodiments of the present disclosure, the instruction preprocessing unit is further configured to ignore the second instruction fetch request in one or more intermediate operation cycles between the second clock cycle and the first clock cycle.

[0030] For example, in the instruction scheduling device provided in some embodiments of the present disclosure, the instruction fetch arbitration unit is further configured to:

[0031] In the second clock cycle, if a third instruction fetch request initiated by a third warp to fetch an instruction for the first instruction address is a candidate, ignoring the third instruction fetch request;

[0032] In a third clock cycle between the second clock cycle and the first clock cycle, if the third instruction fetch request is a candidate, the third instruction fetch request is selected to perform an instruction fetch operation on the first instruction address.

[0033] For example, in the instruction scheduling device provided by some embodiments of the present disclosure, the first thread warp and the second thread warp belong to the same thread block or to different thread blocks.

[0034] For example, in some embodiments of the present disclosure, the instruction scheduling device further includes an instruction broadcast determination unit configured to record attribute information of each instruction fetch request.

[0035] The attribute information includes the thread block number of the thread block to which the thread warp that initiates the instruction fetch request belongs and the thread warp number of the thread warp that initiates the instruction fetch request in the thread block to which it belongs.

[0036] For example, in the instruction scheduling device provided in some embodiments of the present disclosure, the instruction broadcast determination unit is further configured to:

[0037] Obtaining the first instruction data returned from the instruction preprocessing unit, and obtaining a thread block number corresponding to the first warp and a warp number corresponding to the first warp according to the attribute information of the first instruction fetch request; and

[0038] The second instruction fetch request is determined based on the thread block number corresponding to the first warp, the warp number corresponding to the first warp, and the thread block number and the warp number included in the attribute information of each of the candidate instruction fetch requests.

[0039] For example, in the instruction scheduling apparatus provided in some embodiments of the present disclosure, the instruction broadcast determination unit is further configured to generate a broadcast mask based on the program block number and program bundle number corresponding to the first warp, and the thread block number and warp number included in the attribute information of each of the candidate multiple instruction fetch requests, wherein the broadcast mask includes an information bit corresponding to the second instruction fetch request;

[0040] The instruction pre-processing unit broadcasts the first instruction data to a write address of an instruction data access area of ​​the second warp in a first clock cycle using the broadcast mask.

[0041] For example, in the instruction scheduling device provided in some embodiments of the present disclosure, the broadcast mask also includes an information bit corresponding to the first instruction fetch request, and the instruction preprocessing unit is further configured to use the broadcast mask to broadcast the first instruction data to the write address of the instruction data access area of ​​the first thread warp and the write address of the instruction data access area of ​​the second thread warp in the first clock cycle.

[0042] For example, in the instruction scheduling device provided in some embodiments of the present disclosure, the attribute information of each instruction fetch request also includes the write address of the instruction data access area.

[0043] The instruction preprocessing unit is further configured to:

[0044] obtaining a write address of the instruction data access area of ​​the first warp through the attribute information of the first instruction fetch request, and

[0045] The write address of the instruction data access area of ​​the second warp is obtained according to the attribute information of the second instruction fetch request.

[0046] For example, in the instruction scheduling device provided in some embodiments of the present disclosure, the attribute information of each instruction fetch request also includes a program counter,

[0047] The instruction preprocessing unit is further configured to obtain a first instruction fetch address corresponding to the first instruction fetch request through a program counter of the attribute information of the first instruction fetch request.

[0048] For example, in the instruction scheduling device provided in some embodiments of the present disclosure, the attribute information of each instruction fetch request also includes the state of the instruction data access area.

[0049] The instruction cache state includes whether the instruction data access area of ​​the thread warp that initiates the instruction fetch request is not full.

[0050] At least one embodiment of the present disclosure further provides a processor, comprising at least one computing unit, wherein the computing unit comprises the instruction scheduling device provided by any of the above embodiments.

[0051] At least one embodiment of the present disclosure further provides an instruction scheduling device, which includes: a memory for non-temporarily storing computer-executable instructions; and a processor for executing the computer-executable instructions, wherein when the computer-executable instructions are executed by the processor, the instruction scheduling method provided by any embodiment of the present disclosure is executed.

[0052] At least one embodiment of the present disclosure further provides a non-transitory storage medium that non-transitorily stores computer-executable instructions, wherein when the computer-executable instructions are executed by a computer, the instruction scheduling method provided by any embodiment of the present disclosure is executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, rather than limiting the present disclosure.

[0054] Figure 1 A schematic diagram of the structure of a general-purpose graphics processing unit (GPGPU);

[0055] Figure 2 A pipeline operation flow chart for a general-purpose graphics processing unit (GPGPU);

[0056] Figure 3 A schematic diagram of an instruction fetch flow of a computing unit;

[0057] Figure 4 A schematic diagram of an instruction scheduling device performing an instruction fetch operation;

[0058] Figure 5 Schematic diagram of the running cycle of different thread warps in the same thread block on the computing unit;

[0059] Figure 6 A flowchart of an instruction scheduling method provided by an embodiment of the present disclosure;

[0060] Figure 7 A schematic diagram of a flow chart of an instruction scheduling method provided in one embodiment of the present disclosure applied to an instruction scheduling device in a computing unit;

[0061] Figure 8A schematic diagram of a command broadcast mode during operation of an embodiment of the present disclosure;

[0062] Figure 9 This is a schematic diagram of another command broadcast mode during the operation of an embodiment of the present disclosure;

[0063] Figure 10 A schematic block diagram of an instruction scheduling device provided by an embodiment of the present disclosure;

[0064] Figure 11 A schematic block diagram of another instruction scheduling device provided in one embodiment of the present disclosure;

[0065] Figure 12 A schematic diagram of a non-transitory storage medium provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0066] To make the purpose, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0067] Unless otherwise defined, the technical or scientific terms used in this disclosure should have the usual meanings understood by people with ordinary skills in the field to which this disclosure belongs. The words "first", "second" and similar words used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one", "an" or "the" do not indicate a quantity limitation, but rather indicate the existence of at least one. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0068] The present disclosure is described below through several specific embodiments. To keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits detailed descriptions of known functions and known components. When any component of an embodiment of the present disclosure appears in more than one figure, the component is represented by the same or similar reference numeral in each figure.

[0069] For example, when the instruction dispatcher in a GPGPU fetches instructions, because instructions are executed too quickly, data updates cannot keep up with the rate of instruction execution. Once the data required for instruction execution does not exist in the data cache, it must be obtained from the shared cache. Even if the data required for instruction execution does not exist in the data cache at all levels, it will be obtained from external storage space (for example, main memory), resulting in a longer path and higher latency to obtain the data required for instruction execution. Therefore, the computing unit's demand for data bandwidth is far greater than its demand for instruction bandwidth, resulting in greater data bandwidth pressure.

[0070] For example, since there is only one interface between the computing core and the instruction cache, the instruction scheduling device can only fetch instructions for one thread warp in one clock cycle, so that multiple computing cores can run the thread warp simultaneously or in time-sharing mode. Figure 4 As shown, the instruction scheduling device includes: an instruction fetch arbitration unit and an instruction preprocessing unit. One or more thread bundles to be run or currently running on the computing unit (these thread bundles may belong to the same thread block or may belong to different thread blocks) will send the initiated instruction fetch request to the instruction fetch arbitration unit. The instruction fetch arbitration unit selects a thread bundle according to the selection rule in one clock cycle to perform instruction fetch operation for it, and the instruction fetch requests of other thread bundles will be blocked until they are selected by the instruction fetch arbitration unit. For example, the instruction fetch arbitration unit selects the instruction fetch request of thread bundle 1 in thread block 1 which has the most urgent instruction demand. The instruction fetch arbitration unit sends the instruction fetch request initiated by a selected thread bundle (for example, thread bundle 1 in thread block 1) to the instruction cache (or other level cache areas such as the shared cache area) to perform instruction fetch operation from the instruction address. The instruction preprocessing unit then receives the instruction data returned by the instruction cache (e.g., the instruction data for warp 1 in thread block 1) and determines the valid portion of the returned instruction data, such as whether the instruction data is for instruction fetching warp 1 in thread block 1, the state of the instruction data access area for the instruction data, and the write address of the instruction data access area. Finally, the instruction data is written to the write address of the instruction data access area for the warp, completing the instruction fetch for the warp (e.g., warp 1 in thread block 1), facilitating subsequent decoding, execution, memory access, and writeback, thereby completing the execution of the warp on the corresponding compute unit.

[0071] For example, Figure 5The diagram shows the cycle of four thread bundles (thread bundle 0 to thread bundle 4) belonging to the same thread block, running from start to finish on the computing unit. Although these four thread bundles are pipelined and run continuously, the data update speed required by each thread bundle varies, so the thread bundle that starts running first may not necessarily end first, that is, the overall running time of each thread bundle is not necessarily the same. Therefore, different thread bundles may access the same instruction address in the instruction cache in the same clock cycle, or when different thread bundles access the same instruction address in the instruction cache in different clock cycles, the difference between the two different clock cycles is small. Since the four thread bundles belonging to the same thread block execute the same core instruction code (kernel), the instruction data retrieved by one thread bundle can be broadcast to the other thread bundles that are currently retrieving or about to retrieve this instruction data.

[0072] In addition, a parallel computing task on the CPU side may be divided into multiple thread blocks to complete together. However, because the overall running time difference between thread warps belonging to different thread blocks is larger than the overall running time difference between thread warps belonging to the same thread block, the possibility of two thread warps in different thread blocks accessing the same instruction address in the instruction cache in the same clock cycle is relatively low, and the probability of thread warps in different thread blocks achieving synchronous operation is low. However, thread warps belonging to different thread blocks also have the possibility of instruction broadcast.

[0073] At least some embodiments of the present disclosure provide an instruction scheduling method, comprising: selecting a first instruction fetch request for a first instruction address initiated by a first thread warp, and performing an instruction fetch operation on the first instruction address; receiving first instruction data corresponding to the first instruction fetch request returned from the first instruction address; and in response to a second instruction fetch request for the first instruction address initiated by a second thread warp, broadcasting the first instruction data in a first clock cycle to a write address of an instruction data access area of ​​the first thread warp and a write address of an instruction data access area of ​​the second thread warp.

[0074] Some embodiments of the present disclosure also provide an instruction scheduling device corresponding to the instruction scheduling method, the instruction scheduling device comprising: an instruction fetch arbitration unit configured to select a first instruction fetch request for a first instruction address initiated by a first thread bundle, and perform an instruction fetch operation on the first instruction address; an instruction preprocessing unit configured to receive first instruction data corresponding to the first instruction fetch request returned from the first instruction address, and configured to respond to a second instruction fetch request for the first instruction address initiated by a second thread bundle, broadcast the first instruction data to the write address of the instruction data access area of ​​the first thread bundle and the write address of the instruction data access area of ​​the second thread bundle in a first clock cycle.

[0075] Some embodiments of the present disclosure also provide a non-temporary storage medium corresponding to the above-mentioned instruction scheduling method, which non-temporarily stores computer-readable instructions. When the computer-readable instructions are executed by a computer, the instruction scheduling method provided by the above-mentioned embodiments of the present disclosure is executed.

[0076] The instruction scheduling method provided by the above-mentioned embodiment of the present disclosure, when receiving the first instruction data returned from the first instruction address, broadcasts the first instruction data to the write address of the instruction data access area of ​​each thread bundle that has an instruction fetch request for the first instruction address, effectively reducing the access of the computing unit to the instruction cache or other cache systems such as several levels of cache when performing instruction fetch operations, that is, reducing the access bandwidth of the instruction cache or other cache systems such as several levels of cache, and thereby reducing the access bandwidth of the data cache or other cache systems such as several levels of cache for the data required to execute instructions.

[0077] The following describes some embodiments and examples of the present disclosure in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure.

[0078] Figure 6 A flowchart of an instruction scheduling method provided in some embodiments of the present disclosure. Figure 7 A schematic diagram of an instruction scheduling method provided in one embodiment of the present disclosure applied to an instruction scheduling device in a computing unit.

[0079] like Figure 7 As shown, the instruction scheduling device includes an instruction fetch arbitration unit, an instruction preprocessing unit and an instruction broadcast determination unit. The instruction fetch arbitration unit is used to arbitrate the instruction fetch requests initiated by each thread bundle that needs to be run on the computing unit in each clock cycle, and select the instruction fetch request of a thread bundle to perform an instruction fetch operation. The instruction preprocessing unit is used to receive and analyze the instruction data corresponding to the instruction fetch request of the thread bundle returned by the instruction fetch operation in each clock cycle. For example, the instruction data includes multiple executable instructions, and the size of the multiple executable instructions can be 8 double words (dword, 32 bits, 4 bytes), and broadcast the instruction data to the write address of the instruction data access area of ​​one or more thread bundles in the clock cycle of receiving the returned instruction data. The instruction broadcast determination unit is used to determine one or more thread bundles to which the instruction data needs to be broadcast.

[0080] For example, Figure 6 As shown, the instruction scheduling method includes the following steps S100 to S300, which will be combined with Figure 7 The instruction scheduling method of the embodiment of the present disclosure is introduced with reference to the instruction scheduling device shown.

[0081] Step S100: selecting a first instruction fetch request for a first instruction address initiated by a first warp, and performing an instruction fetch operation for the first instruction address.

[0082] For example, Figure 7 Thread blocks 0 through M are shown running on a compute unit, where each thread block includes warps 0 through N (both N and M are positive integers greater than or equal to 0). One or more thread blocks can run on a compute unit, and each thread block can include one or more warps. During each clock cycle, multiple warps initiate corresponding instruction fetch requests and are in a candidate state, awaiting selection by an instruction fetch arbitration unit. For example, the attribute information of each instruction fetch request includes: the thread block number of the thread block to which the warp initiating the instruction fetch request belongs (e.g., thread block 0), the warp number of the warp in the thread block to which the warp initiating the instruction fetch request belongs (e.g., warp 0), the status of the instruction data access area, the write address of the instruction data access area, the program counter, etc. For example, the attribute information of each instruction fetch request may also include a single warp identifier. The instruction data access area is typically RAM (random access memory) used for temporary access to instruction data. For example, the returned instruction data may include a mixture of double-word instructions (dword, 32 bits, 4 bytes) and quad-word instructions (qword, 64 bits, 8 bytes). Multiple double-word instructions are not immediately sent to the computing core for execution, but are instead sent sequentially and executed continuously. Therefore, the instruction data access area needs to cache one or more instruction data returned. The status of the instruction data access area includes whether the instruction data access area corresponding to the thread warp that initiated the instruction fetch request is not full. A single warp identifier indicates that the thread block to which the warp that initiated the instruction fetch request belongs has only one warp, namely, the warp that initiated the instruction fetch request. For example, in this state, it can be determined that the instruction data returned by the instruction fetch request does not need to be broadcast.

[0083] It should be pointed out that, as mentioned above, the selected "first thread warp" can be any thread warp from thread block 0 to thread block M as the description object; the "first instruction fetch request" can be the instruction fetch request initiated by the thread warp as the description object; and the "first instruction address" can be the instruction address corresponding to the instruction fetch request.

[0084] For example, the instruction fetch arbitration unit selects the first instruction fetch request initiated by thread warp 1 in thread block 1 as the first thread warp. The attribute information of the first instruction fetch request includes: the first thread block number (for example, thread block number 1), the first thread warp number (for example, thread warp number 1), the first instruction address recorded by the program counter, the status of the instruction data access area of ​​the first thread warp, and the write address of the instruction data access area. Among the instruction fetch requests of multiple thread warps in the candidate state, the instruction fetch arbitration unit selects the first instruction fetch request of the first thread warp after arbitration, sends the first instruction fetch request to the instruction cache, and determines whether the instruction data targeted by the first instruction fetch request of the first thread warp exists in the first instruction address. If so, it indicates to perform an instruction fetch operation on the first instruction address. If not, it sends the first instruction fetch request to the shared cache or more levels of cache, and performs an instruction fetch operation on the first instruction address.

[0085] Step S200: Receive first instruction data corresponding to a first instruction fetch request returned from a first instruction address.

[0086] For example, after several clock cycles, in a first clock cycle serving as a description object, the instruction preprocessing unit receives first instruction data corresponding to a first instruction fetch request returned from a first instruction address, analyzes the received first instruction data, and obtains analysis result information. For example, the analysis result information may include attribute information of the first instruction fetch request of the first instruction data: the thread block number of the thread block to which the thread warp initiating the first instruction fetch request belongs, the thread warp number of the thread warp initiating the first instruction fetch request within the thread block to which it belongs, and information indicating whether a broadcast function needs to be enabled for the first instruction data.

[0087] Then, the instruction preprocessing unit sends the above analysis result information to the instruction broadcast determination unit. In each clock cycle, the instruction fetch request initiated by the candidate thread bundle is updated to the instruction broadcast determination unit, and the instruction broadcast determination unit records the attribute information of each candidate instruction fetch request. These attribute information are saved, for example, after the instruction data corresponding to the instruction fetch request is sent to the instruction data access area of ​​the corresponding thread bundle, or after the instruction data corresponding to the instruction fetch request is executed or retired. The instruction broadcast determination unit determines the instruction data access area of ​​the thread bundle that initiated the candidate instruction fetch request to which the first instruction data needs to be broadcast based on the analysis result information and the attribute information of the currently recorded candidate instruction fetch request. These candidate instruction fetch requests all correspond to the first instruction fetch address (i.e., correspond to the first instruction data).

[0088] In at least some examples, an exemplary process of determining the instruction data access areas of candidate warps to which the first instruction data needs to be broadcast may include the following steps:

[0089] Step S210: The thread block number (e.g., thread block 1) and the bundle number (e.g., bundle 1 in thread block 1) corresponding to the first thread warp are obtained based on the attribute information of the first instruction fetch request corresponding to the first instruction data. It should be noted that if the attribute information of the first instruction fetch request corresponding to the first instruction data includes a single bundle identifier, i.e., the first bundle is the only bundle in the thread block to which the first bundle belongs, then the first instruction data does not need to enable the broadcast function; if the single bundle identifier is not included, then the first instruction data does need to enable the broadcast function. Then, the obtained thread block number and bundle number corresponding to the first thread warp, as well as whether the first data needs to enable the broadcast function, are sent to the instruction broadcast determination unit.

[0090] In step S220, the instruction broadcast determination unit determines, based on the thread block number (e.g., thread block 1) corresponding to the first warp and the warp number (e.g., warp 1 in thread block 1), the thread block number and warp number (e.g., warps 2 to N in thread block 1) included in the attribute information of the candidate instruction fetch requests in the first clock cycle, to broadcast the first instruction data to the write address of the instruction data access area of ​​the second warp (e.g., one or more warps of warps 2 to N in thread block 1). The instruction broadcast determination unit sends information about the instruction data access areas of the candidate warps to which the first instruction data is to be broadcast, such as the attribute information of the first instruction fetch request of the first warp and the attribute information of the second instruction fetch request of the second warp, to the instruction preprocessing unit, instructing the instruction preprocessing unit to broadcast the first instruction data to the write address of the instruction data access area corresponding to the first warp and the write address of the instruction data access area corresponding to the second warp in the first clock cycle.

[0091] It should be noted that in the present disclosure, the "second thread warp" may include one of the other thread warps in thread blocks 0 to M except the "first thread warp". In some examples, the number of second thread warps may be one or more (for example, one of them is used as the description object).

[0092] For example, in different examples, the first thread warp and the second thread warp belong to the same thread block or to different thread blocks. As described above, because the multiple thread warps belonging to the same thread block execute the same kernel instruction code, their instruction fetch requests in similar time periods target the same instruction fetch address. In addition, in some rare cases, thread warps in different thread blocks may also have the same instruction fetch address.

[0093] Step S300: In response to a second instruction fetch request initiated by the second warp to fetch a first instruction address, broadcast the first instruction data to the write address of the instruction data access area of ​​the first warp and the write address of the instruction data access area of ​​the second warp in the first clock cycle.

[0094] For example, the instruction preprocessing unit obtains the broadcast object of the first instruction data from the instruction broadcast determination unit (for example, the first thread bundle and the second thread bundle; in other examples, it may also include one or more other thread bundles that initiate instruction fetch of the first instruction address in the same manner as the second thread), obtains the write address of the instruction data access area of ​​the first thread bundle through the attribute information of the first instruction fetch request, obtains the write address of the instruction data access area of ​​the second thread bundle through the attribute information of the second instruction fetch request, and simultaneously obtains the status of the instruction data access area of ​​the first thread bundle through the attribute information of the first instruction fetch request, obtains the status of the instruction data access area of ​​the second thread bundle through the attribute information of the first instruction fetch request, determines whether the status of the instruction data access area of ​​the first thread bundle and the status of the instruction data access area of ​​the second thread bundle are occupied or idle, and if both are idle, then in the first clock cycle, broadcasts the received first instruction data to the write address of the instruction data access area of ​​the first thread bundle and the write address of the instruction data access area of ​​the second thread bundle.

[0095] If, in the first clock cycle, the second warp has been selected by the instruction arbitration unit but has not yet been issued, the instruction scheduling method of the embodiment of the present disclosure may further include the following step S400:

[0096] Step S400: cancel the second instruction fetch request.

[0097] For example, since the instruction preprocessing unit has broadcast the first instruction data returned from the first instruction address to the write address of the instruction data access area of ​​the second thread warp in the first clock cycle, there is no need to respond to the value request for the first instruction address initiated by the second thread warp to perform an instruction fetch operation, so the instruction preprocessing unit cancels the second instruction fetch request.

[0098] For example, in at least one example, the instruction broadcast determination unit generates a broadcast mask based on the thread block number corresponding to the first warp and the warp number corresponding to the first warp, as well as the thread block number and warp number included in the attribute information of each of the candidate multiple instruction fetch requests. For example, the broadcast mask includes information bits corresponding to the second instruction fetch request and information bits corresponding to the first instruction fetch request. The instruction broadcast determination unit sends the broadcast mask and the write address of the instruction data access area corresponding to the broadcast mask to the instruction preprocessing unit. The instruction preprocessing unit uses the broadcast mask to broadcast the first instruction data to the write address of the instruction data access area of ​​the first warp and the write address of the instruction data access area of ​​the second warp in the first clock cycle.

[0099] For example, the m thread warps running on a computing unit can be virtually recorded as a thread warp slot. For example, the m thread warps can belong to the same thread block or different thread blocks. For example, the thread warp slot is a one-dimensional array with m bits, each bit representing a thread warp belonging to a thread block. For example, the maximum number of thread warps running on a computing unit can be 10, so m=10. Because the instruction broadcast determination unit determines the second instruction fetch request based on the thread block number corresponding to the first thread warp and the thread warp number corresponding to the first thread warp, as well as the thread block number and thread warp number included in the attribute information of each of the candidate multiple instruction fetch requests, it determines the second thread warp that needs to be broadcasted with instruction data (for example, it can also include more other thread warps that meet the instruction data broadcast conditions), and the first thread warp also needs to be broadcasted with instruction data. The broadcast mask is used to represent the second thread warp and the first thread warp (and possibly more other thread warps that meet the instruction data broadcast conditions) that need to be broadcasted with instruction data.

[0100] For example, the broadcast mask is a one-dimensional array with a total of m bits, which corresponds one-to-one to the m bits of the thread warp mask. For example, the broadcast mask can be expressed as inst_broadcast_mask[m], and the maximum value of m should be the maximum value of the thread warp mask. Whether a bit in the broadcast mask is set to 1 indicates whether the thread warp corresponding to the bit needs to be broadcast by the instruction data, that is, the bits set to 1 include the information bits of the first instruction fetch request and the information bits of the second instruction fetch request. For example, the broadcast mask is 0011000000, and the third bit set to 1 is the information bit of the first instruction fetch request. For example, the third bit indicates that the first thread warp is thread warp 1 in thread block 1; the fourth bit set to 1 is the information bit of the second instruction fetch request. For example, the fourth bit indicates that the second thread warp is thread warp 2 in thread block 1.

[0101] For example, in at least some examples, there may be two broadcast modes for broadcasting the first instruction data returned from the first instruction address to the write address of the instruction data access area of ​​the first thread warp and the write address of the instruction data access area of ​​the second thread warp in the first clock cycle.

[0102] For example, after the instruction preprocessing unit receives the first instruction data corresponding to the first instruction fetch request returned from the first instruction address in the first clock cycle, and determines the second instruction fetch request initiated by the second thread warp to fetch the first instruction address, if the second instruction fetch request is a candidate and is selected in the first clock cycle, the instruction preprocessing unit cancels the instruction fetch operation corresponding to the second instruction fetch request, and at the same time broadcasts the first instruction data to the write address of the instruction data access area of ​​the second thread warp.

[0103] Alternatively, when the second instruction fetch request initiated by the second thread warp is a candidate and is ignored in the first clock cycle, the instruction preprocessing unit cancels the instruction fetch operation corresponding to the second instruction fetch request, and at the same time broadcasts the first instruction data to the write address of the instruction data access area of ​​the second thread warp.

[0104] For example, in at least some examples, based on the two broadcast modes described above, in different situations, the first instruction fetch request and the second instruction fetch request are candidates in the second clock cycle (the first clock cycle is after the second clock cycle), the first instruction fetch request is selected, and the second instruction fetch request is ignored, or the second instruction fetch request is ignored in one or more intermediate operation cycles between the second clock cycle and the first clock cycle, or the third instruction fetch request initiated by the third thread warp to fetch the first instruction address is a candidate in the second clock cycle, but is ignored; and, in the third clock cycle between the second clock cycle and the first clock cycle, the third instruction fetch request is a candidate and is selected to perform an instruction fetch operation on the first instruction address.

[0105] like Figure 8 In the exemplary case of the broadcast mode shown, warps 0 to 3 in the same thread block all have instruction fetch requests for instruction address 0 (IF, PC=0, an example of the first instruction address) within T*N clock cycles. At this time, the instruction fetch requests of warps 0 to 3 are all in the candidate state.

[0106] In the T*N clock cycle, the instruction fetch request of thread warp 0 is selected by the instruction fetch arbitration unit (displayed in bold, the same below), and an instruction fetch operation is performed on instruction address 0. At the same time, the instruction fetch requests of other thread warps are ignored (displayed in light color, the same below). For example, the instruction fetch request of thread warp 3 is a candidate but is ignored. Similarly, the instruction fetch requests of thread warps 1 and thread warp 2 are candidates but are ignored.

[0107] In the T*N+1 clock cycle, the instruction fetch request of thread warp 1 is selected by the instruction fetch arbitration unit and an instruction fetch operation is performed on instruction address 0. At the same time, the instruction fetch requests of other thread warps are in the candidate state but are ignored. For example, the instruction fetch request of thread warp 3 is ignored. Similarly, the instruction fetch request of thread warp 2 is in the candidate state but is ignored.

[0108] In the T*N+2 clock cycle, the instruction fetch request of warp 2 is selected by the instruction fetch arbitration unit and an instruction fetch operation is performed on instruction address 0. Meanwhile, the instruction fetch request of warp 3 is in the candidate state but is ignored.

[0109] In the T*N+3 clock cycle, the instruction data of thread warp 0 returns from instruction address 0 (INST RTN, PC=0). At this time, the instruction fetch request of thread warp 3, which also wants to fetch instruction address 0, is still in the candidate state. Then the instruction data of thread warp 0 is broadcast simultaneously to the write address of the instruction data access area of ​​the instruction fetch request of thread warp 3, which was in the candidate state and ignored in the previous T*N~T*N+2 clock cycles. This is equivalent to responding to the instruction fetch request of thread warp 3 to fetch instruction address 0 (therefore, INST RTN, PC=0 is also marked for thread warp 3 in the figure), and at the same time canceling the value-taking operation of the instruction fetch request of thread warp 3.

[0110] On the other hand, the warps 1 and 2 that have performed instruction fetch operations in clock cycles T*N to T*N+2 will wait for instruction data retrieved for their instruction fetch operations.

[0111] For example, within the T*N+4 clock cycle, the instruction data of thread warp 1 is returned from instruction address 0. At this time, the instruction fetch request of thread warp 5 for instruction address 0 is a candidate and is selected by the instruction fetch arbitration unit. Then the instruction data of thread warp 1 can be broadcast to thread warp 5, and the value fetch operation of thread warp 5's instruction fetch request is canceled.

[0112] Similarly, within the T*N+5 clock cycle, the instruction data of thread warp 2 is returned from instruction address 0. At this time, the instruction fetch request of thread warp 4 for instruction address 0 is a candidate and is selected by the instruction fetch arbitration module. Then the instruction data of thread warp 2 can be broadcast to thread warp 4, and the value fetch operation of thread warp 4's instruction fetch request is canceled.

[0113] It should be noted that, in the scenario shown above, examples of the first clock cycle include T*N+3 clock cycles and T*N+5 clock cycles. Correspondingly, examples of the second clock cycle include T*N clock cycles. Examples of intermediate operation cycles include T*N+1 clock cycles and T*N+2 clock cycles. For example, an example of the third clock cycle includes T*N+1 clock cycles. In the above scenario, examples of the first instruction fetch request may include an instruction fetch request from warp 0 for instruction address 0 and an instruction fetch request from warp 2 for instruction address 0. Correspondingly, examples of the second instruction fetch request may include an instruction fetch request initiated by warp 3 for instruction address 0, an instruction fetch request from warp 4 for instruction address 0, and an instruction fetch request from warp 5 for instruction address 0. An example of the third instruction fetch request may include an instruction fetch request from warp 1 for instruction address 0.

[0114] In addition, Figure 8In the exemplary case of the broadcast mode shown, the processing process of thread bundles 0 to 5 for multiple instruction fetch requests for instruction address 32 (IF, PC = 32, an example of the first instruction address) in different instruction cycles is also shown. This is similar to the above-mentioned instruction fetch request for instruction address 0 and will not be repeated here.

[0115] like Figure 9 In the exemplary case of the broadcast mode shown, warps 0 to 3 in the same thread block all have instruction fetch requests for instruction address 0 (IF, PC=0, an example of the first instruction address) within T*N clock cycles. At this time, the instruction fetch requests of warps 0 to 3 are all in the candidate state.

[0116] In the T*N clock cycle, the instruction fetch request of warp 0 is selected by the instruction fetch arbitration unit, and an instruction fetch operation is performed on instruction address 0. Meanwhile, the instruction fetch requests of other warps in the candidate state, for example, the instruction fetch requests of warps 1 to 3, are ignored.

[0117] In the T*N+1 clock cycle and the T*N+2 clock cycle, the instruction fetch requests of the thread warps 1 to 3 in the candidate state are also ignored.

[0118] Until the instruction data of thread warp 0 returns from instruction address 0 within the T*N+3 clock cycle, the instruction data of thread warp 0 is broadcast simultaneously to the instruction cache write addresses of thread warps 1 to 3 that have been waiting before, which actually responds to the value requests of thread warps 1 to 3.

[0119] In the T*N+4 clock cycle, thread warp 5 also initiates an instruction fetch request for instruction address 0. Since the instruction data of thread warp 0 has been broadcast, the instruction fetch request of thread warp 5 for address 0 will not be ignored at this moment. Instead, it will go to instruction address 0 again to fetch the instruction data, repeating the previous thread warp 0 instruction fetch broadcast process.

[0120] In the T*N+5 and T*N+6 clock cycles, warp 4 also initiates instruction fetch requests for instruction address 0, but both are ignored.

[0121] In the T*N+7 clock cycle, the instruction data of thread warp 5 returns from instruction address 0, and then the instruction data of thread 5 is broadcast to the instruction cache write address of thread warp 4 that has been waiting before, which actually responds to the value request of thread warp 4.

[0122] It should be noted that in the illustrated scenario, examples of the first clock cycle include T*N+3, T*N+4, and T*N+7. Correspondingly, examples of the second clock cycle include T*N, and examples of intermediate operation cycles include T*N+1 and T*N+2. Examples of first instruction fetch requests include an instruction fetch request from warp 0 for instruction address 0, and an instruction fetch request from warp 5 for instruction address 0. Correspondingly, examples of second instruction fetch requests include an instruction fetch request from warps 1 through 3 for instruction address 0, and an instruction fetch request from warp 4 for instruction address 0.

[0123] In addition, Figure 9 In the exemplary case of the broadcast mode shown, the processing process of multiple instruction fetch requests for instruction address 32 in different instruction cycles by thread warps 0 to 5 is also shown, which is similar to the above-mentioned instruction fetch request for instruction address 0 and is not repeated here.

[0124] At least some embodiments of the present disclosure further provide an instruction scheduling device, which is used, for example, in a parallel processor, such as a general-purpose graphics processing unit (GPGPU), which is not limited in the embodiments of the present disclosure.

[0125] Figure 10 This is a schematic block diagram of an instruction scheduling device provided in some embodiments of the present disclosure. Figure 10 As shown, the instruction scheduling device 100 includes an instruction fetch arbitration unit 110 , an instruction preprocessing unit 120 and an instruction broadcast determination unit 130 .

[0126] The instruction fetch arbitration unit 110 is configured to select a first instruction fetch request for a first instruction address initiated by a first warp, and perform an instruction fetch operation for the first instruction address.

[0127] The instruction preprocessing unit 120 is configured to receive first instruction data corresponding to a first instruction fetch request returned from a first instruction address, and is configured to respond to a second instruction fetch request initiated by a second thread warp to fetch the first instruction address, and broadcast the first instruction data to the write address of the instruction data access area of ​​the first thread warp and the write address of the instruction data access area of ​​the second thread warp in a first clock cycle.

[0128] For example, the instruction pre-processing unit 120 is further configured to cancel the second instruction fetch request.

[0129] For example, the instruction pre-processing unit 120 is further configured to, when the second instruction fetch request is a candidate and is selected in the first clock cycle, cancel the instruction fetch operation corresponding to the second instruction fetch request.

[0130] For example, the instruction pre-processing unit 120 is further configured to ignore the second instruction fetch request when the second instruction fetch request is a candidate in the first clock cycle.

[0131] For example, the instruction preprocessing unit 120 is further configured to, when the first instruction fetch request and the second instruction fetch request are candidates in the second clock cycle, select the first instruction fetch request and ignore the second instruction fetch request, wherein the first clock cycle is after the second clock cycle.

[0132] For example, the instruction pre-processing unit 120 is further configured to ignore the second instruction fetch request in one or more intermediate operation cycles between the second clock cycle and the first clock cycle.

[0133] For example, the instruction preprocessing unit 120 is further configured to, in the second clock cycle, ignore the third instruction fetch request initiated by the third thread warp to fetch the first instruction address when the third instruction fetch request is a candidate; in the third clock cycle between the second clock cycle and the first clock cycle, select the third instruction fetch request to perform the instruction fetch operation on the first instruction address when the third instruction fetch request is a candidate.

[0134] For example, the first warp and the second warp belong to the same thread block or to different thread blocks.

[0135] For example, the instruction broadcast determination unit 130 is configured to record attribute information of each instruction fetch request, wherein the attribute information includes the thread block number of the thread block to which the thread warp initiating the instruction fetch request belongs and the thread warp number of the thread warp initiating the instruction fetch request in the thread block to which it belongs.

[0136] For example, the instruction broadcast determination unit 130 is further configured to: obtain the first instruction data returned from the instruction preprocessing unit 110, obtain the thread block number corresponding to the first thread warp and the thread warp number corresponding to the first thread warp according to the attribute information of the first instruction fetch request; and determine the second instruction fetch request based on the thread block number corresponding to the first thread warp and the thread warp number corresponding to the first thread warp and the thread block number and thread warp number included in the attribute information of each of the multiple candidate instruction fetch requests.

[0137] For example, the instruction broadcast determination unit 130 is further configured to generate a broadcast mask based on the program block number corresponding to the first thread warp and the program bundle number corresponding to the first thread warp, and the thread block number and the thread bundle number included in the attribute information of each of the multiple candidate instruction fetch requests, wherein the broadcast mask includes information bits corresponding to the second instruction fetch request; the instruction preprocessing unit 120 is further configured to use the broadcast mask to broadcast the first instruction data to the write address of the instruction data access area of ​​the second thread warp in the first clock cycle.

[0138] For example, the broadcast mask also includes information bits corresponding to the first instruction fetch request, and the instruction preprocessing unit 120 is also configured to use the broadcast mask to broadcast the first instruction data to the write address of the instruction data access area of ​​the first thread warp and the write address of the instruction data access area of ​​the second thread warp in the first clock cycle.

[0139] For example, the attribute information of each instruction fetch request also includes the write address of the instruction data access area, and the instruction preprocessing unit 120 is also configured to obtain the write address of the instruction data access area of ​​the first instruction fetch request through the attribute information of the first instruction fetch request, and to obtain the write address of the instruction data access area of ​​the second instruction fetch request through the attribute information of the second instruction fetch request.

[0140] For example, the attribute information of each instruction fetch request further includes a program counter, and the instruction preprocessing unit 120 is further configured to obtain the first instruction fetch address corresponding to the first instruction fetch request through the program counter of the attribute information of the first instruction fetch request.

[0141] For example, the attribute information of each instruction fetch request further includes the status of the instruction data access area. The status of the instruction data access area includes whether the instruction data access area of ​​the thread warp that initiates the instruction fetch request is not full.

[0142] For example, each unit in the instruction fetch arbitration unit 110 , the instruction pre-processing unit 120 , and the instruction broadcast determination unit 130 may be implemented by hardware, firmware, or software.

[0143] Therefore, for example, the processor may execute code and programs to implement some or all of the functions of the above modules. For another example, each unit in the instruction fetch arbitration unit 110, the instruction preprocessing unit 120, and the instruction broadcast determination unit 130 may be a hardware device to implement some or all of the functions of the above modules.

[0144] It should be noted that the instruction scheduling device 100 can be used to implement the aforementioned instruction scheduling method. For example, the instruction fetch arbitration unit 110 can be used to implement step S100 in the aforementioned instruction scheduling method. The specific implementation process and details can refer to the relevant description of the aforementioned step S100, which will not be repeated here. For example, the instruction preprocessing unit 120 can be used to implement steps S200 and S300 in the aforementioned instruction scheduling method. The specific implementation process and details can refer to the relevant description of the aforementioned steps S200 and S300, which will not be repeated here. For example, the instruction preprocessing unit 120 can also be used to implement step S400 in the aforementioned instruction scheduling method. The specific implementation process and details can refer to the relevant description of the aforementioned step S400, which will not be repeated here. For example, the instruction broadcast determination unit 130 can also be used to implement steps S210 and S220 in the aforementioned instruction scheduling method. The specific implementation process and details can refer to the relevant description of the aforementioned steps S210 and S220, which will not be repeated here.

[0145] Figure 11 A schematic block diagram of another instruction scheduling device provided in some embodiments of the present disclosure. Figure 11 As shown, the instruction scheduling device 500 includes a memory 510 and a processor 520. The instruction scheduling device can be used in a parallel processor, such as a general purpose graphics processing unit (GPGPU), which is not limited in the embodiments of the present disclosure.

[0146] For example, the memory 510 is used to non-temporarily store computer-executable instructions, and the processor 520 is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor 520, the instruction scheduling method provided by any embodiment of the present disclosure is executed.

[0147] For example, the memory 510 and the processor 520 may communicate with each other directly or indirectly. Figure 11 As shown, the instruction scheduling device 500 may further include a system bus 530, and the memory 510 and the processor 520 may communicate with each other via the system bus 530. For example, the processor 520 may access the memory 510 via the system bus 1006. For example, in other examples, the memory 510 and the processor 520 and other components may communicate via a network on chip (NOC) connection.

[0148] For example, the processor 520 can control other components in the instruction scheduling device to perform the desired functions. The processor 520 can be a device with data processing capabilities and / or program execution capabilities, such as a central processing unit (CPU), a tensor processing unit (TPU), a network processor (NP), or a graphics processing unit (GPU). It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0149] For example, the memory 510 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, a flash memory, etc.

[0150] For example, one or more computer instructions may be stored in the memory 510, and the processor 520 may execute the computer instructions to implement various functions. Various applications and various data may also be stored in the computer-readable storage medium, such as instruction scheduling code and various data used and / or generated by the application.

[0151] For example, when some computer instructions stored in the memory 510 are executed by the processor 520 , one or more steps in the instruction scheduling method described above may be performed.

[0152] For example, Figure 11 As shown, the instruction scheduling apparatus 500 may further include an input interface 540 that allows external devices to communicate with the instruction scheduling apparatus 500. For example, the input interface 540 may be used to receive instructions from an external computer device, from a user, etc. The instruction scheduling apparatus 500 may further include an output interface 550 that interconnects the instruction scheduling apparatus 500 and one or more external devices. For example, the instruction scheduling apparatus 500 may display images, etc., via the output interface 550.

[0153] For example, for a detailed description of the processing process of the instruction scheduling method in the above embodiment of the present disclosure, reference can be made to the relevant description in the embodiment of the above instruction scheduling method, and repeated parts will not be repeated.

[0154] It should be noted that the instruction scheduling device provided in the embodiments of the present disclosure is exemplary and not restrictive. According to actual application needs, the instruction scheduling device may also include other conventional components or structures. For example, in order to realize the necessary functions of the instruction scheduling device, those skilled in the art may set other conventional components or structures according to the specific application scenarios, and the embodiments of the present disclosure do not limit this.

[0155] The technical effects of the instruction scheduling device provided by the embodiments of the present disclosure can be referred to the corresponding description of the instruction scheduling method in the above embodiments, and will not be repeated here.

[0156] At least some embodiments of the present disclosure further provide a processor comprising at least one computing unit, wherein the computing unit comprises an instruction scheduling device as provided in any of the aforementioned embodiments. For example, the processor is a general-purpose graphics processor, which comprises a plurality of computing units, each of which comprises an instruction scheduling device, and may further comprise a plurality of computing cores, register files, instruction caches, data caches, and the like. Each computing core comprises an arithmetic logic unit (ALU), a floating-point computing unit, and the like. For example, when multiple thread blocks are executed in the same computing unit, the thread warps in the computing unit may come from the same thread block or different thread blocks, and all threads in the same thread warp may be executed in a SIMD manner.

[0157] At least some embodiments of the present disclosure also provide a non-transitory storage medium. Figure 12 A schematic diagram of a non-transitory storage medium provided in some embodiments of the present disclosure. For example, Figure 12 As shown, the storage medium 600 non-temporarily stores computer executable instructions 601. When the non-temporary computer executable instructions 601 are executed by a computer (including a processor), the instruction scheduling method provided by any embodiment of the present disclosure can be executed.

[0158] For example, one or more computer instructions may be stored in the storage medium 600. Some of the computer instructions stored in the storage medium 600 may be, for example, instructions for implementing one or more steps in the above-mentioned instruction scheduling method.

[0159] For example, the storage medium may include a tablet computer storage component, a personal computer hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), compact disc read-only memory (CD-ROM), flash memory, or any combination thereof, or other suitable storage medium. For example, the storage medium 600 may include the memory 510 in the aforementioned instruction scheduling device 500.

[0160] The technical effects of the storage medium provided by the embodiments of the present disclosure can be referred to the corresponding description of the instruction scheduling method in the above embodiments, which will not be repeated here.

[0161] Regarding this disclosure, the following points need to be explained:

[0162] (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure, and other structures may refer to conventional designs.

[0163] (2) Unless there is any conflict, the features of the same embodiment and different embodiments of the present disclosure may be combined with each other.

[0164] The above are only specific embodiments of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. An instruction scheduling method, comprising: Selecting a first instruction fetch request for a first instruction address initiated by a first thread warp, and performing an instruction fetch operation for the first instruction address; receiving first instruction data corresponding to the first instruction fetch request returned from the first instruction address; In response to a second instruction fetch request initiated by the second thread warp to fetch the first instruction address, the first instruction data is broadcast to the write address of the instruction data access area of ​​the first thread warp and the write address of the instruction data access area of ​​the second thread warp in a first clock cycle, wherein the instruction data includes multiple executable instructions, the first thread warp and the second thread warp belong to the same thread block or different thread blocks, and the instruction fetch addresses targeted by the instruction fetch requests of the first thread warp and the second thread warp are the same.

2. The method according to claim 1, further comprising: The second instruction fetch request is canceled.

3. The method according to claim 1, wherein In a case where the second instruction fetch request is a candidate and is selected in the first clock cycle, an instruction fetch operation corresponding to the second instruction fetch request is canceled.

4. The method according to claim 3, wherein: The second instruction fetch request is a candidate in the first clock cycle and is ignored.

5. The method according to claim 1, wherein The first instruction fetch request and the second instruction fetch request are candidates in a second clock cycle, the first instruction fetch request is selected, and the second instruction fetch request is ignored. The first clock cycle is located after the second clock cycle.

6. The method according to claim 5, wherein: In one or more intermediate operation cycles between the second clock cycle and the first clock cycle, the second instruction fetch request is ignored.

7. The method according to claim 5, wherein: In the second clock cycle, a third instruction fetch request initiated by a third warp to fetch an instruction for the first instruction address is a candidate but is ignored; In a third clock cycle between the second clock cycle and the first clock cycle, the third instruction fetch request is a candidate and is selected to perform an instruction fetch operation on the first instruction address.

8. The method according to claim 1, further comprising: Recording attribute information of each instruction fetch request, wherein the attribute information includes the thread block number of the thread block to which the thread warp initiating the instruction fetch request belongs and the thread warp number of the thread warp initiating the instruction fetch request in the thread block to which it belongs.

9. The method according to claim 8, further comprising: In response to returning the first instruction data, obtaining, according to the attribute information of the first instruction fetch request, a thread block number corresponding to the first warp and a warp number corresponding to the first warp; The second instruction fetch request is determined based on the thread block number corresponding to the first warp, the warp number corresponding to the first warp, and the thread block number and the warp number included in the attribute information of each of the candidate instruction fetch requests.

10. The method according to claim 9, wherein: Determining the second instruction fetch request based on the thread block number corresponding to the first warp and the warp number corresponding to the first warp, and the thread block number and the warp number included in the attribute information of each of the plurality of candidate instruction fetch requests, includes: generating a broadcast mask, wherein the broadcast mask includes information bits corresponding to the second instruction fetch request; The first instruction data is broadcasted to a write address of an instruction data access area of ​​the second warp in the first clock cycle using the broadcast mask.

11. The method according to claim 10, wherein: The broadcast mask further includes an information bit corresponding to the first instruction fetch request, The first instruction data is broadcasted to a write address of an instruction data access area of ​​the first warp and a write address of an instruction data access area of ​​the second warp in the first clock cycle using the broadcast mask.

12. The method according to claim 9, wherein The attribute information of each instruction fetch request also includes the write address of the instruction data access area. Acquire a write address of the instruction data access area of ​​the first instruction fetch request through the attribute information of the first instruction fetch request; The write address of the instruction data access area of ​​the second instruction fetch request is obtained through the attribute information of the second instruction fetch request.

13. The method according to claim 9, wherein: The attribute information of each instruction fetch request also includes a program counter, A first instruction fetch address corresponding to the first instruction fetch request is obtained through a program counter of the attribute information of the first instruction fetch request.

14. The method according to claim 9, wherein The attribute information of each instruction fetch request further includes a status of an instruction data access area. The status of the instruction data access area includes whether the instruction data access area of ​​the thread warp that initiates the instruction fetch request is not full.

15. An instruction scheduling device, comprising: an instruction fetch arbitration unit configured to select a first instruction fetch request for a first instruction address initiated by a first thread warp, and perform an instruction fetch operation for the first instruction address; An instruction preprocessing unit is configured to receive first instruction data corresponding to the first instruction fetch request returned from the first instruction address, and is configured to respond to a second instruction fetch request initiated by the second thread warp to fetch the first instruction address, and broadcast the first instruction data to the write address of the instruction data access area of ​​the first thread warp and the write address of the instruction data access area of ​​the second thread warp in a first clock cycle, wherein the instruction data includes multiple executable instructions, the first thread warp and the second thread warp belong to the same thread block or different thread blocks, and the instruction fetch requests of the first thread warp and the second thread warp are targeted at the same instruction fetch address.

16. The instruction scheduling device according to claim 15, wherein: The instruction pre-processing unit is further configured to cancel the second instruction fetch request.

17. The instruction scheduling device according to claim 15, wherein: The instruction preprocessing unit is further configured to, when the second instruction fetch request is a candidate and is selected in the first clock cycle, cancel an instruction fetch operation corresponding to the second instruction fetch request.

18. The instruction scheduling device according to claim 17, wherein: The instruction pre-processing unit is further configured to ignore the second instruction fetch request when the second instruction fetch request is a candidate in the first clock cycle.

19. The instruction scheduling device according to claim 15, wherein: The instruction preprocessing unit is further configured to, when the first instruction fetch request and the second instruction fetch request are candidates in a second clock cycle, select the first instruction fetch request and ignore the second instruction fetch request; The first clock cycle is located after the second clock cycle.

20. The instruction scheduling device according to claim 19, wherein: The instruction pre-processing unit is further configured to ignore the second instruction fetch request in one or more intermediate operation cycles between the second clock cycle and the first clock cycle.

21. The apparatus according to claim 19, wherein The instruction preprocessing unit is further configured to: In the second clock cycle, if a third instruction fetch request initiated by a third warp to fetch an instruction for the first instruction address is a candidate, ignoring the third instruction fetch request; In a third clock cycle between the second clock cycle and the first clock cycle, if the third instruction fetch request is a candidate, the third instruction fetch request is selected to perform an instruction fetch operation on the first instruction address.

22. The instruction scheduling device according to claim 15, further comprising an instruction broadcast determination unit, wherein: The instruction broadcast determination unit is configured to record attribute information of each instruction fetch request, The attribute information includes the thread block number of the thread block to which the thread warp that initiates the instruction fetch request belongs and the thread warp number of the thread warp that initiates the instruction fetch request in the thread block to which it belongs.

23. The instruction scheduling device according to claim 22, wherein: The instruction broadcast determination unit is further configured to: Obtaining the first instruction data returned from the instruction preprocessing unit, and obtaining a thread block number corresponding to the first warp and a warp number corresponding to the first warp according to attribute information of the first instruction fetch request; as well as The second instruction fetch request is determined based on the thread block number corresponding to the first warp, the warp number corresponding to the first warp, and the thread block number and the warp number included in the attribute information of each of the candidate instruction fetch requests.

24. The instruction scheduling device according to claim 23, wherein: The instruction broadcast determination unit is further configured to generate a broadcast mask based on the block number and the warp number corresponding to the first warp, and the thread block number and the warp number included in the attribute information of each of the candidate instruction fetch requests, wherein the broadcast mask includes an information bit corresponding to the second instruction fetch request; The instruction pre-processing unit is further configured to broadcast the first instruction data to a write address of the instruction data access area of ​​the second warp in a first clock cycle using the broadcast mask.

25. The instruction scheduling device according to claim 24, wherein: The broadcast mask further includes an information bit corresponding to the first instruction fetch request, The instruction preprocessing unit is further configured to broadcast the first instruction data to a write address of the instruction data access area of ​​the first warp and a write address of the instruction data access area of ​​the second warp in a first clock cycle using the broadcast mask.

26. The instruction scheduling device according to claim 23, wherein: The attribute information of each instruction fetch request also includes the write address of the instruction data access area, The instruction preprocessing unit is further configured to: obtaining a write address of the instruction data access area of ​​the first warp through the attribute information of the first instruction fetch request, and The write address of the instruction data access area of ​​the second warp is obtained according to the attribute information of the second instruction fetch request.

27. A processor comprising at least one computing unit, wherein: The computing unit includes the instruction scheduling device as described in any one of claims 15-26.

28. An instruction scheduling device, comprising: a memory for non-transitory storage of computer-executable instructions; as well as a processor for executing the computer-executable instructions, The computer-executable instructions, when executed by the processor, execute the instruction scheduling method according to any one of claims 1 to 14.

29. A non-transitory storage medium that non-transitory stores computer-executable instructions, wherein: When the computer-executable instructions are executed by a computer, the instruction scheduling method according to any one of claims 1 to 14 is executed.

Images