Methods and apparatus for scheduling threads, compilation methods and apparatus, electronic devices, storage media

Abstract

This disclosure provides a method and apparatus for scheduling threads, a compilation method and apparatus, an electronic device, and a storage medium. The method for scheduling threads includes: mapping a first register to a register for recording a return program counter for a thread bundle to be split and synchronized, using a first software instruction; mapping a second register to a register for recording thread synchronization information, including a thread mask and a target program counter, whereby the thread mask records whether the currently participating thread in the split and synchronization has been executed, and the target program counter records the target instruction address to jump to after synchronization; and performing thread bundle split and synchronization using the mapped return program counter and thread synchronization information. This disclosure makes the use of registers and thread bundle split and synchronization more flexible.

Description

Technical Field

[0001] One or more embodiments of this disclosure relate to the field of data processing, and more specifically to methods and apparatus for scheduling threads, compilation methods and apparatus, electronic devices, and storage media. Background Technology

[0002] Some types of processors, such as graphics processing units (GPUs) or general-purpose graphics processing units (GPGPUs), execute groups of threads called warps in a single-instruction, multi-threaded (SIMT) manner, where multiple threads within a warp execute the same instructions in parallel. A warp is a group of threads grouped together and executing simultaneously on a processor. For example, a warp might contain 32 threads, such as those in a ray tracing application, each tracing a single ray.

[0003] When threads take different execution paths, they cannot continue to execute in parallel; these different paths are temporarily serialized. This is called warp divergence. For example, in the case of conditional statements, the hardware will execute one branch completely (the AB code segment of the if branch) before switching to another branch (the XY code segment of the else branch). The next instruction to be executed in the first thread and the next instruction to be executed in the second thread are located at different program counter positions. The processor may choose one path to execute, while idle threads will execute one or more other paths.

[0004] However, when an execution thread stalls, the idle thread remains idle, waiting for the stall to end, and the execution resources of the thread bundle cannot be fully utilized. Typically, GPUs or GPGPUs executing SIMT thread bundles can handle thread stalls by switching execution between the entire bundle (e.g., executing code segment X while code segment B stalls), but solutions to improve computational resource utilization and runtime efficiency are still needed. Summary of the Invention

[0005] According to one aspect of this disclosure, at least one embodiment provides a method for scheduling threads, comprising: mapping a first register in a processor to a register for recording a return program counter for a thread bundle to be subjected to thread bundle differentiation and synchronization via a first software instruction, wherein the return program counter is used to record the address of the next instruction to be executed in each thread of the thread bundle; mapping a second register in a processor to a register for recording thread synchronization information for the thread bundle via a second software instruction, wherein the thread synchronization information includes a thread mask and a target program counter, the thread mask is used to record whether the thread currently participating in the thread bundle differentiation and synchronization has been executed, and the target program counter is used to record the address of the target instruction to jump to after synchronization is completed; and performing thread bundle differentiation and synchronization using the mapped return program counter and thread synchronization information.

[0006] In some embodiments, thread bundle differentiation and synchronization are performed using the mapped return program counter and thread synchronization information, including: in response to a branch instruction, recording the return program counter, thread mask, and target program counter of the threads in the thread bundle; in response to a thread switching instruction, selecting the first address of the thread to be switched as the return program counter to continue execution; in response to a thread synchronization instruction, determining, based on the thread mask, that there are still unfinished threads among the currently participating threads in synchronization, selecting the second address among the unfinished threads as the return program counter to continue execution, and in response to determining, based on the thread mask, that all currently participating threads in synchronization have been completed, continuing execution based on the target program counter.

[0007] In some embodiments, the method further includes: searching for a free vector register region in the processor that meets the vector register space size required by the thread bundle; if a free vector register region exists, returning the program counter to the free vector register region and releasing the first register; if no free vector register region exists, returning the program counter to the first register.

[0008] In some embodiments, the storage space of the free vector register region is smaller than the storage space of the first register.

[0009] In some embodiments, the method further includes: retrieving two free scalar register regions in the processor that satisfy the scalar register space size according to the scalar register space size required by the thread bundle; if two free scalar register regions exist, mapping the thread mask and the target program counter to the two free scalar register regions respectively, and releasing the second register; if two free scalar register regions do not exist, mapping the thread mask and the target program counter to the second register respectively.

[0010] In some embodiments, the total storage space of the two free scalar register regions is less than the storage space of the second register.

[0011] In some embodiments, in response to a thread switching instruction, selecting the first address of the thread to be switched to as the return program counter to continue execution includes: selecting the address of the highest priority thread from the threads that have not yet finished executing as the return program counter to continue execution.

[0012] In some embodiments, selecting the address of the highest priority thread from the threads that have not yet finished executing as the return program counter to continue execution includes: counting the number of repetitions of each address in the return program counter of the switchable threads; selecting the address with the most repetitions to continue execution in response to the existence of an address with the most repetitions; and selecting the address with the smallest value among the multiple addresses with the most repetitions to continue execution in response to the existence of multiple addresses with the most repetitions.

[0013] In some embodiments, selecting a second address as the return program counter from the threads that have not yet finished executing to continue execution includes: selecting the address of the thread with the highest priority from the threads that have not yet finished executing as the second address, and using the second address as the return program counter to continue execution.

[0014] In some embodiments, selecting the address of the highest priority thread among the threads that have not yet finished executing as the second address, and using the second address as the return program counter to continue execution, includes: counting the number of repetitions of each address in the return program counter of the threads that have not yet finished executing; in response to the existence of an address with the most repetitions, selecting the address with the most repetitions to continue execution; in response to the existence of multiple addresses with the most repetitions, selecting the address with the smallest value among the multiple addresses with the most repetitions to continue execution.

[0015] In some embodiments, the method further includes: calculating priorities by: grouping the addresses of threads that have not yet finished executing into candidate thread groups, wherein multiple threads with duplicate addresses are grouped into one group, and different threads with different addresses are each grouped into different groups; performing multi-dimensional scoring on the candidate thread groups according to a multi-dimensional scoring formula based on a weighting factor and feedback information from the runtime hardware predictor; and using the score as a priority.

[0016] In some embodiments, the hardware predictor includes an instruction cache predictor and a data cache predictor. The instruction cache predictor maintains a hit prediction table and obtains an instruction cache hit score for a subset of address bits by indexing a portion of the address bits in the hit prediction table based on a portion of the address bits of the thread's program counter. This score represents the probability that a subset of address bits will be predicted to hit in the instruction cache. The data cache predictor calculates a data locality score between the candidate thread group and the currently executing thread group by recording an address record table that records a portion of the address bits of the thread's most recent memory operation. This score represents the distance between the representative address of the candidate thread group and the representative address of the currently executing thread group.

[0017] In some embodiments, the hit prediction table includes four instruction cache hit rate states: a first predicted miss state, a second predicted miss state, a third predicted hit state, and a fourth predicted hit state. The probability of a predicted miss in the first predicted miss state is higher than that in the second predicted miss state, and the probability of a predicted hit in the third predicted hit state is lower than that in the fourth predicted hit state. The higher the probability of a certain address bit being predicted to hit in the instruction cache, the higher the instruction cache hit rate score. The method further includes updating the instruction cache hit rate state of the hit prediction table according to the actual hit or miss result of the highest priority address in the instruction cache each time an instruction is fetched from the highest priority address to continue execution.

[0018] In some embodiments, the data cache predictor calculates a data locality score by: maintaining an address record table that records a portion of the address of the most recent memory operation of a thread; selecting a representative address of a thread from the address record table of a portion of the address of the most recent memory operation of a thread; calculating the distance between the representative address of the candidate thread group and the representative address of the currently executing thread group; and calculating a data locality score based on the distance, wherein the closer the distance, the higher the data locality score.

[0019] In some embodiments, the formula for multi-dimensional scoring is:

[0020] Score = w_i * ICache_Score + w_d * Dcache_Score + w_g * Group_Size_Score,

[0021] Where w_i, w_d, and w_g are weighting factors.

[0022] ICache_Score is the instruction cache hit score output by the instruction cache predictor.

[0023] Dcache_Score is the data locality score output by the data cache predictor.

[0024] Group_Size_Score is the score for the number of threads in a candidate thread group. The more threads there are, the higher the score.

[0025] In some embodiments, the method further includes: adjusting the weighting factor according to the application type of the thread, wherein for compute-intensive applications, w_d is greater than w_g and w_g is greater than w_i; for memory-intensive applications, w_d is greater than w_g and w_g is greater than w_i; and for control flow-intensive applications, w_i is greater than w_d and w_d is greater than w_g.

[0026] According to another aspect of this disclosure, at least one embodiment provides an apparatus for scheduling threads, comprising: a first mapping module configured to map a first register in a processor to a register for recording a return program counter for a thread bundle to be split and synchronized, via a first software instruction, wherein the return program counter is used to record the processor address of the next instruction to be executed in each thread of the thread bundle; a second mapping module configured to map a second register in a processor to a register for recording thread synchronization information for the thread bundle via a second software instruction, wherein the thread synchronization information includes a thread mask and a target program counter, the thread mask being used to record whether the thread currently participating in the split and synchronized thread bundle has been executed, and the target program counter being used to record the processor address of the target instruction to jump to after synchronization is completed; and an execution module configured to execute the split and synchronized thread bundle using the mapped return program counter and the thread synchronization information.

[0027] According to another aspect of this disclosure, at least one embodiment provides an apparatus for scheduling threads, comprising: at least one memory storing computer instructions; and at least one processor configured to execute the computer instructions in the memory to perform a method according to at least one embodiment of this disclosure.

[0028] According to another aspect of this disclosure, at least one embodiment provides a compilation method, comprising: inserting a first software instruction before a branch instruction in the source code, for mapping a first register in the processor to a register recording a return program counter for a thread bundle to be split and synchronized, wherein the return program counter is used to record the address of the next instruction to be executed in each thread of the thread bundle; and inserting a second software instruction before the branch instruction, for mapping a second register in the processor to a register recording thread synchronization information for the thread bundle, wherein the thread synchronization information includes a thread mask and a target program counter, the thread mask being used to record whether the thread currently participating in the split and synchronized thread bundle has been executed, and the target program counter being used to record the address of the target instruction to jump to after synchronization is completed.

[0029] In some embodiments, the compilation method further includes: inserting a save-the-state instruction at the branch instruction to record the return program counter, thread mask, and target program counter of the threads in the thread bundle; inserting a thread switching instruction at the instruction requiring thread switching to select the first address of the thread to be switched as the return program counter to continue execution; and inserting a thread synchronization instruction at the instruction requiring synchronization to determine, based on the thread mask, whether there are still unfinished threads participating in the synchronization, selecting the second address of the unfinished threads as the return program counter to continue execution, and continuing execution based on the target program counter in response to determining, based on the thread mask, that all currently participating threads in the synchronization have been completed.

[0030] According to another aspect of this disclosure, at least one embodiment provides a compilation apparatus, comprising: a first insertion module configured to insert a first software instruction before a branch instruction in source code, for mapping a first register in a processor to a register recording a return program counter for a thread bundle to be split and synchronized, wherein the return program counter is used to record the address of the next instruction to be executed in each thread of the thread bundle; and a second insertion module configured to insert a second software instruction before a branch instruction, for mapping a second register in a processor to a register recording thread synchronization information for the thread bundle, wherein the thread synchronization information includes a thread mask and a target program counter, the thread mask being used to record whether the thread currently participating in the split and synchronized thread bundle has been executed, and the target program counter being used to record the target instruction address to jump to after synchronization is completed.

[0031] In some embodiments, the compilation apparatus further includes: a third insertion module configured to insert a save-the-state instruction at a branch instruction for recording the return program counter, thread mask, and target program counter of the threads in the thread bundle; a fourth insertion module configured to insert a thread switching instruction at an instruction requiring thread switching for selecting a first address of the thread to be switched as the return program counter to continue execution; and a fifth insertion module configured to insert a thread synchronization instruction at an instruction requiring synchronization for determining, based on the thread mask, that there are still unfinished threads among the currently participating threads in synchronization, selecting a second address as the return program counter among the unfinished threads to continue execution, and continuing execution based on the target program counter in response to determining, based on the thread mask, that all currently participating threads in synchronization have been completed.

[0032] According to another aspect of this disclosure, at least one embodiment provides a compilation apparatus, comprising: at least one memory for storing computer instructions; and at least one processor for reading the computer instructions from the at least one memory and performing a method according to at least one embodiment of this disclosure.

[0033] According to another aspect of this disclosure, at least one embodiment provides an electronic device including: means according to at least one embodiment of the present disclosure described above.

[0034] According to another aspect of this disclosure, at least one embodiment provides a non-transitory computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed by at least one processor, cause at least one processor to perform a method according to at least one embodiment of this disclosure.

[0035] According to another aspect of this disclosure, at least one embodiment provides a computer program product including computer instructions, wherein, when executed by at least one processor, the computer instructions cause the at least one processor to perform a method according to at least one embodiment of this disclosure.

[0036] Thus, by using software instructions to utilize general-purpose vector / scalar registers as dedicated registers for recording the return program counter, and dedicated registers for recording the thread mask and the target program counter, respectively, it is not limited to the dedicated register locations specified by the hardware, and register resources can be flexibly reused to achieve thread bundle differentiation and synchronization. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 A schematic diagram of a general-purpose graphics processing unit (GPGPU) is shown.

[0039] Figure 2 This diagram illustrates a serial execution process involving SIMT branches, handled through thread bundle differentiation.

[0040] Figure 3 A schematic diagram is shown of an improved scheme for handling serial execution processes containing SIMT branches through thread bundle differentiation.

[0041] Figure 4 A flowchart of a compilation method according to at least one embodiment of the present disclosure is shown.

[0042] Figure 5 A schematic diagram is shown of a compiler according to at least one embodiment of the present disclosure inserting operators into conditional branch instructions executed by a thread bundle.

[0043] Figure 6 At least one embodiment of the present disclosure is shown as follows Figure 5 The diagram shows the overall flowchart of program execution after the insertion operator.

[0044] Figure 7 A flowchart of a method for scheduling threads according to at least one embodiment of the present disclosure is shown.

[0045] Figure 8A A schematic diagram of registers within an execution unit (EU) in a processor according to at least one embodiment of the present disclosure is shown.

[0046] Figure 8B An example diagram of an instruction cache prediction hit table according to at least one embodiment of the present disclosure is shown.

[0047] Figure 8C An example diagram of an address record table according to at least one embodiment of the present disclosure is shown.

[0048] Figure 9 A block diagram of an apparatus for scheduling threads according to at least one embodiment of the present disclosure is shown.

[0049] Figure 10A block diagram of an apparatus for another scheduling thread according to at least one embodiment of the present disclosure is shown.

[0050] Figure 11 A block diagram of a compilation apparatus according to at least one embodiment of the present disclosure is shown.

[0051] Figure 12 A block diagram of another compilation apparatus according to at least one embodiment of the present disclosure is shown.

[0052] Figure 13 A block diagram of an electronic device according to at least one embodiment of the present disclosure is shown.

[0053] Figure 14 A schematic block diagram of an electronic device according to at least one embodiment of the present disclosure is shown. Detailed Implementation

[0054] Referring now to specific embodiments of this disclosure, examples of which are illustrated in the accompanying drawings. Although this application will be described in conjunction with specific embodiments, it will be understood that it is not intended to limit this application to the described embodiments. Rather, it is intended to cover variations, modifications, and equivalents included within the spirit and scope of this disclosure. It should be noted that the method steps described herein can be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of both.

[0055] In this article, "multiple" refers to two or more. "Or" describes the relationship between related objects, indicating that there can be three relationships. For example, A or B can represent: A alone, both A and B, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0056] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0057] The following is an explanation of some of the terms used in this article:

[0058] Single Instruction Multiple Threads (SIMT) is a core parallel computing model in GPU architecture. It drives multiple threads to execute in parallel through a single instruction stream, with each thread processing an independent dataset. Computational tasks are decomposed into thread blocks, each containing multiple warps (typically groups of 32 threads). A thread warp is the basic scheduling unit in SIMT; all threads receive the same instruction within the same instruction cycle but can independently execute branch paths based on their own data.

[0059] Program Counter (PC): After each instruction is executed, the PC automatically increments (usually by 4 or the instruction length) to point to the next instruction. When a jump instruction (such as JMP or CALL) is encountered, the PC is forcibly modified to the target address to achieve program flow control.

[0060] Barrier Participation Mask: Used to track which threads in a warp have reached a specific barrier point (i.e., a merge synchronization point). Each warp maintains a mask bitmap, with each bit corresponding to the barrier participation state of a thread. For example, when thread 0 reaches the barrier, bit 0 of the mask is set to 1; when thread 1 arrives, bit 1 is set to 1, and so on.

[0061] Artificial intelligence processors, such as graphics processing units (GPUs) or general-purpose graphics processing units (GPGPUs), can be used for data computation in neural networks and deep learning. Specific applications include image processing (real-time video processing and analysis, image super-resolution and enhancement, style transfer and generation), computer vision (face recognition and authentication, object detection and tracking), and other computationally intensive tasks (gene sequencing and bioinformatics, financial analysis and risk management, virtual reality and augmented reality). When GPUs or GPGPUs process large amounts of data, it is necessary to improve the utilization of computing resources and operational efficiency in terms of thread scheduling.

[0062] Figure 1 A schematic diagram of a general-purpose graphics processing unit (GPGPU) is shown.

[0063] like Figure 1 As shown, a general-purpose graphics processor is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can be a streaming processor cluster (SPC), such as including... Figure 1The diagram shows the 1st streaming processor cluster, ..., the Mth streaming processor cluster, where M is a positive integer greater than 1. In a general-purpose graphics processor, one streaming processor cluster processes one computational task, or multiple streaming processor clusters process one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0064] like Figure 1 As shown, taking the first streaming processor cluster as an example, a streaming processor cluster includes multiple computing units, for example... Figure 1 The computation unit (CU) is structured as the 1st, 2nd, ..., Nth computation unit, where N is a positive integer. Each CU performs arithmetic and logical operations, such as accumulation, reduction, and standard addition, subtraction, multiplication, and division. A CU contains multiple cores, each including an arithmetic logic unit (ALU), a floating-point unit, etc., which execute specific computational tasks. Furthermore, the CU also includes registers (e.g., ...). Figure 1 The register file and shared memory in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared memory in a computing unit is used to share data between the cores of that computing unit.

[0065] Each computing unit also provides a tensor core for performing tensor-related computations, such as tensor shrinking operations. Tensor cores can accelerate tensor operations such as matrix multiplication. Tensor cores across multiple computing units can be scheduled and controlled uniformly.

[0066] Each streaming processor cluster also provides a buffer for caching data across N computing units within the streaming processor cluster.

[0067] In parallel computing, computational tasks are typically executed by multiple threads. These threads are divided into multiple thread blocks before execution in a general-purpose graphics processor (or parallel computing processor), and then dispatched via a thread block distribution module. Figure 1 (Not shown in the image) Multiple thread blocks are distributed to various computation units. All threads in a thread block must be assigned to the same computation unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute in the same computation unit or in different computation units.

[0068] In each computing unit, the thread beam scheduling / distribution module ( Figure 1(Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can be executed concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory execution instructions are issued to shared memory within the computing unit or further issued to intermediate-level caches, global caches, or global memory (e.g., High Bandwidth Memory, HBM) for read and write operations, etc.

[0069] Figure 2 This diagram illustrates a serial execution process involving SIMT branches, handled through thread bundle differentiation.

[0070] When a thread bundle encounters a conditional branch, it undergoes differentiation. The hardware first executes the branch that meets the condition (e.g., code segment AB in an if branch). While code segment A stalls, it waits for the stall to finish before executing code segment B, blocking other threads that do not meet the condition. After code segment B finishes execution and stalls, it switches to executing code segment X in another branch (e.g., code segment XY in an else branch). While code segment X stalls, it waits for the stall to finish before executing code segment Y. After code segment Y finishes, the next thread Z is executed synchronously. The above describes the process of thread bundle differentiation and synchronization.

[0071] Here, "stagnation" refers to delays caused by, for example, waiting for memory access, synchronization, or other operations.

[0072] In this scenario, it is necessary to wait for code segments A and B in one branch to finish executing before another branch can be executed, which cannot make good use of other idle threads, resulting in low resource utilization and low running efficiency.

[0073] Figure 3 A schematic diagram is shown of an improved scheme for handling serial execution processes containing SIMT branches through thread bundle differentiation.

[0074] In this scenario, when a thread bundle encounters a conditional branch, the hardware first executes the branch that satisfies the condition. If (e.g., code segment A in the if branch) finishes execution and pauses, an idle thread is scheduled to execute code segment X in the else branch. During the pause after code segment X finishes execution, an idle thread is scheduled to execute code segment B in the if branch. During the pause after code segment B finishes execution, an idle thread is scheduled to execute code segment Y in the else branch. During the pause after code segment Y finishes execution, an idle thread is scheduled to execute a portion of code segment Z1 within the next code segment Z. After the pause after code segment Y finishes execution, an idle thread is scheduled to execute another portion of code segment Z2 within code segment Z. Then, the subsequent thread bundles are executed synchronously.

[0075] In this scenario, when a code segment on one branch is stalled, other idle threads can be scheduled to switch to another branch to execute the code segment of that branch, thereby improving resource utilization and running efficiency.

[0076] Implementing the above independent thread scheduling method in hardware requires switching control between branches of different execution paths when the execution branch encounters a stall.

[0077] When a thread bundle encounters a conditional branch, thread scheduling allows threads to independently execute different paths based on the condition (such as if / else branches). Each thread maintains an independent program counter (PC) and thread state register, eliminating the need for synchronization across the entire thread bundle. Hardware dynamically divides threads within a bundle into active and inactive groups. Threads in the active group execute instructions for the current path, while threads in the inactive group are suspended (without consuming execution resources). Grouping can be dynamically adjusted, such as merging them again when a branch path ends.

[0078] The following describes the entire process of hardware implementation of the above independent thread scheduling method:

[0079] 1. Before executing branch code, the BSSY (Barrier Sync) instruction (used to mark thread synchronization points) passes the currently active thread mask to the barrier participation mask, indicating which threads are participating in this synchronization (if subsequent threads exit or suspend due to branching, the recorded mask will not be affected). When a thread executes the BSSY barrier ID (barid) (barid is a parameter of the BSSY instruction, representing the unique identifier of the barrier) and label instructions, the hardware will: record the currently active thread mask (ActiveThreadMask): marking which threads are currently active (not exited or suspended); set the barrier participation mask (BarrierParticipationMask): copying the ActiveThreadMask to the participation mask of the barrier ID (barid), clarifying which threads need to participate in subsequent synchronization; and jump to the label: continuing execution of the branch code.

[0080] 2. Each time an instruction is executed, the next program count (Next_pc) is recorded in the thread program counter (rPC) register for each thread, which is the address of the next instruction to be executed.

[0081] 3. The scheduler selects a group of threads with the same program count (PC) from the rPC register and issues them (if blocked). When a thread encounters an if / else branch, it updates the rPC independently based on the condition: For threads that meet the if condition, the rPC jumps to the if block address. For threads that do not meet the condition, the rPC jumps to the else block address.

[0082] 4. If an instruction is stalled or has a long latency, the corresponding thread state is set to yield.

[0083] 5. The scheduler selects another group of threads from rPC (BarrierState == 0, Thread State == ready) and issues them; (otherwise, it blocks).

[0084] 6. Upon encountering the BSYNC (Barrier Sync) instruction (used to wait for all threads that have joined the same barrier to reach the synchronization point), the barrier status (BarrierStat) of the corresponding barrier ID (barid) is set to 1 (synchronization point reached).

[0085] 7. Return to step 5;

[0086] 8. When BarrierState == BarrierParticipationMask, all threads reach the synchronization point, synchronization is completed, all thread bundles have finished executing, and the thread bundles continue to execute subsequent instructions synchronously.

[0087] The above scheme implicitly implements thread switching and independent thread scheduling through the BSSY / BSYNC instructions. The above information needs to be recorded for each thread bundle, as shown in Table 1 below, which is a total of 1120 bits. This is costly and requires a large hardware implementation area.

[0088] Table 1

[0089] name Number of digits BarrierParticipationMask 32 BarrierState 32 Thread State 32 Thread rPC 32*32 = 1024

[0090] Furthermore, implementing the above scheme requires a dedicated rPC register and a complex hardware scheduler specifically for this thread scheduling, which increases hardware costs (circuit size, power consumption, etc.). Since different thread bundles have different register resource requirements, certain register regions on the physical register bank often become idle, failing to fully utilize register resources. In addition, thread switching and thread synchronization rely on predefined instruction logic, lacking flexibility and failing to meet actual thread requirements.

[0091] The defects in the above solutions were discovered by the inventors of this disclosure through careful research and creative labor. The discovery process of the above problems and the solutions proposed by at least one embodiment disclosed below for the above problems are all creative contributions.

[0092] According to at least one embodiment of this disclosure, by designing specific instructions, general-purpose registers can be flexibly reused in software and idle register resources can be fully utilized. In terms of hardware implementation, only a small amount of logic needs to be implemented to achieve the function of independent thread scheduling. Moreover, by considering the actual thread requirements, the thread to which to switch and the thread to be executed before reaching the synchronization point can be flexibly determined in order to optimize the execution performance of the thread.

[0093] Specifically, according to at least one embodiment of this disclosure, a method and apparatus for scheduling threads, a compilation method, and a compilation apparatus are proposed. This involves mapping a first register in the processor to a register that records a return program counter for a thread bundle to be differentiated and synchronized, using a first software instruction. The return program counter records the address of the next instruction to be executed in each thread of the thread bundle. Then, a second register in the processor is mapped to a register that records thread synchronization information for the thread bundle, using a second software instruction. This thread synchronization information includes a thread mask and a target program counter. The thread mask records whether the currently participating thread bundle differentiation and synchronization has been completed, and the target program counter records the address of the target instruction to jump to after synchronization. The mapped return program counter and thread synchronization information are used to perform thread bundle differentiation and synchronization, thereby achieving flexible reuse of general-purpose registers.

[0094] For example, further, by responding to branch instructions, the return program counter, thread mask, and target program counter of the threads in the thread bundle are recorded; in response to thread switching instructions, the first address of the thread to be switched to is selected as the return program counter to continue execution; in response to thread synchronization instructions, the thread mask is used to determine whether there are still unfinished threads participating in the synchronization, and the second address of the unfinished threads is selected as the return program counter to continue execution; and in response to determining that all the threads participating in the synchronization have been completed according to the thread mask, execution continues according to the target program counter, thereby flexibly utilizing software instructions to implement independent thread scheduling.

[0095] For example, further, free vector / scalar register regions in the processor that meet the vector / scalar register space size required by the thread bundle are searched according to the vector / scalar register space size; if a free vector / scalar register region exists, the program counter is returned and mapped to the free vector / scalar register region, and the first / second register is released; if no free vector / scalar register region exists, the program counter is returned and mapped to the first / second register, so as to make full use of the free register resources.

[0096] For example, further, by selecting the address of the highest priority thread from the threads that have not yet finished executing as the return program counter to continue execution, and by taking into account the actual thread requirements, the thread to which to switch and the thread to execute before reaching the synchronization point can be flexibly determined in order to optimize the execution performance of the threads.

[0097] It should be noted that the thread scheduling method and apparatus provided in at least one embodiment of this disclosure may have different physical meanings depending on the application scenario. For example, the method or apparatus provided in at least one embodiment of this disclosure can be applied in fields such as speech processing, image processing, text processing, and video processing.

[0098] For example, in the field of speech processing, thread scheduling can refer to thread scheduling in tasks such as feature extraction, speech enhancement, and speech recognition.

[0099] For example, in the field of image processing, thread scheduling can refer to thread scheduling in tasks such as image recognition, feature extraction, image segmentation, object detection, image classification, and scene reconstruction.

[0100] For example, in the field of text processing, thread scheduling can refer to thread scheduling in tasks such as text classification, sentiment analysis, and text generation.

[0101] For example, in the field of video processing, thread scheduling can be the scheduling of computing threads as in the field of image processing, or thread scheduling specific to the field of video processing, such as optical flow operators (used to estimate motion between video frames) and target tracking operators (used to track specific targets in video).

[0102] Of course, this disclosure is not limited to this. For other application scenarios or fields, as long as the scheduling of computing threads is required, the method of at least one embodiment of this disclosure can be applied, which will not be described in detail here.

[0103] Note that the method and apparatus provided in at least one embodiment of this disclosure can be applied to different systems or devices, and can be as follows: Figure 14 The electronic equipment 1400 shown can be, for example, a terminal such as a mobile phone, tablet computer, laptop computer, AR device, VR device, vehicle terminal, etc., or a server. The method provided in at least one embodiment of this disclosure can be applied to scenarios involving thread scheduling in electronic equipment, such as those related to High Performance Computing (HPC) and Artificial Intelligence (AI). Of course, this disclosure is not limited to this; any scenario, device, or apparatus involving thread scheduling can employ the method or apparatus provided in at least one embodiment of this disclosure.

[0104] To perform independent thread scheduling, it is necessary to separately record the execution status of each branch instruction, the next instruction after that branch instruction, and the next instruction to be executed synchronously after all branch instructions are completed, etc. First, let's define the software aspects:

[0105] 1. Define an operand rPC (return program counter) to represent the address of the next instruction to be executed (including branch instructions).

[0106] 2. Define an operator `setrPC` (or the first software instruction) to establish a mapping between the rPC and general-purpose vector registers (each of the 32 threads is independent, 32 threads * 32 bits = 1024 bits), such as `setrPC UR0, R0`. After executing this instruction, each time the rPC is indexed, the hardware will automatically index the general-purpose vector register R0 to retrieve the address of the next instruction to be executed. This allows the use of general-purpose vector registers instead of relying on the dedicated rPC register, making register utilization more flexible.

[0107] 3. Define a register type cbr (convergence barrier register) to express BarrierParticipationMask (indicating which threads participate in this synchronization, i.e., thread mask, for example, 1 indicates that the thread participates in this synchronization) and the target PC (target program counter, targetPC, i.e. target instruction address) to jump to after synchronization is completed.

[0108] 4. Define an operator setCBR (or a second software instruction) to establish a mapping between the CBR and a general-purpose scalar register (shared by 32 threads, 32 bits): setCBR UR2, cbr0, {UR0, UR1}; After executing this instruction, each time cbr0 is indexed, the hardware will automatically index to the general-purpose scalar registers UR0 and UR1 to retrieve the thread mask to understand which threads are participating in this synchronization, and to retrieve the target program counter to understand the target PC to jump to after synchronization is completed.

[0109] 5. Define an operator `mx.JOIN` (or thread synchronization instruction) to express the semantics of reaching the synchronization point. If all threads reach the synchronization point, jump to `targetPC`; if not all threads reach the synchronization point, select one thread PC from `rPC` that has not reached the synchronization point. For example, `mx.JOIN UR7, CBR0`.

[0110] 6. Define an operator `mx.YIELD` (or thread switching instruction) to express the semantics of switching threads even when synchronization is incomplete, for example, `mx.YIELD UR7, CBR0`. Its implementation involves updating the next PC and nextPC corresponding to the currently active thread to the corresponding positions of the rPC, so that the address of the next instruction is known when switching back. A PC is selected from the group where cbr bit = 1 to return to (i.e., which instruction to switch to, such as branch instruction X). The selection method will be described in detail later.

[0111] Once the operands and operators are defined, the compiler can insert the defined operators into the conditional branch instructions executed by the thread bundle in order to complete independent thread scheduling.

[0112] Figure 4 A flowchart of a compilation method 400 according to at least one embodiment of the present disclosure is shown.

[0113] Compilation method 400 includes steps 401 and 402.

[0114] In step 401, a first software instruction is inserted before the branch instruction in the source code. This first software instruction maps a first register in the processor to a register that records the return program counter for the thread bundle to be split and synchronized. The return program counter is used to record the address of the next instruction to be executed in each thread of the thread bundle.

[0115] In step 402, a second software instruction is inserted before the branch instruction to map the second register in the processor to a register that records thread synchronization information for the thread bundle. The thread synchronization information includes a thread mask and a target program counter. The thread mask is used to record whether the thread currently participating in the thread bundle differentiation and synchronization has been executed. The target program counter is used to record the target instruction address processor to jump to after synchronization is completed.

[0116] In some embodiments, the compilation method 400 further includes: at branch instructions, inserting a save context instruction to record the return program counter, thread mask, and target program counter of the threads in the thread bundle; at instructions requiring thread switching, inserting a thread switching instruction to select the first address of the thread to be switched as the return program counter to continue execution; at instructions requiring synchronization, inserting a thread synchronization instruction to determine, based on the thread mask, whether there are still unfinished threads participating in the synchronization, selecting the second address of the unfinished threads as the return program counter to continue execution, and continuing execution based on the target program counter in response to determining, based on the thread mask, that all currently participating threads in the synchronization have been completed.

[0117] Figure 5 A schematic diagram is shown of a compiler according to at least one embodiment of the present disclosure inserting operators into conditional branch instructions executed by a thread bundle.

[0118] Prior to this, the compiler inserts the setrPC operator (or the first software instruction) before the program starts to establish the mapping between rPC and general-purpose vector registers, and inserts the setCBR operator (or the second software instruction) to establish the mapping between cbr and general-purpose scalar registers.

[0119] like Figure 5 As shown, when a thread bundle encounters a conditional branch, it needs to perform thread bundle differentiation and independent thread scheduling. Before the conditional branch, the compiler inserts record next_pc, mask, targetPC at #0, which saves the return program counter, thread mask, and target program counter as the context.

[0120] If the condition is true at #1, then instructions A and B are executed at #2 and #3 respectively. If the condition is false at #4, then instructions X and Y are executed at #5 and #6 respectively.

[0121] As previously mentioned, the thread stalls during the execution of instruction A at #2, requiring a thread switch. Therefore, a thread switch instruction is inserted here, namely the `mx.YIELD` operator at #2.5 (only `yield` is shown in the diagram), which allows execution to jump to other instructions and continue, followed by instruction B. After executing instruction B, a synchronization point should be reached so that subsequent instructions can continue synchronously after the thread bundle ends. Therefore, a synchronization point reaching instruction instruction is inserted after B, namely the `mx.JOIN` operator at #3.5 (only `join` is shown in the diagram).

[0122] Similarly, if the thread stalls while executing instruction X at #5, a thread switching instruction is inserted here, namely the `mx.YIELD` operator at #5.5 (only `yield` is shown in the diagram). This allows the thread to jump to other instructions to continue execution, and then the `Y` instruction continues. After executing instruction Y, a synchronization point should be reached so that subsequent instructions can continue to be executed synchronously after the thread bundle ends. Therefore, a synchronization point reaching instruction instruction is inserted after instruction Y, namely the `mx.JOIN` operator at #7 (only `join` is shown in the diagram).

[0123] Figure 6 At least one embodiment of the present disclosure is shown as follows Figure 5 The diagram shows the overall flowchart of program execution after the insertion operator.

[0124] exist Figure 6 In step 601, before the program starts, the compiler uses free general-purpose registers to map rPC (i.e., the return program counter) and cbr (including the thread mask and the target program counter). For example, the operator setrPC UR2, R0 is used to map the free vector register UR2 as the vector register R0 of rPC, and the operator setCBR UR3, cbr0{UR0, UR1} is used to map the free scalar register UR3 as the scalar registers UR0 and UR1 of cbr.

[0125] In step 602, the compiler checks the location of the branch in the source code, corresponding to Figure 5Before #1; save the thread mask and target program counter (targetPC) involved in the synchronization. Figure 6 The example shows operators used to store the thread mask and the target program counter (targetPC) as the context.

[0126] In step 603, the compiler executes... Figure 5 The instruction at PC#1 is the conditional branch.

[0127] In step 604, after #2 is executed, the mx.YIELD instruction is executed, which switches threads.

[0128] In step 605, it is determined whether mx == CBR0.mask, which means that the thread mask is used to determine whether there are any threads that have not yet finished executing among the threads currently participating in the synchronization.

[0129] If none of the threads currently participating in the synchronization have finished executing (i.e., mx == CBR0.mask), then in step 611, the return program counter of #2, i.e., PC#3, is selected to continue execution. In step 612, after PC#3 is executed, the synchronization point (JOIN) is reached. Then, it is checked whether all the threads participating in the synchronization have finished executing (i.e., whether the synchronization is completed). If not, then continue to step 606.

[0130] If it is determined that there are any threads that have not finished executing among the currently participating threads, in step 606, PC#1 is selected to continue execution. Then, based on the conditional branch at PC#1, the branch to enter PC#4 is determined. In step 607, the #5 X instruction is executed. Then, the thread switching instruction (mx.YIELD) is executed.

[0131] Then in step 608, it is determined whether mx == CBR0.mask, that is, whether there are any threads that have not yet finished executing among the current threads participating in synchronization.

[0132] If any of the threads currently participating in the synchronization have not yet finished executing, then in step 611, PC#3 is selected to continue execution.

[0133] If there are no unfinished threads currently participating in the synchronization, i.e., mx == CBR0.mask, then in step 609, the PC#6 Y instruction is selected to continue execution.

[0134] Then, in step 610, after executing the #6 Y instruction, the thread synchronization instruction (JOIN) is executed to check whether the synchronization is complete. If all threads have completed synchronization, then in step 613, the process jumps to #7 to continue executing the instruction indicated by the target program counter.

[0135] In other words, by defining software operands and operators to map general-purpose registers (instead of dedicated hardware), the compiler can execute independent thread scheduling, including thread switching and thread synchronization, according to the corresponding software instructions, while making full use of register resources and reducing the required hardware resources and area.

[0136] The above describes scenarios and process examples of application according to at least one embodiment of the present disclosure. Next, specific details and further improvements according to at least one embodiment of the present disclosure will be described.

[0137] Figure 7 A flowchart of a method for scheduling threads according to at least one embodiment of the present disclosure is shown. The method 700 for scheduling threads includes the following steps.

[0138] In step 701, for the thread bundle to be differentiated and synchronized, the first register in the processor is mapped to a register that records the return program counter rPC through the first software instruction. The return program counter rPC is used to record the address of the next instruction to be executed in each thread of the thread bundle.

[0139] In step 702, the second register in the processor is mapped to a register for recording thread synchronization information by the second software instruction for the thread bundle. The thread synchronization information includes a thread mask and a target program counter (targetPC). The thread mask is used to record whether the thread currently participating in the thread bundle differentiation and synchronization has been executed. The target program counter is used to record the target instruction address to jump to after synchronization is completed.

[0140] In step 703, thread bundle differentiation and synchronization are performed using the returned program counter and thread synchronization information obtained after mapping.

[0141] Thus, by using software instructions to utilize general-purpose vector / scalar registers as dedicated registers for recording the return program counter, and dedicated registers for recording the thread mask and the target program counter, respectively, it is not limited to the dedicated register locations specified by the hardware, and register resources can be flexibly reused to achieve thread bundle differentiation and synchronization.

[0142] Figure 8A A schematic diagram of registers within an execution unit (EU) in a processor according to at least one embodiment of the present disclosure is shown.

[0143] like Figure 8AAs shown, for a code segment, the execution unit in the processor performs instruction decoding and operand fetching. The execution unit sends an operand fetch request to the registers to obtain the register file. The registers return the operands to the vector computation unit, instruction cache predictor, and data cache predictor within the execution unit. The vector computation unit performs calculations on the operands and writes the results back to the registers. When needed, the vector computation unit and registers access data via shared memory / cache. The instruction cache predictor and data cache predictor will be discussed later.

[0144] For different warp threads, software instructions are used to establish dedicated registers in the processor for that specific warp thread (including vector registers and scalar registers used to store the return program counter (rPC) and the thread mask and target program counter (targetPC), respectively). For example, for warp0, software instructions establish a mapping between the processor's free registers, warp0 dedicated registers, and the return program counter (rPC); for warp1, the same mapping applies; for warp2, the same mapping applies; and for warp3, the same mapping applies.

[0145] In this way, by using software instructions to specify special-purpose registers, it is not limited to the location of special-purpose registers specified by the hardware, and register resources can be reused flexibly.

[0146] However, in some cases, after specifying dedicated registers for different thread bundles using software instructions, the different register resource requirements of these thread bundles may lead to certain register regions on the physical register set remaining idle. Therefore, the resources of the dedicated registers specified by the software instructions may be wasted. Thus, a scheme is designed to further utilize register resources more flexibly.

[0147] For example, previously, the software instruction `setrPC UR0` specified R0 as the dedicated vector register for the thread warp0 to store the return program counter. If the storage space required by the thread warp0 might be smaller than the storage space occupied by the dedicated vector register R0 specified by the software instruction `setrPC UR0`, then other free vector register regions of suitable size can be searched for and used, thus freeing up the larger vector register R0.

[0148] For example, the method 700 described above may further include retrieving a free vector register region in the processor that meets the vector register space size requirement based on the vector register space size of the thread bundle (e.g., ...). Figure 7 If a free vector register region (e.g., Rx) exists, the program counter is mapped to the free vector register region (Rx), and the first register (R0) is released; if no free vector register region exists, the program counter is mapped to the first register (R0).

[0149] Here, the required vector register space size for a thread bundle can be the storage space required to store the return program counter that the thread bundle needs to store.

[0150] Subsequently, each time the operand of the thread-beam instruction is indexed back to the program counter rPC, the hardware will automatically index and determine whether to access register Rx or R0 to read the corresponding data according to the aforementioned process.

[0151] For example, by assigning a value of 0 to a result register UR0, it can be indicated that there is a free vector register region (e.g., Rx), and the first register (R0) specified by the software instruction is not used by the thread bundle warp0, so it can be used as a general-purpose register by other thread bundles.

[0152] In some embodiments, the storage space of the free vector register region is smaller than the storage space of the first register.

[0153] In this way, after the software instruction specifies the dedicated vector register, the appropriate size of the free vector register can still be used according to the vector register space required by the thread bundle, while the larger space of the dedicated vector register specified by the software instruction can be released for use by other thread bundles, so as not to waste the resources of the dedicated vector register.

[0154] For example, the software instruction `setCBR UR2, cbr0, {UR0, UR1}` previously specified the dedicated scalar registers UR0 and UR1 for storing the thread mask and target program counter. If the storage space required by thread warp0 is smaller than the storage space occupied by the dedicated scalar registers UR0 and UR1 specified by the software instruction `setCBR UR2, cbr0, {UR0, UR1}`, then other suitable free scalar register regions can be searched for and used, freeing up the larger scalar registers UR0 and UR1.

[0155] For example, the method 700 described above may further include retrieving two free scalar register regions in the processor that meet the scalar register space size requirement based on the scalar register space size required by the thread bundle; if two free scalar register regions exist, the thread mask and the target program counter are mapped to the two free scalar register regions respectively, and the second register is released; if two free scalar register regions do not exist, the thread mask and the target program counter are mapped to the second register respectively.

[0156] For example, by assigning a value of 0 to a result register UR2, it can be indicated that there is a free scalar register region (e.g., URx, URy), and the second register (UR0, UR1) specified by the software instruction is not used by the thread bundle warp0, so it can be used as a general-purpose register by other thread bundles.

[0157] In some embodiments, the total storage space of the two free scalar register regions is less than the storage space of the second register.

[0158] Subsequently, each time the thread bundle instruction operand is indexed to cbr0 (including the thread mask and the target program counter), the hardware will automatically index it and determine whether to access registers URx, URy or UR0, UR1 to read the corresponding data according to the aforementioned process.

[0159] In this way, after the software instruction specifies the dedicated scalar register, the appropriate size of the free scalar register can still be used flexibly according to the scalar register space required by the thread bundle, while the larger space of the dedicated scalar register specified by the software instruction can be released for use by other thread bundles, so as not to waste the resources of the dedicated register.

[0160] In some embodiments, step 703 may include, in response to a branch instruction, recording the return program counter, thread mask, and target program counter of the threads in the thread bundle; in response to a thread switching instruction, selecting the first address of the thread to be switched as the return program counter to continue execution; in response to a thread synchronization instruction, determining, based on the thread mask, that there are still unfinished threads among the currently participating threads in the synchronization process, selecting the second address among the unfinished threads as the return program counter to continue execution, and in response to determining, based on the thread mask, that all currently participating threads in the synchronization process have been completed, continuing execution based on the target program counter.

[0161] In the step of recording the return program counter, thread mask, and target program counter of the thread in the thread bundle in response to a branch instruction, that is, before the branch instruction is executed, since the thread mask and target program counter have been mapped to registers URx, URy or UR0, UR1 by means of software instructions or by retrieving free registers, the return program counter, thread mask, and target program counter of the thread in the thread bundle can be recorded in the mapped registers URx, URy or UR0, UR1.

[0162] In the step of responding to a thread switching instruction and selecting the first address of the thread to be switched to as the return program counter to continue execution, it is usually possible to pre-select the address of the thread on the other conditional branch of the branch instruction to continue execution. However, this method is not flexible enough. For example, there may be instructions with higher priority addresses that need to be executed. For example, there may be instructions with lower address values ​​in the unfinished thread, indicating that they are earlier in the instruction sequence and should be executed earlier. Or, for example, there may be addresses where the next instruction address is repeated multiple times in the unfinished thread. In other words, many threads need to jump to the same instruction address to continue execution, so the instruction at that address should be executed earlier. If multiple instruction addresses are repeated the same number of times, the instruction with the lower address value should be selected for execution, and so on.

[0163] Therefore, in some embodiments, in response to a thread switching instruction, the first address of the thread to be switched to is selected as the return program counter to continue execution, including: selecting the address of the highest priority thread from the threads that have not yet finished executing as the return program counter to continue execution.

[0164] In some embodiments, selecting the address of the highest priority thread from the threads that have not yet finished executing as the return program counter to continue execution includes: counting the number of repetitions of each address in the return program counter of the switchable threads; selecting the address with the most repetitions to continue execution in response to the existence of an address with the most repetitions; and selecting the address with the smallest value among the multiple addresses with the most repetitions to continue execution in response to the existence of multiple addresses with the most repetitions.

[0165] For example, taking the thread switching instruction mx.YIELD UR7, CBR0 as an example, its implementation is as follows:

[0166] Update the next PC (i.e. the address of the instruction following the YIELD instruction) corresponding to the current active thread (e.g., the current active thread shown in the current active thread mask mx, where a mask of 1 indicates inactivity and a mask of 0 indicates activity) to the corresponding position of the return program counter rPC in the aforementioned mapped vector register according to mx, and preserve its current context.

[0167] Check the thread mask CBR0.Mask[] in the scalar register of the aforementioned mapping (a mask of 0 indicates that the merge synchronization point has been reached, and a mask of 1 indicates that the merge synchronization point has not yet been reached).

[0168] If multiple threads in CBR0.Mask[] (with corresponding bit positions set to 1) have not completed this synchronization, i.e. have not finished executing, assuming it is the unfinished thread mask mask_unjoin (mask_unjoin includes the mask of the currently active thread, i.e., the position that is 1 in the mask of the currently active thread must be 1 in mask_unjoin), then it is necessary to select the most suitable thread to continue execution in order to optimize program performance.

[0169] For example, specifically, according to mask_unjoin & (~mx) (i.e., the NOT AND operation between the mask of the thread that has not completed execution and the mask of the currently active thread), corresponding to "other threads that have not completed this synchronization and are not active threads of this YIELD instruction" (because this YIELD instruction is also a thread that has not completed execution, but should be excluded from switching to this YIELD instruction), search for the PCs corresponding to the return program counter rpc[] positions (i.e., the bit position corresponding to mask_unjoin & (~mx) is 1) of those threads that have not completed execution and are excluded from the currently active threads, and select the PC with the highest repetition frequency, that is, the address of the thread with the highest priority, to continue execution.

[0170] For example, if the content of mask_unjoin is 1111 and the content of mx is 10, it means that threads 0-3 have not completed synchronization. The currently active thread 1 is executing the YIELD instruction, mask_unjoin & (~mx), which is 1011. This means that this step needs to select the corresponding PC from threads 0, 2, and 3 to execute.

[0171] Assume that the return program counter (rpc) contents of each thread 0, 1, 2, and 3 are as follows: rpc[0] = 0x0; rpc[1] = 0x1; rpc[2] = 0x2; rpc[3] = 0x2.

[0172] According to at least one embodiment of this disclosure, the address with the most repetitions, i.e. the most common value 0x2, is selected and returned to the result register UR7 of the YIELD instruction as a return program counter to continue execution.

[0173] If there are multiple PC values ​​with the same number of repetitions, for example:

[0174] rpc[0] = 0x0; rpc[1] = 0x0;

[0175] rpc[2] = 0x2; rpc[3] = 0x2;

[0176] Since both 0x0 and 0x2 are repeated twice, the address with the smallest PC value is selected and returned according to at least one embodiment of this disclosure, i.e., 0x0 is returned. This is because instructions with smaller PC values ​​are generally considered to be located earlier in the code segment and should be executed earlier. Therefore, for load balancing among different threads, instructions with smaller PC values ​​are preferentially selected for continued execution according to at least one embodiment of this disclosure.

[0177] Then, 0x0 is returned to the result register UR7 of the JOIN instruction, which is used as the return program counter to continue execution.

[0178] On the other hand, if mask_unjoin & (~mx) = 0, which means "only the active thread of this YIELD instruction has not completed the synchronization of this CBR0", it means that the thread switching failed and the instruction at PC+1 (i.e. the position of the next instruction) will continue to be executed based on the YIELD instruction.

[0179] For example, if the content of mask_unjoin is 1111 and the content of mx is 1111, then the PC+1 of the current YIELD instruction (i.e., the position of the next instruction) is returned to the result register UR7 of the YIELD instruction, and used as the return program counter to continue execution.

[0180] In this way, it is possible to flexibly choose to switch to a more suitable thread to continue execution.

[0181] Next, in response to the thread synchronization instruction, the thread mask is used to determine whether there are any threads that have not yet finished executing among the threads currently participating in the synchronization. Among the threads that have not yet finished executing, the second address is selected as the return program counter to continue execution. In response to the determination that all the threads currently participating in the synchronization have finished executing based on the thread mask, execution continues based on the target program counter.

[0182] For example, consider the instruction instance of the thread synchronization instruction mx.JOIN UR7, CBR0:

[0183] Update the thread mask CBR0.Mask based on the current active mask (mx) to indicate which corresponding threads have reached the merge synchronization point. A mask of 0 indicates that the merge synchronization point has been reached, and a mask of 1 indicates that the merge synchronization point has not yet been reached.

[0184] Check if the thread mask CBR0.Mask is all 0. If it is, it means that all the threads currently participating in the synchronization have been executed and all threads have reached the merge synchronization point. The next synchronization execution instruction of the SIMT thread bundle should be executed. Then return the target program counter and targetPC to the result register UR7 of the JOIN instruction so as to jump to the instruction address of the next synchronization execution instruction.

[0185] Check if the thread mask CBR0.Mask is all 0. If not, it means that there are still threads that have not finished executing among the threads currently participating in the synchronization. Then select a PC address n (whose CBR0.Mask[n] is equal to 1) from the group where the bit = 1 in CBR0.Mask[]. Return rpc[n], that is, the PC address of the nth thread, to the result register UR7 of the JOIN instruction so that the instruction address of the next branch instruction indicated by the PC address can be jumped to execute the branch instruction.

[0186] The choice here can also consider prioritizing instructions with higher priority addresses. For example, if an instruction with a lower address value is in an unfinished thread, it means that it is earlier in the instruction sequence and should be executed earlier. Or, if the address of the next instruction in an unfinished thread is repeated multiple times, that is, many threads need to jump to the same instruction address to continue execution, then the instruction at that address should be executed earlier. If multiple instruction addresses are repeated the same number of times, then the instruction with the lower address value should be selected for execution, and so on.

[0187] Therefore, in some embodiments, selecting a second address as the return program counter from the threads that have not yet finished executing to continue execution includes: selecting the address of the highest priority thread from the threads that have not yet finished executing as the second address, and using the second address as the return program counter to continue execution.

[0188] In some embodiments, selecting the address of the highest priority thread among the threads that have not yet finished executing as the second address, and using the second address as the return program counter to continue execution, includes: counting the number of repetitions of each address in the return program counter of the threads that have not yet finished executing; in response to the existence of an address with the most repetitions, selecting the address with the most repetitions to continue execution; in response to the existence of multiple addresses with the most repetitions, selecting the address with the smallest value among the multiple addresses with the most repetitions to continue execution.

[0189] For example, by searching for the position corresponding to rpc[] (i.e., the bit position corresponding to mask_unjoin has a value of 1), the PC with the highest repetition frequency is selected.

[0190] For example, if the content of mask_unjoin is 1111, it means that threads 0-3 have not completed synchronization and need to select the corresponding pc from threads 0-3 to execute.

[0191] Assuming the contents of rpc are: rpc[0] = 0x0; rpc[1] = 0x1; rpc[2] = 0x2; rpc[3] = 0x2; then select the most frequently repeated and most common value 0x2 and return it to the result register UR7 of the JOIN instruction as the return program counter to continue execution.

[0192] If multiple PC values ​​are the most common, such as rpc[0] = 0x0; rpc[1] = 0x0; rpc[2] = 0x2; rpc[3] = 0x2, then the smaller PC value is selected and returned. This is because instructions with smaller PC values ​​are generally considered to be earlier in the code segment and should be executed earlier. To balance the load of different threads, instructions with smaller PC values ​​are selected first. Then, 0x0 is returned to the result register UR7 of the JOIN instruction as the return program counter to continue execution.

[0193] In this way, a more suitable thread can be flexibly selected to continue execution.

[0194] In addition to the above method of selecting the address of the highest priority thread as the return program counter to continue execution, at least one embodiment of this disclosure also provides a method for calculating the priority of the addresses of each thread.

[0195] In some embodiments, method 700 may further include: calculating priorities by the following steps: grouping the addresses of threads that have not yet finished executing into candidate thread groups, wherein multiple threads with duplicate addresses are grouped into one group, and different threads with different addresses are each grouped into different groups; performing multi-dimensional scoring on the candidate thread groups according to a multi-dimensional scoring formula based on a weighting factor and in conjunction with feedback information from the runtime hardware predictor; and using the score as a priority.

[0196] Specifically, for example, according to the mask_unjoin of the thread mask, search for the return program counter (i.e. the instruction fetch address of the thread that has not completed execution) at the corresponding position of rpc[] of the thread that has not completed execution (i.e. the bit position corresponding to mask_unjoin is 1), and group the valid program counter PC (i.e. the instruction fetch address).

[0197] For example, the content of mask_unjoin is 1111; that is, threads 0-3 have not completed synchronization and need to select the corresponding PC from threads 0-3 to continue execution. Assume that the RPC content at this time is: rpc[0] = 0x0; rpc[1] = 0x1; rpc[2] = 0x2; rpc[3] = 0x2 (note that thread 0 uses rpc[0], ... thread n uses rpc[n]). That is, there are different addresses 0x0, 0x1, and 0x2 (repeated twice). Then, the two threads (thread 2 and thread 3) with the repeated address 0x2 are grouped together, and the threads 0 and 1 with different addresses 0x0 and 0x1 are each grouped into different groups.

[0198] The grouping result is as follows:

[0199] Group 1: {Thread 0} / / rpc=0x0

[0200] Group 2: {Thread 1} / / rpc=0x1

[0201] Group 3: {Thread 2, Thread 3} / / rpc=0x2.

[0202] Next, based on the weighting factors and the feedback information from the runtime hardware predictor, the candidate thread group is scored in multiple dimensions according to the multi-dimensional scoring formula; the score is then used as the priority.

[0203] In some embodiments, the hardware predictor includes an instruction cache predictor and a data cache predictor. The instruction cache predictor maintains a hit prediction table and obtains an instruction cache hit score for a subset of address bits by indexing a portion of the address bits in the hit prediction table based on a portion of the address bits of the thread's program counter. This score represents the probability that a subset of address bits will be predicted to hit in the instruction cache. The data cache predictor calculates a data locality score between the candidate thread group and the currently executing thread group by recording an address record table that records a portion of the address bits of the thread's most recent memory operation. This score represents the distance between the representative address of the candidate thread group and the representative address of the currently executing thread group.

[0204] The instruction cache predictor mainly considers the prediction hit rate of the instruction to be fetched in the cache, while the data cache predictor mainly considers the location of the data to be accessed in the cache. It aims to select the thread with the instruction with the high prediction hit rate and the data that is closer in the cache to continue execution. This can reduce the time spent searching and reading instructions and / or data in the cache, improve data throughput, and speed up program execution.

[0205] In some embodiments, the formula for multi-dimensional scoring is:

[0206] Score = w_i * ICache_Score + w_d * Dcache_Score + w_g * Group_Size_Score.

[0207] Among them, w_i, w_d, and w_g are weighting factors, which can be configured by the software through instructions or memory-mapped I / O.

[0208] w_i stands for I-Cache Weight. The higher the value, the more likely it is to select the PC group with a high instruction cache hit rate (i.e., it is easier to hit in the cache, so fetching instructions from that address in the cache will be faster than from shared memory).

[0209] w_d stands for D-Cache Weight. The higher the value, the more likely it is to select PC groups with high data affinity (i.e., closer to the address of the current thread, such as the address in the L1 cache) to improve the data cache hit rate of the L1 cache.

[0210] w_g stands for Group Size Weight. The higher the value, the more likely it is to select PC groups containing more threads in order to maximize the utilization of vector computing units.

[0211] ICache_Score, DCache_Score, and Group_Size_Score are dynamically generated by the hardware based on runtime information. Specifically, ICache_Score is the instruction cache hit rate score output by the instruction cache predictor. Dcache_Score is the data locality score output by the data cache predictor. Group_Size_Score is the thread count score for a candidate thread group; the more threads, the higher the thread count score. For example, setting Group_Size_Score to the number of threads in a PC group, using the example above, the scores for each PC group are as follows:

[0212] Group 1: {Thread 0} / / Group_Size_Score = 1

[0213] Group 2: {Thread 1} / / Group_Size_Score = 1

[0214] Group 3: {Thread 2, Thread 3} / / Group_Size_Score = 2.

[0215] ICache_Score is calculated by the instruction cache predictor (Icache predictor) in the execution unit.

[0216] The hit prediction table maintained by the instruction cache predictor can include four instruction cache hit rate states: first prediction miss, second prediction miss, third prediction hit, and fourth prediction hit. The probability of a prediction miss in the first prediction miss state is higher than that in the second prediction miss state, and the probability of a prediction hit in the third prediction hit state is lower than that in the fourth prediction hit state. The higher the probability of a prediction hit for a certain address bit in the instruction cache, the higher the instruction cache hit rate score.

[0217] Method 700 may further include: each time an instruction is fetched from the highest priority address to continue execution, updating the instruction cache hit rate status of the hit prediction table according to the actual hit or miss result of the highest priority address in the instruction cache.

[0218] Specifically, for example, an instruction cache predictor maintains an instruction cache prediction hit table with N entries (N is a positive integer). Each entry is indexed by an address, and each entry contains a 2-bit hit prediction table. The lower-order bits of the PC address can be used for indexing to reduce the amount of data in the index while maintaining index accuracy. The table size N is configurable, for example, 64 or 128 entries.

[0219] The reason for using the lower-order bits of the PC address for indexing is as follows:

[0220] If we understand the instruction cache line size as one instruction: for example, PC 0x0 and PC 0x1 belong to different instruction cache lines and should have their status recorded in different "Instruction Cache Prediction Hit Table" entries. However, if we use high-order indexing, the high-order 6 or 7 bits of 0x0 and 0x1 are indistinguishable and mapped to entry 0 in the table, resulting in the same prediction result. Therefore, low-order indexing is better at distinguishing the position of index entries, while omitting high-order indexing does not affect the retrieval results and also reduces the amount of index data and the storage space of the index table.

[0221] In practice, the specific number of bits for the instruction cache prediction hit table index can be designed based on the size of the instruction cache line, with the aim of assigning different prediction hit table entries to different cache lines.

[0222] Assume the prediction hit table has 64 entries, N=64. If the cache line size of the instruction cache is one instruction, then use PC& 0x3f to take the lower 6 bits; if the cache line size of the instruction cache is two instructions, then use (PC>>1) &0x3f, right-shift by 1 bit, and take the lower 6 bits.

[0223] The lower bits can record different states for different cache lines, while the higher bits may record the states of multiple cache lines together.

[0224] Figure 8B An example diagram of an instruction cache prediction hit table according to at least one embodiment of the present disclosure is shown.

[0225] If N=64, the lower 6 bits of the PC address are used as the index to access the instruction cache prediction hit table.

[0226] If N=128, the lower 7 bits of the PC address are used as the index to access the instruction cache prediction hit table.

[0227] The number of entries in the instruction cache prediction hit table is assumed to be N, where N is a positive integer. The contents of the instruction cache prediction hit table are 2-bit saturation counts.

[0228] The 2-bit saturation count can be used to represent the state as: 00 (strong prediction miss), 01 (weak prediction miss), 10 (weak prediction hit), 11 (strong prediction hit).

[0229] A strong prediction miss (or first prediction miss state) means that the address of the above index has a relatively high probability of being predicted as a miss in the instruction cache. A weak prediction miss (or second prediction miss state) means that the address of the above index has a relatively low probability of being predicted as a miss in the instruction cache. In other words, the probability of a strong prediction miss is higher than the probability of a weak prediction miss.

[0230] A weak prediction hit (or third prediction miss) means that the address of the above index has a low probability of being predicted to hit in the instruction cache. A strong prediction hit (or fourth prediction miss) means that the address of the above index has a high probability of being predicted to hit in the instruction cache. In other words, the probability of a weak prediction hit is lower than the probability of a strong prediction hit.

[0231] The higher the probability of a predicted hit in the instruction cache at the indexed address, the higher the instruction cache hit score. Based on the predicted hit rate ranking, the order should be: Strong Prediction Hit > Weak Prediction Hit > Weak Prediction Miss > Strong Prediction Miss.

[0232] Whenever an instruction is fetched from a certain PC address, the corresponding counter is updated based on the actual hit / miss result of that address in the instruction cache. For example, if there is a hit, the counter is incremented by 1 (maximum not exceeding 11), and if there is a miss, the counter is decremented by 1 (minimum not less than 00).

[0233] For example, the 2-bit count value above can be directly mapped to a score (0-3), with a higher value representing a higher prediction hit rate. That is, ICache_Score(PC) = I_Cache_Prediction_Table[PC].count_value. Of course, the score is not limited to 0-3; other scores can also be involved to reflect the higher the probability of the index address bit being predicted to hit in the instruction cache, the higher the instruction cache hit rate score.

[0234] The storage cost for the instruction cache hit rate score is 64 * 2 = 128 bits.

[0235] DCache_Score is calculated by the data cache predictor in the execution unit.

[0236] The data cache predictor calculates the data locality score through the following steps: maintaining an address record table that records a portion of the address of the most recent memory operation of a thread; selecting a representative address of a thread from the address record table of a portion of the address of the most recent memory operation of a thread; calculating the distance between the representative address of the candidate thread group and the representative address of the currently executing thread group; and calculating the data locality score based on the distance, where the closer the distance, the higher the data locality score.

[0237] The data locality score indicates whether the data that the predicted thread is about to access is localized to the data currently in the cache (or to the data accessed by the currently executing thread), i.e., whether they are nearby.

[0238] The implementation involves maintaining an address record table, such as REMA_Table (recently memory access, REMA), in the data cache predictor. This table records a portion of the address of the most recent memory operation performed by each thread. To save costs, only the lower bits of the address can be recorded; for example, the 8 bits of the address of the most recent memory operation can be recorded. Figure 8C An example diagram of an address record table according to at least one embodiment of the present disclosure is shown. The address record table includes 32 entries, and the content of each entry is 8 bits of the address of the most recently executed memory operation.

[0239] In order to update the REMA_Table, each time a load / store instruction for data is executed, the data cache predictor extracts the lower 8 bits of the address corresponding to the active thread and writes them into the address record table REMA_Table[active thread].

[0240] Calculate DCache_Score: Obtain the representative address Group_REMA for this PC group. The representative address can be the REMA_Table[thread_id] value of any thread in the PC group (e.g., the one with the smallest identifier ID). Calculate data affinity: Compare the selected Group_REMA with the representative address of the currently executing thread group (e.g., the Curr_REMA of the thread group currently running the JOIN instruction). Of course, there are many other ways to obtain the representative address, as long as it represents the approximate location of the addresses of this group of threads and can be distinguished from the approximate locations of the addresses of other thread groups; these will not be described in detail here.

[0241] Distance = |Group_REMA - Current_REMA |.

[0242] Data locality score DCache_Score = f(distance).

[0243] The function f can be a piecewise function, for example:

[0244] Distance < 64: DCache_Score = 3.

[0245] 64 <= Distance < 128: DCache_Score = 2.

[0246] Distance >= 128: DCache_Score = 1.

[0247] In other words, the closer the distance, the easier it is to hit that nearby address, and the higher the score.

[0248] In summary, based on the software-configured w_i, w_d, and w_g, and the hardware-generated ICache_Score, Dcache_Score, and Group_Size_Score, a weighted score is assigned to each candidate PC group using the formula Score = w_i * ICache_Score + w_d * Dcache_Score + w_g * Group_Size_Score. The PC group with the highest score (as the highest priority) is selected as the return value and returned to the result register UR7 as the return program counter to continue execution.

[0249] Thus, through the built-in hardware predictors (instruction cache predictor and data cache predictor), the performance risks that a thread might face after switching (such as cache misses or data access misses) can be predicted in advance. The scheduler will actively avoid these high-risk operations and prioritize scheduling those threads that are "highly likely to run smoothly", thereby transforming long-latency operations that would otherwise block the pipeline into background waiting, significantly improving the effective instructions per cycle (IPC) of the computing unit.

[0250] In some embodiments, method 700 further includes: adjusting a weighting factor according to the application type of the thread, wherein for compute-intensive applications, w_d is greater than w_g and w_g is greater than w_i; for memory-intensive applications, w_d is greater than w_g and w_g is greater than w_i; and for control flow-intensive applications, w_i is greater than w_d and w_d is greater than w_g.

[0251] The above strategies enable high-performance support for various computing scenarios using the same hardware. For example: Scenario A: Computationally intensive applications (such as artificial intelligence (AI) inference), characterized by complex computations but regular data access patterns and high cache hit rates. Recommended weight: w_d > w_g > w_i. The reason is that maintaining high computational density is key, and data locality has the greatest impact on performance. Scenario B: Memory-intensive applications (such as sparse matrices), characterized by random data access and memory latency as the main bottleneck. Recommended weight: w_d > w_g > w_i. The reason is that data locality is given extreme priority, even at the cost of sacrificing some SIMT parallelism, to avoid long latency due to cache misses. Scenario C: Control-flow intensive applications (such as complex shaders). Characterized by large code size, many branches, and significant impact from I-Cache misses. Recommended weight: w_i > w_d > w_g. The reason is that maintaining a smooth instruction pipeline is the primary task, avoiding resource waste caused by instruction fetch pauses.

[0252] In this way, the above specific formula can be used to quantitatively select the more suitable thread for operation and environment to continue execution based on the operation data and environment data collected by the hardware (such as cache hit rate, address distance, scenario, etc.), to achieve adaptive trade-offs, optimize performance and energy efficiency, and the multi-dimensional weighted arbitration mechanism enables the GPGPU to "think for itself" based on application characteristics to select the more suitable thread to continue execution.

[0253] In summary, the hardware does not need to pre-allocate fixed storage space. Instead, it uses software to map general-purpose registers to record the necessary state information, saving hardware storage resources and significantly reducing storage resource consumption (e.g., from 1120 bits / thread bundle to 32 bits / thread bundle). Moreover, through hardware-software co-design, it offers high flexibility, allowing the insertion of appropriate operators (such as mx.JOIN and mx.YIELD instructions) according to compiler requirements, enabling threads to switch execution autonomously when stalled, thus improving the utilization of computing resources.

[0254] At least one embodiment of this disclosure is compatible with the traditional SIMT architecture, achieving independent thread scheduling with only a limited number of added opcodes. For example, in image super-resolution tasks, threads can dynamically switch to non-stagnant branches to process other pixels, reducing waiting time and improving overall computational efficiency by more than 10%. Through hardware and software co-design, the device achieves independent thread scheduling with only a few added operators while maintaining compatibility with the traditional SIMT architecture, reducing the hardware implementation area by approximately 65%. For example, in deep learning inference tasks, thread switching latency is reduced to 15% of traditional solutions, while access locality is improved by 15%.

[0255] Figure 9 A block diagram of an apparatus for scheduling threads according to at least one embodiment of the present disclosure is shown.

[0256] The scheduling device 900 provided in at least one embodiment of the present disclosure includes a first mapping module 901, a second mapping module 902, and an execution module 903.

[0257] The first mapping module 901 is configured to map a first register in the processor to a register that records the return program counter for a thread bundle to be split and synchronized, using a first software instruction. The return program counter is used to record the address of the next instruction to be executed in each thread of the thread bundle.

[0258] The second mapping module 902 is configured to map a second register in the processor to a register for recording thread synchronization information via a second software instruction. The thread synchronization information includes a thread mask and a target program counter. The thread mask is used to record whether the thread currently participating in thread bundle differentiation and synchronization has been executed. The target program counter is used to record the target instruction address processor to jump to after synchronization is completed.

[0259] Execution module 903 is configured to perform thread bundle differentiation and synchronization using the returned program counter and thread synchronization information obtained after mapping.

[0260] Figure 10A block diagram of an apparatus 1000 for another scheduling thread according to at least one embodiment of the present disclosure is shown.

[0261] The apparatus 1000 may include a processor 1010 and a memory 1020, the memory 1020 being coupled to the processor 1010 and storing computer instructions therein for performing steps of various methods of at least one embodiment of the present disclosure when executed by the processor 1010.

[0262] The processor 1010 may include, but is not limited to, one or more processors or microprocessors.

[0263] The memory 1020 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disk, floppy disk, solid-state drive, removable disk, CD-ROM, DVD-ROM, Blu-ray disc, etc.).

[0264] Figure 11 A block diagram of a compilation apparatus 1100 according to at least one embodiment of the present disclosure is shown.

[0265] A compilation device 1100 includes a first insertion module 1101 and a second insertion module 1102.

[0266] The first insertion module 1101 is configured to insert a first software instruction before the branch instruction of the source code, for mapping a first register in the processor to a register that records the return program counter for the thread bundle to be split and synchronized, wherein the return program counter is used to record the address of the next instruction to be executed in each thread of the thread bundle.

[0267] The second insertion module 1102 is configured to insert a second software instruction before the branch instruction, which is used to map a second register in the processor to a register that records thread synchronization information for the thread bundle. The thread synchronization information includes a thread mask and a target program counter. The thread mask is used to record whether the thread currently participating in the thread bundle differentiation and synchronization has been executed. The target program counter is used to record the target instruction address processor to jump to after synchronization is completed.

[0268] Figure 12 A block diagram of another compilation apparatus 1200 according to at least one embodiment of the present disclosure is shown.

[0269] The apparatus 1200 may include a processor 1210 and a memory 1220, the memory 1220 being coupled to the processor 1210 and storing computer instructions therein for performing steps of various methods of at least one embodiment of the present disclosure when executed by the processor 1210.

[0270] The processor 1210 may include, but is not limited to, one or more processors or microprocessors.

[0271] The memory 1220 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disk, floppy disk, solid-state drive, removable disk, CD-ROM, DVD-ROM, Blu-ray disc, etc.).

[0272] Figure 13 A block diagram of an electronic device 1300 according to at least one embodiment of the present disclosure is shown.

[0273] Electronic device 1300 includes any of the aforementioned devices 900 / 1000 / 1100 / 1200.

[0274] The electronic device of at least one embodiment may further include one or more first registers 1301 and one or more second registers 1302.

[0275] Figure 14 A schematic block diagram of an electronic device 1400 according to at least one embodiment of the present disclosure is shown.

[0276] like Figure 14 As shown, the electronic device 1400 is, for example, suitable for implementing the methods provided in the embodiments of this disclosure. It should be noted that... Figure 14 The components of the electronic equipment 1400 shown are merely exemplary and not limiting. The electronic equipment 1400 may also have other components depending on the actual application requirements.

[0277] like Figure 14 As shown, electronic equipment 1400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1401, which can perform various appropriate actions and processes according to non-transitory computer-readable instructions stored in memory to achieve various functions.

[0278] For example, when the computer-readable instructions are executed by the processing device 1401, one or more steps of the method according to any of the above embodiments can be performed. It should be noted that a detailed description of the method's processing procedure can be found in the relevant descriptions in the embodiments of the above methods.

[0279] For example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 1403 and / or cache memory, etc., for example, computer-readable instructions may be loaded from storage device 1408 into RAM 1403 to execute computer-readable instructions. Non-volatile memory may include, for example, read-only memory (ROM) 1402, hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. Various applications and various data, such as style images, and various data used and / or generated by the applications, may also be stored in the computer-readable storage medium.

[0280] For example, processing device 1401, ROM 1402, and RAM 1403 are interconnected via bus 1404. Input / output (I / O) interface 1405 is also connected to bus 1404.

[0281] Typically, the following devices can be connected to I / O interface 1405: input devices 1406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 1407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1408 including, for example, magnetic tapes, hard disks, flash memory, etc.; and communication devices 1409. Communication device 1409 allows electronic equipment 1400 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although... Figure 14 Electronic device 1400 with various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown, and electronic device 1400 may alternatively implement or possess more or fewer devices. For example, processing device 1401 can control other components in electronic device 1400 to perform desired functions. Processing device 1401 may be a device with data processing capabilities and / or program execution capabilities, such as a central processing unit (CPU), tensor processor (TPU), or graphics processing unit (GPU). The central processing unit (CPU) may be an x86, ARM, RISC-V architecture, etc. The GPU may be directly integrated into the SOC, directly integrated into the motherboard, or built into the northbridge chip of the motherboard.

[0282] In one embodiment, the at least one computer instruction may also be compiled into or comprise a computer program product or software product, wherein one or more computer instructions, when executed by a processor, perform the steps of the various functions or methods in the embodiments described herein.

[0283] Non-transitory computer-readable storage media are also disclosed according to at least one embodiment of the present disclosure.

[0284] Instructions, such as computer instructions, are stored on a non-transitory computer-readable storage medium. When the computer instructions are executed by a processor, the various methods described above can be performed. Non-transitory computer-readable storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disks, floppy disks, solid-state drives, removable disks, CD-ROMs, DVD-ROMs, Blu-ray discs, etc.). For example, a non-transitory computer-readable storage medium can be connected to a computing device such as a computer, and then, when the computing device executes the computer instructions stored on the computer-readable storage medium, the various methods described above can be performed.

[0285] According to at least one embodiment of this disclosure, a computer program product is also disclosed, which can perform the methods, steps, and operations given herein. For example, such a computer program product may be a computer software package, computer code instructions, or a computer-readable tangible medium having computer instructions tangibly stored (or encoded) thereon, which can be executed by a processor to perform the operations described herein. The computer program product may include packaging materials.

[0286] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The term “such as / for example” as used herein refers to the phrase “such as / for example but not limited to,” and is used interchangeably with it.

[0287] The flowcharts and method descriptions in this disclosure are merely illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the given order. As those skilled in the art will recognize, the steps in the above embodiments can be performed in any order. Words such as "then," "next," etc., are not intended to limit the order of the steps; these words are only used to guide the reader through the description of these methods. Furthermore, any reference to a singular element, such as the use of the articles "a," "one," or "the," is not to be construed as limiting that element to the singular.

[0288] Furthermore, the steps and apparatus in the various embodiments herein are not limited to any one embodiment. In fact, new embodiments can be conceived by combining relevant steps and apparatus in the various embodiments herein based on the concepts of this disclosure, and these new embodiments are also included within the scope of this disclosure.

[0289] The above methods can be implemented in hardware, software, firmware, or any combination thereof.

[0290] Furthermore, modules or other suitable means for carrying out the methods and techniques described herein can be downloaded from a server wirelessly when appropriate. Alternatively, the various methods described herein can be provided via a storage component so that the various methods are available when coupled to the storage component. Additionally, any other suitable techniques for providing the methods and techniques described herein to the device can be utilized.

[0291] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit at least one embodiment of the present disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for scheduling threads, characterized in that, The method includes: In order to perform thread bundle differentiation and synchronization, the first register in the processor is mapped to a register that records the return program counter through the first software instruction, wherein the return program counter is used to record the address of the next instruction to be executed in each thread of the thread bundle; The second register in the processor is mapped to a register for recording thread synchronization information by the second software instruction for the thread bundle. The thread synchronization information includes a thread mask and a target program counter. The thread mask is used to record whether the thread currently participating in the thread bundle differentiation and synchronization has been executed. The target program counter is used to record the target instruction address to jump to after the synchronization is completed. The thread bundle differentiation and synchronization are performed using the returned program counter obtained after mapping and the thread synchronization information. The step of using the returned program counter and the thread synchronization information obtained after mapping to perform thread bundle differentiation and synchronization includes: In response to a branch instruction, the return program counter, the thread mask, and the target program counter of the threads in the thread bundle are recorded; In response to a thread switching instruction, the first address of the thread to be switched to is selected as the return program counter to continue execution; In response to a thread synchronization instruction, based on the thread mask, it is determined that there are still threads that have not finished executing among the currently participating threads in the synchronization process. A second address is selected from the threads that have not finished executing as the return program counter to continue execution. In response to determining based on the thread mask that all currently participating threads in the synchronization process have finished executing, execution continues based on the target program counter.

2. The method according to claim 1, characterized in that, The method further includes: Based on the required vector register space size of the thread bundle, a free vector register region in the processor that meets the vector register space size is retrieved; If the free vector register region exists, the return program counter is mapped to the free vector register region, and the first register is released; If the free vector register region does not exist, the return program counter is mapped to the first register.

3. The method according to claim 2, characterized in that, The storage space of the free vector register region is smaller than the storage space of the first register.

4. The method according to claim 1, characterized in that, The method further includes: Based on the scalar register space size required by the thread bundle, two free scalar register regions in the processor that satisfy the scalar register space size are retrieved. If there are two free scalar register regions, then the thread mask and the target program counter are mapped to the two free scalar register regions respectively, and the second register is released; If the two free scalar register regions do not exist, the thread mask and the target program counter are mapped to the second register, respectively.

5. The method according to claim 4, characterized in that, The total storage space of the two free scalar register regions is less than the storage space of the second register.

6. The method according to claim 1, characterized in that, The step of responding to a thread switching instruction by selecting the first address of the thread to be switched to as the return program counter to continue execution includes: The address of the highest priority thread among the unfinished threads is selected as the return program counter to continue execution.

7. The method according to claim 6, characterized in that, The step of selecting the address of the highest-priority thread from the incomplete threads as the return program counter to continue execution includes: Count the number of times each address in the returned program counter is repeated for the threads that can be switched to; In response to the existence of an address with the highest number of repetitions, the address with the highest number of repetitions is selected to continue execution; In response to the existence of multiple addresses with the highest number of repetitions, the address with the smallest value among the multiple addresses with the highest number of repetitions is selected to continue execution.

8. The method according to claim 1, characterized in that, The step of selecting a second address from the incomplete thread as the return program counter to continue execution includes: The address of the highest priority thread among the unfinished threads is selected as the second address, and the second address is used as the return program counter to continue execution.

9. The method according to claim 8, characterized in that, The step of selecting the address of the highest-priority thread from the incomplete threads as the second address, and using the second address as the return program counter to continue execution, includes: Count the number of times each address in the program counter is returned for the threads that have not yet finished executing. In response to the existence of an address with the highest number of repetitions, the address with the highest number of repetitions is selected to continue execution; In response to the existence of multiple addresses with the highest number of repetitions, the address with the smallest value among the multiple addresses with the highest number of repetitions is selected to continue execution.

10. The method according to claim 6 or 8, characterized in that, The method further includes: The priority is calculated using the following steps: The addresses of the threads that have not finished executing are grouped into candidate thread groups, where multiple threads with duplicate addresses are grouped into one group, and different threads with different addresses are each grouped into different groups. Based on the weighting factors and the feedback information from the runtime hardware predictor, the candidate thread group is scored in multiple dimensions according to the multi-dimensional scoring formula. The rating is used as the priority.

11. The method according to claim 10, characterized in that, The hardware predictor includes an instruction cache predictor and a data cache predictor; The instruction cache predictor maintains a hit prediction table, indexes the hit prediction table based on a portion of the address bits of the thread's program counter, and obtains the instruction cache hit rate score of the portion of address bits, which represents the probability that the portion of address bits is predicted to hit in the instruction cache. The data cache predictor calculates the data locality score between the candidate thread group and the currently executing thread group by recording a portion of the address bits of the address of the most recent memory operation of the thread in the address record table. This score represents the distance between the representative address of the candidate thread group and the representative address of the currently executing thread group.

12. The method according to claim 11, characterized in that, The hit prediction table includes four instruction cache hit rate states: first predicted miss, second predicted miss, third predicted hit, and fourth predicted hit. The probability of a predicted miss in the first predicted miss state is higher than that in the second predicted miss state, and the probability of a predicted hit in the third predicted hit state is lower than that in the fourth predicted hit state. The higher the probability of a predicted hit for a given set of address bits within the instruction cache, the higher the instruction cache hit rate score. The method further includes: Each time an instruction is fetched from the highest priority address to continue execution, the instruction cache hit rate status of the hit prediction table is updated according to the actual hit or miss result of the highest priority address in the instruction cache.

13. The method according to claim 11, characterized in that, The data cache predictor calculates the data locality score through the following steps: Maintain an address record table that records a portion of the address of the most recent memory operation of the thread; Select a representative address of a thread from the candidate thread group from the address record table of a portion of the address of the thread's most recent memory operation; Calculate the distance between the representative address of the candidate thread group and the representative address of the currently executing thread group; The data locality score is calculated based on the distance, wherein the closer the distance, the higher the data locality score.

14. The method according to claim 11, characterized in that, The formula for the multi-dimensional scoring is: Score = w_i * ICache_Score + w_d * Dcache_Score + w_g * Group_Size_Score, Where w_i, w_d, and w_g are weighting factors. ICache_Score is the instruction cache hit score output by the instruction cache predictor. Dcache_Score is the data locality score output by the data cache predictor. Group_Size_Score is the score for the number of threads in the candidate thread group. The more threads there are, the higher the score for the number of threads.

15. The method according to claim 14, characterized in that, The method further includes: The weighting factor is adjusted according to the application type of the thread, wherein for compute-intensive applications, w_d is greater than w_g and w_g is greater than w_i; for memory-intensive applications, w_d is greater than w_g and w_g is greater than w_i; and for control flow-intensive applications, w_i is greater than w_d and w_d is greater than w_g.

16. An apparatus for scheduling threads, characterized in that, The device includes: A first mapping module is configured to map a first register in the processor to a register that records a return program counter for a thread bundle to be split and synchronized, using a first software instruction. The return program counter is used to record the address of the next instruction to be executed in each thread of the thread bundle. The second mapping module is configured to map the second register in the processor to a register for recording thread synchronization information via a second software instruction for the thread bundle. The thread synchronization information includes a thread mask and a target program counter. The thread mask is used to record whether the thread currently participating in the thread bundle differentiation and synchronization has been executed. The target program counter is used to record the target instruction address processor to jump to after the synchronization is completed. The execution module is configured to perform thread bundle differentiation and synchronization using the returned program counter obtained after mapping and the thread synchronization information. The execution module is configured as follows: In response to a branch instruction, the return program counter, the thread mask, and the target program counter of the threads in the thread bundle are recorded; In response to a thread switching instruction, the first address of the thread to be switched to is selected as the return program counter to continue execution; In response to a thread synchronization instruction, based on the thread mask, it is determined that there are still threads that have not finished executing among the currently participating threads in the synchronization process. A second address is selected from the threads that have not finished executing as the return program counter to continue execution. In response to determining based on the thread mask that all currently participating threads in the synchronization process have finished executing, execution continues based on the target program counter.

17. An apparatus for scheduling threads, characterized in that, The device includes: At least one memory storing computer instructions; At least one processor is configured to execute the computer instructions in the at least one memory to perform the method according to any one of claims 1-15.

18. A compilation method, characterized in that, The method includes: A first software instruction is inserted before the branch instruction in the source code to map a first register in the processor to a register that records the return program counter for the thread bundle to be split and synchronized. The return program counter is used to record the address of the next instruction to be executed in each thread of the thread bundle. A second software instruction is inserted before the branch instruction to map the second register in the processor to a register that records thread synchronization information for the thread bundle. The thread synchronization information includes a thread mask and a target program counter. The thread mask is used to record whether the thread currently participating in the thread bundle differentiation and synchronization has been executed. The target program counter is used to record the target instruction address processor to jump to after the synchronization is completed. At the branch instruction, a save context instruction is inserted to record the return program counter, the thread mask, and the target program counter of the thread in the thread bundle; At the instruction where thread switching is required, insert a thread switching instruction to select the first address of the thread to be switched to as the return program counter to continue execution; At the instruction that requires synchronization, a thread synchronization instruction is inserted to determine, based on the thread mask, whether there are any threads that have not yet finished executing among the currently participating threads, to select a second address from the threads that have not yet finished executing as the return program counter to continue execution, and in response to determining, based on the thread mask, that all the currently participating threads have finished executing, to continue execution based on the target program counter.

19. A compilation apparatus, characterized in that, The device includes: A first insertion module is configured to insert a first software instruction before a branch instruction in the source code, for mapping a first register in the processor to a register that records a return program counter for a thread bundle to be split and synchronized, wherein the return program counter is used to record the address of the next instruction to be executed in each thread of the thread bundle. The second insertion module is configured to insert a second software instruction before the branch instruction, for mapping a second register in the processor to a register that records thread synchronization information for the thread bundle. The thread synchronization information includes a thread mask and a target program counter. The thread mask is used to record whether the thread currently participating in the thread bundle differentiation and synchronization has been executed. The target program counter is used to record the target instruction address processor to jump to after the synchronization is completed. The third insertion module is configured to insert a save context instruction at the branch instruction, for recording the return program counter, the thread mask, and the target program counter of the thread in the thread bundle; The fourth insertion module is configured to insert a thread switching instruction at the instruction where a thread switching is required, and to select the first address of the thread to be switched as the return program counter to continue execution; The fifth insertion module is configured to insert thread synchronization instructions at the instruction requiring synchronization. It is used to determine, based on the thread mask, whether there are any threads that have not yet finished executing among the currently participating threads, select a second address from the threads that have not finished executing as the return program counter to continue execution, and, in response to determining, based on the thread mask, that all the currently participating threads have finished executing, continue execution based on the target program counter.

20. A compilation apparatus, characterized in that, The device includes: At least one memory storing computer instructions; At least one processor is configured to execute the computer instructions in the at least one memory to perform the method according to claim 18.

21. An electronic device, characterized in that, The device includes the apparatus according to any one of claims 16, 17, 19, and 20.

22. A non-transitory computer-readable storage medium, characterized in that, It stores computer instructions. When the computer instructions are executed by at least one processor, the at least one processor causes the processor to perform the method according to any one of claims 1-15 and 18.

Images