hardware execution unit, processor, electronic device

Abstract

A hardware execution unit, a processor and an electronic device. The hardware execution unit comprises a data exchange module and L selectors coupled to the output ends of L hardware resources. The data exchange module is configured to, in response to the input data of N threads entering the hardware execution unit earliest being output from the last hardware resource on a pipeline and being cached to the storage resource corresponding to the last hardware resource, collect the input data of T threads from L blocks of storage resources corresponding to the L hardware resources respectively, perform data exchange on the input data of the T threads according to the instruction parameters of a first instruction, and distribute the exchange results of the T threads to the L selectors, so as to sequentially output the exchange results of the T threads as the execution results of the first instruction through the L selectors and the L hardware resources. The hardware execution unit can reduce the storage area and the input-output delay when executing the first instruction.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of data processing, and more specifically to a hardware execution unit, processor, and electronic device. Background Technology

[0002] Single Instruction Multiple Threads (SIMT) vector instructions are a common type of processor programming instruction widely used for large-scale parallel computing tasks, such as graphics rendering and artificial intelligence (AI) computations. SIMT vector instructions execute parallel computations at the warp level. A warp contains multiple threads, and a warp (typically 32 threads) is the smallest unit of execution, with all threads executing the same instructions. For example, the SIMT32 instruction represents an instruction containing 32 threads; a single SIMT32 instruction allows threads within a warp of 32 threads to execute the same computation in parallel. This allows processors or chips to process multiple data points or computational tasks simultaneously, thereby improving computational efficiency.

[0003] In the SIMT architecture, inter-thread data exchange is a high-frequency requirement (such as reduction, matrix operations, and communication coordination). If data is transferred through global or shared memory, it incurs latency and bandwidth overhead. The shuffle instruction (also known as warp shuffle) can directly achieve inter-thread data exchange at the register level, without accessing memory, resulting in lower latency and higher efficiency. Currently, the shuffle instruction requires a large storage area, has high input / output latency, and involves numerous enable operations for the swapping logic, leading to relatively poor energy efficiency. Summary of the Invention

[0004] This invention provides a hardware execution unit for performing parallel computation in units of thread bundles. Each thread bundle includes T threads executing a first instruction, which is used for inter-thread data exchange. The hardware execution unit includes N thread hardware units, which are divided into M hardware resources. The M hardware resources are serially coupled and form a pipeline. The hardware execution unit is configured to execute the first instruction through the M hardware resources via M stages. In each stage, one hardware resource executes the corresponding operations of the N threads. M, N, and T are positive integers, and M × N is greater than or equal to T. The hardware execution unit also includes M storage resources corresponding one-to-one with the M hardware resources. Each storage resource is used to cache the intermediate results of the N threads output by the corresponding hardware resource. The hardware execution unit further includes a data exchange module and L selectors coupled to the outputs of the L hardware resources. The data exchange module is configured to, for a first thread bundle, respond to the earliest entered... The input data of N threads in the hardware execution unit has been output from the last hardware resource in the pipeline and cached in the storage resource corresponding to the last hardware resource. The input data of T threads is collected from the L storage resources corresponding to the L hardware resources respectively. The input data of the T threads is exchanged according to the instruction parameters of the first instruction. The exchange results of the T threads are distributed to the L selectors, where L is a positive integer less than or equal to M, and L = T / N. The L selectors are configured to, when receiving the exchange results of the T threads, enable data from the data exchange module to replace the thread data in the processing pipeline composed of the L hardware resources, and use the replaced thread data as the input of the hardware resource and the storage resource corresponding to the next stage. The hardware execution unit is also configured to, through the L selectors and the L hardware resources, sequentially output the exchange results of the T threads as the execution result of the first instruction.

[0005] For example, in the hardware execution unit provided in at least one embodiment of this application, the data exchange module includes an exchange enabling subunit, a data collection and distribution subunit, and a data exchange subunit. The exchange enabling subunit is configured to generate an enable signal in response to the fact that the input data of the N threads that first entered the hardware execution unit has been output from the last hardware resource and cached in the storage resource corresponding to the last hardware resource. The data collection and distribution subunit is configured to: in response to the enable signal, collect the input data of the T threads from the L blocks of storage resources and send it to the data exchange subunit; receive the exchange results of the T threads returned by the data exchange subunit, split the exchange results of the T threads into L exchange result blocks according to the thread number order, and distribute them to the L selectors respectively, wherein each exchange result block includes the exchange results of the N threads; the data exchange subunit is configured to, in response to the enable signal, perform data exchange on the received input data of the T threads according to the instruction parameters of the first instruction, and return the exchange results of the T threads to the data collection and distribution subunit.

[0006] For example, in the hardware execution unit provided in at least one embodiment of this application, the data collection and distribution subunit includes a first subunit and a second subunit. The first subunit is coupled to the L blocks of storage resources and is configured to, in response to receiving the enable signal, collect the input data of the T threads in the L blocks of storage resources, sort them in descending order of thread number, and transmit the input data of the T threads to the data exchange subunit through thread merging logic. The second subunit is configured to receive the exchange results of the T threads returned by the data exchange subunit, and, with N as the granularity, split the exchange results of the T threads into L exchange result blocks in ascending order of thread number, and distribute the L exchange result blocks to the L selectors respectively. The exchange result block containing the smallest thread number among the L exchange result blocks is distributed to the selector coupled to the output of the last hardware resource.

[0007] For example, in the hardware execution unit provided in at least one embodiment of this application, the data exchange subunit includes T T-way selectors, each of the T T-way selectors corresponding to one of the T threads. The T input ports of each T-way selector are respectively input to the input data of the T threads. Each T-way selector selects one input data to output according to the instruction parameters of the first instruction, as the exchange result of the thread corresponding to the T-way selector.

[0008] For example, in the hardware execution unit provided in at least one embodiment of this invention application, each selector is a two-to-one multiplexer. One input port of the L selectors is coupled to the output of the L hardware resources respectively, and the other input port of the L selectors is coupled to the data exchange module. For a selector coupled to the output of the last hardware resource, the output port of the selector is coupled to the output of the hardware execution unit. For the other selectors among the L selectors, the output ports of the other selectors are coupled to the input of the next-level hardware resource in the pipeline.

[0009] For example, in the hardware execution unit provided in at least one embodiment of this application, the M hardware resources are turned on sequentially in M ​​clock cycles to transmit data from each thread. When each hardware resource outputs data, the output data is cached in the storage resource corresponding to the hardware resource. In the M clock cycles, the hardware execution unit sequentially inputs M sets of input data, each set of input data including input data from N threads. Through the M clock cycles, the hardware execution unit caches the input data of the T threads into the L blocks of storage resources.

[0010] For example, in the hardware execution unit provided in at least one embodiment of this application, the switching enable subunit generates the enable signal in the first clock cycle, the data collection and distribution subunit collects the input data of the T threads in the first clock cycle, obtains the switching results of the T threads, and distributes the switching results of the T threads to the L selectors, and the hardware execution unit outputs the first switching result block in the first clock cycle.

[0011] For example, in the hardware execution unit provided in at least one embodiment of the present invention, in L-1 clock cycles after the first clock cycle, the L selectors turn on data from the previous level hardware resources of the pipeline, and the other L-1 exchange result blocks, except for the first exchange result block, are output sequentially in the L-1 clock cycles.

[0012] For example, in the hardware execution unit provided in at least one embodiment of the present invention, the L hardware resources are adjacent in the pipeline and include the last hardware resource. In response to the input data of the N threads that first enter the hardware execution unit in the first clock cycle, the input data is output from the selector coupled to the output terminal of the last hardware resource. Starting from the first clock cycle, the exchange results of the T threads are output sequentially.

[0013] For example, in the hardware execution unit provided in at least one embodiment of this application, the data exchange module is implemented using combinational logic.

[0014] For example, in the hardware execution unit provided in at least one embodiment of this application, each thread hardware independently executes the instruction operation of one thread; or, the M hardware resources are time-division multiplexed to execute the instruction operations of the T threads in parallel.

[0015] At least one embodiment of this application provides a processor including a plurality of hardware execution units as described in at least one embodiment of this application, wherein the plurality of hardware execution units are configured to execute the first instruction in parallel to process different data.

[0016] This application provides at least one embodiment of an electronic device, including a processor as described in at least one embodiment of this application.

[0017] In at least one embodiment, the hardware execution unit provided by this invention reuses the original storage resources of each hardware resource, eliminating the need for additional instantiation of a dedicated thread bundle data storage unit. The storage area only requires the storage resources of M×N threads, which is half that of the current solution, effectively saving storage space. Furthermore, this invention can start data exchange after the input data of the earliest N threads entering the hardware execution unit has been output from the last hardware resource in the pipeline, without waiting for the data of all threads in the receiving thread bundle to be output from the last hardware resource in the pipeline before exchanging data. This effectively reduces input / output latency during instruction execution, shortens the overall instruction processing time, improves the response speed of instruction execution and data transmission efficiency, and thus optimizes the overall operating performance of the hardware system. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0019] Figure 1 This is a schematic diagram of the architecture of a general-purpose graphics processing unit (GPGPU).

[0020] Figure 2 A schematic diagram of the structure of a threaded hardware;

[0021] Figure 3A This is a schematic diagram of the structure of a hardware execution unit;

[0022] Figure 3B for Figure 3A The timing diagram shown is a sequence of events when the hardware execution unit executes the shuffle instruction.

[0023] Figure 4A schematic structural diagram of a hardware execution unit provided in at least one embodiment of this disclosure;

[0024] Figure 5 A schematic structural diagram of a data exchange module provided in at least one embodiment of this disclosure;

[0025] Figure 6 A schematic structural diagram of a data exchange module provided in an embodiment of this disclosure;

[0026] Figure 7A A schematic structural diagram of a hardware execution unit provided in an embodiment of this disclosure;

[0027] Figure 7B A timing diagram for executing a first instruction provided in an embodiment of this disclosure;

[0028] Figure 8 A schematic structural diagram of a processor provided for at least one embodiment of this disclosure;

[0029] Figure 9 A schematic block diagram of an electronic device provided in one embodiment of this disclosure;

[0030] Figure 10 This is a schematic diagram of the specific structure of another electronic device provided in at least one embodiment of the present disclosure. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0032] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described object changes. To keep the following description of the embodiments of this disclosure clear and concise, detailed descriptions of some known functions and components are omitted.

[0033] Figure 1 This is a schematic diagram of a general-purpose graphics processing unit (GPGPU).

[0034] like Figure 1 As shown, a general-purpose graphics processor is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can be a streaming processor cluster (SPC), such as including... Figure 1 The diagram shows streaming processor clusters 1, ..., M, where M is a positive integer greater than 1. In a general-purpose graphics processor, one streaming processor cluster handles one computational task, or multiple streaming processor clusters handle one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0035] like Figure 1 As shown, a graphics processing unit (GPU) is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can be a streaming processor cluster (SPC), such as including... Figure 1 The diagram shows multiple streaming processor clusters. In a graphics processing unit (GPU), one streaming processor cluster handles one computational task, or multiple streaming processor clusters handle one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0036] like Figure 1 As shown, a streaming processor cluster includes multiple computing units (CUs). Each computing unit (CU) is used to perform arithmetic and logical operations, such as accumulation, reduction, and regular addition, subtraction, multiplication, and division.

[0037] A computing unit comprises multiple cores (also called computing kernels or computing cores). Each computing core includes an arithmetic logic unit (ALU), a floating-point unit, etc., and is used to execute specific computational tasks. In addition, the computing unit also includes registers (e.g., ... Figure 1 The register file and shared memory in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared memory in a computing unit is used to share data between the cores of that computing unit.

[0038] like Figure 1 As shown, each computing unit also provides a tensor core for performing tensor-related computations, such as tensor shrinking operations. Tensor cores can accelerate tensor operations such as matrix multiplication. Tensor cores in multiple computing units can be scheduled and controlled uniformly.

[0039] like Figure 1 As shown, each streaming processor cluster also provides a buffer for caching data across the N computing units within the streaming processor cluster.

[0040] In parallel computing, computational tasks are typically executed by multiple threads. These threads are divided into multiple thread blocks before execution in the graphics processing unit (or parallel computing processor), and then dispatched via a thread block distribution module. Figure 1 (Not shown in the image) Multiple thread blocks are distributed to various computation units. All threads in a thread block must be assigned to the same computation unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute in the same computation unit or in different computation units.

[0041] In each computing unit, the thread beam scheduling / distribution module ( Figure 1 (Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can execute concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory-executed instructions are issued to shared memory within the computing unit or further issued to intermediate-level caches, global caches, or global memory (e.g., [example cache]). Figure 1 High Bandwidth Memory (HBM) is used for read and write operations.

[0042] For example, each computing core includes multiple thread hardware, which can execute multiple threads in parallel, with one thread bundle running within one computing core.

[0043] Figure 2 This is a schematic diagram of the structure of a threaded hardware.

[0044] like Figure 2 As shown, the hardware resources of the thread hardware can be divided into M blocks, which are represented as follows for descriptive purposes: Figure 2 The hardware resource block is named 1, 2, ..., M, where M is a positive integer and can be determined based on the maximum number of stages required in the instructions executed by the thread.

[0045] An instruction can be completed through M hardware resources via M stages, meaning that in each clock cycle (stage), a portion of the instruction's operation is executed through one hardware resource.

[0046] For example, taking the floating-point multiply-add instruction as an example, the instruction can be completed in four stages. The first stage performs the multiplication operation through hardware resource block 1, the second stage performs the alignment operation through hardware resource block 2, the third stage performs the addition operation through hardware resource block 3, and the fourth stage performs the normalization operation through hardware resource block 4. The four stages are performed in a pipelined sequence and are executed separately through the four hardware resource blocks on the thread hardware.

[0047] For multiplication instructions, the instruction is completed in two stages. However, since the hardware itself is fixed, it still needs to be completed through the above four stages and four hardware resource blocks. For example, the second and third stages may only perform signal transmission without performing specific operations. The final result of the instruction is still the result of the multiplication instruction.

[0048] For example, N-threaded hardware can execute N threads in parallel, with each thread executing instructions independently.

[0049] For example, N threaded hardware can be time-division multiplexed using the aforementioned M hardware resource blocks, allowing M×N threads to be executed in parallel over multiple clock cycles.

[0050] For example, suppose there are N threads, described as thread 1, thread 2 to thread N respectively. In the first clock cycle, hardware resource block 1 of the thread hardware performs the stage 1 operation of thread 1, obtaining the output of stage 1. In the second clock cycle, hardware resource block 2 of the thread hardware performs the stage 2 operation of thread 1 based on the output of stage 1, obtaining the output of stage 2. At the same time, in the second clock cycle, hardware resource block 1 of the thread hardware performs the stage 1 operation of thread 1+N. In the third clock cycle, hardware resource block 3 of the thread hardware performs the stage 3 operation of thread 1. At the same time, in the third clock cycle, hardware resource block 2 of the thread hardware performs the stage 2 operation of thread 1+N, hardware resource block 1 of the thread hardware performs the stage 1 operation of thread 1+2×N, and so on.

[0051] For general-purpose graphics processing units (GPUs) or graphics processing units (GPUs), the shuffle instruction is a commonly used data exchange instruction that directly implements inter-thread data exchange at the register level. It does not require memory access, resulting in lower latency and higher efficiency. Threads within the same thread bundle exchange data directly using the shuffle instruction. Data can be transferred directly between thread registers without memory access overhead, and the latency is typically within a few clock cycles.

[0052] Common shuffle commands include data exchange patterns such as up, down, and butterfly. For example, the shuffle up command (upward exchange command) is used for thread i to retrieve data from a thread with a smaller ID (i - offset), where offset is the specified step size; the shuffle down command (downward exchange command) is used for thread i to retrieve data from a thread with a larger ID (i + offset), where offset is the specified step size; the shuffle butterfly command (butterfly command) exchanges data between threads in a "butterfly topology," where thread i exchanges data with thread i (i ^ mask) ("^" represents bitwise XOR), and mask is the specified mask (controlling the exchange distance).

[0053] Figure 3A This is a schematic diagram of the structure of a hardware execution unit.

[0054] like Figure 3A As shown, the hardware execution unit includes N thread hardware units. For details regarding the specific structure of the thread hardware, please refer to [reference needed]. Figure 2 Examples are not repeated here.

[0055] To execute the shuffle instruction, a swap enable and loop control unit, a thread bundle data storage unit, and an inter-thread data exchange unit are set up in the hardware execution unit.

[0056] The switching enable and cycle control unit is used to provide an enable signal when switching enable is required.

[0057] To implement inter-thread data exchange instructions, a thread bundle data storage unit was instantiated, which can store the data of the entire thread bundle, totaling M×N threads. The thread bundle data storage unit is used to store the data that all threads in a thread bundle need to exchange.

[0058] The inter-thread data exchange unit is used to perform inter-thread data exchange according to the parameters of the shuffle instruction.

[0059] The hardware execution unit has an input throughput and an output throughput of N threads, and the entire thread bundle that requires inter-thread data exchange has a total of M×N threads. Meanwhile, the thread hardware, such as... Figure 2 As shown, each thread's hardware is divided into M hardware resource blocks to execute instructions through an M-stage pipeline.

[0060] To store intermediate results, each of the M hardware resource blocks of the thread hardware has corresponding storage resources capable of storing data for N threads. Figure 3A (Not shown in the diagram) is used to cache the operation results output by the hardware resource block. The thread bundle data storage unit is used to store all the data of M×N threads, which can be represented as M storage blocks, each storing the data of N threads.

[0061] Figure 3B for Figure 3A The diagram shown illustrates the timing of the hardware execution unit executing the shuffle instruction. Figure 3B In the example, M=4, N=8, and each thread bundle includes 32 threads, referred to as Thread 1, Thread 2, ..., Thread 32 for clarity. Each thread's hardware is divided into 4 hardware resource blocks. Hardware resource block 1 of the 8 thread hardware blocks forms one hardware resource (hereinafter referred to as Hardware Resource 1), Hardware resource block 2 of the 8 thread hardware blocks forms another hardware resource (hereinafter referred to as Hardware Resource 2), and so on, thus the 8 thread hardware blocks have a total of 4 hardware resources. Storage blocks 1 through 4 correspond to these 4 hardware resources, and each storage block stores the data for 8 threads.

[0062] In clock cycle t0, the input data of threads 1-8 are fed into the hardware execution unit, and the corresponding operations are performed by hardware resource 1. In clock cycle t1, hardware resource 1 outputs the operation results of threads 1-8, and the data of threads 9-16 enters the hardware execution unit. In clock cycle t4, hardware resource 4 outputs the final operation results of threads 1-8 and stores them in storage block 1 of the thread bundle data storage unit. Hardware resources 1-3 execute the corresponding operations of threads 25-32, threads 17-24, and threads 9-16, respectively. In clock cycle t5, hardware resource 4 outputs the final operation results of threads 9-16 and stores them in storage block 2 of the thread bundle data storage unit, and so on.

[0063] Therefore, as Figure 3B As shown, the thread bundle data storage unit only obtains all the input data of the 32 threads at clock cycle t8. After detecting that the thread bundle data storage unit has collected the data of all threads, at clock cycle t8, the exchange enable and loop control unit will continuously enable the inter-thread data exchange unit to execute the same exchange logic for M cycles starting from the current clock cycle. Each cycle generates an exchange result block that selects N threads and outputs M exchange results with different numbers from the hardware execution unit in sequence.

[0064] For example, in clock cycle t8, the output is selected according to the mode and parameters of the shuffle instruction to obtain the swap result 1 of threads 1-8 and output it; in clock cycle t9, the swap result 2 of threads 9-16 is obtained according to the mode and parameters of the shuffle instruction and output it, and so on.

[0065] In the above scheme, it can be seen that the hardware execution unit that can execute shuffle instructions currently occupies a large area and requires instantiation of a dedicated thread bundle data storage unit, resulting in a significant storage area consumption. The total storage area of ​​the hardware execution unit is 2 × M × N threads. Furthermore, after the thread hardware completes the final stage of threads 1-8, it still needs to wait for the thread bundle data storage unit to receive data from all threads within the thread bundle before the swap can be performed and the final output completed. This is because all input data needs to be stored in the thread bundle storage unit before the swap can be performed, thus resulting in higher input / output latency. Figure 3B As shown, due to throughput limitations, it takes 12 clock cycles (t0-t11) to complete the shuffle instruction.

[0066] Furthermore, this approach involves numerous logic enable operations, resulting in poorer energy efficiency. For example... Figure 3B As shown, due to throughput limitations, even after receiving data from all threads within the thread bundle, it is still necessary to continuously perform M exchange logic enable operations to output the exchange results of all threads in the thread bundle. This results in higher dynamic power consumption and lower efficiency.

[0067] This disclosure provides at least one embodiment of a hardware execution unit, processor, and electronic device. The hardware execution unit is used to perform parallel computation in units of thread bundles. Each thread bundle includes T threads executing a first instruction, which is used for inter-thread data exchange. The hardware execution unit includes N thread hardware units, which are divided into M hardware resources. The M hardware resources are serially coupled and form a pipeline. The hardware execution unit is configured to execute the first instruction through M hardware resources and M stages. In each stage, one hardware resource executes the corresponding operations of the N threads. M, N, and T are positive integers, and M × N is greater than or equal to T. The hardware execution unit also includes M storage resources corresponding one-to-one with the M hardware resources. Each storage resource is used to cache the intermediate results of the N threads output by the corresponding hardware resource. The hardware execution unit also includes a data exchange module and L selectors coupled to the outputs of the L hardware resources. The data exchange module is configured to, for a first thread bundle, respond to the earliest entered... The input data of N threads in the hardware execution unit has been output from the last hardware resource in the pipeline and cached in the storage resource corresponding to the last hardware resource. The input data of T threads is collected from the L storage resources corresponding to the L hardware resources respectively. The input data of the T threads is exchanged according to the instruction parameters of the first instruction. The exchange results of the T threads are distributed to L selectors, where L is a positive integer less than or equal to M, and L = T / N. The L selectors are configured to, when receiving the exchange results of the T threads, turn on the data from the data exchange module to replace the thread data in the processing pipeline composed of the L hardware resources, and use the replaced thread data as the input of the hardware resource and the storage resource corresponding to the next stage. The hardware execution unit is also configured to, through the L selectors and the L hardware resources, sequentially output the exchange results of the T threads as the execution result of the first instruction.

[0068] For example, the first thread bundle can be any thread bundle currently being executed by the hardware execution unit for executing the first instruction.

[0069] In at least one embodiment, the hardware execution unit provided by this disclosure reuses the original storage resources of each hardware resource, eliminating the need for additional instantiation of a dedicated thread bundle data storage unit. The storage area only requires the storage resources of M×N threads, which is half that of the current solution, effectively saving storage space. Furthermore, this disclosure allows data exchange to begin after the input data of the earliest N threads entering the hardware execution unit has been output from the last hardware resource in the pipeline. It eliminates the need to wait for the data of all threads in the receiving thread bundle to be output from the last hardware resource in the pipeline before exchanging data, effectively reducing input / output latency during instruction execution, shortening the overall instruction processing time, improving the response speed of instruction execution and data transmission efficiency, and thus optimizing the overall operating performance of the hardware system.

[0070] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments.

[0071] Figure 4 This is a schematic structural diagram of a hardware execution unit provided in at least one embodiment of the present disclosure.

[0072] For example, the hardware execution unit can be the execution unit (EU) in the aforementioned graphics processor. Of course, in different processor hardware architectures, the hardware execution unit can also take other forms, and this disclosure does not impose any specific limitations.

[0073] The hardware execution unit is used to perform parallel computation in units of thread bundles. A thread bundle includes T threads that execute a first instruction. The first instruction is used to exchange data between threads. The first instruction is, for example, a shuffle instruction. T is a positive integer. For example, in different hardware architectures, T can be 32, 64, etc.

[0074] like Figure 4 As shown, the hardware execution unit includes N thread hardware units, where N is a positive integer. More details about the thread hardware can be found in the previous section. Figure 2 The relevant descriptions will not be repeated here.

[0075] refer to Figure 2 The description states that thread hardware can be divided into M hardware resource blocks to execute complex instructions through multiple stages. For example, refer to... Figure 2 The thread hardware can be divided into hardware resource block 1, hardware resource block 2, hardware resource block 3, ..., hardware resource block M. The M hardware resource blocks are serially coupled and form a pipeline.

[0076] like Figure 4 As shown, the N threads are divided into M hardware resources 101. For clarity, in... Figure 4The M hardware resources are labeled as hardware resources 101_1, 101_2, 101_3, ..., 101_M. These M hardware resources are serially coupled and form a pipeline. In other words, the M hardware resources are connected sequentially to form a unidirectional link, with data or signals transmitted in a fixed direction. The output of one hardware resource is coupled to the input of the next.

[0077] Specifically, hardware resource 101_1 includes hardware resource block 1 in N-thread hardware, hardware resource 101_2 includes hardware resource block 2 in N-thread hardware, hardware resource 101_3 includes hardware resource block 3 in N-thread hardware, and so on. That is to say, in Figure 4 In this system, each hardware resource can execute the corresponding operations of N threads in parallel.

[0078] The hardware execution unit is configured to execute the first instruction through M hardware resources and M stages. In each stage, one hardware resource executes the corresponding operations of N threads, where M, N, and T are positive integers, and M×N is greater than or equal to T.

[0079] When executing the first instruction used for inter-thread data exchange, the hardware resources do not perform specific calculations, but instead establish a connection to transmit data. They do not perform actual calculations on the data, nor do they change the input data.

[0080] like Figure 4 As shown, the hardware execution unit 100 also includes M blocks of storage resources 103 corresponding one-to-one with the M hardware resources 101. For clarity, in... Figure 4 The values ​​are 103_1 to 103_M. Each storage resource is used to cache the intermediate results of N threads output by the corresponding hardware resource. For example, the output result of hardware resource 101_1 is cached in storage resource 103_1, the output result of hardware resource 101_2 is cached in storage resource 103_2, the output result of storage resource 101_M is cached in storage resource 103_M, and so on.

[0081] For example, in the current clock cycle t0, input data from N threads is entered into hardware execution unit 100 and then into hardware resource 101_1; in the next clock cycle t1, the input data is cached in storage resource 103_1, and hardware resource 101_1 outputs the input data from the N threads to hardware resource 101_2; in the next clock cycle t2, the input data is cached in storage resource 103_2, and hardware resource 101_2 outputs the input data from the N threads to hardware resource 101_3; ...; in the next clock cycle tM, the input data is cached in storage resource 103_M, and hardware resource 101_M outputs the input data from the N threads.

[0082] For example, in some embodiments, each thread hardware can independently execute the instruction operations of one thread, for example, N equals T. In this case, each thread hardware can independently execute the instruction operations of one thread, and during clock cycle tM, the input data of T threads are cached in storage resource 103_M.

[0083] For example, in other embodiments, M hardware resources are time-division multiplexed to execute the instructions of T threads in parallel.

[0084] For example, the thread numbers of the T threads are thread 1 to thread T.

[0085] For example, in clock cycle t0, input data for threads 1 to N is input. In clock cycle t1, input data for threads N+1 to 2×N is input. At this time, the input data for threads 1 to N has entered hardware resource 101_2, and hardware resource 101_1 is idle and can be reused. Afterwards, the input data for threads N+1 to 2×N continues to flow down the pipeline step by step according to the above process.

[0086] The same principle applies to the input data of threads 2×N+1 to 3×N, which will not be elaborated here.

[0087] In the thread hardware, each hardware resource is provided with storage resources for caching data of N threads. Since this disclosure reuses the storage resources to cache the input data of T threads, it is necessary to ensure that the storage resources corresponding to the hardware resources can completely store the input data of T threads. Therefore, it is necessary to ensure that M×N is greater than or equal to T.

[0088] like Figure 4 As shown, the hardware execution unit also provides a data exchange module 102 and L selectors coupled to the outputs of L hardware resources.

[0089] It should be noted that, in Figure 4 In the example, L=M, but this disclosure is not limited thereto; L=T / N, and in some scenarios L can be less than M.

[0090] The data exchange module 102 is configured to, in response to the fact that the input data of the N threads that first entered the hardware execution unit has been output from the last hardware resource 101_M on the pipeline and cached in the storage resource 103_M corresponding to the last hardware resource 101_M, collect the input data of T threads from the L storage resources corresponding to the L hardware resources respectively, exchange the input data of the T threads according to the instruction parameters of the first instruction, and distribute the exchange results of the T threads to the L selectors, where L is a positive integer less than or equal to M, and L=T / N.

[0091] As previously described, this disclosure reuses the storage resources provided in the thread hardware to cache the input data of T threads. Therefore, L blocks of storage resources can be used to cache the input data of T threads. For example, when T=32, N=8, L=4, and 4 blocks of storage resources can be used to obtain all the input data of 32 threads.

[0092] When the input data of the N threads that first entered the hardware execution unit has been output from the last hardware resource 101_M in the pipeline and cached in the storage resource 103_M corresponding to the last hardware resource 101_M, the input data of the T threads has entered the storage resources corresponding to the L hardware resources.

[0093] For example, a pipeline consists of L hardware resources, which are adjacent to each other in the pipeline, and include the last hardware resource 101_M in the pipeline. For example, L blocks of storage resources can be storage resources 103_M-L+1 to storage resource 103_M.

[0094] Input data from T threads is collected from L memory blocks corresponding to L hardware resources. Data exchange is performed on the input data of the T threads according to the instruction parameters of the first instruction. The exchange results of the T threads are then distributed to L selectors. Therefore, this disclosure allows data exchange to begin after the input data of the first group of N threads completes its final stage, without waiting for the input data of the last group of N threads to complete its final stage. This reduces the input / output latency of the first instruction, shortens the overall instruction processing time, improves the instruction execution response speed and data transmission efficiency, and ultimately optimizes the overall operating performance of the hardware system.

[0095] Furthermore, in this disclosure, the exchange logic is executed directly through the data exchange module, eliminating the need for data storage and saving storage space.

[0096] like Figure 4 As shown, the data exchange module 102 can collect input data 105 from various storage resources 103, which, for clarity, is represented as... Figure 4 The input data is 105_1 to 105_M. After the data exchange module 102 performs the data exchange, it redistributes the exchange result block 106 to L selectors 104, which are represented as follows for clarity: Figure 4 The swap result blocks 106_1 to 106_M in the middle.

[0097] exist Figure 4In the example, L=M, meaning that a two-to-one multiplexer 104 is coupled to the output of each hardware resource 101, referred to as multiplexer 104_1 to multiplexer 104_M for clarity. However, it should be noted that, depending on the specific embodiment, those skilled in the art can flexibly set the number of multiplexers L based on the relationship between N and T, i.e., L=T / N. For example, in some embodiments, L=M / 2, in which case selectors can be set at the outputs of M / 2 hardware resources from hardware resource 101_M / 2+1 to hardware resource 101_M, which will not be elaborated further here.

[0098] The L selectors are configured to, upon receiving the exchange results from the T threads, enable data from the data exchange module 102 to replace the thread data in the processing pipeline composed of the L hardware resources, and use the replaced thread data as the input to the hardware resources and the corresponding storage resources of the next stage.

[0099] For example, when L is less than M, the processing pipeline can be understood as a part of a pipeline composed of M hardware resources; or, when M=L, the processing pipeline can be that pipeline itself. For example, in Figure 4 In the example, M=L, the processing pipeline can be the entire pipeline.

[0100] like Figure 4 As shown, for example, when the exchange result is received from the data exchange module 102 in the current clock cycle, each selector 104 selects the exchange result block 106 to output, thereby replacing the thread data in the pipeline composed of L hardware resources. The exchange result block 106_1 is directly output through the selector 104_M. Other exchange result blocks are used as the input of the hardware resources of the next stage and the corresponding storage resources of the hardware resources, and continue to be passed step by step in the pipeline, and are output sequentially in the following clock cycles.

[0101] like Figure 4 As shown, each selector is a 2-to-1 multiplexer, meaning it includes two input ports and one output port. L selectors are located at the outputs of L hardware resources, with one input port of each of the L selectors coupled to the output of one of the L hardware resources, and the other input port of each of the L selectors coupled to a data exchange module to receive the exchange result. For a selector coupled to the output of the last hardware resource, its output port is coupled to the output of the hardware execution unit. For the other selectors among the L selectors, their output ports are coupled to the inputs of the next-level hardware resource in the pipeline.

[0102] like Figure 4As shown, one input port of selector 104_1 is coupled to the output of hardware resource 101_1, another input port is coupled to data exchange module 102, and its output port is coupled to the input of the next-level hardware resource 101_2. One input port of selector 104_M is coupled to the output of hardware resource 101_M, another input port is coupled to data exchange module 102, and its output port is coupled to the output of hardware execution unit.

[0103] The hardware execution unit is also configured to sequentially output the exchange results of T threads as the execution result of the first instruction through L selectors and L hardware resources.

[0104] like Figure 4 As shown, after receiving the exchange result, selector 104_M can output the data exchange results between threads in sequence, avoiding the time-consuming process of receiving data from all threads in the thread bundle before exchanging data, and reducing the input-output delay of executing the first instruction.

[0105] Figure 5 This is a schematic structural diagram of a data exchange module provided in at least one embodiment of the present disclosure.

[0106] like Figure 5 As shown, the data exchange module 102 includes an exchange enable subunit 1021, a data collection and distribution subunit 1022, and a data exchange subunit 1023.

[0107] The switching enable subunit 1021 is configured to generate an enable signal in response to the fact that the input data of the N threads that first entered the hardware execution unit has been output from the last hardware resource and cached in the storage resource corresponding to the last hardware resource. It should be noted that the switching enable subunit generates an enable signal when the hardware execution unit executes the first instruction, provided this condition is met. If the hardware execution unit is not executing the first instruction, it follows the normal execution process, and the switching enable subunit does not generate an enable signal when the above condition is met.

[0108] The data collection and distribution subunit 1022 is configured to, in response to receiving an enable signal, collect input data from T threads from L blocks of storage resources and send it to the data exchange subunit; receive the exchange results of T threads returned by the data exchange subunit, split the exchange results of T threads into L exchange result blocks according to the thread number order, and distribute them to L selectors respectively, wherein each exchange result block includes the exchange results of N threads.

[0109] exist Figure 5 In the example, L=M is ​​still used as an example for description, but as mentioned before, in some embodiments, L can be less than M.

[0110] For example, such as Figure 5As shown, the data exchange subunit 1023 is configured to receive input data from T threads, perform data exchange between the T threads according to the instruction parameters of the first instruction, and return the exchange results of the T threads to the data collection and distribution subunit.

[0111] Figure 6 This is a schematic structural diagram of a data exchange module provided in an embodiment of the present disclosure.

[0112] like Figure 6 As shown, the data collection and distribution subunit 1022 includes a first subunit and a second subunit.

[0113] The first sub-unit is coupled to the L-block storage resources. Figure 6 In the example, L=M, the first sub-unit and Figure 4 The M storage resources 103 in the middle are coupled accordingly.

[0114] In addition, Figure 6 In the example, T = M × N and L = M, and the T threads include thread 1, thread 2, ..., thread N, ..., thread M × N.

[0115] exist Figure 6 In the above, the input data of N threads collected from storage resource 103_1 includes the input data of threads ((M-1)×N+1) to M×N, the input data of N threads collected from storage resource 103_M includes the input data of threads 1 to N, and so on. The exchange data block 106_M distributed to selector 104_1 includes the exchange results of threads ((M-1)×N+1) to M×N, the exchange data block 106_1 distributed to selector 104_M includes the exchange results of threads 1 to N, and so on.

[0116] The first subunit is configured to collect input data from T threads in L blocks of storage resources in response to receiving an enable signal generated by the switching enable subunit 1021. Figure 6 In the example, input data 105_1 to input data 105_M are collected and sorted in descending order of thread number. The input data from T threads is then transferred to the data exchange subunit 1023 using a data bandwidth of M×N×B via thread merging logic. Here, B represents the data volume of a single thread.

[0117] The second subunit is configured to receive the exchange results of T threads returned by the data exchange subunit 1023, and, with N as the granularity, split the exchange results of the T threads into L exchange result blocks in ascending order of thread number, for example... Figure 6The L result blocks 106_1 to 106_M are exchanged and distributed to L selectors 104, namely selectors 104_1 to 104_M.

[0118] For example, the swap result block containing the smallest thread number in L swap result blocks is distributed to the selector coupled to the output of the last hardware resource. Figure 6 As shown, the exchange result block 106_1, which includes the exchange results of threads 1 to N, is distributed to selector 104_M; the exchange result block 106_2, which includes the exchange results of threads N+1 to 2×N, is distributed to selector 104_M-1, and so on. This ensures that the output exchange result blocks are output in a predetermined order and are consistent with the thread number of the input data to the input hardware execution unit.

[0119] For example, the data exchange subunit 1023 includes T T-way selectors, each corresponding to one of the T threads. Each T-way selector's T input ports receive input data from the T threads respectively. Each T-way selector selects one input data point for output according to the instruction parameters of the first instruction, serving as the exchange result for the thread corresponding to that selector. The parameters of the first instruction may include, for example, the data exchange mode between threads, such as up, down, etc., and may also include specific parameters, such as the aforementioned offset.

[0120] In this disclosure, as shown above, the data collection and distribution subunit 1022 is composed of combinational logic. Furthermore, the data exchange subunit 1023 may also be composed of combinational logic, neither of which contains storage components. The data exchange module 102 can complete data collection, exchange, splitting, and distribution within one clock cycle, without the need to instantiate dedicated storage resources, effectively saving storage space.

[0121] In at least one embodiment, in response to the input data of the N threads that first enter the hardware execution unit in the first clock cycle being output from the selector coupled to the output of the last hardware resource, the exchange results of T threads are output sequentially starting from the first clock cycle. That is, this disclosure can output the results of data exchange between threads sequentially after the thread hardware completes the last stage, without waiting for the data from all threads within the thread bundle to be received before the exchange process, thus improving the execution efficiency of the first instruction and reducing the input-output latency of the first instruction.

[0122] Furthermore, unlike current methods that require enabling the inter-thread data exchange sub-unit's exchange logic M times consecutively, this disclosure only requires enabling the exchange logic once (generating an enable signal once) to complete the exchange process, resulting in lower dynamic power consumption and higher efficiency.

[0123] The following describes the overall execution process of the hardware execution unit provided in at least one embodiment of this disclosure when executing the first instruction from a timing perspective.

[0124] For example, if the hardware execution unit 100 does not receive the first instruction, the conventional approach can be adopted to execute the instruction through multiple hardware resources that make up the pipeline. In this case, the data exchange module will not be triggered, and the selector will continue to select to conduct and transmit the output data of the previous level hardware resource.

[0125] For example, when the first instruction is executed, M hardware resources are turned on sequentially over M clock cycles to transmit data from each thread. When each hardware resource outputs data, the output data is cached in the corresponding storage resource. It is understandable that if hardware resource 101_1 outputs the second set of received input data at this time, it will overwrite the first set of input data originally cached in storage resource 103_1.

[0126] In M clock cycles, the hardware execution unit sequentially inputs M sets of input data, each set of input data including input data from N threads, and caches the input data of T threads into L blocks of storage resources over M clock cycles.

[0127] In other words, during these M clock cycles, the hardware resources of the N threads are turned on sequentially from front to back, transmitting data from each thread.

[0128] For example, in the first clock cycle, the input data of the N threads that entered the hardware execution unit earliest has been output from the last hardware resource on the pipeline and cached in the storage resource corresponding to the last hardware resource. The switching enable subunit generates an enable signal in the first clock cycle. The data collection and distribution subunit collects the input data of T threads from L blocks of storage resources in the first clock cycle, obtains the switching results of T threads, and distributes the switching results of T threads to L selectors.

[0129] Since data collection, merging, splitting, and distribution are completed in the first clock cycle, there is no need for data storage, effectively saving hardware storage space.

[0130] In addition, the hardware execution unit can output the first exchange result block 106_1 in the first clock cycle, without having to wait for the data of all threads in the receiving thread bundle before performing the exchange process.

[0131] In the L-1 clock cycles following the first clock cycle, the L selectors turn on the data from the previous level hardware resources of the pipeline. Since the data in the pipeline has been replaced by the exchange result blocks in the first clock cycle, the other L-1 exchange result blocks, except for the first exchange result block, can be output sequentially in the L-1 clock cycles.

[0132] The following is combined with Figure 7A and Figure 7B Describe the structure of the hardware execution unit and the workflow of executing the first instruction when M=4, N=8, L=4, T=32.

[0133] Figure 7A This is a schematic structural diagram of a hardware execution unit provided in an embodiment of the present disclosure.

[0134] like Figure 7A As shown, in this embodiment, the hardware execution unit includes eight threaded hardware units, each of which is divided into four hardware resource blocks to execute instructions through a four-stage pipeline. The four hardware resource blocks of the eight threaded hardware units constitute hardware resources 101_1 to hardware resources 101_4 as shown in 7A.

[0135] The thread bundle includes 32 threads, referred to as thread 1, thread 2, ..., thread 32.

[0136] Each hardware resource 101 also has a corresponding storage resource 103, for example Figure 7A Storage resources 103_1 to 103_4 are used to cache the intermediate results of the eight threads output by the corresponding hardware resources.

[0137] Each of the four hardware resources 101 has a selector 104 coupled to its output. The specific connection relationship of the selectors can be referred to the previous description, and will not be repeated here.

[0138] When executing regular instructions, the switching enable subunit 1021 of the hardware execution unit 100 will not generate an enable signal. The various stages of the N thread hardware will be turned on in sequence from front to back, and each stage will have a storage resource of size N×B to store intermediate results.

[0139] Figure 7B A timing diagram for executing a first instruction provided in an embodiment of this disclosure.

[0140] When the first instruction is executed, the first set of input data, namely the input data of threads 1-8, is input at clock cycle t0. For the next 4 clock cycles starting from clock cycle t0, all hardware resources are normally turned on from top to bottom to transmit the data of each thread.

[0141] like Figure 7BAs shown, during clock cycle t1, the input data of threads 1-8 is buffered in storage resource 103_1 and output from hardware resource 101_1; during clock cycle t2, the input data of threads 1-8 is buffered in storage resource 103_2 and output from hardware resource 101_2; during clock cycle t3, the input data of threads 1-8 is buffered in storage resource 103_3 and output from hardware resource 101_3; during clock cycle t4, the input data of threads 1-8 is buffered in storage resource 103_4 and output from hardware resource 101_4.

[0142] like Figure 7B As shown, in clock cycle t1, the second set of input data, namely the input data of threads 9-16, is input; in clock cycle t2, the input data of threads 9-16 is buffered in storage resource 103_1 and output from hardware resource 101_1; in clock cycle t3, the input data of threads 9-16 is buffered in storage resource 103_2 and output from hardware resource 101_2; in clock cycle t4, the input data of threads 9-16 is buffered in storage resource 103_3 and output from hardware resource 101_3.

[0143] The same applies to the input data for threads 17-24 and threads 25-32, so it will not be repeated here.

[0144] Therefore, during clock cycle t4, the input data of threads 1-8 is cached in storage resource 103_4, the input data of threads 9-16 is cached in storage resource 103_3, the input data of threads 17-24 is cached in storage resource 103_2, and the input data of threads 25-32 is cached in storage resource 103_1.

[0145] At clock cycle t4, the first 8 threads to enter the hardware execution unit (i.e., threads 1 to 8) output from the last hardware resource of the thread hardware. The storage resource corresponding to each hardware resource already has thread data. At this time, the enable subunit 1021 generates an enable signal.

[0146] During clock cycle t4, the data collection and distribution subunit 1022 receives an enable signal, collects 32 thread input data from 4 storage resources, and transmits them to the data exchange subunit 1023 at the same time. Therefore, no data storage is required during this stage.

[0147] During clock cycle t4, the data exchange subunit 1023 responds to the enable signal to perform data exchange between 32 threads. It completes the data exchange between threads in the form of up / down / butterfly according to the instruction parameters of the first instruction, and generates the exchange results of 32 threads at the same time. The data exchange subunit 1023 does not need to store data.

[0148] During clock cycle t4, the data collection and distribution subunit 1022 splits the received exchange results from the 32 threads into 4 exchange result blocks (i.e., exchange result block 106_1 to exchange result block 106_4), and distributes the 4 split exchange result blocks to each selector 104 at the same time. Therefore, no data storage is required during this stage.

[0149] Furthermore, during clock cycle t4, all selectors are turned on to transmit the switching result blocks from the data collection and distribution subunit 1022, replacing the original thread data in the pipeline with each switching result block, and using each switching result block as the input to the next hardware resource and its storage resources.

[0150] At the same time, such as Figure 7B As shown, during clock cycle t4, the hardware execution unit outputs the swap result block 1, which is the execution result of the first instruction. Figure 7A The swap result block 106_1 in the middle includes the swap results of threads 1 to 8.

[0151] exist Figure 7A In the diagram, the data transmission associated with clock cycle t4 is shown by a bold black line.

[0152] Subsequently, from clock cycle t5 to clock cycle t7, the four selectors 104 are reactivated to transmit data normally in the hardware resource pipeline, i.e., they switch back to normal operation. Figure 7A The data transmission paths for 105_1 and other data outputs from various hardware resources in the pipeline are as follows: However, since the data in the pipeline has been replaced by various exchange result blocks, ... Figure 7B As shown, the result block 2 (i.e. Figure 7A The swap result blocks 106_2 to 4 (i.e.) are in the middle. Figure 7A The swap result block 106_4 will be output sequentially from clock cycle t5 to clock cycle t7.

[0153] Furthermore, since the hardware execution unit is implemented in a pipelined manner, the first set of input data for the next first instruction can be input to the hardware execution unit at clock cycle t5, and the above process is repeated. The relevant content of the next first instruction is... Figure 7B The text is shown with a gray background to distinguish it from the first instruction in this context. The execution process of the next first instruction can be found in the preceding content and will not be repeated here.

[0154] from Figure 7B As can be seen, the next first instruction and part of the operation of the current first instruction overlap in time. Therefore, the hardware execution unit can continuously execute each first instruction in a pipeline manner without waiting, which effectively improves the execution efficiency of the first instruction.

[0155] In this embodiment, compared to the traditional solution which requires instantiating a dedicated thread bundle data storage unit (storage capacity of 2×M×N×B) in the hardware execution unit, this disclosure reuses the original pipeline storage resources in the hardware execution unit, and the total storage area is only M×N×B, which is only half of the traditional solution, effectively saving storage area.

[0156] This disclosure can output the results of data exchange between threads sequentially after the thread hardware completes the last stage of the first set of input data, that is, in clock cycle t4. Compared with the traditional solution, which requires clock cycle t8 to output the results of data exchange between threads, this effectively reduces the input-output latency of executing the first instruction and improves the execution efficiency of executing the first instruction.

[0157] Furthermore, compared to traditional solutions that require enabling the switching logic of the inter-thread data exchange sub-unit M times consecutively, this disclosure only requires enabling the switching logic once, reducing the number of times the switching logic is enabled, resulting in lower dynamic power consumption, higher efficiency, and better energy efficiency ratio.

[0158] At least one embodiment of this disclosure also provides a processor. Figure 8 A schematic structural diagram of a processor provided for at least one embodiment of this disclosure.

[0159] like Figure 8 As shown, the processor 200 includes multiple hardware execution units 100. The specific structure and operation of the hardware execution units 100 can be found in the descriptions of the foregoing embodiments, and will not be repeated here.

[0160] Multiple hardware execution units are configured to execute the first instruction in parallel to process different data. For example, as mentioned earlier, for graphics processing units (GPUs), computational tasks are typically executed through multiple threads, and all threads in a thread block must be assigned to the same computational unit for execution. Thread blocks are broken down into minimum execution thread bundles, and a single hardware execution unit is configured to execute multiple threads within a thread bundle, thus enabling parallel processing of computational tasks through multiple hardware execution units.

[0161] In at least one embodiment of this disclosure, the processor can be a processor of any architecture, such as a graphics processor, a tensor processor, or a data processor. The following description uses a graphics processor as an example to illustrate the illustrative structure of a graphics processor provided in at least one embodiment of this disclosure.

[0162] For example, refer to this disclosure Figure 1 The schematic diagram of the graphics processor shows that the four cores of the computing unit can adopt the structure of the hardware execution unit provided in at least one embodiment of this disclosure. For a more detailed description of the hardware execution unit, please refer to the relevant description of the hardware execution unit 100 as described above, and repeated details will not be repeated.

[0163] In at least one embodiment, by setting a data exchange module implemented with combinational logic in the computing core of the graphics processor, there is no need to instantiate a dedicated thread bundle data storage unit. The storage area only requires the storage resources of M×N threads, which is half of the storage area of ​​the current traditional solution, effectively saving storage area. Furthermore, this disclosure can start data exchange after the input data of the first N threads entering the hardware execution unit has been output from the last hardware resource in the pipeline. There is no need to wait for the data of all threads in the receiving thread bundle to be output from the last hardware resource in the pipeline before the exchange is performed. This effectively reduces the input and output latency in the instruction execution process, shortens the overall instruction processing time, improves the response speed of instruction execution and the data transmission efficiency, and thus optimizes the overall operating performance of the hardware system.

[0164] In addition, it should be noted that Figure 8 The components of the graphics processor 200 shown are merely exemplary and not limiting. The graphics processor 200 may have other components as needed for the actual application.

[0165] Figure 9 A schematic diagram of an electronic device provided in at least one embodiment of the present disclosure is shown.

[0166] For example, such as Figure 9 As shown, the electronic device 300 includes a processor 200. For example, the processor 200 may employ... Figure 8 The structure shown is implemented as described. For example, the electronic device 300 can be any electronic device including those with computing capabilities, such as a laptop, tablet, desktop computer, web server, etc., and the embodiments disclosed herein are not limited thereto.

[0167] For example, electronic device 300 can be implemented in hardware, software, or a combination of hardware and software, and this disclosure does not impose any specific restrictions on it.

[0168] In some embodiments, the electronic device 300 provided in at least one embodiment of this disclosure can be a single-machine multi-card form, where multiple GPU cards are integrated in a single server or workstation, and inter-card communication is achieved through a bus (such as PCIe bus, AMD bus, etc.). For example, the electronic device 300 can also be a multi-machine multi-card system, where multiple single-machine multi-card servers are interconnected through a high-speed network to form a cluster, and each card can not only communicate with other cards on its own machine, but also interact with cards on other nodes through the network. For example, the electronic device 300 can also be a TPU cluster, a multi-FPGA accelerator card system, etc., and this disclosure does not impose specific limitations in this regard.

[0169] For example, this electronic device 300 can overcome the limitations of single-card computing power or memory through parallel computing, and is therefore widely used in scenarios requiring large-scale data processing, high-throughput computing, or low-latency parallel tasks. Examples include deep learning and large model training, high-performance computing, large-scale data processing and AI inference, real-time rendering and visualization, and it can be applied to fields such as large model training, scientific computing, high-concurrency AI services, and real-time rendering.

[0170] For example, the electronic device 300 may also include other forms of processing units with data processing capabilities and / or instruction execution capabilities, such as a central processing unit (CPU) or a digital signal processor (DSP), as well as storage units. The electronic device may also be equipped with an operating system and application programming interfaces (e.g., OpenGL (Open Graphics Library), Metal, etc.). For example, the electronic device may also include an output component, such as a display component, which may be a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, a quantum dot light-emitting diode (QLED) display, etc. The embodiments disclosed herein do not impose limitations on this.

[0171] It should be noted that, for clarity and brevity, this disclosure does not show all the constituent units of the electronic device 300. To achieve the necessary functions of the electronic device 300, those skilled in the art can provide or set other constituent units (not shown) according to specific needs, and this disclosure does not limit this.

[0172] The following is for reference. Figure 10 It illustrates a specific structural diagram of an electronic device (e.g., a terminal device or a server) 300 suitable for implementing a processor including embodiments of the present disclosure.

[0173] The terminal devices in this disclosure can include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. For example, the electronic device can be in the form of a server, used for various application scenarios such as deep learning and artificial intelligence, scientific computing, graphics rendering and video editing, virtual reality and game development, and cloud services. For example, the electronic device can be a dedicated server such as a data center or cloud computing center that is deployed with tasks such as deep learning training, large-scale data analysis, and high-performance computing.

[0174] Figure 10 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0175] like Figure 10 As shown, the electronic device 300 may include a processing unit 301, such as the aforementioned processor 200, which can execute various appropriate actions and processes according to non-transitory computer-readable instructions stored in memory to achieve various functions. The processing unit 301 may also include devices with instruction optimization capabilities and / or program execution capabilities, such as a central processing unit (CPU) or a tensor processor (TPU). The CPU can be based on x86, ARM, or RISC-V architectures. The GPU can be directly integrated into the SOC, directly integrated onto the motherboard, or built into the northbridge chip of the motherboard.

[0176] like Figure 10 As shown, for example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 303 and / or cache memory, etc., for example, computer-readable instructions may be loaded from storage device 308 into RAM 303 to execute computer-readable instructions. Non-volatile memory may include, for example, read-only memory (ROM) 302, hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. Various applications and various data, such as various data used and / or generated by applications, may also be stored in the computer-readable storage medium.

[0177] For example, the processing device 301, the read-only memory (ROM) 302, and the random access memory (RAM) 303 are interconnected via a bus 304. The input / output (I / O) interface 305 is also connected to the bus 304.

[0178] Typically, the following devices can be connected to the input / output (I / O) interface 305: input devices 306 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 308 including, for example, magnetic tape, hard disk, flash memory, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 10An electronic device 300 with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown, and the electronic device 300 may alternatively implement or have more or fewer devices. For example, the processing device 301 may control other components in the electronic device 300 to perform desired functions.

[0179] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0180] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0181] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0182] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire at least two Internet Protocol (IP) addresses; send a node evaluation request including the at least two IP addresses to a node evaluation device, wherein the node evaluation device selects an IP address from the at least two IP addresses and returns it; and receive the IP address returned by the node evaluation device; wherein the acquired IP address indicates an edge node in a content delivery network.

[0183] Alternatively, the aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: receive a node evaluation request including at least two Internet Protocol (IP) addresses; select an IP address from the at least two IP addresses; and return the selected IP address; wherein the received IP address indicates an edge node in the content delivery network.

[0184] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0185] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0186] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0187] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0188] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

[0189] The following points should be noted regarding this disclosure:

[0190] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0191] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0192] The above description is only a specific embodiment of this disclosure, but the protection scope of this disclosure is not limited thereto. The protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A hardware execution unit, characterized in that, This is used to perform parallel computations in units of thread bundles, where each thread bundle includes T threads executing a first instruction, which is used for inter-thread data exchange. The hardware execution unit includes N thread hardware units, which are divided into M hardware resources. The M hardware resources are serially coupled and form a pipeline. The hardware execution unit is configured to execute the first instruction through M stages using the M hardware resources, wherein in each stage, one hardware resource executes the corresponding operations of N threads, where M, N, and T are positive integers, and M×N is greater than or equal to T. The hardware execution unit also includes M storage resources corresponding one-to-one with the M hardware resources. Each storage resource is used to cache the intermediate results of the N threads output by the corresponding hardware resources. The hardware execution unit also includes a data exchange module and L selectors coupled to the outputs of the L hardware resources. The data exchange module is configured to, for the first thread bundle, in response to the fact that the input data of the N threads that entered the hardware execution unit earliest in the first thread bundle has been output from the last hardware resource on the pipeline and cached in the storage resource corresponding to the last hardware resource, collect the input data of the T threads from the L storage resources corresponding to the L hardware resources respectively, exchange the input data of the T threads according to the instruction parameters of the first instruction, and distribute the exchange results of the T threads to the L selectors, where L is a positive integer less than or equal to M, and L=T / N; The L selectors are configured to, upon receiving the exchange results from the T threads, enable data from the data exchange module to replace the thread data in the processing pipeline composed of the L hardware resources, and use the replaced thread data as the input of the hardware resources and corresponding storage resources of the next stage. The hardware execution unit is further configured to sequentially output the exchange results of the T threads as the execution result of the first instruction through the L selectors and the L hardware resources.

2. The hardware execution unit according to claim 1, characterized in that, The data exchange module includes an exchange enabling subunit, a data collection and distribution subunit, and a data exchange subunit. The switching enable subunit is configured to generate an enable signal in response to the fact that the input data of the N threads that first entered the hardware execution unit has been output from the last hardware resource and cached in the storage resource corresponding to the last hardware resource. The data collection and distribution subunit is configured as follows: In response to the enable signal, the input data of the T threads is collected from the L blocks of storage resources and sent to the data exchange subunit; The system receives the exchange results of the T threads returned by the data exchange subunit, splits the exchange results of the T threads into L exchange result blocks according to the thread number order, and distributes them to the L selectors respectively. Each exchange result block consists of the exchange results of N threads. The data exchange subunit is configured to, in response to the enable signal, exchange the input data of the received T threads according to the instruction parameters of the first instruction, and return the exchange results of the T threads to the data collection and distribution subunit.

3. The hardware execution unit according to claim 2, characterized in that, The data collection and distribution subunit includes a first subunit and a second subunit. The first subunit is coupled to the L block storage resources and is configured to, in response to receiving the enable signal, collect the input data of the T threads in the L block storage resources, sort the input data of the T threads in descending order of thread number, and transmit the input data of the T threads to the data exchange subunit in one go through thread merging logic; The second subunit is configured to receive the exchange results of the T threads returned by the data exchange subunit, and split the exchange results of the T threads into L exchange result blocks in ascending order of thread number with N as the granularity. The L exchange result blocks are then distributed to the L selectors. The exchange result block with the smallest thread number among the L exchange result blocks is distributed to the selector coupled to the output of the last hardware resource.

4. The hardware execution unit according to claim 2, characterized in that, The data exchange subunit includes T T-way selectors, and each of the T T-way selectors corresponds one-to-one with one of the T threads. Each T-way selector has T input ports that respectively receive the input data from the T threads. Each T-way selector selects an input data output according to the instruction parameters of the first instruction, which serves as the exchange result for the thread corresponding to the T-way selector.

5. The hardware execution unit according to claim 1, characterized in that, Each selector is a two-to-one multiplexer. One input port of each of the L selectors is coupled to the output of one of the L hardware resources, and the other input port of each of the L selectors is coupled to the data exchange module. For a selector coupled to the output of the last hardware resource, the output port of the selector is coupled to the output of the hardware execution unit. For the L selectors other than the first selector, the output port of the other selector is coupled to the input port of the next stage hardware resource in the pipeline.

6. The hardware execution unit according to claim 1, characterized in that, The M hardware resources are turned on sequentially over M clock cycles to transmit data from each thread. When each hardware resource outputs data, the output data is cached in the storage resource corresponding to the hardware resource. During the M clock cycles, the hardware execution unit sequentially inputs M sets of input data, each set of input data including input data from N threads. Over the M clock cycles, the hardware execution unit caches the input data from the T threads into the L blocks of storage resources.

7. The hardware execution unit according to claim 2, characterized in that, The switching enable subunit generates the enable signal in the first clock cycle. The data collection and distribution subunit collects the input data of the T threads in the first clock cycle, obtains the exchange results of the T threads, and distributes the exchange results of the T threads to the L selectors. The hardware execution unit outputs the first exchange result block in the first clock cycle.

8. The hardware execution unit according to claim 7, characterized in that, In the L-1 clock cycles following the first clock cycle, the L selectors enable data from the previous stage hardware resources of the pipeline. Of the L exchange result blocks, the other L-1 exchange result blocks, excluding the first exchange result block, are output sequentially during the L-1 clock cycles.

9. The hardware execution unit according to claim 1, characterized in that, The L hardware resources are adjacent in the pipeline, including the last hardware resource. In response to the input data of the N threads that first enter the hardware execution unit in the first clock cycle being output from the selector coupled to the output of the last hardware resource, the exchange results of the T threads are output sequentially starting from the first clock cycle.

10. The hardware execution unit according to claim 2, characterized in that, The data exchange module is implemented using combinational logic.

11. The hardware execution unit according to any one of claims 1-10, characterized in that, Each thread executes its own instructions independently; or, The M hardware resources are time-division multiplexed to execute the instructions of the T threads in parallel.

12. A processor, characterized in that, It includes a plurality of hardware execution units as described in any one of claims 1-11, the plurality of hardware execution units being configured to execute the first instruction in parallel to process different data.

13. An electronic device, characterized in that, Includes the processor as described in claim 12.

Images