Method and device for processing instruction

Abstract

The invention relates to a method and device for processing instructions. A computer-implemented method for processing instructions in a multi-processing device, the method comprising: obtaining a first instruction of a first process; decoding the first instruction to detect a continuation indicator associated with the first instruction; determining whether to enforce the continuation indicator; and when it is determined to enforce the continuation indicator: continuing to execute the first process until the first instruction of the first process and the at least next sequential second instruction are completed. For example, a normal eviction process may continue to be temporarily suppressed based on a fair algorithm.

Description

[0001] The present technology relates to data processing and data processing devices and systems.

[0002] In a data processing system (e.g., a CPU or a graphics processing system (GPU)), multiple software processes (e.g., warps, where a warp may include one or more threads, and where a thread may include a piece of work) that need to access hardware resources on the GPU may need to run substantially simultaneously, where a scheduler is provided to select processes so as to access the hardware resources in a fair manner.

[0003] The present technology relates to process control in such a multiprocessing device. The technology is particularly applicable to multiprocessing devices having a main processor (execution engine) and attached coprocessors (texture mapper, load / store unit, ray tracing unit, variation unit, etc.). In one example of the technology, the main processor assigns tasks to coprocessors by sending messages to the coprocessors on a communication channel such as a message structure, the message including the necessary input data and a description of the task to be performed (opcodes, control data). The coprocessor will respond at a later time with the result of the executed task. In the meantime, the main processor can continue to execute instructions from the same or different processes until it reaches an instruction (an instruction executed by the coprocessor on the message structure) that depends on the result of the preceding message instruction. In this example of the multiprocessing technology, the message is sent by a "message instruction unit". The present technology applies to what happens to the process after the message has been sent - whether and under what circumstances the process is evicted from the execution engine (and the execution engine switches to running instructions from a different process), or whether and under what circumstances the execution of the same process continues with the next instruction.

[0004] According to a first technique, a computer-implemented method for processing instructions in a multiprocessing device is provided, the method comprising: obtaining a first instruction of a first process; decoding the first instruction to detect a continue indicator associated with the first instruction; determining whether to force the continue indicator; and when it is determined that the continue indicator is forced: continuing to execute the first process until the first instruction of the first process and at least a next sequential second instruction are completed.

[0005] According to another technique, a multiprocessing device operable to process instructions is provided, the multiprocessing device comprising: a receiver component for obtaining a first instruction of a first process; a decoder component, the decoder component decoding the first instruction of the first process to detect a continue indicator associated with the first instruction and determine whether to force its continue indicator; and an eviction inhibitor component, the eviction inhibitor component for, when determining to force the continue indicator, inhibiting the eviction of the first process from a message instruction unit to continue processing until the first instruction of the first process and at least a next second sequential instruction are completed.

[0006] According to another technology, a computer program is provided, which includes computer program code, which, when loaded into a processor and executed thereon, causes the processor to perform the method as described above. The computer program can be implemented as a non-transitory data carrier carrying code, which, when implemented on a processor, causes the processor to perform the method described herein.

[0007] In yet another technique, the present invention may be implemented by a computer program operable to adapt a host processing system to provide an execution environment enabling operation of non-native processor instructions to perform the steps of the above-described method.

[0008] These techniques are schematically illustrated by way of example in the accompanying drawings, in which:

[0009] Figure 1A schematically illustrates a simplified block diagram of a multi-processing system according to an embodiment;

[0010] Figure 1B A simplified example of a data processor operable to process warps including threads according to an implementation of the present technology is shown;

[0011] Figure 2 illustratively showing a block diagram of an execution engine of a graphics processing unit;

[0012] Figure 3 illustratively showing a block diagram of an execution engine of a graphics processing unit according to an embodiment;

[0013] Figure 4 shows one possible structure of the instruction fetch stage; and

[0014] Figure 5 Schematically shows the Figure 3 An example flow chart of processing instructions at an execution engine of .

[0015] As described above, in a data processing system, multiple software processes that need to access hardware resources on a GPU may need to run substantially simultaneously, wherein a scheduler is provided to select processes to access the hardware resources, typically using an algorithm to select processes in a fair manner.

[0016] The data processing system may include a CPU and various forms of accelerators, such as, for example, co-processors, graphics processing units, and the like.

[0017] In one example of the interaction of a system in which the present technology may be implemented, components such as a CPU, a display controller, and a GPU may be connected to an interconnect. The CPU executes an application that requires a graphics processing operation to be performed by the GPU. To this end, the application generates an API call that is interpreted by a GPU driver (on the CPU) that is operable to control the GPU. The driver executed on the CPU generates commands, data structures, and instructions to the GPU, where, for example, a shader core may be activated to perform graphics processing operations using an execution engine. In an example implementation, the graphics processing operation may be executed as a bundle of threads or threads in a system of execution channels in the execution engine, where each thread of the bundle is executed by an execution channel.

[0018] The present technology therefore relates to process control in such a multiprocessing arrangement. The technology is particularly applicable to a multiprocessing arrangement having a main processor (execution engine) and attached coprocessors (texture mapper, load / store unit, ray tracing unit, transformation unit, etc.). The main processor can assign tasks to the coprocessor by, for example, sending a message to the coprocessor on a message structure, the message including the necessary input data and a description of the task to be performed (opcodes, control data). In other architectures, other communication techniques may be used to convey the task assignments.

[0019] The coprocessor will respond at a later time with the results of the executed task. Meanwhile, the main processor may continue to execute instructions from the same or a different process until it reaches an instruction that depends on the result of a preceding message instruction (an instruction executed by the coprocessor on the message structure). Messages are sent by a "message instruction unit." The present technique applies to what happens to the process after the message has been sent - whether it is evicted from the execution engine (and the execution engine switches to run instructions from a different process), or execution of the same process continues with the next instruction.

[0020] Figure 1A is a simplified block diagram of a multiprocessing system 1. In the present embodiment, a multiprocessing system (hereinafter referred to as a "data processing system") refers to a data processing system having one or more hardware (HW) units (e.g., functional units) to which two or more software (SW) processes may need to access (i.e., the number of HW units is less than the number of SW processes).

[0021] The data processing system 1 includes a plurality of processing units. Figure 1A1 is depicted as a host processor 2 and a target processor unit, here depicted as a graphics processor unit (GPU) 4 and a neural processing unit (NPU) 6. The data processing system 1 also includes a storage device 8, such as a random access memory, and a storage device 10, such as a solid state drive, hard disk or other non-volatile memory. The elements of the data processing system 1 are connected via an interconnect 12, which can be, for example, a coherent interconnect, a network or a bus.

[0022] Host processor 2 may include, for example, a general-purpose processing core, and is referred to herein as a central processing unit (CPU 2 ).

[0023] GPU 4 executes a graphics processor pipeline that includes one or more processing stages ("shaders"). For example, the graphics processor pipeline executed by GPU 4 may include one or more of the following, and typically all of the following: geometry shaders, vertex shaders, and fragment (pixel) shaders. These shaders are programmable processing stages that execute a shader program on input data to generate a desired set of output data.

[0024] To execute shader programs, GPU 4 includes one or more graphics execution units or execution engines for executing graphics programs (eg, shader programs). GPU 4 may include one or more execution engines.

[0025] The actual data processing operations performed by the execution engine when executing a graphics program may be performed by a corresponding graphics processing or functional unit (circuitry), such as an arithmetic unit (circuitry), in response to and under the control of the instructions in the (shader) program being executed. Thus, for example, an appropriate graphics functional unit such as an arithmetic unit will respond to the instructions in the (shader) program being executed and perform data processing operations as required by those instructions.

[0026] When executing instructions in a graphics program, the execution engine (e.g., using an appropriate graphics functional unit, such as an arithmetic unit) will typically perform processing operations using one or more input data values ​​to generate one or more output data values, and then return the output data values, e.g., for further processing by subsequent instructions in the executed program and / or for output (for a purpose other than during execution of the executed program).

[0027] Input data values ​​to be used when executing instructions will typically be stored in a storage device (e.g., a register file, or perhaps one or more caches accessible to the execution engine), and output data values ​​generated by the graphics functional unit executing the instructions will correspondingly be written back to the appropriate storage device (e.g., register file) for future use. Thus, when an instruction is executed, input data values ​​will be read from the appropriate storage device (e.g., register file) and output values ​​will be written back to the same or different storage device. Typically, the data structure (e.g., input data array, filter, output data array, etc.) used to represent the data to be used for processing is multi-dimensional (e.g., 2D or 3D) image data.

[0028] NPU 6 typically includes one or more neural execution units (hereinafter referred to as "neural engines"), where the neural engine is configured to more efficiently perform one or more specific types of neural network processing operations.

[0029] A neural engine may include one or more neural functional units to perform neural network operations. For example, a neural engine configured to perform tensor arithmetic operations (such as tensor MAC operations) may include multiple neural functional units in the form of multiplier-accumulator circuits ("MAC units") that are arranged to perform such MAC operations on tensor data structures. Typically, data structures used to represent data to be used for neural network processing (e.g., input data arrays, filters, output data arrays, etc.) are multi-dimensional (e.g., 4D+ tensors). Therefore, arithmetic operations typically include tensor arithmetic, such as tensor multiplication, addition, etc.

[0030] The NPU 6 may be coupled to the GPU 4 or Figure 2 4 to provide neural processing capabilities for GPU 4, whereby in an embodiment, each execution engine of the GPU may have its own dedicated neural engine to provide neural network capabilities therefor. Alternatively, each execution engine of the GPU may share the neural engine of NPU 6 with one or more other GPU execution engines, wherein one neural engine provides neural network capabilities for each GPU sharing the neural engine.

[0031] An application 14 (such as a game) executing on the host processor 2 may require graphics processing operations to be performed by the associated graphics processing unit (graphics processing pipeline). To do this, the application will generate API (application programming interface) calls, which are interpreted by the driver 16 running on the host processor 2 to generate appropriate commands for the graphics processor 4 to perform the operations required by the application 14. To facilitate this, a collection of "commands" will be provided to the GPU 4 in response to commands from the application 14 running on the host system 2 for graphics output.

[0032] The program (e.g., shader program) to be executed by GPU 4 will be provided by application 14, which requires graphics processing using, for example, a high-level shader program language (such as GLSL, HLSL, OpenCL, etc.). The program is then translated into binary code for GPU 4 by a compiler 15 (e.g., a shader language compiler) of driver 16. The binary code includes a process with "instructions" specified in the instruction set specification for a given GPU 4. The compilation process for converting a program into binary code instructions can be performed via multiple intermediate representations of the program within the compiler. Therefore, a program written in the high-level language (e.g., shader language) can be translated into an intermediate representation for a specific compiler (and there may be several consecutive intermediate representations within the compiler), where the final intermediate representation is translated into binary code instructions for the target GPU.

[0033] In one example, here Figure 1B As shown, the techniques described herein may be implemented in a data processor 100B (such as a graphics processing unit) that may execute execution threads 102B to execute an instruction set for a program to perform data processing operations on data values, and wherein the threads 102B to execute instructions of the program may be grouped together into thread groups (“warps”) 104B, wherein the execution threads 102B of the warps 104B execute the instructions of the program in a lockstep manner, e.g., such that each execution thread 102B within the warp 104B executes the same instruction in the program (to perform a processing operation) before executing the next instruction in the program (and so on). The data processor 100B (e.g., a graphics processing unit) includes instruction execution processing circuitry 106B configured as a plurality of execution lanes 108B, instruction decoding circuitry 110B, and execution thread issuing circuitry 112B that issues the execution threads 102B of the warps 104B to the execution lanes 108B for execution. As will be apparent to one of ordinary skill in the art, this arrangement is merely one example and one of many possible arrangements for data processing, and the present technology is in no way limited to such an environment.

[0034] Figure 2 A block diagram of execution engine 20 of a graphics processing unit is illustratively shown.

[0035] Execution engine 20 includes one or more hardware units or blocks, and in this illustrative example, execution engine 20 includes a front end control unit 22 (hereinafter referred to as a "front end unit") having a work queue 23 and a scheduler 24, and wherein execution engine 20 is coupled to an instruction cache 26. Work queue 23 may be a first-in, first-out queue, although the claims are not limited in this respect.

[0036] The execution engine 20 also includes a message instruction unit or message block 28 for executing message instructions, and one or more processor units 30a-n for executing arithmetic instructions (eg, fused multiply-add instructions (FMA); conversion (CVT) instructions; special function unit (SFU) instructions).

[0037] It should be understood that the execution engine 20 may include Figure 2 For example, execution engine 20 may include a divergence unit (not shown) that may be used to manage a program counter for branching situations.

[0038] The various hardware units may communicate with each other and with SW processes and / or HW units external to the execution engine 20. For example, the front end unit 22 will receive work (e.g., from a warp manager (not shown)) and schedule and distribute the work in the work queue 23 to the message blocks 28 and processing units 30a-n for execution using the scheduler 24. Although the embodiments below primarily describe the scheduling and processing of message instructions, it should be understood that the scheduler 24 may also schedule other types of instructions (e.g., arithmetic), or a different scheduler may be provided to schedule messages other than message instructions.

[0039] In the present illustrative embodiment, message instructions are instructions that, when executed at message block 28, result in work requests being sent to various target client units (e.g., hardware accelerators such as texture mapper units and transform units or other units such as load / store caches (LSCs)) via the message fabric interconnect. Such work requests may include requests to perform requested operations on the target client units.

[0040] The scheduler 24 may employ a fair selection mechanism to select a process to be run (i.e., the process from which instructions are executed at the message block 28) from the work queue 23, for example, in an unbiased manner, where a typical fair selection mechanism (i.e., without favoring any particular process) ensures that one instruction from each process is executed in turn.

[0041] When a process is selected by the scheduler (24) and posted to a message block (MB), the MB will issue an instruction fetch request for that process to the instruction cache (26). The instruction cache will respond to the MB with an instruction. The received instruction will be executed, and a further continuation decision for the process will be made based on the continuation indicator on the received instruction.

[0042] The L0 cache (32) within the MB will store the instruction fetched from the instruction cache (26) and the type of the next sequential instruction of the fetched instruction. There are other storage elements (relatively small) in the MB that store information about the process / warp, including the warp ID (for identifying the process), the warp type (for fault detection), the discard channel of the warp, the pointer to the instruction in the instruction cache cache line (for issuing instruction fetch requests), etc. This information is reused by the continuation instructions of a particular process until the process is evicted.

[0043] The process is evicted from the MB shortly after the operands of the current instruction are fetched from the register file; that is, the eviction of the process does not depend on the result being written to the register file. It only needs to ensure that the dependencies of the process have been incremented in FE (22). After the eviction, the instruction will continue to create request messages on the message structure interconnect. The result is returned some time later and processed by the MB, and the dependency is decremented.

[0044] When the message block 28 completes execution of the current instruction of the selected process, the result may be written, for example, to a storage device (e.g., a register file) in the processor unit 30a-n, and the process is then "evicted" from the message block 28, whereupon the process is returned to the work queue 23 to be selected by the scheduler according to the unbiased selection mechanism. In this embodiment, eviction may also include deleting state data associated with the process at the message block 28 and / or data in the data storage device (e.g., a data cache) at the message block 28 required for the message block 28 to process the instruction.

[0045] Using an unbiased selection mechanism avoids deadlock and allows all processes in the queue to move forward to completion. However, this unbiased fairness mechanism may lead to inefficient processing. As an example, inefficient processing occurs due to cache thrashing of the client units (e.g., texture mappers) of the MB due to large delays between sending messages for subsequent instructions of the same process from the MB to the client units. This is caused by the fairness mechanism.

[0046] As an illustrative example, cache thrashing may occur at the client unit, in one example case at the texture mapper unit. For example, if a shader has 4 back-to-back texture instructions, the current scheduler will not schedule the instructions one after the other due to the fairness mechanism (one instruction from one process). On the client side, the delay between the messages receiving the 4 texture instructions causes TEX cache thrashing, and the data required to process the TEX instructions in the TEX unit must be fetched 4 times. Embodiments of the present invention solve this problem by attempting to process the 4 instructions back-to-back in the MB, thereby reducing the delay between messages sent to the client for instructions of the same process.

[0047] The present technology is described herein with reference to the use of message blocks. However, as will be apparent to those skilled in the art, the present technology may operate in other architectures that do not have message blocks.

[0048] Figure 3 Illustratively, a block diagram of an execution engine 100 in accordance with the present technology is shown. Similar numbering will be used to describe similar features above.

[0049] The message block 128 of the execution engine 100 includes a decoder component 134 and an eviction inhibitor component 136. The message block 128 may also include an interface to / from the instruction cache 126 to communicate therewith, and further include an interface to / from the front end unit 122 to communicate therewith, and an interface to / from the processor unit.

[0050] In the present embodiment, a modifier indicator or a continuation indicator (e.g., a specified value or flag) is encoded within the instruction received by the front end (e.g., within a binary code instruction). In the present illustrative example, the continuation indicator (CI) comprises a 1-bit value (e.g., "0" or "1"), which, when set, indicates how the execution engine 100 will operate. For example, when the CI is set (e.g., "1"), the execution engine 100 may operate in a different manner than when the CI is not set ("0"). It should be understood that the CI may not be limited to a 1-bit value, and may take any form.

[0051] The CI may be set by the compiler, where the compiler may use various heuristics to determine whether to set the CI. Additionally or alternatively, the value of the CI may be defined in the high-level shader program itself and translated by the compiler, where the programmer may define the CI by direct intervention (e.g., for safety-critical code or library code).

[0052] Typically, a heuristic will need to predict whether the next instruction is likely to hit the same line in the cache as the first instruction. An example in a GPU is a texture cache that holds image data. Typically, a cache line will hold a rectangular area of ​​pixels, such as 4x4. If a program does multiple texture reads from the same image / texture, it has often been seen that these reads are close together on the image and will likely hit the same cache line. Therefore, one heuristic is to set the CI for consecutive texture instructions that read from the same texture. On the other hand, if they read from different textures, the CI should not be set. Another example would be for consecutive memory loads, where the compiler can infer that the memory addresses are close together, such as is the case for MEMCPY, for example.

[0053] Upon receiving an instruction in which the CI is set (ie, the instruction does have an associated CI), the execution engine 100 may operate differently than it would otherwise if the CI was not set (ie, the instruction did not have an associated CI).

[0054] In some embodiments, the CI is deterministic, and the execution engine 100 always operates according to the value of the CI. In other embodiments, the CI is indicative or informational, and the execution engine 100 may operate according to the value of the CI or may choose to ignore the value of the CI, as will be described in detail below.

[0055] On the slave host processor ( Figure 3 When a driver on the CPU 120 (not shown) receives a workload including code, the code including multiple processes is stored in the work queue 123.

[0056] Scheduler 124 determines whether the next process in work queue 123 includes instructions that need to be issued to message block 128 (e.g., message instructions that may require data from an external client unit) or instructions that need to be issued to processing unit 130a-130n for internal processing (e.g., arithmetic instructions that do not execute external retrieval).

[0057] When the scheduler 124 selects a process including a message instruction, the scheduler 124 issues or dispatches the selected process to the message block 128 .

[0058] State information and dependency information associated with a process (e.g., if an instruction depends on the result of another instruction) may be stored in a process information storage device 125, which may be updated by, for example, a message block 128 or a scheduler 124 or any other appropriate unit. Each instruction in the instruction stream creates a dependency for a warp when it is executed. This is accomplished by incrementing the dependency slots assigned to the process. Each instruction executed by a warp will use a specific slot (e.g., TEX_FETCH).

[0059] For example, one or more dependency counters may be maintained in the process information storage 125, where each instruction executed in the message block 128 creates a dependency for the process (e.g., where a first instruction of a process may depend on the result of a second instruction of the process before it can complete). Dependencies may be tracked by dependency counters, where the dependency counters are incremented upon execution of instructions and decremented when client units (texture mappers, mutation units, LSC units) return results.

[0060] The instruction acquisition component in message block 128 ( Figure 3126, and receives the current instruction of the process to be executed and stores the instruction in the local storage device 132. The message block 128 also receives additional information with the current instruction from the instruction cache 126, for example during the fetch operation, which will provide details about at least the next instruction in the process. This additional information may indicate or can be used to determine, for example, the type of the next instruction or the next two or more instructions; whether the instruction link (pointer to the instruction in the cache line) of the instruction has reached the end of the cache line; whether the termination condition is decoded in the current instruction (this information is decoded from the instruction itself after receiving the fetch response from the IC in the MB); or whether the front-end dependency condition is about to overflow; and the priority of the instruction. For dependency conditions, the current value of the dependency counter is received at the time of the thread warp issuance from the FE to the MB. This is incremented internally each time a valid continue instruction is fetched. When it reaches its maximum value (a fixed dependency resource available for each process), we stop continuing.

[0061] In another example, the additional information could be the type of the next instruction. For example, if there are three different pipelines in the MB to handle LSC, VAR, and TEX instructions, we treat these as three different instruction types. The additional information would indicate which pipeline the next instruction is intended for. If it is different from the pipeline the current instruction is executing on, the continue indicator will be ignored. An instruction link (a pointer to the instruction in the cache line) is received when a warp is issued from the FE to the MB. This is incremented internally each time a valid continue instruction is fetched. In this case, the continue is stopped when the maximum value (end of cache line) is reached.

[0062] A program counter (not shown) may also be updated by one or more units to track the progress of instructions of a program as they are fetched.

[0063] Decoder component 134 (a pre-decode unit within the instruction fetch stage of the MB) decodes the current instruction to determine whether a continue indicator is associated with the instruction. Additional information may be provided by pre-decoding or may be required for decoding.

[0064] When a continuation indicator is not associated with an instruction (i.e., when there is no continuation indicator or a value is set to indicate that the process should be evicted from the message block when the instruction completes), the message block will evict the process after execution of the instruction is completed, and the process is evicted from the message block 128, whereupon the process is returned to the work queue 123 for future selection by the scheduler 124. As described above, eviction may also include deleting state data related to the process at the message block 128 and / or data in the data storage at the message block 128 required for the message block 128 to process the instruction. Eviction may also include updating the program counter at the front end unit 122.

[0065] When a continuation indicator is associated with an instruction (i.e., when the value is set to indicate that the process should not be evicted from the message block when the instruction completes), the eviction inhibitor component 136 can inhibit the eviction of the process from the message block after execution of the instruction is completed (i.e., without the process returning to the work queue 123). If eviction is inhibited in this manner, the message block 128 will decode at least the next sequential instruction of the process.

[0066] When a continue indicator is not associated with the next sequential instruction, the message block 128 will evict the process after the next sequential instruction completes.

[0067] When the continue indicator is associated with the next sequential instruction, the eviction inhibitor component 136 may inhibit the eviction of the process from the message block 128 after the next sequential instruction is completed, and the message block may continue to execute the next instruction from the same process. Similar functionality may be applied to further sequential instructions of the process with the continue indicator, where the eviction of the process will be inhibited until the eviction point is reached.

[0068] The eviction point may be reached when the current instruction being executed does not have an associated continue indicator.

[0069] As described above, the continuation indicator may simply indicate how the message block should be used, so that even when the continuation indicator is associated with a particular instruction, the message block 128 may evict the process after the instruction is completed based on various factors. For example, an eviction point may be met when the current instruction does have an associated continuation indicator but the instruction link reaches the end of the cache line (the instruction link is actually a pointer to the instruction in the FE cache line. This is received on the thread warp issue from the FE to the MB. Each time a valid continue instruction is fetched and decoded from the IC, the value of the link will be incremented in the MB. Therefore, when the maximum number of instructions in the IC cache line has been executed, the continuation is stopped). In further examples, the eviction point may be reached where the current instruction and the next instruction are of different types (e.g., to be sent to different client units (e.g., LSC unit vs. VAR unit)); where the termination condition is decoded in the current instruction; or where the dependency condition is not met. An example dependency condition may be one where each process is allocated a limited resource in the message block 128, so when a particular process is determined to exceed a threshold for that resource, then any instructions of that particular process that have an associated continue indicator will be evicted after being processed, even if those instructions have an associated continue indicator. As an illustrative example, the front end unit may send a dependency count to the message block, where the dependency count may be used to calculate the minimum headroom available in the dependency counter in the storage device 125 for the process to issue to the message block 128. If it is determined that the headroom is / will be exhausted when fetching / executing instructions for that process, then the eviction point will be reached (i.e., the continue indicator will not be enforced for that process).

[0070] In an embodiment, respecting the continue indicator (i.e., suppressing the eviction of the process even after the current instruction completes) may cause performance problems, and message block 128 may ignore continue indicators in certain instructions that are known to cause such performance problems (e.g., failures in hardware such as stalls, crashes). Additionally or alternatively, the operation of the compiler may be modified to avoid including continue indicators in certain types of instructions, thereby avoiding any performance problems.

[0071] In addition, the compiler can access information about which application is running and can use specific heuristics to determine whether to include a continue indicator in one or more instructions while the application is running. In addition, depending on the program being compiled, there may be one or more instances of the compiler with different operating modes. This functionality can provide customized compilation based on the program being run.

[0072] The message block 128 may also determine whether a dependency overflow is satisfied, and if the dependency overflow condition for the process is satisfied or the dependency counter has not reached a defined limit, the continuation indicator of the current instruction of the process may be simply continued to be followed. For example, a dependency counter (not shown) may be maintained in the front end 122 (e.g., in the process information storage device 125), wherein each time a process is issued to the message block 128 and a request is issued to the client unit as a result of the current instruction being executed, the dependency counter is incremented after all operand acquisitions are completed for the instruction, and the dependency counter is decremented when the instruction for the process returns the required result.

[0073] Furthermore, the additional information may indicate a priority of the instruction, wherein when the continuation indicator indicates how the message block should execute the instruction, the message block may enforce the continuation indicator for instructions having a higher priority more frequently than instructions having a lower priority.

[0074] In these implementations, a balance may be sought between the benefits of adhering to the continuation indicator and the benefits of preserving a fair algorithm. For example, the decision process may be implemented using a trained machine learning and reasoning engine that is capable of comparing weights associated with proposed processing paths to determine the best possible outcome.

[0075] A compiler on a host processor (not shown) may use heuristics when determining whether to modify an instruction to add a continue indicator thereto. In addition to, or as an alternative to, using heuristics, the compiler may add a continue indicator based on characteristics of the instruction itself. For example, when a first instruction of a process and a next sequential instruction of the process will not be read from the same memory area (e.g., the same image), then the compiler will not set the continue indicator for the first instruction.

[0076] As described above, the execution engine 100 may be configured to treat the continuation indicator as deterministic (where the execution engine 100 always acts according to the value of the continuation indicator), or the execution engine 100 may be configured to treat the continuation indicator as informational or indicative (where the execution engine 100 may determine whether to act according to the value of the continuation indicator). Even when the execution engine 100 is configured to treat the continuation indicator as informational / indicative, then in some cases the message block 128 may evict an instruction after completion despite the instruction having a continuation indicator associated therewith.

[0077] It should be understood that the message block 128 may conceptually include one or more processing pipelines, where each pipeline will process instructions of different types of processes issued to the message block 128. For example, a first pipeline may process texture instructions that result in work requests to a texture mapper unit (texture mapper pipeline), a second pipeline may process LSC instructions that result in work requests to an LSC unit (LSC pipeline); and a third pipeline may process variation instructions that result in work requests to a variation unit (variation pipeline).

[0078] Each pipeline may include one or more processing stages to process instructions.

[0079] As an illustrative example, the acquisition phase may include components for receiving a process issued by a front end and acquiring its instructions.

[0080] Figure 4 A possible structure of the message block instruction acquisition stage 40 is shown. Figure 4 In the example of FIG. 2 , the warps from the FE arrive at the input buffer 204, after which the warp state entities 206a, 206b and the warp control entities 208a, 208b feed into the warp selection 209. The warp selection is then passed to the instruction extractor 210 to fetch the instruction. In one example of the present technique, the instruction pre-decoder 212 then intercepts the incoming instruction and determines whether there is a continue indicator before passing the relevant information to the decode stage 218 via the output buffer 214.

[0081] The decode stage 218 may determine the operation to be performed, while in another “operand fetch stage”, operand data for the decoded instruction is fetched, for example, from one or more of the processing units 130a-n, and may be stored in a storage device at the message block (e.g., in a data cache at the message block).

[0082] In another stage (not shown), the work request may be sent via the message fabric interconnect to one or more client units to undertake the requested work according to the instructions.

[0083] As the client unit performs the requested work, results returned from the client unit are processed in a "write-back phase" where the result data is written to the appropriate memory (e.g., one or more caches and / or register files) according to the instructions.

[0084] In an embodiment, there may be a latency (e.g., multi-cycle latency) for an instruction response from instruction cache 126 following a fetch request from message block 128. Thus, fetching and executing instructions of the same process back-to-back in message block 128 may result in "bubbles" or delays or latencies in the pipeline for executing instructions.

[0085] The present technique alleviates latency by providing an acquisition phase in which two or more processes can proceed simultaneously.

[0086] therefore, Figure 4 An illustrative example of an acquisition phase 40 is shown, which may handle two or more processes substantially simultaneously.

[0087] The fetch stage 40 described above allows a current instruction and a next sequential instruction of the same process to be fetched (e.g., from storage at the front end (e.g., an instruction cache)), wherein a determination may be made as to whether the current instruction has a continue indicator associated with it, and an eviction scenario may be identified that prevents the continue indicator from being honored (e.g., if the current instruction and the next instruction are of different types).

[0088] The acquisition phase 40 also allows identifying eviction scenarios, for example based on additional information received from the front end.

[0089] The fetch stage 40 also allows multiple different processes to be ongoing in a message block at any one time, thereby reducing the impact of latency when fetching instructions from the front end.

[0090] Figure 5 Schematically shows the Figure 3 An example flow chart 400 of processing instructions at an execution engine of .

[0091] At S402 , the flowchart 400 starts.

[0092] At S404, a compiler (e.g., a shader language compiler) translates the high-level program to provide the execution engine with binary code for a process including "instructions", where the instructions are specified in an instruction set specification for a GPU including the execution engine. The compiler may encode a continuation indicator into one or more of the instructions based on or in response to one or more heuristics and / or in response to one or more characteristics of the instructions or application running on the host processor.

[0093] At S406 , the process is added to a work queue on a front-end unit of the execution engine, wherein the work queue includes all processes to be executed by the execution engine.

[0094] At S408, the scheduler in the front end selects the process to be issued to the message block and the processing unit. The selection of the process may be based on a first-in-first-out order, where the work queue may be a FIFO queue.

[0095] At S410, the message block accepts the process published from the front end.

[0096] At S412, the message block checks whether the current instruction of the process is in the local storage device (e.g., L0 cache), and when the current instruction of the process is not available in the local storage device, then at S414, the message block issues an instruction acquisition request to the front end (e.g., to the instruction cache at the front end).

[0097] When the instruction is available at S412, the decoder component of the message block (eg, depicted as Figure 4 The pre-decoder component in the decoder decodes the fetched instruction, and at S416, detects whether the current instruction has a continue indicator associated with it.

[0098] When there is a continuation indicator associated with the current instruction, at S418, the inhibitor component determines whether to enforce or comply with the continuation indicator. This determination can be based on additional information received, for example, during the fetch operation, and can include, for example, a determination of whether the current instruction and the next instruction are of different types (e.g., to be executed by different pipelines of the message block (e.g., a variation instruction vs. an LSC instruction); where a termination condition is decoded in the current instruction, or where a front-end dependency condition is about to overflow; or where the next instruction is not available in the local memory; and the priority of the instruction.

[0099] When the inhibitor component enforces the continuation indicator at S418, at S420, the message block executes the instruction and updates the state information and / or dependency information, and processes the next sequential instruction of the process. The next sequential instruction (and any other instruction) is processed in the same manner as at S412 to S414 until the eviction point of the instruction is reached. The MB can also communicate with the front-end unit to prevent the scheduler from issuing any further processes to the message block when the continuation indicator of the current process is followed. If the MB is fully loaded, the MB instruction acquisition stage will push the thread warp release interface back to the scheduler. The thread warp state registers 206a and 206b will not be de-allocated until the continuation stops due to reaching the eviction point. The new thread warp will only be accepted when the current thread warp is de-allocated. When the inhibitor component does not enforce the continuation indicator at S418, at S422, the current instruction is executed, and when completed, the process is expelled back to the work queue. The inhibitor component can also communicate with the front-end unit to instruct the scheduler to issue the next process to the message block.

[0100] When the current instruction does have an associated continuation indicator at S416 , the current instruction is executed at S422 , and upon completion, the process is evicted back to the work queue.

[0101] At S424, the flowchart ends.

[0102] The present technique provides a biased execution mechanism where instructions of the same process are executed and the process is retained in the message block instead of being evicted after each instruction is completed.

[0103] This functionality means that data required by instructions of the same process is available to at least one subsequent instruction as it is executed, rather than having to be fetched each time (e.g., from a cache). This avoids data cache thrashing, and reduces energy consumption and improves overall execution speed.

[0104] This functionality also means that the message block can determine whether to execute consecutive instructions of a process independently of the front end, rather than having to evict the process after executing each individual instruction of the process.

[0105] The functions of the various elements shown in the drawings (including any functional elements marked as "blocks", "components", "modules" or "processors") can be provided by using dedicated hardware and hardware capable of executing software in association with appropriate software. When provided by a processor, the function can be provided by a single dedicated processor, by a single shared processor, or by multiple separate processors that can be shared by some of the processors. In addition, the explicit use of the term "processor" or "controller" should not be interpreted as exclusively referring to hardware capable of executing software, but may implicitly include, but is not limited to, digital signal processor (DSP) hardware, network processors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), read-only memories (ROMs) for storing software, random access memories (RAMs), and non-volatile storage devices. Other hardware that is conventional and / or customized may also be included.

[0106] Software modules or simply modules implied as software may be represented herein as any combination of flow chart elements or other elements indicating process steps and / or execution of the text description. Such modules may be executed by hardware shown explicitly or implicitly.

[0107] Reference throughout this document to "one embodiment," "certain embodiments," "embodiments," "implementations," "aspects," or similar terms means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, the appearances of such phrases in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments without limitation.

[0108] As used herein, the term "or" will be interpreted as an inclusive or meaning any one or any combination. Thus, "A, B, or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B, and C." An exception to this definition will only occur if a combination of elements, functions, steps, or actions are inherently mutually exclusive in some way.

[0109] As used herein, the term "configured to" when applied to an element means that the element can be designed or constructed to perform a specified function, or has a desired structure that enables it to be reconfigured or adapted to perform that function.

[0110] Many details are set forth to provide an understanding of the embodiments described herein. The embodiments may be practiced without these details. In other cases, well-known methods, procedures, and components are not described in detail to avoid obscuring the embodiments. The present disclosure should not be construed as being limited to the scope of the embodiments described herein.

[0111] Those skilled in the art will recognize that the present disclosure has been described by way of example. The present disclosure may be implemented using hardware component equivalents such as dedicated hardware and / or dedicated processors, which are equivalents of the present disclosure as described and claimed.

[0112] These technologies also provide processor control code to implement the above-mentioned systems and methods, for example, on a general-purpose computer system or a digital signal processor (DSP). These technologies also provide a carrier carrying processor control code to implement any of the above-mentioned methods at runtime, particularly on a non-transient data carrier such as a disk, a microprocessor, a CD or DVD-ROM, a programming memory such as a read-only memory (firmware), or on a data carrier such as an optical or electrical signal carrier. The code may be provided on a carrier such as a disk, a microprocessor, a CD or DVD-ROM, a programming memory such as a non-volatile memory (e.g., flash memory) or a read-only memory (firmware). The code (and / or data) used to implement the embodiments of the present technology may include source code, object code or executable code in a conventional programming language (interpreted or compiled) such as C, or assembly code, code for setting or controlling an ASIC (application-specific integrated circuit) or FPGA (field programmable gate array), or for a hardware description language such as Verilog TM , VHDL (Very High Speed ​​Integrated Circuit Hardware Description Language) or SystemVerilog hardware description and hardware verification language. As will be appreciated by a skilled person, such code and / or data may be distributed among a plurality of coupled components that communicate with each other. These techniques may include a controller including a microprocessor, a working memory, and a program memory coupled to one or more components of the system.

[0113] The above-discussed technique can be implemented in a data processing device, and the data processing device has a hardware circuit provided for realizing an instruction decoder and the expulsion control mechanism discussed above. However, the same technique can also be implemented in a computer program, and the computer program is executed on a host data processing device to provide an instruction execution environment for executing a target code. Such a computer program can control the host data processing device to simulate the architecture environment to be provided on the target data processing device, and the target data processing device actually supports the target code according to a specific ISA, even if the host data processing device itself does not support the architecture. For example, when executing the legacy code written for a kind of ISA on a host processor supporting different ISAs, this simulator is useful. In addition, simulation can allow the software development for the newer version of ISA to be started before the processing hardware supporting the new architecture version is ready, because the execution of software on the simulated execution environment can make software testing and the ongoing development of the hardware device supporting the new architecture be carried out in parallel. The simulator can be stored on a storage medium, and the storage medium can be a non-transient storage medium.

[0114] Thus, the computer program may include instruction decoding program logic that decodes program instructions of the target code to control the host data processing device to perform data processing in response to the non-native program instructions (e.g., mapping each instruction in the target code to a series of one or more instructions in the host's native instruction set that implement an equivalent function). In addition, the computer program may have register emulation program logic that maintains data structures in a host storage device of the host data processing device (e.g., a register or memory of the host) to emulate the register storage device of the target ISA being emulated (the register storage device was originally expected to be provided in hardware in a processor that actually supports the target ISA).

[0115] The various representative embodiments described in detail herein are given by way of example and not limitation. It should be understood by those skilled in the art that various changes can be made to the form and details of the embodiments described to obtain equivalent embodiments that remain within the scope of the appended claims.

Claims

1. A computer-implemented method for processing instructions in a multiprocessing device, the method comprising: Obtaining a first instruction of a first process; decoding the first instruction to detect a continue indicator associated with the first instruction; determining whether to enforce the continuation indicator; When it is determined that the continuation indicator is enforced: The first process continues to be executed until the first instruction and at least the next sequential second instruction of the first process are completed.

2. The method of claim 1, wherein continuing comprises suppressing an algorithmically predetermined eviction.

3. The method according to claim 1 or claim 2, comprising: decoding the second instruction of the first process with a decoder component to detect a continuation indicator associated with the second instruction; determining, with the decoder component, whether to enforce the continue indicator of the second instruction; as well as When it is determined to enforce the continuation indicator of the second instruction, eviction of the first process from the instruction unit is suppressed in response to the detected continuation indicator until the second instruction and at least a next sequential third instruction of the process are completed.

4. The method according to claim 3, further comprising: receiving, at the instruction unit, the first process from a control unit; The first instruction for the first process is fetched with a decoder component from the control unit.

5. The method according to claim 4, further comprising: Additional information associated with the first instruction is obtained from the control unit, wherein the additional information includes one or more of the following: the location of the first instruction in the cache; the type of the current instruction and the type of the next sequential second instruction; a termination condition; a dependency condition; and a priority of the first instruction.

6. The method according to claim 5, comprising: An eviction point for the first instruction is identified based on or in response to the additional information associated with the first instruction.

7. The method according to claim 6, further comprising: In response to the identified eviction point, the first process is evicted from the instruction unit upon completion of the first instruction.

8. A method according to any preceding claim, further comprising: storing the instructions of the first process and the instructions of the second process in a storage device at a first pipeline stage; At the second pipeline stage, the instructions of the first process and the instructions of the second process are issued in an interleaved instruction stream.

9. A method according to any preceding claim, wherein the determining detects an "always continue" indication and always continues to execute the first process until the first instruction and at least the next sequential second instruction of the first process are completed.

10. A method according to any preceding claim, further comprising: storing instructions of the first process in a first storage device at a first pipeline stage; storing instructions of a second process in a second storage device at the first pipeline stage; Selecting an instruction from the first storage device and / or the second storage device; To the second pipeline stage, the selected instructions are issued as an instruction stream for processing.

11. The method according to any one of claims 6 to 10, wherein evicting the first process from the instruction unit upon completion of the first instruction comprises: The first process is returned to the work queue at the control unit for selection by a scheduler.

12. The method according to any one of claims 3 to 11, further comprising: A work request is issued from the instruction unit to a client unit to execute the work request according to the first instruction.

13. A method according to any preceding claim, wherein when it is determined that the continuation indicator is not to be enforced: The first process is evicted from the instruction unit upon completion of the first instruction.

14. The method according to any one of claims 1 to 12, wherein when no continuation indicator is detected: The first process is evicted from the instruction unit upon completion of the first instruction.

15. The method according to claim 13 or 14, comprising: When the first process is evicted, another process is received from the control unit.

16. A method according to any preceding claim, wherein the continuation indicator is encoded in the instruction.

17. A method according to any preceding claim, wherein at least one of the first process and the second process comprises a warp.

18. A multiprocessing apparatus operable to process instructions, the multiprocessing apparatus comprising: a receiver component, the receiver component being configured to obtain a first instruction of a first process; a decoder component that decodes a first instruction of a first process to detect a continue indicator associated with the first instruction and determines whether to force the continue indicator thereof; An eviction inhibitor component is configured to, when it is determined that the continue indicator is enforced, inhibit eviction of the first process from the instruction unit to continue processing until the first instruction and at least a next second sequential instruction of the first process are completed.

19. A computer program comprising computer program code which, when loaded into a processor and executed thereon, causes the processor to perform the method according to any one of claims 1 to 17.

20. A computer program operable to adapt a host processing system to provide an execution environment enabling operation of non-native processor instructions to perform a method according to any one of claims 1 to 17.

Images