Apparatus and method for parallel computing work offloading

Abstract

To provide an apparatus and method for offloading parallel computation tasks.SOLUTION: The parallel computation tasks insert requests to execute multiple parallel thread groups 13 into at least one or more parallel thread group queues. When preset order of priority exists, the requests to execute is inserted into the at least one parallel thread group queues corresponding to the order of priority according to the preset order of priority. Parallel threads 12 of the parallel thread groups are executed using a parallel thread group execution request entry extracted from the parallel thread group queue according to the order of priority. When the execution of the parallel threads according to an execution sequence scheduled in an execution startup routine code is terminated, an execution result is inserted into an execution result queue, and the execution termination state of the parallel thread groups is checked by checking the execution result queue.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to parallel computing task techniques, and more particularly to parallel computing task offloading techniques. [Background technology]

[0002] The central processing unit (CPU), the central processing unit of a computing device, has evolved into a multicore or many-core processor to meet the ever-increasing demand for high-performance computing. However, simply increasing the number of CPU cores is not enough to meet the demands of high-performance computing. Computing systems have evolved to include one or more processors specialized for various types of parallel processing, such as the graphic processing unit (GPU), field-programmable gate array (FPGA), digital signal processor (DSP), neural processing unit (NPU), and accelerated processing unit (APU). Such processors specialized for parallel processing are generally called accelerators. Accelerators offload highly parallel operations from the CPU for faster processing. While the CPU and accelerator were previously built on separate system-on-chip (SoC), heterogeneous many-core processors have emerged, integrating the CPU core and accelerator core into a single SoC. Such heterogeneous architecture combination reduces communication costs by reducing the physical distance and increasing bandwidth between the host core and the heterogeneous acceleration core, and enables global memory sharing, thereby reducing the offloading overhead of parallel computing tasks and achieving high performance.

[0003] To offload parallel computing tasks to such accelerators, device-specific private programming models such as CUDA (Compute Unified Device Architecture) and HIP (Heterogeneous-Compute Interface for Portability) and industry-wide open standard programming models such as OpenCL, OpenMP, and OpenACC are used.

[0004] Among these, Open Computing Language (OpenCL), a widely used industry-standard programming model, allows users to create highly compatible programs that can run on various accelerator platforms. Furthermore, OpenCL programs can be executed on CPUs and provide a method for executing a large number of data-parallel threads that can accelerate user calculations on various accelerator platforms, such as GPUs and FPGAs. OpenCL is a programming model similar to C, and supports the creation of parallel computing kernel programs that can be executed on accelerators. It also supports the creation of host programs by providing the OpenCL API through the OpenCL runtime. OpenCL programs created in this way are compiled into host and accelerator executable code by a compiler, and can be executed on both the host and accelerator with the help of the OpenCL runtime library and accelerator driver.

[0005] CPUs and accelerators are designed with various hardware architectures. For example, recent CPUs generally have a simultaneous multi-threading (SMT) design pattern, while accelerators may have hardware design patterns that support various data-level parallelism or thread-level parallelism, such as single instruction, multiple data (SIMD), single instruction, multiple threads (SIMT), and multiple instructions, multiple threads (MIMD). MIMD has a large number of scalar cores or multi-threaded cores and can flexibly map data-parallel tasks to these cores, but has the disadvantage of lower data-level parallelism efficiency compared to SIMD. Conversely, SIMD excels in data-level parallelism but is weak in irregular data parallelism. SIMT is a design pattern with intermediate characteristics between SIMD and MIMD, and has been applied to Nvidia GPGPUs to demonstrate its effectiveness as a method for offering an alternative between programmability and computational efficiency.

[0006] As described above, accelerators may have various types of cores and control structures according to the architecture design pattern, and may also have various memory hierarchy structures. Therefore, a method for effectively executing parallel computing tasks (e.g., OpenCL kernels) suitable for various types of accelerator hardware designs is required.

[0007] Nvidia and AMD GPUs are equipped with various levels of hardware schedulers to efficiently execute and manage the parallel computing tasks. Nvidia provides a thread group scheduler configured as hardware, and also includes a Warp Scheduler that schedules threads of thread groups scheduled to each Streaming Multiprocessor (corresponding to an OpenCL compute unit) in SIMD computation units called Warps.

[0008] Meanwhile, Korean Patent Publication No. 10-2018-0076051, "Method for processing OpenCL kernel and computer device for performing the same," discloses a hierarchical control core structure for performing parallel computing tasks on many-core processors.

[0009] However, Korean Patent Publication No. 10-2018-0076051 discloses a scheduling mechanism using dedicated HW, but the accelerator HW structure and the resulting control structure are complicated, making it difficult to flexibly develop and apply new scheduling methods. Summary of the Invention [Problem to be solved by the invention]

[0010] The present invention aims to quickly execute parallel computing tasks in a heterogeneous core processor system having heterogeneous acceleration cores without a hardware scheduler.

[0011] Another object of the present invention is to provide a hardware and software execution support structure and flow that can be effectively implemented in a large-scale parallel processing environment.

[0012] Another object of the present invention is to simultaneously provide the execution performance of a large-scale parallel processing arithmetic unit and the flexibility to handle a variety of workloads.

[0013] Another object of the present invention is to prevent low priority scheduling items from falling into an infinite starvation state without SW intervention. [Means for solving the problem]

[0014] In accordance with one embodiment of the present invention, a parallel computing task offloading device includes one or more processors and a memory for storing at least one program executed by the one or more processors, wherein the at least one program inserts execution requests of a plurality of parallel thread groups into at least one parallel thread group queue, and if a pre-set priority exists, the execution requests are inserted into the at least one parallel thread group queue corresponding to the pre-set priority. The parallel threads of the parallel thread groups are executed using parallel thread group execution request entries extracted from the parallel thread group queues according to the priority. When the execution of the parallel threads is completed, the device inserts execution results into an execution result queue, checks the execution results reported in the execution result queue to confirm an execution completion state of the parallel thread group, and executes the parallel threads of the parallel thread groups corresponding to the execution completion state.

[0015] In this case, the at least one program may use a programmable timer in a parallel thread group queue corresponding to the priority to find a parallel thread group execution request that has not been scheduled for a previously set time.

[0016] In this case, when the at least one program finds a parallel thread group execution request that has not been scheduled for the previously set time, the at least one program may move the parallel thread group execution request that has not been scheduled for the previously set time to the last execution request entry of the parallel thread group queue with the next highest priority.

[0017] In this case, the at least one program executes an execution startup routine code for each parallel thread of the parallel thread group, loads information necessary for executing the parallel operation kernel code from the execution state information into a register of the acceleration core, and executes the parallel operation kernel code.

[0018] In this case, the execution state information may include common state information for identifying the parallel thread group, and individual parallel thread state information for identifying the parallel threads included in the parallel thread group.

[0019] In this case, the at least one program may use a thread switching logic to switch the context block of the stopped parallel thread to a scratch memory if the total number of parallel threads in the parallel thread group is greater than the number of hardware threads included in the acceleration core group.

[0020] In this case, the at least one program may have a representative parallel thread preset among the parallel threads included in the parallel thread group insert the parallel thread group execution result into the execution result queue.

[0021] In this case, the at least one program may include executing a first parallel thread group of a plurality of parallel thread groups on one acceleration core group.

[0022] In this case, the at least one program can read the value of the IDLE state register of the acceleration core group and, when it confirms that the acceleration core group is in the IDLE state, execute all parallel threads included in any one of the first parallel thread groups.

[0023] In this case, the at least one program may change the value of the IDLE status register from IDLE to BUSY when all parallel threads included in any one of the first parallel thread groups are executed, and may change the value of the IDLE status register from BUSY to IDLE when execution of all the parallel threads is completed.

[0024] In order to achieve the above object, a parallel computing work offloading method according to one embodiment of the present invention is a parallel computing work offloading method for a parallel computing work offloading device, the method including: inserting execution requests of a plurality of parallel thread groups into one of at least one parallel thread group queue; if a pre-set priority exists, the execution requests are inserted into the at least one parallel thread group queue corresponding to the pre-set priority; executing parallel threads of the parallel thread groups using parallel thread group execution request entries extracted from the parallel thread group queues according to the priority; inserting execution results into an execution result queue when execution of the parallel threads is completed; checking the execution results reported in the execution result queue to confirm an execution completion state of the parallel thread group; and executing the parallel threads of the parallel thread groups corresponding to the execution completion state.

[0025] In this case, the executing step may use a programmable timer in a parallel thread group queue corresponding to the priority to find a parallel thread group execution request that has not been scheduled for a set time.

[0026] In this case, if the executing step finds a parallel thread group execution request that has not been scheduled for the previously set time, the executing step may move the parallel thread group execution request that has not been scheduled for the previously set time to the last execution request entry of the parallel thread group queue with the next highest priority.

[0027] In this case, the executing step may execute an execution startup routine code for each parallel thread of the parallel thread group, preload information required for executing the parallel operation kernel code from execution state information into a register of the acceleration core, and execute the parallel operation kernel code.

[0028] In this case, the execution state information may include common state information for identifying the parallel thread group, and individual parallel thread state information for identifying the parallel threads included in the parallel thread group.

[0029] In this case, the executing step may use a thread switching logic to switch the context block of the stopped parallel thread to a scratch memory if the total number of parallel threads in the parallel thread group is greater than the number of hardware threads included in the acceleration core group.

[0030] In this case, the inserting step may involve a representative parallel thread, which is pre-set among the parallel threads included in the parallel thread group, inserting the parallel thread group execution result into the execution result queue.

[0031] In this case, the parallel computing work offloading method may include a step of executing a first parallel thread group of a plurality of parallel thread groups on one of the acceleration core groups before the step of inserting the first parallel thread group into one of the parallel thread group queues.

[0032] In this case, the step of executing on any one of the acceleration core groups may read the value of the IDLE state register of the acceleration core group, and if it is confirmed that the acceleration core group is in an IDLE state, all parallel threads included in any one of the first parallel thread groups may be executed.

[0033] In this case, the step of executing in any one of the acceleration core groups may change the value of the IDLE status register from IDLE to BUSY when all parallel threads included in any one of the first parallel thread groups are executed, and change the value of the IDLE status register from BUSY to IDLE when execution of all the parallel threads is completed. [Effects of the Invention]

[0034] The present invention allows parallel computing tasks to be quickly executed in a heterogeneous core processor system having heterogeneous acceleration cores without a hardware scheduler.

[0035] Furthermore, the present invention can provide a hardware and software execution support structure and flow that can be effectively implemented in a large-scale parallel processing environment.

[0036] Furthermore, the present invention can simultaneously provide the execution performance of a large-scale parallel processing unit and the flexibility to handle a variety of workloads.

[0037] The present invention also prevents low priority items from entering into an infinite starvation state without SW intervention. [Brief explanation of the drawings]

[0038] [Figure 1] FIG. 1 is a diagram illustrating parallel computing operations according to one embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram illustrating a parallel computing task offloading device according to one embodiment of the present invention. [Figure 3] FIG. 3 is a diagram illustrating a process of executing one parallel thread group between a device driver of a host core and an acceleration core group (ACG) according to an embodiment of the present invention. [Figure 4] FIG. 4 illustrates a process of sequential parallel thread group scheduling between a device driver and an ACG of a host core according to an embodiment of the present invention. [Figure 5] FIG. 5 illustrates a process of scheduling multiple parallel thread groups with priority applied between a device driver and an ACG of a host core according to an embodiment of the present invention. [Figure 6] FIG. 6 illustrates a pipeline of parallel thread group execution requests and execution result reports between a device driver and an execution startup routine according to one embodiment of the present invention. [Figure 7] FIG. 7 is a diagram illustrating a parallel computing work context according to one embodiment of the present invention. [Figure 8] FIG. 8 is a flowchart illustrating a method for offloading parallel computing work by executing one parallel thread group according to an embodiment of the present invention. [Figure 9] FIG. 9 is an operational flowchart illustrating a method for offloading parallel computing work to multiple parallel thread groups (PTGs) according to one embodiment of the present invention. [Figure 10] FIG. 10 is an operational flowchart showing in detail an example of the steps for inserting a PTG execution request into the PTGQ shown in FIG. [Figure 11] FIG. 11 is a diagram illustrating a computer system according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0039] The present invention will be described in detail with reference to the accompanying drawings as follows. Here, repeated explanations and detailed explanations of known functions and configurations that may unnecessarily obscure the gist of the present invention will be omitted. The embodiments of the present invention are provided to more completely explain the present invention to those having average knowledge in the art. Therefore, the shapes and sizes of elements in the drawings may be exaggerated for clarity.

[0040] Throughout the specification, when a part is said to "comprise" a certain element, this does not mean that it excludes other elements, but that it may further include other elements, unless otherwise specified to the contrary.

[0041] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0042] The present invention provides a method for efficiently mapping a large number of threads of a parallel computing task (e.g., an OpenCL kernel) to a heterogeneous many-core processor, including an MIMD-style heterogeneous many-core processor, which supports a relatively small number of multithreads per core compared to a GPGPU, to rapidly execute the parallel computing task.

[0043] FIG. 1 is a diagram illustrating parallel computation tasks (PCT) according to one embodiment of the present invention.

[0044] Referring to Figure 1, parallel computing is a computational task based on thread-level parallelism, similar to the concept of CUDA or OpenCL offloading. A multithread-based programming model such as OpenMP can perform a relatively large number of operations by having one thread process data repeatedly and sequentially using threads.

[0045] The parallel computation task disclosed in the present invention can be understood as a kernel function that performs relatively small-scale operations, such as the parallel computation kernel 11 shown in Fig. 1. An example of a parallel computation task (PCT) is OpenCL NDRagnge.

[0046] A large number of parallel threads (PT) 12 can execute parallel computation kernel 11 code to perform parallel computation tasks. The parallel threads 12 can be, for example, OpenCL work-items.

[0047] Since the parallel threads 12 of parallel computational work target the MIMD core, the control flow of each thread can diverge, making it more effective for work requiring irregular data parallelism.

[0048] Running parallel threads 12 on separate heterogeneous MIMD cores can induce significant overhead.

[0049] Therefore, it is necessary to bundle the parallel threads 12 into a parallel thread group (PTG) 13 and perform collective scheduling in thread group units. An example of the parallel thread group 13 is an OpenCL work-group.

[0050] The parallel computing task (PCT) execution model is divided into two parts, similar to the CUDA or OpenCL execution model. PCT kernels run on heterogeneous acceleration cores, and the host program (PCT App.) runs on the host core. The host program, in conjunction with the accelerator device driver, configures a PCT context for the PCT kernel in memory and requests and manages the execution of the PCT kernel on the heterogeneous acceleration core. PCT kernel threads are executed in parallel, taking full advantage of the accelerator's parallel processing structure. As the number of parallel threads increases, multiple PTs can be bundled together to form parallel thread groups to reduce the scheduling overhead. Typically, a PCT kernel may include an execution startup routine (similar to crt0) that manages the execution of PTs on specific accelerator computing cores.

[0051] FIG. 2 is a block diagram illustrating a parallel computing task offloading device according to one embodiment of the present invention.

[0052] Referring to FIG. 2, it can be seen that a parallel computing task offloading apparatus according to an embodiment of the present invention corresponds to a target heterogeneous core processor system for offloading parallel computing tasks.

[0053] In this case, the heterogeneous core may correspond to a computing core that is distinct from a host core on which an OS runs.

[0054] A parallel computing task offloading device according to one embodiment of the present invention may include at least one host core 110, at least one accelerating core group (ACG) 120, at least one accelerator memory management unit (Accel. MMU) 130, and a memory controller 140.

[0055] The acceleration core group 120 may include parallel thread group scheduling support registers (PTG Scheduling Registers, PSR) 121 that simultaneously run at least one hardware thread (HT), at least one acceleration core 122, and a scratchpad memory (SM) 123 to improve execution efficiency.

[0056] In this case, the accelerator core group 120 may be configured by grouping at least one accelerator core 122 together.

[0057] The acceleration core may provide at least one hardware thread (HT) and may include one or more single instruction multiple data (SIMD) processors shared by the HTs.

[0058] In this case, if the execution of at least one PT executed in the HT enters a stall state, the acceleration core can provide a thread switching function to allow other PTs assigned to the acceleration core to be executed.

[0059] One or more ACGs 120 may share one accelerator memory management unit (Accel.MMU) 130. The Accel.MMU 130 may provide a translation function between virtual addresses and physical addresses so that parallel threads executed in each AC of the ACG 120 can be executed in a virtual address space. The host core 110 and the accelerator core group 120 share memory, and an operating system, accelerator device drivers, and contexts required for executing parallel computation tasks may be allocated as needed.

[0060] The ACG 120 may be requested by a parallel computing host program (PCT App.) running on the host core 110 to execute a parallel thread group (PTG) with a specified number of parallel threads of a certain size, as shown in FIG. 1.

[0061] Generally, the number of parallel threads in a parallel computation task is greater than the maximum number of threads that the entire accelerator core can simultaneously execute at a given time. Scheduling a large number of parallel threads individually to the accelerator cores (ACs) can result in a large number of parallel threads to be scheduled, which can lead to a decrease in the accelerator's computational performance due to scheduling load and delays. Therefore, parallel threads are scheduled to the accelerator cores (ACs) in parallel thread groups (PTGs). This scheduling can accommodate the setting of multiple PSRs to execute PTs on the ACs.

[0062] A parallel computing work offloading device according to one embodiment of the present invention supports and executes scheduling using a data structure (PTG Scheduling Register, PSR) capable of collectively scheduling multiple PTs for multiple ACs, and may include a management register.

[0063] FIG. 3 illustrates a process of executing one parallel thread group between a host device driver and an ACG according to an embodiment of the present invention.

[0064] 3, the PCT Host Process can request a parallel computation task from a device driver and deliver a binary execution code (parallel computation kernel) of a parallel thread to the driver. The parallel computation kernel can be comprised of a user-defined opcode and an execution startup routine that manages the start and end of parallel threads so that the user-defined opcode can be executed as a parallel thread on an accelerator core, and schedules subsequent parallel threads to the accelerator core group.

[0065] 3, the process of performing one PTG on one ACG is shown, and for this purpose, the PSR includes a Scheduling Support Register (SSR) and an Execution Control Register (ECR).

[0066] The scheduling support register (SSR) can include a scheduling control register (SCR) that can specify various scheduling-related options, and an IDLE status register that indicates the status of ACG usage (occupancy by parallel threads). The device driver can selectively apply various scheduling policies to the execution startup routine through the scheduling control register.

[0067] The Execution Control Register (ECR) may include an identifier register to distinguish between parallel thread groups to be executed or running, data structure registers required for PTG execution, registers to specify the execution code and PT concurrency registers for the PTG, and registers to control and report the PTG termination status.

[0068] A portion of the execution control register may be associated with logic for simultaneously executing parallel threads on all acceleration cores of the ACG.

[0069] 1) First, the device driver of the host core 110 can read the value of the IDLE status register of any ACG and confirm that the ACG is in the IDLE state.

[0070] At this time, 2) the device driver can prepare for the PTG execution by setting valid values ​​in the execution control registers, such as the PTG identifier, and registers specifying the code and data structure required for execution.

[0071] At this time, 3) the device driver can execute all PTs of the PTG by setting a valid value in the PT concurrent execution register in the execution control register.

[0072] At this time, 4) the ACG can change the value of the IDLE status register, which is hardwired to the PT concurrent execution register, from IDLE to BUSY at the moment the value of the PT concurrent execution register, one of the execution control registers, changes.

[0073] At this time, 5) all ACs 122 can execute PT to perform PCT kernel functions.

[0074] In this case, 6) when all the executed PTs finish executing, a representative PT among the PTs can report the completion status such as success / failure through the PTG completion register. The representative PT can correspond to a thread that has been set in advance to check the status or the last completed thread.

[0075] At this time, 7) the representative PT can update the value of the IDLE status register to change the IDLE status from BUSY to IDLE.

[0076] At this time, 8) the device driver can confirm the end of PTG execution through an interrupt or polling the IDLE status register.

[0077] At this time, 9) the device driver can check the execution result of the executed PTG from the PTG completion register.

[0078] Additionally, the parallel computational work offloading process can repeat the above procedure to execute a new PTG.

[0079] FIG. 4 illustrates a process of sequential parallel thread group scheduling between a device driver and an ACG of a host core according to an embodiment of the present invention.

[0080] Referring to FIG. 4, it can be seen that the process of continuously scheduling PTGs to ACGs is shown in order to effectively schedule a large number of PTGs.

[0081] For this purpose, the device driver and the representative PT can perform scheduling using the queue management registers of PSR 121. The queue management registers can include a queue base pointer register, a head offset register, and a tail offset register.

[0082] At this time, the device driver can set a policy to execute consecutive PTGs using a parallel thread group queue (PTGQ) in the execution startup routine code of the representative PT through a scheduling control register.

[0083] First, 1) The execution of the first PTG among consecutive PTGs follows the process disclosed in Figure 3, but after the execution of the first PTG is completed, the procedure for executing a new PTG may be different, specifically in terms of how the completion result is reported.

[0084] In this case, 2) the execution startup routine code of the representative PT does not change the value of the IDLE status register after the execution of the first PTG is completed (it continues to maintain the "BUSY" state), and can report the execution result through the execution result queue (ResultQ) instead of the termination register.

[0085] At this time, 3) the execution startup routine code of the representative PT can insert a result entry of the completed PTG into ResultQ.

[0086] In this case, 4) the representative PT execution startup routine code can extract one PTG execution request entry from the PTGQ when the execution of one PTG ends if the PTGQ is not empty.

[0087] At this time, 5) the execution startup routine code of the representative PT can set the execution control register to execute the PTG corresponding to the extracted PTG execution request entry.

[0088] At this time, the execution startup routine code of the representative PT checks whether the PTGQ is in an empty state, and if it is not, it can repeat steps 2) to 5) above until the PTGQ is in an empty state.

[0089] In addition, 1-1) After starting the execution of the first PTG, the device driver immediately inserts multiple PTG execution requests (ERs) into the PTGQ, and can insert further PTG execution requests when an empty slot occurs in the PTGQ through appropriate status monitoring of the PTGQ management register.

[0090] Also, 1-2) the device driver can check the ResultQ through appropriate status monitoring of the ResultQ management register, extract the PTG result entry from the queue as soon as possible, and perform subsequent processing.

[0091] At this time, AC122 can use a thread switching logic to switch (In / Out) the parallel thread context block (PTCB) of a parallel thread that has been stalled for a while due to memory access or other reasons in the scratchpad memory (SM) 123 if the total number of PTs in the PTG is greater than the total number of HTs in the ACG.

[0092] At this time, the representative PT execution startup routine code can transmit an interrupt to the host core 110 where the device driver is executed depending on whether the scheduling control register is set or not when there are no more PTG execution request entries to perform in the PTGQ or when the rate falls below the already set rate.

[0093] The device driver of the host core 110 that receives the interrupt can insert an additional PTG execution request entry into the PTGQ.

[0094] At this time, the execution startup routine code of the representative PT can transmit an interrupt to the host core 110 when the ResultQ is Full or exceeds a certain ratio.

[0095] At this time, the device driver of the host core 110 that receives the interrupt can immediately perform subsequent processing, including checking the PTG execution result.

[0096] FIG. 5 illustrates a process of executing multiple parallel thread groups with priority applied between a device driver and an ACG of a host core according to an embodiment of the present invention.

[0097] Referring to FIG. 5, it can be seen that a PTG scheduling process based on multiple priority PTGQs for each ACG 120 is shown.

[0098] In this case, the PTG scheduling process to which priority is applied can schedule PTGs with other priorities through a separate priority queue according to the already set priority.

[0099] In this case, the PTG scheduling process to which priority is applied can insert execution requests for multiple PTGs into one of the PTGQs corresponding to the priorities according to the pre-set priorities.

[0100] For example, the already set priority may be the priority of a PCT host process, or the priority among multiple parallel computation tasks executed by one PCT host process.

[0101] It can be seen that the multiple PTGQs and one ResultQ shown in FIG. 5 are configured in SM123 or memory.

[0102] Priority-based PTGQ can configure and maintain links in hardware.

[0103] The number of valid entries in each PTGQ can be changed by a separate register value programmed by the SW.

[0104] The device driver and representative PT execution startup routine code can manage multiple PTGQs using multiple PTGQ management register sets.

[0105] In addition, the execution startup routine code of the representative PT can select various scheduling policies for extracting PTG execution requests from multiple queues by setting the PTG scheduling control register 121.

[0106] In addition, each ACG 120 can use a programmable timer for each priority PTGQ to find PTG execution requests that have not been scheduled for a set time, so that execution requests in lower priority PTGQs do not fall into an infinite wait state (starvation).

[0107] In this case, each ACG 120 may include automatic PTGQ management HW that, when it finds an unscheduled PTG execution request, moves the PTG execution request to the last execution request entry in the PTGQ with the next highest priority.

[0108] One PTG execution process shown in FIG. 3 can be used when executing a kernel interactively with the host core, such as for kernel code debugging.

[0109] The continuous PTG scheduling process shown in FIG. 4 can be used to achieve high performance with low HW cost.

[0110] FIG. 6 is a diagram illustrating a PTG execution request and execution result report pipelining between a device driver and a representative PT execution startup routine according to an embodiment of the present invention.

[0111] Referring to FIG. 6, it can be seen that FIGS. 4 and 5 illustrate a parallel thread group execution request and execution result report pipelining between the execution startup routine of the device driver and the representative PT.

[0112] When the device driver specifies that PTGQ is to be used as the scheduling control register, as shown in Figures 4 and 5, when execution of all parallel threads of the PTG is completed, the designated representative PT of the PTG executes the execution startup routine code, saves the parallel thread execution completion information (corresponding to PTGxRslt in Figure 6) in an entry of ResultQ, and modifies the queue management register value of RequltQ.

[0113] In addition, the execution startup routine code of the representative PT reads one PT execution request information (PTG Execution Request, corresponding to PTGxER in Figure 6) from the PTGQ and sets it in the execution control register, thereby scheduling subsequent PTGs to the ACG with low latency.

[0114] The host core's device driver can continuously insert PTG execution request information into the PTGQ.

[0115] At this time, the representative PT of the acceleration core can insert PTG execution completion information into the ResultQ.

[0116] At this time, the representative PT of the acceleration core can extract one PTG execution request information from the PTGQ and execute the subsequent PTG for the ACG it is executing with low latency.

[0117] FIG. 7 is a diagram illustrating a parallel computing work context according to one embodiment of the present invention.

[0118] Referring to Figure 7, a PCT context configured in memory is shown. A PCT context is a collection of data objects generated for each PCT in memory, and may include objects such as context management information, page tables, PCT kernel binaries, and data buffers used by the PCT, as well as execution state information objects required for the execution of individual PTs in the PTG. As described above, to execute parallel thread groups in the ACG, the device driver must perform advance preparations for the execution of parallel threads for each parallel thread group. To execute parallel threads in the PCT simply by setting the PTG scheduling registers in the ACG, execution state information corresponding to the individual thread contexts of each parallel thread must be prepared in memory in advance.

[0119] The execution state information may include parallel thread group common state (parallel_thread_group_common_state) information and individual parallel thread state (parallel_thread_state) information for all parallel threads included in the parallel thread group.

[0120] In this case, the execution state information may be allocated in a contiguous memory space.

[0121] The parallel thread group common state (parallel_thread_group_common_state) information may include the ID of the parallel thread group in the index space, the size of the parallel thread group, the size of the parallel computation work (index space), the total number of parallel thread groups, kernel factor information, etc.

[0122] Individual parallel thread state (parallel_thread_state) information may include the local ID, global ID, etc. of the parallel thread.

[0123] In this case, when one parallel thread (PT) is executed in the hardware thread (HT) of each AC, the execution state information can be loaded and saved in the control and status register of each hardware thread (HT). Thereafter, the user function code of the PCT kernel is executed, and when the execution of the user function code of the PCT kernel is completed, an interrupt to the host core according to the execution startup routine, or an IDLE status register update, or the next parallel thread group scheduling may be performed. In fact, the PCT Context can be used in various ways within the scope of supporting the flow of the present invention.

[0124] FIG. 8 is an operational flowchart illustrating a method for offloading parallel computing tasks according to one embodiment of the present invention.

[0125] Referring to Figure 8, it can be seen that the parallel computing work offloading method according to one embodiment of the present invention shows an operational flowchart illustrating the execution process of one parallel thread group between the host device driver and ACG shown in Figure 3.

[0126] The parallel computing task offloading method according to an embodiment of the present invention may check the IDLE state (S210).

[0127] That is, in step S210, the device driver of the host core 110 can read the value of the IDLE status register of any ACG and confirm that the ACG is in the IDLE state.

[0128] Furthermore, the parallel computing task offloading method according to an embodiment of the present invention may prepare for PTG execution (S220).

[0129] That is, in step S220, the device driver can prepare for the PTG execution by setting valid values ​​in the execution control registers, such as the PTG identifier, and registers specifying the code and data structure required for execution.

[0130] Furthermore, the parallel computing task offloading method according to an embodiment of the present invention may perform PTG (S230).

[0131] That is, in step S230, the ACG can change the value of the IDLE status register, which is hardwired to the PT concurrent execution register, from IDLE to BUSY at the moment the value of the PT concurrent execution register in the execution control register changes.

[0132] At this time, step S230 allows all ACs 122 to execute the PT and perform the PCT kernel function.

[0133] In addition, the parallel computing task offloading method according to an embodiment of the present invention may report the PTG execution completion status (S240).

[0134] That is, in step S240, when all the executed PTs finish executing, a representative PT preset among the PTs can report the end status such as success / failure through the PT end register. The representative PT can correspond to a thread preset to check the status or the last completed thread.

[0135] At this time, in step S240, the representative PT can update the value of the IDLE status register to change the IDLE status from BUSY to IDLE.

[0136] In addition, the parallel computing task offloading method according to an embodiment of the present invention can check the PTG execution result (S250).

[0137] That is, step S250 allows the device driver to check the completion of PTG execution through an interrupt or polling of the IDLE status register.

[0138] At this time, step S250 allows the device driver to check the execution result of the executed PTG from the PTG end register.

[0139] Furthermore, the parallel computing task offloading method according to an embodiment of the present invention can repeat the above procedure to execute a new PTG.

[0140] 9 is an operational flowchart illustrating a method for offloading parallel computational work to multiple PTGs according to one embodiment of the present invention. FIG. 10 is an operational flowchart illustrating in detail an example of the steps of inserting a PTG execution request into the PTGQ shown in FIG.

[0141] Referring to Figure 9, it can be seen that the method for offloading parallel computation work to multiple PTGs according to one embodiment of the present invention shows an operational flowchart illustrating the process of scheduling multiple parallel thread groups to which priority is applied between the device driver and ACG of the host core shown in Figure 5.

[0142] The parallel computing task offloading method according to an embodiment of the present invention may check the IDLE state (S310).

[0143] That is, in step S310, the device driver of the host core 110 can read the value of the IDLE status register of any ACG and confirm that the ACG is in the IDLE state.

[0144] Furthermore, the parallel computing task offloading method according to an embodiment of the present invention may prepare for PTG execution (S320).

[0145] That is, in step S320, the device driver can prepare for the PTG execution by setting valid values ​​in the execution control registers, such as the PTG identifier, and registers specifying the code and data structure required for execution.

[0146] In addition, the method for offloading parallel computation work for multiple PTGs according to an embodiment of the present invention may execute the first PTG (S330).

[0147] That is, in step S330, the ACG can change the value of the IDLE status register, which is hardwired to the PT concurrent execution register, from IDLE to BUSY at the moment the value of the PT concurrent execution register in the execution control register changes.

[0148] At this time, in step S330, all ACs can execute the PT to perform the PCT kernel function.

[0149] In addition, the method for offloading parallel computation work for multiple PTGs according to an embodiment of the present invention can insert a PTG execution request into the PTGQ (S340).

[0150] Referring to FIG. 10, step 340 can first execute the PT of the PTG (S341).

[0151] That is, in step S341, after the execution startup routine code of the representative PT finishes executing the first PTG, the value of the IDLE status register does not change (continues to remain in the "BUSY" state), and the PT of the subsequent PTG can be executed.

[0152] At this time, in step S341, if the total number of PTs in the PTG is greater than the total number of HTs included in the ACG, the AC 122 can switch (In / Out) the parallel thread context block (PTCB) of a parallel thread that has been stalled for a while due to memory access or other reasons in the scratchpad memory (SM) 123 using the thread switching logic.

[0153] Step S340 may also insert a completed PTG result entry into ResultQ (S342).

[0154] That is, in step S342, after the execution of the PTG is completed, the value of the IDLE state register is not changed, and the execution result can be reported through the execution result queue (ResultQ) instead of the end register.

[0155] At this time, step S342 can insert a result entry of the PTG for which the execution startup routine code of the representative PT has finished into ResultQ.

[0156] At this time, step S342 can transmit an interrupt to host core 110 when ResultQ is Full or equals or exceeds a certain ratio.

[0157] Also, step S340 can check whether the PTGQ is empty (EMPTY) (S343).

[0158] That is, in step S343, if the PTGQ is not empty, at the end of the execution of one PTG, one PTG execution request entry can be extracted and inserted from the PTGQ (S344), and if the PTGQ is empty, the PTG execution procedure can be terminated and the PTG execution results can be reported (S350).

[0159] Step S340 may also extract and insert a PTG execution request into the PTGQ (S344).

[0160] That is, at this time, in step S344, if the PTGQ is not empty, the queue management register can extract one PTG execution request entry from the PTGQ when the execution of one PTG is completed.

[0161] At this time, in step S344, the execution startup routine code of the representative PT can extract a PTG execution request entry.

[0162] At this time, step S344 can extract the PTG execution request from one of the PTGQs corresponding to the priority according to the previously set priority.

[0163] At this time, step S344 allows the host core 110 to insert an additional PTG execution request entry into the PTGQ.

[0164] At this time, in step S344, the device driver inserts multiple PTG execution requests (ERs) into the PTGQ, and if an empty slot occurs in the PTGQ through appropriate status monitoring, it can insert additional PTG execution requests.

[0165] In addition, step S344 allows each ACG 120 to find PTG execution requests that have not been scheduled for a previously set time using a programmable timer for each priority PTGQ so that PTG execution requests existing in lower priority PTGQs do not fall into an infinite starvation state.

[0166] At this time, in step S344, when each ACG 120 finds an unscheduled PTG execution request, it can move the PTG execution request to the last execution request entry in the PTGQ with the next highest priority.

[0167] At this time, step S344 can transmit an interrupt to the host core 110 where the device driver is running depending on whether the scheduling control register is set or not when the ACG 120 has no more execution request entries to perform in the parallel thread group queue or the PTG falls below a previously set rate.

[0168] Also, step S345 can schedule the extracted subsequent PTGs.

[0169] That is, step S345 can schedule the PTG using the parallel thread group execution state information included in the parallel computing work context previously stored in memory.

[0170] In this case, the execution state information may include common state information for identifying the parallel thread group, and individual parallel thread state information for identifying the parallel threads included in the parallel thread group.

[0171] Step S345 can also execute PTG through the execution control register.

[0172] At this time, step S345 can execute the PT of the PTG scheduled by the execution startup routine code.

[0173] At this time, step S345 can execute the PTG corresponding to the PTG execution request entry extracted by the representative PT execution startup routine from the PTGQ.

[0174] At this time, step S345 allows the scheduled PTG to execute the PT.

[0175] In addition, the method for offloading parallel computation work for multiple PTGs according to an embodiment of the present invention can check the PTG execution results (S350).

[0176] At this time, step S350 can check the execution result queue to confirm the execution completion status of the PTG.

[0177] At this time, in step S350, the device driver checks the ResultQ through appropriate status monitoring, and extracts the PTG execution result entry from the queue as soon as possible for subsequent processing.

[0178] In addition, the method for offloading parallel computation work for multiple PTGs according to an embodiment of the present invention may check whether there are any unexecuted PTGs in the PCT (S360).

[0179] That is, step S360 checks whether there are any unexecuted PTGs in the PCT, and if there are any unexecuted PTGs, inserts a PTG execution request into the PTGQ for the unexecuted PTGs (S340), and checks the PTG execution results (S350).This step can be repeated to execute the parallel threads of the parallel thread group corresponding to the execution end state until there are no unexecuted PTGs in the PCT.

[0180] Through this, the present invention can schedule the continuous execution of another PTG after the completion of one PTG using the concurrent execution logic of the heterogeneous acceleration core group and execution startup routine software that executes individual parallel processing threads.

[0181] The execution startup routine software may be created as part of the parallel computing kernel when the PCT Host Process compiles the parallel computing kernel provided to the device driver. Therefore, an additional scheduling algorithm can be added as the execution startup routine code at compile time, thereby ensuring scheduling flexibility to accommodate various workloads.

[0182] FIG. 11 is a diagram illustrating a computer system according to one embodiment of the present invention.

[0183] Referring to FIG. 11, a parallel computing task offloading apparatus according to an embodiment of the present invention may be embodied as a computer system 1100, such as a computer-readable storage medium. As shown in FIG. 11, the computer system 1100 may include one or more processors 1110, memory 1130, user interface input devices 1140, user interface output devices 1150, and storage 1160, all of which communicate with each other via a bus 1120. The computer system 1100 may also include a network interface 1170 coupled to a network 1180. The processor 1110 may be a central processing unit or a semiconductor device that executes processing instructions stored in the memory 1130 or storage 1160. The memory 1130 and storage 1160 may be various forms of volatile or non-volatile storage media. For example, the memory may include a read only memory (ROM) 1131 or a random access memory (RAM) 1132.

[0184] A parallel computing work offloading device according to one embodiment of the present invention includes one or more processors 1110 and a memory 1130 storing at least one program executed by the one or more processors 1110, wherein the at least one program inserts execution requests of a plurality of parallel thread groups into at least one parallel thread group queue, and if a pre-set priority exists, the execution requests are inserted into the at least one parallel thread group queue corresponding to the pre-set priority. The parallel threads of the parallel thread groups are executed using parallel thread group execution request entries extracted from the parallel thread group queues according to the priority. When the execution of the parallel threads is completed, the execution results are inserted into an execution result queue, the execution results reported in the execution result queue are checked to confirm the execution completion status of the parallel thread group, and the parallel threads of the parallel thread groups corresponding to the execution completion status are executed.

[0185] In this case, the one or more processors 1110 may include one or more host cores 110 , one or more accelerator cores 120 , an accelerator memory management unit 130 , and a memory controller 140 .

[0186] In this case, the at least one program may use a programmable timer in a parallel thread group queue corresponding to the priority to find a parallel thread group execution request that has not been scheduled for a previously set time.

[0187] In this case, when the at least one program finds a parallel thread group execution request that has not been scheduled for the previously set time, the at least one program may move the parallel thread group execution request that has not been scheduled for the previously set time to the last execution request entry of the parallel thread group queue with the next highest priority.

[0188] In this case, the at least one program may schedule its execution procedure according to the priority using parallel thread group execution state information included in a parallel computation work context previously stored in memory.

[0189] In this case, the execution state information may include common state information for identifying the parallel thread group, and individual parallel thread state information for identifying the parallel threads included in the parallel thread group.

[0190] In this case, the at least one program can use a thread switching logic to switch the context block of the stopped parallel thread to a scratch memory if the total number of parallel threads in the parallel thread group is greater than the number of hardware threads included in the acceleration core group.

[0191] In this case, the at least one program may cause a representative parallel thread, which is preset among the parallel threads included in the parallel thread group, to insert the parallel thread group execution result into the execution result queue.

[0192] In this case, the at least one program may include executing a first parallel thread group of a plurality of parallel thread groups on one acceleration core group.

[0193] In this case, the at least one program can read the value of the IDLE state register of the acceleration core group and, when it confirms that the acceleration core group is in the IDLE state, execute all parallel threads included in any one of the first parallel thread groups.

[0194] In this case, the at least one program may change the value of the IDLE status register from IDLE to BUSY when all parallel threads included in any one of the first parallel thread groups are executed, and may change the value of the IDLE status register from BUSY to IDLE when execution of all the parallel threads is completed.

[0195] As described above, the parallel computing task offloading device and method according to one embodiment of the present invention is not limited to the configurations and methods of the embodiments described above, and the embodiments may be configured by selectively combining all or part of each embodiment to enable various modifications. [Explanation of symbols]

[0196] 11: Parallel computing kernel 12: Parallel threads 13: Parallel Thread Group 110: Host Core 120: Acceleration Core Group 130: Accelerator Memory Management Unit 140: Memory controller 1100: Computer Systems 1110: Processor 1120: Bus 1130:Memory 1131: ROM 1132: Ram 1140: User interface input device 1150: User interface output device 1160:Storage 1170: Network interface 1180:Network

Claims

1. one or more processors; a memory for storing at least one program to be executed by the one or more processors; The at least one program Inserting execution requests of a plurality of parallel thread groups into at least one parallel thread group queue, and if a pre-set priority exists, inserting the execution requests into the at least one parallel thread group queue corresponding to the priority according to the pre-set priority; Execute the parallel threads of the parallel thread group using the parallel thread group execution request entry extracted from the parallel thread group queue according to the priority; When the execution of the parallel thread is completed, insert the execution result into an execution result queue; confirming the execution result reported in the execution result queue to confirm the execution end state of the parallel thread group; a parallel thread of a parallel thread group corresponding to the execution end state;

2. The at least one program 2. The parallel computing work offloading device of claim 1, wherein a programmable timer is used in a parallel thread group queue corresponding to the priority to find a parallel thread group execution request that has not been scheduled for a set time.

3. The at least one program 3. The parallel computing work offloading device of claim 2, wherein when a parallel thread group execution request that has not been scheduled for the previously set time is found, the parallel thread group execution request that has not been scheduled for the previously set time is moved to a last execution request entry of a parallel thread group queue with a next highest priority.

4. The at least one program 2. The parallel computing work offloading device of claim 1, wherein the device executes an execution startup routine code for each parallel thread of the parallel thread group, loads information required for executing the parallel computing kernel code from execution state information into a register of an acceleration core, and executes the parallel computing kernel code.

5. The execution state information is 5. The parallel computing work offloading apparatus of claim 4, further comprising common state information for identifying the parallel thread group, and individual parallel thread state information for identifying the parallel threads included in the parallel thread group.

6. The at least one program 2. The parallel computing work offloading device of claim 1, wherein, when the total number of parallel threads in the parallel thread group is greater than the number of hardware threads included in the acceleration core group, the context block of the stopped parallel thread is switched to a scratch memory using a thread switching logic.

7. The at least one program 2. The parallel computing work offloading device of claim 1, wherein a representative parallel thread, which is preset among the parallel threads included in the parallel thread group, inserts the execution result of the parallel thread group into the execution result queue.

8. The at least one program 2. The parallel computing task offloading device according to claim 1, further comprising the step of executing a first parallel thread group of a plurality of parallel thread groups on one of the acceleration core groups.

9. The at least one program 9. The parallel computing work offloading device of claim 8, wherein the device reads the value of the IDLE status register of the acceleration core group, and executes all parallel threads included in any one of the first parallel thread groups when it is confirmed that the acceleration core group is in the IDLE state.

10. The at least one program 10. The parallel computing task offloading device of claim 9, wherein when all parallel threads included in any one of the first parallel thread groups are executed, the value of the IDLE status register is changed from IDLE to BUSY, and when execution of all the parallel threads is completed, the value of the IDLE status register is changed from BUSY to IDLE.

11. A parallel computing task offloading method for a parallel computing task offloading device, comprising: inserting execution requests of a plurality of parallel thread groups into one of at least one or more parallel thread group queues, and if a pre-established priority exists, inserting the execution requests into the at least one or more parallel thread group queues corresponding to the pre-established priority according to the pre-established priority; executing the parallel threads of the parallel thread group using the parallel thread group execution request entry extracted from the parallel thread group queue according to the priority; When the execution of the parallel thread is completed, inserting the execution result into an execution result queue; checking the execution results reported in the execution result queue to confirm the execution end status of the parallel thread group; executing a parallel thread of a parallel thread group corresponding to the execution end state; 1. A method for offloading parallel computing work, comprising:

12. The performing step includes:

12. The method of claim 11, further comprising: using a programmable timer in a parallel thread group queue corresponding to the priority level to find a parallel thread group execution request that has not been scheduled for a set time.

13. The performing step includes:

13. The parallel computing work offloading method of claim 12, wherein, when a parallel thread group execution request that has not been scheduled for the previously set time is found, the parallel thread group execution request that has not been scheduled for the previously set time is moved to a last execution request entry of a parallel thread group queue with a next highest priority.

14. The performing step includes:

12. The parallel computing work offloading method of claim 11, wherein an execution startup routine code is executed for each parallel thread of the parallel thread group, and information required for executing the parallel computing kernel code is preloaded from execution state information into a register of an acceleration core, and the parallel computing kernel code is executed.

15. The execution state information is 15. The method of claim 14, further comprising common state information for identifying the parallel thread group and individual parallel thread state information for identifying the parallel threads that the parallel thread group comprises.

16. The performing step includes:

12. The parallel computing work offloading method of claim 11, wherein, when the total number of parallel threads in the parallel thread group is greater than the number of hardware threads included in the acceleration core group, the context block of the stopped parallel thread is switched to a scratch memory using a thread switching logic.

17. The inserting step includes:

12. The parallel computing work offloading method of claim 11, wherein a representative parallel thread that is preset among the parallel threads included in the parallel thread group inserts the parallel thread group execution result into the execution result queue.

18. The parallel computing task offloading method includes: before inserting the thread into one of the parallel thread group queues, 12. The method of claim 11, further comprising the step of executing a first parallel thread group of a plurality of parallel thread groups on one of the acceleration core groups.

19. The step of performing the above on any one of the acceleration core groups includes:

20. The parallel computing work offloading method of claim 18, further comprising: reading a value of an IDLE status register of the acceleration core group; and executing all parallel threads included in any one of the first parallel thread groups when it is confirmed that the acceleration core group is in an IDLE state.

20. The step of performing the above on any one of the acceleration core groups includes:

20. The parallel computing work offloading method of claim 19, wherein when all parallel threads included in any one of the first parallel thread groups are executed, the value of the IDLE status register is changed from IDLE to BUSY, and when execution of all the parallel threads is completed, the value of the IDLE status register is changed from BUSY to IDLE.

Images