Task processing method, device and system

Abstract

The embodiments of the present disclosure provide a task processing method, device and system, wherein the method may include: a graphics processing unit (GPU) runs a management task on a first thread, wherein the management task remains in a runtime state; the GPU schedules at least one second thread to execute at least one computing task through the management task on the first thread.

Description

Technical Field

[0001] The present disclosure relates to machine learning technology, and in particular to a task processing method, device and system. Background Art

[0002] A large number of AI-related tasks are running in the computing clusters of data centers, especially tasks such as deep learning that require high parallel computing capabilities. Accordingly, a large number of computing accelerators, mainly general-purpose GPUs (Graphics Processing Units), are configured in the computing clusters, but the GPU resources in the clusters are not fully utilized.

[0003] Compared with CPUs, the rental price of GPUs in data centers is more expensive, and the low utilization rate has further increased the price of unit computing power, resulting in many users being unwilling to purchase GPU resources and preferring to run lightweight tasks such as deep learning model reasoning on CPUs. However, as more and more computing resources are required for these tasks, for example, the scale of deep learning models is increasing, CPU operations are gradually unable to meet users' low-latency requirements, and the trend of using general-purpose computing accelerators such as GPUs is inevitable. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure at least provide a task processing method, device and system to improve the utilization rate of the GPU.

[0005] In a first aspect, a task processing method is provided, the method comprising:

[0006] The graphics processor GPU runs a management task on the first thread, wherein the management task remains in a runtime state;

[0007] The GPU schedules at least one second thread to execute at least one computing task through the management task on the first thread.

[0008] In some optional embodiments, the at least one computing task is a plurality of computing tasks; the GPU schedules at least one second thread to execute the computing task through the management task on the first thread, including: for each of the plurality of computing tasks, the GPU schedules at least one second thread to execute the computing task through the management task on the first thread, wherein the plurality of computing tasks are executed in parallel by the plurality of second threads.

[0009] In some optional embodiments, the GPU schedules at least one second thread to execute at least one computing task through a management task on the first thread, including: the GPU responds to receiving task information of the at least one computing task sent by the central processing unit CPU, and schedules at least one second thread to execute the at least one computing task through a management task on the first thread.

[0010] In some optional implementations, the GPU executes the management task on the first thread, including: the GPU executes the management task on the first thread in response to receiving a task scheduling instruction from the CPU.

[0011] In a second aspect, a task processing method is provided, comprising:

[0012] The CPU sends a task scheduling instruction to the GPU, so that the GPU runs a management task on a first thread based on the task scheduling instruction, wherein the management task remains in a runtime state;

[0013] When receiving the first computing task, the CPU sends task information of the first computing task to the GPU, so that the GPU schedules at least one second thread to execute the first computing task through the management task on the first thread.

[0014] In some optional implementations, the CPU sending the task scheduling instruction to the GPU includes: in response to the task processing device where the CPU is located being started, the CPU sending the task scheduling instruction to the GPU.

[0015] In some optional embodiments, the task information of the first computing task sent by the CPU to the GPU is shared by multiple computing tasks including the first computing task on the GPU side, wherein the multiple computing tasks have the same task operation information.

[0016] In some optional implementations, the method further includes: placing the plurality of computing tasks to be processed into a task queue; and determining that the plurality of computing tasks have the same task operation information by comparing the task codes of the plurality of computing tasks.

[0017] In some optional embodiments, when the CPU and the GPU are included in a server virtual machine, the method further includes: the CPU, in response to receiving a task processing request sent by the client virtual machine, jumps to the system call entry of the server virtual machine to perform a system call according to the value of a special module register MSR in a pre-configured server page table; wherein, before the client virtual machine switches to the server virtual machine, the value of the client virtual address GVA in the MSR in the server page table is configured to be mapped to the client physical address GPA of the system call entry code page of the server virtual machine.

[0018] In some optional embodiments, when the CPU and the GPU are included in a server virtual machine, the method further includes: the CPU, in response to receiving a task processing request sent by a client virtual machine, determines a data access address corresponding to the task processing request according to a value of a flag segment register FS or a global segment register GS of the client virtual machine, wherein, before the client virtual machine switches to the server virtual machine, a value of FS or GS in a virtual machine control structure VMCS field of the client virtual machine is configured to a corresponding value in the server virtual machine.

[0019] In some optional embodiments, when the CPU and the GPU are included in a server-side virtual machine, the method further includes: in response to a crash caused by a page fault in an extended page table, the CPU performs task processing based on a mapping relationship between a server-side virtual machine address corresponding to the page fault and a physical memory address added in the extended page table.

[0020] In a third aspect, a task processing device is provided, the device being applied to a GPU, the device comprising:

[0021] A management and operation module, used for running a management task on the first thread, wherein the management task remains in a runtime state;

[0022] The task scheduling module is used to run a management task on the first thread, wherein the management task remains in a runtime state.

[0023] In some optional embodiments, the task scheduling module is specifically used to: when the at least one computing task is multiple computing tasks, for each of the multiple computing tasks, the GPU schedules at least one second thread to execute the computing task through the management task on the first thread, wherein the multiple computing tasks are executed in parallel by multiple second threads.

[0024] In some optional implementations, the task scheduling module is specifically used to: in response to receiving task information of the at least one computing task sent by the CPU, schedule at least one second thread to execute the at least one computing task through the management task on the first thread.

[0025] In some optional implementations, the management execution module is specifically used to: execute the management task on the first thread in response to receiving a task scheduling instruction from the CPU.

[0026] In a fourth aspect, a task processing device is provided, the device being applied to a CPU, the device comprising:

[0027] an instruction sending module, configured to send a task scheduling instruction to the GPU, so that the GPU runs a management task on a first thread based on the task scheduling instruction, wherein the management task remains in a runtime state;

[0028] A task sending module is used for, when receiving a first computing task, the CPU to send task information of the first computing task to the GPU, so that the GPU schedules at least one second thread to execute the first computing task through the management task on the first thread.

[0029] In some optional implementations, the instruction sending module is used to respond to the task processing device where the CPU is located being started, and the CPU sends a task scheduling instruction to the GPU.

[0030] In some optional embodiments, the task information of the first computing task sent by the CPU to the GPU is shared by multiple computing tasks including the first computing task on the GPU side, wherein the multiple computing tasks have the same task operation information.

[0031] In a fifth aspect, a task processing system is provided, the system comprising the CPU provided by any embodiment of the present disclosure and the GPU provided by any embodiment of the present disclosure.

[0032] In some optional embodiments, the system further includes: a virtual machine monitor for adding a mapping relationship between a server virtual machine address and a physical memory address corresponding to an extended page table when a crash occurs due to an extended page table page fault.

[0033] The task processing method, device and system provided by the embodiments of the present disclosure enable the parallel execution of multiple computing tasks on the GPU by scheduling the execution of computing tasks by the management task on the GPU, thereby reducing the frequent task switching overhead on the GPU and improving the utilization rate of the GPU. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in one or more embodiments of the present disclosure or related technologies, the drawings required for use in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in one or more embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0035] Figure 1 A flowchart of a task processing method provided by at least one embodiment of the present disclosure is shown;

[0036] Figure 2 A schematic diagram of managing task scheduling computing tasks provided by at least one embodiment of the present disclosure is shown;

[0037] Figure 3 A GPU virtualization system architecture provided by at least one embodiment of the present disclosure is shown;

[0038] Figure 4 Example Figure 3 Schematic diagram of the task processing flow under the architecture;

[0039] Figure 5 A flowchart of a task processing method provided by at least one embodiment of the present disclosure is shown;

[0040] Figure 6 A schematic diagram of kernel function scheduling provided by at least one embodiment of the present disclosure is shown;

[0041] Figure 7 A flowchart showing another task processing method provided by at least one embodiment of the present disclosure is shown;

[0042] Figure 8 A flowchart of another task processing method provided by at least one embodiment of the present disclosure is shown;

[0043] Fig. 9 A schematic diagram of extended page table switching provided by at least one embodiment of the present disclosure is shown;

[0044] Fig.10 A schematic diagram of a task processing device provided by at least one embodiment of the present disclosure is shown;

[0045] Fig.11 A schematic diagram showing another task processing device provided by at least one embodiment of the present disclosure is shown;

[0046] Fig.12 A schematic diagram of another task processing device provided by at least one embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of the present disclosure, the technical solutions in one or more embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in one or more embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on one or more embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present disclosure.

[0048] The present disclosure provides a task processing method, which can be executed by a GPU. Figure 1 As shown, the method may include the following processing.

[0049] In step 100 , a GPU runs a management task on a first thread, wherein the management task remains in a runtime state.

[0050] For example, each GPU may include multiple stream processors (Streaming Multiprocessor, SM), each stream processor may run a thread block (Thread Block), the thread block includes a group of threads, that is, one or more threads, each thread can be executed in parallel. When processing a task, the GPU may process the task through at least one thread, wherein the at least one thread may be one or more groups of threads, or part of a group of threads, and the embodiments of the present disclosure do not limit this.

[0051] In some optional embodiments, the management task may be executed on one thread of the GPU, or on multiple threads (such as a group of threads). The thread that executes the management task is referred to as the "first thread" in this document. The "first" here has no restrictive meaning and is only used to distinguish it from other thread names that appear later.

[0052] In the embodiment of the present disclosure, the management task is used to initiate the execution of the computing task. In some examples, the management task may include a kernel function, and the GPU may run the kernel function. The management task may remain in the runtime state after being started, that is, the management task may be kept running. For example, the management task may remain in the runtime state during the GPU operation, but the embodiment of the present disclosure is not limited thereto.

[0053] In some optional embodiments, the management task may be initiated by the CPU, and accordingly, the management task may be that the GPU runs the management task on at least one thread in response to receiving a task scheduling instruction for the management task sent by the CPU. As an example, the CPU may send a task scheduling instruction to the GPU, and the task scheduling instruction includes task information and scheduling information of the management task, for example, the task scheduling instruction includes information such as the task code of the management task and the number of at least one first thread, which is not limited in the embodiments of the present disclosure.

[0054] The CPU may initiate the management task when a specific trigger condition is met. As an optional implementation, the CPU may send the above-mentioned task scheduling instruction for instructing the execution of the management task to the GPU when the task processing device where the CPU is located is started, so that the GPU runs the management task on at least one thread based on the task scheduling instruction. The CPU and the GPU may belong to the same task processing device, for example, the GPU and the GPU may belong to the same server virtual machine, but the embodiments of the present disclosure are not limited thereto.

[0055] In another optional implementation, the CPU may initiate the above management task to the GPU when determining that the GPU is about to start executing a computing task, but the embodiments of the present disclosure are not limited thereto.

[0056] In step 102, the GPU schedules at least one second thread to execute at least one computing task through the management task on the first thread.

[0057] When the GPU determines that a computing task needs to be executed, it initiates the execution of the computing task through the management task. For example, the GPU receives computing task information sent by the CPU, wherein the computing task information may optionally include a task code and data to be processed. The GPU may determine the thread that executes the computing task corresponding to the computing task information based on the computing task information, and schedule the computing task through the management task. In some optional examples, the computing task may be the processing of a deep learning model. For example, the processing of operators or modules in a neural network model, or the computing task may be matrix multiplication or other operations, etc. The embodiments of the present disclosure do not limit the specific implementation of the computing task.

[0058] In an optional implementation, assuming that the CPU receives a request for a computing task sent by a client virtual machine, the computing task may be temporarily referred to as a "first computing task", and the CPU may send task information of the first computing task to the GPU, wherein the information sent by the CPU to the GPU may not include scheduling information of the first computing task. After receiving the task information, the GPU may schedule at least one second thread to execute the first computing task by maintaining the management task in the runtime state, that is, the first computing task is initiated by the management task, and does not need to be initiated by the CPU.

[0059] For example, assuming that the GPU runs a management task on 32 threads, when the GPU receives the task information of the first computing task, it can select one thread from the above 32 threads, and the management task on the thread schedules another one or more threads (called second threads) on the GPU to execute the first computing task.

[0060] In one example, when the GPU is to execute multiple computing tasks, for each of the multiple computing tasks, the GPU schedules at least one second thread to execute the computing task through the management task on the first thread. The multiple computing tasks can be executed in parallel by multiple second threads.

[0061] See also Figure 2 As shown in FIG. 1 , threads S1, S2, and S3 on the GPU are all running management tasks and remain in the runtime state. The management task on thread S1 can schedule thread S4 to perform the first computing task, the management task on thread S2 can schedule thread S5 to perform the second computing task, and the management task on thread S3 can schedule thread S6 to perform the third computing task, so that the first computing task, the second computing task, and the third computing task can be executed in parallel.

[0062] The task processing method of this embodiment provides a scheduling control method for executing tasks on the GPU. This method enables the GPU to execute multiple computing tasks in parallel by scheduling the execution of computing tasks by the management task on the GPU, thereby reducing the frequent task switching overhead on the GPU and improving the utilization rate of the GPU.

[0063] In another embodiment, when there are multiple computing tasks that need to be sent by the CPU to the GPU for processing, it is also possible that the multiple computing tasks have the same task operation information. For example, the same task operation information may have the same kernel function, or the same task operation information may be substantially the same task code. Then, the CPU may send a copy of the task operation information to the GPU, and the task operation information may be shared by multiple computing tasks.

[0064] As an example, the CPU may send task operation information (such as task code) shared by the multiple tasks to the GPU, and send the to-be-processed data of each of the multiple tasks to the GPU, wherein the to-be-processed data of the multiple tasks may be carried in the same or different messages, which is not limited in the embodiments of the present disclosure.

[0065] The CPU can determine that multiple computing tasks have the same task operation information in a variety of ways. As an optional example, the CPU can determine that the multiple computing tasks have the same task operation information by comparing the task codes of the multiple computing tasks. For example, a hash operation can be performed on the kernel function of each computing task. If the hash values ​​obtained by the kernel functions of two computing tasks are the same, then the kernel functions can be considered to be the same, that is, the two computing tasks have the same task operation information, but the embodiments of the present disclosure are not limited to this.

[0066] In some embodiments, after receiving multiple computing tasks to be processed, the CPU may put the multiple computing tasks into a task queue; and compare whether the multiple tasks in the task queue have the same task operation information.

[0067] By sending a copy of multiple computing tasks with the same task operation information to the GPU, multiple computing tasks can be shared on the GPU side, thereby reducing the communication overhead between the CPU and the GPU and improving the task processing efficiency.

[0068] The following will take the above task processing method in the GPU virtualization system architecture as an example to explain the communication interaction and task processing between the GPU and the CPU in the GPU virtualization architecture. GPU virtualization can adopt a variety of virtualization methods, one of which is the API forwarding method. This virtualization method adopts a client / server architecture (C / S structure), providing the abstraction of vGPU to the user on the client, and the server is responsible for managing the allocation and scheduling of the physical GPU. The client forwards the call request for the GPU API to the server, and the server returns the calculation result to the client after the calculation is completed.

[0069] See also Figure 3 As shown, Figure 3 The system architecture under the GPU virtualization mode of API forwarding is illustrated. In this architecture, the physical GPU resources are controlled and managed by the server virtual machine. When the client virtual machine needs to call the GPU to perform a computing task, it can send a GPU call request to the server virtual machine, and the server virtual machine allocates physical GPU resources to the client virtual machine to perform the computing task.

[0070] The task processing flow under this architecture can be found in Figure 4 Example shown.

[0071] In step 400, the client virtual machine sends a GPU call request to the server virtual machine.

[0072] For example, a task code is running on the client virtual machine 11, and during the running process, a GPU needs to be called to assist in executing a certain computing task. At this time, the client virtual machine 11 can send a GPU call request to the server virtual machine 12, and the server virtual machine 12 can control the physical GPU device to execute the computing task that needs to assist the client according to the GPU call request.

[0073] The GPU call request may also be referred to as a GPU API call request. In the GPU virtualization mode of API forwarding, the use of the GPU may be to provide an API interface, and the client application may call the GPU device by calling the API interface.

[0074] In actual implementation, the client virtual machine can transmit the kernel function and the data to be processed to the server virtual machine, and the server virtual machine calls the physical GPU device to execute the kernel function to process the processing. Among them, the kernel function is a function that needs to be run on the GPU, and the kernel function can be executed by multiple threads on the GPU, and each thread will execute the code in the kernel function.

[0075] In step 402, the server virtual machine obtains the kernel function and data to be processed on the client virtual machine side, and calls the GPU to execute the kernel function to process the data.

[0076] For example, the kernel function and data are transmitted from the client virtual machine to the server virtual machine, and the transmission direction is from the host memory to the device memory. Among them, the CPU side is also called the host side, and the CPU memory is called the host memory; the GPU side is also called the device side, and its memory is called the device memory.

[0077] After the kernel function and the data are copied to the GPU, the thread on the physical GPU device can execute the kernel function and process the data through the kernel function to obtain the task processing result.

[0078] In step 404, the server virtual machine returns the task processing result to the client virtual machine.

[0079] After the GPU calculates the task processing result, the server virtual machine can copy the task processing result back to the host memory and return it to the client virtual machine.

[0080] After describing the above basic GPU virtualization system architecture and the task processing flow under the architecture, the improvements made by the embodiments of the present disclosure under the system architecture are explained as follows: In the related art, multiple tasks (i.e., multiple processes) can only use the same GPU hardware resources in a serialized manner, which is also one of the reasons for the low utilization of the GPU. Based on this, the task processing method provided by the above embodiment of the present disclosure can enable multiple tasks to occupy the same GPU resources in parallel, thereby improving the utilization of the GPU. Specifically, the task can be a lightweight GPU task, which can refer to a task that cannot occupy the entire GPU resource alone.

[0081] Figure 5 The task processing method of the embodiment of the present disclosure applied in the system architecture of GPU virtualization is illustrated, and the method may include the following processing:

[0082] In step 500, the CPU of the server virtual machine sends a task scheduling instruction to the GPU, controls multiple first threads of the GPU to run the parent kernel function corresponding to the management task, and keeps the management task in a runtime state.

[0083] In this step, when the server virtual machine is started, the CPU can send a task scheduling instruction to the GPU to schedule and control the multiple first threads on the GPU to run the kernel function of the management task, which is referred to as the parent kernel function in this embodiment. In addition, the CPU can control the management task to remain in the runtime state.

[0084] For example, please refer to Figure 3 , one GPU resource "physical GPU-0" can include multiple stream processors (Streaming Multiprocessor, SM), each stream processor can run a thread block (ThreadBlock), the thread block includes a group of threads, and each thread can be executed in parallel. For example, the parent kernel function of the management task can run on multiple threads on physical GPU-0.

[0085] The multiple threads on the physical GPU-0 may be, for example, Figure 3 The number of threads on a stream processor on the physical GPU-0 in the GPU-0, or the number of threads on two stream processors on the physical GPU-0, etc., or even all threads on all stream processors on the physical GPU-0. The specific number of threads occupied on the physical GPU-0 can be determined by the server virtual machine according to the thread resource occupation parameter carried in the GPU call request. The thread resource occupation parameter can be a related parameter for specifying the number of GPU resources occupied. For example, it can be specified to run the parent kernel function on all threads of one of the SMs.

[0086] The CPU of the server virtual machine can control the above-mentioned multiple threads of the GPU to keep the management tasks in a runtime state. For example, the parent kernel function that is always in runtime can be set to a loop execution state to ensure that it is always in operation.

[0087] Please refer to Figure 6 The hint, Figure 6 It shows that a group of threads on the GPU are all running the above-mentioned parent kernel function, so that the parent kernel function maintains a runtime state on these threads.

[0088] In step 502, the CPU of the server virtual machine receives the computing task sent by the client virtual machine, and sends the task information of the computing task to the GPU.

[0089] In this step, the computing task may be a GPU call request sent by a client virtual machine, which requests to call a kernel function in a GPU execution task, which is called a sub-kernel function in this embodiment. Figure 6 Task 1 / Task 2 / Task 3 in the example may be that the server-side virtual machine receives three computing tasks, and each computing task requests to call the GPU to execute the corresponding sub-kernel function. Figure 6 Three computing tasks are illustrated, but in actual implementation, there may be four computing tasks, five computing tasks, etc., and the number of computing tasks received by the server-side virtual machine is not limited.

[0090] In this step, the CPU can send the task information of the computing task to be processed by the GPU to the GPU. However, unlike the management task, the CPU is not responsible for scheduling the threads on the GPU to execute the computing task. Instead, in the subsequent step 504, the GPU itself schedules the execution of the computing task.

[0091] In step 504, the GPU schedules the execution of the computing task through the management task running on the first thread, so that the computing task is executed on the second thread of the GPU.

[0092] For different computing tasks, the GPU can call the child kernel function corresponding to the computing task through the parent kernel function on different first threads.

[0093] For example, for Figure 6 In the example of Task 1, Task 2, and Task 3, for any of the three computing tasks, the GPU can schedule another second thread to execute the child kernel function corresponding to the computing task through the parent kernel function running in one of the first threads. For different computing tasks, the corresponding child kernel functions can be called through the parent kernel functions running in different first threads, so that the three computing tasks can be run in parallel on the GPU.

[0094] like Figure 6 As shown, the parent core function running by thread 41 on the physical GPU-0 schedules a second thread (the second thread is not shown) to execute the sub-core function 1 corresponding to task 1, and the parent core function running by the control thread 42 schedules another second thread to execute the sub-core function 2 corresponding to task 2, and the parent core function running by the control thread 43 schedules another second thread to execute the sub-core function 3 corresponding to task 3, ultimately realizing the concurrent execution of task 1, task 2 and task 3 on the GPU.

[0095] The task processing method of this embodiment uses a calling method in which a management task running on a GPU thread schedules the execution of computing tasks, so that multiple computing tasks can be run in parallel on the same physical GPU, thereby improving the utilization rate of the GPU.

[0096] In another example, the disclosed embodiment further provides a task processing method, the purpose of the method of this embodiment is to speed up the processing of computing tasks when the task operation information (for example, kernel function) of multiple computing tasks is the same. Figure 7 As shown, the task processing method may include the following processing:

[0097] In step 700, the CPU of the server virtual machine receives multiple computing tasks.

[0098] For example, the multiple computing tasks may be GPU call requests sent by the client virtual machine to the server virtual machine, requesting to call GPU resources to execute corresponding kernel functions to complete a certain computing task.

[0099] In step 702, the CPU puts a plurality of computing tasks to be processed into a task queue, and compares the task codes of the plurality of computing tasks to determine that the plurality of computing tasks have the same task operation information.

[0100] In this step, the CPU of the server virtual machine can put the received multiple computing tasks into the task queue, which stores the various computing tasks waiting to be processed. In addition, a database can be maintained. For each received computing task, the kernel function contained in the task is hashed and the hash value is stored in the database. If kernel functions with the same hash value are found (i.e., the kernel functions are consistent), the computing tasks corresponding to the kernel functions with the same hash value can be batch processed.

[0101] For example, assuming that the kernel functions corresponding to computing tasks A and B are the same, the CPU of the server virtual machine can send a copy of the kernel function to the GPU. In other words, a copy of the kernel function is stored in the device memory of the physical GPU resource. When computing tasks A and B are executed, the GPU thread will execute this kernel function. Specifically, the CPU of the server virtual machine can take a computing task to be processed from the task queue and send the task to the GPU for processing.

[0102] In step 704, the CPU sends task information of the first computing task to the GPU, which is shared by multiple computing tasks including the first computing task on the GPU side.

[0103] For example, when it is determined that the task operation information of multiple computing tasks is the same, the CPU can send the task information of the first computing task to the GPU, and the task operation information of the first computing task is shared by multiple computing tasks including the first computing task on the GPU side.

[0104] In a specific implementation, it is possible that multiple computing tasks use the same kernel function, but the data to be processed by the kernel function is different. In this case, the CPU of the server virtual machine can send the data corresponding to each computing task to the GPU respectively, so that the GPU can process each computing task respectively through the same kernel function and each data, and return the task processing result to the CPU after obtaining it.

[0105] The task processing method of this embodiment batches multiple requests with the same kernel function so that the CPU only needs to transmit one copy of the kernel function to the GPU, thereby reducing the communication copies between the CPU and the GPU and improving the task processing efficiency.

[0106] In another embodiment, whether it is the batch processing of computing tasks mentioned in the above embodiment, or multiple tasks running in parallel on the same GPU, when the client virtual machine sends a GPU call request to the server virtual machine, in order to execute the computing task corresponding to the request on the GPU side, the client virtual machine needs to provide the kernel function and the data to be processed to the server virtual machine, so that the server virtual machine can control the GPU device to execute the kernel function to process the data.

[0107] Please refer to Figure 3 As shown, the virtual machine monitor can configure corresponding extended page tables for the client virtual machine and the server virtual machine. Figure 3 The extended page table 0 in the two client virtual machines corresponds to the extended page table 1 and the extended page table 2 respectively.

[0108] Each extended page table records the mapping of the virtual machine physical address (GPA) to the host physical address (HPA). For a certain client virtual machine, assuming that the client virtual machine wants to send a GPU call request to the server virtual machine, it is necessary to provide the corresponding kernel function and data to the server virtual machine. And, for example, the kernel function and data are actually stored in HPA-1. When the client virtual machine accesses, it must also find the corresponding HPA-1 access according to the mapping relationship "GPA-1——HPA-1" in the extended page table (where "GPA-1 is the address of the client virtual machine). The client virtual machine can send the address HPA-1 to the virtual machine monitor, and the virtual machine monitor adds the mapping "GPA-2——HPA-1" in the extended page table of the server virtual machine, where GPA-2 is the address of the server virtual machine.

[0109] From the above, it can be seen that, assuming that a client virtual machine stores the kernel function and data to be executed later in one of the physical memory addresses HPA-2, a mapping relationship "GPA-12——HPA-2" (GPA-12 is the address of the client virtual machine) can be added to the extended page table corresponding to the client virtual machine. This mapping relationship can be added to the extended page table by the virtual machine monitor. At the same time, the virtual machine monitor can also add a mapping relationship of "GPA-21——HPA-2" to the extended page table corresponding to the server virtual machine bound to the client virtual machine, where "GPA-21" is the address of the server virtual machine, that is, the server virtual machine address and the client virtual machine address GPA both point to the same HPA.

[0110] From the above description, we can see that the server virtual machine and the client virtual machine configure and synchronously modify the extended page table so that the GPA of the server virtual machine and the client virtual machine can point to the same HPA, which can speed up the efficiency of the server virtual machine in obtaining kernel functions and data and reduce the memory copy between the server virtual machine and the client virtual machine.

[0111] Figure 8 A task processing method of another embodiment is provided, which is used to describe the process of implementing data acquisition between a server virtual machine and a client virtual machine through an extended page table, such as Figure 8 As shown, the method may include the following processing:

[0112] In step 800, the server virtual machine receives a GPU call request sent by the client virtual machine.

[0113] For example, when the client virtual machine wants to make a GPU call, it can execute the springboard code page and switch the extended page table through the springboard code page, that is, use the extended page table corresponding to the server virtual machine to access data. Among them, the springboard code page can be understood as the code responsible for switching between processes. As long as the springboard code page executes the VMFUNC instruction, it can switch to execute the extended page table of the server virtual machine. The springboard code page stores the correspondence between the client virtual machine and each server virtual machine it can access. According to the execution of the springboard code page, a GPU call request can be sent to a server virtual machine.

[0114] like Fig. 9 As shown, the Fig. 9 The client virtual machine switches to the extended page table corresponding to the server virtual machine by executing the springboard code page, and the server virtual machine accesses the physical memory to obtain the kernel function and data to be executed according to the extended page table. Among them, the GPA of the server virtual machine and the client virtual machine corresponds to the same physical memory address HPA, and the kernel function and data can be obtained from the HPA.

[0115] In step 802, the server virtual machine queries the corresponding extended page table according to the GPU call request, and accesses the kernel function and data corresponding to the task GPU call request according to the physical memory address indicated in the address mapping information of the extended page table.

[0116] In this step, the server virtual machine queries its corresponding extended page table, and based on the address mapping information of the extended page table (for example, "GPA-2——HPA-1"), it can access the kernel functions and data stored in the physical memory address HPA-1. These kernel functions and data are what the client virtual machine needs the GPU to execute and process.

[0117] In step 804, the server-side virtual machine calls the GPU to execute the kernel function to process data.

[0118] The server virtual machine can control the physical GPU to execute the kernel function and return the task processing result to the client virtual machine.

[0119] Before the client VM executes the VMFUNC instruction to switch to the server VM, the client VM first performs a series of preparatory work before the control flow switch, and then switches to the server VM after the preparation is completed. The preparatory work can be some pre-configuration done by the virtual machine monitor or the client VM, as follows:

[0120] In one example, in order to enable the server virtual machine to perform system calls normally during task processing after switching, the virtual machine monitor can pre-configure the value of the client virtual address GVA (guest virtual address) in the MSR (Model Specific Register) in the server page table to map to the client physical address GPA (guest physical address) of the system call entry code page of the server virtual machine.

[0121] The purpose of the above pre-configured mapping relationship is to correctly execute the system call after the user-mode application initiates the system call when the execution flow is in the address space of the server virtual machine. Most modern operating systems use the SYSCALL instruction to make system calls. When executing the SYSCALL instruction, the CPU will jump to the entry of the system call according to the value of the LSTAR MSR. Under the default configuration, modifying the LSTAR MSR in the virtual machine will trigger the virtual machine to sink and the virtual machine monitor will complete the operation. Therefore, the above-mentioned method of adding mappings avoids the virtual machine sinking during the subsequent control flow switching process. After adding the above mapping, when initiating a system call during the proxy execution process, the entry of the server system call can be accessed by relying on the value of the client LSTAR MSR, so as to interact with the kernel normally.

[0122] As described above, after pre-configuration of the mapping relationship, the CPU of the server virtual machine can respond to the task processing request sent by the client virtual machine and jump to the system call entry of the server virtual machine to perform the system call normally according to the value of the special module register MSR in the pre-configured server page table.

[0123] In another example, the client virtual machine returns to the user state, calls the interface provided by the springboard code page, and starts executing the code related to the control flow switching. However, in order to enable the server virtual machine to access data normally during the task processing after the switch, before switching the address space, the client process needs to temporarily modify the FS.base or GS.base value in the VMCS (Virtual Machine Control Structure) field of the client virtual machine to the corresponding value of the FS (Flag segment register) or GS (Global segment register) in the server virtual machine.

[0124] The purpose of the above processing is to ensure that the segment register access mechanism can work normally in the server virtual machine during the execution of the server virtual machine. In modern operating systems, a large amount of data needs to be accessed through the segment register mechanism. Their base addresses are stored in certain fields in the VMCS of the corresponding virtual machine and will not change with the change of the address space. If no corresponding adjustments are made, the wrong data may be accessed according to the wrong address during the execution of the server virtual machine. The above-mentioned process of modifying the segment register value replaces the correct value in advance, so that the addresses obtained by the segment register access mechanism before and after the address space switch are all legal addresses. Although this step adopts the method of replacing the register value instead of adding a mapping, the virtual machine monitor does not intercept the operations of reading and writing the two registers FS.base and GS.base in the virtual machine by default, so it will still not cause any virtual machine sinking.

[0125] After the above configuration, the CPU of the switched server virtual machine can respond to the task processing request sent by the client virtual machine, determine the data access address corresponding to the task processing request according to the value of the flag segment register FS or the global segment register GS of the client virtual machine, and perform data access in the process of processing the task. For example, the kernel stack will use the value of GS plus an offset as the address to access the data corresponding to the address.

[0126] In another example, after the client virtual machine executes the VMFUNC instruction to switch to the server virtual machine, during the process of the server virtual machine executing the task, it is also possible that the virtual machine sinks, and this sink is caused by a page fault in the extended page table, and the cause of this sink comes from the server virtual machine. When a sink caused by a page fault in the extended page table occurs, the virtual machine monitor can determine whether it is a page fault error that occurs during the task processing of the server virtual machine. If so, extract the parameters related to the page fault in the client virtual machine VMCS, execute the page fault processing function for the server virtual machine, and add the mapping of the GPA to the HPA of the server virtual machine corresponding to the page fault to the extended page table of the server virtual machine. After the above configuration, the CPU of the server virtual machine can continue to process the task based on the above address mapping relationship added in the extended page table. As above, this processing ensures that the server virtual machine correctly handles the extended page table page fault error during the task processing.

[0127] Fig.10 An example of a task processing device is provided, which can be applied to a GPU so that the GPU can execute the task processing method of any embodiment of the present disclosure. Fig.10 As shown, the device may include: a management and operation module 1001 and a task scheduling module 1002.

[0128] The management execution module 1001 is used to execute a management task on the first thread, wherein the management task remains in a runtime state.

[0129] The task scheduling module 1002 is used to run a management task on the first thread, wherein the management task remains in a runtime state.

[0130] In one example, the task scheduling module 1002 is specifically used to: when the at least one computing task is a plurality of computing tasks, for each of the plurality of computing tasks, the GPU schedules at least one second thread to execute the computing task through the management task on the first thread, wherein the plurality of computing tasks are executed in parallel by the plurality of second threads.

[0131] In one example, the task scheduling module 1002 is specifically used to: in response to receiving task information of the at least one computing task sent by the CPU, schedule at least one second thread to execute the at least one computing task through the management task on the first thread.

[0132] In an example, the management execution module 1001 is specifically configured to execute the management task on the first thread in response to receiving a task scheduling instruction from the CPU.

[0133] Fig.11 Another example of a task processing device is provided, which can be applied to a CPU so that the CPU can execute the task processing method of any embodiment of the present disclosure. Fig.11 As shown, the device may include: an instruction sending module 1101 and a task sending module 1102.

[0134] The instruction sending module 1101 is used to send a task scheduling instruction to the GPU, so that the GPU runs a management task on the first thread based on the task scheduling instruction, wherein the management task remains in a runtime state.

[0135] The task sending module 1102 is used for, when receiving a first computing task, the CPU to send task information of the first computing task to the GPU, so that the GPU schedules at least one second thread to execute the first computing task through the management task on the first thread.

[0136] In one example, the instruction sending module 1101 is used for sending a task scheduling instruction from the CPU to the GPU in response to the task processing device where the CPU is located being started.

[0137] In one example, the task information of the first computing task sent by the CPU to the GPU is shared by multiple computing tasks including the first computing task on the GPU side, wherein the multiple computing tasks have the same task operation information.

[0138] In one example, the task sending module 1102 is further used to: put the multiple computing tasks to be processed into a task queue; and determine that the multiple computing tasks have the same task operation information by comparing the task codes of the multiple computing tasks.

[0139] In one example, when the CPU and the GPU are included in a server-side virtual machine, Fig.12 As shown, the device also includes at least one of the following modules:

[0140] The call processing module 1103 is used to respond to the task processing request sent by the client virtual machine and jump to the system call entry of the server virtual machine to make a system call according to the value of the register MSR in the pre-configured server page table; wherein, before the client virtual machine switches to the server virtual machine, the value of the GVA in the register MSR in the server page table is configured to be mapped to the GPA of the system call entry code page of the server virtual machine.

[0141] The access processing module 1104 is used to determine, in response to receiving a task processing request sent by a client virtual machine, a data access address corresponding to the task processing request according to the value of the segment register FS or GS of the client virtual machine, wherein, before the client virtual machine switches to the server virtual machine, the value of the segment register FS or GS in the VMCS domain of the client virtual machine is configured to the corresponding value in the server virtual machine.

[0142] The task processing module 1105 is used for responding to the sink caused by the extended page table page fault, the CPU performing task processing based on the mapping relationship between the server virtual machine address corresponding to the page fault and the physical memory address added in the extended page table.

[0143] In some embodiments, the above-mentioned apparatus may be used to execute any corresponding method described above, which will not be described in detail here for the sake of brevity.

[0144] The embodiment of the present disclosure further provides a task processing system, which may include a CPU and a GPU. The CPU may be the CPU provided by any embodiment of the present disclosure. The GPU may be the GPU provided by any embodiment of the present disclosure.

[0145] For example, the CPU is configured to send a task scheduling instruction for starting a management task to the GPU, and, upon receiving a first computing task, send task information of the first computing task to the GPU. The GPU is configured to run a management task on a first thread based on the task scheduling instruction, wherein the management task remains in a runtime state; and upon receiving the task information of the first computing task, schedule at least one second thread to execute the first computing task through the management task on the first thread.

[0146] In one example, the system further includes: a virtual machine monitor, which is used to add a mapping relationship between a server virtual machine address and a physical memory address corresponding to an extended page table when a crash occurs due to a page fault in the extended page table.

[0147] An embodiment of the present disclosure further provides an electronic device, the device comprising a memory and a processor, the memory being used to store computer-readable instructions, and the processor being used to call the computer instructions to implement the method of any embodiment of the present specification.

[0148] The embodiments of the present disclosure further provide a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method of any embodiment of the present specification.

[0149] It should be understood by those skilled in the art that one or more embodiments of the present disclosure may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, one or more embodiments of the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0150] An embodiment of the present disclosure further provides a computer-readable storage medium, which may store a computer program. When the program is executed by a processor, it implements the steps of the training method of a neural network for text recognition described in any embodiment of the present disclosure, and / or implements the steps of the text recognition method described in any embodiment of the present disclosure.

[0151] Among them, the "and / or" described in the embodiments of the present disclosure means at least one of the two, for example, "many and / or B" includes three options: many, B, and "many and B".

[0152] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the data processing device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0153] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0154] Embodiments of the subject matter and functional operations described in the present disclosure may be implemented in the following: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in the present disclosure and their structural equivalents, or a combination of one or more of them. Embodiments of the subject matter described in the present disclosure may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory program carrier to be executed by a data processing device or to control the operation of the data processing device. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information and transmit it to a suitable receiver device for execution by a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0155] The processes and logic flows described in the present disclosure may be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating output. The processes and logic flows may also be performed by special purpose logic circuits, such as FPGAs (field programmable gate arrays) or SICs (application specific integrated circuits), and the apparatus may also be implemented as special purpose logic circuits.

[0156] Computers suitable for executing computer programs include, for example, general and / or special microprocessors, or any other type of central processing unit. Typically, the central processing unit will receive instructions and data from a read-only memory and / or a random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more large-capacity storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operably coupled to this large-capacity storage device to receive data from it or to transmit data to it, or both situations have both. However, a computer does not necessarily have such a device. In addition, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PD), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0157] Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0158] Although the present disclosure includes many specific implementation details, these should not be interpreted as limiting the scope of any disclosure or the scope of protection claimed, but are mainly used to describe the features of the specific disclosed embodiments. Certain features described in multiple embodiments in the present disclosure may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or implemented in any suitable sub-combination. In addition, although features may work in certain combinations as described above and even initially claim protection, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may point to a sub-combination or a variation of a sub-combination.

[0159] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or requiring that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

[0160] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0161] The above description is only a preferred embodiment of one or more embodiments of the present disclosure and is not intended to limit one or more embodiments of the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present disclosure shall be included in the scope of protection of one or more embodiments of the present disclosure.

Claims

1. A task processing method, characterized in that: The method comprises: A graphics processor GPU runs a management task on at least one first thread, wherein the management task is maintained in a runtime state; the management task includes a parent kernel function; The GPU schedules at least one second thread to execute at least one computing task through a management task on at least one of the first threads, and the computing task includes a sub-kernel function; when the GPU needs to execute multiple computing tasks, for each of the multiple computing tasks, the GPU schedules the second thread to execute the sub-kernel function corresponding to the computing task through a parent kernel function running on a different first thread; the multiple computing tasks are executed in parallel by multiple second threads.

2. The method according to claim 1, characterized in that The GPU schedules at least one second thread to execute at least one computing task through the management task on the first thread, including: In response to receiving the task information of the at least one computing task sent by the central processing unit (CPU), the GPU schedules at least one second thread to execute the at least one computing task through the management task on the first thread.

3. The method according to any one of claims 1 to 2, characterized in that The GPU runs management tasks on the first thread, including: In response to receiving the task scheduling instruction from the CPU, the GPU runs the management task on the first thread.

4. A task processing method, characterized in that: include: The CPU sends a task scheduling instruction to the GPU, so that the GPU runs a management task on at least one first thread based on the task scheduling instruction, wherein the management task remains in a runtime state; the management task includes a parent kernel function; When receiving a first computing task, the CPU sends task information of the first computing task to the GPU, so that the GPU schedules at least one second thread to execute the first computing task through a management task on at least one of the first threads, and the computing task includes a sub-kernel function; when the GPU is to execute multiple computing tasks, for each of the multiple computing tasks, the GPU schedules the second thread to execute the sub-kernel function corresponding to the computing task through a parent kernel function running on a different first thread; the multiple computing tasks are executed in parallel by multiple second threads.

5. The method according to claim 4, characterized in that The CPU sends a task scheduling instruction to the GPU, including: In response to the task processing device where the CPU is located being started, the CPU sends a task scheduling instruction to the GPU.

6. The method according to claim 4 or 5, characterized in that: The task information of the first computing task sent by the CPU to the GPU is shared by a plurality of computing tasks including the first computing task on the GPU side, wherein the plurality of computing tasks have the same task operation information.

7. The method according to claim 6, characterized in that The method further comprises: Putting the plurality of computing tasks to be processed into a task queue; By comparing the task codes of the plurality of computing tasks, it is determined that the plurality of computing tasks have the same task operation information.

8. The method according to claim 4, characterized in that In the case where the CPU and the GPU are included in a server-side virtual machine, the method further includes: In response to receiving a task processing request sent by the client virtual machine, the CPU jumps to the system call entry of the server virtual machine to make a system call according to the value of the special module register MSR in the pre-configured server page table; wherein, before the client virtual machine switches to the server virtual machine, the value of the client virtual address GVA in the MSR in the server page table is configured to be mapped to the client physical address GPA of the system call entry code page of the server virtual machine.

9. The method according to claim 4, characterized in that In the case where the CPU and the GPU are included in a server-side virtual machine, the method further includes: In response to receiving a task processing request sent by a client virtual machine, the CPU determines a data access address corresponding to the task processing request according to a value of a flag segment register FS or a global segment register GS of the client virtual machine, wherein before the client virtual machine switches to the server virtual machine, a value of FS or GS in a virtual machine control structure VMCS domain of the client virtual machine is configured to a corresponding value in the server virtual machine.

10. The method according to claim 4, characterized in that In the case where the CPU and the GPU are included in a server-side virtual machine, the method further includes: In response to a crash caused by a page fault in an extended page table, the CPU performs task processing based on a mapping relationship between a server virtual machine address corresponding to the page fault and a physical memory address added in the extended page table.

11. A task processing device, characterized in that: The device is applied to a GPU, and the device includes: A management and operation module, configured to execute a management task on at least one first thread, wherein the management task is maintained in a runtime state; the management task includes a parent kernel function; A task scheduling module is used to schedule at least one second thread to execute at least one computing task through a management task on at least one of the first threads, wherein the computing task includes a sub-kernel function; when the GPU needs to execute multiple computing tasks, for each of the multiple computing tasks, the GPU schedules the second thread to execute the sub-kernel function corresponding to the computing task through a parent kernel function running on a different first thread; the multiple computing tasks are executed in parallel by multiple second threads.

12. The device according to claim 11, characterized in that The task scheduling module is specifically used to: in response to receiving the task information of the at least one computing task sent by the CPU, schedule at least one second thread to execute the at least one computing task through the management task on the first thread.

13. The device according to any one of claims 11 to 12, characterized in that: The management and execution module is specifically configured to execute the management task on the first thread in response to receiving a task scheduling instruction from the CPU.

14. A task processing device, characterized in that: The device is applied to a CPU, and comprises: An instruction sending module, configured to send a task scheduling instruction to the GPU, so that the GPU runs a management task on at least one first thread based on the task scheduling instruction, wherein the management task remains in a runtime state; the management task includes a parent kernel function; A task sending module is used for, upon receiving a first computing task, the CPU sending task information of the first computing task to the GPU, so that the GPU schedules at least one second thread to execute the first computing task through at least one management task on the first thread, wherein the computing task includes a sub-kernel function; when the GPU is to execute multiple computing tasks, for each of the multiple computing tasks, the GPU schedules the second thread to execute the sub-kernel function corresponding to the computing task through the parent kernel function running on different first threads; the multiple computing tasks are executed in parallel by multiple second threads.

15. The device according to claim 14, characterized in that The instruction sending module is used for sending a task scheduling instruction from the CPU to the GPU in response to the task processing device where the CPU is located being started.

16. The device according to claim 14 or 15, characterized in that The task information of the first computing task sent by the CPU to the GPU is shared by a plurality of computing tasks including the first computing task on the GPU side, wherein the plurality of computing tasks have the same task operation information.

17. A task processing system, characterized in that: The method comprises the CPU according to any one of claims 14 to 16 and the GPU according to any one of claims 11 to 13.

18. The system according to claim 17, characterized in that The system further comprises: The virtual machine monitor is used for adding a mapping relationship between a server virtual machine address and a physical memory address corresponding to an extended page table when a crash occurs due to a page fault in the extended page table.

Images