Task processing method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides a task processing method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of computers, in particular to the technical fields of graphics processors, task allocation, graphics tasks and the like. According to the specific implementation scheme, the method comprises the steps of obtaining a graphics processor task, and obtaining a token from a token bucket; under the condition that the token is successfully acquired, the acquired token is distributed to the graphics processor task, and a graphics processor operator is requested based on the token; the graphics processor operator based on the request processes the graphics processor task.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to the technical fields of graphics processors, task allocation, graphics tasks, etc. Specifically, the present disclosure relates to a task processing method and apparatus, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of artificial intelligence technology, more and more functions are relying on GPU (Graphics Processing Unit) computing, especially image processing-related tasks such as image enhancement, image watermark removal, and OCR (Optical Character Recognition).

[0003] GPU tasks have the characteristics of high single-task occupancy, long occupancy time, resource constraints and difficulty in scalability. In order to avoid sudden user requests occupying GPU resources and causing an avalanche, a reasonable overload protection mechanism is needed to ensure the stability of GPU tasks. Summary of the Invention

[0004] The present disclosure provides a task processing method and device, an electronic device, and a computer-readable storage medium.

[0005] According to a first aspect of the present disclosure, a task processing method is provided, the method comprising:

[0006] Get GPU task and get token from token bucket;

[0007] If the token is successfully acquired, the acquired token is assigned to the GPU task, and a GPU operator is requested based on the token;

[0008] The GPU task is processed based on the requested GPU operator.

[0009] According to a second aspect of the present disclosure, there is provided a task processing device, the device comprising:

[0010] The token acquisition module is used to obtain GPU tasks and obtain tokens from the token bucket;

[0011] An operator acquisition module, configured to assign the acquired token to the GPU task if the token is successfully acquired, and request a GPU operator based on the token;

[0012] The task processing module is configured to process the GPU task based on the requested GPU operator.

[0013] According to a third aspect of the present disclosure, an electronic device is provided, including:

[0014] at least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can perform the task processing method.

[0017] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the above-mentioned task processing method.

[0018] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements the above-mentioned task processing method when executed by a processor.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0021] Figure 1 is a flowchart of a task processing method provided by an embodiment of the present disclosure;

[0022] Figure 2 is a flowchart of some steps of another task processing method provided by an embodiment of the present disclosure;

[0023] Figure 3 is a flowchart of some steps of another task processing method provided by an embodiment of the present disclosure;

[0024] Figure 4 is a flowchart of some steps of another task processing method provided by an embodiment of the present disclosure;

[0025] Figure 5 is a flowchart of some steps of another task processing method provided by an embodiment of the present disclosure;

[0026] Figure 6 is a flowchart of some steps of another task processing method provided by an embodiment of the present disclosure;

[0027] Figure 7 is a structural diagram of a task processing device provided by an embodiment of the present disclosure;

[0028] Figure 8 It is a block diagram of an electronic device used to implement the task processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0030] In some related technologies, when a GPU task is submitted to a backend server according to a user request, the backend server will synchronously request a processing operator to process the GPU task, obtain the processing results, and synchronously return them to the frontend.

[0031] After receiving a GPU task, the backend server will immediately request the operator to process it. When there is a sudden increase in traffic, the operator will be overloaded, the task execution will take longer or even fail, and the user may be triggered to retry, causing the problem to escalate and eventually cause a service avalanche.

[0032] The task processing method and device, electronic device, and computer-readable storage medium provided by the embodiments of the present disclosure are intended to solve at least one of the above technical problems in the prior art.

[0033] The task processing method provided in the embodiments of the present disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be an in-vehicle device, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, etc. The method can be implemented by a processor calling computer-readable program instructions stored in a memory. Alternatively, the method can be executed by a server.

[0034] Figure 1 FIG. 1 shows a flow chart of a task processing method provided by an embodiment of the present disclosure. Figure 1 As shown in , the task processing method provided by the embodiment of the present disclosure may include steps S110, S120, and S130.

[0035] S110: Obtain a graphics processor task and obtain a token from a token bucket.

[0036] The image processor task, namely the GPU task, may be a task implemented based on the GPU.

[0037] GPUs can handle highly parallel, data-intensive computing needs and excel in areas such as graphics rendering, scientific simulation, deep learning, and batch optimization. Therefore, in some specific scenarios, GPU tasks can include image enhancement, watermark removal, and OCR (Optical Character Recognition).

[0038] The token bucket maintains virtual tokens and supports the insertion and removal of tokens. The bucket also has a fixed upper limit on its capacity. The token insertion policy is implemented by controlling the insertion of tokens within a period. By limiting the number of tokens required to execute GPU tasks, the token bucket can be used to control the maximum number of GPU tasks that can be executed simultaneously.

[0039] Therefore, after receiving a request to execute a GPU task and obtaining the GPU task, a token is first obtained from the token bucket to determine whether the GPU supports executing the GPU task.

[0040] If there are tokens in the token bucket, the token is obtained from the token bucket and the token acquisition is successful; if there are no tokens in the token bucket, the token acquisition fails.

[0041] The GPU task may be obtained immediately after receiving the request to execute the GPU task, or may be obtained by sorting the GPU tasks in the order in which the request to execute the GPU task was received and then sorting the GPU tasks in the order in which the request to execute the GPU task was received.

[0042] S120 : When the token is successfully acquired, the acquired token is allocated to the GPU task, and a GPU operator is requested based on the token.

[0043] Among them, a graphics processing unit operator, namely a GPU operator, is a basic computing unit used to perform specific mathematical operations or processing tasks.

[0044] If token acquisition fails, it means that the number of GPU tasks executed simultaneously has reached the maximum number of GPU tasks that the GPU can support, and the GPU can no longer support processing GPU tasks. Therefore, the GPU task needs to queue and wait for GPU resources to be released.

[0045] If the token is successfully acquired, it means that the number of GPU tasks being executed simultaneously has not reached the maximum number of GPU tasks that the GPU can support. Therefore, if the token is successfully acquired, you can continue to request the GPU operator to process CPU tasks.

[0046] Specifically, the obtained token can be "bound" to the GPU task, and a GPU operator for processing the GPU task can be requested based on the token.

[0047] S130 : Processing the GPU task based on the requested GPU operator.

[0048] After obtaining the GPU operator, the GPU task is processed based on the obtained GPU operator. When the GPU task processing is completed, the token and operator corresponding to the GPU are released for use by other GPU tasks.

[0049] The embodiments of the present disclosure do not impose any limitation on the specific implementation method of processing GPU tasks based on GPU operators. Any method that can implement processing GPU tasks based on GPU operators is within the scope of protection of the embodiments of the present disclosure.

[0050] In the task processing method provided by the embodiment of the present disclosure, a token bucket is applied to GPU task processing, and the token is "bound" to the GPU operator. That is, the GPU operator can be requested only when the token is successfully obtained, and the GPU task is processed based on the obtained GPU operator, thereby controlling the number of GPU tasks that the GPU can process simultaneously. This avoids excessive pressure on the GPU operator when traffic suddenly increases, causing task execution to take longer or fail, triggering user retries, and causing the problem to escalate, ultimately causing a service avalanche, thereby ensuring the stability of the GPU service.

[0051] The following is a detailed introduction to the task processing method provided by the embodiment of the present disclosure.

[0052] In some possible implementations, the token bucket may be implemented based on an ordered set of cache middleware.

[0053] The cache middleware may be Redis, which is an open-source, high-performance Key-Value database that supports multiple data formats and has the characteristics of ease of use, high speed, and high concurrency support.

[0054] An ordered set can be a Redis zset structure. Zset is an ordered set supported in Redis. It is a collection of objects, each of which has two values: Value and Score. Objects in the collection are automatically sorted by Score. It supports batch query and deletion operations based on Score values, and supports querying the number of objects in the entire collection.

[0055] Figure 2 A flowchart of an implementation method of acquiring tokens from a token bucket using an ordered set of cache middleware is shown. Figure 2 As shown, step S210 may be included.

[0056] S210 : When the token is acquired successfully, the acquired token is assigned to the GPU task, an ordered set object is written into the ordered set based on the token, and a GPU operator is requested based on the token.

[0057] The ordered set object includes at least an object value and an object score; the object value is a task identifier of the graphics processor task; and the object score is a timestamp corresponding to the time when the acquired token is allocated to the graphics processor task.

[0058] If the token is successfully acquired, the acquired token is "bound" to the GPU task, and the GPU operator used to process the GPU task is requested based on the token. At the same time, the task identifier of the GPU task, which can be specifically the task ID (Identity Document, unique code) of the GPU task, is used as the Value of the ordered set object corresponding to the GPU task, and the timestamp corresponding to the time when the acquired token is assigned to the GPU task is used as the Score of the ordered set object corresponding to the GPU task.

[0059] In some specific implementations, an ordered set object may be used as a specific implementation of a token, that is, one ordered set object corresponds to one token, and the number of tokens in a token bucket can be controlled by controlling the number of ordered set objects in the ordered set.

[0060] Figure 3 The following is a flow chart showing a method for implementing a token bucket based on an ordered set. Figure 3 As shown, it may include steps S310 and S320.

[0061] S310: Obtain a graphics processor task, query and obtain the number of ordered set objects in the ordered set.

[0062] Since the ordered set object is a concrete implementation of the token, one ordered set object corresponds to one token. Therefore, the number of ordered set objects in the ordered set is the number of tokens that have been used.

[0063] When a new GPU task is received, the number of used tokens in the token bucket can be determined by querying the number of objects in the entire ordered set.

[0064] S320: When the number of objects is less than the number threshold, obtain tokens from the token bucket.

[0065] When the number of objects in the ordered set is less than the preset number threshold, it means that the number of used tokens in the token bucket is less than the token capacity of the token bucket, that is, there are still tokens that have not been used, so tokens can be obtained from the token bucket.

[0066] The number threshold is the upper limit of tokens in the token bucket, which can be determined based on the maximum number of tasks that the GPU can support. In some specific implementations, the maximum number of tasks that the GPU operator can support can be estimated based on the average value of the resources occupied by a single GPU task.

[0067] Obtaining tokens from the token bucket can specifically be initializing an ordered set object, and initially setting the Value and Score of the ordered set object to empty. In the subsequent processing process (such as the process corresponding to S210), the GPU task ID is used to assign a value to the Value, and the timestamp is used to assign a value to the Score.

[0068] After the assignment is completed and the operator is obtained, the GPU task is processed based on the obtained operator. During the process of processing the GPU task based on the obtained operator, the ordered set object corresponding to the GPU task is stored in the ordered set.

[0069] When the GPU task processing is completed, the ordered set object corresponding to the GPU task is deleted from the ordered set, and the token corresponding to the GPU task is released. At the same time, the operator corresponding to the GPU task is released for use by other GPU tasks.

[0070] In some specific implementations, since the ordered set object is used as a specific implementation of the token, that is, one ordered set object corresponds to one token, deleting the ordered set object corresponding to the GPU task from the ordered set is equivalent to releasing the token corresponding to the GPU task.

[0071] To sum up, since the ordered set of cache middleware is easy to use, fast, and supports high concurrency, implementing a token bucket based on the ordered set of cache middleware does not consume too much resources, and can ensure the stability of the service without increasing the task time.

[0072] In some specific implementations, the processing of GPU tasks can also be monitored through the ordered set objects in the ordered set to prevent some GPU tasks from being abnormally executed or not releasing resources in time, resulting in occupying resources for too long.

[0073] Figure 4 FIG. 4 shows a flow chart of an implementation method for monitoring the processing of GPU tasks through ordered set objects in an ordered set, as shown in FIG. Figure 4 As shown, it may include steps S410 and S420.

[0074] S410: Determine a timed out ordered set object according to the object score of the ordered set object in the ordered set, and delete the timed out ordered set object from the ordered set.

[0075] S420: Release the token corresponding to the timeout ordered set object, and set the graphics processor task corresponding to the timeout ordered set object to a failed state.

[0076] Since the object score of the ordered set object in the ordered set is the time when the token is assigned to the GPU task, it can also be regarded as the time when the GPU task starts to be executed. Therefore, the execution time of the GPU task, or the time that the GPU task occupies the GPU operator, can be determined by using the current timestamp and the timestamp corresponding to the object score of the ordered set object in the ordered set to perform calculations.

[0077] If the time that the GPU task occupies the GPU operator exceeds the preset timeout period, it can be determined that the GPU task abnormally occupies the GPU operator. Therefore, it can be determined that the ordered set object corresponding to the GPU task is a timeout ordered set object.

[0078] Therefore, the ordered set object corresponding to the GPU task can be deleted from the ordered set, releasing the token corresponding to the GPU task and freeing up the operator corresponding to the GPU task for use by other GPU tasks. This prevents the GPU task from executing abnormally or not releasing resources in a timely manner, which results in excessive resource usage. The GPU task is then set to a failed state to notify the user or other server of the GPU task execution anomaly, allowing the user or other server to resubmit the GPU task.

[0079] In some specific implementations, since the ordered set object is used as a specific implementation of the token, that is, one ordered set object corresponds to one token, deleting the ordered set object corresponding to the GPU task from the ordered set is equivalent to releasing the token corresponding to the GPU task.

[0080] In some possible implementations, a local queue may be added so that, in the event of a token acquisition failure, the GPU task can wait for other GPU tasks to complete their execution and then release the token.

[0081] Figure 5 A flowchart of an implementation method of adding a local queue so that when a token acquisition fails, the GPU task can wait for other GPU tasks to complete execution and release the token is shown. Figure 5 As shown, it may include steps S510 and S520.

[0082] S510: If the token acquisition fails, place the graphics processor task into a task waiting queue.

[0083] If token acquisition fails, it means the number of concurrently executing GPU tasks has reached the maximum number of GPU tasks the GPU can support. The GPU can no longer process GPU tasks and the task must wait in queue for GPU resources to be released. Therefore, if token acquisition fails, the task is placed in the task waiting queue.

[0084] In the case of using ordered set objects as the specific implementation of tokens, by querying the number of objects in the entire ordered set, when the number of objects in the ordered set is equal to the preset number threshold, it means that the number of tokens already used in the token bucket is equal to the token capacity of the token bucket, that is, no tokens are not used, so the GPU task needs to queue and wait for the release of GPU resources.

[0085] S520: Obtain a token from the token bucket. If the token is successfully obtained, assign the obtained token to the GPU task that is placed at the front of the task waiting queue according to the order in which the GPU tasks are placed in the task waiting queue, and delete the GPU task from the task waiting queue.

[0086] The GPU tasks waiting in the task waiting queue continuously try to obtain tokens from the token bucket. If the acquisition is successful, the acquired token is assigned to the GPU task at the front according to the position of the GPU task in the task waiting queue, and the GPU task is deleted from the task waiting queue.

[0087] The position of the GPU task in the task waiting queue is determined according to the order in which the GPU tasks are placed in the task waiting queue. The GPU tasks at the front are placed earlier than those at the back.

[0088] After allocating the acquired token to the GPU task at the front, it is only necessary to continue processing the GPU task according to the task processing method provided in the embodiment of the present disclosure, which will not be described in detail here.

[0089] By setting up a task waiting queue, GPU tasks that are not assigned to tokens can be processed in an orderly manner to avoid task processing confusion.

[0090] In some specific implementations, the GPU tasks in the task waiting queue can also be monitored. If no token can be obtained, the waiting can be ended to avoid excessive resource consumption.

[0091] Figure 6 FIG. 1 shows a flow chart of an implementation method for monitoring GPU tasks in a task waiting queue, as shown in FIG. Figure 6 As shown, step S610 may be included.

[0092] S610: When the waiting time of the graphics processor task in the task waiting queue exceeds a preset timeout time, the graphics processor task is deleted from the task waiting queue and the graphics processor task is set to a failed state.

[0093] When a GPU task is placed in the task waiting queue, the time when the GPU task is placed in the task waiting queue is recorded. At regular intervals, the time difference between the current time and the time when all GPU tasks in the task waiting queue are placed in the task waiting queue is calculated. If the time difference exceeds the preset timeout period, the GPU task can be deleted from the task waiting queue and the graphics processor task can be set to a failed state to avoid excessive resource occupation by task waiting and to avoid long task execution time that affects the user experience.

[0094] Due to the existence of the task waiting queue, when there are too many GPU tasks and the system resources are running at full load, the GPU tasks will not fail directly, but will wait for a while before executing. This reduces the probability of GPU task failure due to insufficient resources, and indirectly improves system resource utilization and user experience.

[0095] In some possible implementations, token buckets can be set to isolate virtual resources of different business parties, thereby preventing GPU tasks of different business parties from affecting each other.

[0096] Specifically, different token buckets may be used for different business methods, that is, in response to receiving a GPU task, a token is obtained from a token bucket corresponding to the business party to which the GPU task belongs.

[0097] That is to say, different business parties use different token buckets, and the tokens in the token bucket are only used for the GPU tasks of the business party. This can limit the occupation of GPU resources by different business parties and prevent the GPU tasks of one business party from occupying too much GPU resources and affecting the GPU tasks of other business parties.

[0098] The following is a specific example of a task processing method provided by the present disclosure. Specifically, the specific example includes the following steps:

[0099] 1. Based on the average resource consumption of a single GPU task, pre-estimate the maximum number of tasks that the GPU operator can support. Configure the token bucket threshold.

[0100] 2. When a GPU task arrives, first query the number of existing tokens in the zset. If the threshold has been reached, it is considered that the acquisition has failed and go to step 4. If the threshold has not been reached, it is considered that the token can be acquired and go to step 3.

[0101] 3. Write an object to the zset. The value of the object is the GPU task ID for this execution, and the score of the object is the current second-level timestamp. Then request the GPU operator service to execute the task. After the execution is completed, delete the token from the zset.

[0102] 4. If the token is not obtained, the task enters the local task waiting queue. Tasks in the task waiting queue will continue to try to obtain the token. If they cannot obtain the token, after the task execution timeout period is reached, the waiting period ends and the task is set to failed. If the token is obtained successfully, go to step 3.

[0103] 5. Since the score in zset is the timestamp of token acquisition, tokens that have exceeded the timeout period can be deleted regularly to avoid the failure to release the token in time after executing the task in step 3, which may cause subsequent tasks to fail to execute.

[0104] This token bucket and queue control the maximum number of concurrent requests to operators, preventing sudden increases in traffic from overwhelming all request operators and potentially saturating GPU resources and causing service unavailability. Furthermore, this approach only involves Redis, resulting in minimal additional processing time (less than 20 milliseconds), with little impact on user experience.

[0105] Based on Figure 1 The same principle as shown in the method, Figure 7 A schematic diagram of the structure of a task processing device provided by an embodiment of the present disclosure is shown. Figure 7 As shown, the task processing device 70 may include:

[0106] The token acquisition module 710 is used to acquire a graphics processor task and obtain a token from a token bucket;

[0107] The operator acquisition module 720 is configured to allocate the acquired token to the GPU task if the token is successfully acquired, and request a GPU operator based on the token;

[0108] The task processing module 730 is configured to process a GPU task based on the requested GPU operator.

[0109] In the task processing device provided by the embodiment of the present disclosure, a token bucket is applied to GPU task processing, and the token is "bound" to the GPU operator. That is, the GPU operator can be requested only when the token is successfully obtained, and the GPU task is processed based on the obtained GPU operator, thereby controlling the number of GPU tasks processed simultaneously by the GPU. This avoids excessive pressure on the GPU operator when traffic suddenly increases, causing task execution to take longer or fail, triggering user retries, and causing the problem to escalate, ultimately causing a service avalanche, thereby ensuring the stability of the GPU service.

[0110] In some possible implementations, the task processing device also includes a task waiting module for: placing the graphics processor task into a task waiting queue if token acquisition fails; obtaining a token from a token bucket, and if token acquisition succeeds, allocating the obtained token to the graphics processor task with the highest order of placement in the task waiting queue according to the order in which the graphics processor tasks in the task waiting queue are placed, and deleting the graphics processor task from the task waiting queue.

[0111] In some possible implementations, the task waiting module is further configured to: when the waiting time of the GPU task in the task waiting queue exceeds a preset timeout period, delete the GPU task from the task waiting queue and set the GPU task to a failed state.

[0112] In some possible implementations, the token bucket is implemented based on an ordered set of cache middleware;

[0113] If the token is successfully acquired, it is assigned to the GPU task. The operator acquisition module is used to:

[0114] If the token is successfully acquired, the acquired token is assigned to the GPU task, an ordered set object is written into the ordered set based on the token, and a GPU operator is requested based on the token.

[0115] The ordered set object includes at least an object value and an object score; the object value is a task identifier of the graphics processor task; and the object score is a timestamp corresponding to the time when the acquired token is allocated to the graphics processor task.

[0116] In some possible implementations, the task processing apparatus further includes a task deletion module configured to:

[0117] When the GPU task processing is completed, the ordered set object corresponding to the GPU task is deleted from the ordered set, and the token corresponding to the GPU task is released.

[0118] In some possible implementations, the task deletion module is further configured to:

[0119] Determine a timed out ordered set object according to the object score of the ordered set object in the ordered set, and delete the timed out ordered set object from the ordered set;

[0120] The token corresponding to the timeout ordered set object is released, and the GPU task corresponding to the timeout ordered set object is set to a failed state.

[0121] In some possible implementations, the token acquisition module is further configured to:

[0122] Get the GPU task and query the number of objects in the sorted set.

[0123] When the number of objects is less than the number threshold, a token is obtained from the token bucket.

[0124] In some possible implementations, the quantity threshold is determined according to the maximum number of tasks that the graphics processor can support.

[0125] In some possible implementations, the token acquisition module is further configured to:

[0126] Get the GPU task and obtain the token from the token bucket corresponding to the business party to which the GPU task belongs.

[0127] It is understandable that the above modules of the task processing device in the embodiment of the present disclosure have the function of implementing Figure 1 The functions of the corresponding steps of the task processing method in the embodiment shown in . This function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. The above modules can be software and / or hardware, and the above modules can be implemented separately or integrated with multiple modules. For the functional description of each module of the above task processing device, please refer to Figure 1 The corresponding description of the task processing method in the embodiment shown in is not repeated here.

[0128] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0129] In the technical solution disclosed herein, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0130] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0131] The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the task processing method provided in the embodiment of the present disclosure.

[0132] Compared with the existing technology, this electronic device applies a token bucket to GPU task processing and "binds" the token to the GPU operator. That is, the GPU operator can only be requested if the token is successfully obtained, and the GPU task is processed based on the obtained GPU operator. The number of GPU tasks that the GPU can process simultaneously is controlled to avoid excessive pressure on the GPU operator when traffic suddenly increases, which causes task execution to take longer or fail, triggering user retries, causing the problem to escalate, and eventually causing service avalanche, thereby ensuring the stability of the GPU service.

[0133] The readable storage medium is a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the task processing method provided by the embodiment of the present disclosure.

[0134] Compared with the existing technology, this readable storage medium applies a token bucket to GPU task processing and "binds" the token to the GPU operator. That is, the GPU operator can only be requested if the token is successfully obtained, and the GPU task is processed based on the obtained GPU operator, thereby controlling the number of GPU tasks that the GPU can process simultaneously. This avoids excessive pressure on the GPU operator when traffic suddenly increases, causing task execution to take longer or fail, triggering user retries, and causing the problem to escalate, eventually causing a service avalanche, thereby ensuring the stability of the GPU service.

[0135] The computer program product includes a computer program, and when the computer program is executed by a processor, the computer program implements the task processing method provided in the embodiment of the present disclosure.

[0136] Compared with the existing technology, this computer program product applies a token bucket to GPU task processing and "binds" the token to the GPU operator. That is, the GPU operator can be requested only when the token is successfully obtained, and the GPU task is processed based on the obtained GPU operator. The number of GPU tasks that the GPU can process simultaneously is controlled to avoid excessive pressure on the GPU operator when traffic suddenly increases, which causes task execution to take longer or fail, triggering user retries, causing the problem to escalate, and eventually causing service avalanche, thereby ensuring the stability of the GPU service.

[0137] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0138] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0139] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0140] The computing unit 801 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the task processing method. For example, in some embodiments, the task processing method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the task processing method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the task processing method by any other appropriate means (e.g., by means of firmware).

[0141] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0142] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0143] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0144] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0145] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0146] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0147] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0148] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A task processing method, comprising: Get GPU task and get token from token bucket; If the token is successfully acquired, the acquired token is assigned to the GPU task, and a GPU operator is requested based on the token; The GPU task is processed based on the requested GPU operator.

2. The method according to claim 1, further comprising: In the case where the token acquisition fails, placing the graphics processor task into a task waiting queue; Obtain a token from the token bucket. If the token is successfully obtained, assign the obtained token to the graphics processor task with the earliest order in the task waiting queue according to the order in which the graphics processor tasks in the task waiting queue are placed, and delete the graphics processor task from the task waiting queue.

3. The method according to claim 2, further comprising: When the waiting time of the graphics processor task in the task waiting queue exceeds a preset timeout time, the graphics processor task is deleted from the task waiting queue and is set to a failed state.

4. The method according to claim 1, wherein The token bucket is implemented based on an ordered set of cache middleware; When the token is successfully acquired, allocating the acquired token to the GPU task and requesting a GPU operator based on the token includes: If the token is successfully acquired, the acquired token is assigned to the GPU task, an ordered set object is written into the ordered set based on the token, and a GPU operator is requested based on the token; The ordered set object includes at least an object value and an object score; the object value is a task identifier of the graphics processor task; and the object score is a timestamp corresponding to a time when the acquired token is allocated to the graphics processor task.

5. The method according to claim 4, further comprising: When the processing of the graphics processor task is completed, the ordered set object corresponding to the graphics processor task is deleted from the ordered set, and the token corresponding to the graphics processor task is released.

6. The method according to claim 4, further comprising: Determine a timed out ordered set object according to the object score of the ordered set object in the ordered set, and delete the timed out ordered set object from the ordered set; The token corresponding to the timeout ordered set object is released, and the graphics processor task corresponding to the timeout ordered set object is set to a failed state.

7. The method according to claim 4, wherein: The step of obtaining a graphics processor task and obtaining a token from a token bucket includes: Obtaining a graphics processor task, querying and obtaining the number of objects in the ordered set; When the number of objects is less than the number threshold, a token is obtained from the token bucket.

8. The method according to claim 7, wherein: The quantity threshold is determined according to the maximum number of tasks that the graphics processor can support.

9. The method according to claim 1, wherein The step of obtaining a graphics processor task and obtaining a token from a token bucket includes: A graphics processing unit task is obtained, and a token is obtained from a token bucket corresponding to the business party to which the graphics processing unit task belongs.

10. A task processing device comprising: The token acquisition module is used to obtain GPU tasks and obtain tokens from the token bucket; An operator acquisition module, configured to assign the acquired token to the GPU task if the token is successfully acquired, and request a GPU operator based on the token; The task processing module is configured to process the GPU task based on the requested GPU operator.

11. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.

13. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images