Control method, apparatus, and electronic device

Abstract

The application discloses a control method, device and electronic equipment, the method comprises the following steps: obtaining a data processing task; the data processing task is used for requesting a target processing resource in a resource set, the resource set contains at least one processing resource, and each processing resource corresponds to at least a first set; the number of first labels issued per unit time in the first set is related to the minimum parallel task processing capacity of the corresponding processing resource; obtaining a first label for the data processing task in the first set corresponding to the target processing resource; obtaining a second label for the data processing task in a second set in the case of obtaining the first label for the data processing task; the number of second labels issued per unit time in the second set is related to the maximum parallel task processing capacity of the resource set; obtaining a processing result by executing a data processing operation corresponding to the data processing task based on the target processing resource in the case of obtaining the second label for the data processing task.

Description

Technical Field

[0001] This application relates to the field of control technology, and in particular to a control method, device and electronic device. Background Technology

[0002] The cloud platform provides users with resource pools to facilitate the execution of computing models and deliver computation results to users according to their requests. For example, after receiving an inference request, the cloud platform sends the inference request to an inference model that is encapsulated as a RESTful API, and the inference model returns the inference result according to the inference request.

[0003] However, there may be situations where the number of requests fluctuates significantly. Therefore, a technical solution capable of controlling the flow of requests is needed. Summary of the Invention

[0004] In view of the above, this application provides a control method, apparatus, and electronic device, as follows:

[0005] A control method, comprising:

[0006] Obtain a data processing task; the data processing task is used to request a target processing resource in a resource set, the resource set contains at least one processing resource, each processing resource corresponds to at least a first set containing a first tag, the number of first tags issued per unit time in the first set is related to the minimum parallel task processing volume of its corresponding processing resource;

[0007] In the first set corresponding to the target processing resources, a first tag is obtained for the data processing task;

[0008] If the first tag is obtained for the data processing task, a second tag is obtained for the data processing task in the second set. The number of second tags issued per unit time in the second set is related to the maximum parallel task processing capacity of the resource set. The maximum parallel task processing capacity is greater than the sum of the minimum parallel task processing capacity of all the processing resources in the resource set.

[0009] If the second tag is obtained for the data processing task, the data processing operation corresponding to the data processing task is executed based on the target processing resource to obtain the processing result.

[0010] Optionally, if the first tag is not obtained for the data processing task, the above method further includes:

[0011] A third tag is obtained for the data processing task in the third set, and the number of third tags issued per unit time in the third set is related to the amount of redundant parallel tasks corresponding to the maximum parallel task processing volume.

[0012] If the third tag is obtained for the data processing task, the following step is performed: obtain the second tag for the data processing task from the second set.

[0013] Optionally, in the case where neither the first tag nor the third tag is obtained for the data processing task, the method further includes any one of the following:

[0014] Discard the data processing task;

[0015] The data processing task is cached to wait for a new third tag to be issued in the third set, so that if the new third tag is obtained for the data processing task, the step of obtaining a second tag for the data processing task in the second set is performed.

[0016] Optionally, in the case where the first tag is obtained for the data processing task but the second tag is not obtained for the data processing task, the method further includes:

[0017] The data processing task is cached to wait for a new second tag to be issued from the second set, so that when the new second tag is obtained for the data processing task, the following step is executed: perform the data processing operation corresponding to the data processing task based on the target processing resource to obtain the processing result.

[0018] Optionally, in the case where the first tag, the third tag, and the second tag are not obtained for the data processing task, the method further includes:

[0019] Discard the data processing task.

[0020] Optionally, in the above method, the minimum parallel task processing capacity of the processing resources is obtained based on the maximum parallel task processing capacity and the resource ratio of the processing resources in the resource set; wherein, the resource ratio is the ratio between the amount of video memory required by the processing resources and the sum of the amount of video memory required by all processing resources in the resource set.

[0021] The redundant parallel task processing capacity is obtained by multiplying the maximum parallel task processing capacity by a preset redundancy coefficient.

[0022] Optionally, the above method includes obtaining the data processing task, which comprises:

[0023] Receive a data processing request, the data processing request being used to request the target processing resource;

[0024] Add the received data processing request to the request set corresponding to the target processing resource;

[0025] Determine whether the request merging condition corresponding to the request set is met;

[0026] If the request merging condition is met, merge the data processing requests in the request set to obtain a data processing task, clear the request set, and return to the step of receiving a data processing request.

[0027] If the request merging conditions are not met, return to the step of receiving a data processing request.

[0028] Optionally, the request merging condition in the above method includes: the cache duration of the first data processing request added to the request set reaches a preset maximum duration, wherein the maximum duration is set according to the running status of the target processing resource;

[0029] or,

[0030] The request merging condition includes: the number of data processing requests stored in the request set reaches the maximum number of parallel processing requests corresponding to the target processing resource.

[0031] A control device, comprising:

[0032] A task acquisition unit is used to acquire a data processing task; the data processing task is used to request a target processing resource in a resource set, the resource set contains at least one processing resource, each processing resource corresponds to at least a set containing a first tag, and the data of the first tag issued in the first set per unit time is related to the minimum parallel task processing amount of its corresponding processing resource;

[0033] The first acquisition unit is used to acquire a first tag for the data processing task from the first set corresponding to the target processing resource;

[0034] The second acquisition unit is used to acquire a second tag for the data processing task in a second set when the first acquisition unit acquires the first tag for the data processing task. The number of second tags issued per unit time in the second set is related to the maximum parallel task processing capacity of the resource set. The maximum parallel task processing capacity is greater than the sum of the minimum parallel task processing capacity of all the processing resources in the resource set.

[0035] The task execution unit is used to execute the data processing operation corresponding to the data processing task based on the target processing resource when the second acquisition unit acquires the second tag for the data processing task, so as to obtain the processing result.

[0036] An electronic device, comprising:

[0037] A memory for storing computer programs and the data generated by the execution of said computer programs;

[0038] A processor is configured to execute the computer program to: obtain a data processing task; the data processing task requests a target processing resource from a resource set, the resource set containing at least one processing resource, each processing resource corresponding to at least a first set containing a first tag, the number of first tags issued per unit time in the first set being related to the minimum parallel task processing capacity of its corresponding processing resource; obtain a first tag for the data processing task from the first set corresponding to the target processing resource; if the first tag is obtained for the data processing task, obtain a second tag for the data processing task from a second set, the number of second tags issued per unit time in the second set being related to the maximum parallel task processing capacity of the resource set, the maximum parallel task processing capacity being greater than the sum of the minimum parallel task processing capacities of all processing resources in the resource set; if the second tag is obtained for the data processing task, perform a data processing operation corresponding to the data processing task based on the target processing resource to obtain a processing result.

[0039] As can be seen from the above technical solutions, in the control method, apparatus, and electronic device disclosed in this application, after obtaining a data processing task for requesting a target processing resource, a first tag is obtained for the data processing task in a first set corresponding to the target processing resource. After obtaining the first tag, a second tag is obtained for the data processing task in a second set. Only when the second tag is obtained for the data processing task can the data processing operation corresponding to the data processing task be executed based on the target processing resource to obtain the processing result. The number of first tags issued per unit time in the first set is related to the minimum parallel task processing capacity of its corresponding processing resource, and the number of second tags issued per unit time in the second set is related to the maximum parallel task processing capacity of the resource set containing all processing resources. Therefore, in this application, the target processing resource can only be used to execute the data processing task when both the first and second tags are obtained simultaneously. This ensures that each processing resource in the resource set reaches its minimum parallel task processing capacity and does not exceed the maximum parallel task processing capacity of the resource set. Thus, even if the number of requests fluctuates greatly, it can avoid task congestion of a certain processing resource and exceeding the maximum processing capacity of the resource set, thereby achieving flow control of requests. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart of a control method provided in Embodiment 1 of this application;

[0042] Figure 2 This is an example diagram illustrating the processing of resources and the first set in an embodiment of this application;

[0043] Figure 3 This is an example diagram illustrating the processing of resources and the first and second sets in embodiments of this application;

[0044] Figure 4 Another flowchart of a control method provided in Embodiment 1 of this application;

[0045] Figure 5 This is an example diagram illustrating the processing of resources with the first set, the second set, and the third set in embodiments of this application;

[0046] Figures 6-9 Each of the following is a flowchart of a control method provided in Embodiment 1 of this application;

[0047] Figure 10 This is a partial flowchart of a control method provided in Embodiment 1 of this application;

[0048] Figure 11 This is an example diagram illustrating the processing of resources, a first set, a second set, a third set, and a request set in embodiments of this application.

[0049] Figure 12 This is a schematic diagram of the structure of a control device provided in Embodiment 2 of this application;

[0050] Figures 13-17 These are another structural schematic diagrams of a control device provided in Embodiment 2 of this application;

[0051] Figure 18 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of this application;

[0052] Figure 19 This is a flowchart illustrating request batching for GPU resource pools in this application;

[0053] Figure 20 This is a schematic diagram illustrating the process of flow control for requests in a GPU resource pool, applicable to this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] refer to Figure 1 The diagram shown is a flowchart illustrating the implementation of a control method according to Embodiment 1 of this application. This method can be applied to electronic devices, such as computers or servers. The technical solution in this embodiment is mainly used for flow control of data processing tasks for request processing resources.

[0056] Specifically, the method in this embodiment may include the following steps:

[0057] Step 101: Obtain the data processing task.

[0058] In this system, a data processing task requests a target processing resource from a resource set. Each data processing task may contain one or more data processing requests, each requesting a specific processing resource (i.e., the target processing resource) from the resource set. The resource set contains at least one processing resource, and each processing resource corresponds to at least one first set containing a first tag. The number of first tags issued per unit time in the first set is related to the minimum parallel task processing capacity of its corresponding processing resource. For example, ... Figure 2 As shown, each processing resource in the resource set corresponds to a first set. In the first set, a corresponding number of first tags are issued per unit time according to the minimum parallel task processing volume of the corresponding processing resource. The better the resource performance of the processing resource, the higher the number of first tags issued per unit time.

[0059] It should be noted that the minimum parallel task processing capacity refers to the lowest parallel task processing capacity that the corresponding processing resources can provide. In other words, the processing resources, based on their resource performance, can at least execute the minimum parallel task processing capacity of data processing tasks simultaneously.

[0060] Specifically, the minimum parallel task processing capacity of the processing resource can be obtained based on the maximum parallel task processing capacity and the resource proportion of the processing resource in the resource set. Here, the resource proportion is the ratio between the amount of GPU memory required for the processing resource and the total amount of GPU memory required by all processing resources in the resource set.

[0061] The processing resources can be processing models based on graphics processing units (GPUs). For example, a GPU resource pool contains one or more GPUs. Correspondingly, the resource set contains multiple processing models deployed on one or more GPUs. A processing model can be deployed on multiple GPUs, and one or more processing models can be deployed on a single GPU. Each processing model runs a corresponding number of instances according to resource performance and can provide data processing services externally via a RESTful API. Based on this, the terminal can send data processing requests to the resource set to invoke the RESTful API of the processing resource model to perform data processing.

[0062] In practical implementation, the first set corresponding to each processing resource can be understood as the first-level token bucket corresponding to that processing resource. First-level tokens (i.e., first tags) are issued in the first-level token bucket according to the token issuance rate corresponding to the minimum parallel task processing volume. For example, the GPU resource pool contains K GPUs. N processing models are deployed on these K GPUs. The maximum parallel task processing capacity, i.e., the maximum concurrency, is C, and the number of instances in the processing model is... The amount of GPU memory required to process a single instance in the model is GM. n This indicates that, based on this, the processing model is... n The corresponding rate of issuance of Level 1 tokens in the Level 1 token bucket (CIR) n The minimum parallel task processing capacity of the processing model is expressed by the following formula (1):

[0063]

[0064] Step 102: Obtain the first label for the data processing task from the first set corresponding to the target processing resources.

[0065] Specifically, a first label is obtained for the data processing task in the first set corresponding to the target processing resource. If there are remaining first labels in the first set corresponding to the target processing resource, it means that the current task processing volume of the target processing resource has not yet reached the minimum parallel task processing volume. At this time, a first label can be taken from the first set for the data processing task. If there are no remaining first labels in the first set corresponding to the target processing resource, it means that the current task processing volume of the target processing resource has exceeded the minimum parallel task processing volume.

[0066] For example, a first-level token is obtained for the data processing task from the first-level token bucket corresponding to the target processing resource requested by the data processing task. If there are remaining first-level tokens in the first-level token bucket of the target processing model, it means that the current task processing volume of the target processing model has not yet reached the minimum parallel task processing volume. At this time, a first-level token can be taken from the first-level token bucket for the data processing task, and a green label is set for the data processing task to indicate that a first-level token has been obtained for the data processing task. If there are no remaining first-level tokens in the first-level token bucket of the target processing model, it means that the current task processing volume of the target processing model has at least reached the corresponding minimum parallel task processing volume.

[0067] Step 103: Determine if the first tag has been obtained for a data processing task. If the first tag has been obtained for a data processing task, proceed to step 104.

[0068] Step 104: Obtain the second label for the data processing task in the second set.

[0069] The number of second tags issued per unit time in the second set is related to the maximum parallel task processing capacity in the resource set, which is greater than the sum of the minimum parallel task processing capacity of all processing resources in the resource set.

[0070] It should be noted that the second set is the set corresponding to the resource set used for issuing the second tag, such as... Figure 3As shown, each resource set corresponds to a second set, and each processing resource in the resource set corresponds to a first set. The second set issues a corresponding number of second tags per unit time according to the maximum parallel task processing volume of the resource set. The higher the overall resource performance of the resource set, the higher the number of second tags issued per unit time.

[0071] It should be noted that the maximum parallel task processing capacity refers to the maximum number of parallel tasks that can be processed by all processing resources in the resource set. In other words, the resource set can execute data processing tasks with the maximum parallel task processing capacity at the same time, based on the resource performance of all the processing resources it contains.

[0072] In the specific implementation, the second set corresponding to the resource set can be understood as the third-level token bucket corresponding to the resource set. In the third-level token bucket, the third-level token (i.e., the second tag) is issued according to the issuance rate corresponding to the maximum parallel task processing volume. For example, taking the GPU resource pool as an example, the GPU resource pool has a third-level token bucket. The CIR of the third-level token bucket, that is, the rate at which the third-level token is issued, is set to the maximum concurrency of the GPU resource pool, that is, the maximum parallel task processing volume, as shown in the following formula (2):

[0073] CIR = C Formula (2)

[0074] Based on this, a second label is obtained for the data processing task in the second set. If there are remaining second labels in the second set corresponding to the resource set, it means that the current overall task processing volume of all processing resources in the resource set has not yet reached the maximum parallel task processing volume. At this time, a second label can be taken from the second set for the data processing task. If there are no remaining second labels in the second set, it means that the current overall task processing volume of the resource set has exceeded the maximum parallel task processing volume.

[0075] For example, after obtaining a Level 1 token for the data processing task from the Level 1 token bucket corresponding to the target processing resource requested by the data processing task, a Level 3 token is then obtained from the Level 3 token bucket of the resource pool. If there are remaining Level 3 tokens in the Level 3 token bucket, it means that the current task processing volume of the resource pool has not yet reached the maximum parallel task processing volume, and a Level 3 token can be obtained from the Level 3 token bucket for the data processing task. If there are no remaining Level 3 tokens in the Level 3 token bucket, it means that the current task processing volume of the resource pool has exceeded the maximum parallel task processing volume.

[0076] Step 105: Determine if the second tag was obtained by a data processing task. If the second tag was obtained by a data processing task, proceed to step 106.

[0077] Step 106: Execute the data processing operation corresponding to the data processing task based on the target processing resources to obtain the processing result.

[0078] For example, in this embodiment, the model RESTful API of the target processing resource is called to process the data processing requests in the data processing task, so as to obtain the processing result of each data processing request.

[0079] As can be seen from the above technical solution, in the control method provided in Embodiment 1 of this application, after obtaining a data processing task for requesting a target processing resource, a first tag is obtained for the data processing task in a first set corresponding to the target processing resource. If the first tag is obtained for the data processing task, a second tag is obtained for the data processing task in a second set. Only when the second tag is obtained for the data processing task can the data processing operation corresponding to the data processing task be executed based on the target processing resource to obtain the processing result. The number of first tags issued per unit time in the first set is related to the minimum parallel task processing capacity of its corresponding processing resource. Furthermore, the number of second tags issued per unit time in the second set is related to the maximum parallel task processing capacity of the resource set containing all processing resources. Therefore, in this embodiment, the target processing resource can only be used to execute the data processing task if both the first and second tags are obtained simultaneously. This ensures that each processing resource in the resource set reaches its minimum parallel task processing capacity and does not exceed the maximum parallel task processing capacity of the resource set. Thus, even if the number of requests fluctuates significantly, it can avoid task congestion of a certain processing resource and exceeding the maximum processing capacity of the resource set, thereby achieving traffic control of the requests.

[0080] In one implementation, if it is determined in step 103 that no first tag has been obtained for the data processing task, the method in this embodiment may further include the following steps, such as... Figure 4 As shown:

[0081] Step 107: Obtain the third label for the data processing task in the third set.

[0082] The number of third tags issued per unit time in the third set is related to the redundant parallel task processing volume corresponding to the maximum parallel task processing volume. Redundant parallel task processing volume refers to the number of additional data processing tasks that can be executed in parallel after all processing resources in the resource set have reached their minimum parallel task processing volume.

[0083] It should be noted that the third set is the set corresponding to the resource set used for issuing third tags, such as... Figure 5As shown, the resource set corresponds to a second set and one or more third sets. Each processing resource in the resource set corresponds to a first set. The second set issues a corresponding number of second tags per unit time according to the maximum parallel task processing volume of the resource set. The third set issues a corresponding number of third tags per unit time according to the redundant parallel task processing volume.

[0084] Specifically, the processing capacity of redundant parallel tasks can be obtained by multiplying the maximum processing capacity of parallel tasks by a preset redundancy coefficient. In one implementation, the redundancy coefficient can be a value less than 1, meaning that the processing capacity of redundant parallel tasks in the resource set is less than the maximum processing capacity of parallel tasks. In another implementation, the redundancy coefficient can be a value greater than or equal to 1, meaning that the processing capacity of redundant parallel tasks in the resource set is greater than or equal to the maximum processing capacity of parallel tasks.

[0085] For example, the third set corresponding to the resource set can be understood as the secondary token bucket corresponding to the resource set. Secondary tokens (i.e., the third tag) are issued in the secondary token bucket according to the issuance rate corresponding to the redundant parallel task processing volume. For example, taking the GPU resource pool as an example, the GPU resource pool has a token bucket and a secondary token bucket. The CIR of the secondary token bucket, that is, the rate at which secondary tokens are issued, is set to the redundant concurrency of the GPU resource pool, that is, the redundant parallel task processing volume, as shown in the following formula (3):

[0086] CIR = th cir *C Formula (3)

[0087] Based on this, a third label is obtained for the data processing task in the third set. If there are remaining third labels in the third set corresponding to the resource set, it means that the redundant parallel task processing volume in the resource set has not been fully utilized. At this time, a third label can be obtained for the data processing task from the third set. If there are no remaining third labels in the third set, it means that the redundant parallel task processing volume in the resource set has been fully utilized.

[0088] For example, if a data processing task cannot obtain a Level 1 token from the Level 1 token bucket corresponding to the target processing resource requested by the task, we can only consider whether there is any remaining redundant parallel task processing capacity in the resource pool. In this case, we obtain a Level 2 token for the data processing task from the Level 2 token bucket of the resource pool, and then mark the data processing task with a red label to indicate that a Level 2 token has been obtained. If there are remaining Level 2 tokens in the Level 2 token bucket, it means that there is still remaining redundant parallel task processing capacity in the resource pool, and a Level 2 token can be obtained from the Level 2 token bucket for the data processing task. If there are no remaining Level 2 tokens in the Level 2 token bucket, it means that the redundant parallel task processing capacity of the resource pool has been completely occupied.

[0089] Step 108: Determine if the third tag was obtained by a data processing task. If the third tag was obtained by a data processing task, proceed to step 109.

[0090] Step 109: Obtain the second label for the data processing task in the second set.

[0091] Step 110: Determine if the second tag was obtained by a data processing task. If the second tag was obtained by a data processing task, proceed to step 111.

[0092] Step 111: Execute the data processing operation corresponding to the data processing task based on the target processing resources to obtain the processing result.

[0093] Therefore, if a third label is obtained for a data processing task even if a first label is not obtained, it means that although the current task processing capacity of the target processing resource in the resource set has reached the minimum parallel task processing capacity, there may still be remaining resources to execute the data processing task. At this time, we try to obtain a second label for the data processing task in the second set, that is, check whether the resource set exceeds the maximum parallel task processing capacity. If a second label is obtained for the data processing task in the second set, it means that the resource set has not exceeded the maximum parallel task processing capacity. At this time, the target processing resource can still be used to execute the data processing operation corresponding to the data processing task to obtain the processing result. However, if a second label is not obtained for the data processing task in the second set, it means that the resource set has exceeded the maximum parallel task processing capacity. At this time, the target processing resource can no longer be used.

[0094] based on Figure 4 In the implementation scheme shown, if it is determined in step 110 that no second tag has been obtained for the data processing task, the method in this embodiment may further include the following steps, such as... Figure 6 As shown:

[0095] Step 112: Discard the data processing task.

[0096] In other words, in this embodiment, if the minimum parallel task processing capacity of the requested target processing resources has been reached, and the maximum parallel task processing capacity of the resource set has been exceeded, even if there is still a remaining redundant parallel task processing capacity, the requested processing resources cannot be used to process the data processing task, and the data processing task is discarded. However, if the maximum parallel task processing capacity of the resource set has not been exceeded, then since there is a remaining redundant parallel task processing capacity, the requested processing resources can be used to process the data processing task to obtain the processing result.

[0097] For example, if a data processing task cannot obtain a Level 1 token from the Level 1 token bucket corresponding to the target processing resource requested by the data processing task, and a Level 3 token cannot be obtained from the Level 3 token bucket, then even if a Level 2 token can be obtained from the Level 2 token bucket, the target processing resource cannot be used to provide processing services for the data processing request in the data processing task; only when a Level 3 token is obtained from the Level 3 token bucket and a Level 2 token is obtained from the Level 2 token bucket can the target processing resource be used to provide processing services for the data processing request in the data processing task in order to obtain the processing result.

[0098] based on Figure 4 The implementation scheme shown, in step 108, determines that no third tag has been obtained for the data processing task. The method in this embodiment may include the following steps: Figure 7 As shown:

[0099] Step 113: Discard the data processing task.

[0100] In other words, in this embodiment, if the minimum parallel task processing capacity of the requested target processing resources has been reached and there is no remaining redundant parallel task processing capacity, it indicates that there are many data processing requests at this time, so that the target processing resources have both reached the minimum parallel task processing capacity and have fully occupied the redundant parallel task processing capacity in the resource set. At this time, it is not necessary to determine whether the maximum parallel task processing capacity has been exceeded, and the data processing tasks can be discarded.

[0101] For example, if a data processing task cannot obtain a first-level token from the first-level token bucket corresponding to the target processing resource requested by the data processing task, and also cannot obtain a second-level token from the second-level token bucket, then the data processing request in the data processing task can be discarded directly.

[0102] based on Figure 4 The implementation scheme shown, in step 108, determines that no third tag has been obtained for the data processing task. The method in this embodiment may include the following steps: Figure 8 As shown:

[0103] Step 114: Cache the data processing task, return to step 107 to wait for a new third label to be issued in the third set, so that if a new third label is obtained for the data processing task, step 109 is executed, that is, a second label is obtained for the data processing task in the second set.

[0104] In other words, in this embodiment, if the minimum parallel task processing capacity of the requested target processing resource has been reached, and there is no remaining redundant parallel task processing capacity, it indicates that there are many data processing requests, causing the target processing resource to both reach the minimum parallel task processing capacity and fully occupy the redundant parallel task processing capacity in the resource set. At this time, the data processing tasks can be cached in a preset storage area to wait for the redundant parallel task processing capacity to be released, that is, to wait for the third tag that can be obtained in the third set. Thus, when the third tag is obtained, that is, when there is remaining redundant parallel task processing capacity that can be occupied, it is then determined whether the maximum parallel task processing capacity has been exceeded. Therefore: if the maximum parallel task processing capacity of the resource set has been exceeded, even if the remaining redundant parallel task processing capacity is occupied, the requested processing resource can no longer be used to process the data processing task, and the data processing task is discarded; if the maximum parallel task processing capacity of the resource set has not been exceeded, then since there is remaining redundant parallel task processing capacity, the requested processing resource can be used to process the data processing task to obtain the processing result.

[0105] For example, if a data processing task cannot obtain a Level 1 token from the Level 1 token bucket corresponding to the target processing resource requested by the data processing task, and also cannot obtain a Level 2 token from the Level 2 token bucket, the data processing task can be cached in a preset storage area to wait for a new Level 2 token to be issued from the Level 2 token bucket. Once a Level 2 token is obtained, it can then attempt to obtain a Level 3 token from the Level 3 token bucket. If a Level 3 token cannot be obtained from the Level 3 token bucket, even if a Level 2 token can be obtained from the Level 2 token bucket, the target processing resource cannot be used to process the data processing request in the data processing task. Only when both a Level 3 token and a Level 2 token are obtained from the Level 2 token bucket can the target processing resource be used to process the data processing request in the data processing task and obtain the processing result.

[0106] based on Figure 4 In the implementation scheme shown, if it is determined in step 105 that no second tag has been obtained for the data processing task, the method in this embodiment may further include the following steps, such as... Figure 9 As shown:

[0107] Step 115: Cache the data processing task to wait for a new second label to be issued in the second set, so that if a new second label is obtained for the data processing task, execute step 106: Execute the data processing operation corresponding to the data processing task based on the target processing resource to obtain the processing result.

[0108] In other words, in this embodiment, if the minimum parallel task processing capacity of the requested processing resources has not been reached, but the maximum parallel task processing capacity of the resource set has been exceeded, it indicates that there are many data processing requests. This means that although the target processing resources have not reached the minimum parallel task processing capacity, they have already fully occupied the maximum parallel task processing capacity in the resource set. At this time, the data processing tasks can be cached in a preset storage area to wait for the maximum parallel task processing capacity to be released, that is, to wait for the second tag that can be obtained in the second set. Thus, when the second tag is obtained, that is, when there is remaining maximum parallel task processing capacity that can be occupied, the requested processing resources can be used to process the data processing tasks to obtain the processing results.

[0109] For example, if a data processing task has already obtained a Level 1 token from the Level 1 token bucket corresponding to the target processing resource requested by the data processing task, but has not obtained a Level 3 token from the Level 3 token bucket, the data processing task can be cached in a preset storage area to wait for a new Level 3 token to be issued from the Level 3 token bucket. Thus, after obtaining the Level 3 token, the target processing resource can be used to provide processing services for the data processing requests in the data processing task to obtain the processing results.

[0110] In one implementation, when obtaining the data processing task in step 101, it can be specifically implemented in the following way, such as... Figure 10 As shown:

[0111] Step 1001: Receive a data processing request, which is used to request the target processing resource.

[0112] Step 1002: Add the received data processing request to the request set corresponding to the target processing resource.

[0113] Each processing resource in the resource set can correspond to a request set. Each request set is used to store the data processing requests corresponding to the respective processing resource, such as... Figure 11 As shown, each processing resource corresponds to a first set and also to a request set, which is used to cache data processing requests for that processing resource.

[0114] It should be noted that each processing resource has its own request set with its own request merging conditions. The request merging conditions are used to indicate when to merge the cached data processing requests in the request set.

[0115] Step 1003: Determine whether the request merging condition corresponding to the request set is met. If the request merging condition is met, proceed to step 1004. If the request merging condition is not met, return to step 1001, that is, continue to wait to receive new data processing requests and add the data processing requests to the request set corresponding to the processing model they requested, until the request merging condition corresponding to the request set is met.

[0116] Step 1004: Merge the data processing requests in the request set to obtain a data processing task, clear the request set, and return to step 1001, that is, continue to wait to receive new data processing requests and add the data processing requests to the request set corresponding to the processing model they requested, until the request merging conditions corresponding to the request set are met, then perform a request merging process to obtain a data processing task, thereby achieving the purpose of batch processing data processing requests.

[0117] In this embodiment, request merging conditions can be configured for processing resources based on their quality of service (QoS) or system load. QoS can be characterized by the response time of the processing resource executing data processing tasks, while system load can be characterized by resource utilization, such as GPU utilization. Specifically, if the target processing resource currently has many data processing requests in its concurrent data processing tasks, its response time will be longer, resulting in a lower QoS. Conversely, if the target processing resource currently has fewer data processing requests in its concurrent data processing tasks, its response time will be shorter, resulting in a higher QoS. Similarly, if the target processing resource currently has many data processing requests in its concurrent data processing tasks, its resource utilization will be higher, resulting in a higher system load. Conversely, if the target processing resource currently has fewer data processing requests in its concurrent data processing tasks, its resource utilization will be lower, resulting in a lower system load.

[0118] In one implementation, the request merging condition may include: the cache duration of the first data processing request added to the request set reaches a preset maximum duration. In other words, if the cache duration of any data processing request in the request set reaches the maximum duration, the data processing requests in the request set can be merged.

[0119] The maximum duration here can be set fixedly based on the resource performance of the target processing resource, or it can be set dynamically based on the running status of the target processing resource. The running status of the target processing resource includes its response time or GPU utilization.

[0120] Based on this, in this embodiment, when the response time of the target processing resource is short or the GPU utilization is low, the maximum duration can be increased, so that the number of data processing requests in the data processing task obtained by merging requests is higher, thereby improving the service quality of the target processing resource; in this embodiment, when the response time of the target processing resource is long or the GPU utilization is high, the maximum duration can be decreased, so that the number of data processing requests in the data processing task obtained by merging requests is lower, thereby ensuring the minimum service quality of the target processing resource.

[0121] In another implementation, the request merging condition may include: the number of data processing requests stored in the request set reaches the maximum number of parallel processing requests corresponding to the target processing resource. In other words, if the number of data processing requests in the request set reaches the maximum number that the target processing resource can support, then the data processing requests in the request set can be merged.

[0122] The maximum number of parallel processing requests can be set fixedly based on the resource performance of the target processing resource, or it can be dynamically set based on the operating status of the target processing resource. For example, in this embodiment, when the response time of the target processing resource is short or the GPU utilization is low, the maximum number of parallel processing requests can be increased, resulting in a higher number of data processing requests in the data processing task obtained by merging requests, thereby improving the service quality of the target processing resource. In this embodiment, when the response time of the target processing resource is long or the GPU utilization is high, the maximum number of parallel processing requests can be decreased, resulting in a lower number of data processing requests in the data processing task obtained by merging requests, thereby ensuring the minimum service quality of the target processing resource.

[0123] refer to Figure 12 This is a schematic diagram of a control device provided in Embodiment 2 of this application. This device can be configured in an electronic device, such as a computer or server. The technical solution in this embodiment is mainly used for flow control of data processing tasks for request processing resources.

[0124] Specifically, the device in this embodiment may include the following units:

[0125] The task acquisition unit 1201 is used to acquire a data processing task; the data processing task is used to request a target processing resource in a resource set, the resource set contains at least one processing resource, each processing resource corresponds to at least a set containing a first tag, and the data of the first tag issued in the first set per unit time is related to the minimum parallel task processing amount of its corresponding processing resource;

[0126] The first acquisition unit 1202 is used to acquire a first tag for the data processing task from the first set corresponding to the target processing resource;

[0127] The second acquisition unit 1203 is used to acquire a second tag for the data processing task in a second set when the first acquisition unit 1202 acquires the first tag for the data processing task. The number of second tags issued per unit time in the second set is related to the maximum parallel task processing capacity of the resource set. The maximum parallel task processing capacity is greater than the sum of the minimum parallel task processing capacity of all the processing resources in the resource set.

[0128] The task execution unit 1204 is used to execute the data processing operation corresponding to the data processing task based on the target processing resource when the second acquisition unit 1203 acquires the second tag for the data processing task, so as to obtain the processing result.

[0129] As can be seen from the above technical solution, in the control device provided in Embodiment 2 of this application, after obtaining a data processing task for requesting a target processing resource, a first tag is obtained for the data processing task in a first set corresponding to the target processing resource. If the first tag is obtained for the data processing task, a second tag is obtained for the data processing task in a second set. Only when the second tag is obtained for the data processing task can the data processing operation corresponding to the data processing task be executed based on the target processing resource to obtain the processing result. The number of first tags issued per unit time in the first set is related to the minimum parallel task processing capacity of its corresponding processing resource, and the number of second tags issued per unit time in the second set is related to the maximum parallel task processing capacity of the resource set containing all processing resources. Therefore, in this embodiment, the target processing resource can only be used to execute the data processing task when both the first and second tags are obtained simultaneously. This ensures that each processing resource in the resource set reaches its minimum parallel task processing capacity and does not exceed the maximum parallel task processing capacity of the resource set. Thus, even if the number of requests fluctuates greatly, it can avoid task congestion of a certain processing resource and exceeding the maximum processing capacity of the resource set, thereby achieving traffic control of requests.

[0130] In one implementation, the apparatus in this embodiment further includes the following units, such as... Figure 13 As shown:

[0131] The third acquisition unit 1205 is configured to: acquire a third tag for the data processing task in a third set when the first acquisition unit 1202 has not acquired the first tag for the data processing task, wherein the number of third tags issued per unit time in the third set is related to the redundant parallel task processing volume corresponding to the maximum parallel task processing volume; and trigger the second acquisition unit 1203 to acquire a second tag for the data processing task in the second set when the third tag is requested for the data processing task.

[0132] In one implementation, the apparatus in this embodiment further includes the following units, such as... Figure 14 As shown:

[0133] The first discarding unit 1206 is used to discard the data processing task when the first acquiring unit 1202 fails to acquire the first tag for the data processing task and the third acquiring unit 1205 fails to acquire the third tag for the data processing task.

[0134] In one implementation, the apparatus in this embodiment further includes the following units, such as... Figure 15 As shown:

[0135] The first caching unit 1207 is used to cache the data processing task when the first acquisition unit 1202 fails to acquire the first tag for the data processing task and the third acquisition unit 1205 fails to acquire the third tag for the data processing task, so as to wait for a new third tag to be issued in the third set, so that when the third acquisition unit 1205 acquires the new third tag for the data processing task, the second acquisition unit 1203 is triggered.

[0136] In one implementation, the apparatus in this embodiment further includes the following units, such as... Figure 16 As shown:

[0137] The second caching unit 1208 is configured to: cache the data processing task when the first acquisition unit 1202 acquires the first tag for the data processing task and the second acquisition unit 1203 does not acquire the second tag for the data processing task, so as to wait for a new second tag to be issued in the second set, so that when the second acquisition unit 1203 acquires the new second tag for the data processing task, the task execution unit 1204 is triggered.

[0138] In one implementation, the apparatus in this embodiment further includes the following units, such as... Figure 17 As shown:

[0139] The second discarding unit 1209 is configured to discard the data processing task when the first acquiring unit 1202 fails to acquire the first tag for the data processing task, the third acquiring unit 1205 acquires the third tag for the data processing task, and the second acquiring unit 1203 fails to acquire the second tag for the data processing task.

[0140] In one implementation, the minimum parallel task processing capacity of the processing resource is obtained based on the maximum parallel task processing capacity and the resource proportion of the processing resource in the resource set; wherein, the resource proportion is the ratio between the amount of video memory required by the processing resource and the sum of the amount of video memory required by all processing resources in the resource set.

[0141] The redundant parallel task processing capacity is obtained by multiplying the maximum parallel task processing capacity by a preset redundancy coefficient.

[0142] In one implementation, the task acquisition unit 1201 is specifically configured to: receive a data processing request, the data processing request being used to request the target processing resource; add the received data processing request to the request set corresponding to the target processing resource; determine whether the request merging condition corresponding to the request set is met; if the request merging condition is met, merge the data processing requests in the request set to obtain a data processing task, clear the request set, and return to execute the step: receiving a data processing request; if the request merging condition is not met, return to execute the step: receiving a data processing request.

[0143] Optionally, the request merging condition includes: the cache duration of the first data processing request added to the request set reaches a preset maximum duration, wherein the maximum duration is set according to the running status of the target processing resource;

[0144] Optionally, the request merging condition includes: the number of data processing requests stored in the request set reaches the maximum number of parallel processing requests corresponding to the target processing resource.

[0145] It should be noted that the specific implementation of each unit in this embodiment can be referred to the corresponding content above, and will not be described in detail here.

[0146] refer to Figure 18 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of this application. The electronic device includes the following structure:

[0147] Memory 1801 is used to store computer programs and data generated by the execution of said computer programs;

[0148] Processor 1802 is configured to execute the computer program to: obtain a data processing task; the data processing task requests a target processing resource from a resource set, the resource set containing at least one processing resource, each processing resource corresponding to at least a first set containing a first tag, the number of first tags issued per unit time in the first set being related to the minimum parallel task processing capacity of its corresponding processing resource; obtain a first tag for the data processing task from the first set corresponding to the target processing resource; if the first tag is obtained for the data processing task, obtain a second tag for the data processing task from a second set, the number of second tags issued per unit time in the second set being related to the maximum parallel task processing capacity of the resource set, the maximum parallel task processing capacity being greater than the sum of the minimum parallel task processing capacities of all processing resources in the resource set; if the second tag is obtained for the data processing task, perform a data processing operation corresponding to the data processing task based on the target processing resource to obtain a processing result.

[0149] As can be seen from the above technical solution, in the electronic device provided in Embodiment 3 of this application, after obtaining a data processing task for requesting a target processing resource, a first tag is obtained for the data processing task in a first set corresponding to the target processing resource. If the first tag is obtained for the data processing task, a second tag is obtained for the data processing task in a second set. Only when the second tag is obtained for the data processing task can the data processing operation corresponding to the data processing task be executed based on the target processing resource to obtain the processing result. The number of first tags issued per unit time in the first set is related to the minimum parallel task processing capacity of its corresponding processing resource, and the number of second tags issued per unit time in the second set is related to the maximum parallel task processing capacity of the resource set containing all processing resources. Therefore, in this embodiment, the target processing resource can only be used to execute the data processing task when both the first and second tags are obtained simultaneously. This ensures that each processing resource in the resource set reaches its minimum parallel task processing capacity and does not exceed the maximum parallel task processing capacity of the resource set. Thus, even if the number of requests fluctuates greatly, it can avoid task congestion of a certain processing resource and exceeding the maximum processing capacity of the resource set, thereby achieving traffic control of requests.

[0150] Taking a cloud platform providing a GPU resource pool as an example, the resources in the GPU resource pool are used for model inference. Users can apply for a GPU resource pool containing multiple GPUs, set the maximum concurrency (i.e., the maximum concurrent task processing capacity, shared by all models), and deploy multiple models in the GPU resource pool. Models execute in parallel, with multiple models running on the same or multiple GPUs. Users can specify the number of instances of a model running on the GPU, and the model inference service is provided externally via a RESTful API. End users can call its RESTful API for model inference. Because each model shares the resources of the GPU resource pool, the call patterns of each model's inference API are unpredictable. To ensure that each model can provide normal external services and that the system will not crash when encountering sudden traffic surges, this application proposes a traffic control method to achieve this goal.

[0151] The following are the core technical points of this application:

[0152] First, for models that support batch processing, this application automatically batches inference requests (i.e., data processing requests) based on service quality (such as model inference response time) and system load (such as GPU utilization). Under the premise of ensuring the minimum service quality, the requests will be automatically combined into the largest possible batch (upper limit is the maximum batch size supported by the model) to improve the overall throughput of the system.

[0153] In addition, this application uses a multi-level token bucket to ensure the minimum service quality of each model and that the overall traffic of each model does not exceed the maximum concurrency. Specifically: the first-level token bucket contains N token buckets, each corresponding to one of the N models, to ensure the minimum service quality of each model; the second-level token bucket contains one or more token buckets, which can be selected according to the actual situation, to handle secondary priority traffic, i.e., inference requests other than the minimum service quality guarantee for each model; and the third-level token bucket contains one token bucket to ensure that the overall traffic of each model does not exceed the maximum concurrency.

[0154] The specific plan is as follows:

[0155] Taking the GPU resource pool as an example, it contains a total of K GPUs. Deploy N models Set the maximum concurrency C, and the number of model instances is... The system automatically calculates the GM of video memory required for a single instance of this model. n The system uses automatic batching and multi-level token buckets to ensure a minimum service quality for each model, improve the overall system throughput, and prevent the overall traffic of each model from exceeding the maximum concurrency, as follows:

[0156] 1. For models that support batch processing, inference requests can be automatically batched based on service quality and system load. While ensuring the minimum service quality, requests can be automatically combined into the largest possible batch to improve the overall throughput of the system.

[0157] (1) The system continuously monitors the model inference request response time and system load. When an inference request arrives, it uses a time series forecasting algorithm (e.g., ARIMA differential integrated moving average autoregressive model) to predict the response time and system load. Figure 19 The flowchart for requesting batching is shown in the figure.

[0158] (2) When the predicted response time is short or the system load is low (below the threshold), increase the maximum cache time of the request (i.e. the maximum duration mentioned above); when the predicted response time is long or the system load is high (not below the threshold), reduce the maximum cache time of the request in order to ensure the minimum quality of service.

[0159] (3) When the request reaches the maximum cache time, or the cached requests reach the maximum batch size supported by the model (i.e. the maximum request processing volume mentioned above), the cached requests are merged into a single batch, and the merged requests enter the first-level token bucket.

[0160] 2. This application uses a multi-level token bucket to ensure a minimum service quality for each model and that the overall traffic of all models does not exceed the maximum concurrency. Figure 20 The flowchart illustrating the flow control of requests is shown below:

[0161] (1) The first-level token bucket contains N token buckets (each token bucket can contain one or more token buckets, which can be selected according to the actual situation; the following uses one token bucket as an example), corresponding to N models, to ensure the minimum service quality of each model. After each inference request (a batched data processing request, i.e., the data processing task mentioned above) arrives, one token is taken. If successful, the request is given a green label and enters the third-level token bucket; otherwise, it enters the second-level token bucket. The CIR of the token bucket, i.e., the rate at which tokens are issued in the token bucket, can be calculated by comprehensively considering factors such as maximum concurrency, required GPU memory, and number of instances. The calculation method can be designed according to the actual situation. For example, the Model can be calculated based on the GPU memory ratio. n CIR n As shown in formula (1) above.

[0162] (2) The secondary token bucket contains one or more token buckets, which can be selected according to the actual situation. The following example uses one token bucket to handle secondary priority traffic, that is, inference requests (entering the secondary token bucket from the primary token bucket) other than the minimum service quality guarantee for each model. After each inference request arrives, one token is taken. If successful, the request is marked with a red tag and enters the tertiary token bucket; otherwise, it is discarded or cached. The CIR of the token bucket, that is, the rate at which tokens are issued in the token bucket, is set to the maximum concurrency multiplied by the coefficient th set by the user. cir ≥1 (less than 1 is also acceptable), as shown in formula (3) above.

[0163] (3) The three-level token bucket contains one token bucket to ensure that the overall traffic of each model does not exceed the maximum concurrency. After each inference request arrives, one token is taken. If successful, inference is performed (i.e., the data processing operation mentioned above). If it fails, green-labeled requests are cached and red-labeled requests are discarded. The CIR of the token bucket, that is, the rate at which tokens are issued in the token bucket, is set to the maximum concurrency as shown in formula (2) above.

[0164] As can be seen, the traffic control scheme proposed in this application for physical GPU model inference scenarios can not only ensure that each model achieves the minimum service quality, but also improve the overall throughput of the system, thereby ensuring that the overall traffic of each model does not exceed the maximum concurrency.

[0165] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0166] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0167] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0168] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A control method, comprising: Obtain data processing tasks; The data processing task is used to request target processing resources in a resource set. The resource set contains at least one processing resource. Each processing resource corresponds to at least one first set containing a first tag. The number of first tags issued per unit time in the first set is related to the minimum parallel task processing volume of its corresponding processing resource. In the first set corresponding to the target processing resources, a first tag is obtained for the data processing task; If the first tag is obtained for the data processing task, a second tag is obtained for the data processing task in the second set. The number of second tags issued per unit time in the second set is related to the maximum parallel task processing capacity of the resource set. The maximum parallel task processing capacity is greater than the sum of the minimum parallel task processing capacity of all the processing resources in the resource set. If the second tag is obtained for the data processing task, the data processing operation corresponding to the data processing task is executed based on the target processing resource to obtain the processing result.

2. The method according to claim 1, wherein if the first tag is not obtained for the data processing task, the method further comprises: A third tag is obtained for the data processing task in the third set, and the number of third tags issued per unit time in the third set is related to the amount of redundant parallel tasks corresponding to the maximum parallel task processing volume. If the third tag is obtained for the data processing task, the following step is performed: Obtain the second tag for the data processing task from the second set.

3. The method according to claim 2, wherein if neither the first tag nor the third tag is obtained for the data processing task, the method further comprises any one of the following: Discard the data processing task; The data processing task is cached to wait for a new third tag to be issued in the third set, so that if the new third tag is obtained for the data processing task, the step of obtaining a second tag for the data processing task in the second set is performed.

4. The method according to claim 1 or 2, wherein if the first tag is obtained for the data processing task but the second tag is not obtained for the data processing task, the method further comprises: The data processing task is cached to wait for a new second tag to be issued from the second set, so that when the new second tag is obtained for the data processing task, the following steps are performed: the data processing operation corresponding to the data processing task is executed based on the target processing resource to obtain the processing result.

5. The method according to claim 2, wherein if the first tag is not obtained for the data processing task, the third tag is not obtained for the data processing task, and the second tag is not obtained for the data processing task, the method further comprises: Discard the data processing task.

6. The method according to claim 2, wherein the minimum parallel task processing capacity of the processing resource is obtained based on the maximum parallel task processing capacity and the resource proportion of the processing resource in the resource set; wherein, The resource ratio is the ratio between the amount of video memory required for the processing resources and the total amount of video memory required for all processing resources in the resource set. The redundant parallel task processing capacity is obtained by multiplying the maximum parallel task processing capacity by a preset redundancy coefficient.

7. The method according to claim 1 or 2, wherein the data processing task includes: Receive a data processing request, the data processing request being used to request the target processing resource; Add the received data processing request to the request set corresponding to the target processing resource; Determine whether the request merging condition corresponding to the request set is met; If the request merging condition is met, merge the data processing requests in the request set to obtain a data processing task, clear the request set, and return to the execution step: receive a data processing request; If the request merging conditions are not met, return to the execution step: receive a data processing request.

8. The method according to claim 7, wherein the request merging conditions include: The cache duration of the first data processing request added to the request set reaches the preset maximum duration, which is set according to the running status of the target processing resource. or, The request merging condition includes: the number of data processing requests stored in the request set reaches the maximum number of parallel processing requests corresponding to the target processing resource.

9. A control device, comprising: The task acquisition unit is used to acquire data processing tasks; The data processing task is used to request target processing resources in a resource set. The resource set contains at least one processing resource. Each processing resource corresponds to at least one first set containing a first tag. The data of the first tag issued in the first set per unit time is related to the minimum parallel task processing amount of its corresponding processing resource. The first acquisition unit is used to acquire a first tag for the data processing task from the first set corresponding to the target processing resource; The second acquisition unit is used to acquire a second tag for the data processing task in a second set when the first acquisition unit acquires the first tag for the data processing task. The number of second tags issued per unit time in the second set is related to the maximum parallel task processing capacity of the resource set. The maximum parallel task processing capacity is greater than the sum of the minimum parallel task processing capacity of all the processing resources in the resource set. The task execution unit is used to execute the data processing operation corresponding to the data processing task based on the target processing resource when the second acquisition unit acquires the second tag for the data processing task, so as to obtain the processing result.

10. An electronic device, comprising: A memory for storing computer programs and the data generated by the execution of said computer programs; A processor is configured to execute the computer program to: obtain a data processing task; the data processing task is configured to request a target processing resource in a resource set, the resource set containing at least one processing resource, each processing resource corresponding to at least a first set containing a first tag, the number of first tags issued per unit time in the first set being related to the minimum parallel task processing capacity of its corresponding processing resource; In the first set corresponding to the target processing resources, a first tag is obtained for the data processing task; if the first tag is obtained for the data processing task, a second tag is obtained for the data processing task in the second set, wherein the number of second tags issued per unit time in the second set is related to the maximum parallel task processing capacity of the resource set, and the maximum parallel task processing capacity is greater than the sum of the minimum parallel task processing capacities of all processing resources in the resource set; if the second tag is obtained for the data processing task, the data processing operation corresponding to the data processing task is executed based on the target processing resources to obtain the processing result.

Images