GPU cache structure, resource allocation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a GPU cache structure, a resource allocation method and device, electronic equipment and a storage medium. The GPU cache structure comprises a private domain space associated with a specified task and a shared space of full tasks. According to the embodiment of the invention, different tasks can be isolated and mutual influence can be avoided through the private domain space associated with the specified task, and different tasks can be used and GPU resources can be utilized to the maximum extent through the shared space of the total tasks, so that resource fragmentation caused by different tasks can be reduced, and the resource utilization rate can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cloud computing, and in particular to a GPU cache structure, a resource allocation method and device, an electronic device and a storage medium. BACKGROUND

[0002] At present, an intelligent application platform supports various application creation scenarios, such as model training, model debugging, online inference and the like. Many of the applications depend on GPU creation services, but GPU cards are limited, and the cost of increasing GPU cards is high, which cannot meet all application deployment tasks. Different application scenarios require different deployment specifications and deployment quantities for GPU application tasks, some tasks only require one card, and some tasks require eight cards to be deployed simultaneously. With frequent creation and deletion of applications, GPU fragmentation is inevitable, resulting in that some tasks cannot find available resources and the resource utilization rate is low. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide a GPU cache structure, a resource allocation method and device, an electronic device and a storage medium to improve resource utilization. The specific technical solutions are as follows: In the first aspect of the present application, a GPU cache structure is first provided, comprising a private domain space associated with a specified task and a shared space of all tasks.

[0004] In the second aspect of the present application, a GPU cache resource allocation method is further provided, comprising: receiving a task to be executed; determining allocation resources from the private domain space and / or the shared space in the GPU cache structure according to the identification information of the task to be executed; allocating the allocation resources to the task to be executed.

[0005] In the third aspect of the present application, a GPU cache resource allocation device is further provided, comprising: a task receiving module for receiving a task to be executed; a resource determining module for determining allocation resources from the private domain space and / or the shared space in the GPU cache structure according to the identification information of the task to be executed; a resource allocation module for allocating the allocation resources to the task to be executed.

[0006] In another aspect of the present application, an electronic device is further provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus; the memory is used to store a computer program; A processor is configured to implement the steps of the GPU cache resource allocation method described above when executing a program stored in a memory.

[0007] In another aspect of the present application, a computer readable storage medium is provided, which stores instructions that, when executed on a computer, cause the computer to perform the GPU cache resource allocation method described above.

[0008] In another aspect of the present application, a computer program product is provided, which contains instructions that, when executed on a computer, cause the computer to perform the GPU cache resource allocation method described above.

[0009] The GPU cache structure, resource allocation method, device, and storage medium provided by the embodiments of the present application can isolate different tasks through the private space associated with a specified task, avoid mutual influence, and enable different tasks to use the shared space of the full task, thereby maximizing the use of GPU resources, reducing resource fragmentation caused by different tasks, and improving resource utilization. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below.

[0011] Figure 1 is a schematic diagram of a GPU cache structure provided by the embodiments of the present application; Figure 2 is a flowchart of a GPU cache resource allocation method provided by the embodiments of the present application; Figure 3 is an example diagram of an application scenario of the embodiments of the present application; Figure 4 is an example diagram of a GPU cache structure in a cluster in the embodiments of the present application; Figure 5 is a flowchart of a GPU cache resource allocation method in the embodiments of the present application; Figure 6 is a structural schematic diagram of a GPU cache resource allocation device provided by the embodiments of the present application; Figure 7 is a structural schematic diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0012] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

[0013] Figure 1is a schematic diagram of a GPU cache structure provided by an embodiment of the present application, as shown in the figure, the GPU cache structure 100 includes a private domain space 110 associated with a specified task and a shared space 120 of all tasks. Figure 1

[0014] The private domain space can also be referred to as a private resource pool, and includes GPU resources that can be used by the specified task. The shared space can also be referred to as a shared resource pool, and includes GPU resources that can be used by all tasks.

[0015] The association of the specified task and the private domain space can be associated by the task identifier of the specified task and the resource identifier of the private domain space, or can also be associated by the business identifier corresponding to the specified task and the resource identifier of the private domain space.

[0016] The GPU cache structure can be a GPU cache structure in an electronic device containing a GPU component, or can also be a GPU cache structure in a cluster composed of multiple electronic devices containing a GPU component.

[0017] The GPU cache structure provided by the embodiment of the present application can isolate different tasks through the private domain space associated with the specified task, avoid mutual influence, and use different tasks through the shared space of all tasks, so as to maximize the use of GPU resources, thereby reducing resource fragmentation caused by different tasks and improving resource utilization.

[0018] On the basis of the above technical solution, the resource allocation principle of the shared space is to determine the resource allocation order based on the task priority. When there are multiple tasks that need to use the shared space, the resources in the shared space can be allocated to each task according to the priority of each task, that is, the resources in the shared space are allocated to the task with higher priority first. In this way, the task priority can coordinate the order of using the shared space by different tasks.

[0019] On the basis of the above technical solution, the task type of the specified task or the all tasks includes a model inference task. The model inference task can include a model training task, a model debugging task, a model online application task and other tasks related to machine learning models. The machine learning model can be a neural network model or other machine learning model. The neural network model can be a video processing model, an image processing model, a speech processing model, etc. The video processing model can be used to remove sensitive information, redundant information, etc. in the video. The speech processing model can be used to convert one language into another language, such as converting Chinese into English, etc.

[0020] Figure 2 ​This is a flowchart illustrating a GPU cache resource allocation method provided in an embodiment of the present invention. This GPU cache resource allocation method can be applied to electronic devices such as computers, mobile phones, and servers, or it can also be applied to service clusters, such as... Figure 2 As shown, the method includes: Step 210: Receive the task to be executed.

[0021] It can receive tasks to be executed from business partners. These tasks include model inference tasks, which can include tasks for deploying model training, model debugging, online inference, and other applications. Model inference tasks are related to machine learning models, which can be neural network models or other types of machine learning models. Neural network models can be, for example, video processing models, image processing models, or speech processing models. Video processing models can be used to remove sensitive or redundant information (such as TV station logos) from videos, while speech processing models can be used to convert one language to another, such as converting Chinese to English.

[0022] Figure 3 This is an example diagram illustrating an application scenario of an embodiment of the present invention, such as... Figure 3 As shown, the application scenarios of the GPU cache resource allocation method provided in this embodiment of the invention can include model training, model inference, model delivery, model validation, etc. For each scenario, the application platform can uniformly receive tasks to be executed and prepare instances, in the corresponding cluster ( Figure 3 (Taking cluster 1 and cluster 2 as examples) or a single electronic device, the task to be executed is performed and the corresponding service instance (i.e., application) is created. Figure 3 In this example, we have deployed Service 1 instance and Service 2 instance in Cluster 1, and Service 3 instance and Service 4 instance in Cluster 2.

[0023] Step 220: Based on the identification information of the task to be executed, determine the allocation of resources from the private space and / or shared space in the GPU cache structure.

[0024] Private space, also known as a private resource pool, includes GPU resources available for specific tasks. Shared space, also known as a shared resource pool, includes GPU resources available for all tasks.

[0025] A GPU cache structure can be a GPU cache structure in an electronic device that contains a GPU component, or it can be a GPU cache structure in a cluster of multiple electronic devices that contain GPU components.

[0026] Each to-be-executed task can be associated with a specific private domain space, and the association of the to-be-executed task and the private domain space can be performed by associating the task identifier of the to-be-executed task with the resource identifier of the private domain space, or by associating the business identifier corresponding to the to-be-executed task with the resource identifier of the private domain space.

[0027] The identification information of the to-be-executed task can include the task identifier of the to-be-executed task or the business identifier corresponding to the to-be-executed task.

[0028] Figure 4 is an example diagram of a GPU cache structure in an embodiment of the application, as shown in Figure 4 The GPU cache structure in an embodiment of the application can include a private domain space and a shared space, Figure 4 two private domain spaces and one shared space are taken as examples.

[0029] The identification information of the to-be-executed task corresponds to a private domain space, and the private domain space corresponding to the identification information is used by the to-be-executed task corresponding to the identification information (such as deploying an application, etc.), and different task application creation can be isolated by the private domain space to avoid mutual influence. The resources in the shared space can be used by all tasks to maximize the use of GPU resources.

[0030] According to the identification information of the to-be-executed task, the allocation resource satisfying the required resource amount of the to-be-executed task is determined from the private domain space and / or the shared space corresponding to the identification information in the GPU cache structure. The allocation resource can be first determined from the private domain space, and when the resources in the private domain space are insufficient, the allocation resource can be determined from the shared space.

[0031] The determined allocation resource can be a GPU card resource in an electronic device, or can also be a GPU card resource in a target device identifier in a cluster. When applied in a cluster, the number of determined target device identifiers can be one or multiple, and is related to the instance number and instance specification contained in the application deployment request requested by the deployed application, and the instance specification can include the number of GPUs, the number of CPUs, the size of memory, etc.

[0032] Step 130, allocating the allocation resource to the to-be-executed task.

[0033] The determined allocation resource is allocated to the to-be-executed task to execute the to-be-executed task using the allocation resource.

[0034] The GPU cache resource allocation method provided by the embodiment of the present application can determine the allocation resource from the private space and / or the shared space in the GPU cache structure according to the identification information of the to-be-executed task when the to-be-executed task is received, and allocate the allocation resource to the to-be-executed task. By dividing the GPU cache structure into the private space and the shared space, different tasks can be isolated, and part of the resources can be shared between the tasks, so that the resource fragmentation caused by different tasks can be reduced, and the resource utilization rate can be improved.

[0035] On the basis of the above technical solution, the determining the allocation resource from the private space and / or the shared space in the GPU cache structure according to the identification information of the to-be-executed task comprises: determining the allocation resource from the target private space corresponding to the identification information according to the identification information of the to-be-executed task and resource configuration information.

[0036] The resource configuration information represents the resource information required by the to-be-executed task, and can include the required GPU card quantity, CPU quantity, memory, environment variable and the like, for example, one GPU card, two GPU cards or the like, or can also include the instance quantity to be deployed by the to-be-executed task (the GPU card quantity required by each instance is the same).

[0037] The identification information of the to-be-executed task and the resource configuration information can be used to first determine whether the remaining resource in the target private space corresponding to the identification information meets the required resource quantity in the resource configuration information, and if the first remaining resource quantity in the target private space is greater than or equal to the required resource quantity in the resource configuration information, the allocation resource of the required resource quantity can be determined from the target private space.

[0038] The second remaining resource information corresponding to the second device identification can be the remaining GPU card quantity corresponding to the second device identification.

[0039] Exemplarily, when the GPU resources in a cluster need to be allocated, first, it can be determined whether the devices in the target private domain space can meet the resource configuration information based on a machine allocation algorithm, the resource configuration information can be compared with the first residual resource information corresponding to each first device identifier in the target private domain space, and the target device identifier of the allocated resource is determined based on the comparison result. Exemplarily, when the resource configuration information includes the configuration information of one instance or multiple instances, the configuration information of each instance can be compared with the first residual resource information corresponding to each first device identifier in the target private domain space, and the device identifier whose first residual resource information is greater than or equal to the configuration information of the instance is determined as the target device identifier. If one or more instances can be configured in the target private domain space, but there are remaining instances without resources for configuration, the remaining instances can be configured in the subsequent shared space; if the target private domain space cannot meet any instance, all instances can be configured in the subsequent shared space. Wherein, the first residual resource information corresponding to the first device identifier can be the number of residual GPU cards corresponding to the first device identifier.

[0040] By determining the allocated resource from the target private domain space corresponding to the identification information according to the identification information of the to-be-executed task and the resource configuration information, different tasks can be isolated through the private domain space, mutual influence between tasks is avoided, and resource stability is improved.

[0041] On the basis of the above technical solutions, the determining the allocated resource from the private domain space and / or the shared space in the GPU cache structure according to the identification information of the to-be-executed task includes: According to the identification information of the to-be-executed task and the resource configuration information, when the first residual resource amount in the target private domain space corresponding to the identification information is less than the required resource amount in the resource configuration information, the allocated resource can be determined from the shared space.

[0042] Exemplarily, when the GPU resources in a cluster need to be allocated, and the first device identifiers in the target private domain space are insufficient in the first residual resource information corresponding thereto, i.e., the target private domain space can configure part of the instances of the to-be-executed task or completely cannot configure all the instances of the to-be-executed task, the target device identifier for allocating resources can be determined in the shared space based on a machine allocation algorithm. The resource configuration information can be compared with the second residual resource information corresponding to each second device identifier in the shared space, and the target device identifier is determined based on the comparison result. Exemplarily, the configuration information of the residual instances in the resource configuration information can be compared with the second residual resource information corresponding to each second device identifier in the shared space respectively, and the device identifier whose second residual resource information is greater than or equal to the configuration information of the residual instances is determined as the target device identifier. The second residual resource information corresponding to the second device identifier can be the number of residual GPU cards corresponding to the second device identifier.

[0043] By first determining the allocation of resources in the private domain space and then determining the allocation of resources from the shared space when the resources in the private domain space are insufficient, different tasks can be isolated in the private domain space to avoid mutual influence between tasks, improve resource stability, and the shortage of the private domain space can be made up through the shared space, thereby solving the problem of resource contention between tasks and improving resource utilization.

[0044] On the basis of the above technical solution, the number of to-be-executed tasks is multiple; The determining of the allocation of resources from the shared space includes: determining the allocation of resources of each to-be-executed task in the shared space in the order of the priority of each to-be-executed task.

[0045] When multiple to-be-executed tasks need to use the resources in the shared space at the same time, the allocation of resources of each to-be-executed task in the shared space can be determined in the order of the priority of each to-be-executed task, i.e., the resources in the shared space are allocated to the to-be-executed tasks with high priority first, so that the order of using the shared space by different tasks can be coordinated through the priority of the tasks.

[0046] On the basis of the above technical solution, the determining of the allocation of resources from the private domain space and / or the shared space in the GPU cache structure according to the identification information of the to-be-executed task includes at least one of the following: determining the difference between the residual resource amount corresponding to the target private domain space and / or the shared space corresponding to the identification information and the required resource amount in the resource configuration information according to the identification information and the resource configuration information, and determining the allocation of resources from the target private domain space and / or the shared space with the smallest difference; determine the allocation resource according to distances between resources in the target private domain space and / or the shared space based on the resource configuration information; determine the allocation resource according to distances between resources in the target private domain space and / or the shared space based on the resource configuration information; determine the allocation resource according to distances between resources in the target private domain space and / or the shared space based on the resource configuration information;

[0047] The resource identifier can be a specific GPU card identifier, or can also be a device identifier in the cluster.

[0048] The embodiment mainly illustrates resource allocation, that is, allocating corresponding resources to the to-be-executed task. Here, a resource allocation algorithm can be used for allocation. The resource allocation algorithm can include at least one of a best fit algorithm, a best performance algorithm, a round robin allocation algorithm, and a sharding allocation algorithm.

[0049] The best fit algorithm will preferentially determine the allocation resource from the target private domain space and / or the shared space with the smallest difference between the remaining resource amount and the required resource amount in the resource configuration information. In this way, resource waste can be avoided, and resource fragmentation can be reduced.

[0050] For example, when applied to a cluster, the best fit algorithm will preferentially allocate resources in a device with the smallest difference between the required resource amount in the resource configuration information and the remaining resource of the device. In this way, resource waste can be avoided, and device fragmentation can be reduced. That is, for each device identifier in the target private domain space and the shared space, the difference between the remaining resource amount corresponding to the device identifier and the required resource amount in the resource configuration information is calculated. From these device identifiers, the device identifier with a difference greater than or equal to 0 and the smallest difference is determined as the target device identifier, and the resources in the device corresponding to the target device identifier are used as the allocation resource.

[0051] Considering the overall performance of the service, the best performance algorithm can tend to deploy application instances of the same service on the same or similar devices to reduce network delay and improve service response speed. That is, based on the distances between devices where resources in the private domain space and the shared space are located (such as machine room information of the devices), based on the instance quantity in the resource configuration information, the allocation resource is determined from the devices with closer distances. For example, the identifiers of the devices with the instance quantity in the same machine room or adjacent machine rooms are determined as the target device identifiers, and the resources in the devices corresponding to the target device identifiers are used as the allocation resource.

[0052] The best performance algorithm analyzes the resource utilization in the target private domain space and / or the shared space, determines the resource identifier of the to-be-configured instance quantity in the order from low to high resource utilization as the allocation resource. For example, when applied to a cluster, the best performance algorithm analyzes the resource utilization (CPU, GPU, etc.) of each device, and the algorithm deploys instances on each device in a round-robin manner to balance the load of each device and avoid single-point overload. That is, the resource utilization of each device identifier in the target private domain space and / or the shared space can be sorted in the order from low to high, and the device identifier of the to-be-configured instance quantity in the front can be selected as the target device identifier, and the resource in the device corresponding to the target device identifier can be used as the allocation resource.

[0053] The sharding allocation algorithm considers that different service applications corresponding to different tasks have different life cycles. The algorithm allocates resources in a sharding manner according to the priority and life cycle of the task to avoid mixed deployment of applications of different priority tasks. That is, each priority task can correspond to a resource identifier of the corresponding priority. Based on the target priority of the to-be-executed task, the resource identifier corresponding to the target priority can be determined from the target private domain space and / or the shared space as the allocation resource. For example, when applied to a cluster, each priority task can correspond to a device identifier that can execute the task. When receiving a to-be-executed task, the device identifier corresponding to the target priority can be determined from the target private domain space and / or the shared space according to the target priority of the to-be-executed task, the device identifier is used as the target device identifier, the resource identifier is determined from the target device identifier as the allocation resource. In this way, high-priority tasks can be deployed on the same device, and low-priority tasks can be deployed on the same device, which can reduce the migration of applications for devices deploying high-priority tasks.

[0054] It should be noted that in the above resource allocation process, the allocation resource is first determined from the target private domain space, and when the resources in the target private domain space do not meet the requirements of the resource configuration information, the allocation resource is determined from the shared space.

[0055] On the basis of the above technical solutions, the allocation resource is determined from the private domain space and / or the shared space in the GPU cache structure according to the identification information of the to-be-executed task, that is, the step 120 can include: when the first remaining resource quantity in the target private domain space corresponding to the identification information and the second remaining resource quantity of the shared space are both less than the required resource quantity in the resource configuration information, migrating the allocated task in the private domain space and the shared space; and determining the allocation resource in the private domain space and / or the shared space after migrating the allocated task according to the identification information and the resource configuration information.

[0056] The remaining resource amount can be the number of remaining GPU cards, i.e., the number of idle GPU cards.

[0057] In determining the allocated resources, the first remaining resource amount in the target private domain space and the second remaining resource amount in the shared space can be compared with the required resource amount in the resource configuration information respectively. If both the first remaining resource amount and the second remaining resource amount are less than the required resource amount in the resource configuration information, the migratable tasks can be determined from the allocated tasks in the private domain space and the shared space, and the migratable tasks are migrated, so that the resources in the private domain space or the shared space after the migration of the tasks can meet the resource requirement of the to-be-executed tasks. In this way, after the migration of the tasks, the allocated resources can be determined again from the target private domain space and / or the shared space.

[0058] For example, when applied to a cluster, in determining the target device identifier corresponding to the allocated resources, the remaining resource amount corresponding to each device identifier in the target private domain space and the shared space can be compared with the required resource amount in the resource configuration information respectively. If the remaining resource amount corresponding to each device identifier is less than the required resource amount in the resource configuration information, i.e., the resources in the target private domain space and the shared space are insufficient to execute the to-be-executed tasks, the migratable tasks can be determined from the allocated tasks in the private domain space and the shared space, and the migratable tasks are migrated, so that the resources in the devices after the migration of the tasks can meet the resource requirement of the to-be-executed tasks. In this way, after the migration of the tasks, the remaining resource amount corresponding to each device identifier in the target private domain space and the shared space can be determined again, and the device identifier with the remaining resource amount greater than the required resource amount is determined as the target device identifier.

[0059] By migrating the allocated tasks, the fragmentation is arranged. The fragmentation arrangement algorithm is an optimization strategy started when the resource allocation algorithm cannot find suitable resources (i.e., suitable resources cannot be found directly from the target private domain space and / or the shared space). The main purpose is to release enough space to meet the resource requirement of the to-be-executed tasks by reorganizing the existing resources.

[0060] By migrating the allocated tasks in the private domain space and the shared space when the first remaining resource amount in the target private domain space and the second remaining resource amount in the shared space are both less than the required resource amount in the resource configuration information, and determining the allocated resources from the private domain space and / or the shared space after the migration of the tasks, the fragmentation of the resources in each private domain space and shared space can be arranged in time, and the resource utilization rate is improved.

[0061] In some embodiments of the application, the migration of the allocated tasks in the private domain space and the shared space can include: determining a task to be migrated and a new space identifier corresponding to the task to be migrated according to the priority and the life cycle of the allocated tasks; determining the remaining resource information of the private domain space and the shared space and the recovered resource information corresponding to the task to be migrated; determining a migration space identifier according to the resource configuration information, the remaining resource information and the recovered resource information, and migrating the task to be migrated corresponding to the migration space identifier to the private domain space or the shared space corresponding to the new space identifier.

[0062] When applied to a cluster, if there is no available device, an available device can be found according to the principle of minimum cost. At the same time, by using a distributed lock mechanism, a distributed lock identifier is added to the device of the task to be migrated when the task is migrated, so as to avoid the problem of device contention. The minimum cost means that the tasks are evaluated according to the urgency and the number of occupied resources, for example, tasks with high priority are avoided to be migrated, and tasks of the synchronous type usually have higher priority than tasks of the production type. The synchronous type task is an application task that provides real-time service. The production type task is an application task that is completed within a certain time range.

[0063] When the application is migrated, the tasks to be migrated are first determined to identify the tasks that can be migrated, and the new deployment positions of the tasks are planned, that is, the new space identifiers of the tasks to be migrated are determined. This step needs to consider the priority of the tasks, the life cycle of the tasks and other factors to ensure that the tasks can normally operate after migration. For example, the tasks to be migrated are preferentially selected from tasks with low priority; or when the priorities are the same, the tasks with short life cycle are preferentially selected; or the tasks with small amount of occupied resources are preferentially selected, so that the migration cost of the tasks can be reduced.

[0064] Secondly, the recoverable resources are counted, that is, the remaining resource information of the private domain space and the shared space and the recovered resource information corresponding to the task to be migrated are counted.

[0065] Finally, the task recovery is performed, and the tasks of the private domain space and / or the shared space can be recovered according to whether the resource requirements of the tasks to be executed can be met. That is, according to the resource configuration information, the remaining resource information of each private domain space and the shared space, and the recovered resource information, it is determined whether the private domain space and / or the shared space after recovery of the resources can meet the resource requirements of the tasks to be executed, the space identifier that can meet the resource requirements of the tasks to be executed is determined as a migration space identifier, and the task to be migrated corresponding to the migration space identifier is migrated to the private domain space and / or the shared space corresponding to the new space identifier. When applied to a cluster, the space identifier can be a device identifier in the cluster.

[0066] The to-be-migrated task and the new space identifier are determined based on the priority and the life cycle of the allocated task, the resource information corresponding to each space identifier can be determined based on the to-be-migrated task, and then the migration space identifier can be determined based on the resource configuration information, the remaining resource information and the recycled resource information, and the migration of the task is performed, so that the fragmentation of the resources can be arranged in time, and the to-be-executed task can be executed based on the arranged resources, and the resource utilization rate can be improved.

[0067] In some embodiments of the present application, the determining of the migration space identifier based on the resource configuration information, the remaining resource information and the recycled resource information can include: determining the space identifier whose sum of the remaining resource information and the recycled resource information is greater than or equal to the resource configuration information as the migration space identifier.

[0068] For each space identifier, the sum of the remaining resource information and the recycled resource information can be calculated as the available resource amount corresponding to the space identifier, the available resource amount is compared with the resource configuration information, the space identifier whose available resource amount is greater than or equal to the required resource amount in the resource configuration information is determined, the migration space identifier is determined from these space identifiers, and the distributed lock is set for the migration space identifier, indicating that the empty resource after this migration belongs to the corresponding to-be-executed task. For example, when applied to a cluster, the migration space identifier is determined from the space identifiers (i.e. the device identifiers in the cluster) whose available resource amount is greater than the required resource amount in the resource configuration information, and the space identifier with the smallest difference between the available resource amount and the required resource amount can be determined as the migration space identifier; or one or more space identifiers whose available resource amount is greater than the required resource amount are selected as the migration space identifier.

[0069] The space identifier whose sum of the remaining resource information and the recycled resource information is greater than or equal to the resource configuration information is determined as the migration space identifier, so that the private space or the shared space after the migration of the task can meet the resource requirement of the to-be-executed task, and the resource utilization rate is improved.

[0070] In some embodiments of the present application, the determining of the allocated resource in the private space and / or the shared space after the migration of the allocated task based on the identifier information and the resource configuration information can include: determining the allocated resource from the private space or the shared space corresponding to the migration space identifier.

[0071] After the to-be-migrated task corresponding to the migration space identifier is migrated, the remaining resource (the sum of the original remaining resource information and the recycled resource information) corresponding to the migration space identifier can meet the resource requirement of the to-be-executed task, and the allocated resource can be determined from the private space or the shared space corresponding to the migration space identifier.

[0072] By determining the allocated resources in the private domain space or the shared space corresponding to the migration space identification after the migration of the task corresponding to the migration space identification, the to-be-executed task can be executed by the allocated resources in the private domain space or the shared space after the migration, so that the resource utilization rate can be improved.

[0073] Figure 5 is a flowchart of a GPU cache resource allocation method in an embodiment of the application, as shown in Figure 5 When a to-be-executed task is received, the to-be-executed task is application creation (instance creation in an application), resource configuration information can be obtained by reading the creation configuration; a cluster is allocated to the to-be-executed task, and device allocation is performed in the cluster, the device allocation can be performed by using the above resource allocation algorithm (at least one of the best fit algorithm, the best performance algorithm, the round robin allocation algorithm, and the sharding allocation algorithm), if an available device is found, application creation can be performed, that is, application deployment is performed, if no available device is found, fragmentation arrangement is performed in the case that the device allocation does not time out, and the allocation is performed again according to the resource allocation algorithm after the fragmentation arrangement, in the case that the device allocation times out, it is determined that the application creation fails, and the result is returned to the requester of the to-be-executed task after the application creation succeeds or fails.

[0074] In an embodiment of the application, all GPU resources are uniformly divided into a private domain space and a shared space according to types, the private domain space ensures the mutual isolation of different tasks, the shared space ensures the stability of business application creation, improves the utilization rate of computing resources, and through fragmentation arrangement, the computing resources can be optimized and the utilization rate of the computing resources can be improved.

[0075] Figure 6 is a structural schematic diagram of a GPU cache resource allocation device provided by an embodiment of the application, as shown in Figure 6 The device comprises: A task receiving module 610 is configured to receive a to-be-executed task. A resource determining module 620 is configured to determine allocated resources from a private domain space and / or a shared space in a GPU cache structure according to identification information of the to-be-executed task. A resource allocation module 630 is configured to allocate the allocated resources to the to-be-executed task.

[0076] Optionally, the resource determining module is specifically configured to: determine the allocated resources from a target private domain space corresponding to the identification information according to the identification information of the to-be-executed task and resource configuration information.

[0077] Optionally, the resource determining module is specifically configured to: determining the allocated resource from the shared space according to the identification information and the resource configuration information when the first remaining resource amount in the target private domain space corresponding to the identification information is less than the required resource amount in the resource configuration information.

[0078] Optionally, the number of the to-be-executed tasks is a plurality. The determining the allocated resource from the shared space comprises: determining the allocated resource of each to-be-executed task in the shared space in order of the priority of each to-be-executed task.

[0079] Optionally, the resource determining module is specifically configured to: migrating the allocated tasks in the private domain space and the shared space according to the identification information and the resource configuration information when the first remaining resource amount in the target private domain space corresponding to the identification information and the second remaining resource amount in the shared space are both less than the required resource amount in the resource configuration information. determining the allocated resource in the private domain space and / or the shared space after migrating the allocated tasks according to the identification information and the resource configuration information.

[0080] Optionally, the resource determining module is configured to perform at least one of the following: determining the difference between the required resource amount in the resource configuration information and the remaining resource amount corresponding to the target private domain space and / or the shared space corresponding to the identification information according to the identification information and the resource configuration information, and determining the allocated resource from the target private domain space and / or the shared space with the smallest difference; determining the allocated resource according to the distance between resources in the target private domain space and / or the shared space based on the resource configuration information; determining the resource identification of the to-be-configured instance number as the allocated resource in order of the resource utilization from low to high in the target private domain space and / or the shared space according to the to-be-configured instance number in the resource configuration information; determining the resource identification corresponding to the target priority as the allocated resource from the target private domain space and / or the shared space according to the target priority corresponding to the to-be-executed task.

[0081] The GPU cache resource allocation device provided by the embodiment of the present application can determine the allocation resource from the private space and / or the shared space in the GPU cache structure according to the identification information of the to-be-executed task when receiving the to-be-executed task, and allocate the allocation resource to the to-be-executed task. By dividing the GPU cache structure into the private space and the shared space, different tasks can be isolated, and part of resources can be shared between the tasks, so that the resource fragmentation caused by different tasks can be reduced, and the resource utilization rate can be improved.

[0082] The embodiment of the present application further provides an electronic device, as shown in the figure, Figure 7 The processor 701, the communication interface 702 and the memory 703 can communicate with each other through the communication bus 704, The memory 703 is used for storing a computer program. The processor 701 is used for executing the program stored in the memory 703, and the following steps are realized: Receiving a to-be-executed task; According to the identification information of the to-be-executed task, the allocation resource is determined from the private space and / or the shared space in the GPU cache structure. The allocation resource is allocated to the to-be-executed task.

[0083] The communication bus mentioned in the above electronic device can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0084] The communication interface is used for the communication between the above electronic device and other devices.

[0085] The memory can include a random access memory (RAM) and can also include a non-volatile memory, for example, at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0086] The processor described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; or can be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0087] In yet another embodiment provided by the present application, a computer readable storage medium is provided, which stores instructions, when executed on a computer, cause the computer to perform the GPU cache resource allocation method according to any one of the above embodiments.

[0088] In yet another embodiment provided by the present application, a computer program product is provided, which contains instructions, when executed on a computer, cause the computer to perform the GPU cache resource allocation method according to any one of the above embodiments.

[0089] In the above embodiments, the implementation can be achieved entirely or partially by software, hardware, firmware or any combination thereof. When implemented by software, the implementation can be achieved entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the flow or function described in the embodiments of the present application is entirely or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, DVD), or a semiconductor medium (for example, solid state disk (SSD)) and the like.

[0090] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0091] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0092] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A GPU cache structure, comprising: include: The private space associated with a specific task and the shared space for all tasks.

2. The GPU cache structure of claim 1, wherein, The resource allocation principle of the shared space is based on the task priority to determine the order of resource allocation.

3. The GPU cache structure of claim 1, wherein, The task type of the specified task or the full task includes model inference tasks.

4. A method for allocating GPU cache resources, the method comprising: include: Receive tasks to be executed; Based on the identification information of the task to be executed, resources are allocated from the private and / or shared spaces in the GPU cache structure. The allocated resources are allocated to the task to be executed.

5. The method of claim 4, wherein, The step of determining and allocating resources from the private and / or shared space in the GPU cache structure based on the identification information of the task to be executed includes: Based on the identification information and resource configuration information of the task to be executed, the allocated resources are determined from the target private space corresponding to the identification information.

6. The method according to claim 4 or 5, characterized in that, The step of determining and allocating resources from the private and / or shared space in the GPU cache structure based on the identification information of the task to be executed includes: Based on the identification information and resource configuration information of the task to be executed, when the first remaining resource amount in the target private domain space corresponding to the identification information is less than the required resource amount in the resource configuration information, the allocated resources are determined from the shared space.

7. The method of claim 6, wherein, The number of tasks to be executed is multiple; Determining the allocated resources from the shared space includes: Based on the priority order of each task to be executed, the resources allocated to each task in the shared space are determined.

8. The method according to claim 4 or 5, characterized in that, The step of determining and allocating resources from the private and / or shared space in the GPU cache structure based on the identification information of the task to be executed includes: Based on the identification information and resource configuration information of the task to be executed, when both the first remaining resource amount in the target private space corresponding to the identification information and the second remaining resource amount in the shared space are less than the required resource amount in the resource configuration information, the assigned tasks in the private space and the shared space are migrated. Based on the identification information and the resource configuration information, the allocated resources are determined in the private and / or shared spaces after the allocated tasks have been migrated.

9. The method of claim 4, wherein, The step of determining and allocating resources from the private and / or shared space in the GPU cache structure based on the identification information of the task to be executed includes at least one of the following: Based on the identification information and the resource configuration information, the difference between the remaining resource amount corresponding to the target private space and / or shared space corresponding to the identification information and the required resource amount in the resource configuration information is determined respectively, and the allocated resources are determined from the target private space and / or shared space with the smallest difference. Based on the resource configuration information, the allocated resources are determined according to the distance between each resource in the target private space and / or shared space; Based on the number of instances to be configured in the resource configuration information, the resource identifiers of the number of instances to be configured are determined in order of resource utilization from low to high in the target private space and / or shared space, and are used as the allocated resources; According to the target priority corresponding to the task to be executed, a resource identifier corresponding to the target priority is determined from the target private domain space and / or the shared space as the allocation resource.

10. An apparatus for allocating GPU cache resources, the apparatus comprising: Comprise: A task receiving module for receiving a task to be executed; A resource determining module for determining an allocation resource from a private domain space and / or a shared space in a GPU cache structure according to identification information of the task to be executed; A resource allocation module for allocating the allocation resource to the task to be executed.

11. An electronic device, comprising: Comprise a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus; The memory is used for storing a computer program; The processor is used for executing the program stored on the memory, and realizes the method steps in any one of claims 4-9.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to realize the method in any one of claims 4-9.

13. A computer program product comprising computer program instructions, characterised in that, The computer program instructions run on the computer to make the computer execute the method in any one of claims 4-9. The computer program instructions run on the computer to make the computer execute the method in any one of claims 4-9.

Images