Resource management method and system, program product and electronic equipment

By virtualizing the TPU device into computing resources and allocating target computing resources according to the needs of the container, the problem of difficult to share TPU devices between containers is solved, and the utilization rate of computing resources is improved.

CN118916170APending Publication Date: 2024-11-08CHINA TELECOM CORP LTD TECHNOLOGY INNOVATION CENTER +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411081829.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

It is difficult to share the computing resources of the TPU device between multiple containers, resulting in the inability to fully utilize the TPU device and the utilization rate of the computing resource is low.

Method used

By virtualizing the TPU device into a computing resource, the resource usage information of the container is obtained, the target computing resource is determined, and the configuration information is updated based on this information to bind the container and the execution unit, so that the container can share the computing resource.

Benefits of technology

The computing resource utilization rate of TPU devices is improved, resource sharing between containers is realized, and the overall utilization efficiency of computing resources is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916170A_ABST
    Figure CN118916170A_ABST
Patent Text Reader

Abstract

The invention provides a resource management method and system, a program product and electronic equipment, and relates to the field of wireless communication. The resource management method comprises the following steps: virtualizing the TPU equipment in the cluster into computing resources; obtaining resource use information of a container in the cluster for the computing resource, and determining target configuration information of a first execution unit to be bound with the container according to the resource use information; determining a target computing resource of the container in a candidate computing resource list according to the resource use information, and obtaining identification information of the target computing resource; the target configuration information is updated based on the identification information of the target computing resource, and the first execution unit is created according to the updated target configuration information, so that the container and the first execution unit are bound, and the container bound with the first execution unit jointly uses the target computing resource. According to the invention, the utilization rate of computing resources in the cluster is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of wireless communication technology, and in particular to a resource management method, a resource management system, a computer program product, and an electronic device. Background Art

[0002] TPU (Tensor Processing Unit) devices can be used to accelerate machine learning and deep learning tasks. With the explosive growth of data, it has become a trend for containers in clusters to rely on TPU devices for large-scale data processing.

[0003] In the related art, it is difficult to share the computing resources of the TPU device between multiple containers, so that the TPU device cannot be fully utilized, resulting in low utilization of computing resources in the cluster. Summary of the invention

[0004] The present disclosure provides a resource management method, a resource management system, a computer program product and an electronic device, thereby improving the problem of low computing resource utilization at least to a certain extent.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to a first aspect of the present disclosure, a resource management method is provided, which is applied to a cluster using tensor processing unit TPU devices, and the method includes: virtualizing the TPU devices in the cluster into computing resources; obtaining resource usage information of containers in the cluster for the computing resources, and determining target configuration information of a first execution unit to be bound to the container based on the resource usage information; determining a target computing resource of the container in a list of candidate computing resources based on the resource usage information, and obtaining identification information of the target computing resource; updating the target configuration information based on the identification information of the target computing resource, and creating the first execution unit based on the updated target configuration information to bind the container to the first execution unit, so that the container bound to the first execution unit uses the target computing resource together.

[0007] Optionally, the computing resources include multiple virtual computing resources corresponding to the same TPU device, and virtualizing the TPU devices in the cluster into computing resources includes: obtaining an available time period of the same TPU device, and dividing the available time period into multiple sub-time periods according to the usage time period segmentation rule; generating multiple usage constraint information corresponding to the same TPU device according to each sub-time period, and obtaining the same number of identification information as the usage constraint information; and making one-to-one correspondence between each identification information of the same TPU device and each usage constraint information to obtain the multiple virtual computing resources corresponding to the same TPU device.

[0008] Optionally, the candidate computing resource list includes a first category of candidate computing resources and a second category of candidate computing resources, the first category of candidate computing resources include second label information, and the second category of candidate computing resources does not include the second label information; determining the target computing resource of the container in the candidate computing resource list based on the resource usage information includes: obtaining resource demand information and / or first label information based on the resource usage information; when the resource usage information includes the first label information, combining the first label information and the resource demand information to determine the target computing resource from the candidate computing resources; when the resource usage information does not include the first label information, preferentially selecting the target computing resource from the second category of candidate computing resources.

[0009] Optionally, combining the first label information with the resource requirement information to determine the target computing resource in the candidate computing resource list includes: obtaining usage constraint information of the first category candidate computing resources; if there is a first category candidate computing resource whose second label information is the same as the first label information among the first category candidate computing resources, determining the target computing resource by combining the usage constraint information of the first category candidate computing resource with the resource requirement information of the container; otherwise, determining the target computing resource among the candidate computing resources of the unloaded container included in the candidate computing resource list.

[0010] Optionally, the configuration information includes a list of available resources, and updating the target configuration information based on the identification information of the target computing resource includes: editing the list of available resources based on the identification information of the target computing resource to update the target configuration information.

[0011] Optionally, the containers associated with the first execution unit jointly use the target computing resources, including: obtaining usage frequency information of the containers for the target computing resources, and obtaining resource demand information of the containers based on the resource usage information of the containers; combining the usage frequency information with the resource demand information, determining priority information for the containers to use the target computing resources, so that the containers use the target computing resources according to the priority information.

[0012] Optionally, the method further includes: in response to monitoring scheduling request information of a container bound to the second execution unit, determining a target working node of the second execution unit by combining the scheduling request information with a preset scheduling strategy.

[0013] According to a second aspect of the present disclosure, a resource management system is provided, which is applied to a cluster using TPU devices, and the system includes: a computing resource acquisition module, configured to virtualize the TPU devices in the cluster into computing resources; a target configuration information acquisition module, configured to acquire resource usage information of containers in the cluster for the computing resources, and determine the target configuration information of the first execution unit to be bound to the container based on the resource usage information; a resource configuration information generation module, configured to determine the target computing resource of the container in a list of candidate computing resources based on the resource usage information, and generate resource configuration information based on the identification information of the target computing resource; a target configuration information update module, configured to update the target configuration information based on the identification information of the target computing resource, and create the first execution unit based on the updated target configuration information, so as to bind the container to the first execution unit, so that the container bound to the first execution unit uses the target computing resource together.

[0014] According to a third aspect of the present disclosure, a computer program is provided, which, when executed by a processor, implements the resource management method of the first aspect and possible implementation manners thereof.

[0015] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor, wherein the processor is configured to execute the resource management method of the first aspect and possible implementation thereof by executing the executable instructions.

[0016] The technical solution disclosed in this disclosure has the following beneficial effects:

[0017] On the one hand, computing resources are obtained based on the TPU devices in the cluster, and the TPU devices are virtualized into unified computing resources of the cluster, providing the possibility of allocating target computing resources to containers in the subsequent process. On the other hand, a first execution unit is provided, and the target configuration information of the first execution unit is updated through the identification information of the target computing resources, and the first execution unit is created based on the updated target configuration information, so that the target computing resources are shared between the containers loaded by the first execution unit, effectively improving the utilization rate of computing resources.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0020] Figure 1 The system operation architecture of this exemplary embodiment is shown;

[0021] Figure 2 A flow chart showing a resource management method in this exemplary embodiment;

[0022] Figure 3 A flowchart showing a method for acquiring computing resources in this exemplary embodiment is shown;

[0023] Figure 4 A flowchart of obtaining a target computing resource in this exemplary embodiment is shown;

[0024] Figure 5 A schematic diagram showing a method of creating a first execution unit in this exemplary embodiment is shown;

[0025] Figure 6 A schematic diagram showing a creation script of a first execution unit in this exemplary embodiment;

[0026] Figure 7 A flowchart showing a method for containers to share target computing resources in this exemplary embodiment is shown;

[0027] Figure 8 A schematic diagram showing computing resources corresponding to the same TPU device in this exemplary embodiment;

[0028] Fig. 9A schematic diagram showing a process of scheduling a second execution unit in this exemplary embodiment;

[0029] Fig.10 A schematic diagram showing the structure of a resource management system in this exemplary embodiment is shown;

[0030] Fig.11 A schematic structural diagram of an electronic device in this exemplary embodiment is shown. DETAILED DESCRIPTION

[0031] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0032] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0033] In the related art, it is difficult to share the computing resources of the TPU device between multiple containers, so that the TPU device cannot be fully utilized, resulting in low utilization of computing resources in the cluster.

[0034] In view of one or more of the above problems, the exemplary embodiment of the present disclosure first provides a resource management method. Figure 1 The system architecture of the operating environment of this exemplary embodiment will be described.

[0035] refer to Figure 1As shown, the system architecture 100 may include a terminal device 110 and a server 120. The terminal device 110 may be an electronic device such as a smart phone, a tablet computer, or a laptop computer. The terminal device 110 may interact with the cluster through a container. For example, the terminal device 110 may be used to send a request to a container in the cluster. After receiving the request, the container may generate corresponding resource usage information according to the request. The server 120 generally refers to a background system that provides resource management related services in this exemplary embodiment, such as a server that implements a resource management method. The server 120 may be a cluster formed by one server or multiple servers. The cluster may use a TPU device to accelerate the computing process, which is not limited in this disclosure. The terminal device 110 and the server 120 may be connected via a wired or wireless communication link for data exchange.

[0036] In one embodiment, the TPU device used by the server 120 can be virtualized into a computing resource. The user can submit a request to the server 120 by interacting with the terminal device 110 for data. In response to receiving the request, the container in the server 120 can generate resource usage information based on the computing resources required to respond to the request, and obtain the target configuration information of the first execution unit to be bound to the container based on the resource usage information, and determine the target computing resource of the container in the candidate computing resource list based on the resource usage information, and obtain the identification information of the target computing resource, update the target configuration information based on the identification information of the target computing resource, and create a first execution unit based on the updated target configuration information to bind the container to the first execution unit so that the container bound to the first execution unit shares the target computing resource, and process the above request through the target computing resource, so as to send the response information corresponding to the above request to the terminal device 110 after generating the corresponding information.

[0037] As can be seen from the above, the resource management method in this exemplary embodiment can be applied to a cluster composed of servers 120 using TPU devices.

[0038] Combine the following Figure 2 Describe the resource management method. Figure 2 An exemplary process of a resource management method is shown, including the following steps S210 to S240:

[0039] Step S210, virtualizing the TPU devices in the cluster into computing resources;

[0040] Step S220, obtaining resource usage information of the container in the cluster for computing resources, and determining target configuration information of the first execution unit to be bound to the container according to the resource usage information;

[0041] Step S230, determining a target computing resource of the container in the candidate computing resource list according to the resource usage information, and obtaining identification information of the target computing resource;

[0042] Step S240, update the target configuration information based on the identification information of the target computing resource, and create a first execution unit according to the updated target configuration information to bind the container to the first execution unit, so that the container bound to the first execution unit uses the target computing resource together.

[0043] Based on the above method, on the one hand, computing resources are obtained based on the TPU devices in the cluster, and the TPU devices are virtualized into unified computing resources of the cluster, providing the possibility of allocating target computing resources to containers in the subsequent process. On the other hand, a first execution unit is provided, and the target configuration information of the first execution unit is updated through the identification information of the target computing resources, and the first execution unit is created based on the updated target configuration information, so that the target computing resources are shared between the containers loaded by the first execution unit, and the utilization rate of computing resources is effectively improved.

[0044] Below Figure 1 Each step in the procedure is described in detail.

[0045] refer to Figure 1 , in step S210, the TPU devices in the cluster are virtualized into computing resources.

[0046] TPU devices can be used to accelerate machine learning and deep learning processes. Through highly optimized hardware architecture and specific instruction sets, they can efficiently accelerate the operation process of tensors. As a processor designed specifically for machine learning tasks, TPU has parallel computing capabilities and large-scale vector processing capabilities. It can efficiently perform matrix operations and tensor operations in large-scale neural network models. Therefore, it is widely used in cloud computing, large-scale machine learning and deep learning applications, and has become an important tool for accelerated computing.

[0047] The above computing resources can be collectively referred to as vTPU (virtual Tensor Processing Unit), which can be the computing power of TPU devices abstracted into virtual resources possessed by the cluster. Virtual resources can include the computing power possessed by TPU devices in the cluster, and can also include multiple virtual computing resources corresponding to the same TPU device, that is, multiple virtual computing resources obtained by dividing the computing power possessed by the same TPU device; it can also include the computing power possessed by multiple TPU devices in parallel; the following is an explanation of how to obtain computing resources:

[0048] Multiple virtual computing resources corresponding to the same TPU device can be obtained according to the usage time of the TPU device. In one embodiment, the TPU devices in the cluster are virtualized into computing resources. Figure 3 As shown, steps S310 to S330 may be included:

[0049] Step S310, obtaining the available time period of the same TPU device, and dividing the available time period into multiple sub-time periods according to the usage time period segmentation rule;

[0050] Step S320, generating a plurality of usage constraint information corresponding to the same TPU device according to each sub-period, and obtaining identification information having the same number as the usage constraint information;

[0051] Step S330 , matching each piece of identification information of the same TPU device with each piece of usage constraint information one by one to obtain multiple virtual computing resources corresponding to the same TPU device.

[0052] Among them, the above identification information is used to characterize the serial number of the virtual computing resource, which can be represented by TPUID. The usage period segmentation rule can be the basis for the division of the sub-period, and the usage period segmentation rule can include the automatic segmentation logic pre-established by the developer. For example, after obtaining the available time period of the TPU device, the available time period is automatically divided into multiple sub-periods according to the usage period segmentation rule; it can also include the rule that the user manually divides the available time period into multiple sub-periods. The above usage constraint information can be used to specify the usage conditions of the virtual computing resources. For example, the usage constraint information of any virtual computing resource may include usage time constraints and usage memory constraints. The usage time constraint may include the specific duration and start time of the sub-period corresponding to the virtual computing resource, and the usage memory constraint may include the memory upper limit of the TPU device corresponding to the virtual computing resource.

[0053] It should be noted that the durations of the above-mentioned sub-periods may be the same or different, and the present disclosure does not make any special limitation on this.

[0054] based on Figure 3 The method can divide the same TPU device into multiple virtual computing resources according to the usage time, providing the possibility of improving the computing resource utilization of the TPU device.

[0055] In addition, multiple TPU devices can be divided into the same computing resource. Accordingly, the usage time constraint in the usage constraint information of the computing resource can be jointly determined according to the available time of multiple TPU devices. For example, it can be determined according to the intersection of the available time periods of multiple TPU devices. The usage memory constraint can be jointly determined according to the memory upper limit of multiple TPU devices. Exemplarily, multiple TPU devices can be virtualized into the same computing resource according to user needs, so that the container that receives the demand can quickly respond to user needs based on parallel TPU devices, and can also reserve some computing resources to process user needs with higher priority. At the same time, users can also customize the virtualization scheme of computing resources by themselves.

[0056] By virtualizing the TPU devices in the cluster as the cluster's computing resources, it is possible to quickly respond to various user needs.

[0057] Continue to refer Figure 2 In step S220, resource usage information of the container in the cluster for computing resources is obtained, and target configuration information of the first execution unit to which the container is to be bound is determined based on the resource usage information.

[0058] The resource usage information may be information related to computing resources required by the container to process user needs, and may include resource demand information, and / or first tag information. The resource demand information may include the upper and lower limits of the resource demand of the container, for example, the minimum duration tpu_request and the maximum duration tpu_limit required for the normal operation of the container to use computing resources, as well as the upper limit tpu_mem of the memory required by the container to occupy computing resources, etc. The above-mentioned first label information can be used to characterize the affinity of the container. The first label information can include any one of the affinity label affinity, the anti-affinity label anti-affinity and the exclusion label exclusion. The container can be bound to the first execution unit specified by the user through the first label information, and the target computing resources available to the container can be obtained, so that the container uses the target computing resources through the first execution unit; exemplarily, if the resource usage information of container A and container B both include the same affinity label, it means that container A and container B can be bound to the same first execution unit; if the resource usage information of container A and container B both include the same anti-affinity label, it means that container A and container B cannot be bound to the same first execution unit; if the resource usage information of container A includes an exclusion label, the label container A cannot share the same computing resources with other containers.

[0059] The first execution unit (shareTPUPod) is a new pod type obtained by developers through the cluster's custom resource definition mechanism to extend the native pod, which is mainly used to associate containers with computing resources. Custom Resource Definition (CRD) is a mechanism in Kubernetes that allows developers to extend the API of the Kubernetes cluster; through CRD, developers can define and use new resource types (for example, specific application configurations or workload types); making Kubernetes more scalable and flexible, and able to meet a variety of different application needs and management requirements.

[0060] Configuration information can be the information required by users to deploy the first execution unit in the cluster, and can be used to create the first execution unit in the subsequent process. The main difference between it and the native pod (second execution unit) of the cluster is that the configuration information of the first execution unit can contain the identification information of the computing resources; the parameters contained in the configuration information can be the same as the resource usage information, and users can bind containers with the same computing resource usage requirements to the same first execution unit by editing the configuration information. The target configuration information is the configuration information of the first execution unit to which the container is to be bound.

[0061] In one embodiment, the above-mentioned determining the target configuration information of the first execution unit to be bound to the container based on the resource usage information may include: generating the target configuration information based on the resource usage information; or matching the configuration information of the first execution unit deployed in the cluster based on the resource usage information to filter out the first execution unit available for the container, and determining the target configuration information in the configuration information of the first running unit available for the container.

[0062] The target configuration information obtained through the resource usage information can enable the first execution unit deployed through the target configuration information to reflect the resource requirements of the container.

[0063] Continue to refer Figure 2 In step S230, based on the resource usage information, the target computing resource of the container is determined in the candidate computing resource list, and identification information of the target computing resource is obtained.

[0064] The candidate computing resource list may include computing resources currently available in the cluster, for example, computing resources not occupied by containers. The target computing resource may be a computing resource available to the container in the candidate computing resource list.

[0065] In one implementation, the candidate computing resource list includes a first category of candidate computing resources and a second category of candidate computing resources, the first category of candidate computing resources include second label information, and the second category of candidate computing resources do not include second label information; the step of determining the target computing resource of the container in the candidate computing resource list according to the resource usage information may include the following steps:

[0066] Acquire resource demand information and / or first tag information based on the resource usage information;

[0067] When the resource usage information includes the first label information, the target computing resource is determined from the candidate computing resources in combination with the first label information and the resource demand information; when the resource usage information does not include the first label information, the target computing resource is preferentially selected from the second category of candidate computing resources.

[0068] Among them, the second label information of the above-mentioned computing resources is the same as the first label information, and may include any one of an affinity label, an anti-affinity label and an exclusion label; the present disclosure does not specifically limit the generation method of the second label information of the computing resources. For example, the above-mentioned second label information may be determined based on the first label information of the container currently loaded by the computing resource; or it may be manually set by the developer during the computing resource generation process.

[0069] The above-mentioned combining the first tag information with the resource requirement information to determine the target computing resource in the candidate computing resource list may include:

[0070] Obtain usage constraint information of the first category of candidate computing resources;

[0071] If there is a first-category candidate computing resource whose second label information is the same as the first label information among the first-category candidate computing resources, the target computing resource is determined by combining the usage constraint information of the first-category candidate computing resource and the resource requirement information of the container; otherwise, the target computing resource is determined among the candidate computing resources that are not loaded with the container and are included in the candidate computing resource list.

[0072] For example, the process of matching target computing resources for containers based on the resource usage information of the container can be referred to Figure 4 As shown, steps S401 to S410 are included:

[0073] Step S401, obtaining resource usage information of the container;

[0074] Step S402, determining whether the resource usage information contains the first tag information; if so, jumping to step S403; otherwise, jumping to step S404;

[0075] Step S403, searching for a second type of candidate computing resources;

[0076] Step S404, whether there is an intermediate computing resource whose second label information is the same as the first label information in the first category of candidate computing resources; if so, jump to step S405; otherwise, jump to step S406;

[0077] Step S405, combining the usage constraint information of the first type of candidate computing resources with the resource requirement information of the container, determining the target computing resource from the intermediate computing resources;

[0078] Step S406, searching the candidate computing resources list for candidate computing resources that are not occupied by the container, and selecting the target computing resource from the unoccupied candidate computing resources;

[0079] Step S407, using the best pseudo algorithm to select a target computing resource from the second category of candidate computing resources;

[0080] Step S408, using the worst-case algorithm to select a target computing resource from the first category of candidate computing resources;

[0081] Step S409, applying for new computing resources from the cluster as target computing resources;

[0082] Step S410, obtaining the TPUID of the target computing resource.

[0083] Furthermore, since the usage constraint information of computing resources may include the usage time constraint and the usage memory constraint of computing resources, and the resource requirement information of the container may include the upper and lower limits of the amount of resources required for the container, such as the upper and lower limits of the duration for which the computing resources need to be used, and the upper limit of the memory of the computing resources that need to be occupied; therefore, the target computing resources can be selected based on the comparison results of the upper and lower limits of the amount of resources required by the container with the usage time constraint and the usage memory constraint of the computing resources.

[0084] Based on the above method for determining the target computing resource in the candidate computing resource list, the selected target computing resource can meet the usage requirements and affinity of the container, which is conducive to achieving efficient utilization of computing resources.

[0085] Continue to refer Figure 2 In step S240, the target configuration information is updated based on the identification information of the target computing resource, and a first execution unit is created according to the updated target configuration information to bind the container to the first execution unit so that the container bound to the first execution unit uses the target computing resource together.

[0086] By updating the target configuration information through the identification information of the target computing resource and creating the first execution unit based on the updated target configuration information, a mapping relationship between the target computing resource and the first execution unit can be established so that the container bound to the first execution unit shares the target computing resource available to the first execution unit.

[0087] In one implementation, the configuration information includes a list of available resources, and the updating of the target configuration information based on the identification information of the target computing resource may include:

[0088] The available resource list is edited based on the identification information of the target computing resource to update the target configuration information.

[0089] The available resource list may include all computing resources that can be used by the container in the first execution unit. Editing the available resource list may include adding the TPUID of the target computing resource to the available resource list, deleting the computing resource, and replacing the computing resource. Exemplarily, different instructions may be generated based on the identification information of the target computing resource, such as the instruction may include adding the TPUID of the target computing resource to the resource list annotation corresponding to the available resource list to update the target configuration information.

[0090] For example, refer to Figure 5 As shown, the processes of step S230 and step S240 can be integrated into the resource scheduler TPUShare-Sched and the resource manager TPUShare-DevMgr through the CRD file. TPUShare-Sched and TPUShare-DevMgr together constitute the computing resource controller TPUShare; developers can deploy TPUShare in the cluster, as well as the CRD file of the first execution unit shareTPUPod. After the deployment is completed, the user can use the APIserver to generate target configuration information based on the resource usage information of the container to deploy shareTPUPod in the cluster; the APIserver can send the resource usage information, target configuration information, etc. to the cluster component Etcd for storage, so that TPUShare-Sched interacts with the cluster component etcd to determine the target computing resources required for the container, and modify the available resource list of shareTPUPod based on the identification information of the target computing resources to update the target configuration information; TPUShare-DevMgr can generate a shareTPUPod creation script based on the updated target configuration information, refer to Figure 6As shown, the creation script may include the naming, type, target configuration information of shareTPUPod, and other configuration specifications of shareTPUPod, such as detailed information of the container to be bound to shareTPUPod, so as to create shareTPUPod in the cluster and initialize the operating environment of the container bound to shareTPUPod; shareTPUPod can find the corresponding worker node in the cluster according to the first label information of the bound container, so that after shareTPUPod is bound to the worker node, the container in shareTPUPod (shareTPUPod container) can use the target computing resource vTPU to implement the running process.

[0091] Based on the above process, the target computing resource can be associated with the container through the first execution unit. The following is an explanation of how the container bound to the first execution unit uses the target computing resource. In one embodiment, the container associated with the first execution unit uses the target computing resource together. Figure 7 As shown, steps S710 to S720 may be included:

[0092] Step S710, obtaining the usage frequency information of the container for the target computing resource, and obtaining the resource demand information of the container based on the resource usage information of the container;

[0093] Step S720 , combining the usage frequency information and the resource demand information, determining the priority information of the container using the target computing resource, so that the container uses the target computing resource according to the priority information.

[0094] The priority information may include information related to the order in which the container uses the target computing resource. The usage frequency information may include a real-time record of the number of times the container uses the target computing resource. For example, Figure 8As shown in the figure, a certain TPU device in the cluster is virtualized into ten computing resources, computing resources 3 to 5 are occupied by shareTPUPod1, that is, computing resources 3 to 5 are the target computing resources of shareTPUPod1, and computing resources 6 to 8 are occupied by shareTPUPod2. Based on the resource usage information issued by the user request for the container bound to shareTPUPod1, the resource demand information in the resource usage information can be obtained. For example, the resource demand information may include the duration of the target computing resource required for the normal operation of the container, or the memory of the target computing resource required for the normal operation of the container to use the target computing resource; in the resource demand information of each container, the container with the largest resource demand is first filtered out, and then the remaining containers are sorted from small to large according to the resource demand and usage frequency information, and the priority information of each container using the target computing resource is determined according to the sorting. If the lower limit of the resource demand of each container is met, the containers can be sorted from small to large according to the usage frequency information of the container, so as to determine the priority information of each container based on the sorting.

[0095] based on Figure 7 This method can ensure that each container uses the target computing resources in a balanced manner, thereby increasing the container's response speed to user requests and improving the user experience.

[0096] In addition, in one embodiment, the above method may further include: in response to monitoring the scheduling request information of the container bound to the second execution unit, determining the target working node of the second execution unit by combining the scheduling request information with a preset scheduling strategy.

[0097] The second execution unit is a native pod in the cluster. The preset scheduling strategy may include a scheduling strategy generated based on the scheduling process of kube-scheduler in the Kubernetes cluster, that is, the resource management method in this exemplary embodiment integrates the scheduling process of the native pod, and can simultaneously implement the scheduling process of shareTPUPod and the native pod.

[0098] For example, refer to Fig. 9 As shown in the figure, after the user deploys the definition file of the second execution unit pod in the cluster, TPUShare-Sched in TPUShare can monitor the scheduling request information of the container in the pod in the API server, and obtain the target working node for the pod according to the preset scheduling strategy to generate the scheduling information of the pod; the kubelet of the target working node can receive the scheduling information and create and run the container of the pod, so that the container in the pod can run in the target working node. Fig. 9As shown, the containers in the pod and shareTPUPod can use the same target working node to implement the operation process. Specifically, the container in the shareTPUPod can use the vTPU to implement the operation process, and the pod can use the corresponding TPU device to run.

[0099] In one embodiment, the method may further include: in response to monitoring state change information of the first execution unit, reacquiring the target computing resource based on the state change information.

[0100] The state change information may include events such as creation, update, and deletion of the first execution unit. By monitoring the state change information of the first execution unit, the target computing resources of the first execution unit may be adjusted, thereby improving the utilization of computing resources.

[0101] Based on the above method, the TPU device is virtualized into a unified computing resource of the cluster, and the target computing resources are shared between the containers loaded by the first execution unit, which effectively improves the utilization rate of computing resources.

[0102] The exemplary embodiment of the present disclosure also provides a resource management system, which is applied to a cluster using a TPU device, referring to Fig.10 As shown, the system may include:

[0103] A computing resource acquisition module 1010 is configured to virtualize TPU devices in the cluster into computing resources;

[0104] The target configuration information acquisition module 1020 is configured to acquire resource usage information of the container in the cluster for computing resources, and determine target configuration information of the first execution unit to be bound to the container according to the resource usage information;

[0105] The resource configuration information generating module 1030 is configured to determine the target computing resource of the container in the candidate computing resource list according to the resource usage information, and generate the resource configuration information based on the identification information of the target computing resource;

[0106] The target configuration information update module 1040 is configured to update the target configuration information based on the identification information of the target computing resource, and create a first execution unit according to the updated target configuration information to bind the container to the first execution unit so that the container bound to the first execution unit uses the target computing resource together.

[0107] In one embodiment, the computing resources include multiple virtual computing resources corresponding to the same TPU device, and the TPU devices in the cluster are virtualized into computing resources, including:

[0108] Obtain the available time period of the same TPU device, and divide the available time period into multiple sub-time periods according to the usage time period segmentation rule;

[0109] Generate multiple usage constraint information corresponding to the same TPU device according to each sub-period, and obtain identification information with the same number as the usage constraint information;

[0110] Each identification information of the same TPU device is matched with each usage constraint information one by one to obtain multiple virtual computing resources corresponding to the same TPU device.

[0111] In one implementation, the candidate computing resource list includes a first category of candidate computing resources and a second category of candidate computing resources, the first category of candidate computing resources include second label information, and the second category of candidate computing resources do not include second label information; determining the target computing resource of the container in the candidate computing resource list according to the resource usage information may include:

[0112] Acquire resource demand information and / or first tag information based on the resource usage information;

[0113] When the resource usage information includes the first label information, the target computing resource is determined from the candidate computing resources in combination with the first label information and the resource demand information; when the resource usage information does not include the first label information, the target computing resource is preferentially selected from the second category of candidate computing resources.

[0114] In one implementation, combining the first tag information with the resource requirement information to determine the target computing resource in the candidate computing resource list may include:

[0115] Obtain usage constraint information of the first category of candidate computing resources;

[0116] If there is a first-category candidate computing resource whose second label information is the same as the first label information among the first-category candidate computing resources, the target computing resource is determined by combining the usage constraint information of the first-category candidate computing resource and the resource requirement information of the container; otherwise, the target computing resource is determined among the candidate computing resources that are not loaded with the container and are included in the candidate computing resource list.

[0117] In one implementation, the configuration information includes a list of available resources, and updating the target configuration information based on the identification information of the target computing resource may include:

[0118] The available resource list is edited based on the identification information of the target computing resource to update the target configuration information.

[0119] In one implementation, the containers associated with the first execution unit jointly use the target computing resources, which may include:

[0120] Obtaining usage frequency information of the container for the target computing resources, and obtaining resource demand information of the container based on the resource usage information of the container;

[0121] The use frequency information and the resource demand information are combined to determine the priority information of the container using the target computing resource, so that the container uses the target computing resource according to the priority information.

[0122] In one embodiment, the above system may further include:

[0123] In response to monitoring the scheduling request information of the container bound to the second execution unit, a target working node of the second execution unit is determined by combining the scheduling request information with a preset scheduling strategy.

[0124] The specific details of each part of the above system have been described in detail in the implementation method part, so they will not be repeated here.

[0125] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0126] The exemplary embodiments of the present disclosure further provide a computer program product, which includes a computer program. When the computer program is executed by a processor, the resource management method is implemented.

[0127] Computer programs for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer program may be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0128] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.

[0129] The code of the computer program can be written in one or more programming languages. Programming languages ​​such as C language, Java, C++, etc. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).

[0130] Computer programs can be carried or transmitted through electrical, magnetic, optical, electromagnetic, infrared and other signals. Electronic devices can convert signals carrying computer programs into digital signals to run computer programs. When a computer program runs on an electronic device, its code is used to enable the electronic device to execute (more specifically, the processor of the electronic device can execute) the method steps of various exemplary embodiments of the present disclosure, such as the above-mentioned resource management method, which includes the following steps: virtualizing the TPU devices in the cluster into computing resources; obtaining resource usage information of the containers in the cluster for computing resources, and determining the target configuration information of the first execution unit to be bound to the container based on the resource usage information; determining the target computing resource of the container in the candidate computing resource list based on the resource usage information, and obtaining the identification information of the target computing resource; updating the target configuration information based on the identification information of the target computing resource, and creating a first execution unit based on the updated target configuration information to bind the container to the first execution unit so that the container bound to the first execution unit uses the target computing resource together.

[0131] By executing the above method steps through a computer program, on the one hand, computing resources are obtained based on the TPU devices in the cluster, and the TPU devices are virtualized into unified computing resources of the cluster, providing the possibility of allocating target computing resources to containers in subsequent processes. On the other hand, a first execution unit is provided, and the target configuration information of the first execution unit is updated through the identification information of the target computing resources, and the first execution unit is created based on the updated target configuration information, so that the target computing resources are shared between the containers loaded by the first execution unit, effectively improving the utilization rate of computing resources.

[0132] The exemplary embodiments of the present disclosure also provide an electronic device, such as the server 120 described above. The electronic device may include a processor and a memory. The memory stores executable instructions of the processor, such as a computer program. The processor executes the method steps of various exemplary embodiments of the present disclosure by executing the executable instructions.

[0133] Reference below Fig.11 , the electronic device is exemplarily described in the form of a general-purpose computing device. It should be understood that Fig.11 The electronic device 1100 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0134] like Fig.11 As shown, the electronic device 1100 may include: a processor 1110 , a memory 1120 , a bus 1130 , an I / O (input / output) interface 1140 , and a network adapter 1150 .

[0135] Processor 1110 may include one or more processing units. For example, processor 1110 may include an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor and / or an NPU (Neural-Network Processing Unit), etc.

[0136] Processor 1110 can be used to execute executable instructions stored in memory 1120, such as the above-mentioned resource management method, which includes the following steps: virtualizing the TPU devices in the cluster into computing resources; obtaining resource usage information of the containers in the cluster for the computing resources, and determining the target configuration information of the first execution unit to be bound to the container based on the resource usage information; determining the target computing resource of the container in the candidate computing resource list based on the resource usage information, and obtaining the identification information of the target computing resource; updating the target configuration information based on the identification information of the target computing resource, and creating a first execution unit based on the updated target configuration information to bind the container to the first execution unit, so that the container bound to the first execution unit uses the target computing resource together.

[0137] By executing the above method steps through a computer program, on the one hand, computing resources are obtained based on the TPU devices in the cluster, and the TPU devices are virtualized into unified computing resources of the cluster, providing the possibility of allocating target computing resources to containers in subsequent processes. On the other hand, a first execution unit is provided, and the target configuration information of the first execution unit is updated through the identification information of the target computing resources, and the first execution unit is created based on the updated target configuration information, so that the target computing resources are shared between the containers loaded by the first execution unit, effectively improving the utilization rate of computing resources.

[0138] The memory 1120 may include a volatile memory, such as a RAM 1121, a cache unit 1122, and may also include a non-volatile memory, such as a ROM 1123. The memory 1120 may also include one or more program modules 1124, such program modules 1124 include but are not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof may include the implementation of a network environment. For example, the program module 1124 may include the modules in the above-mentioned system 1000.

[0139] The bus 1130 is used to realize the connection between different components of the electronic device 1100, and may include a data bus, an address bus, and a control bus.

[0140] The electronic device 1100 can communicate with one or more external devices 1200 (eg, a keyboard, a mouse, an external controller, etc.) through the I / O interface 1140 .

[0141] The electronic device 1100 can communicate with one or more networks through the network adapter 1150. For example, the network adapter 1150 can provide mobile communication solutions such as 3G / 4G / 5G, or provide wireless communication solutions such as wireless LAN, Bluetooth, near field communication, etc. The network adapter 1150 can communicate with other modules of the electronic device 1100 through the bus 1130.

[0142] although Fig.11 Not shown, other hardware and / or software modules may also be provided in the electronic device 1100, including but not limited to: a display, a microcode, a device driver, a redundant processor, an external disk drive array, a RAID system, a tape drive, and a data backup storage system.

[0143] As can be seen from the above, the technical solution of the present disclosure can be implemented as a method, a system, a computer program product, an electronic device, etc. Those skilled in the art can understand that various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software, such as being respectively referred to as a "circuit", "module" or "system".

[0144] It should be understood that the present disclosure is not limited to the specific method steps or structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. Those skilled in the art will easily think of other embodiments based on the specific embodiments provided by the present disclosure. Therefore, the specific embodiments provided by the present disclosure are only exemplary, and the scope and spirit of the present disclosure are indicated by the claims, and any variations, uses or adaptive changes of the present disclosure should be covered, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the technical field that are not disclosed in the present disclosure.

Claims

1. A resource management method, characterized in that: Applied to a cluster using a tensor processing unit (TPU) device, the method comprises: Virtualizing the TPU devices in the cluster as computing resources; Obtain resource usage information of the container in the cluster for the computing resource, and determine target configuration information of the first execution unit to be bound to the container according to the resource usage information; Determine, according to the resource usage information, a target computing resource of the container in a list of candidate computing resources, and obtain identification information of the target computing resource; The target configuration information is updated based on the identification information of the target computing resource, and the first execution unit is created according to the updated target configuration information to bind the container to the first execution unit, so that the container bound to the first execution unit uses the target computing resource together.

2. The method according to claim 1, characterized in that The computing resources include multiple virtual computing resources corresponding to the same TPU device, and virtualizing the TPU devices in the cluster into computing resources includes: Obtaining an available period of the same TPU device, and dividing the available period into a plurality of sub-periods according to the usage period segmentation rule; Generate a plurality of usage constraint information corresponding to the same TPU device according to each of the sub-periods, and obtain the same number of identification information as the number of the usage constraint information; The identification information of the same TPU device is matched one-to-one with the usage constraint information to obtain the multiple virtual computing resources corresponding to the same TPU device.

3. The method according to claim 1, characterized in that The candidate computing resource list includes a first category of candidate computing resources and a second category of candidate computing resources, the first category of candidate computing resources includes second label information, and the second category of candidate computing resources does not include the second label information; The determining, according to the resource usage information, a target computing resource for the container in a list of candidate computing resources includes: Acquire resource demand information and / or first tag information based on the resource usage information; In a case where the resource usage information includes the first label information, the target computing resource is determined from the candidate computing resources in combination with the first label information and the resource demand information; in a case where the resource usage information does not include the first label information, the target computing resource is preferentially selected from the second category of candidate computing resources.

4. The method according to claim 3, characterized in that The combining the first tag information with the resource requirement information to determine the target computing resource in the candidate computing resource list includes: Obtaining usage constraint information of the first category of candidate computing resources; If there is a first-category candidate computing resource whose second label information is identical to the first label information among the first-category candidate computing resources, the target computing resource is determined by combining the usage constraint information of the first-category candidate computing resource and the resource requirement information of the container; otherwise, the target computing resource is determined among the candidate computing resources that are not loaded with the container and are included in the candidate computing resource list.

5. The method according to claim 1, characterized in that: The configuration information includes a list of available resources, and the updating of the target configuration information based on the identification information of the target computing resource includes: The available resource list is edited based on the identification information of the target computing resource to update the target configuration information.

6. The method according to claim 1, characterized in that The containers associated with the first execution unit jointly use the target computing resource, including: Obtaining usage frequency information of the container for the target computing resource, and obtaining resource demand information of the container based on the resource usage information of the container; The priority information of the container using the target computing resource is determined in combination with the usage frequency information and the resource demand information, so that the container uses the target computing resource according to the priority information.

7. The method according to claim 1, characterized in that The method further comprises: In response to monitoring the scheduling request information of the container bound to the second execution unit, the target working node of the second execution unit is determined by combining the scheduling request information with a preset scheduling policy.

8. A resource management system, characterized in that: Applied to a cluster using a TPU device, the system includes: A computing resource acquisition module, configured to virtualize the TPU devices in the cluster into computing resources; a target configuration information acquisition module, configured to acquire resource usage information of the container in the cluster for the computing resource, and determine target configuration information of the first execution unit to be bound to the container according to the resource usage information; a resource configuration information generating module, configured to determine a target computing resource of the container in a list of candidate computing resources according to the resource usage information, and to generate resource configuration information based on identification information of the target computing resource; A target configuration information updating module is configured to update the target configuration information based on the identification information of the target computing resource, and to create the first execution unit according to the updated target configuration information to bind the container to the first execution unit, so that the container bound to the first execution unit can use the target computing resource together.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.