Method and device for obtaining excess resources and computing equipment

Abstract

The invention provides a method and device for obtaining excess resources and computing equipment, and the method comprises the steps that a cloud management platform receives the configuration information of a first virtual accelerator sent by a tenant and the configuration information of the excess resources of the first virtual accelerator, and the excess resources comprise excess storage resources; allocating resources of a first physical accelerator in a first server to the first virtual accelerator according to the specification, size and type of the first virtual accelerator selected by the tenant; on the basis of the configuration information of the excess resources, the excess storage resources are obtained through replacement in a mode of consuming computing power resources of a first physical accelerator; and sending login information of the first virtual accelerator to a client of the tenant, wherein the login information is used for the tenant to log in and use the first virtual accelerator and the excess storage resource. According to the method, the upper limit of available resources on the virtual accelerator is not limited by physical resources.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on August 29, 2023, with application number 202311099298.1 and invention name “Method and computing device for determining excess video memory”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of cloud computing, and more specifically, to a method, apparatus and computing device for acquiring excess resources. Background Art

[0003] An accelerator is a computing unit specifically used for parallel computing of large amounts of data, and can be called a parallel computing unit or graphics card device. As a service parallel computing resource in cloud computing services, accelerators provide flexible and cost-effective computing capabilities for industries with high-density computing requirements such as graphics processing and artificial intelligence. Accelerator virtualization plays a key role in cloud computing services as an important means to reduce costs and improve resource utilization. A virtual accelerator is mainly a virtual device based on a physical accelerator after assembling it into cloud cluster resources. The virtual device provides virtualized computing resources. When users use virtual accelerators, there may be a situation where the storage resources in a virtual accelerator cannot meet user needs, or the storage resources in a virtual accelerator are not fully utilized.

[0004] In a related technical solution, the problem that a single user's video memory resources cannot meet user needs is solved by borrowing video memory / memory space in other locations. However, since the excess video memory resources in this solution are only obtained through space transfer, the use of the excess video memory resources will still be limited by the physical video memory resources on the physical accelerator.

[0005] Therefore, how to ensure that the upper limit of the resources available on the virtual accelerator is not limited by the physical resources has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] The present application provides a method, apparatus and computing device for obtaining excess resources, which can make the upper limit of resources available on a virtual accelerator not limited by physical resources, so that the resources on the virtual accelerator that a user can obtain exceed the physical resources on the physical accelerator.

[0007] In a first aspect, a method for obtaining excess resources is provided. The method is applied to a cloud management platform, and the cloud management platform is used to manage an infrastructure for providing cloud services. The infrastructure includes at least one server, and each server runs at least one virtual accelerator. The method includes: the cloud management platform receives configuration information of a first virtual accelerator and configuration information of excess resources of the first virtual accelerator sent by a tenant, the configuration information of the first virtual accelerator includes a specification size and a type of the first virtual accelerator selected by the tenant, and the excess resources include excess storage resources; the cloud management platform allocates resources of a first physical accelerator in a first server to the first virtual accelerator according to the specification size and type of the first virtual accelerator selected by the tenant, and the resources of the first physical accelerator include a first storage resource, a first computing power resource, and a first bandwidth resource; the cloud management platform obtains the excess storage resource by replacing it based on the configuration information of the excess resources by consuming the first computing power resource; the cloud management platform sends login information of the first virtual accelerator to a client of the tenant, and the login information is used for the tenant to log in and use the first virtual accelerator and the excess storage resource.

[0008] In the above technical solution, additional excess resources can be obtained by replacing computing resources, so that the upper limit of resources available on the virtual accelerator is not limited by physical resources, so that the resources on the virtual accelerator that users can obtain exceed the physical resources on the physical accelerator.

[0009] In combination with the first aspect, in some implementations of the first aspect, the configuration information of the excess resources includes a mode, size, type, and speed requirement of the excess resources.

[0010] In combination with the first aspect, in some implementations of the first aspect, the excess storage resource is located on a second storage resource, and the second storage resource includes at least one of the following: the first storage resource, a storage resource on a second physical accelerator in the first server, and a system storage resource of the first server.

[0011] It should be understood that the system storage resources of the first server may also be referred to as external storage resources.

[0012] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: the cloud management platform determines the location of the second storage resource based on at least one of the following information: a resource size of the first physical accelerator, a resource size of the second physical accelerator, an inter-card bandwidth of the first physical accelerator and the second physical accelerator, and a system resource usage of the first server.

[0013] In combination with the first aspect, in certain implementations of the first aspect, the cloud management platform receives data sent by the tenant; the cloud management platform compresses the data through the first computing power resource to obtain compressed data; and the cloud management platform stores the compressed data to the second storage resource.

[0014] In combination with the first aspect, in some implementations of the first aspect, the second storage resource is the second storage resource, and the cloud management platform stores the compressed data in the second storage resource through the bandwidth between the first physical accelerator and the second physical accelerator.

[0015] In combination with the first aspect, in some implementations of the first aspect, the second storage resource is a system storage resource of the first server, and the cloud management platform stores the compressed data in the second storage resource through the bandwidth between the first physical accelerator and the first server.

[0016] In combination with the first aspect, in certain implementations of the first aspect, the cloud management platform deletes the first intermediate data stored in the second storage resource through a first instruction, so that the tenant can reuse the storage resources occupied by the first intermediate data in the second storage resource; the cloud management platform regains the first intermediate data through calculation using the first computing power resource according to the second instruction sent by the tenant to obtain the first intermediate data.

[0017] In combination with the first aspect, in some implementations of the first aspect, the second storage resource is the second storage resource, and the cloud management platform deletes the first intermediate data stored in the second storage resource through the bandwidth between the first physical accelerator and the second physical accelerator.

[0018] In combination with the first aspect, in some implementations of the first aspect, the second storage resource is a system storage resource of the first server, and the cloud management platform deletes the first intermediate data stored in the second storage resource through the bandwidth between the first physical accelerator and the first server.

[0019] In a second aspect, a device for obtaining excess resources is provided, the device is applied to a cloud management platform, the cloud management platform is used to manage the infrastructure for providing cloud services, the infrastructure includes at least one server, each of the servers runs at least one virtual accelerator, the device includes: a receiving module, a processing module, and a sending module. Among them, the receiving module is used to receive the configuration information of the first virtual accelerator sent by the tenant and the configuration information of the excess resources of the first virtual accelerator, the configuration information of the first virtual accelerator includes the size and type of the first virtual accelerator selected by the tenant, and the excess resources include excess storage resources; the processing module is used to allocate the resources of the first physical accelerator in the first server to the first virtual accelerator according to the size and type of the first virtual accelerator selected by the tenant, and the resources of the first physical accelerator include first storage resources, first computing resources, and first bandwidth resources; the processing module is also used to replace and obtain the excess storage resources by consuming the first computing resources based on the configuration information of the excess resources; the sending module is used to send the login information of the first virtual accelerator to the client of the tenant, and the login information is used for the tenant to log in and use the first virtual accelerator and the excess storage resources.

[0020] In combination with the second aspect, in some implementations of the second aspect, the configuration information of the excess resources includes the mode, size, type, and speed requirement of the excess resources.

[0021] In combination with the second aspect, in some implementations of the second aspect, the excess storage resource is located on a second storage resource, and the second storage resource includes at least one of the following: the first storage resource, a storage resource on a second physical accelerator in the first server, and a system storage resource of the first server.

[0022] In combination with the second aspect, in certain implementations of the second aspect, the processing module is further used to determine the location of the second storage resource according to at least one of the following information: a resource size of the first physical accelerator, a resource size of the second physical accelerator, an inter-card bandwidth of the first physical accelerator and the second physical accelerator, and a system resource usage of the first server.

[0023] In combination with the second aspect, in certain implementations of the second aspect, the receiving module is also used to receive data sent by the tenant; the processing module is also used to compress the data by using the first computing power resource to obtain compressed data; the processing module is also used to store the compressed data in the second storage resource.

[0024] In combination with the second aspect, in some implementations of the second aspect, the second storage resource is the second storage resource, and the processing module stores the compressed data in the second storage resource through a bandwidth between the first physical accelerator and the second physical accelerator.

[0025] In combination with the second aspect, in some implementations of the second aspect, the second storage resource is a system storage resource of the first server, and the processing module stores the compressed data in the second storage resource through the bandwidth between the first physical accelerator and the first server.

[0026] In combination with the second aspect, in certain implementations of the second aspect, the processing module is also used to delete the first intermediate data stored in the second storage resource through a first instruction, so that the tenant can reuse the storage resources occupied by the first intermediate data in the second storage resource; the processing module is also used to regain the first intermediate data by calculating through the first computing power resource according to a second instruction sent by the tenant to obtain the first intermediate data.

[0027] In combination with the second aspect, in some implementations of the second aspect, the second storage resource is the second storage resource, and the processing module deletes the first intermediate data stored in the second storage resource through a bandwidth between the first physical accelerator and the second physical accelerator.

[0028] In combination with the second aspect, in some implementations of the second aspect, the second storage resource is a system storage resource of the first server, and the processing module deletes the first intermediate data stored in the second storage resource through a bandwidth between the first physical accelerator and the first server.

[0029] In a third aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method in the first aspect or any possible implementation of the first aspect.

[0030] Optionally, the processor may be a general-purpose processor, which may be implemented by hardware or software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory, which may be integrated in the processor or located outside the processor and exist independently.

[0031] In a fourth aspect, a chip is provided, which obtains instructions and executes the instructions to implement the method in the above-mentioned first aspect and any implementation manner of the first aspect.

[0032] Optionally, as an implementation manner, the chip includes a processor and a data interface, and the processor reads instructions stored in the memory through the data interface to execute the method in the above-mentioned first aspect and any implementation manner of the first aspect.

[0033] Optionally, as an implementation method, the chip may also include a memory, in which instructions are stored, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the method in the first aspect and any one of the implementation methods of the first aspect.

[0034] In a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device cluster, the computing device cluster executes the method in the first aspect and any one of the implementations of the first aspect.

[0035] In a sixth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method in the first aspect and any one of the implementations of the first aspect.

[0036] By way of example, these computer-readable storages include, but are not limited to, one or more of the following: read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), Flash memory, electrically EPROM (EEPROM), and hard drive.

[0037] Optionally, as an implementation manner, the above-mentioned storage medium may specifically be a non-volatile storage medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic block diagram of a system architecture applied to an embodiment of the present application.

[0039] Figure 2 It is a schematic block diagram of a cloud scenario applied to an embodiment of the present application.

[0040] Figure 3 The diagram is a schematic diagram of the correspondence between resources of a virtual accelerator and resources of a physical accelerator.

[0041] Figure 4 It is another schematic diagram of the correspondence between resources of a virtual accelerator and resources of a physical accelerator.

[0042] Figure 5 It is another schematic diagram of the correspondence between resources of a virtual accelerator and resources of a physical accelerator.

[0043] Figure 6 It is a schematic flowchart of a method for obtaining excess resources provided in an embodiment of the present application.

[0044] Figure 7 It is a schematic diagram of a user interface provided in an embodiment of the present application.

[0045] Figure 8 It is a schematic diagram of obtaining additional excess video memory by replacing computing resources provided in an embodiment of the present application.

[0046] Fig. 9 This is another schematic diagram of obtaining additional excess video memory by replacing computing resources provided in an embodiment of the present application.

[0047] Fig.10 This is another schematic diagram of obtaining additional excess video memory by replacing computing resources provided in an embodiment of the present application.

[0048] Fig.11 It is a schematic diagram of input information and output information of an excess resource controller provided in an embodiment of the present application.

[0049] Fig.12 This is a schematic block diagram of an excess policy between users provided in an embodiment of the present application.

[0050] Fig.13 It is a schematic block diagram of a device 1300 provided in an embodiment of the present application.

[0051] Fig.14 It is a schematic diagram of the architecture of a computing device 1500 provided in an embodiment of the present application.

[0052] Fig.15 It is a schematic diagram of the architecture of a computing device cluster provided in an embodiment of the present application.

[0053] Fig.16 It is a schematic diagram of the connection between computing devices 1500A and 1500B via a network provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0055] The present application will present various aspects, embodiments or features around a system including multiple devices, components, modules, etc. It should be understood and appreciated that each system may include additional devices, components, modules, etc., and / or may not include all devices, components, modules, etc. discussed in conjunction with the figures. In addition, combinations of these schemes may also be used.

[0056] In addition, in the embodiments of the present application, words such as "exemplary" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present concepts in a concrete way.

[0057] In the embodiments of the present application, "corresponding (corresponding, relevant)" and "corresponding (corresponding)" can sometimes be used interchangeably. It should be pointed out that when the distinction between them is not emphasized, the meanings they intend to express are consistent.

[0058] The business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person of ordinary skill in the art can appreciate that, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0059] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0060] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0061] Figure 1 is a schematic block diagram of a system architecture applied to an embodiment of the present application. Figure 1 As shown, the system includes at least one terminal device 110, a network device 120 and at least one host 130.

[0062] The terminal device 110 may be, for example, a mobile phone, a tablet computer, a notebook, a wearable electronic device with a communication function, etc. A user may send information, data, or a request to at least one host 130 in the cloud service through the terminal device 110. For example, a user may request a resource from the cloud service through the terminal device 110.

[0063] The network device 120 is used to transmit data between the terminal device 110 and at least one host 130 in the cloud service through a communication network of any communication mechanism / communication standard. The communication network can be a wide area network, a local area network, a point-to-point connection, or any combination thereof.

[0064] Host 130 refers to a cloud server, which may also be called a cloud server, and is located in a cloud data center. Multiple hosts 130 may communicate with each other. Multiple hosts 130 are located in the same cloud service. Specifically, multiple hosts 130 may be located in the same cloud or in multiple clouds. When multiple hosts 130 are located in multiple clouds, the multiple clouds may communicate with each other.

[0065] Combine the following Figure 2 ,right Figure 1 The following example illustrates the deployment of the system in a cloud scenario.

[0066] Figure 2 is a schematic block diagram of a cloud scenario applied to an embodiment of the present application. Figure 2 As shown, the cloud scenario may include: a cloud management platform 210 , the Internet 220 , and a client 230 .

[0067] like Figure 2 As shown, the cloud management platform 210 is used to manage the infrastructure that provides multiple cloud services. The infrastructure includes multiple cloud data centers, each of which includes multiple servers, each of which includes cloud service resources to provide corresponding cloud services to tenants. As an example, the servers included in each cloud data center may correspond to Figure 1 Host 130 in.

[0068] The cloud management platform 210 may be located in a cloud data center, which may provide an access interface (such as an interface or an application program interface (API)). The tenant may operate the client 230 to remotely access the access interface to register a cloud account and password on the cloud management platform 210, and log in to the cloud management platform 210. After the cloud management platform 210 successfully authenticates the cloud account and password, the tenant may further pay to select and purchase a virtual machine of a specific specification (processor, memory, disk) on the cloud management platform 210. After the paid purchase is successful, the cloud management platform 210 provides the remote login account and password of the purchased virtual machine, and the client 230 may remotely log in to the virtual machine, install and run the tenant's application in the virtual machine. Therefore, the tenant may create, manage, log in and operate a virtual machine in the cloud data center through the cloud management platform 210. Among them, the virtual machine may also be called a cloud server (elastic compute service, ECS) or an elastic instance (different cloud service providers have different names).

[0069] It should be understood that tenants of cloud services can be individuals, enterprises, schools, hospitals, administrative agencies, etc.

[0070] The functions of the cloud management platform 210 include, but are not limited to, user console, computing management service, network management service, storage management service, authentication service, and image management service. The user console provides an interface or API to interact with tenants, the computing management service is used to manage servers running virtual machines and containers and bare metal servers, the network management service is used to manage network services (such as gateways, firewalls, etc.), the storage management service is used to manage storage services (such as data bucket services), the authentication service is used to manage tenants' accounts and passwords, and the image management service is used to manage virtual machine images. Tenants can log in to the cloud management platform 210 through the Internet 220 using the client 230 to manage the rented cloud services.

[0071] by Figure 1For example, each host 130 includes one or more physical accelerators, each of which includes storage resources (video memory) and computing resources. In order to improve resource utilization, the resources of multiple physical accelerators on multiple hosts 130 under the same cloud service can be divided into multiple virtual accelerators. Each virtual accelerator can serve one or more users, wherein the resources in a physical accelerator can be divided into one or more virtual accelerators.

[0072] It should be understood that the above-mentioned accelerator is a computing unit specifically used for parallel computing of large amounts of data, which can be called a parallel computing unit or a graphics card device. As a service parallel computing resource in cloud computing services, accelerators provide flexible and cost-effective computing capabilities for industries with high-density computing requirements such as graphics processing and artificial intelligence. The accelerator may include computing resources and storage resources (video memory), wherein the computing resources may include but are not limited to: graphic processing unit (GPU), tensor processing units (TPU), neural processing unit (NPU).

[0073] It should also be understood that accelerator virtualization plays a key role in cloud computing services as an important means to reduce costs and improve resource utilization. Virtual accelerators are mainly virtual devices based on physical accelerators after assembling them into cloud cluster resources. The virtual devices provide virtualized computing resources.

[0074] It should be noted that the accelerator virtualization process will segment and isolate the physical resources of the original physical accelerator to ensure that users have safe, stable, and independent computing resources. The segmentation and isolation of the original physical accelerator resources mainly include computing power, video memory, and bandwidth.

[0075] An example, such as Figure 3 As shown, assuming that the cloud service includes at least host A, and host A includes physical accelerator 1 and physical accelerator 2, when user 1 requests a service from the cloud service through terminal device 1, the cloud service can divide part or all of the resources in physical accelerator 1 on host A into virtual accelerator 1, and virtual accelerator 1 serves user 1. When user 2 requests the cloud service through terminal device 2, part or all of the resources in physical accelerator 2 on host A can be divided into virtual accelerator 2, and virtual accelerator 2 serves user 2.

[0076] Another example, such as Figure 4As shown, assuming that the cloud service includes at least host A, and host A includes physical accelerator 1 and physical accelerator 2, when user 1 requests a service from the cloud service through terminal device 1, the cloud service can divide part of the resources in physical accelerator 1 on host A into virtual accelerator 1, and virtual accelerator 1 serves user 1. When user 2 requests the cloud service through terminal device 2, the remaining resources in physical accelerator 1 on host A can be divided into virtual accelerator 2, and virtual accelerator 2 serves user 2.

[0077] Another example, such as Figure 5 As shown, assuming that the cloud service includes at least host A, and host A includes physical accelerators 1, 2, and 3, when user 1 requests a service from the cloud service through terminal device 1, the cloud service can divide some or all resources of physical accelerators 1 and 2 on host A into virtual accelerator 1, and virtual accelerator 1 serves user 1. When user 2 requests the cloud service through terminal device 2, some or all resources of physical accelerator 3 on host A can be divided into virtual accelerator 2, and virtual accelerator 2 serves user 2.

[0078] In the above system, there may be a situation where the storage resources in a certain virtual accelerator cannot meet the user's needs. For example, in some task scenarios, when the virtual accelerator executes a certain user's tasks, it is found that after the tasks are completed, more storage resources are needed, but the storage resources in the virtual accelerator serving the user are all occupied and no more storage resources can be provided. In this case, there is a situation where the storage resources in the virtual accelerator cannot meet the user's needs.

[0079] In the above system, there may be a situation where the storage resources in a certain virtual accelerator are not fully utilized. For example, in some scenarios or for a certain user, the storage resources used by the user are relatively small, or in the process of the virtual accelerator executing the user's tasks, it is found that the storage resources required to complete these tasks are relatively small, and many of the storage resources in the virtual accelerator serving the user are idle, and these storage resources are not fully utilized. This situation will inevitably cause a large amount of resource fragmentation and waste, so that the idle video memory resources of all virtual accelerators in the resource pool are not fully utilized within a unit time period, and this amount is very large for the entire resource pool.

[0080] In a related technical solution, the problem that a single user's video memory resources cannot meet user needs is solved by borrowing video memory / memory space in other locations. However, since the excess video memory resources in this solution are only obtained through space transfer, the use of the excess video memory resources will still be limited by the physical video memory resources on the physical accelerator.

[0081] In view of this, an embodiment of the present application provides a method for obtaining excess resources, which can make the upper limit of resources available on the virtual accelerator not limited by physical resources, so that the resources on the virtual accelerator that the user can obtain exceed the physical resources on the physical accelerator.

[0082] As an example, the method provided in the embodiment of the present application can be applied to the scenario of the debugging mode of artificial intelligence training / inference. It should be understood that in the application scenario of artificial intelligence, the training operation (model training) is a process that consumes computing power resources, and it is also a debugging process that requires continuous trial and error.

[0083] Figure 6 FIG. 1 is a schematic flow chart of a method for obtaining excess resources provided in an embodiment of the present application. Figure 6 As shown, the method may include steps 610-620, and steps 610-620 are described in detail below.

[0084] It should be understood that Figure 6 The examples are only intended to help those skilled in the art understand the embodiments of the present application, and are not intended to limit the embodiments of the present application to Figure 6 The specific numerical values ​​or specific scenarios shown in the examples. Figure 6 The examples given are obviously susceptible to various equivalent modifications or changes, and such modifications and changes also fall within the scope of the embodiments of the present application.

[0085] Step 610: The user inputs information about excess resources.

[0086] In the embodiment of the present application, the user can create a virtual accelerator through the cloud management platform, and manage the virtual accelerator through the cloud management platform throughout its life cycle. Specifically, the tenant can input the configuration information of the virtual accelerator to be created through the user interface provided by the cloud management platform, and the cloud management platform can send the configuration information of the virtual accelerator to the cloud server (also referred to as the host) that supports the specification. The cloud server runs a cloud management platform client function, which mainly receives the control plane command sent by the cloud management platform, creates a virtual machine accelerator on the cloud server according to the control plane control command, and manages the virtual machine accelerator through its life cycle.

[0087] In an embodiment of the present application, the above-mentioned user interface also adds the setting of excess resources, so that the tenant can input relevant information of the excess resources through the user interface, so that when the storage resources in a certain virtual accelerator cannot meet the user's needs, the excess resources can be used, thereby improving the overall utilization of physical resources.

[0088] For example, taking the above debugging mode as an example, a user interface provided to tenants is as follows: Figure 7See Figure 7 The user interface may include the setting of target resource specifications and the setting of excess resources in debugging mode. Among them, the tenant can create the corresponding virtual accelerator by setting the target resource specifications, for example, the specification size of GPU / NPU. The tenant can also create the corresponding excess resources by setting the excess resources in debugging mode. For example, after selecting the target resource specifications of the virtual accelerator, the tenant selects the excess mode, excess size, excess type, excess speed, etc.

[0089] The above-mentioned excess mode may be, for example, single-user or multi-user, that is, single-user represents an excess policy within the same user, and multi-user represents an excess policy between users.

[0090] The above-mentioned excess size can be, for example, a percentage of memory excess, a percentage of computing power excess, or a percentage of bandwidth excess. For example, the percentage of memory excess indicates the percentage of memory resource usage to the total amount of storage resources, the percentage of computing power excess indicates the percentage of computing power resource usage to the total amount of computing power resources, and the percentage of bandwidth excess indicates the percentage of bandwidth resource usage to the total amount of bandwidth resources.

[0091] The above-mentioned excess types may include, for example, excess video memory, excess computing power, excess bandwidth, etc.

[0092] The above-mentioned excess speed is represented by setting the speed of different gears. For example, in the embodiment of the present application, three gears of "fast", "normal" and "economic" are set for users to choose.

[0093] Step 620: Acquire additional excess resources by replacing computing resources.

[0094] In the embodiment of the present application, additional excess resources can be obtained by replacing computing resources, so that the upper limit of resources available on the virtual accelerator is not limited by physical resources, so that the resources on the virtual accelerator that the user can obtain exceed the physical resources on the physical accelerator.

[0095] There are many specific implementation methods for obtaining additional excess video memory by replacing computing resources. The embodiments of the present application do not specifically limit this. Several possible replacement methods are introduced below.

[0096] In a possible implementation, the memory resources of other physical accelerators on a host A (also called a cloud server) can be used to obtain excess video memory on other physical accelerators by replacing computing resources. Figure 8As shown, it is assumed that host A includes physical accelerator 1 and physical accelerator 2, wherein the extra excess video memory on physical accelerator 2 can be obtained by replacing the computing power resources of physical accelerator 1, and the extra excess video memory on physical accelerator 2 is used as the video memory of physical accelerator 1. Physical accelerator 1 uses the video memory of physical accelerator 2 through the connection line between physical accelerator 1 and physical accelerator 2.

[0097] It should be understood that in the above implementation, when physical accelerator 1 uses the video memory of physical accelerator 2 through the connection line between physical accelerator 1 and physical accelerator 2, physical accelerator 1 does not directly store data in the video memory of physical accelerator 2, but stores the data in the video memory of physical accelerator 2 in a compressed manner. This compression process requires computing power resources on physical accelerator 1 to be processed, and the video memory consumption of physical accelerator 2 is lower than directly using the video memory on physical accelerator 2.

[0098] It should be understood that compression here is to reduce the physical space required for data, and to perform transformations to reduce storage space, such as re-expressing data through mathematical formulas or adjusting data precision without affecting usage.

[0099] It should be noted that the above implementation needs to consider the inter-card bandwidth of the physical accelerator 1 and the physical accelerator 2. Generally speaking, the above implementation can be used when the inter-card bandwidth of the physical accelerator 1 and the physical accelerator 2 is sufficient.

[0100] In another possible implementation, on the same physical accelerator, excess video memory resources are obtained by replacing computing resources, that is, part of the video memory on the same physical accelerator is reused in space. Fig. 9 As shown, some of the data stored in the video memory resources of the physical accelerator 1 in the host A are intermediate data. We can release some intermediate data (such as intermediate data B) and recalculate the intermediate data B through the computing resources in the physical accelerator 1 when needed.

[0101] In another possible implementation, the computing power resources and memory resources on the system are used to store the data that the physical accelerator's video memory needs to store. We regard part of the memory resources in the system resources as "external memory resources of the accelerator", such as Fig.10 As shown, some of the data stored in the video memory resources of the physical accelerator 1 in the host A are intermediate data (such as intermediate data B). We can store some intermediate data (such as intermediate data B) in the "external memory resources of the accelerator".

[0102] It should be understood that the intermediate data B may also be an intermediate result with a smaller number of accesses or data that only needs to be repeatedly accessed once.

[0103] It should be noted that the above implementation method needs to consider the bandwidth between the system resources and the physical accelerator. Generally speaking, the above implementation method can be used when the bandwidth between the system resources and the physical accelerator is sufficient.

[0104] In the embodiment of the present application, in order to implement the above method and obtain the optimal solution for the resources on the host A, an excess resource controller is required, such as Fig.11 As shown, the excess resource controller input data includes but is not limited to: the usage and bandwidth of system resources; the computing power, video memory, and intra-card bandwidth of the accelerator; the inter-card bandwidth and access load between accelerators, etc. The excess resource controller manages the excess resources based on these input data, and the management may include but is not limited to: managing the creation, location, size, and life cycle of the excess video memory of the virtual accelerator.

[0105] It should be understood that the above-mentioned excess resource controller can be deployed on a cloud management platform, or on a cloud server in a cloud data center, and the embodiments of the present application do not make specific limitations on this.

[0106] As an example, taking the above-mentioned host A including physical accelerator 1 and physical accelerator 2 as an example, the information input by host A to the excess resource controller may include but is not limited to: the usage and bandwidth of system resources; the computing power, video memory, and in-card bandwidth of physical accelerator 1; the computing power, video memory, and in-card bandwidth of physical accelerator 2; the inter-card bandwidth and access load between physical accelerator 1 and physical accelerator 2, etc.

[0107] In the embodiment of the present application, after obtaining the input information, the excess resource controller manages the excess resources based on the balance strategy and the excess strategy. The balance strategy and the excess strategy are explained below.

[0108] The above balance strategy is a strategy to balance different physical accelerators, system resources, and communication links on a host (for example, host A) to optimize all resources on host A. That is, the balance strategy considers the resource usage of different physical accelerators on host A (including computing power, video memory, intra-card bandwidth, inter-card bandwidth), and the usage of system resources (including system resource usage and bandwidth), so as to determine the above balance strategy. Figure 8-Figure 10 Which of the following is the optimal deployment location for excess video memory? There are many specific implementation methods, one of which is based on manual tuning and the other is based on machine learning.

[0109] For manual tuning, since the current operating status of the virtual accelerator is known, the consumption generated by different excess methods can be obtained through testing (historical experience), and then the weighted addition input logic classification judgment is configured according to the input parameters to obtain the best location for the next excess video memory to be opened.

[0110] For machine learning-based methods, this can be achieved through the reinforcement learning method in machine learning. The longer the system runs and the more data is collected, the better the control effect will be.

[0111] The above excess strategy is a strategy that implements the "excess method" on the physical accelerator. There are two types of excess strategies based on the user dimension: one is the excess strategy between users, and the other is the excess strategy within the same user.

[0112] For example, Fig.12 The excess strategy between users is shown. This strategy means that excess resources are obtained through the idle resources of other users. The idle resources of other users can be computing power, bandwidth, system resources, etc. Specifically, suppose that user A and user B each create a virtual accelerator A and B. The resources created by virtual accelerator A are mainly implemented on physical accelerator 1, and the resources created by virtual accelerator B are mainly implemented on physical accelerator 2. User A uses more memory resources, but there are more idle computing resources; user B uses more computing resources, but there is more memory resource space. At this time, we create a ServerA on physical accelerator 1 to be responsible for the external service of computing resources; and create a ServerB on physical accelerator 2 to be responsible for the external service of memory resources. In this way, user A can treat the memory resources on physical accelerator 2 as excess video memory resources; user B can treat the computing resources on physical accelerator 1 as excess computing resources.

[0113] It should be noted that Fig.12 The method shown is the same as Figure 8 It is similar to the method of consuming bandwidth between cards, so it is considered a method of replacing computing power and memory with bandwidth.

[0114] The excess policy within the same user means that the excess resources are not obtained by using the physical resources occupied by other virtual accelerators, but the excess resources are obtained by the virtual accelerator of the same user in an excess manner. Fig. 9 The method can correspond to the excess policy within the same user. For details, please refer to Fig. 9 The description in will not be repeated here.

[0115] Optionally, in the embodiment of the present application, the above method for obtaining excess resources can also be applied to the scenario of the debugging mode of artificial intelligence training / reasoning. Since the resource usage between users in the debugging mode is different, the original way of using the virtual accelerator will cause a large waste of resources, and the method provided in the embodiment of the present application can better improve the utilization rate of resources.

[0116] Combination of the above Figures 1 to 12 , describes in detail the method provided by the embodiment of the present application, and will be combined with Figure 13-Figure 16 , describes in detail the embodiment of the device of the present application. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so the parts not described in detail can refer to the previous method embodiment.

[0117] Fig.13 1 is a schematic block diagram of a device 1300 provided in an embodiment of the present application. The device 1300 can be implemented by software, hardware, or a combination of both. The device 1300 provided in an embodiment of the present application can implement the Figure 6 According to the method flow shown, the device 1300 includes: a receiving module 1310, a processing module 1320, and a sending module 1330. The receiving module 1310 is used to receive the configuration information of the first virtual accelerator and the configuration information of the excess resources of the first virtual accelerator sent by the tenant, wherein the configuration information of the first virtual accelerator includes the size and type of the first virtual accelerator selected by the tenant, and the excess resources include excess storage resources; the processing module 1320 is used to allocate the resources of the first physical accelerator in the first server to the first virtual accelerator according to the size and type of the first virtual accelerator selected by the tenant, and the resources of the first physical accelerator include first storage resources, first computing resources, and first bandwidth resources; the processing module 1320 is also used to replace and obtain the excess storage resources by consuming the first computing resources based on the configuration information of the excess resources; the sending module 1330 is used to send the login information of the first virtual accelerator to the client of the tenant, and the login information is used for the tenant to log in and use the first virtual accelerator and the excess storage resources.

[0118] Optionally, the configuration information of the excess resources includes the mode, size, type and speed requirement of the excess resources.

[0119] Optionally, the excess storage resource is located on a second storage resource, and the second storage resource includes at least one of the following: the first storage resource, a storage resource on a second physical accelerator in the first server, and a system storage resource of the first server.

[0120] Optionally, the processing module 1320 is further used to determine the location of the second storage resource according to at least one of the following information: resource size of the first physical accelerator, resource size of the second physical accelerator, inter-card bandwidth of the first physical accelerator and the second physical accelerator, and system resource usage of the first server.

[0121] Optionally, the receiving module 1310 is also used to receive data sent by the tenant; the processing module 1320 is also used to compress the data by using the first computing power resource to obtain compressed data; the processing module 1320 is also used to store the compressed data to the second storage resource.

[0122] Optionally, the second storage resource is the second storage resource, and the processing module 1320 stores the compressed data in the second storage resource through the bandwidth between the first physical accelerator and the second physical accelerator.

[0123] Optionally, the second storage resource is a system storage resource of the first server, and the processing module 1320 stores the compressed data in the second storage resource through the bandwidth between the first physical accelerator and the first server.

[0124] Optionally, the processing module 1320 is also used to delete the first intermediate data stored in the second storage resource through a first instruction, so that the tenant can reuse the storage resources occupied by the first intermediate data in the second storage resource; the processing module 1320 is also used to regain the first intermediate data by calculating through the first computing power resource according to a second instruction sent by the tenant to obtain the first intermediate data.

[0125] Optionally, the second storage resource is the second storage resource, and the processing module 1320 deletes the first intermediate data stored in the second storage resource through the bandwidth between the first physical accelerator and the second physical accelerator.

[0126] Optionally, the second storage resource is a system storage resource of the first server, and the processing module 1320 deletes the first intermediate data stored in the second storage resource through the bandwidth between the first physical accelerator and the first server.

[0127] The device 1300 here can be embodied in the form of a functional module. The term "module" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0128] For example, a "module" can be a software program, a hardware circuit, or a combination of the two that implements the above functions. Exemplarily, the implementation of the receiving module 1310 is described below by taking the receiving module 1310 as an example. Similarly, the implementation of other modules, such as the processing module 1320 and the sending module 1330, can refer to the implementation of the receiving module 1310.

[0129] The receiving module 1310 is taken as an example of a software functional unit, and the receiving module 1310 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the receiving module 1310 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with similar geographical locations. Among them, usually a region may include multiple AZs.

[0130] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0131] The receiving module 1310 is taken as an example of a hardware functional unit, and the receiving module 1310 may include at least one computing device, such as a server, etc. Alternatively, the receiving module 1310 may also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0132] The multiple computing devices included in the receiving module 1310 can be distributed in the same region or in different regions. The multiple computing devices included in the receiving module 1310 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the receiving module 1310 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0133] Therefore, the modules of each example described in the embodiments of the present application can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0134] It should be noted that: when the device provided in the above embodiment executes the above method, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. For example, the receiving module 1310 can be used to execute any step in the above method, the processing module 1320 can be used to execute any step in the above method, and the sending module 1330 can be used to execute any step in the above method. The steps that the receiving module 1310, the processing module 1320, and the sending module 1330 are responsible for implementing can be specified as needed, and the full functions of the above device are realized by respectively implementing different steps in the above method through the receiving module 1310, the processing module 1320, and the sending module 1330.

[0135] In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation processes are detailed in the method embodiments above, which will not be repeated here.

[0136] The method provided in the embodiment of the present application can be performed by a computing device, which can also be referred to as a computer system. It includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a processing unit, a memory and a memory control unit, and then the function and structure of the hardware are described in detail. The operating system is any one or more computer operating systems that implement business processing through a process, for example, a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a windows operating system. The application layer includes applications such as a browser, an address book, a word processing software, and an instant messaging software. In addition, optionally, the computer system is a handheld device such as a smart phone, or a terminal device such as a personal computer, and the present application is not particularly limited, as long as the method provided by the embodiment of the present application can be used. The execution subject of the method provided in the embodiment of the present application can be a computing device, or a functional module in a computing device that can call a program and execute a program.

[0137] Combine the following Fig.14 , a computing device provided in an embodiment of the present application is described in detail.

[0138] Fig.14 1 is a schematic diagram of the architecture of a computing device 1500 provided in an embodiment of the present application. The computing device 1500 may be a server or a computer or other device with computing capabilities. Fig.14 The computing device 1500 shown includes at least one processor 1510 and a memory 1520 .

[0139] It should be understood that the present application does not limit the number of processors and memories in the computing device 1500 .

[0140] The processor 1510 executes the instructions in the memory 1520, so that the computing device 1500 implements the method provided by the present application. Alternatively, the processor 1510 executes the instructions in the memory 1520, so that the computing device 1500 implements the functional modules provided by the present application, thereby implementing the method provided by the present application.

[0141] Optionally, the computing device 1500 further includes a communication interface 1530. The communication interface 1530 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1500 and other devices or a communication network.

[0142] Optionally, the computing device 1500 further includes a system bus 1540, wherein the processor 1510, the memory 1520, and the communication interface 1530 are respectively connected to the system bus 1540. The processor 1510 can access the memory 1520 through the system bus 1540. For example, the processor 1510 can read and write data or execute code in the memory 1520 through the system bus 1540. The system bus 1540 is a peripheral component interconnect express (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus 1540 is divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.14 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0143] In a possible implementation, the function of the processor 1510 is mainly to interpret the instructions (or codes) of the computer program and process the data in the computer software. The instructions of the computer program and the data in the computer software can be stored in the memory 1520 or the cache 1516.

[0144] Optionally, the processor 1510 may be an integrated circuit chip with signal processing capabilities. As an example and not a limitation, the processor 1510 is a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Among them, the general-purpose processor is a microprocessor, etc. For example, the processor 1510 is a central processing unit (CPU).

[0145] Optionally, each processor 1510 includes at least one processing unit 1512 and a memory control unit 1514 .

[0146] Optionally, the processing unit 1512 is also called a core or kernel, which is the most important component of the processor. The processing unit 1512 is manufactured from single crystal silicon using a certain production process, and all calculations, command reception, command storage, and data processing of the processor are performed by the core. The processing units run program instructions independently and use the ability of parallel computing to speed up the running speed of the program. Various processing units have a fixed logical structure. For example, the processing unit includes logical units such as a first-level cache, a second-level cache, an execution unit, an instruction-level unit, and a bus interface.

[0147] In one implementation example, the memory control unit 1514 is used to control data interaction between the memory 1520 and the processing unit 1512. Specifically, the memory control unit 1514 receives a memory access request from the processing unit 1512, and controls access to the memory based on the memory access request. As an example and not a limitation, the memory control unit is a device such as a memory management unit (MMU).

[0148] In one implementation example, each memory control unit 1514 addresses the memory 1520 through the system bus. And an arbiter ( Fig.14 ), which is responsible for handling and coordinating competing accesses of multiple processing units 1512.

[0149] In an implementation example, the processing unit 1512 and the memory control unit 1514 are connected to each other through connection lines inside the chip, such as address lines, so as to achieve communication between the processing unit 1512 and the memory control unit 1514.

[0150] Optionally, each processor 1510 also includes a cache 1516, wherein the cache is a buffer for data exchange (called cache). When the processing unit 1512 wants to read data, it will first search for the required data from the cache. If it is found, it will be executed directly. If it is not found, it will be searched from the memory. Since the running speed of the cache is much faster than the memory, the role of the cache is to help the processing unit 1512 run faster.

[0151] The memory 1520 can provide a running space for the processes in the computing device 1500. For example, the computer program (specifically, the program code) used to generate the process is stored in the memory 1520. After the computer program is executed by the processor to generate the process, the processor allocates a corresponding storage space for the process in the memory 1520. Furthermore, the above storage space further includes a text segment, an initialized data segment, a bit initialized data segment, a stack segment, a heap segment, etc. The memory 1520 stores the data generated during the running of the process in the storage space corresponding to the above process, such as intermediate data, process data, etc.

[0152] Optionally, the storage is also called memory, and its function is to temporarily store the operation data in the processor 1510 and the data exchanged with the external storage such as the hard disk. As long as the computer is running, the processor 1510 will transfer the data to be calculated to the memory for calculation, and when the calculation is completed, the processing unit 1512 will transmit the result.

[0153] As an example and not limitation, memory 1520 is a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory is a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory is a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DRRAM). It should be noted that the memory 1520 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0154] The structure of the computing device 1500 listed above is only an example, and the present application is not limited thereto. The computing device 1500 of the embodiment of the present application includes various hardware in the computer system in the prior art. For example, the computing device 1500 also includes other memories besides the memory 1520, such as disk storage, etc. Those skilled in the art should understand that the computing device 1500 may also include other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the above-mentioned computing device 1500 may also include hardware devices for implementing other additional functions. In addition, those skilled in the art should understand that the above-mentioned computing device 1500 may also include only the devices necessary to implement the embodiment of the present application, and does not necessarily include Fig.14 All devices shown in .

[0155] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device may be a server. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0156] like Fig.15 As shown, the computing device cluster includes at least one computing device 1500. The memory 1520 in one or more computing devices 1500 in the computing device cluster may store the same instructions for executing the above method.

[0157] In some possible implementations, the memory 1520 in one or more computing devices 1500 in the computing device cluster may also store some instructions for executing the above method. In other words, the combination of one or more computing devices 1500 may jointly execute the instructions of the above method.

[0158] It should be noted that the memory 1520 in different computing devices 1500 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the above-mentioned apparatus. That is, the instructions stored in the memory 1520 in different computing devices 1500 may implement the functions of one or more modules in the above-mentioned apparatus.

[0159] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.16 A possible implementation is shown. Fig.16 As shown, two computing devices 1500A and 1500B are connected via a network. Specifically, they are connected to the network via a communication interface in each computing device.

[0160] It should be understood that Fig.16The functionality of the computing device 1500A shown in FIG. 1 may also be implemented by multiple computing devices 1500. Similarly, the functionality of the computing device 1500B may also be implemented by multiple computing devices 1500.

[0161] In this embodiment, a computer program product including instructions is also provided, and the computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on a computing device, the computing device is caused to execute the method provided above, or the computing device is caused to implement the function of the apparatus provided above.

[0162] In this embodiment, a computer-readable storage medium is also provided. The computer-readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions. When the instructions in the computer-readable storage medium are executed on a computing device, the computing device executes the method provided above.

[0163] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0164] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0165] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0166] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0167] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0168] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0169] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0170] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for obtaining excess resources, characterized in that: The method is applied to a cloud management platform, the cloud management platform is used to manage an infrastructure for providing cloud services, the infrastructure includes at least one server, each of the servers runs at least one virtual accelerator, and the method includes: The cloud management platform receives configuration information of the first virtual accelerator and configuration information of excess resources of the first virtual accelerator sent by the tenant, wherein the configuration information of the first virtual accelerator includes the size and type of the first virtual accelerator selected by the tenant, and the excess resources include excess storage resources; The cloud management platform allocates resources of a first physical accelerator in a first server to the first virtual accelerator according to the size and type of the first virtual accelerator selected by the tenant, where the resources of the first physical accelerator include first storage resources, first computing power resources, and first bandwidth resources; The cloud management platform obtains the excess storage resources by replacing them by consuming the first computing resources based on the configuration information of the excess resources; The cloud management platform sends login information of the first virtual accelerator to the client of the tenant, where the login information is used by the tenant to log in and use the first virtual accelerator and the excess storage resources.

2. The method according to claim 1, characterized in that The configuration information of the excess resources includes the mode, size, type and speed requirement of the excess resources.

3. The method according to claim 1 or 2, characterized in that: The excess storage resource is located on a second storage resource, and the second storage resource includes at least one of the following: the first storage resource, a storage resource on a second physical accelerator in the first server, and a system storage resource of the first server.

4. The method according to claim 3, characterized in that The method further comprises: The cloud management platform determines the location of the second storage resource according to at least one of the following information: resource size of the first physical accelerator, resource size of the second physical accelerator, inter-card bandwidth of the first physical accelerator and the second physical accelerator, and system resource usage of the first server.

5. The method according to claim 3 or 4, characterized in that: The step of replacing and obtaining the excess storage resources by consuming the first computing power resources includes: The cloud management platform receives data sent by the tenant; The cloud management platform compresses the data by using the first computing resource to obtain compressed data; The cloud management platform stores the compressed data in the second storage resource.

6. The method according to claim 3 or 4, characterized in that: The second storage resource stores the first intermediate data, and the step of replacing and obtaining the excess storage resource by consuming the first computing power resource includes: The cloud management platform deletes the first intermediate data stored in the second storage resource through a first instruction, so that the tenant can reuse the storage resources occupied by the first intermediate data in the second storage resource; The cloud management platform regains the first intermediate data by calculating with the first computing resource according to the second instruction sent by the tenant to obtain the first intermediate data.

7. A device for obtaining excess resources, characterized in that: The device is applied to a cloud management platform, and the cloud management platform is used to manage an infrastructure for providing cloud services, wherein the infrastructure includes at least one server, and each of the servers runs at least one virtual accelerator. The device includes: A receiving module, configured to receive configuration information of a first virtual accelerator and configuration information of excess resources of the first virtual accelerator sent by a tenant, wherein the configuration information of the first virtual accelerator includes a specification size and type of the first virtual accelerator selected by the tenant, and the excess resources include excess storage resources; a processing module, configured to allocate resources of a first physical accelerator in a first server to the first virtual accelerator according to a size and type of the first virtual accelerator selected by the tenant, wherein the resources of the first physical accelerator include first storage resources, first computing power resources, and first bandwidth resources; The processing module is further configured to replace and obtain the excess storage resources by consuming the first computing power resources based on the configuration information of the excess resources; A sending module is used to send login information of the first virtual accelerator to the client of the tenant, where the login information is used by the tenant to log in and use the first virtual accelerator and the excess storage resources.

8. The device according to claim 7, characterized in that The configuration information of the excess resources includes the mode, size, type and speed requirement of the excess resources.

9. The device according to claim 7 or 8, characterized in that The excess storage resource is located on a second storage resource, and the second storage resource includes at least one of the following: the first storage resource, a storage resource on a second physical accelerator in the first server, and a system storage resource of the first server.

10. The device according to claim 9, characterized in that The processing module is further used to determine the location of the second storage resource according to at least one of the following information: resource size of the first physical accelerator, resource size of the second physical accelerator, inter-card bandwidth of the first physical accelerator and the second physical accelerator, and system resource usage of the first server.

11. The device according to claim 9 or 10, characterized in that The receiving module is further used to receive data sent by the tenant; The processing module is further used to compress the data using the first computing power resource to obtain compressed data; The processing module is further configured to store the compressed data in the second storage resource.

12. The device according to claim 9 or 10, characterized in that The processing module is further configured to delete the first intermediate data stored in the second storage resource through a first instruction, so that the tenant can reuse the storage resource occupied by the first intermediate data in the second storage resource; The processing module is further configured to retrieve the first intermediate data by calculating with the first computing power resources according to a second instruction sent by the tenant for obtaining the first intermediate data.

13. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 6.

14. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 6.

15. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 6.

Images