Excess resource acquisition method and apparatus and computing device

Abstract

The present application provides an excess resource acquisition method and apparatus and a computing device. The method comprises: a cloud management platform receiving configuration information of a first virtual accelerator sent by a tenant and configuration information of excess resources of the first virtual accelerator, the excess resources comprising an excess storage resource; on the basis of the specification and type of the first virtual accelerator selected by the tenant, allocating a resource of a first physical accelerator in a first server to the first virtual accelerator; on the basis of the configuration information of the excess resources, consuming a computing power resource of the first physical accelerator for replacing to obtain the excess storage resource; and sending login information of the first virtual accelerator to a client of the tenant, wherein the login information is used for the tenant to log in and use the first virtual accelerator and the excess storage resource. The method can enable the upper limit of resources available on virtual accelerators to not be limited by physical resources.

Description

Method, device and computing equipment for obtaining excess resources

[0001] This application claims priority to Chinese patent application number 202311099298.1 filed on August 29, 2023, entitled “Method and computing device for determining excess video memory”, and priority to Chinese patent application number 202311458477.X filed on November 3, 2023, entitled “Method, device and computing device for obtaining excess resources”, all of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of cloud computing, and more specifically, to a method, apparatus, and computing device for acquiring excess resources. Background Art

[0003] An accelerator is a computing unit specifically designed for large-scale data parallel computing, and can be called a parallel computing unit or graphics card device. In cloud computing services, accelerators, as a service parallel computing resource, provide flexible and cost-effective computing power for industries with high-density computing requirements, such as graphics processing and artificial intelligence. Accelerator virtualization, as an important means of reducing costs and improving resource utilization, plays a key role in cloud computing services. A virtual accelerator is essentially a virtual device implemented based on a physical accelerator after being assembled into cloud cluster resources. This virtual device provides virtualized computing resources. When using a virtual accelerator, users may encounter situations where the storage resources in a virtual accelerator do not meet their needs or are not fully utilized.

[0004] In a related technical solution, the problem of a single user's video memory resources not being able to meet user needs is solved by borrowing video memory / memory space in other locations. However, since the excess video memory resources in this solution are only obtained through space transfer, the use of the excess video memory resources is still limited by the physical video memory resources on the physical accelerator.

[0005] Therefore, how to ensure that the upper limit of resources available on the virtual accelerator is not limited by physical resources has become a technical problem that urgently needs to be solved.

[0006] Summary of the Invention

[0007] The present application provides a method, apparatus, and computing device for obtaining excess resources. The method enables the upper limit of resources available on a virtual accelerator to be independent of physical resources, thereby enabling users to obtain resources on the virtual accelerator that exceed the physical resources on the physical accelerator.

[0008] In a first aspect, a method for obtaining excess resources is provided. The method is applied to a cloud management platform, the cloud management platform being used to manage infrastructure for providing cloud services, the infrastructure including at least one server, each server running at least one virtual accelerator. The method comprises: receiving, by the cloud management platform, configuration information of a first virtual accelerator and configuration information of excess resources of the first virtual accelerator sent by a tenant, the configuration information of the first virtual accelerator including a size and type of the first virtual accelerator selected by the tenant, the excess resources including excess storage resources; allocating, by the cloud management platform, resources of a first physical accelerator in a first server to the first virtual accelerator based on the size and type of the first virtual accelerator selected by the tenant, the resources of the first physical accelerator including a first storage resource, a first computing power resource, and a first bandwidth resource; obtaining, by the cloud management platform, the excess storage resource by consuming the first computing power resource based on the configuration information of the excess resources; and sending, by the cloud management platform, login information of the first virtual accelerator to a client of the tenant, the login information being used by the tenant to log in and use the first virtual accelerator and the excess storage resource.

[0009] In the above technical solution, additional excess resources can be obtained by replacing computing resources, so that the upper limit of resources available on the virtual accelerator is not limited by physical resources, so that the resources available to users on the virtual accelerator exceed the physical resources on the physical accelerator.

[0010] In combination with the first aspect, in some implementations of the first aspect, the configuration information of the excess resources includes the mode, size, type, and speed requirement of the excess resources.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the excess storage resource is located on a second storage resource, and the second storage resource includes at least one of the following: the first storage resource, a storage resource on a second physical accelerator in the first server, and a system storage resource of the first server.

[0012] It should be understood that the system storage resources of the first server may also be referred to as external storage resources.

[0013] In conjunction with the first aspect, in certain implementations of the first aspect, the method further includes: the cloud management platform determining the location of the second storage resource based on at least one of the following information: a resource size of the first physical accelerator, a resource size of the second physical accelerator, an inter-card bandwidth between the first physical accelerator and the second physical accelerator, and system resource usage of the first server.

[0014] In combination with the first aspect, in certain implementations of the first aspect, the cloud management platform receives data sent by the tenant; the cloud management platform compresses the data through the first computing power resource to obtain compressed data; and the cloud management platform stores the compressed data in the second storage resource.

[0015] In combination with the first aspect, in some implementations of the first aspect, the second storage resource is the second storage resource, and the cloud management platform stores the compressed data in the second storage resource through the bandwidth between the first physical accelerator and the second physical accelerator.

[0016] In combination with the first aspect, in certain implementations of the first aspect, the second storage resource is a system storage resource of the first server, and the cloud management platform stores the compressed data in the second storage resource through the bandwidth between the first physical accelerator and the first server.

[0017] In combination with the first aspect, in certain implementations of the first aspect, the cloud management platform deletes the first intermediate data stored in the second storage resource through a first instruction, so that the tenant can reuse the storage resources occupied by the first intermediate data in the second storage resource; the cloud management platform regains the first intermediate data through calculation using the first computing power resource based on the second instruction sent by the tenant to obtain the first intermediate data.

[0018] In combination with the first aspect, in some implementations of the first aspect, the second storage resource is the second storage resource, and the cloud management platform deletes the first intermediate data stored in the second storage resource through the bandwidth between the first physical accelerator and the second physical accelerator.

[0019] In combination with the first aspect, in certain implementations of the first aspect, the second storage resource is a system storage resource of the first server, and the cloud management platform deletes the first intermediate data stored in the second storage resource through the bandwidth between the first physical accelerator and the first server.

[0020] In a second aspect, a device for obtaining excess resources is provided, the device being applied to a cloud management platform, the cloud management platform being used to manage an infrastructure providing cloud services, the infrastructure comprising at least one server, each server running at least one virtual accelerator, the device comprising: a receiving module, a processing module, and a sending module. The receiving module is configured to receive configuration information of a first virtual accelerator and configuration information of excess resources of the first virtual accelerator sent by a tenant, the configuration information of the first virtual accelerator comprising the size and type of the first virtual accelerator selected by the tenant, the excess resources comprising excess storage resources; the processing module is configured to allocate resources of a first physical accelerator in a first server to the first virtual accelerator based on the size and type of the first virtual accelerator selected by the tenant, the resources of the first physical accelerator comprising first storage resources, first computing resources, and first bandwidth resources; the processing module is further configured to, based on the configuration information of the excess resources, replace the excess storage resources by consuming the first computing resources; and the sending module is configured to send login information of the first virtual accelerator to a client of the tenant, the login information being used by the tenant to log in and use the first virtual accelerator and the excess storage resources.

[0021] In combination with the second aspect, in some implementations of the second aspect, the configuration information of the excess resources includes the mode, size, type, and speed requirement of the excess resources.

[0022] In combination with the second aspect, in certain implementations of the second aspect, the excess storage resource is located on a second storage resource, and the second storage resource includes at least one of the following: the first storage resource, a storage resource on a second physical accelerator in the first server, and a system storage resource of the first server.

[0023] In conjunction with the second aspect, in certain implementations of the second aspect, the processing module is further configured to determine the location of the second storage resource based on at least one of the following information: a resource size of the first physical accelerator, a resource size of the second physical accelerator, an inter-card bandwidth between the first physical accelerator and the second physical accelerator, and system resource usage of the first server.

[0024] In combination with the second aspect, in certain implementations of the second aspect, the receiving module is also used to receive data sent by the tenant; the processing module is also used to compress the data through the first computing power resource to obtain compressed data; the processing module is also used to store the compressed data in the second storage resource.

[0025] In combination with the second aspect, in some implementations of the second aspect, the second storage resource is the second storage resource, and the processing module stores the compressed data in the second storage resource through the bandwidth between the first physical accelerator and the second physical accelerator.

[0026] In combination with the second aspect, in some implementations of the second aspect, the second storage resource is a system storage resource of the first server, and the processing module stores the compressed data in the second storage resource through the bandwidth between the first physical accelerator and the first server.

[0027] In combination with the second aspect, in certain implementations of the second aspect, the processing module is also used to delete the first intermediate data stored in the second storage resource through a first instruction, so that the tenant can reuse the storage resources occupied by the first intermediate data in the second storage resource; the processing module is also used to regain the first intermediate data through calculation using the first computing power resource according to a second instruction sent by the tenant to obtain the first intermediate data.

[0028] In combination with the second aspect, in some implementations of the second aspect, the second storage resource is the second storage resource, and the processing module deletes the first intermediate data stored in the second storage resource through the bandwidth between the first physical accelerator and the second physical accelerator.

[0029] In combination with the second aspect, in some implementations of the second aspect, the second storage resource is a system storage resource of the first server, and the processing module deletes the first intermediate data stored in the second storage resource through the bandwidth between the first physical accelerator and the first server.

[0030] In a third aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method in the first aspect or any possible implementation of the first aspect.

[0031] Optionally, the processor may be a general-purpose processor, which may be implemented in hardware or software. When implemented in hardware, the processor may be a logic circuit, an integrated circuit, or the like; when implemented in software, the processor may be a general-purpose processor implemented by reading software code stored in a memory, which may be integrated into the processor or located independently of the processor.

[0032] In a fourth aspect, a chip is provided, which obtains an instruction and executes the instruction to implement the method in the above-mentioned first aspect and any implementation manner of the first aspect.

[0033] Optionally, as an implementation, the chip includes a processor and a data interface, and the processor reads instructions stored in the memory through the data interface to execute the method in the above-mentioned first aspect and any implementation of the first aspect.

[0034] Optionally, as an implementation method, the chip may also include a memory, in which instructions are stored, and the processor is used to execute the instructions stored on the memory. When the instructions are executed, the processor is used to execute the method in the first aspect and any one of the implementation methods of the first aspect.

[0035] In a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device cluster, the computing device cluster executes the method in the first aspect and any one of the implementations of the first aspect.

[0036] In a sixth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method according to the first aspect and any one of the implementations of the first aspect.

[0037] By way of example, these computer-readable storages include, but are not limited to, one or more of the following: read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), Flash memory, electrically EPROM (EEPROM), and a hard drive.

[0038] Optionally, as an implementation manner, the above-mentioned storage medium may specifically be a non-volatile storage medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] FIG1 is a schematic block diagram of a system architecture applied to an embodiment of the present application.

[0040] FIG2 is a schematic block diagram of a cloud scenario applied to an embodiment of the present application.

[0041] FIG3 is a schematic diagram of a correspondence between resources of a virtual accelerator and resources of a physical accelerator.

[0042] FIG4 is a schematic diagram of another correspondence relationship between resources of a virtual accelerator and resources of a physical accelerator.

[0043] FIG5 is a schematic diagram of another correspondence relationship between resources of a virtual accelerator and resources of a physical accelerator.

[0044] FIG6 is a schematic flowchart of a method for obtaining excess resources provided in an embodiment of the present application.

[0045] FIG7 is a schematic diagram of a user interface provided in an embodiment of the present application.

[0046] FIG8 is a schematic diagram of obtaining additional excess video memory by replacing computing resources according to an embodiment of the present application.

[0047] FIG9 is a schematic diagram of another method of obtaining additional excess video memory by replacing computing resources according to an embodiment of the present application.

[0048] FIG10 is a schematic diagram of another method for obtaining additional excess video memory by replacing computing resources according to an embodiment of the present application.

[0049] FIG11 is a schematic diagram of input information and output information of an excess resource controller provided in an embodiment of the present application.

[0050] FIG12 is a schematic block diagram of an excess policy between users provided in an embodiment of the present application.

[0051] FIG13 is a schematic block diagram of an apparatus 1300 provided in an embodiment of the present application.

[0052] FIG14 is a schematic diagram of the architecture of a computing device 1500 provided in an embodiment of the present application.

[0053] FIG15 is a schematic diagram of the architecture of a computing device cluster provided in an embodiment of the present application.

[0054] FIG16 is a schematic diagram of a network connection between computing devices 1500A and 1500B provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The technical solution in this application will be described below with reference to the accompanying drawings.

[0056] This application will present various aspects, embodiments, or features around systems including multiple devices, components, modules, etc. It should be understood and appreciated that each system may include additional devices, components, modules, etc., and / or may not include all of the devices, components, modules, etc. discussed in conjunction with the figures. Furthermore, combinations of these aspects may also be used.

[0057] Additionally, in the embodiments of this application, words such as "exemplary" and "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner.

[0058] In the embodiments of the present application, “corresponding” and “relevant” may sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings they intend to express are consistent.

[0059] The business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person skilled in the art will appreciate that, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are equally applicable to similar technical problems.

[0060] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0061] In this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0062] FIG1 is a schematic block diagram of a system architecture used in an embodiment of the present application. As shown in FIG1 , the system includes at least one terminal device 110 , a network device 120 , and at least one host 130 .

[0063] The terminal device 110 may be, for example, a mobile phone, a tablet computer, a notebook, a wearable electronic device with communication capabilities, etc. A user may send information, data, or a request to at least one host 130 in the cloud service through the terminal device 110. For example, a user may request resources from the cloud service through the terminal device 110.

[0064] The network device 120 is used to transmit data between the terminal device 110 and at least one host 130 in the cloud service through a communication network using any communication mechanism / communication standard. The communication network can be a wide area network, a local area network, a point-to-point connection, or any combination thereof.

[0065] Host 130 refers to a cloud server, also known as a cloud server, located in a cloud data center. Multiple hosts 130 can communicate with each other. Multiple hosts 130 are located in the same cloud service. Specifically, multiple hosts 130 can be located in the same cloud or in multiple clouds. In this case, multiple clouds can communicate with each other.

[0066] The following is an example of how the system in FIG1 may be deployed in a cloud scenario, with reference to FIG2 .

[0067] FIG2 is a schematic block diagram of a cloud scenario applied to an embodiment of the present application. As shown in FIG2 , the cloud scenario may include: a cloud management platform 210 , the Internet 220 , and a client 230 .

[0068] As shown in Figure 2 , cloud management platform 210 is used to manage the infrastructure that provides multiple cloud services. This infrastructure includes multiple cloud data centers, each of which includes multiple servers. Each server contains cloud service resources and provides corresponding cloud services to tenants. For example, the servers included in each cloud data center may correspond to host 130 in Figure 1 .

[0069] The cloud management platform 210 can be located in a cloud data center, which can provide an access interface (such as an interface or an application program interface (API)). The tenant can operate the client 230 to remotely access the access interface to register a cloud account and password on the cloud management platform 210, and log in to the cloud management platform 210. After the cloud management platform 210 successfully authenticates the cloud account and password, the tenant can further pay to select and purchase a virtual machine with specific specifications (processor, memory, disk) on the cloud management platform 210. After the payment is successful, the cloud management platform 210 provides the remote login account and password of the purchased virtual machine, and the client 230 can remotely log in to the virtual machine, install and run the tenant's application in the virtual machine. Therefore, the tenant can create, manage, log in and operate virtual machines in the cloud data center through the cloud management platform 210. Among them, the virtual machine can also be called a cloud server (elastic compute service, ECS) or an elastic instance (different cloud service providers have different names).

[0070] It should be understood that tenants of cloud services can be individuals, enterprises, schools, hospitals, administrative agencies, etc.

[0071] The functions of the cloud management platform 210 include, but are not limited to, a user console, computing management services, network management services, storage management services, authentication services, and image management services. The user console provides an interface or API for interacting with tenants. The computing management service is used to manage servers running virtual machines and containers, as well as bare metal servers. The network management service is used to manage network services (such as gateways, firewalls, etc.). The storage management service is used to manage storage services (such as data bucket services). The authentication service is used to manage tenant accounts and passwords. The image management service is used to manage virtual machine images. Tenants can use the client 230 to log in to the cloud management platform 210 through the Internet 220 to manage the rented cloud services.

[0072] Taking Figure 1 as an example, each host 130 includes one or more physical accelerators, each of which includes storage resources (video memory) and computing resources. To improve resource utilization, the resources of multiple physical accelerators on multiple hosts 130 under the same cloud service can be divided into multiple virtual accelerators. Each virtual accelerator can serve one or more users, and the resources of a physical accelerator can be divided into one or more virtual accelerators.

[0073] It should be understood that the above-mentioned accelerator is a computing unit specifically designed for parallel computing of large amounts of data, which can be called a parallel computing unit or graphics card device. In cloud computing services, accelerators serve as a service parallel computing resource, providing flexible and cost-effective computing power for industries with high-density computing requirements such as graphics processing and artificial intelligence. The accelerator may include computing power resources and storage resources (video memory), where the computing power resources may include but are not limited to: graphic processing unit (GPU), tensor processing unit (TPU), neural processing unit (NPU).

[0074] It should also be understood that accelerator virtualization plays a key role in cloud computing services as an important means of reducing costs and improving resource utilization. A virtual accelerator is essentially a virtual device implemented based on a physical accelerator after it is assembled into cloud cluster resources. This virtual device provides virtualized computing resources.

[0075] It's important to note that the accelerator virtualization process divides and isolates the physical resources of the original physical accelerator to ensure users have secure, stable, and independent computing resources. This division and isolation primarily involves computing power, video memory, and bandwidth.

[0076] As an example, as shown in Figure 3, assume that the cloud service includes at least host A, which includes physical accelerators 1 and 2. When user 1 requests a service from the cloud service through terminal device 1, the cloud service can allocate some or all of the resources in physical accelerator 1 on host A to virtual accelerator 1, which then serves user 1. When user 2 requests a service from the cloud service through terminal device 2, the cloud service can allocate some or all of the resources in physical accelerator 2 on host A to virtual accelerator 2, which then serves user 2.

[0077] As another example, as shown in Figure 4, assume that the cloud service includes at least host A, which includes physical accelerators 1 and 2. When user 1 requests a service from the cloud service through terminal device 1, the cloud service can allocate some of the resources in physical accelerator 1 on host A to virtual accelerator 1, which serves user 1. When user 2 requests a service from the cloud service through terminal device 2, the remaining resources in physical accelerator 1 on host A can be allocated to virtual accelerator 2, which serves user 2.

[0078] As another example, as shown in Figure 5, assume that the cloud service includes at least host A, which includes physical accelerators 1, 2, and 3. When user 1 requests a service from the cloud service through terminal device 1, the cloud service can allocate some or all of the resources in physical accelerators 1 and 2 on host A to virtual accelerator 1, which serves user 1. When user 2 requests a service from the cloud service through terminal device 2, some or all of the resources in physical accelerator 3 on host A can be allocated to virtual accelerator 2, which serves user 2.

[0079] In the above system, it is possible that the storage resources in a certain virtual accelerator cannot meet user needs. For example, in certain task scenarios, while executing a user's task, the virtual accelerator may find that after completing the task, more storage resources are required. However, the storage resources in the virtual accelerator serving the user are all occupied, making it impossible to provide more storage resources. In this case, the storage resources in the virtual accelerator cannot meet the user's needs.

[0080] In the above system, it is also possible that the storage resources in a certain virtual accelerator are not fully utilized. For example, in certain scenarios or for a certain user, the user's storage resources are relatively few, or when the virtual accelerator is executing the user's tasks, it is found that the storage resources required to complete these tasks are relatively few. Therefore, many of the storage resources in the virtual accelerator serving the user are idle, and these storage resources are not fully utilized. This situation inevitably leads to a large amount of resource fragmentation and waste, resulting in the idle video memory resources of all virtual accelerators in the resource pool not being fully utilized within a unit time period, and this amount is very large relative to the entire resource pool.

[0081] In a related technical solution, the problem of a single user's video memory resources not being able to meet user needs is solved by borrowing video memory / memory space in other locations. However, since the excess video memory resources in this solution are only obtained through space transfer, the use of the excess video memory resources is still limited by the physical video memory resources on the physical accelerator.

[0082] In view of this, an embodiment of the present application provides a method for obtaining excess resources, which can make the upper limit of resources available on the virtual accelerator not limited by physical resources, so that the resources on the virtual accelerator that users can obtain exceed the physical resources on the physical accelerator.

[0083] As an example, the method provided in the embodiments of the present application can be applied to the debugging mode of artificial intelligence training / inference. It should be understood that in the application scenario of artificial intelligence, training operations (model training) are a process that consumes computing power resources and is also a debugging process that requires continuous trial and error.

[0084] Figure 6 is a schematic flow chart of a method for obtaining excess resources provided by an embodiment of the present application. As shown in Figure 6, the method may include steps 610-620, which are described in detail below.

[0085] It should be understood that the example in FIG6 is merely intended to help those skilled in the art understand the embodiments of the present application, and is not intended to limit the embodiments of the present application to the specific numerical values ​​or specific scenarios illustrated in FIG6. Those skilled in the art can obviously make various equivalent modifications or variations based on the example given in FIG6, and such modifications and variations also fall within the scope of the embodiments of the present application.

[0086] Step 610: The user inputs excess resource information.

[0087] In the embodiment of the present application, users can create a virtual accelerator through the cloud management platform and manage the virtual accelerator through the cloud management platform throughout its life cycle. Specifically, the tenant can input the configuration information of the virtual accelerator to be created through the user interface provided by the cloud management platform. The cloud management platform can send the configuration information of the virtual accelerator to a cloud server (also called a host) that supports the specification. The cloud server runs a cloud management platform client function, which mainly receives the control plane commands sent by the cloud management platform, creates a virtual machine accelerator on the cloud server according to the control plane control commands, and manages the virtual machine accelerator through its life cycle.

[0088] In an embodiment of the present application, the above-mentioned user interface also adds a setting for excess resources, so that tenants can enter relevant information about excess resources through the user interface, so that when the storage resources in a certain virtual accelerator cannot meet user needs, the excess resources can be used, thereby improving the overall utilization of physical resources.

[0089] For example, taking the above-mentioned debugging mode as an example, a user interface provided to the tenant is shown in Figure 7. Referring to Figure 7, the user interface may include the setting of target resource specifications and the setting of excess resources in debugging mode. Among them, the tenant can create the corresponding virtual accelerator by setting the target resource specifications, for example, the specifications of the GPU / NPU. The tenant can also create the corresponding excess resources by setting the excess resources in the debugging mode. For example, after selecting the target resource specifications of the virtual accelerator, the tenant selects the excess mode, excess size, excess type, excess speed, etc.

[0090] The above-mentioned overage mode may be, for example, single-user or multi-user. That is, single-user indicates an overage policy within the same user, and multi-user indicates an overage policy between users.

[0091] The above-mentioned excess size can be, for example, the percentage of memory excess, the percentage of computing power excess, or the percentage of bandwidth excess. For example, the percentage of memory excess indicates the percentage of memory resource usage to the total amount of storage resources, the percentage of computing power excess indicates the percentage of computing power resource usage to the total amount of computing power resources, and the percentage of bandwidth excess indicates the percentage of bandwidth resource usage to the total amount of bandwidth resources.

[0092] The above-mentioned excess types may include, for example, excess video memory, excess computing power, excess bandwidth, etc.

[0093] The above-mentioned excess speed is represented by setting the speed of different gears. For example, in the embodiment of the present application, three gears of "fast", "normal" and "economic" are set for the user to choose.

[0094] Step 620: Acquire additional excess resources by replacing computing resources.

[0095] In the embodiment of the present application, additional excess resources can be obtained by replacing computing resources, so that the upper limit of resources available on the virtual accelerator is not limited by physical resources, thereby allowing users to obtain resources on the virtual accelerator that exceed the physical resources on the physical accelerator.

[0096] There are many specific implementation methods for obtaining additional excess video memory by replacing computing resources. The embodiments of this application do not specifically limit this. Several possible replacement methods are introduced below.

[0097] In one possible implementation, the memory resources of other physical accelerators on a host A (also known as a cloud server) can be used to obtain excess video memory on other physical accelerators by exchanging computing resources. For example, as shown in Figure 8, assuming that host A includes physical accelerators 1 and 2, the additional excess video memory on physical accelerator 2 can be obtained by exchanging computing resources of physical accelerator 1, and the additional excess video memory on physical accelerator 2 can be used as the video memory of physical accelerator 1. Physical accelerator 1 uses the video memory of physical accelerator 2 through the connection line between physical accelerator 1 and physical accelerator 2.

[0098] It should be understood that in the above implementation, when physical accelerator 1 uses the video memory of physical accelerator 2 through the connection line between physical accelerator 1 and physical accelerator 2, physical accelerator 1 does not directly store data in the video memory of physical accelerator 2, but stores the data in the video memory of physical accelerator 2 in a compressed manner. This compression process requires computing power resources on physical accelerator 1 to process, and the video memory consumption of physical accelerator 2 is lower than directly using the video memory on physical accelerator 2.

[0099] It should be understood that compression here is performed to reduce the physical space required for data, and to reduce storage space by performing transformations, such as re-expressing data through mathematical formulas or adjusting data precision without affecting usability.

[0100] It should be noted that the above implementation needs to consider the inter-card bandwidth of physical accelerator 1 and physical accelerator 2. Generally speaking, the above implementation can be used when the inter-card bandwidth of physical accelerator 1 and physical accelerator 2 is sufficient.

[0101] In another possible implementation, excess memory resources can be obtained on the same physical accelerator by replacing computing resources. This means that some of the memory on the same physical accelerator is spatially reused. For example, as shown in Figure 9, some of the data stored in the memory resources of physical accelerator 1 in host A is intermediate data. We can release some of this intermediate data (for example, intermediate data B) and then recalculate intermediate data B using the computing resources in physical accelerator 1 when needed.

[0102] Another possible implementation involves collaborating the system's computing power and memory resources to store the data needed by the physical accelerator's video memory. We consider a portion of the system's memory resources as the "accelerator's external memory resources." As shown in Figure 10, some of the data stored in the video memory resources of physical accelerator 1 on host A is intermediate data (e.g., intermediate data B). This data (e.g., intermediate data B) can be stored in the "accelerator's external memory resources."

[0103] It should be understood that the intermediate data B may also be an intermediate result that is accessed less frequently or data that only needs to be accessed once.

[0104] It should be noted that the above implementation needs to consider the bandwidth between the system resources and the physical accelerator. Generally speaking, the above implementation can be used when the bandwidth between the system resources and the physical accelerator is sufficient.

[0105] In this embodiment of the present application, to implement the above approach and obtain the optimal solution for resources on host A, an excess resource controller is required, as shown in Figure 11. The excess resource controller's input data includes, but is not limited to, system resource usage and bandwidth; the accelerator's computing power, video memory, and intra-card bandwidth; and the inter-card bandwidth and access load between accelerators. The excess resource controller manages excess resources based on this input data, including, but not limited to, managing the creation, location, size, and lifecycle of excess video memory for the virtual accelerator.

[0106] It should be understood that the above-mentioned excess resource controller can be deployed on a cloud management platform, or on a cloud server in a cloud data center, and the embodiments of the present application do not make specific limitations on this.

[0107] As an example, taking the above-mentioned host A including physical accelerator 1 and physical accelerator 2 as an example, the information input by host A to the excess resource controller may include but is not limited to: system resource usage and bandwidth; computing power, video memory, and on-card bandwidth of physical accelerator 1; computing power, video memory, and on-card bandwidth of physical accelerator 2; inter-card bandwidth and access load between physical accelerator 1 and physical accelerator 2, etc.

[0108] In the embodiment of the present application, after obtaining the input information, the excess resource controller manages the excess resources based on the balance strategy and the excess strategy. The balance strategy and the excess strategy are explained below.

[0109] The aforementioned balancing strategy balances the different physical accelerators, system resources, and communication links on a host (for example, host A) to optimize all resources on host A. Specifically, the balancing strategy considers the resource usage of the different physical accelerators on host A (including computing power, video memory, intra-card bandwidth, and inter-card bandwidth) and the system resource usage (including system resource usage and bandwidth) to determine which of the above Figures 8-10 represents the optimal deployment location for excess video memory. There are multiple specific implementation methods, one based on manual tuning and another based on machine learning.

[0110] For manual tuning, since the current operating status of the virtual accelerator is known, the consumption generated by different over-commitment methods can be obtained through testing (historical experience). Then, based on the input parameter configuration, weighted addition and input logic classification judgment can be performed to obtain the optimal location for the next excess video memory to be allocated.

[0111] For machine learning-based methods, this can be achieved through the reinforcement learning method in machine learning. The longer the system runs and the more data is collected, the better the control effect will be.

[0112] The above-mentioned oversubscription strategy implements an oversubscription method on a physical accelerator. Oversubscription strategies can be categorized into two types based on the user: inter-user oversubscription and intra-user oversubscription.

[0113] For example, Figure 12 illustrates an inter-user oversubscription strategy. This strategy involves obtaining excess resources from other users' idle resources, which can include computing power, bandwidth, system resources, and so on. Specifically, assume that user A and user B each create a virtual accelerator A and B. The resources created by virtual accelerator A are primarily implemented on physical accelerator 1, while the resources created by virtual accelerator B are primarily implemented on physical accelerator 2. User A uses a large amount of memory resources but has a large amount of idle computing power resources; user B uses a large amount of computing power resources but has a large amount of free memory resources. In this case, we create Server A on physical accelerator 1 to provide external computing power resources, and Server B on physical accelerator 2 to provide external memory resources. This allows user A to use the memory resources on physical accelerator 2 as excess video memory resources, while user B can use the computing power resources on physical accelerator 1 as excess computing power resources.

[0114] It should be noted that the method shown in FIG12 is similar to the method in FIG8 , and requires consumption of inter-card bandwidth, so it is considered a method of replacing computing power and memory with bandwidth.

[0115] The above-mentioned intra-user excess resource strategy means that excess resources are obtained through excess quota on the same user's virtual accelerator, not by using the physical resources occupied by other virtual accelerators. The method in Figure 9 can correspond to the intra-user excess resource strategy. For details, please refer to the description in Figure 9 and will not be repeated here.

[0116] Optionally, the above method for obtaining excess resources can also be applied to the debugging mode of artificial intelligence training / inference in the embodiments of the present application. Because resource usage varies among users in debugging mode, the original method of using virtual accelerators can result in significant resource waste. The method provided in the embodiments of the present application can effectively improve resource utilization.

[0117] The above describes in detail the method provided by the embodiment of the present application in conjunction with Figures 1 to 12. The following describes in detail the embodiment of the device of the present application in conjunction with Figures 13 to 16. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment. Therefore, for parts not described in detail, reference can be made to the above method embodiment.

[0118] Figure 13 is a schematic block diagram of an apparatus 1300 provided in an embodiment of the present application. Apparatus 1300 can be implemented using software, hardware, or a combination of both. Apparatus 1300 provided in an embodiment of the present application can implement the method flow shown in Figure 6 of the embodiment of the present application. Apparatus 1300 includes a receiving module 1310, a processing module 1320, and a sending module 1330. Among them, the receiving module 1310 is used to receive the configuration information of the first virtual accelerator and the configuration information of the excess resources of the first virtual accelerator sent by the tenant, the configuration information of the first virtual accelerator includes the specifications and type of the first virtual accelerator selected by the tenant, and the excess resources include excess storage resources; the processing module 1320 is used to allocate the resources of the first physical accelerator in the first server to the first virtual accelerator according to the specifications and type of the first virtual accelerator selected by the tenant, the resources of the first physical accelerator include first storage resources, first computing power resources, and first bandwidth resources; the processing module 1320 is also used to replace the excess storage resources by consuming the first computing power resources based on the configuration information of the excess resources; the sending module 1330 is used to send the login information of the first virtual accelerator to the tenant's client, and the login information is used by the tenant to log in and use the first virtual accelerator and the excess storage resources.

[0119] Optionally, the configuration information of the excess resources includes the mode, size, type and speed requirement of the excess resources.

[0120] Optionally, the excess storage resource is located on a second storage resource, and the second storage resource includes at least one of the following: the first storage resource, a storage resource on a second physical accelerator in the first server, and a system storage resource of the first server.

[0121] Optionally, the processing module 1320 is further configured to determine the location of the second storage resource based on at least one of the following information: a resource size of the first physical accelerator, a resource size of the second physical accelerator, an inter-card bandwidth between the first physical accelerator and the second physical accelerator, and system resource usage of the first server.

[0122] Optionally, the receiving module 1310 is also used to receive data sent by the tenant; the processing module 1320 is also used to compress the data using the first computing power resource to obtain compressed data; the processing module 1320 is also used to store the compressed data in the second storage resource.

[0123] Optionally, the second storage resource is the second storage resource, and the processing module 1320 stores the compressed data in the second storage resource through the bandwidth between the first physical accelerator and the second physical accelerator.

[0124] Optionally, the second storage resource is a system storage resource of the first server, and the processing module 1320 stores the compressed data in the second storage resource through the bandwidth between the first physical accelerator and the first server.

[0125] Optionally, the processing module 1320 is also used to delete the first intermediate data stored in the second storage resource through a first instruction, so that the tenant can reuse the storage resources occupied by the first intermediate data in the second storage resource; the processing module 1320 is also used to regain the first intermediate data through calculation using the first computing power resource according to a second instruction sent by the tenant to obtain the first intermediate data.

[0126] Optionally, the second storage resource is the second storage resource, and the processing module 1320 deletes the first intermediate data stored in the second storage resource through the bandwidth between the first physical accelerator and the second physical accelerator.

[0127] Optionally, the second storage resource is a system storage resource of the first server, and the processing module 1320 deletes the first intermediate data stored in the second storage resource through the bandwidth between the first physical accelerator and the first server.

[0128] The apparatus 1300 herein may be embodied in the form of a functional module. The term "module" herein may be implemented in the form of software and / or hardware, and is not specifically limited thereto.

[0129] For example, a "module" can be a software program, hardware circuit, or a combination of the two that implements the aforementioned functions. For example, the implementation of receiving module 1310 will be described below using receiving module 1310 as an example. Similarly, the implementation of other modules, such as processing module 1320 and sending module 1330, can refer to the implementation of receiving module 1310.

[0130] The receiving module 1310 is taken as an example of a software functional unit, and the receiving module 1310 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the receiving module 1310 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0131] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0132] As an example of a hardware functional unit, receiving module 1310 may include at least one computing device, such as a server. Alternatively, receiving module 1310 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0133] The multiple computing devices included in the receiving module 1310 can be distributed in the same region or in different regions. The multiple computing devices included in the receiving module 1310 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the receiving module 1310 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0134] Therefore, the modules of each example described in the embodiments of this application can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0135] It should be noted that: when the device provided in the above embodiment executes the above method, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. For example, the receiving module 1310 can be used to execute any step in the above method, the processing module 1320 can be used to execute any step in the above method, and the sending module 1330 can be used to execute any step in the above method. The steps that the receiving module 1310, the processing module 1320, and the sending module 1330 are responsible for implementing can be specified as needed, and all the functions of the above device can be realized by respectively implementing different steps in the above method through the receiving module 1310, the processing module 1320, and the sending module 1330.

[0136] In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation processes are detailed in the method embodiments above and will not be repeated here.

[0137] The method provided in the embodiment of the present application can be performed by a computing device, which can also be referred to as a computer system. It includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a processing unit, a memory and a memory control unit, and the function and structure of the hardware are subsequently described in detail. The operating system is any one or more computer operating systems that implement business processing through a process, for example, a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system or a Windows operating system. The application layer includes applications such as a browser, an address book, a word processing software, and an instant messaging software. Furthermore, optionally, the computer system is a handheld device such as a smart phone, or a terminal device such as a personal computer, and this application is not particularly limited, as long as the method provided in the embodiment of the present application can be used. The execution subject of the method provided in the embodiment of the present application can be a computing device, or a functional module in a computing device that can call a program and execute a program.

[0138] 14 , a computing device provided in an embodiment of the present application is described in detail below.

[0139] FIG14 is a schematic diagram of the architecture of a computing device 1500 provided in an embodiment of the present application. The computing device 1500 may be a server, a computer, or other device with computing capabilities. The computing device 1500 shown in FIG14 includes: at least one processor 1510 and a memory 1520.

[0140] It should be understood that this application does not limit the number of processors and memories in the computing device 1500.

[0141] The processor 1510 executes the instructions in the memory 1520 so that the computing device 1500 implements the method provided in this application. Alternatively, the processor 1510 executes the instructions in the memory 1520 so that the computing device 1500 implements the functional modules provided in this application, thereby implementing the method provided in this application.

[0142] Optionally, the computing device 1500 further includes a communication interface 1530. The communication interface 1530 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1500 and other devices or a communication network.

[0143] Optionally, the computing device 1500 further includes a system bus 1540, wherein the processor 1510, the memory 1520, and the communication interface 1530 are respectively connected to the system bus 1540. The processor 1510 can access the memory 1520 through the system bus 1540. For example, the processor 1510 can read and write data or execute code in the memory 1520 through the system bus 1540. The system bus 1540 is a peripheral component interconnect express (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus 1540 is divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in Figure 14, but this does not mean that there is only one bus or one type of bus.

[0144] In one possible implementation, the processor 1510 is primarily responsible for interpreting computer program instructions (or codes) and processing data in the computer software. The computer program instructions and the data in the computer software may be stored in the memory 1520 or the cache 1516.

[0145] Optionally, the processor 1510 may be an integrated circuit chip having signal processing capabilities. By way of example and not limitation, the processor 1510 is a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor is a microprocessor, etc. For example, the processor 1510 is a central processing unit (CPU).

[0146] Optionally, each processor 1510 includes at least one processing unit 1512 and a memory control unit 1514 .

[0147] Processing unit 1512, also known as a core, is the most important component of a processor. Processing unit 1512 is manufactured from single-crystal silicon using a specific production process. All processor calculations, command reception, command storage, and data processing are performed by the core. Each processing unit independently executes program instructions, leveraging parallel computing capabilities to accelerate program execution. Each processing unit has a fixed logical structure. For example, a processing unit includes logical units such as a level 1 cache, a level 2 cache, an execution unit, an instruction-level unit, and a bus interface.

[0148] In one implementation example, the memory control unit 1514 is configured to control data exchange between the memory 1520 and the processing unit 1512. Specifically, the memory control unit 1514 receives memory access requests from the processing unit 1512 and controls access to the memory based on the memory access requests. By way of example and not limitation, the memory control unit is a device such as a memory management unit (MMU).

[0149] In one implementation example, each memory control unit 1514 addresses the memory 1520 via a system bus. An arbiter (not shown in FIG. 14 ) is configured in the system bus to handle and coordinate competing accesses by multiple processing units 1512 .

[0150] In an implementation example, the processing unit 1512 and the memory control unit 1514 are connected to each other via connection lines inside the chip, such as address lines, so as to achieve communication between the processing unit 1512 and the memory control unit 1514.

[0151] Optionally, each processor 1510 also includes a cache 1516, which is a buffer for data exchange (called a cache). When a processing unit 1512 needs to read data, it first searches the cache for the required data. If the data is found, it executes the request directly; if not, it searches the memory. Because the cache runs much faster than the memory, the cache helps the processing unit 1512 run faster.

[0152] Memory 1520 can provide runtime space for processes in computing device 1500. For example, memory 1520 stores computer programs (specifically, program code) used to generate processes. After the computer program is executed by the processor to generate a process, the processor allocates corresponding storage space for the process in memory 1520. Furthermore, the aforementioned storage space further includes a text segment, an initialized data segment, a bit-initialized data segment, a stack segment, a heap segment, and the like. Memory 1520 stores data generated during the execution of the process, such as intermediate data or process data, in the storage space corresponding to the aforementioned process.

[0153] Optionally, the memory is also called the internal memory. Its function is to temporarily store the data processed by the processor 1510 and the data exchanged with external storage such as a hard disk. As long as the computer is running, the processor 1510 will load the data to be calculated into the internal memory for calculation. When the calculation is completed, the processing unit 1512 will transmit the result.

[0154] By way of example and not limitation, memory 1520 is a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Non-volatile memory is read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory is random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM). It should be noted that the memory 1520 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0155] The structure of the computing device 1500 listed above is only an example, and the present application is not limited thereto. The computing device 1500 of the embodiment of the present application includes various hardware in the computer system in the prior art. For example, the computing device 1500 also includes other memories in addition to the memory 1520, such as disk storage, etc. Those skilled in the art should understand that the computing device 1500 may also include other devices necessary to achieve normal operation. At the same time, according to specific needs, those skilled in the art should understand that the above-mentioned computing device 1500 may also include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the above-mentioned computing device 1500 may also include only the devices necessary to implement the embodiment of the present application, and does not necessarily include all the devices shown in Figure 14.

[0156] The present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device may be a server. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0157] As shown in Figure 15, the computing device cluster includes at least one computing device 1500. The memory 1520 in one or more computing devices 1500 in the computing device cluster may store the same instructions for executing the above method.

[0158] In some possible implementations, the memory 1520 of one or more computing devices 1500 in the computing device cluster may also store some instructions for executing the above method. In other words, the combination of one or more computing devices 1500 can jointly execute the instructions of the above method.

[0159] It should be noted that the memory 1520 in different computing devices 1500 in the computing device cluster can store different instructions, each for performing part of the functions of the above-mentioned apparatus. In other words, the instructions stored in the memory 1520 in different computing devices 1500 can implement the functions of one or more modules in the above-mentioned apparatus.

[0160] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. The network can be a wide area network (WAN) or a local area network (LAN), among others. FIG. 16 illustrates one possible implementation. As shown in FIG. 16 , two computing devices 1500A and 1500B are connected via a network. Specifically, each computing device is connected to the network via a communication interface within the computing device.

[0161] It should be understood that the functionality of the computing device 1500A shown in FIG16 may also be implemented by multiple computing devices 1500. Similarly, the functionality of the computing device 1500B may also be implemented by multiple computing devices 1500.

[0162] This embodiment also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on a computing device, it causes the computing device to perform the method provided above, or causes the computing device to implement the functions of the apparatus provided above.

[0163] This embodiment also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of storing data on a computing device, or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, or a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that, when executed on a computing device, cause the computing device to perform the method provided above.

[0164] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0165] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0166] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0167] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0168] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0169] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0170] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0171] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for obtaining excess resources, characterized in that: The method is applied to a cloud management platform, the cloud management platform is used to manage an infrastructure for providing cloud services, the infrastructure includes at least one server, each of the servers runs at least one virtual accelerator, and the method includes: The cloud management platform receives configuration information of the first virtual accelerator and configuration information of excess resources of the first virtual accelerator sent by the tenant, wherein the configuration information of the first virtual accelerator includes the size and type of the first virtual accelerator selected by the tenant, and the excess resources include excess storage resources; The cloud management platform allocates resources of a first physical accelerator in a first server to the first virtual accelerator according to the size and type of the first virtual accelerator selected by the tenant, where the resources of the first physical accelerator include first storage resources, first computing power resources, and first bandwidth resources; The cloud management platform obtains the excess storage resources by replacing them by consuming the first computing resources based on the configuration information of the excess resources; The cloud management platform sends login information of the first virtual accelerator to the client of the tenant, where the login information is used by the tenant to log in and use the first virtual accelerator and the excess storage resources.

2. The method according to claim 1, characterized in that The configuration information of the excess resources includes the mode, size, type and speed requirement of the excess resources.

3. The method according to claim 1 or 2, characterized in that: The excess storage resource is located on a second storage resource, and the second storage resource includes at least one of the following: the first storage resource, a storage resource on a second physical accelerator in the first server, and a system storage resource of the first server.

4. The method according to claim 3, characterized in that The method further comprises: The cloud management platform determines the location of the second storage resource according to at least one of the following information: resource size of the first physical accelerator, resource size of the second physical accelerator, inter-card bandwidth of the first physical accelerator and the second physical accelerator, and system resource usage of the first server.

5. The method according to claim 3 or 4, characterized in that: The step of replacing and obtaining the excess storage resources by consuming the first computing power resources includes: The cloud management platform receives data sent by the tenant; The cloud management platform compresses the data by using the first computing resource to obtain compressed data; The cloud management platform stores the compressed data in the second storage resource.

6. The method according to claim 3 or 4, characterized in that: The second storage resource stores the first intermediate data, and the step of replacing and obtaining the excess storage resource by consuming the first computing power resource includes: The cloud management platform deletes the first intermediate data stored in the second storage resource through a first instruction, so that the tenant can reuse the storage resources occupied by the first intermediate data in the second storage resource; The cloud management platform regains the first intermediate data by calculating with the first computing resource according to the second instruction sent by the tenant to obtain the first intermediate data.

7. A device for obtaining excess resources, characterized in that: The device is applied to a cloud management platform, and the cloud management platform is used to manage an infrastructure for providing cloud services, wherein the infrastructure includes at least one server, and each of the servers runs at least one virtual accelerator. The device includes: A receiving module, configured to receive configuration information of a first virtual accelerator and configuration information of excess resources of the first virtual accelerator sent by a tenant, wherein the configuration information of the first virtual accelerator includes a specification size and type of the first virtual accelerator selected by the tenant, and the excess resources include excess storage resources; a processing module, configured to allocate resources of a first physical accelerator in a first server to the first virtual accelerator according to a size and type of the first virtual accelerator selected by the tenant, wherein the resources of the first physical accelerator include first storage resources, first computing power resources, and first bandwidth resources; The processing module is further configured to replace and obtain the excess storage resources by consuming the first computing power resources based on the configuration information of the excess resources; A sending module is used to send the login information of the first virtual accelerator to the client of the tenant, and the login information is used to The tenant logs in and uses the first virtual accelerator and the excess storage resources.

8. The device according to claim 7, characterized in that The configuration information of the excess resources includes the mode, size, type and speed requirement of the excess resources.

9. The device according to claim 7 or 8, characterized in that The excess storage resource is located on a second storage resource, and the second storage resource includes at least one of the following: the first storage resource, a storage resource on a second physical accelerator in the first server, and a system storage resource of the first server.

10. The device according to claim 9, characterized in that The processing module is further used to determine the location of the second storage resource according to at least one of the following information: resource size of the first physical accelerator, resource size of the second physical accelerator, inter-card bandwidth of the first physical accelerator and the second physical accelerator, and system resource usage of the first server.

11. The device according to claim 9 or 10, characterized in that The receiving module is further used to receive data sent by the tenant; The processing module is further used to compress the data using the first computing power resource to obtain compressed data; The processing module is further configured to store the compressed data in the second storage resource.

12. The device according to claim 9 or 10, characterized in that The processing module is further configured to delete the first intermediate data stored in the second storage resource through a first instruction, so that the tenant can reuse the storage resource occupied by the first intermediate data in the second storage resource; The processing module is further configured to retrieve the first intermediate data by calculating with the first computing power resources according to a second instruction sent by the tenant for obtaining the first intermediate data.

13. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 6.

14. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 6.

15. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 6.

Images