Resource scheduling method, computing device and storage medium

Abstract

The present invention provides a resource scheduling method, computing device, and storage medium. For a created target virtual device, the method determines the resource usage of the target virtual device. If the resource usage does not meet the resource application of the target virtual device, the method directly recreates the target virtual device based on the resource usage. This allows the recreated target virtual device to replace the existing target virtual device, and the available resources of the recreated target virtual device can meet its resource usage. If the current resources cannot meet the resource application, the method can quickly and automatically dynamically adjust the available resources of the target virtual device.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a resource scheduling method, computing device, and storage medium. Background Art

[0002] With the rapid development of the Internet, cloud services have many advantages over traditional computing services, such as low cost, easy expansion, scalability, easy management, and high availability. Therefore, they occupy an increasingly critical position in the information Internet era.

[0003] Cloud servers can be deployed with multiple services, providing users with a variety of cloud services, such as model creation, model application, data storage, image analysis, and more. However, many of these services are online and will occupy cloud server resources for a long time as long as the services are online. Therefore, how to effectively improve the resource utilization of online services has become a crucial issue. Summary of the Invention

[0004] Various aspects of the present application provide a resource scheduling method, a computing device, and a storage medium, which are used to automatically and relatively quickly adjust appropriate resources for utilization.

[0005] An embodiment of the present application provides a resource scheduling method, comprising: determining the resource usage of a created target virtual device; and recreating the target virtual device directly based on the resource usage if the resource usage does not meet the resource application amount of the target virtual device.

[0006] An embodiment of the present application also provides a resource scheduling method, including: determining the resource usage of a created target virtual device; when the target virtual device determines that the resource usage is greater than the resource application amount within a first preset time, receiving a reallocation request sent by the target virtual device; determining whether the number of CPUs in the resource usage carried by the reallocation request meets the preset number, and whether the number of memories in the resource usage meets the number of memories in the resource application amount; if so, recreating the target virtual device based on the number of CPUs in the resource usage and the number of memories in the resource application amount.

[0007] An embodiment of the present application also provides a resource scheduling method, including: receiving a reallocation request, and determining a new resource application amount directly based on the resource usage carried in the reallocation request; and recreating a target virtual device based on the new resource application amount carried in the reallocation request to replace the created target virtual device, so that the available resource amount of the recreated target virtual device is the resource usage.

[0008] An embodiment of the present application also provides a resource scheduling method, including: monitoring the resource usage of the target virtual device; when it is determined that the resource usage does not meet the resource application amount of the target virtual device, sending a reallocation request to the scheduling center, so that the scheduling center determines a new resource application amount based on the resource usage carried in the reallocation request, and sends the new resource usage to the management device through the reallocation request, so that the management device recreates the target virtual device based on the new resource application amount carried in the reallocation request.

[0009] An embodiment of the present application also provides a model deployment method, including: determining the resource usage of a created target virtual device; if the resource usage does not meet the resource application amount of the target virtual device, directly recreating the target virtual device based on the resource usage; and deploying a service model based on the recreated target virtual device to provide services to users according to the service model.

[0010] An embodiment of the present application also provides a computing device, comprising: a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program, so as to: determine the resource usage of the target virtual device that has been created; and if the resource usage does not meet the resource application amount of the target virtual device, directly recreate the target virtual device based on the resource usage.

[0011] An embodiment of the present application also provides a computing device, comprising: a memory, a processor, and the communication component; the memory is used to store a computer program; the processor is used to execute the computer program to: determine the resource usage of the target virtual device that has been created; the communication component is used to receive a reallocation request sent by the target virtual device when it is determined through the target virtual device that the resource usage is greater than the resource application amount within a first preset time; the processor is used to determine whether the number of CPUs in the resource usage carried by the reallocation request meets the preset number, and whether the number of memory in the resource usage meets the number of memory in the resource application amount, and if so, recreate the target virtual device based on the number of CPUs in the resource usage and the number of memory in the resource application amount.

[0012] An embodiment of the present application also provides a computing device, comprising: a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program, so as to: receive a reallocation request, and directly determine a new resource application amount based on the resource usage carried by the reallocation request; and recreate a target virtual device based on the new resource application amount carried by the reallocation request, so as to replace the created target virtual device, so that the available resource amount of the recreated target virtual device is the resource usage.

[0013] An embodiment of the present application also provides a computing device, comprising: a memory, a processor, and a communication component; the memory is used to store a computer program; the processor is used to execute the computer program to: monitor the resource usage of the target virtual device; the communication component is used to send a reallocation request to a scheduling center when it is determined that the resource usage does not meet the resource application amount of the target virtual device, so that the scheduling center determines a new resource application amount based on the resource usage carried in the reallocation request, and sends the new resource usage to the management device through the reallocation request, so that the management device recreates the target virtual device based on the new resource application amount carried in the reallocation request.

[0014] An embodiment of the present application also provides a computing device, comprising: a memory, a processor, and a communication component; the memory is used to store a computer program; the processor is used to execute the computer program, so as to: determine the resource usage of the target virtual device that has been created; if the resource usage does not meet the resource application amount of the target virtual device, directly recreate the target virtual device based on the resource usage; based on the recreated target virtual device, deploy a service model to provide services to users according to the service model.

[0015] An embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer program is executed by one or more processors, the one or more processors implement the steps in the above method.

[0016] In an embodiment of the present application, the resource usage of the target virtual device that has been created is determined; if the resource usage does not meet the resource application amount of the target virtual device, the target virtual device is recreated directly based on the resource usage. This allows the recreated target virtual device to replace the created target virtual device, and the available resources of the recreated target virtual device can meet its resource usage. If the current resources cannot meet the requirement, the available resources of the target virtual device can be quickly and automatically adjusted dynamically. At the same time, since the resources are adjusted directly based on the resource usage, the available resources of the recreated target virtual device will not be excessively wasted, and a certain amount of resources can be more appropriately utilized, thereby improving resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 This is a schematic diagram of the structure of a resource scheduling system according to an exemplary embodiment of the present application;

[0019] Figure 2 A schematic diagram of a flow chart of a resource scheduling method according to an exemplary embodiment of the present application;

[0020] Figure 3A A schematic diagram of a process for recreating a target virtual device according to an exemplary embodiment of the present application;

[0021] Figure 3B A schematic diagram of a process for recreating a target virtual device according to an exemplary embodiment of the present application;

[0022] Figure 3C A schematic diagram of a process for recreating a target virtual device according to an exemplary embodiment of the present application;

[0023] Figure 4 A schematic diagram of a flow chart of a resource scheduling method according to an exemplary embodiment of the present application;

[0024] Figure 5 A schematic diagram of a flow chart of a resource scheduling method according to an exemplary embodiment of the present application;

[0025] Figure 6 A schematic diagram of a flow chart of a resource scheduling method according to an exemplary embodiment of the present application;

[0026] Figure 7 A schematic structural diagram of a resource scheduling device provided in yet another exemplary embodiment of the present application;

[0027] Figure 8 A schematic structural diagram of a resource scheduling device provided in yet another exemplary embodiment of the present application;

[0028] Figure 9 A schematic structural diagram of a resource scheduling device provided in yet another exemplary embodiment of the present application;

[0029] Figure 10 A schematic structural diagram of a resource scheduling device provided in yet another exemplary embodiment of the present application;

[0030] Figure 11 A schematic diagram of the structure of a computing device provided for an exemplary embodiment of the present application;

[0031] Figure 12 A schematic diagram of the structure of a computing device provided for an exemplary embodiment of the present application;

[0032] Figure 13 A schematic diagram of the structure of a computing device provided for an exemplary embodiment of the present application;

[0033] Figure 14 A schematic diagram of the structure of a computing device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0034] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0035] As AI (Artificial Intelligence) becomes increasingly prevalent, more and more users are looking to leverage the power of algorithmic models to enhance the value of their services. NLP (Natural Language Processing) and MT (Machine Translation) are two typical examples. Their capabilities, such as text distribution, splicing and error correction, keyword extraction, and translation, can effectively reduce user labor costs, thereby increasing productivity and empowering their services. Unlike training tasks, online services are resident processes that occupy resources for a long time as long as the service is online. Therefore, effectively improving resource utilization for online services is a crucial issue.

[0036] A typical solution for allocating resources for online model services is to estimate the resources the model might use before serving it. For example, some platforms require users to fill in the estimated resource requirements for the model service, while others rely on empirical estimates. However, this approach fails to dynamically adjust resource usage based on the actual load of the model service. If the initial resource estimate is too low, the model service capacity will be insufficient, while if it is too high, resources will be wasted.

[0037] Therefore, the method provided in the embodiment of the present application can automatically and relatively quickly adjust appropriate resources for utilization.

[0038] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0039] Figure 1 This is a schematic diagram of a resource scheduling system provided by an exemplary embodiment of the present application. Figure 1 As shown, the system 100 may include: a first device 101 and a second device 102 .

[0040] The first device 101 may be a device with certain computing capabilities, capable of sending a request to the second device 102 and receiving a successful request message from the second device 102. The basic structure of the first device 101 may include at least one processor. The number of processors may depend on the configuration and type of the device with certain computing capabilities. The device with certain computing capabilities may also include memory, which may be volatile, such as RAM, or non-volatile, such as read-only memory (ROM), flash memory, or both. The memory typically stores an operating system (OS), one or more application programs, and may also store program data. In addition to the processing unit and memory, the device with certain computing capabilities also includes some basic configurations, such as a network card chip, an I / O bus, a display component, and some peripheral devices. Optionally, some peripheral devices may include, for example, a keyboard, a stylus, etc. Other peripheral devices are well known in the art and are not described in detail here. Optionally, the first device 101 may be a smart terminal, such as a mobile phone, a desktop computer, a laptop, a tablet computer, etc.

[0041] The second device 102 is a device that can provide computing and processing services in a network virtual environment and can use the network to process data, such as creating a virtual device. It can receive requests sent by the first device 101 and return a message indicating that the first device 101's request was successful. In physical implementation, the second device 102 can be any device that can provide computing services, respond to service requests, and return data processing results. For example, it can be a cloud server, cloud host, virtual center, conventional server, etc. The second device 102 mainly includes a processor, hard disk, memory, system bus, etc., which is similar to the general computer architecture.

[0042] It should be noted that multiple functional modules and virtual devices, such as a scheduling center 104, a management device 105, and a virtual device 106, can be deployed in the second device 102. In an embodiment of the present application, the first device 101 sends a creation request to the scheduling center 104 in the second device 102. In response to the creation request, the scheduling center 104 sends a creation request to the management device 105, such as kubernetes (used to manage containerized applications in cloud servers, making the deployment of containerized applications simple and efficient), and the management device 105 creates the virtual device 106 based on the request. In addition, after the virtual device is created, the management device 105 can send a creation success message to the scheduling center 104, and the scheduling center sends the message to the first device 101 through the second device 102.

[0043] Among them, the management device 105 determines the resource usage of the target virtual device for the created target virtual device; when the resource usage does not meet the resource application amount of the target virtual device, the target virtual device is recreated directly based on the resource usage to replace the created target virtual device, so that the available resources of the recreated target virtual device are the resource usage.

[0044] In the application scenario of the embodiment of the present application, user 103 can use a general computing model service on the cloud server, such as a pre-created MT model, to perform online translation. User 103 can also customize a specific computing model on the cloud server, providing data to train the model, and then servicing the model to meet their needs. Based on this, user 103 can perform the following operations:

[0045] User 103 logs in to a browser through first device 101, such as a computer, and opens the browser's web interface. The user can create a request through this web interface. In response to the user's operation, the computer sends a creation request to the scheduling center in second device 102, such as scheduling center 104 in a cloud server. This creation request is for creating a container for creating and deploying a specific computing model, or for directly deploying a general model to provide model services. The request also carries the resource usage requested by user 103 for creating the container. Before receiving this request, scheduling center 104 may pass through the cloud server's load balancing and ingress controller before entering the corresponding scheduling center 104. After receiving this request, scheduling center 104 forwards it to management device 105, such as Kubernetes. Kubernetes creates a virtual device controller, such as a deployment, based on the resource usage contained in the creation request. This controller may contain at least two target virtual devices, such as two containers. Container A may be used for model creation and deployment, and container B may be used to monitor the resource usage of container A in the controller. After creating the virtual device controller, user 103 can create and deploy a model on the container in the created virtual device controller, or directly use the universal model already deployed in the container for machine translation, etc. The virtual device controller can also be called a model application service. The specific process is similar to the above. Requests, such as translation requests, can also be sent through the web interface. After passing through the cloud server's load balancing and the ingress controller, the request is sent to the container providing the service. The container then receives the request, performs the translation, and returns the translation result to user 103's web interface.

[0046] It should be noted that the B container can also implement other functions as needed, such as data management functions, user management functions, etc. It should be understood that according to the different needs of the user 103, such as model deployment, data storage, image analysis, etc., the number of target virtual devices can be different, that is, the target virtual devices in a virtual device controller are different.

[0047] When container B monitors that the resource usage of container A and itself (i.e., the virtual device controller to which it belongs) is greater than or less than the resource application amount of the container (i.e., the virtual device controller to which it belongs) within a certain period of time, it sends a scaling request or expansion request to the scheduling center 104, and the request carries the resource usage. After receiving the request, the scheduling center 104 forwards the request to the management device. The management device recreates the virtual device controller based on the resource usage in the request, such as a deployment, which can have two target containers. After the re-creation is completed, the virtual device controller replaces the original virtual device controller, performs the services provided by the original virtual device controller, and destroys the original virtual device controller.

[0048] In addition, in addition to directly creating based on resource usage, if the CPU in resource usage is higher than the resource application, and the memory amount in resource usage is lower than or equal to the resource application, then you can only modify the CPU amount, and keep the memory amount unchanged to recreate the target virtual device. The creation process is similar to the previous one and will not be repeated here.

[0049] Furthermore, the scheduling center 104 can also, upon receiving two scaling-down requests, merge the scaling-down requests to recreate the target virtual device. Alternatively, the scheduling center 104 can also receive an expansion request and split it, requesting the management device 105 to create at least two virtual device controllers to provide the service. The specific implementation details will be elaborated upon later and will not be further elaborated here.

[0050] In addition, as can be seen from the foregoing, during the process of scaling down or expanding, the translation service can be continuously provided to user 103. User 103 can be a translator in a translation company or a patent agent in a patent agency. It can also be an enterprise, such as a patent applicant and a database provider, or it can be an examiner in a patent office. All of them can perform translation through the translation service provided by the cloud server. As mentioned above, the above-mentioned user 103 can provide a translation service interface through the web interface of the computer, and input the text to be translated on the interface, and trigger the online translation instruction. The web interface responds to the translation instruction through the computer and sends a translation request to the cloud server. The translation request carries the text to be translated. The translation request passes through the load balancing of the cloud server and the ingress controller to enter the container corresponding to the service provider, so that the container receives the request and translates the text to be translated, and returns the translation result to the web interface of the computer of user 103, and the computer displays the translation result so that user 103 obtains the translation result.

[0051] In the above embodiment, the above devices can all be connected to the network, and the network connection can be a wireless connection. If the above devices are connected for communication, the network standard of the mobile network can be any one of 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4G+ (LTE+), WiMax, 5G, etc.

[0052] The resource scheduling process is described in detail below in conjunction with the method embodiment.

[0053] Figure 2 This is a flow chart of a resource scheduling method according to an exemplary embodiment of the present application. The method 200 provided in the embodiment of the present application is executed by a server, such as a cloud server. More specifically, it can be a scheduling center (also referred to as a scheduler) in the server. The method 200 includes the following steps:

[0054] 201: For the created target virtual device, determine the resource usage of the target virtual device.

[0055] 202: When the resource usage does not satisfy the resource application amount of the target virtual device, directly recreate the target virtual device based on the resource usage.

[0056] The following is a detailed explanation of the above steps:

[0057] 201: For the created target virtual device, determine the resource usage of the target virtual device.

[0058] A virtual device is an application created using physical host resources to provide an isolated runtime environment, such as a container. Virtual machines and sandboxes can also be used in different scenarios.

[0059] Resource usage refers to the physical host resources used by the target virtual device during its operation. Conversely, for a virtual machine, it can also refer to the virtual resources. Resources can include the CPU (central processing unit) and memory.

[0060] The creation process of the created target virtual device can be:

[0061] Specifically, a creation request is received, and according to the resource application amount carried in the creation request, at least one corresponding target virtual device is created through the management device.

[0062] Among them, the management device is used to manage containerized applications in the cloud server, making the deployment of containerized applications simple and efficient, such as Kubernetes.

[0063] Figure 3A 300A is a flow chart showing a process of recreating a target virtual device. Figure 3A As shown, a user logs into the cloud server's model creation web interface via computer 101. In response to the user's model service creation operation, computer 101 sends a creation request to the scheduling center 104 on the cloud server to create the model service. This involves executing step 301: sending a creation request to scheduling center 104. This request also includes the resource requirements for the service, such as 1 CPU (also known as 1 core CPU) and 2GB of memory. After receiving this request, scheduling center 104 forwards it to management device 105, such as Kubernetes, or more specifically, the Kubernetes API (Application Programming Interface) server. This involves executing step 302: sending a creation request to management device 105. The API server then creates at least one container based on the resource requirements. This involves executing step 303: creating the target virtual device.

[0064] It should be noted that the number of target virtual devices can vary depending on the service to be created, such as one container, two containers, three containers, etc. This can be determined by the API server based on the service being created.

[0065] To facilitate management and control of the target virtual device, the target virtual device may usually be created through a virtual device controller.

[0066] Specifically, according to the resource application amount carried in the creation request, at least one corresponding target virtual device is created through the management device, including: sending the creation request to the management device so that the management device creates a corresponding virtual device controller according to the resource application amount, and the virtual device controller has at least one corresponding target virtual device.

[0067] The virtual device controller refers to a controller used to control a virtual device, such as a deployment controller used to control a container, which can also be called a deployment.

[0068] For example, Figure 3AAs shown, according to the above description, the scheduling center sends a creation request to a management device, such as the Kubernetes API server. The Kubernetes API server creates a deployment controller 3011 based on the resource application amount. The deployment controller 3011 has two containers: the platform service 3012 (which can be called the Platform Service) and the model service 3013 (which can be called the Model Service). The Model Service is responsible for processing model requests, specifically model creation and model deployment, or model application. The Platform Service is responsible for processing various service logics, specifically monitoring the resource usage of the deployment controller 3011, which can also be called the resource usage of the target container.

[0069] It should be noted that, for virtual devices belonging to the same deployment controller 3011, these virtual devices may belong to the same pod (deployment unit), and they share the same resources, such as 1 CPU and 2G memory.

[0070] The method of determining the resource usage of the target virtual device may include: monitoring the resource usage of the target virtual device through the target virtual device, and determining the resource usage of the target virtual device.

[0071] For example, as described above, when a container is created for model serving, it can serve the model and also needs to determine its own resource usage. This can be queried using the container resource query command. After the resource usage is queried, it can be sent to the scheduling center in the form of a request.

[0072] It should be noted that, based on the foregoing, when creating a target virtual device by creating a virtual device controller, for example, if the deployment controller has only one container, the resource usage of the deployment is determined by the container.

[0073] When the number of target virtual devices is at least two, resource usage of the at least two target virtual devices may be monitored by one of the target virtual devices, and the resource usage of the at least two target virtual devices may be determined.

[0074] Wherein, at least two target virtual devices belong to the same virtual device controller.

[0075] For example, as described above, when there are at least two virtual devices, such as two containers, Platform Service and Model Service, resource usage can be monitored through the Platform Service. The Platform Service can determine its own resource usage and that of the Model Service by calling container resource query commands, thereby determining the resource usage of both, which also constitutes the resource usage of the deployment.

[0076] It should be noted that the virtual device controller can determine the resource usage through the virtual device, that is, the resource usage is determined by the virtual device.

[0077] In addition, each virtual device in the virtual device controller can determine its own resource usage respectively, and then any one of the virtual devices can determine the sum of the resource usage, that is, determine the final resource usage.

[0078] After creating the target virtual device, users can use it to provide services. However, during the service process, there may be a situation where resource usage exceeds the resource application amount. In order to ensure that the service can be carried out smoothly and without service crashes, the following methods can be used to improve it:

[0079] Specifically, the method 200 further includes: determining a limited resource amount that is greater than the resource usage amount according to the resource usage amount; and recreating the target virtual device according to the limited resource amount.

[0080] The resource limit can be determined by the management device or by the dispatch center itself. The following example uses the management device to determine the resource limit:

[0081] For example, as described above, after receiving a request, the Kubernetes API server obtains the resource request amount from the request, such as 1 CPU and 2GB of memory. To ensure smooth service operation, a resource limit (i.e., resource limit) greater than the resource request amount can be determined, such as 1.2 CPUs and 2.1GB of memory. The corresponding container or deployment is then created based on the resource limit.

[0082] It should be noted that when creating a virtual device or a virtual device controller, you can create it according to the corresponding preset creation code. Alternatively, you can directly set a fixed resource limit, but the resource limit must be large enough to satisfy any virtual device.

[0083] In addition, the target virtual device is recreated based on the limited resource amount determined by the scheduling center, which is similar to the previous description and will not be repeated here.

[0084] 202: When the resource usage does not satisfy the resource application amount of the target virtual device, directly recreate the target virtual device based on the resource usage.

[0085] The recreated target virtual device can be used to replace the created target virtual device, so that the available resource amount of the recreated target virtual device is the resource usage amount.

[0086] The resource usage not meeting the resource application amount of the target virtual device means that the resource usage exceeds the resource application amount or the resource usage is too small to meet the resource application amount. The resource usage being too small to meet the resource application amount means that the resource usage is smaller than a certain threshold value of the resource application amount.

[0087] For example, as mentioned above, the Platform Service will regularly monitor the resource usage of the current deployment controller. When the resource usage is monitored to be less than the resource application value (i.e., the resource application amount) for a period of time (such as a preset time, which can be the first preset time below) (or less than a certain threshold), a scaling-in request will be sent to the scheduling center to perform scaling-in. After the scheduling center receives the scaling-in request, it can send the scaling-in request to kubernetes. So that after the kubernetes API server receives the scaling-in request, it will rebuild the corresponding container according to the resource usage in the scaling-in request, and then destroy the original container, and continue to run the model service through the rebuilt container. In addition, it can also be a capacity expansion request, that is, when the resource usage is greater than the resource application value, the specific implementation process is similar to the above, and will not be repeated here.

[0088] The more specific implementation of step 202 is as follows:

[0089] Specifically, when the resource usage does not meet the resource application amount of the target virtual device, the target virtual device is recreated directly based on the resource usage, including: horizontal expansion or horizontal contraction based on the resource usage to recreate the target virtual device; or vertical expansion or vertical contraction based on the resource usage to recreate the target virtual device.

[0090] Among them, vertical expansion refers to expansion when the number of CPUs in resource usage is less than or equal to a certain threshold, such as less than or equal to 1. This expansion is still aimed at creating a virtual device controller.

[0091] Vertical scaling refers to scaling when the number of CPUs in resource usage is less than or equal to a certain threshold, such as less than or equal to 1. This scaling is still for creating a virtual device controller.

[0092] Horizontal expansion refers to expansion when the number of CPUs in resource usage exceeds a certain threshold, such as greater than 1. This expansion is performed to create at least two virtual device controllers.

[0093] Horizontal scaling refers to scaling down when the number of CPUs in the resource usage corresponding to multiple virtual device controllers is less than or equal to a certain threshold, such as less than or equal to 1. This scaling down is to merge at least two virtual device controllers into one virtual device controller.

[0094] Therefore, capacity can be expanded or reduced based on the number of CPUs. The following is a detailed explanation of the above expansion and reduction:

[0095] Specifically, when the resource usage does not meet the resource application amount of the target virtual device, the target virtual device is recreated directly based on the resource usage, including: comparing the resource usage with the resource application amount through the target virtual device, and based on the comparison result, receiving the capacity configuration request sent by the target virtual device, and sending the capacity configuration request to the management device, so that the management device recreates the target virtual device based on the resource usage carried in the capacity configuration request.

[0096] The comparison result indicates whether the resource usage is greater than the resource application amount or the resource usage is less than the resource application amount. Based on the different comparison results, the capacity reduction or expansion is determined so that the management device recreates the target virtual device according to the specific request.

[0097] The scaling request is as follows:

[0098] 1) Compare the resource usage with the resource application amount through the target virtual device, receive a capacity configuration request sent by the target virtual device based on the comparison result, and send the capacity configuration request to the management device, including: determining through the target virtual device that the resource usage is less than the resource application amount within a first preset time, receiving a capacity reduction request sent by the target virtual device, and sending the capacity reduction request to the management device, so that the management device recreates the target virtual device according to the resource usage carried in the capacity reduction request.

[0099] It should be noted that if resource usage is less than the resource request amount over a period of time, the maximum resource usage during that period can be selected as the final resource usage for that period. Alternatively, the minimum resource usage can be selected as the final resource usage for that period. Alternatively, the average resource usage during that period can be selected as the final resource usage for that period.

[0100] Since resources may include CPU and memory, when determining resource usage, the final resource usage, such as the average number, maximum number, and minimum number of CPUs, may be determined based on the number of CPUs and the amount of memory.

[0101] For example, according to the above, Figure 3A As shown, the Platform Service monitors that resource usage is consistently less than the resource application amount within a first preset time period, such as one hour. The maximum resource usage is then used as the final resource usage within the first preset time period. The Platform Service executes step 304: sending a scaling-down request to the scheduling center 104, which carries the final resource usage. After receiving the scaling-down request from the Platform Service, the scheduling center 104 sends the scaling-down request to the management device 105, such as the Kubernetes API server, executing step 305: sending the scaling-down request to the management device 105. When scheduling center 104 sends the scale-down request, it may be a rolling update request for deployment controller 3011, updating the resource usage of the already created deployment controller 3011. After receiving the request, the API server recreates deployment controller 3011 based on the final resource usage in the request, executing step 306: recreating the target virtual device. At this point, the recreated deployment controller 3014 is version 2 (i.e., Version 2). The original deployment controller 3011 is version 1 (i.e., Version 1).

[0102] It should be noted that, because this embodiment utilizes the rolling update request of deployment controller 3011 to recreate the deployment controller, version 1 deployment controller 3011 is destroyed only after version 2 deployment controller 3014 is started, thereby ensuring continuous service availability. In the case of scaling down, the rolling update request indicates a rolling update for scaling down.

[0103] A rolling update involves creating a Version 2 deployment controller 3014 while retaining Version 1's deployment controller 3011. Once Version 2's deployment controller 3014 is created, Version 1's deployment controller 3011 is destroyed, allowing Version 2's deployment controller 3014 to continue providing services, replacing Version 1's. At this point, any pending data or tasks sent by users will be sent to Version 2's deployment controller 3014 for service execution.

[0104] In addition, if capacity expansion is required, the target virtual device can be rebuilt as follows:

[0105] 2) Compare the resource usage with the resource application amount through the target virtual device, receive a capacity configuration request sent by the target virtual device based on the comparison result, and send the capacity configuration request to the management device, including: determining through the target virtual device that the resource usage is greater than the resource application amount within a second preset time, receiving a capacity expansion request sent by the target virtual device, and sending the capacity expansion request to the management device, so that the management device recreates the target virtual device according to the resource usage carried in the capacity expansion request.

[0106] If resource usage is greater than resource application amount over a period of time, the maximum resource usage during that period can be selected as the final resource usage for that period. Alternatively, the minimum resource usage can be selected as the final resource usage for that period. Alternatively, the average resource usage during that period can be selected as the final resource usage for that period.

[0107] For example, according to the above, Figure 3AAs shown, the newly created Platform Service, i.e., Platform Service 3012 (i.e., Platform Service 3012 in Deployment Controller 3014), monitors that resource usage is always greater than the resource application amount within a second preset time, such as 1 hour. Therefore, the maximum resource usage is used as the final resource usage within the second preset time. Platform Service 3012 executes step 307: sends a capacity expansion request to the scheduling center 104, and the capacity expansion request carries the final resource usage. After receiving the capacity expansion request from Platform Service 3012, Scheduling Center 104 sends the capacity expansion request to the management device 105, such as the Kubernetes API server, i.e., executes step 308: sends the capacity expansion request to the management device 105. This expansion request can be a rolling update request for deployment controller 3011, updating the resource usage of deployment controller 3011. After receiving this request, the API server recreates deployment controller 3015 based on the final resource usage in the request, executing step 309: recreating the target virtual device. In this case, the recreated deployment controller 3015 is version 3. Here, a rolling update request indicates a request for expansion.

[0108] It should be noted that the first preset time may be the same as or different from the second preset time.

[0109] In addition, embodiments of the present application can also implement horizontal pod scaling by using the HPA (horizontal pod autoscaler) control in the management device. Target virtual devices, such as containers, can be created by creating a pod. In other words, each pod can have at least one target virtual device, such as two containers.

[0110] When the user's service traffic increases, resource usage will increase. In this case, the HPA control can be used to increase the number of pods. When the user's service traffic decreases, resource usage will decrease. In this case, the HPA control can be used to reduce the number of pods.

[0111] However, increasing or decreasing the number of pods through HPA controls presents some issues: For example, increasing the number of pods through HPA controls allows the number of pods to be adjusted as traffic increases. For example, if a pod's resource request is 0.2 CPUs and 1GB of memory, and initially only one pod is providing services to users, as traffic increases, the number of pods may increase to 3 due to the increase in the number of pods. This means that the current service now occupies a total of 0.2*3=0.6 CPUs and 1*3=3GB of memory. However, under actual traffic load, the CPU may need to be increased to 0.6, but the memory does not need to be increased. Maintaining 1GB of memory is sufficient for service operation, thus wasting 2GB of memory.

[0112] Similarly, using HPA controls to reduce the number of pods can reduce it to at least one. However, when traffic is low, even a single pod's resource usage is far less than its requested amount. For example, when creating a model service, 2GB of memory is requested, but the actual traffic load only uses 0.5GB of memory. In this case, HPA controls cannot solve the problem of resource waste through resource scheduling.

[0113] Directly creating a virtual device controller based on resource usage can reduce resource waste, utilize resources more reasonably and appropriately, and provide better services to users.

[0114] In order to make rational use of resources and avoid wasting unnecessary resources, you can save memory resources in the following ways.

[0115] Specifically, the method 200 also includes: determining through the target virtual device that the resource usage is greater than the resource application amount within a third preset time, then receiving a reallocation request sent by the target virtual device; determining whether the number of CPUs in the resource usage carried in the reallocation request meets the preset number, and whether the number of memory in the resource usage meets the number of memory in the resource application amount; if so, recreating the target virtual device based on the number of CPUs in the resource usage and the number of memory in the resource application amount.

[0116] Among them, whether the number of CPUs in the resource usage meets the preset number, and whether the number of memory in the resource usage meets the number of memory in the resource application, can mean that the number of CPUs is less than or equal to 1. Then, the number of CPUs in the resource usage and the number of memory in the resource application can be used as the new resource application, and a reallocation request can be generated to recreate the target virtual device. The specific implementation method is similar to the previous one and will not be repeated here. Only the different situations of the above resource usage are explained:

[0117] Specifically, the method 200 also includes: determining through the target virtual device that the resource usage is greater than the resource application amount within a third preset time, then receiving a reallocation request sent by the target virtual device; determining that the number of CPUs in the resource usage carried by the reallocation request is less than or equal to 1, and the number of memory in the resource usage is less than or equal to the number of memory in the resource application amount, then using the number of CPUs in the resource usage and the number of memory in the resource application amount as new resource application amounts, and sending a reallocation request to the management device based on the new resource application amounts, so that the management device receives the reallocation request and recreates the target virtual device based on the new resource application amounts carried in the reallocation request.

[0118] Among them, in order to arrange resources reasonably, improve resource utilization, and avoid excessive resource division, the third preset time can be set to be greater than the second preset time, that is, based on longer-term monitoring, it is determined that the memory demand can be met by the memory quantity in the resource application amount.

[0119] Since the resource usage is greater than the resource application amount during the third preset time period, the maximum resource usage during that period can be selected as the final resource usage for that period. Alternatively, the minimum resource usage can be selected as the final resource usage for that period. Alternatively, the average resource usage during that period can be selected as the final resource usage for that period.

[0120] For example, as described above, if the resource usage of a created Platform Service is consistently greater than the resource application amount within a third preset time period, such as five hours, the maximum resource usage is used as the final resource usage within the third preset time period. The Platform Service sends a reallocation request, also known as an expansion request, to the scheduling center. This reallocation request carries the final resource usage amount. After receiving the Platform Service's reallocation request, the scheduling center determines whether the CPU number in the resource usage carried in the request is less than or equal to 1. If so, it then checks whether the memory amount is consistently less than or equal to the memory amount in the resource application amount. If so, the scheduling center uses the CPU number in the resource usage and the memory amount in the resource application amount as the new resource application amount. The scheduling center then sends a reallocation request to a management device, such as the Kubernetes API server, carrying the new resource application amount. This reallocation request can be a rolling update request for the deployment controller, updating the resource application amount of the newly created deployment controller. Upon receiving this request, the API server recreates the deployment controller based on the new resource application amount in the request. At this point, the recreated deployment controller maintains the original amount of memory, avoids excessive resource waste, and meets user needs.

[0121] In addition to the case where the resource usage is always greater than the resource application amount according to the third preset time, it can also be triggered according to other trigger conditions. For example, if the resource usage is greater than the resource application amount, and the excess is greater than a certain threshold, reallocation can also be triggered. It should be noted that the third preset time can also be selected to be equal to the second preset time, or different from the second preset time. In addition, within the third preset time, an expansion request can be executed to expand the capacity. Similarly, expansion does not affect the reallocation of resources within the third preset time. In addition, if the CPU is greater than 1, reallocation can be performed in the following manner, such as creating at least two deployment controllers, which will not be repeated here.

[0122] In addition to scaling capacity directly based on resource usage, as described above, you can also increase resources appropriately based on resource usage to ensure service needs are met as closely as possible, reduce the number of scaling operations, and reduce the need to rebuild virtual devices, saving time and resources. For example, you can increase the CPU and memory allocations. Alternatively, you can increase the CPU allocation while maintaining the memory allocation in the resource request. Then, you can create virtual devices based on the increased resource allocations. The specific implementation is similar to the previous description and will not be further explained here.

[0123] To ensure that resources are distributed as evenly as possible and to reuse multi-core performance, when the number of CPUs in resource usage is greater than 1, the number of deployment controllers can be determined based on the number of CPUs (or the number of CPU cores).

[0124] Specifically, the method 200 also includes: determining through the target virtual device that the resource usage is greater than the resource application amount within a third preset time, and receiving a reallocation request sent by the target virtual device; increasing the number of CPUs and the number of memories in the resource application amount according to the resource usage, and when the increased number of CPUs is greater than 1, determining the amount of memory corresponding to each CPU, and sending the memory amount corresponding to each CPU as a corresponding reallocation request to the management device, so that the management device receives the reallocation request, and creates a virtual device controller corresponding to each CPU according to the memory amount corresponding to each CPU, and each virtual device controller has at least one corresponding virtual device.

[0125] The method for determining resource usage has been previously described and will not be repeated here.

[0126] Increasing the CPU and memory amounts in the resource application based on resource usage can mean that the increased resource application amounts can be the same as the resource usage, that is, the resource application amounts are still determined directly based on resource usage. Alternatively, the pre-set resource amounts can be directly increased based on the resource usage. The following example only illustrates that the increased resource application amounts can be the same as the resource usage:

[0127] Figure 3B 300B is a flow chart showing the process of recreating the target virtual device. Figure 3BAs shown, the created platform service 3012 (i.e., the platform service 3012 in the deployment controller 3011) monitors that the resource usage is always greater than the resource application amount within the third preset time, such as 5 hours. Then, the maximum resource usage is used as the final resource usage within the third preset time. The platform service 3012 executes step 310: sends a reallocation request, i.e., an expansion request, to the scheduling center 104, and the request carries the final resource usage. After receiving the request from the platform service 3012, the scheduling center 104 can determine the new resource application amount based on the resource usage, and at the same time determine whether the number of CPUs is greater than 1. If the number of CPUs is 2 and the amount of memory is 2G, the scheduling center divides the resource usage into two groups of new resource application amounts, each group of new resource application amounts is 1 CPU and 1G memory. Based on the two new resource request amounts, the scheduling center 104 sends a reallocation request to the management device 105, such as the Kubernetes API server, executing step 311: sending a reallocation request to the management device 105. This request can include the two new resource request amounts to create two deployment controllers. Alternatively, two requests can be sent, each including a new set of resource request amounts. This request can be a rolling update request for deployment controller 3011, updating the resource request amounts of deployment controller 3011. After receiving the request, the API server recreates the corresponding deployment controllers 3016 and 3017 based on each set of new resource request amounts in the request, executing step 312: recreating the target virtual device. In this case, the recreated deployment controller 3016 is version 4 (i.e., Version 4), and the recreated deployment controller 3017 is version 5 (i.e., Version 5).

[0128] When user traffic decreases and the monitored resource usage remains at a low level (which may be below the resource usage threshold), a request to reduce the resource usage is sent to the dispatch center. This means that the capacity can be reduced using the above-mentioned scaling-in method. If the dispatch center receives two scaling-in requests at the same time, it can merge the two scaling-in requests to save resources:

[0129] Specifically, when the resource usage does not meet the resource application amount of the target virtual device, the target virtual device is recreated directly based on the resource usage, including: when at least two scaling-down requests sent by the target virtual device are received, the at least two scaling-down requests are merged to obtain a merged scaling-down request; and the target virtual device is recreated based on the merged scaling-down request.

[0130] In this case, merging at least two scaling-down requests also requires merging their resource usage, so that a new capacity is created based on the merged scaling-down requests. The specific creation process has been explained above and will not be repeated here.

[0131] Specifically, when the resource usage does not meet the resource application amount of the target virtual device, the target virtual device is recreated directly based on the resource usage, including: when at least two scaling-down requests are received, the at least two scaling-down requests are merged, and the resource usage carried in the at least two scaling-down requests are merged, and the merged CPU is less than or equal to 1; sending the merged scaling-down request to the management device, so that the management device creates a virtual device controller based on the merged resource usage carried in the merged scaling-down request, and each virtual device controller has at least one corresponding target virtual device.

[0132] The method for determining resource usage has already been determined and will not be repeated here.

[0133] Figure 3C 300C is a flow chart showing a process of recreating a target virtual device. Figure 3CAs shown, platform service 3012 in deployment controller 3018 and platform service 3012 in deployment controller 3019 monitor that resource usage is consistently less than the requested resource amount within a first preset time period, such as one hour. The maximum resource usage is then used as the final resource usage within the first preset time period. Platform service 3012 in deployment controller 3018 and platform service 3012 in deployment controller 3019 respectively execute step 320: Sending a Scale-Down Request and step 321: Sending a Scale-Down Request to scheduling center 104, each of which carries the final resource usage. After receiving these two scale-down requests, scheduling center 104 combines the CPU and memory amounts in these two requests to generate a set of CPU and memory amounts, such as CPU amount = 0.4 + 0.2 = 0.6 and memory amount = 1 + 1 = 2 GB. This is used as the combined resource usage. Based on this, the scheduling center 104 executes a merge and scale-down request to the management device 105, such as the Kubernetes API server, that is, executes step 322: sends a merge and scale-down request to the management device 105. The request can carry the merged resource usage to create a deployment controller. This request can be a rolling update request for the deployment controller 3018 and the deployment controller 3019. After receiving the request, the API server recreates the corresponding deployment controller 3020 based on the merged resource application amount in the request, that is, executes step 323: recreates the target virtual device. At this time, the recreated deployment controller 3020 is version 8 (i.e., Version 8). Among them, the deployment controller 3018 is version 6 (i.e., Version 6) and the deployment controller 3019 is version 7 (i.e., Version 7).

[0134] After merging the scaling-down requests, if further scaling-down is required, the number of CPUs can be maintained and the amount of memory can be reduced during the subsequent scaling-down process, thereby saving resources while maintaining the normal operation of the service.

[0135] Specifically, the method 200 includes: after recreating the target virtual device by merging the scaling-down requests, determining that the resource usage is less than the resource application amount through the recreated target virtual device, receiving the scaling-down request sent by the target virtual device; and recreating the target virtual device according to the scaling-down request.

[0136] The content here is similar to that in the previous article, so I will not elaborate on it here.

[0137] Specifically, the resource usage is determined to be less than the resource application amount by the re-created target virtual device, and a scaling-down request sent by the target virtual device is received, including: determining that the resource usage is less than the resource application amount within a first preset time by the re-created target virtual device, and receiving the scaling-down request sent by the target virtual device; re-creating the target virtual device according to the scaling-down request, including: using the amount of memory in the resource usage and the amount of CPU in the resource application amount as new resource application amounts, and sending the scaling-down request to the management device, so that the management device receives the scaling-down request, and re-creates the target virtual device according to the new resource usage carried in the scaling-down request.

[0138] The method for determining resource usage has already been determined and will not be repeated here.

[0139] It's important to note that the scaling-down process here is similar to the one described above, so I won't go into detail here. Simply put, after receiving a scaling-down request, the scheduling center uses only the memory usage in the resource usage and the CPU usage in the resource request as the new resource request, allowing the Kubernetes API server, for example, to scale down.

[0140] In addition, in addition to automatically rebuilding the virtual device, the embodiment of the present application can also interact with the user during the establishment process, or before and after the establishment, to improve the user's autonomy and user experience.

[0141] In order to enable the user to perceive the process of creating the virtual device, the user may be informed when the virtual device is created, such as by providing the user with a process interface for creation.

[0142] Specifically, the method 200 further includes: during the process of recreating the target virtual device, providing the recreated information to the presentation device for presentation to the user.

[0143] For example, as described above, when the management device creates a virtual device, it can send corresponding creation information to the user's computer through the scheduling center, such as the creation time, end time, and the current completion level of the creation, such as 60%, 90%, etc. The user's computer displays this information on the display interface to inform the user.

[0144] In addition, the resource limit can be modified according to user needs.

[0145] Specifically, the method 200 further includes: receiving a modification request, and modifying the current restricted resource amount of the target virtual device according to the restricted resource amount carried in the modification request.

[0146] For example, as described above, the dispatch center can receive a modification request from a user's computer that includes resource limits. The dispatch center can then modify the deployment controller's current resource limits. After modifying the limits, the center can also send the modified resource limits to Kubernetes for deployment controller creation.

[0147] When creating a virtual device, if the creation fails, the relevant user can also be informed so that the user can investigate the cause of the failure.

[0148] Specifically, the method 200 further includes: when re-creating the target virtual device fails, providing a failure message to the target user, so that the target user performs detection according to the failure message.

[0149] For example, as mentioned above, when Kubernetes fails to create a deployment controller, it sends a failure message to the dispatch center, which then sends it to the computer of the backend staff so that the backend staff can detect the cause of the failure.

[0150] In addition, priorities can be set for virtual devices or virtual device controllers, and capacity can be expanded or reduced based on the priorities, so that when a large number of expansions and reductions occur concurrently, expansions and reductions can be performed in sequence.

[0151] Specifically, the method 200 further includes: receiving calibration information, determining the priority of the created target virtual device according to the calibration information; and recreating the target virtual device according to the priority.

[0152] For example, as described above, the dispatch center can receive bid information sent by the user's computer, such as information setting the priority of an already created deployment controller. When the dispatch center receives multiple expansion and / or reduction requests, in addition to merging the reduction and reduction requests, it can also determine which deployment controller to create based on the priority.

[0153] Based on the above similar inventive concept, Figure 4 A flow chart of a resource scheduling method provided by another exemplary embodiment of the present application is shown. The method 400 provided in the embodiment of the present application is executed by a server, such as a cloud server, more specifically a scheduling center (also referred to as a scheduler) in the cloud server.

[0154] The method 400 includes the following steps:

[0155] 401: Determine resource usage of the created target virtual device.

[0156] 402: When it is determined through the target virtual device that the resource usage is greater than the resource application amount within the first preset time, a reallocation request sent by the target virtual device is received.

[0157] 403: Determine whether the CPU quantity in the resource usage carried in the reallocation request meets the preset quantity, and whether the memory quantity in the resource usage meets the memory quantity in the resource application quantity. If so, recreate the target virtual device based on the CPU quantity in the resource usage and the memory quantity in the resource application quantity.

[0158] Since the specific implementation of steps 401-403 has been described in detail above, they will not be repeated here. It is only explained that the first preset time in step 402 is the third preset time mentioned above.

[0159] In addition, the method 400 also includes: determining through the target virtual device that the resource usage is greater than the resource application amount within a first preset time, then receiving a reallocation request sent by the target virtual device; increasing the number of CPUs and the number of memories in the resource application amount based on the resource usage, and when the increased number of CPUs is greater than 1, determining the amount of memory corresponding to each CPU, and sending the memory amount corresponding to each CPU as a corresponding reallocation request to the management device, so that the management device receives the reallocation request, and creates a virtual device controller corresponding to each CPU based on the memory amount corresponding to each CPU, and each virtual device controller has at least one corresponding target virtual device.

[0160] In addition, the method 400 includes: when receiving two scaling-down requests sent by the target virtual device, merging the two scaling-down requests to obtain a merged scaling-down request; recreating the target virtual device according to the merged scaling-down request; determining that the resource usage is less than the resource application amount through the recreated target virtual device, receiving the scaling-down request sent by the target virtual device; and recreating the target virtual device according to the scaling-down request.

[0161] Since the above specific implementation has been described in detail above, it will not be repeated here. It is only explained that the second preset time here refers to the first preset time above.

[0162] In addition, for the contents not described in detail in the present method 400 , reference may also be made to the various steps in the above-mentioned method 200 .

[0163] Based on the above similar inventive concept, Figure 5 A flow chart of a resource scheduling method provided by another exemplary embodiment of the present application is shown. The method 400 provided in the embodiment of the present application is executed by a server, such as a cloud server, more specifically a management device for virtual devices in the cloud server, such as Kubernetes. The method 500 includes the following steps:

[0164] 501: Receive a reallocation request and directly determine a new resource application amount based on the resource usage amount carried in the reallocation request.

[0165] 502: Recreate the target virtual device according to the new resource application amount carried in the reallocation request to replace the created target virtual device, so that the available resource amount of the recreated target virtual device is the resource usage amount.

[0166] It should be noted that the execution entity of the above method 500, in addition to Kubernetes, can also be other applications that manage virtual devices. Conversely, virtual devices can also include virtual machines and sandboxes in addition to corresponding containers. Accordingly, method 500 can also be executed by a management device that manages virtual machines and sandboxes.

[0167] Since the specific implementation of steps 501-502 has been described in detail above, they will not be repeated here. It is only explained that the reallocation request in step 501 can include the above-mentioned shrinking request and expansion request, and of course it can also be the above-mentioned reallocation request.

[0168] Specifically, the new resource application amount is determined directly according to the resource usage amount carried in the reallocation request, including: using the resource usage amount as the new resource application amount.

[0169] It should be noted that the new resource request amount can be determined by the scheduling center. That is, the scheduling center uses the resource usage as the new resource request amount and sends it to the management device via a reallocation request. The new resource request amount can also be determined by the management device itself.

[0170] In addition, the method 500 also includes: determining through the scheduling center that the number of CPUs in the new resource application amount is greater than 1, then determining the amount of memory corresponding to each CPU, and sending the amount of memory corresponding to each CPU as a corresponding reallocation request; receiving the reallocation request, and creating a virtual device controller corresponding to each CPU based on the amount of memory corresponding to each CPU, each virtual device controller having at least one corresponding target virtual device.

[0171] In addition, the method 500 also includes: receiving at least two scaling-down requests through a scheduling center, merging the at least two scaling-down requests to obtain a merged scaling-down request, and sending the merged scaling-down request; receiving the merged scaling-down request, and creating a virtual device controller based on the merged resource usage carried in the merged scaling-down request, each virtual device controller having at least one corresponding target virtual device.

[0172] Since the above specific implementation methods have been explained in detail above, they will not be repeated here.

[0173] In addition, for the contents not described in detail in the present method 500 , reference may also be made to the steps in the above-mentioned method 200 .

[0174] Based on the above similar inventive concept, Figure 6 A flow chart of a resource scheduling method provided by another exemplary embodiment of the present application is shown. The method 600 provided in the embodiment of the present application is executed by a server, such as a cloud server, and more specifically, a virtual device in the cloud server, such as a container. The method 600 includes the following steps:

[0175] 601: Monitors the resource usage of the target virtual device.

[0176] 602: When it is determined that the resource usage does not meet the resource application amount of the target virtual device, a reallocation request is sent to the scheduling center so that the scheduling center determines a new resource application amount based on the resource usage carried in the reallocation request, and sends the new resource usage to the management device through the reallocation request so that the management device recreates the target virtual device based on the new resource application amount carried in the reallocation request.

[0177] Since the specific implementation of steps 601-602 has been described in detail above, they will not be repeated here. It is only explained that the reallocation request in step 601 may include the aforementioned shrinking request and expansion request, and of course it can also be the aforementioned reallocation request.

[0178] In addition, for the contents not described in detail in the present method 600 , reference may also be made to the steps in the above-mentioned method 200 .

[0179] Figure 7 This is a schematic diagram of the structural framework of a resource scheduling device provided in an exemplary embodiment of the present application. The device 700 can be applied to a server, such as a cloud server, and specifically can be a scheduling center (also called a scheduler) in the cloud server. The device 700 includes: a determination module 701 and a creation module 702; the functions of each module are described in detail below:

[0180] The determination module 701 is configured to determine the resource usage of the created target virtual device.

[0181] The creation module 702 is configured to recreate the target virtual device directly based on the resource usage when the resource usage does not meet the resource application amount of the target virtual device.

[0182] In addition, the creation module 702 is further configured to receive a creation request and create at least one corresponding target virtual device through the management device according to the resource request amount carried in the creation request.

[0183] Specifically, the creation module 702 is configured to send a creation request to the management device, so that the management device creates a corresponding virtual device controller according to the resource application amount, and the virtual device controller has at least one corresponding target virtual device.

[0184] Specifically, the determination module 701 is configured to monitor the resource usage of the target virtual device through the target virtual device and determine the resource usage of the target virtual device.

[0185] When the number of target virtual devices is at least two, the determination module 701 is configured to monitor resource usage of at least two target virtual devices through one of the target virtual devices and determine the resource usage of the at least two target virtual devices.

[0186] Specifically, the creation module 702 is used to recreate the target virtual device by performing horizontal expansion or contraction according to resource usage; or to recreate the target virtual device by performing vertical expansion or contraction according to resource usage.

[0187] Specifically, creation module 702 is used to determine through the target virtual device that the resource usage is less than the resource application amount within the first preset time, receive the scaling down request sent by the target virtual device, and send the scaling down request to the management device so that the management device recreates the target virtual device according to the resource usage carried in the scaling down request.

[0188] Specifically, creation module 702 is used to determine through the target virtual device that the resource usage is greater than the resource application amount within the second preset time, receive the expansion request sent by the target virtual device, and send the expansion request to the management device so that the management device can recreate the target virtual device according to the resource usage carried in the expansion request.

[0189] Wherein, at least two target virtual devices belong to the same virtual device controller.

[0190] In addition, the determination module 701 is further configured to determine a limited resource amount that is greater than the resource usage amount based on the resource usage amount; and the creation module 702 is configured to recreate the target virtual device based on the limited resource amount.

[0191] In addition, the device 700 also includes: a receiving module, which is used to determine through the target virtual device that the resource usage is greater than the resource application amount within a third preset time, and then receive a reallocation request sent by the target virtual device; a determination module 701, which is also used to determine whether the number of CPUs in the resource usage carried by the reallocation request meets the preset number, and whether the number of memory in the resource usage meets the number of memory in the resource application amount. If so, the target virtual device is recreated based on the number of CPUs in the resource usage and the number of memory in the resource application amount.

[0192] In addition, the receiving module is used to determine through the target virtual device that the resource usage is greater than the resource application amount within a third preset time, and then receive the reallocation request sent by the target virtual device; the determination module 701 is also used to determine that the number of CPUs in the resource usage carried by the reallocation request is less than or equal to 1, and the number of memory in the resource usage is less than or equal to the number of memory in the resource application amount, and then use the number of CPUs in the resource usage and the number of memory in the resource application amount as new resource application amounts, and send a reallocation request to the management device based on the new resource application amounts, so that the management device receives the reallocation request and recreates the target virtual device based on the new resource application amounts carried in the reallocation request.

[0193] In addition, the sending module is also used to determine through the target virtual device that the resource usage is greater than the resource application amount within a third preset time, and receive a reallocation request sent by the target virtual device; the determination module 701 is also used to increase the number of CPUs and the number of memories in the resource application amount based on the resource usage. When the increased number of CPUs is greater than 1, the amount of memory corresponding to each CPU is determined, and the amount of memory corresponding to each CPU is sent to the management device as a corresponding reallocation request, so that the management device receives the reallocation request and creates a virtual device controller corresponding to each CPU based on the amount of memory corresponding to each CPU, and each virtual device controller has at least one corresponding virtual device.

[0194] In addition, the creation module 702 includes: a merging unit for merging at least two shrinking requests sent by the target virtual device to obtain a merged shrinking request upon receiving at least two shrinking requests; and a creation unit for recreating the target virtual device according to the merged shrinking request.

[0195] In addition, the sending module is also used to recreate the target virtual device by merging the reduction request, determine that the resource usage is less than the resource application amount through the recreated target virtual device, and receive the reduction request sent by the target virtual device; the creation module 702 is used to recreate the target virtual device according to the reduction request.

[0196] In addition, the sending module is used to determine that the resource usage of the re-created target virtual device is less than the resource application amount within the first preset time, and receive the reduction request sent by the target virtual device; the creation module 702 is used to use the memory amount in the resource usage and the CPU amount in the resource application amount as the new resource application amount, and send them to the management device through a reduction request, so that the management device receives the reduction request and recreates the target virtual device according to the new resource usage carried in the reduction request.

[0197] Among them, virtual devices include: containers.

[0198] The recreated target virtual device is used to replace the created target virtual device, so that the available resource amount of the recreated target virtual device is the resource usage amount.

[0199] In addition, the apparatus 700 further includes: a providing module, configured to provide re-creation information to a display device during the process of re-creating the target virtual device, so as to display the information to the user.

[0200] In addition, the apparatus 700 further includes: a modification module, configured to receive a modification request and modify the current restricted resource amount of the target virtual device according to the restricted resource amount carried in the modification request.

[0201] In addition, a module is provided for providing a failure message to a target user when re-creation of the target virtual device fails, so that the target user can perform detection according to the failure message.

[0202] In addition, the determination module 701 is further configured to receive calibration information and determine the priority of the created target virtual device according to the calibration information; the creation module 702 is further configured to recreate the target virtual device according to the priority.

[0203] Figure 8 A schematic diagram of the structural framework of a resource scheduling device provided by another exemplary embodiment of the present application is shown. The device 800 can be applied to a server, such as a cloud server, and specifically can be a scheduling center (also called a scheduler) in the cloud server. The device 800 includes: a determination module 801 and a sending module 802. The functions of each module are described in detail below:

[0204] The determination module 801 is configured to determine the resource usage of the created target virtual device.

[0205] The sending module 802 is configured to receive a reallocation request sent by the target virtual device when it is determined through the target virtual device that the resource usage is greater than the resource application amount within a first preset time.

[0206] Determination module 801 is used to determine whether the number of CPUs in the resource usage carried by the reallocation request meets the preset number, and whether the number of memory in the resource usage meets the number of memory in the resource application. If so, recreate the target virtual device based on the number of CPUs in the resource usage and the number of memory in the resource application.

[0207] In addition, the sending module 802 is used to determine through the target virtual device that the resource usage is greater than the resource application amount within the first preset time, and then receive the reallocation request sent by the target virtual device; the determination module 801 is used to increase the number of CPUs and the number of memories in the resource application amount according to the resource usage. When the increased number of CPUs is greater than 1, the amount of memory corresponding to each CPU is determined, and the memory amount corresponding to each CPU is sent to the management device as the corresponding reallocation request, so that the management device receives the reallocation request and creates a virtual device controller corresponding to each CPU according to the memory amount corresponding to each CPU, and each virtual device controller has at least one corresponding target virtual device.

[0208] Furthermore, the apparatus 800 includes: a merging module configured to, upon receiving two scaling-down requests from a target virtual device, merge the two scaling-down requests to obtain a combined scaling-down request; a creating module configured to recreate the target virtual device based on the combined scaling-down request; and a receiving module configured to determine, through the recreated target virtual device, that resource usage is less than resource application amount, and receive the scaling-down request from the target virtual device; and a creating module configured to recreate the target virtual device based on the scaling-down request.

[0209] It should be noted that for some contents not mentioned in the apparatus 800 , reference may be made to the contents of the above-mentioned apparatus 700 .

[0210] Figure 9 A schematic diagram of the structural framework of a resource scheduling device provided by another exemplary embodiment of the present application is shown. The device 900 can be applied to a server, such as a cloud server, and specifically can be a management device for virtual devices in a cloud server, such as Kubernetes. The device 900 includes: a determination module 901 and a creation module 902. The functions of each module are described in detail below:

[0211] The determination module 901 is configured to receive a reallocation request and directly determine a new resource application amount based on the resource usage amount carried in the reallocation request.

[0212] The creation module 902 is configured to recreate a target virtual device according to the new resource application amount carried in the reallocation request, to replace the created target virtual device, so that the available resource amount of the recreated target virtual device is the resource usage amount.

[0213] Specifically, the determination module 901 is configured to use the resource usage as a new resource application amount.

[0214] In addition, the determination module 901 is also used to determine through the scheduling center that the number of CPUs in the new resource application amount is greater than 1, then determine the amount of memory corresponding to each CPU, and send the amount of memory corresponding to each CPU as the corresponding reallocation request; the creation module 902 is also used to receive the reallocation request and create a virtual device controller corresponding to each CPU based on the amount of memory corresponding to each CPU, and each virtual device controller has at least one corresponding target virtual device.

[0215] In addition, the apparatus 900 further includes: a merging module, configured to receive at least two scaling-down requests through a scheduling center, merge the at least two scaling-down requests to obtain a merged scaling-down request, and send the merged scaling-down request.

[0216] In addition, the creation module 902 is further configured to receive a merge and shrink request, and create a virtual device controller according to the merged resource usage carried in the merge and shrink request, where each virtual device controller has at least one corresponding target virtual device.

[0217] It should be noted that for some contents not mentioned in the apparatus 900 , reference may be made to the contents of the above-mentioned apparatus 700 .

[0218] Figure 10 The following is a schematic diagram of the structural framework of a resource scheduling device provided by another exemplary embodiment of the present application. The device 1000 can be applied to a server, such as a cloud server, and specifically can be a virtual device in the cloud server, such as a container. The device 1000 includes: a monitoring module 1001 and a sending module 1002. The functions of each module are described in detail below:

[0219] Monitoring module 1001, used to monitor resource usage of target virtual devices;

[0220] The sending module 1002 is used to send a reallocation request to the scheduling center when it is determined that the resource usage does not meet the resource application amount of the target virtual device, so that the scheduling center determines the new resource application amount based on the resource usage carried in the reallocation request, and sends the new resource usage to the management device through the reallocation request, so that the management device recreates the target virtual device based on the new resource application amount carried in the reallocation request.

[0221] It should be noted that for some contents not mentioned in the apparatus 1000 , reference may be made to the contents of the above-mentioned apparatus 700 .

[0222] The above describes Figure 7 The internal functions and structure of the device 700 shown, in one possible design, Figure 7 The structure of the apparatus 700 shown can be implemented as a computing device, such as a server. Figure 11 As shown, the device 1100 may include: a memory 1101 and a processor 1102 .

[0223] The memory 1101 is used to store computer programs.

[0224] Processor 1102 is configured to execute a computer program to: determine resource usage of a created target virtual device; and recreate the target virtual device directly based on the resource usage if the resource usage does not meet the resource application amount of the target virtual device.

[0225] In addition, the processor 1102 is further configured to: receive a creation request, and create at least one corresponding target virtual device through the management device according to the resource request amount carried in the creation request.

[0226] Specifically, the processor 1102 is specifically configured to send a creation request to the management device, so that the management device creates a corresponding virtual device controller according to the resource application amount, where the virtual device controller has at least one corresponding target virtual device.

[0227] Specifically, the processor 1102 is specifically configured to: monitor the resource usage of the target virtual device through the target virtual device, and determine the resource usage of the target virtual device.

[0228] When the number of target virtual devices is at least two, the processor 1102 is specifically configured to: monitor the resource usage of at least two target virtual devices through one of the target virtual devices, and determine the resource usage of the at least two target virtual devices.

[0229] Specifically, the processor 1102 is specifically configured to: recreate the target virtual device by horizontally expanding or shrinking the capacity according to resource usage; or recreate the target virtual device by vertically expanding or shrinking the capacity according to resource usage.

[0230] Specifically, processor 1102 is specifically used to: compare the resource usage with the resource application amount through the target virtual device, receive the capacity configuration request sent by the target virtual device based on the comparison result, and send the capacity configuration request to the management device, so that the management device can recreate the target virtual device according to the resource usage carried in the capacity configuration request.

[0231] Specifically, processor 1102 is specifically used to: determine through the target virtual device that the resource usage is less than the resource application amount within the first preset time, receive a scaling-down request sent by the target virtual device, and send the scaling-down request to the management device, so that the management device recreates the target virtual device according to the resource usage carried in the scaling-down request.

[0232] Specifically, processor 1102 is specifically used to: determine through the target virtual device that the resource usage is greater than the resource application amount within the second preset time, receive the expansion request sent by the target virtual device, and send the expansion request to the management device, so that the management device can recreate the target virtual device according to the resource usage carried in the expansion request.

[0233] Wherein, at least two target virtual devices belong to the same virtual device controller.

[0234] In addition, the processor 1102 is further configured to: determine a limited resource amount that is greater than the resource usage amount according to the resource usage amount; and recreate the target virtual device according to the limited resource amount.

[0235] In addition, processor 1102 is also used to: determine through the target virtual device that the resource usage is greater than the resource application amount within a third preset time, then receive a reallocation request sent by the target virtual device; determine whether the number of CPUs in the resource usage carried by the reallocation request meets the preset number, and whether the number of memories in the resource usage meets the number of memories in the resource application amount; if so, recreate the target virtual device based on the number of CPUs in the resource usage and the number of memories in the resource application amount.

[0236] In addition, processor 1102 is also used to: determine through the target virtual device that the resource usage is greater than the resource application amount within a third preset time, then receive a reallocation request sent by the target virtual device, determine that the number of CPUs in the resource usage carried by the reallocation request is less than or equal to 1, and the amount of memory in the resource usage is less than or equal to the amount of memory in the resource application amount, then use the number of CPUs in the resource usage and the amount of memory in the resource application amount as new resource application amounts, and send a reallocation request to the management device based on the new resource application amounts, so that the management device receives the reallocation request and recreates the target virtual device based on the new resource application amounts carried in the reallocation request.

[0237] In addition, processor 1102 is also used to: determine through the target virtual device that the resource usage is greater than the resource application amount within a third preset time, and receive a reallocation request sent by the target virtual device; increase the number of CPUs and the number of memories in the resource application amount according to the resource usage, and when the increased number of CPUs is greater than 1, determine the amount of memory corresponding to each CPU, and send the memory amount corresponding to each CPU as the corresponding reallocation request to the management device, so that the management device receives the reallocation request sent by the scheduling center, and creates a virtual device controller corresponding to each CPU according to the memory amount corresponding to each CPU, and each virtual device controller has at least one corresponding virtual device.

[0238] In addition, the processor 1102 is specifically configured to: upon receiving at least two scaling-down requests sent by the target virtual device, merge the at least two scaling-down requests to obtain a merged scaling-down request; and recreate the target virtual device according to the merged scaling-down request.

[0239] In addition, processor 1102 is also used to: after recreating the target virtual device by merging the reduction request, determine that the resource usage is less than the resource application amount through the recreated target virtual device, receive the reduction request sent by the target virtual device; and recreate the target virtual device according to the reduction request.

[0240] In addition, processor 1102 is also used to: determine through the recreated target virtual device that the resource usage is less than the resource application amount within the first preset time, and receive a scaling-down request sent by the target virtual device; use the amount of memory in the resource usage and the amount of CPU in the resource application amount as the new resource application amount, and send the scaling-down request to the management device, so that the management device receives the scaling-down request and recreates the target virtual device according to the new resource usage carried in the scaling-down request.

[0241] Among them, virtual devices include: containers.

[0242] The recreated target virtual device is used to replace the created target virtual device, so that the available resource amount of the recreated target virtual device is the resource usage amount.

[0243] In addition, the processor 1102 is further configured to: during the process of recreating the target virtual device, provide the recreated information to the presentation device for presentation to the user.

[0244] In addition, the processor 1102 is further configured to receive a modification request, and modify the current restricted resource amount of the target virtual device according to the restricted resource amount carried in the modification request.

[0245] In addition, the processor 1102 is further configured to: when re-creating the target virtual device fails, provide a failure message to the target user, so that the target user can perform detection according to the failure message.

[0246] In addition, the processor 1102 is further configured to: receive calibration information, determine the priority of the created target virtual device according to the calibration information; and recreate the target virtual device according to the priority.

[0247] In addition, an embodiment of the present invention provides a computer storage medium, which, when a computer program is executed by one or more processors, causes the one or more processors to implement Figure 2 The present invention provides steps of a resource scheduling method in a method embodiment.

[0248] The above describes Figure 8 The internal functions and structure of the device 800 shown, in one possible design, Figure 8 The structure of the apparatus 800 shown can be implemented as a computing device, such as a server. Figure 12 As shown, the device 1200 may include: a memory 1201 , a processor 1202 , and a communication component 1203 .

[0249] The memory 1201 is used to store computer programs.

[0250] The processor 1202 is configured to execute a computer program to determine, for a created target virtual device, a resource usage of the target virtual device.

[0251] The communication component 1203 is configured to receive a reallocation request sent by the target virtual device when it is determined through the target virtual device that the resource usage is greater than the resource application amount within a first preset time.

[0252] Processor 1202 is used to determine whether the number of CPUs in the resource usage carried in the reallocation request meets the preset number, and whether the number of memory in the resource usage meets the number of memory in the resource application. If so, recreate the target virtual device based on the number of CPUs in the resource usage and the number of memory in the resource application.

[0253] In addition, processor 1202 is also used to: determine through the target virtual device that the resource usage is greater than the resource application amount within the first preset time, then receive a reallocation request sent by the target virtual device; increase the number of CPUs and the number of memories in the resource application amount according to the resource usage, and when the increased number of CPUs is greater than 1, determine the amount of memory corresponding to each CPU, and send the memory amount corresponding to each CPU as the corresponding reallocation request to the management device, so that the management device receives the reallocation request, and creates a virtual device controller corresponding to each CPU according to the memory amount corresponding to each CPU, and each virtual device controller has at least one corresponding target virtual device.

[0254] In addition, processor 1202 is also used to: when receiving two scaling-down requests sent by the target virtual device, merge the two scaling-down requests to obtain a merged scaling-down request; recreate the target virtual device based on the merged scaling-down request; determine that the resource usage is less than the resource application amount through the recreated target virtual device, and receive the scaling-down request sent by the target virtual device; and recreate the target virtual device based on the scaling-down request.

[0255] It should be noted that for some contents not mentioned in the device 1200 , reference may be made to the contents of the above-mentioned device 1100 .

[0256] In addition, an embodiment of the present invention provides a computer storage medium, which, when a computer program is executed by one or more processors, causes the one or more processors to implement Figure 4 The present invention provides steps of a resource scheduling method in a method embodiment.

[0257] The above describes Figure 9 The internal functions and structure of the device 900 shown, in one possible design, Figure 9 The structure of the apparatus 900 shown can be implemented as a computing device, such as a server. Figure 13 As shown, the device 1300 may include: a memory 1301 and a processor 1302 .

[0258] The memory 1301 is used to store computer programs.

[0259] The processor 1302 is configured to execute a computer program to: receive a reallocation request, and directly determine a new resource application amount according to the resource usage amount carried in the reallocation request.

[0260] According to the new resource application amount carried in the reallocation request, a target virtual device is recreated to replace the created target virtual device, so that the available resource amount of the recreated target virtual device is the resource usage amount.

[0261] Specifically, the processor 1302 is configured to use the resource usage amount as a new resource application amount.

[0262] In addition, processor 1302 is also used to: determine through the scheduling center that the number of CPUs in the new resource application amount is greater than 1, then determine the amount of memory corresponding to each CPU, and send the amount of memory corresponding to each CPU as the corresponding reallocation request; receive the reallocation request, and create a virtual device controller corresponding to each CPU based on the amount of memory corresponding to each CPU, each virtual device controller having at least one corresponding target virtual device.

[0263] In addition, the processor 1302 is further configured to: receive at least two scaling-down requests through the scheduling center, merge the at least two scaling-down requests to obtain a merged scaling-down request, and send the merged scaling-down request.

[0264] In addition, the device may further include a communication component 1303 for receiving a merge and shrink request, and creating a virtual device controller according to the merged resource usage carried in the merge and shrink request, wherein each virtual device controller has at least one corresponding target virtual device.

[0265] It should be noted that for some contents not mentioned in device 1300, reference may be made to the contents of the above-mentioned device 1100.

[0266] In addition, an embodiment of the present invention provides a computer storage medium, which, when a computer program is executed by one or more processors, causes the one or more processors to implement Figure 5 The present invention provides steps of a resource scheduling method in a method embodiment.

[0267] The above describes Figure 10 The internal functions and structure of the device 1000 shown, in one possible design, Figure 10 The structure of the apparatus 1000 shown can be implemented as a computing device, such as a server. Figure 14 As shown, the device 1400 may include: a memory 1401 , a processor 1402 , and a communication component 1403 .

[0268] The memory 1401 is used to store computer programs.

[0269] The processor 1402 is configured to execute a computer program to monitor resource usage of a target virtual device.

[0270] Communication component 1403 is used to send a reallocation request to the scheduling center when it is determined that the resource usage does not meet the resource application amount of the target virtual device, so that the scheduling center determines the new resource application amount based on the resource usage carried in the reallocation request, and sends the new resource usage to the management device through the reallocation request, so that the management device recreates the target virtual device based on the new resource application amount carried in the reallocation request.

[0271] It should be noted that for some contents not mentioned in the device 1400 , reference may be made to the contents of the above-mentioned device 1100 .

[0272] In addition, an embodiment of the present invention provides a computer storage medium, which, when a computer program is executed by one or more processors, causes the one or more processors to implement Figure 6 The present invention provides steps of a resource scheduling method in a method embodiment.

[0273] Based on the similar inventive concept described above, another exemplary embodiment of the present application provides a model deployment method. The method 1500 provided in the embodiment of the present application is executed by a server, such as a cloud server, more specifically a dispatch center in the cloud server. The method 1500 includes the following steps:

[0274] 1501: For the created target virtual device, determine the resource usage of the target virtual device.

[0275] 1502: When the resource usage does not meet the resource application amount of the target virtual device, directly recreate the target virtual device based on the resource usage.

[0276] 1503: Based on the re-creation of the target virtual device, deploy the service model to provide services to the user according to the service model.

[0277] Since the specific implementation of steps 1501-1503 has been explained in detail above, they will not be repeated here.

[0278] Among them, the service models include MT models and NLP models.

[0279] In addition, the method 1500 further includes: receiving a service request, determining a corresponding service model according to the service request; obtaining output information of the determined service model according to the service input information carried in the service request and the determined service model, and returning the output information.

[0280] Since this has been explained above, I will not elaborate on it here. I will just explain that the input information can be the input information of the model, such as the text to be translated, and the output information can be the output information of the model, such as the translated text.

[0281] In addition, for the contents not described in detail in the present method 1500 , reference may also be made to the various steps in the above-mentioned method 200 .

[0282] Another exemplary embodiment of the present application provides a model deployment device. The device 1600 can be applied to a server, such as a cloud server, and specifically can be a dispatch center in the cloud server. The device 1600 includes: a determination module 1601, a creation module 1602, and a sending module 100. The functions of each module are described in detail below:

[0283] The determination module 1601 is configured to determine the resource usage of the created target virtual device.

[0284] The creation module 1602 is configured to directly recreate the target virtual device based on the resource usage when the resource usage does not meet the resource application amount of the target virtual device.

[0285] The deployment module 1603 is used to deploy the service model based on the re-creation of the target virtual device, so as to provide services to the user according to the service model.

[0286] Among them, the service models include MT models and NLP models.

[0287] In addition, the determination module 1601 is also used to receive a service request and determine the corresponding service model based on the service request; the device 1600 also includes: a return module, which is used to obtain the output information of the determined service model based on the service input information carried by the service request and the determined service model, and return the output information.

[0288] It should be noted that for some contents not mentioned in the apparatus 1600 , reference may be made to the contents of the above-mentioned apparatus 700 .

[0289] The above describes the internal functions and structure of the apparatus 1600 . In one possible design, the structure of the apparatus 1600 can be implemented as a computing device, such as a server. The apparatus 1700 may include: a memory 1701 and a processor 1702 .

[0290] The memory 1701 is used to store computer programs.

[0291] Processor 1702 is used to execute a computer program to: determine the resource usage of the target virtual device that has been created; if the resource usage does not meet the resource application amount of the target virtual device, directly recreate the target virtual device based on the resource usage; based on the recreated target virtual device, deploy a service model to provide services to users according to the service model.

[0292] Among them, the service models include MT models and NLP models.

[0293] In addition, the processor 1702 is further used to: receive a service request, determine a corresponding service model according to the service request; obtain output information of the determined service model according to the service input information carried in the service request and the determined service model, and return the output information.

[0294] It should be noted that for some contents not mentioned in device 1700, reference may be made to the contents of the above-mentioned device 1100.

[0295] In addition, an embodiment of the present invention provides a computer storage medium. When a computer program is executed by one or more processors, the computer storage medium causes the one or more processors to implement the steps of a resource scheduling method in the embodiment of method 1500.

[0296] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The sequence numbers of the operations, such as 201, 202, 203, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0297] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0298] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by a combination of hardware and software. Based on this understanding, the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0299] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable multimedia data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable multimedia data processing device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0300] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable multimedia data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0301] These computer program instructions can also be loaded onto a computer or other programmable multimedia data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0302] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0303] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0304] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0305] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A resource scheduling method, characterized in that: include: For the created target virtual device, determining the resource usage of the target virtual device; In the case that the resource usage does not satisfy the resource application amount of the target virtual device, directly re-creating the target virtual device according to the resource usage; Among them, directly recreating the target virtual device based on the resource usage includes: determining through the target virtual device that the resource usage is greater than the resource application amount within a third preset time, then receiving a reallocation request sent by the target virtual device; determining whether the CPU resource usage in the resource usage carried by the reallocation request meets the preset quantity, and whether the memory amount in the resource usage meets the memory amount in the resource application amount, and if so, recreating the target virtual device based on the CPU resource usage in the resource usage and the memory amount in the resource application amount.

2. The method according to claim 1, characterized in that The method further comprises: A creation request is received, and according to the resource application amount carried in the creation request, at least one corresponding target virtual device is created through a management device.

3. The method according to claim 2, characterized in that The step of creating at least one corresponding target virtual device by a management device according to the resource application amount carried in the creation request includes: The creation request is sent to the management device, so that the management device creates a corresponding virtual device controller according to the resource application amount, and the virtual device controller has at least one corresponding target virtual device.

4. The method according to claim 1, wherein Determining the resource usage of the target virtual device includes: The resource usage of the target virtual device is monitored through the target virtual device, and the resource usage of the target virtual device is determined.

5. The method according to claim 4, characterized in that When the number of the target virtual devices is at least two, resource usage of the at least two target virtual devices is monitored by one of the target virtual devices, and the resource usage of the at least two target virtual devices is determined.

6. The method according to claim 1, characterized in that When the resource usage does not satisfy the resource application amount of the target virtual device, directly recreating the target virtual device according to the resource usage includes: Performing horizontal expansion or contraction according to the resource usage to recreate the target virtual device; or Based on the resource usage, vertical expansion or vertical contraction is performed to recreate the target virtual device.

7. The method according to claim 1, characterized in that When the resource usage does not satisfy the resource application amount of the target virtual device, directly recreating the target virtual device according to the resource usage includes: The resource usage is compared with the resource application amount through the target virtual device, and based on the comparison result, a capacity configuration request sent by the target virtual device is received, and the capacity configuration request is sent to the management device, so that the management device recreates the target virtual device according to the resource usage carried in the capacity configuration request.

8. The method according to claim 7, characterized in that The comparing the resource usage with the resource application amount by the target virtual device, receiving a capacity configuration request sent by the target virtual device according to the comparison result, and sending the capacity configuration request to the management device, includes: The target virtual device determines that the resource usage is less than the resource application amount within a first preset time, receives a scaling-down request sent by the target virtual device, and sends the scaling-down request to the management device, so that the management device recreates the target virtual device according to the resource usage carried in the scaling-down request.

9. The method according to claim 7, characterized in that The comparing the resource usage with the resource application amount by the target virtual device, receiving a capacity configuration request sent by the target virtual device according to the comparison result, and sending the capacity configuration request to the management device, includes: The target virtual device determines that the resource usage is greater than the resource application amount within a second preset time, receives an expansion request sent by the target virtual device, and sends the expansion request to the management device so that the management device recreates the target virtual device according to the resource usage carried in the expansion request.

10. The method according to claim 5, characterized in that The at least two target virtual devices belong to the same virtual device controller.

11. The method according to claim 1, 6 or 7, characterized in that The method further comprises: Determining, based on the resource usage, a limited resource amount that is greater than the resource usage; Recreate the target virtual device based on the limited resource amount.

12. The method according to claim 1, characterized in that The method further comprises: If the target virtual device determines that the resource usage is greater than the resource application amount within a third preset time, receiving a reallocation request sent by the target virtual device; Determine that the number of CPUs in the resource usage carried in the reallocation request is less than or equal to 1, and the number of memory in the resource usage is less than or equal to the number of memory in the resource application amount, then use the number of CPUs in the resource usage and the number of memory in the resource application amount as new resource application amounts, and send a reallocation request to the management device based on the new resource application amounts, so that the management device receives the reallocation request and recreates the target virtual device based on the new resource application amounts carried in the reallocation request.

13. The method according to claim 1, wherein The method further comprises: Determining, by the target virtual device, that the resource usage is greater than the resource application amount within a third preset time, and receiving a reallocation request sent by the target virtual device; According to the resource usage, the number of CPUs and the amount of memory in the resource application amount are increased. When the increased number of CPUs is greater than 1, the amount of memory corresponding to each CPU is determined, and the amount of memory corresponding to each CPU is sent to the management device as the corresponding reallocation request, so that the management device receives the reallocation request and creates a virtual device controller corresponding to each CPU according to the amount of memory corresponding to each CPU, and each virtual device controller has at least one corresponding virtual device.

14. The method according to claim 1, wherein When the resource usage does not satisfy the resource application amount of the target virtual device, directly recreating the target virtual device according to the resource usage includes: When receiving at least two scaling-down requests sent by the target virtual device, merging the at least two scaling-down requests to obtain a merged scaling-down request; Recreate the target virtual device according to the merge and shrink request.

15. The method according to claim 14, characterized in that After recreating the target virtual device by merging the shrinking requests, the method further includes: Determining, by the recreated target virtual device, that the resource usage is less than the resource application amount, and receiving a scaling-down request sent by the target virtual device; Recreate the target virtual device according to the shrink request.

16. The method according to claim 15, characterized in that The recreated target virtual device determines that the resource usage is less than the resource application amount, and receives a scaling-down request sent by the target virtual device, including: Determining, by the recreated target virtual device, that the resource usage is less than the resource application amount within a first preset time, and receiving a scaling-down request sent by the target virtual device; The re-creating the target virtual device according to the scaling-down request includes: The memory amount in the resource usage and the CPU amount in the resource application amount are used as new resource application amounts, and a scaling-down request is sent to a management device so that the management device receives the scaling-down request and recreates the target virtual device according to the new resource usage carried in the scaling-down request.

17. The method according to any one of claims 1 to 10, characterized in that The virtual device includes: a container.

18. The method according to any one of claims 1 to 10, characterized in that The recreated target virtual device is used to replace the created target virtual device, so that the available resource amount of the recreated target virtual device is the resource usage amount.

19. The method according to claim 1, wherein The method further comprises: When re-creating the target virtual device fails, a failure message is provided to the target user, so that the target user can perform detection according to the failure message.

20. A resource scheduling method, characterized in that: include: For the created target virtual device, determining the resource usage of the target virtual device; The created target virtual device is created according to the resource application amount of the target virtual device; When it is determined through the target virtual device that the resource usage is greater than the resource application amount within a first preset time, receiving a reallocation request sent by the target virtual device; Determine whether the CPU quantity in the resource usage carried in the reallocation request meets the preset quantity, and whether the memory quantity in the resource usage meets the memory quantity in the resource application quantity. If so, recreate the target virtual device based on the CPU quantity in the resource usage and the memory quantity in the resource application quantity.

21. The method according to claim 20, characterized in that The method further comprises: Determining, by the target virtual device, that the resource usage is greater than the resource application amount within a first preset time, then receiving a reallocation request sent by the target virtual device; According to the resource usage, the number of CPUs and the amount of memory in the resource application amount are increased. When the increased number of CPUs is greater than 1, the amount of memory corresponding to each CPU is determined, and the amount of memory corresponding to each CPU is sent to the management device as the corresponding reallocation request, so that the management device receives the reallocation request and creates a virtual device controller corresponding to each CPU according to the amount of memory corresponding to each CPU, and each virtual device controller has at least one corresponding target virtual device.

22. The method according to claim 21, characterized in that The method comprises: When receiving two scaling-down requests sent by the target virtual device, the two scaling-down requests are merged to obtain a merged scaling-down request; Re-creating the target virtual device according to the merge and shrink request; Determining, by the recreated target virtual device, that the resource usage is less than the resource application amount, and receiving a scaling-down request sent by the target virtual device; Recreate the target virtual device according to the shrink request.

23. A resource scheduling method, characterized in that: include: receiving a reallocation request and determining a new resource application amount directly based on the resource usage amount carried in the reallocation request; recreating a target virtual device according to the new resource application amount carried in the reallocation request to replace the created target virtual device, so that the available resource amount of the recreated target virtual device is the resource usage amount; If the number of CPUs in the new resource request is greater than 1, the scheduling center determines the amount of memory corresponding to each CPU and sends the corresponding reallocation request with the amount of memory corresponding to each CPU; The reallocation request is received, and a virtual device controller corresponding to each CPU is created according to the memory quantity corresponding to each CPU, wherein each virtual device controller has at least one corresponding target virtual device.

24. The method according to claim 23, wherein The determining of a new resource application amount directly based on the resource usage amount carried in the reallocation request includes: The resource usage amount is used as the new resource application amount.

25. The method according to claim 23, characterized in that The method further comprises: receiving at least two scaling-down requests through a scheduling center, merging the at least two scaling-down requests to obtain a merged scaling-down request, and sending the merged scaling-down request; The merge and shrink request is received, and a virtual device controller is created according to the merged resource usage carried in the merge and shrink request, where each virtual device controller has at least one corresponding target virtual device.

26. A model deployment method, characterized in that: include: For the created target virtual device, determining the resource usage of the target virtual device; In the case that the resource usage does not satisfy the resource application amount of the target virtual device, directly re-creating the target virtual device according to the resource usage; Deploying a service model based on the recreated target virtual device to provide services to users according to the service model; Among them, directly recreating the target virtual device based on the resource usage includes: determining through the target virtual device that the resource usage is greater than the resource application amount within a third preset time, then receiving a reallocation request sent by the target virtual device; determining whether the CPU resource usage in the resource usage carried by the reallocation request meets the preset quantity, and whether the memory amount in the resource usage meets the memory amount in the resource application amount, and if so, recreating the target virtual device based on the CPU resource usage in the resource usage and the memory amount in the resource application amount.

27. The method according to claim 26, characterized in that The service model includes an MT model and an NLP model.

28. The method according to claim 26, characterized in that The method further comprises: Receive a service request, and determine a corresponding service model according to the service request; According to the service input information carried in the service request and the determined service model, output information of the determined service model is obtained, and the output information is returned.

29. A computing device comprising: Memory and processor; The memory is used to store computer programs; The processor is configured to execute the computer program to: For the created target virtual device, determining the resource usage of the target virtual device; In the case that the resource usage does not satisfy the resource application amount of the target virtual device, directly re-creating the target virtual device according to the resource usage; Among them, directly recreating the target virtual device based on the resource usage includes: determining through the target virtual device that the resource usage is greater than the resource application amount within a third preset time, then receiving a reallocation request sent by the target virtual device; determining whether the CPU resource usage in the resource usage carried by the reallocation request meets the preset quantity, and whether the memory amount in the resource usage meets the memory amount in the resource application amount, and if so, recreating the target virtual device based on the CPU resource usage in the resource usage and the memory amount in the resource application amount.

30. A computing device comprising: memory, processors, and communication components; The memory is used to store computer programs; The processor is configured to execute the computer program to: For the created target virtual device, determining the resource usage of the target virtual device; The communication component is configured to receive a reallocation request sent by the target virtual device when it is determined through the target virtual device that the resource usage is greater than the resource application amount within a first preset time; The processor is used to determine whether the CPU resource usage in the resource usage carried by the reallocation request meets the preset quantity, and whether the memory quantity in the resource usage meets the memory quantity in the resource application quantity. If so, recreate the target virtual device based on the CPU resource usage in the resource usage and the memory quantity in the resource application quantity.

31. A computing device comprising: Memory and processor; The memory is used to store computer programs; The processor is configured to execute the computer program to: receiving a reallocation request and determining a new resource application amount directly based on the resource usage amount carried in the reallocation request; recreating a target virtual device according to the new resource application amount carried in the reallocation request to replace the created target virtual device, so that the available resource amount of the recreated target virtual device is the resource usage amount; If the number of CPUs in the new resource request is greater than 1, the scheduling center determines the amount of memory corresponding to each CPU and sends the corresponding reallocation request with the amount of memory corresponding to each CPU; The reallocation request is received, and a virtual device controller corresponding to each CPU is created according to the memory quantity corresponding to each CPU, wherein each virtual device controller has at least one corresponding target virtual device.

32. A computing device comprising: memory, processors, and communication components; The memory is used to store computer programs; The processor is configured to execute the computer program to: monitor resource usage of a target virtual device; The communication component is configured to, when it is determined that the resource usage does not satisfy the resource application amount of the target virtual device, send a reallocation request to the scheduling center, so that the scheduling center determines a new resource application amount based on the resource usage carried in the reallocation request, and sends the new resource usage to the management device through the reallocation request, so that the management device recreates the target virtual device based on the new resource application amount carried in the reallocation request; Among them, directly recreating the target virtual device based on the resource usage includes: determining through the target virtual device that the resource usage is greater than the resource application amount within a third preset time, then receiving a reallocation request sent by the target virtual device; determining whether the CPU resource usage in the resource usage carried by the reallocation request meets the preset quantity, and whether the memory amount in the resource usage meets the memory amount in the resource application amount, and if so, recreating the target virtual device based on the CPU resource usage in the resource usage and the memory amount in the resource application amount.

33. A computing device comprising: memory, processors, and communication components; The memory is used to store computer programs; The processor is configured to execute the computer program to: determine, for a created target virtual device, a resource usage of the target virtual device; In the case that the resource usage does not satisfy the resource application amount of the target virtual device, directly re-creating the target virtual device according to the resource usage; Deploying a service model based on the recreated target virtual device to provide services to users according to the service model; Among them, directly recreating the target virtual device based on the resource usage includes: determining through the target virtual device that the resource usage is greater than the resource application amount within a third preset time, then receiving a reallocation request sent by the target virtual device; determining whether the CPU resource usage in the resource usage carried by the reallocation request meets the preset quantity, and whether the memory amount in the resource usage meets the memory amount in the resource application amount, and if so, recreating the target virtual device based on the CPU resource usage in the resource usage and the memory amount in the resource application amount.

34. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by one or more processors, the one or more processors are caused to implement the steps of the method according to any one of claims 1 to 28.

Images