Edge resource scheduling method and system driven by business request and service quality

By querying the status of service instances, determining the priority queue and dynamically adjusting the upper limit of concurrent requests in the serverless computing platform, the resource contention problem between multiple service instances among edge nodes is solved, and the service quality and resource utilization rate are improved.

CN116366727BActive Publication Date: 2024-11-12INFORMATION & COMM BRANCH OF STATE GRID JIANGSU ELECTRIC POWER
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310300866.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-24
Publication Date
2024-11-12
Estimated Expiration
2043-03-24

AI Technical Summary

Technical Problem

In a serverless computing platform, multiple service instances compete for node resources lead to resource competition and affect service quality. The existing technology has failed to effectively solve this problem in the edge node scenario.

Method used

By querying the service status of all service instances on edge nodes, determining the priority queue, and adjusting the upper limit of concurrent requests based on priority information, using pre-trained machine learning model to predict the upper limit of concurrent requests based on different resource volumes and concurrent requests, dynamic Adjust to avoid resource contention.

Benefits of technology

The proportion of service instances completing request processing within the set request delay limit is significantly increased, resource contention is reduced, resource utilization is improved, and resource requirements and delay requirements of different service instances.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116366727B_ABST
    Figure CN116366727B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and application of edge resource scheduling driven by business request and service quality, which is applied to a serverless computing cluster, wherein the serverless computing cluster includes multiple edge nodes, each edge node includes multiple service instances, and the method specifically includes: querying the service status of all service instances on the edge node to determine whether to issue an alarm message; if so, determining a priority queue based on the service status and delay sensitivity of all service instances on the edge node, wherein the priority queue includes service instances whose resource usage is greater than the application amount and the number of real-time concurrent requests is greater than 0, and the service instances in the priority queue have priority information; based on the priority information, adjusting the upper limit of the number of concurrent requests of the service instances in the priority queue. The method can adapt to the situation where different service instances require different resource upper limits, and can also adapt to the requirements of different service requests for delay.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of resource scheduling, and in particular to an edge resource scheduling method and application driven by business request and service quality. Background Art

[0002] The emergence of cloud computing has reduced the burden of users managing physical facilities, and has lowered the operation and maintenance management capabilities to the infrastructure layer, making out-of-the-box use a trend. Serverless computing, as a new cloud computing model, only requires users to provide the core code of the business, and the platform that supports the operation of the code is managed and maintained by the cloud vendor.

[0003] In a serverless computing platform, a service instance can handle multiple concurrent requests at the same time. However, under different resource allocations, the number of concurrent requests that a single instance can handle is different, and the latency requirements and request rates of different service instances may also be different. At the same time, when running multiple service instances on a single node, there may be resource contention. The resources allocated to an instance are limited. The higher the instance load, the more serious the resource contention will be, and the longer the request latency will be.

[0004] There is a positive correlation between instance load and request execution time. The greater the load, the greater the delay in instance processing requests. Therefore, it is necessary not only to set a reasonable initial load limit for the service instance under a certain amount of resources, but also to dynamically modify the load limit according to the real-time situation during the operation of the service instance. Therefore, how to ensure that the service instance is reasonably scheduled while meeting the service quality has become an urgent problem to be solved.

[0005] In the traditional monolithic application model, in order not to affect latency-sensitive applications, such applications are often allowed to monopolize nodes, but this will lead to low cluster resource utilization. Some studies have obtained the impact of application contention through offline testing, and used regression models or classification models to determine whether offline applications can be mixed. Some studies have also used online monitoring of the service quality of applications to determine whether latency-sensitive applications can run on the same node at the same time as more batch applications, or used collaborative filtering to obtain scheduling strategies using a small amount of offline test data. However, no research has been conducted on the competition for node resources among multiple service instances in serverless computing scenarios.

[0006] Cluster resource management platforms (such as YARN and Kubernetes) allow the resource limit of multiple instances on a node to exceed the node's allocatable resource limit, helping to handle application request loads at peak times. However, when multiple instances on a node are under high load, due to the limited node resources, processes between multiple instances will compete for resources, resulting in mutual service quality being affected.

[0007] Most of the existing service instance scheduling methods mentioned above do not consider the edge node service-free computing scenario. Multiple service instances on a node compete for node resources, and there are certain requirements for the latency of each service instance, but the sensitivity varies. At the same time, because a service instance can carry multiple concurrent requests, multiple requests will be limited by the resource limit of this instance, and resource contention will also occur. It is necessary to be able to set a reasonable upper limit on the number of concurrent requests based on the resource limit of the instance.

[0008] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0009] The purpose of the present invention is to provide an edge resource scheduling method and application driven by business request and service quality, which is used to solve the technical problem of how to ensure that service instances are reasonably scheduled while meeting the service quality.

[0010] To achieve the above object, an embodiment of the present invention provides a business request and service quality driven edge resource scheduling method, the method is applied to a serverless computing cluster, the serverless computing cluster includes multiple edge nodes, each of the edge nodes includes multiple service instances, the method includes:

[0011] Query the service status of all service instances on the edge node to determine whether to issue an alarm; if so,

[0012] Determine a priority queue based on the service status and latency sensitivity of all service instances on the edge node, wherein the priority queue includes service instances whose resource usage is greater than the application amount and whose number of real-time concurrent requests is greater than 0, and the service instances in the priority queue have priority information;

[0013] Based on the priority information, an upper limit on the number of concurrent requests for the service instances in the priority queue is adjusted.

[0014] In one or more embodiments of the present invention, querying the service status of all service instances on the edge node to determine whether to issue an alarm message specifically includes:

[0015] Query the proportion of all service instances on the edge node that complete request processing within the set request delay limit;

[0016] Based on the query ratio, it is determined whether to issue an alarm message.

[0017] In one or more embodiments of the present invention, adjusting the upper limit of the number of concurrent requests for the service instance in the priority queue based on the priority information specifically includes:

[0018] When the edge node receives the alarm information, it lowers the upper limit of the number of concurrent requests for the service instances in the priority queue in order of priority from low to high;

[0019] and / or, when the edge node does not receive the alarm information, periodically checking whether the resource usage of all service instances on the edge node is less than the total concurrent requests; if so,

[0020] In descending order of priority, the upper limit of the number of concurrent requests of the service instances in the priority queue whose upper limit of the number of concurrent requests has been lowered is adjusted upward.

[0021] In one or more embodiments of the present invention, the method further comprises:

[0022] When deploying the service instance to a cluster, the resource limit, standard request latency, and latency growth ratio of the service instance are input into a pre-trained machine learning model to obtain an upper limit on the number of concurrent requests for the service instance.

[0023] The standard request delay is the delay of a request of a service instance under no resource contention, and the delay growth ratio is the maximum growth ratio of a request of a service instance relative to the standard request delay under resource contention.

[0024] In one or more embodiments of the present invention, the method specifically comprises:

[0025] Obtain the request delay of the service instance in offline mode under the same resource limit and different numbers of concurrent requests, and the request delay under different resource limits and different numbers of concurrent requests;

[0026] Constructing a sample data set based on the resource limit, the number of concurrent requests, and the corresponding request latency;

[0027] Training the machine learning model based on the sample data set until a training condition is met;

[0028] Wherein, the machine learning model is a regression model.

[0029] In one or more embodiments of the present invention, the method further comprises:

[0030] Obtain the resource limit, number of concurrent requests, and request latency of new service instances in online mode under different resource limits and different numbers of concurrent requests, and build an updated data set.

[0031] The machine learning model is updated based on the updated dataset.

[0032] In one or more embodiments of the present invention, the method further comprises:

[0033] Obtain the request delays of different types of service instances under different resource limits and different numbers of concurrent requests to build a set of request delay growth ratios;

[0034] Determine the service level objective as the upper limit of service request delays of different types of service instances at different delay growth ratios, and screen out the partial request delays of each service instance that are greater than the upper limit of service request delays at different concurrent request numbers;

[0035] Inputting partial request delays, upper limits of different resource amounts, and different delay growth ratios of each of the service instances into the machine learning model to obtain a plurality of different upper limits of the number of concurrent requests corresponding to each of the service instances;

[0036] The maximum value among a plurality of different upper limits of the number of concurrent requests corresponding to each of the service instances is determined as the upper limit of the number of concurrent requests corresponding to the service instance.

[0037] In another aspect of the present invention, a service request and service quality driven edge resource scheduling system is provided, the system comprising:

[0038] The query module is used to query the service status of all service instances on the edge node to determine whether to issue an alarm message;

[0039] A determination module, configured to determine a priority queue based on the service status and delay sensitivity of all service instances on the edge node, wherein the priority queue includes service instances whose resource usage is greater than the application amount and whose number of real-time concurrent requests is greater than 0, and the service instances in the priority queue have priority information;

[0040] An adjustment module is used to adjust the upper limit of the number of concurrent requests for the service instance in the priority queue based on the priority information.

[0041] In another aspect of the present invention, there is provided an electronic device, comprising:

[0042] at least one processor; and

[0043] A memory storing instructions, which, when executed by the at least one processor, enables the at least one processor to execute the edge resource scheduling method driven by the business request and quality of service as described above.

[0044] In another aspect of the present invention, a machine-readable storage medium is provided, which stores executable instructions, and when the instructions are executed, the machine executes the edge resource scheduling method driven by business requests and quality of service as described above.

[0045] Compared with the prior art, the business request and service quality driven edge resource scheduling method and application according to the implementation mode of the present invention, by introducing a dynamic concurrent request limit adjustment strategy, can timely adjust the request concurrency limit of the service instance under the edge node when edge node resource contention occurs and an alarm is triggered, thereby reducing resource contention and surviving the cluster load peak period. At the same time, it can significantly improve the proportion of request processing completed by each service instance within the set request delay limit.

[0046] On the other hand, this method trains a machine learning model based on test data of the service instance's request latency under different resource limits and different concurrent request volumes, and uses the machine learning model to fit a value as the request concurrency limit when the service instance is deployed, so that it can adapt to situations where different service instances require different resource limits, and can also adapt to the latency requirements of requests from different services. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of a method for scheduling edge resources driven by business requests and quality of service according to an embodiment of the present invention;

[0048] Figure 2 It is a schematic diagram of the structural framework of a serverless computing cluster in a business request and service quality driven edge resource scheduling method according to an embodiment of the present invention;

[0049] Figure 3 It is a flowchart of lowering the upper limit of the number of concurrent requests for service instances in a priority queue in a method for scheduling edge resources driven by business requests and service quality according to an embodiment of the present invention;

[0050] Figure 4 It is a framework diagram for setting the upper limit of the number of concurrent requests for a service instance based on a pre-trained machine learning model in a business request and service quality driven edge resource scheduling method according to an embodiment of the present invention;

[0051] Figure 5 The structure of the edge resource scheduling system driven by business request and service quality according to one embodiment of the present invention is shown in FIG. Figure 1 ;

[0052] Figure 6 The structure of the edge resource scheduling system driven by business request and service quality according to one embodiment of the present invention is shown in FIG. Figure 2 ;

[0053] Figure 7 It is a hardware structure diagram of an electronic device for edge resource scheduling driven by business request and service quality according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation modes.

[0055] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising”, etc., will be understood to include the stated elements or components but not to exclude other elements or components.

[0056] Ginseng Figure 1 , introduces an embodiment of the edge resource scheduling method driven by business request and service quality of the present invention, the method is applied to a serverless computing cluster, the serverless computing cluster includes multiple edge nodes, and each edge node includes multiple service instances.

[0057] In a serverless computing cluster, a single service instance may handle multiple concurrent requests at the same time, but resources are allocated at the level of service instances. Therefore, multiple requests are limited by the resources of the service instance in which they are located, competing for resources and resulting in unsatisfactory service quality. At the same time, multiple service instances on the same edge node may exceed the resource request volume due to high load at the same time, which in turn causes high load on the node. Processes between multiple instances compete for resources and result in unsatisfactory service quality.

[0058] Based on the two technical problems of resource contention among multiple concurrent requests of a single instance and resource contention among multiple instances on the same node, the present invention mainly takes the setting and adjustment of the upper limit of the number of concurrent instance requests as the starting point, studies a service instance scheduling mechanism with service quality as the guarantee, solves the resource contention problem of different applications on the same host machine, analyzes the performance of typical applications under differentiated resource configuration conditions, counts the performance interference of applications in the host machine, and formulates a service instance scheduling strategy according to the application load of the host machine.

[0059] In this embodiment, the method includes the following steps.

[0060] S101, querying the service status of all service instances on the edge node to determine whether to issue an alarm message.

[0061] Specifically, the proportion of all service instances on the edge node that complete request processing within the set request delay limit is queried, and based on the queried proportion, it is determined whether to issue an alarm message.

[0062] Ginseng Figure 2 , the serverless computing cluster will have an overall cluster monitoring, which is responsible for pulling indicators on all edge nodes. Each node will have an alarm receiver, which is responsible for receiving alarms from service instances on its own node. The alarm receiver contains information about all service instances on its own node, including the priority information of the service instance, the upper limit of the number of concurrent requests for the service instance, etc.

[0063] The alarm receiver will periodically query all service instances under each edge node to see whether the alarm rules are triggered. For example, whether the request delay of a service instance is greater than the set request delay upper limit for a period of time. In this embodiment, the service level objective (SLO) of the request can be set as the request delay upper limit.

[0064] When multiple service instances are running on the same edge node, they may be under high load at the same time, resulting in resource competition. When resource contention occurs, the request latency of the service instance will be affected, and the proportion of service instances completing request processing within the set request latency limit will decrease. When it drops to the set threshold, an alarm will be triggered.

[0065] When an alarm is issued, a dynamic concurrent request limit adjustment strategy will be introduced.

[0066] S102: Determine a priority queue based on the service status and delay sensitivity of all service instances on the edge node.

[0067] The priority queue includes service instances whose resource usage is greater than the application amount and whose number of real-time concurrent requests is greater than 0. The service instances in the priority queue have priority information.

[0068] Specifically, in this embodiment, it is possible to determine which service instances are selected to join the priority queue according to the service status of all service instances on the edge node (for example, the resource usage is greater than the application amount, and the number of real-time concurrent requests is greater than 0). Then, according to the delay sensitivity of all service instances, the priority can be sorted from high to low (or from low to high), thereby obtaining a priority queue containing the priority information of the service instances.

[0069] S103: Based on the priority information, adjust the upper limit of the number of concurrent requests for the service instance in the priority queue.

[0070] Specific reference Figure 3When the alarm receiver of the edge node receives the alarm information, it can lower the upper limit of the number of concurrent requests of the service instances in the priority queue in order from low to high priority. For example, the upper limit of the number of concurrent requests of the service instance can be reduced by step (step>0). If the alarm is triggered repeatedly, the upper limit of the number of concurrent requests of the service instance will be continuously lowered until there is no resource contention between the service instances in the node and the alarm is issued.

[0071] It should be noted that for service instances of the same type, the upper limit of the number of concurrent requests on the same edge node will be lowered to avoid unbalanced request loads under multiple instances.

[0072] When the edge node does not receive the alarm information, the alarm receiver will periodically check whether the upper limit of the number of concurrent requests of all service instances on the edge node needs to be restored.

[0073] The recovery strategy is equivalent to the reverse downward adjustment strategy. Each time, the priority is adjusted upward in descending order, and the service instance that has been adjusted upward beforehand has had its concurrent request limit lowered.

[0074] Specifically, the alarm receiver will periodically check whether the resource usage of all service instances on the edge node is less than the total number of concurrent requests. If the resource usage of all service instances is less than the total number of concurrent requests, the total resource usage on the node will not exceed the node resource allocatable amount, and a recovery strategy can be adopted, that is, in order of priority from high to low, the upper limit of the number of concurrent requests in the priority queue is raised for the service instances whose upper limit of concurrent requests has been lowered. In other words, through the strategy of increasing the number of concurrent requests, the node scheduler can schedule more requests in, which can significantly increase the proportion of request processing completed by each service instance within the set request delay limit, thereby improving resource utilization.

[0075] The above solution mainly solves the problem of resource contention among multiple service instances in the same edge node. By introducing a dynamic concurrent request limit adjustment strategy, when edge node resource contention occurs and an alarm is triggered, the concurrent request limit of the service instance under the edge node can be adjusted in time to reduce resource contention and survive the cluster load peak period. At the same time, it can significantly improve the proportion of request processing completed by each service instance within the set request delay limit.

[0076] Ginseng Figure 4 , the business request and service quality driven edge resource scheduling method also includes: in the stage of deploying the service instance to the cluster, the resource limit, standard request delay and delay growth ratio of the service instance are input into the pre-trained machine learning model to obtain the upper limit of the number of concurrent requests of the service instance.

[0077] The standard request delay is the delay of the service instance request under no resource contention, and the delay growth ratio is the maximum growth ratio of the service instance request relative to the standard request delay under resource contention.

[0078] In this embodiment, the pre-training of the machine learning model is performed under offline testing.

[0079] Specifically, first obtain the service instance f in offline mode i Request latency under the same resource limit Limit and different numbers of concurrent requests con∈{1,...,N}, as well as request latency under different resource limits Limit∈{k,2k...,nk} and different numbers of concurrent requests, where k is the basic unit of resource allocation. For example, when the resource is CPU, k is 0.5 cores.

[0080] Then, based on the upper limit of all resource quantities, the number of concurrent requests, and the corresponding request latency, a sample data set is constructed. Finally, the machine learning model is trained based on the sample data set until the training conditions are met.

[0081] In this embodiment, the machine learning model may be a regression model, and the specific model structure is:

[0082] (latency, ratio, f, limit) → con

[0083] Among them, the input vector of the regression model is:

[0084] f represents a certain type of service; limit is the upper limit of the resource volume of service f.

[0085] Latency indicates the latency of the service request in the absence of resource contention, that is, the standard request latency. You can select the average latency, the 95th percentile latency, or the 99th percentile latency.

[0086] ratio is related to the set request latency upper limit SLO (Service Level Objective), ratio∈{100+k,100+2k...,100+nk}, where the value is a percentage of latency, k represents the unit of percentage change, and ratio specifically indicates how many times the request latency upper limit is the average latency when resource contention occurs. If the upper limit is exceeded, the request is deemed to have failed to meet the request SLO. The system should try its best to increase the satisfaction ratio of the request SLO. For example, if service f sets ratio to 115% of latency, the request latency cannot exceed 115%*latency.

[0087] The output of the regression model is the number of concurrent requests con under the limit setting that meets the ratio requirement.

[0088] Based on the above scheme, the request latency of the service instance was tested under different resource limits and different concurrent request amounts, and a certain sample data set was generated according to the input and output design of the regression model. When the service instance needs to be deployed to the cluster, the regression model will output the setting value of the upper limit of the number of concurrent requests based on the resource limit set for the instance and the service request latency quality of service requirements.

[0089] This method initially needs to obtain a certain data set based on offline testing to train the regression model. During the online process, data such as the request latency, the number of concurrent requests carried at the time, and the upper limit of the instance's resource volume during the operation of the serverless computing cluster will be collected to update the model.

[0090] Continue to participate Figure 4 , obtain the resource limit, number of concurrent requests, and request latency of the new service instance in online mode under different resource limits and different numbers of concurrent requests, build an updated data set, and update the machine learning model based on the updated data set.

[0091] Specifically, when the deployment request of service f reaches the gateway, the trained regression model will output the upper limit of concurrent requests con based on the upper limit of resource volume limit, latency latency and latency growth ratio ratio of service f. When a service instance needs to be generated, con will be loaded into the service instance as a configuration and limit the upper limit of concurrent requests when the service instance is running.

[0092] Cluster monitoring will continuously collect and save data from service f. When data from a new service f that has not been trained offline is deployed to the cluster, a default concurrency limit will be set for the new service f instance. When the latency data of the new service f has collected a certain amount of data under different resource allocation limits and different numbers of concurrent requests, the regression model will be updated.

[0093] Different types of services have different latency requirements. Obtain the request latency of different types of service instances with different resource limits and different numbers of concurrent requests to build a request latency growth ratio set.

[0094] The request latency growth ratio contains the growth ratios of different latencies. For different percentile latencies, you can get the specific percentile latency of service f, that is, the standard request latency, or the average latency. Then, different data is generated based on different resource allocation upper limits and different request latency growth ratios.

[0095] The service level objective SLO is determined as the upper limit of service request latency for different types of service instances at different latency growth ratios. Some service requests may have large differences between average latency and high percentile latency. The request latency of some services that is greater than the upper limit of service request latency under different numbers of concurrent requests is screened out.

[0096] Then, the partial request delay, different resource limits, and different delay growth ratios of each service instance are input into the machine learning model to obtain multiple different concurrent request limits corresponding to each service instance. Finally, the maximum value among the multiple different concurrent request limits corresponding to each service instance is determined as the concurrent request limit corresponding to the service instance.

[0097] Ginseng Figure 5 , introduces an embodiment of the edge resource scheduling system driven by business request and service quality of the present invention. In this embodiment, the system includes a query module 201, a determination module 202 and an adjustment module 203.

[0098] A query module 201 is used to query the service status of all service instances on the edge node to determine whether to issue an alarm message;

[0099] A determination module 202 is used to determine a priority queue based on the service status and delay sensitivity of all service instances on the edge node, wherein the priority queue includes service instances whose resource usage is greater than the application amount and whose number of real-time concurrent requests is greater than 0, and the service instances in the priority queue have priority information;

[0100] The adjustment module 203 is used to adjust the upper limit of the number of concurrent requests of the service instance in the priority queue based on the priority information.

[0101] In one implementation, the query module 201 is specifically used to: query the proportion of all service instances on the edge node that complete request processing within a set request delay upper limit; and determine whether to issue an alarm based on the queried proportion.

[0102] In one embodiment, the adjustment module 203 is specifically used to: when the edge node receives an alarm message, lower the upper limit of the number of concurrent requests for the service instances in the priority queue in order of priority from low to high; and / or, when the edge node does not receive an alarm message, periodically check whether the resource usage of all service instances on the edge node is less than the total number of concurrent requests; if so, increase the upper limit of the number of concurrent requests for the service instances in the priority queue whose upper limit of the number of concurrent requests has been lowered in order of priority from high to low.

[0103] Ginseng Figure 6In one embodiment, the system may further include an input module 204 , a construction module 205 and an update module 206 .

[0104] The input module 204 is used to input the resource limit, standard request delay and delay growth ratio of the service instance into the pre-trained machine learning model when the service instance is deployed to the cluster, so as to obtain the upper limit of the number of concurrent requests for the service instance; wherein the standard request delay is the delay of the service instance request under no resource contention, and the delay growth ratio is the maximum growth ratio of the service instance request relative to the standard request delay under resource contention.

[0105] Construction module 205 is used to obtain the request delay of the service instance in offline mode under the same resource limit and different concurrent request numbers, as well as the request delay under different resource limits and different concurrent request numbers; based on the resource limit, the number of concurrent requests and the corresponding request delay, a sample data set is constructed; based on the sample data set, a machine learning model is trained until the training conditions are met; wherein the machine learning model is a regression model.

[0106] Update module 206 is used to obtain the resource limit, concurrent request number, and request delay under different resource limits and different concurrent request numbers of the new service instance in the online mode, and construct an updated data set; update the machine learning model based on the updated data set.

[0107] Figure 7 FIG. 2 shows a hardware structure diagram of an electronic device 30 for edge resource scheduling driven by business request and quality of service according to an embodiment of this specification. Figure 7 As shown, the electronic device 30 may include at least one processor 301, a memory 302 (e.g., a non-volatile memory), a memory 303, and a communication interface 304, and the at least one processor 301, the memory 302, the memory 303, and the communication interface 304 are connected together via a bus 305. At least one processor 301 executes at least one computer-readable instruction stored or encoded in the memory 302.

[0108] It should be understood that the computer executable instructions stored in the memory 302, when executed, cause at least one processor 301 to perform the above combined operations in various embodiments of this specification. Figure 1-4 Describes the various operations and functions.

[0109] In the embodiments of the present specification, the electronic device 30 may include, but is not limited to, personal computers, server computers, workstations, desktop computers, laptop computers, notebook computers, mobile computing devices, smart phones, tablet computers, cellular phones, personal digital assistants (PDAs), handheld devices, messaging devices, wearable computing devices, consumer electronic devices, and the like.

[0110] According to one embodiment, a program product such as a computer-readable storage medium is provided. The computer-readable storage medium may have instructions (i.e., the above-mentioned elements implemented in the form of software), which, when executed by a computer, causes the computer to perform the above-mentioned combination of various embodiments of this specification. Figure 1-4 Specifically, a system or device equipped with a readable storage medium may be provided, on which a software program code implementing the functions of any of the above-mentioned embodiments is stored, and a computer or processor of the system or device reads and executes the instructions stored in the readable storage medium.

[0111] According to the business request and service quality driven edge resource scheduling method and application of the implementation mode of the present invention, by introducing a dynamic concurrent request limit adjustment strategy, when edge node resource contention occurs and an alarm is triggered, the concurrent request limit of the service instance under the edge node can be adjusted in time to reduce resource contention and survive the cluster load peak period. At the same time, it can significantly improve the proportion of request processing completed by each service instance within the set request delay limit.

[0112] On the other hand, this method trains a machine learning model based on test data of the service instance's request latency under different resource limits and different concurrent request volumes, and uses the machine learning model to fit a value as the request concurrency limit when the service instance is deployed, so that it can adapt to situations where different service instances require different resource limits, and can also adapt to the latency requirements of requests from different services.

[0113] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0117] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. A service request and service quality driven edge resource scheduling method, characterized in that: Applied to a serverless computing cluster, the serverless computing cluster includes multiple edge nodes, each of the edge nodes includes multiple service instances, the method includes: Query the service status of all service instances on the edge node to determine whether to issue an alarm; if so, Determine a priority queue based on the service status and latency sensitivity of all service instances on the edge node, wherein the priority queue includes service instances whose resource usage is greater than the application amount and whose number of real-time concurrent requests is greater than 0, and the service instances in the priority queue have priority information; Based on the priority information, adjusting the upper limit of the number of concurrent requests for the service instance in the priority queue; Among them, querying the service status of all service instances on the edge node to determine whether to issue an alarm message specifically includes: querying the proportion of all service instances on the edge node that complete request processing within the set request delay upper limit; based on the queried proportion, determining whether to issue an alarm message.

2. The edge resource scheduling method driven by business request and service quality as claimed in claim 1, characterized in that: Based on the priority information, the upper limit of the number of concurrent requests for the service instance in the priority queue is adjusted, specifically including: When the edge node receives the alarm information, it lowers the upper limit of the number of concurrent requests for the service instances in the priority queue in order of priority from low to high; and / or, when the edge node does not receive the alarm information, periodically checking whether the resource usage of all service instances on the edge node is less than the total concurrent requests; if so, In descending order of priority, the upper limit of the number of concurrent requests of the service instances in the priority queue whose upper limit of the number of concurrent requests has been lowered is adjusted upward.

3. The edge resource scheduling method driven by business request and service quality as claimed in claim 1, characterized in that: The method further comprises: When deploying the service instance to the cluster, the resource limit, standard request latency, and latency growth ratio of the service instance are input into the pre-trained machine learning model to obtain the upper limit of the number of concurrent requests for the service instance; The standard request delay is the delay of a request of a service instance under no resource contention, and the delay growth ratio is the maximum growth ratio of a request of a service instance relative to the standard request delay under resource contention.

4. The method for scheduling edge resources driven by business request and service quality as claimed in claim 3, characterized in that: The method specifically comprises: Obtain the request delay of the service instance in offline mode under the same resource limit and different numbers of concurrent requests, and the request delay under different resource limits and different numbers of concurrent requests; Constructing a sample data set based on the resource limit, the number of concurrent requests, and the corresponding request latency; Training the machine learning model based on the sample data set until a training condition is met; Wherein, the machine learning model is a regression model.

5. The method for scheduling edge resources driven by business request and service quality as claimed in claim 3, characterized in that: The method further comprises: Obtain the resource limit, number of concurrent requests, and request latency of new service instances in online mode under different resource limits and different numbers of concurrent requests, and build an updated data set. The machine learning model is updated based on the updated dataset.

6. The method for scheduling edge resources driven by business request and service quality as claimed in claim 3, characterized in that: The method further comprises: Obtain the request delays of different types of service instances under different resource limits and different numbers of concurrent requests to build a set of request delay growth ratios; Determine the service level objective as the upper limit of service request delays of different types of service instances at different delay growth ratios, and screen out the partial request delays of each service instance that are greater than the upper limit of service request delays at different concurrent request numbers; Inputting partial request delays, upper limits of different resource amounts, and different delay growth ratios of each of the service instances into the machine learning model to obtain a plurality of different upper limits of the number of concurrent requests corresponding to each of the service instances; The maximum value among a plurality of different upper limits of the number of concurrent requests corresponding to each of the service instances is determined as the upper limit of the number of concurrent requests corresponding to the service instance.

7. A business request and service quality driven edge resource scheduling system, characterized in that: The system comprises: A query module is used to query the service status of all service instances on the edge node to determine whether to issue an alarm message; wherein, querying the service status of all service instances on the edge node to determine whether to issue an alarm message specifically includes: querying the proportion of all service instances on the edge node that complete request processing within a set request delay upper limit; based on the queried proportion, determining whether to issue an alarm message; A determination module, configured to determine a priority queue based on the service status and delay sensitivity of all service instances on the edge node, wherein the priority queue includes service instances whose resource usage is greater than the application amount and whose number of real-time concurrent requests is greater than 0, and the service instances in the priority queue have priority information; An adjustment module is used to adjust the upper limit of the number of concurrent requests for the service instance in the priority queue based on the priority information.

8. An electronic device comprising: at least one processor; as well as A memory storing instructions, which, when executed by the at least one processor, enables the at least one processor to execute the edge resource scheduling method driven by business requests and quality of service as described in any one of claims 1 to 6.

9. A machine-readable storage medium storing executable instructions, which, when executed, enable the machine to execute the business request and service quality driven edge resource scheduling method according to any one of claims 1 to 6.