Online container scheduling method and system based on interference scoring

Abstract

The application provides an online container scheduling method and system based on interference scoring, comprising: an offline training step: monitoring CPU utilization and deployment of instance runtime of an online service, and performing application profiling and training a CPU utilization prediction model according to the monitoring data; an online running step: performing candidate host filtering, generating a candidate set and interference scoring based on the CPU utilization prediction model; obtaining the comprehensive interference of online instances on the whole CPU, and selecting a candidate group with the minimum comprehensive interference to deploy the scheduled instance. According to the evaluation result of the interference, the application can quickly recommend the candidate host with the minimum interference and the corresponding vcpus set, so as to improve the performance of online applications and reduce their delay.

Description

Technical Field

[0001] This invention relates to the field of scheduling system technology, and more specifically, to an online container scheduling method and system based on interference scoring. Background Technology

[0002] Online services are applications that provide services to users. Online services are typically latency-sensitive and are therefore also called Latency-sensitive Services (LS services). Common LS services include search engines, email, and e-commerce websites. Typically, LS services have the following characteristics: First, multiple LS services collaborate to process user requests in the form of microservices, forming a call stack. Second, as business volume grows, LS services are usually deployed in clusters, with each LS service having dozens to thousands of instances running the same code, and a load balancer handling a portion of the user requests. If the performance of a portion of the instances of an LS service degrades, it will affect the tail latency of the service. Third, LS instances run as containers on different hosts in the cluster, and their lifecycles typically range from a few hours to several months. Compared to batch jobs, LS services generally have longer lifecycles and, due to their user-facing nature, have more stringent performance requirements. In this invention, the terms "container" or "instance" refer to a single replica entity of an online service used to process requests.

[0003] To improve resource utilization, multiple LS service instances typically run on a single physical server. While this deployment improves resource utilization and reduces costs, competition for physical resources can cause them to interfere with each other, potentially leading to performance degradation. To reduce interference between LS instances, existing solutions employ exclusive binding, where a set of vCPUs is allocated to a single LS instance on a server, preventing other LS instances from using this set of vCPUs (in this article, lowercase vCPU represents a logical core in a Linux system; typically, an Intel CPU with Hyperthreading enabled uses one hyperthread, while uppercase CPU represents a processor block; therefore, in multi-core processors, a single CPU may have dozens of vCPUs). However, this exclusive binding approach still cannot alleviate interference caused by contention for shared hardware resources, such as LLC (last level cache) and memory bandwidth contention. This is due to multi-socket (CPU socket) servers and Intel's Hyperthreading technology (where a physical core can be virtualized into two logical cores, each with its own registers but sharing L1 / L2 cache). This makes the interference caused by competition for CPU internal resources by LS instances more complex. Take a dual-socket server as an example: each server has two CPUs. Each CPU has n physical cores, which are virtualized into 2n vCPUs. Two vCPUs sharing a physical core share the L1 / L2 cache, vCPUs within the same socket share the LLC, and vCPUs belonging to two different CPUs do not share cache, therefore there is no cache contention between them. This phenomenon of different cores sharing different cache resources provides an opportunity to reduce interference by optimizing core-binding strategies.

[0004] Patent document CN115562829A (application number: CN202211211699.7) discloses an interference-aware scheduling system based on task classification and interference scoring, including a BE task classifier, a host interference quantifier, and an interference-aware scheduler. The task classifier performs category inference on BE task instances to obtain task categories. The host interference quantifier evaluates the interference level of candidate hosts based on the BE distribution on the target host to be scheduled, and obtains the interference levels of different candidate hosts. The interference-aware scheduler schedules BE task instances to candidate hosts corresponding to the specified interference levels.

[0005] Existing schedulers, such as those for Kubernetes or Docker Swarm, support container core binding. However, these general-purpose schedulers are unaware of the running status of applications within containers or the interference between them. Therefore, they do not consider inter-container interference during the scheduling phase. Other existing interference-aware scheduling techniques typically only schedule to a single machine without further recommending a set of vcpus (a container is not bound to more than one vCPU; vcpus is a set of vCPUs, such as vCPU 0–vCPU 3). Therefore, this invention proposes a method that can evaluate the interference of a scheduled container deployed on candidate hosts and corresponding vcpus, and based on this interference evaluation, recommend candidate hosts and corresponding vcpus that minimize interference. Summary of the Invention

[0006] In view of the shortcomings of the prior art, the purpose of this invention is to provide an online container scheduling method and system based on interference scoring.

[0007] The online container scheduling method based on interference scoring provided by the present invention includes:

[0008] Offline training steps: Monitor the CPU utilization and deployment status of online service instances during runtime, and create application profiles and train CPU utilization prediction models based on the monitoring data;

[0009] Online operation steps: Candidate host filtering, candidate set generation, and interference scoring based on CPU utilization prediction model;

[0010] The candidate host filtering includes filtering out hosts in the cluster that do not meet the resource requirements, and obtaining a batch of candidate hosts.

[0011] The generation of the candidate set includes using a sliding window method to further generate a candidate set of vcpus on the candidate hosts, thereby generating a candidate set of candidate hosts and vcpus set tuples for container scheduling;

[0012] The interference scoring includes scoring individual instances and comprehensive scoring. Before scheduling an instance of an online service, the interference experienced by the instance if it is scheduled to a specified host and vCPU number in each candidate group of the candidate set is evaluated, as well as the interference experienced by the original instances on the candidate hosts. The interference experienced by each instance is aggregated to obtain the comprehensive interference of online instances on the entire CPU. The candidate group with the least comprehensive interference is selected to deploy the scheduled instance.

[0013] Preferably, for a single instance, the data that needs to be monitored includes:

[0014] (instance_id,node_ip,cpu_list,cpu_util)

[0015] Where instance_id, node_ip, cpu_list, and cpu_util are the instance's unique ID, the machine's IP address, the vcpu number bound to the instance, and the instance's CPU utilization, respectively.

[0016] (node_ip,cpu_num_total,socket_num)

[0017] Where node_ip, cpu_num_total, and socket_num are the machine's IP address, the total number of vCPUs on the machine, and the number of socket slots on the machine, respectively.

[0018] Preferably, based on the average utilization rate of the service As a quantitative indicator of the interference of an example with its neighbors, the expression is:

[0019]

[0020] Where m represents the number of instances of service i; This represents the CPU utilization of each instance k belonging to service i;

[0021] The value is weighted by the number of vCPUs requested by each instance, serving as a quantification of the interference from multiple instances to their neighbors. The expression is:

[0022]

[0023]

[0024] Where K represents the number of shared core SC neighbors; N represents the number of shared slot SS neighbors; requests.vcpu i This indicates the number of vCPUs requested by the instance of service i;

[0025] The CPU utilization prediction model is trained using multiple linear regression, and the expression is:

[0026] CPU inst =b0+k1*CPU SC +k2*CPU ss

[0027] Here, b0, k1, and k2 are the parameters of the linear regression model, which are obtained by fitting historical data.

[0028] Preferably, a sliding window is used to select the candidate set of vCPUs. For each candidate host, the candidate vCPUs are selected from smallest to largest according to their idle vCPU numbers, and a sliding window is used to select the candidate vCPU set, thereby generating a set of candidates.

[0029] Candidates = {(Server i ,vcpus)}

[0030] Among them, Server i Indicates the machine's IP address; vcpus indicates the server's IP address. i A set of vcpu IDs that can be exclusively used by the current instance.

[0031] Preferably, for each (Server) in the candidate set i The interference is evaluated using the tuple (vcpus), which includes an interference score for a single instance and a combined score for multiple instances:

[0032] The interference score for a single instance is calculated using a portion of the CPU utilization prediction model. It selects the portion of utilization increase caused by interference from neighboring applications, expressed as:

[0033] Score inst =k1*CPU SC +k2*CPU Sd

[0034] The comprehensive score for multiple instances, that is, when evaluating interference, not only the increase in CPU utilization of the host itself, but also the increase in the utilization of existing instances on the candidate host, is expressed as:

[0035]

[0036] Among them, requests.vcpu i This indicates the number of vCPUs requested by instance i; Score the interference for instance i; cpu_num socket The number of vCPUs on a single socket.

[0037] The online container scheduling system based on interference scoring provided by the present invention includes:

[0038] Offline training module: Monitors the CPU utilization and deployment status of online service instances during runtime, and creates application profiles and trains CPU utilization prediction models based on monitoring data;

[0039] Online execution module: Based on the CPU utilization prediction model, it performs candidate host filtering, generates candidate sets, and scores interference.

[0040] The candidate host filtering includes filtering out hosts in the cluster that do not meet the resource requirements, and obtaining a batch of candidate hosts.

[0041] The generation of the candidate set includes using a sliding window method to further generate a candidate set of vcpus on the candidate hosts, thereby generating a candidate set of candidate hosts and vcpus set tuples for container scheduling;

[0042] The interference scoring includes scoring individual instances and comprehensive scoring. Before scheduling an instance of an online service, the interference experienced by the instance if it is scheduled to a specified host and vCPU number in each candidate group of the candidate set is evaluated, as well as the interference experienced by the original instances on the candidate hosts. The interference experienced by each instance is aggregated to obtain the comprehensive interference of online instances on the entire CPU. The candidate group with the least comprehensive interference is selected to deploy the scheduled instance.

[0043] Preferably, for a single instance, the data that needs to be monitored includes:

[0044] (instance_id,node_ip,cpu_list,cpu_util)

[0045] Where instance_id, node_ip, cpu_list, and cpu_util are the instance's unique ID, the machine's IP address, the vcpu number bound to the instance, and the instance's CPU utilization, respectively.

[0046] (node_ip,cpu_num_total,socket_num)

[0047] Where node_ip, cpu_num_total, and socket_num are the machine's IP address, the total number of vCPUs on the machine, and the number of socket slots on the machine, respectively.

[0048] Preferably, based on the average utilization rate of the service As a quantitative indicator of the interference of an example with its neighbors, the expression is:

[0049]

[0050] Where m represents the number of instances of service i; This represents the CPU utilization of each instance k belonging to service i;

[0051] The value is weighted by the number of vCPUs requested by each instance, serving as a quantification of the interference from multiple instances to their neighbors. The expression is:

[0052]

[0053]

[0054] Where K represents the number of shared core SC neighbors; N represents the number of shared slot SS neighbors; requests.vcpu i This indicates the number of vCPUs requested by the instance of service i;

[0055] The CPU utilization prediction model is trained using multiple linear regression, and the expression is:

[0056] CPU inst =b0+k1*CPU SC +k2*CPU ss

[0057] Here, b0, k1, and k2 are the parameters of the linear regression model, which are obtained by fitting historical data.

[0058] Preferably, a sliding window is used to select the candidate set of vCPUs. For each candidate host, the candidate vCPUs are selected from smallest to largest according to their idle vCPU numbers, and a sliding window is used to select the candidate vCPU set, thereby generating a set of candidates.

[0059] Candidates = {(Server i ,vcpus)}

[0060] Among them, Server i Indicates the machine's IP address; vcpus indicates the server's IP address. i A set of vcpu IDs that can be exclusively used by the current instance.

[0061] Preferably, for each (Server) in the candidate set i The interference is evaluated using the tuple (vcpus), which includes an interference score for a single instance and a combined score for multiple instances:

[0062] The interference score for a single instance is calculated using a portion of the CPU utilization prediction model. It selects the portion of utilization increase caused by interference from neighboring applications, expressed as:

[0063] Score inst =k1*CPU SC +k2*CPU SS

[0064] The comprehensive score for multiple instances, that is, when evaluating interference, not only the increase in CPU utilization of the host itself, but also the increase in the utilization of existing instances on the candidate host, is expressed as:

[0065]

[0066] Among them, requests.vcpu i This indicates the number of vCPUs requested by instance i; Score the interference for instance i; cpu_num socket The number of vCPUs on a single socket.

[0067] Compared with the prior art, the present invention has the following beneficial effects:

[0068] (1) This invention uses an interference scoring model that considers CPU architecture to evaluate the interference of a single LS instance by its neighboring instances, and further measures the comprehensive interference of all LS instances deployed on this CPU. Based on the interference evaluation results, it guides scheduling and can quickly recommend the candidate host with the lowest interference and the corresponding vcpus set, thereby improving the performance of online applications and reducing their latency.

[0069] (2) The present invention obtains the average utilization of each LS service based on historical monitoring data and trains a CPU prediction model for each LS service. This model can evaluate the amount of interference a single LS instance receives from its neighbors, combine the interference received by multiple online applications, and obtain the overall interference. During scheduling, the candidate host and the corresponding vcpus set that minimize the overall interference are selected, thereby reducing the interference between LS instances.

[0070] (3) This invention proposes an interference scoring model that considers CPU architecture. For an instance, the interference from instances deployed in peer vcpus with shared physical cores and instances deployed in non-peer vcpus with shared sockets can be evaluated separately. When scheduling an LS instance, the interference experienced by the scheduled instance and the interference of the scheduled container on the original instances on the candidate host are considered at the same time. When scheduling, not only are recommended machines given, but also recommended vcpu numbers are given for container core-binding scenarios. Attached Figure Description

[0071] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0072] Figure 1 This is a structural diagram of the present invention;

[0073] Figure 2 A simplified architecture diagram of a CPU with hyper-threading enabled;

[0074] Figure 3 This describes how to select the vcpus collection using a sliding window. Detailed Implementation

[0075] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0076] Example 1:

[0077] This invention provides an online container scheduling method based on interference scoring, comprising:

[0078] Offline training steps: Monitor the CPU utilization and deployment status of online service instances during runtime, and create application profiles and train CPU utilization prediction models based on the monitoring data;

[0079] Online operation steps: Candidate host filtering, candidate set generation, and interference scoring based on CPU utilization prediction model;

[0080] The candidate host filtering includes filtering out hosts in the cluster that do not meet the resource requirements, and obtaining a batch of candidate hosts.

[0081] The generation of the candidate set includes using a sliding window method to further generate a candidate set of vcpus on the candidate hosts, thereby generating a candidate set of candidate hosts and vcpus set tuples for container scheduling;

[0082] The interference scoring includes scoring individual instances and comprehensive scoring. Before scheduling an instance of an online service, the interference experienced by the instance if it is scheduled to a specified host and vCPU number in each candidate group of the candidate set is evaluated, as well as the interference experienced by the original instances on the candidate hosts. The interference experienced by each instance is aggregated to obtain the comprehensive interference of online instances on the entire CPU. The candidate group with the least comprehensive interference is selected to deploy the scheduled instance.

[0083] For a single instance, the data that needs to be monitored includes:

[0084] (instance_id,node_ip,cpu_list,cpu_util)

[0085] Where instance_id, node_ip, cpu_list, and cpu_util are the instance's unique ID, the machine's IP address, the vcpu number bound to the instance, and the instance's CPU utilization, respectively.

[0086] (node_ip,cpu_num_total,socket_num)

[0087] Where node_ip, cpu_num_total, and socket_num are the machine's IP address, the total number of vCPUs on the machine, and the number of socket slots on the machine, respectively.

[0088] Based on the average utilization rate of the service As a quantitative indicator of the interference of an example with its neighbors, the expression is:

[0089]

[0090] Where m represents the number of instances of service i; This represents the CPU utilization of each instance k belonging to service i;

[0091] The value is weighted by the number of vCPUs requested by each instance, serving as a quantification of the interference from multiple instances to their neighbors. The expression is:

[0092]

[0093]

[0094] Where K represents the number of shared core SC neighbors; N represents the number of shared slot SS neighbors; requests.vcpu i This indicates the number of vCPUs requested by the instance of service i;

[0095] The CPU utilization prediction model is trained using multiple linear regression, and the expression is:

[0096] CPU inst =b0+k1*CPU SC +k2*CPU ss

[0097] Here, b0, k1, and k2 are the parameters of the linear regression model, which are obtained by fitting historical data.

[0098] A sliding window is used to select a set of candidate VCPUs. For each candidate host, the candidate VCPUs are selected from smallest to largest using a sliding window, thus generating a set of candidate VCPUs.

[0099] Candidates = {(Server i ,vcpus)}

[0100] Among them, Server i Indicates the machine's IP address; vcpus indicates the server's IP address. i A set of vcpu IDs that can be exclusively used by the current instance.

[0101] For each (Server) in the candidate set i The interference is evaluated using the tuple (vcpus), which includes an interference score for a single instance and a combined score for multiple instances:

[0102] The interference score for a single instance is calculated using a portion of the CPU utilization prediction model. It selects the portion of utilization increase caused by interference from neighboring applications, expressed as:

[0103] Score inst =k1*CPU SC +k2*CPU Sd

[0104] The comprehensive score for multiple instances, that is, when evaluating interference, not only the increase in CPU utilization of the host itself, but also the increase in the utilization of existing instances on the candidate host, is expressed as:

[0105]

[0106] Among them, requests.vcpu i This indicates the number of vCPUs requested by instance i; Score the interference for instance i; cpu_num socket The number of vCPUs on a single socket.

[0107] Example 2:

[0108] like Figure 1 As shown, this invention provides an online container scheduling system based on interference scoring, including an online execution part and an offline training part. The offline part extracts the average utilization of the LS service based on historical monitoring data and trains a CPU utilization predictor. The online part is the scheduling process. First, the online container at the head of the scheduling queue is selected. Based on its metadata, such as the required number of cores, a candidate host filter will filter out candidate hosts that meet the requirements. Then, a sliding window method is used to further select a set of vcpus that meet the conditions from the filtered candidate hosts, resulting in a candidate set of a binary pair of candidate hosts and candidate vcpus sets. Based on the trained predictor and combined with the interference scorer evaluation, the interference of each candidate host and its corresponding vcpus set in the candidate set is evaluated. The host with the smallest interference score and its corresponding vcpus set are then selected to bind the container.

[0109] The offline training component includes:

[0110] (1) Monitoring module

[0111] We need monitoring data to train our model. Analysis of historical data revealed that, with the same request-per-second (RPS), the CPU utilization of different instances of the same LS service can vary by up to 20%. This difference is due to interference, and with the same RPS, higher CPU utilization leads to higher response latency. Therefore, our goal is to minimize instance CPU utilization as much as possible. Since we need to characterize the interference within the cluster, we also need to know which machine each container is deployed on and which cores it is bound to, so we can identify each container's SC and SS neighbors. Therefore, from the container's perspective, the monitoring data mainly includes:

[0112] (instance_id,node_ip,cpu_list,cpu_util)

[0113] Where `instance_id` is the unique identifier of the LS instance; `node_ip` is the IP address of the machine where the LS instance is deployed; `cpu_list` is a list storing the vCPU numbers exclusively used by the LS instance; and `cpu_util` is the CPU utilization of the LS instance. From the machine's perspective, the monitoring data mainly includes:

[0114] (node_ip,cpu_num_total,socket_num)

[0115] We need to know the total number of vCPUs on the machine and the number of sockets (socket_num). Combining these two numbers with the numbering rules of the Linux system, we can calculate the ID of the vCPU sharing a physical core with a given vCPU. We can also determine the ID of the vCPUs within the same socket. The calculation method is as follows:

[0116] Number of vCPUs within a single socket:

[0117]

[0118] The vCPU number (starting from 0) of the i-th slot is:

[0119] in, CPU number and The CPUs share a physical core, and the others are similar. In this way, we can identify the instances that share a physical core with each LS instance and the instances that share a socket.

[0120] (2) Application profile

[0121] To train the CPU utilization predictor, we need not only instances in the peer vcpus sharing the physical core of each instance, but also instances sharing socket resources. We also need some features to represent instances of each application. Figure 2 This describes the application deployment on a CPU with Hyper-Threading enabled. There are five instances: A, B, C, D, E, and F. Instance A shares physical cores with instances B and C, running on virtual cores within the same set of physical core virtualizations. Therefore, we call instances B and C A's SC (sharing core) neighbors. While instances A, D, E, and F do not share physical cores, they share the entire CPU's LLC (lowest level cache). Therefore, we call instances D, E, and F A's SS (sharing socket) neighbors. Here, instance A is affected by interference from instances B, C, D, E, and F. To predict the CPU utilization of instance A, we need to represent the pressure exerted on instance A by other instances. Here, we use the average utilization of the service to which each instance belongs to represent the pressure exerted on that service:

[0122]

[0123] In the above formula, m represents the number of instances of service i; This represents the CPU utilization of each instance belonging to service i. Figure 2 The text indicates that in actual deployments, there is usually more than one SC neighbor and one SS neighbor. Therefore, we need to aggregate them to represent the collective pressure on instance A.

[0124]

[0125]

[0126] Where K represents the number of SC neighbors; N represents the number of SS neighbors; requests.vcpu i This indicates the number of vCPUs requested by instance i.

[0127] (3) Training the CPU utilization predictor

[0128] Within the cluster, due to load balancing, different instances of the same LS service have the same RPS (requests per second). SC neighbors share the L1 / L2 cache with the target instance, while SS neighbors share the LLC with the target instance. Therefore, the CPU utilization of the target instance is mainly affected by SC neighbors and SS neighbors. We use multiple linear regression to predict the CPU utilization of the target instance. The pressure from SC neighbors and the pressure from SS neighbors described in the application profile are as follows:

[0129] CPU inst =b0+k1*CPU SC +k2*CPU ss

[0130] The online operation component includes:

[0131] (1) Generate candidate set

[0132] When selecting a candidate host, two conditions must generally be met. First, it must meet certain operational constraints, such as disaster recovery requirements: different instances of the same service cannot be deployed on the same machine. Second, it must meet resource requirements, that is, the number of remaining idle vCPUs on the candidate host must be greater than or equal to the number of vCPUs required by the instance currently waiting to be scheduled.

[0133] Unlike typical schedulers, the candidate set here includes not only the host but also which vCPUs to select on that host. Since a single socket contains many vCPUs—for example, our cluster uses a processor with 52 vCPUs—if a newly arriving container waiting to be scheduled requests 8 vCPUs, the possible combinations of vCPUs could be as high as [number missing]. There are many types. This number would result in a huge evaluation overhead. Therefore, we used a sliding window to determine the candidate set. Figure 3As shown, the idle vCPUs in the CPU are first sorted by their numbers in ascending order. The advantage of this is that one vCPU on a physical core is selected first, leaving the other vCPU on that core available for other applications. This allows two different applications to use the two vCPUs on a single physical core, thus allocating CPU-intensive and non-CPU-intensive applications to the same physical core. This avoids two scenarios: high-utilization instances occupying both ends of a physical core, violating the SLA; or low-utilization instances occupying both ends of a physical core, wasting resources. A sliding window is then used to select candidate vCPUs. The sliding window has two parameters: the window size is determined by the number of vCPUs requested by the applications, and the step size is usually determined by the smallest granularity of vCPU allocation in the cluster. For example, in our cluster, most applications request two or more vCPUs, so we set the step size to 2. Using the sliding window, we can select a candidate set:

[0134] Candidates = {(Server i ,vcpus)}

[0135] Among them, Server i The number of idle vCPUs on the server must be greater than or equal to the number of vCPUs requested by the current application. vcpus is a set of vCPUs selected using the sliding window method described above. Since cross-socket deployments degrade instance performance and cause our prediction model to fail, we must ensure that instances are not deployed across sockets.

[0136] (2) For each candidate in the candidate set (Server) i (vcpus) rating

[0137] We evaluate the interference scores of all candidates with the aim of selecting the best candidate to run the currently scheduled instance. This achieves the goal of mitigating interference through scheduling.

[0138] First, for a single instance, based on the aforementioned CPU utilization prediction formula, we can see that the instance's CPU utilization is mainly affected by SC neighbors and SS neighbors. We consider this impact as interference experienced by a single instance.

[0139]

[0140] Among them, the two coefficients k1 and k2 are learned from historical data in the aforementioned multiple linear regression model for CPU utilization prediction. And CPU SC and CPU SSThis represents the pressure exerted on the target instance by SC and SS neighbors. The value is obtained by aggregating the application profiles of the SC and SS neighbors respectively. Physically, this score represents the difference in CPU utilization between the instance under interference and without interference. A larger value indicates greater interference.

[0141] For a target instance, once it is scheduled onto some vCPUs of a host, it will not only be affected by existing instances on that machine, but will also interfere with existing instances. Therefore, it is insufficient to consider only the interference of the target instance; we need to evaluate the interference of all LS instances on the sockets where the candidate vCPU sets of candidate machines reside:

[0142]

[0143] The number of vcpus requested per instance: requests.vcpu i As weights, a weighted sum is performed. The physical meaning of this socket-level interference score can be considered as the increase in CPU utilization of a single socket due to interference. Therefore, the score... socket A higher interference score indicates a greater increase in CPU utilization due to interference, suggesting potentially greater interference between LS instances within that socket. Data analysis, and based on the CPU's microarchitecture, shows minimal instance interference between the two sockets. Therefore, we ignore the mutual interference between this socket and another socket. Using only the interference score of a single socket as the scheduling criterion, after calculating the interference scores of all candidates in the candidate set, we select the candidate with the least interference to run the target instance.

[0144] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0145] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. An online container scheduling method based on interference scoring, characterized in that, include: Offline training steps: Monitor the CPU utilization and deployment status of online service instances during runtime, and build instance profiles and train CPU utilization prediction models based on monitoring data; Online operation steps: Candidate host filtering, candidate set generation, and interference scoring based on CPU utilization prediction model; The candidate host filtering includes filtering out hosts in the cluster that do not meet the resource requirements, and obtaining a batch of candidate hosts. The generation of the candidate set includes using a sliding window method to further generate a candidate set of vcpus on the candidate hosts, thereby generating a candidate set of candidate hosts and vcpus set tuples for container scheduling; The interference scoring includes scoring individual instances and comprehensive scoring. Before scheduling an instance of an online service, the interference experienced by the instance if it is scheduled to a specified host and vCPU number in each candidate group of the candidate set is evaluated, as well as the interference experienced by the original instances on the candidate hosts. The interference experienced by each instance is aggregated to obtain the comprehensive interference of online instances on the entire CPU, and the candidate group with the least comprehensive interference is selected to deploy the scheduled instance. Based on the average utilization rate of the service As a quantitative indicator of the interference of an example with its neighbors, the expression is: Where m represents The number of instances; Indicates belonging to CPU utilization for each instance k; The value is weighted by the number of vCPUs requested by each instance, serving as a quantification of the interference from multiple instances to their neighbors. The expression is: Where K represents the number of shared core SC neighbors; N represents the number of shared slot SS neighbors; This indicates the number of vCPUs requested by the instance of service i; The CPU utilization prediction model is trained using multiple linear regression, and the expression is: in, , and These are the parameters of a linear regression model, obtained by fitting historical data.

2. The online container scheduling method based on interference scoring according to claim 1, characterized in that, For a single instance, the data that needs to be monitored includes: in, These are the instance's unique ID, the machine's IP address, the vCPU ID bound to the instance, and the instance's CPU utilization, respectively. in, These are the machine's IP address, the total number of vCPUs on the machine, and the number of sockets on the machine, respectively.

3. The online container scheduling method based on interference scoring according to claim 1, characterized in that, A sliding window is used to select a set of candidate VCPUs. For each candidate host, the candidate VCPUs are selected from smallest to largest using a sliding window, thus generating a set of candidate VCPUs. in, Indicates the machine's IP address; v Indicates in A set of vcpu IDs that can be exclusively used by the current instance.

4. The online container scheduling method based on interference scoring according to claim 3, characterized in that, For each candidate in the candidate set The tuple is used for interference evaluation, which includes interference scores for individual instances and combined scores for multiple instances: The interference score for a single instance is calculated using a portion of the CPU utilization prediction model. It selects the portion of utilization increase caused by interference from neighboring applications, expressed as: The comprehensive score for multiple instances, that is, when evaluating interference, not only the increase in CPU utilization of the host itself, but also the increase in the utilization of existing instances on the candidate host, is expressed as: in, This indicates the number of vCPUs requested by instance i; Score the interference for instance i; The number of vCPUs on a single socket.

5. An online container scheduling system based on interference scoring, characterized in that, include: Offline training module: Monitors the CPU utilization and deployment status of online service instances during runtime, and builds instance profiles and trains CPU utilization prediction models based on monitoring data; Online execution module: Based on the CPU utilization prediction model, it performs candidate host filtering, generates candidate sets, and scores interference. The candidate host filtering includes filtering out hosts in the cluster that do not meet the resource requirements, and obtaining a batch of candidate hosts. The generation of the candidate set includes using a sliding window method to further generate a candidate set of vcpus on the candidate hosts, thereby generating a candidate set of candidate hosts and vcpus set tuples for container scheduling; The interference scoring includes scoring individual instances and comprehensive scoring. Before scheduling an instance of an online service, the interference experienced by the instance if it is scheduled to a specified host and vCPU number in each candidate group of the candidate set is evaluated, as well as the interference experienced by the original instances on the candidate hosts. The interference experienced by each instance is aggregated to obtain the comprehensive interference of online instances on the entire CPU, and the candidate group with the least comprehensive interference is selected to deploy the scheduled instance. Based on the average utilization rate of the service As a quantitative indicator of the interference of an example with its neighbors, the expression is: Where m represents The number of instances; Indicates belonging to CPU utilization for each instance k; The value is weighted by the number of vCPUs requested by each instance, serving as a quantification of the interference from multiple instances to their neighbors. The expression is: Where K represents the number of shared core SC neighbors; N represents the number of shared slot SS neighbors; This indicates the number of vCPUs requested by the instance of service i; The CPU utilization prediction model is trained using multiple linear regression, and the expression is: in, , and These are the parameters of a linear regression model, obtained by fitting historical data.

6. The online container scheduling system based on interference scoring according to claim 5, characterized in that, For a single instance, the data that needs to be monitored includes: in, These are the instance's unique ID, the machine's IP address, the vCPU ID bound to the instance, and the instance's CPU utilization, respectively. in, These are the machine's IP address, the total number of vCPUs on the machine, and the number of sockets on the machine, respectively.

7. The online container scheduling system based on interference scoring according to claim 5, characterized in that, A sliding window is used to select a set of candidate VCPUs. For each candidate host, the candidate VCPUs are selected from smallest to largest using a sliding window, thus generating a set of candidate VCPUs. in, Indicates the machine's IP address; v Indicates in A set of vcpu IDs that can be exclusively used by the current instance.

8. The online container scheduling system based on interference scoring according to claim 7, characterized in that, For each candidate in the candidate set The tuple is used for interference evaluation, which includes interference scores for individual instances and combined scores for multiple instances: The interference score for a single instance is calculated using a portion of the CPU utilization prediction model. It selects the portion of utilization increase caused by interference from neighboring applications, expressed as: The comprehensive score for multiple instances, that is, when evaluating interference, not only the increase in CPU utilization of the host itself, but also the increase in the utilization of existing instances on the candidate host, is expressed as: in, This indicates the number of vCPUs requested by instance i; Score the interference for instance i; The number of vCPUs on a single socket.

Images