Method, device, electronic equipment, storage medium and program product for capacity adjustment of computing cluster

Abstract

The application provides a capacity adjustment method and device of a computing cluster, electronic equipment, a storage medium and a program product; the computing cluster comprises a plurality of computing nodes; the method comprises: in response to an exception of the computing cluster, performing at least one round of capacity expansion on the computing cluster, wherein in each round of capacity expansion, the following processing is performed: determining a capacity expansion threshold of the computing cluster in the current round based on current load data of the computing cluster, wherein the capacity expansion threshold is the maximum number of computing nodes included in the computing cluster after the capacity expansion in the current round; determining a target capacity expansion number of the computing cluster based on the capacity expansion threshold and a preset capacity expansion ratio for the computing cluster, wherein the target capacity expansion number is the number of computing nodes actually included in the computing cluster after the capacity expansion in the current round; and expanding the capacity of the computing cluster according to the target capacity expansion number. Through the application, the stable operation of the business can be ensured.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, electronic device, storage medium, and program product for adjusting the capacity of a computing cluster. Background Technology

[0002] Against the backdrop of rapid development in information technology, and with the dynamic changes in business needs and the continuous expansion of application scenarios, computing clusters often face the problem of resource shortage or surplus.

[0003] In related technologies, computing clusters can automatically expand or shrink during peak or off-peak periods to optimize resource utilization. However, when a computing cluster malfunctions, the above-mentioned automatic expansion or shrinkage scheme fails, and the computing cluster may be unable to support normal business operations. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, storage medium, and program product for adjusting the capacity of a computing cluster, which can ensure the stable operation of services.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a method for adjusting the capacity of a computing cluster, the computing cluster including multiple computing nodes; the method includes:

[0007] In response to an anomaly in the computing cluster, at least one round of scaling up is performed on the computing cluster, wherein the following processing is performed in each round of scaling up:

[0008] Based on the current load data of the computing cluster, the expansion threshold of the computing cluster in the current round is determined, wherein the expansion threshold is the maximum number of computing nodes included in the computing cluster after the expansion in the current round;

[0009] Based on the expansion threshold and the preset expansion ratio for the computing cluster, the target expansion quantity of the computing cluster is determined, wherein the target expansion quantity is the actual number of computing nodes included in the computing cluster after the current round of expansion.

[0010] The computing cluster is expanded according to the target expansion amount.

[0011] In the above scheme, the computing cluster is used to carry the target service; before performing at least one round of capacity expansion for the computing cluster in response to an anomaly, the method further includes:

[0012] Obtain the fourth load data of the computing cluster and the second indicator data of the target service. If the fourth load data is greater than the preset second load threshold, determine that the computing cluster is abnormal. The fourth load data is the load data of the computing cluster before it performs the at least one round of expansion.

[0013] If it is determined that the computing cluster needs to be expanded based on the fourth load data and the second indicator data, a second expansion quantity is determined based on the fourth load data and a preset first load threshold.

[0014] Multiple preset ratios are sequentially determined and multiplied by the number of fourth nodes to obtain the number of nodes to be added corresponding to each preset ratio. The number of fourth nodes is the number of computing nodes included in the computing cluster before the first round of expansion. The multiple preset ratios increase exponentially.

[0015] The number of nodes added for each preset ratio is summed with the number of fourth nodes to obtain the fourth expansion number for each preset ratio;

[0016] The third indicator data of the target service is predicted after the computing cluster is expanded according to the fourth expansion quantity.

[0017] From the plurality of preset ratios, select the first target preset ratio whose third indicator data meets the range of the indicator data, and determine the fourth expansion quantity corresponding to the target preset ratio as the fifth expansion quantity;

[0018] The maximum value between the second expansion quantity and the fifth expansion quantity is determined as the first expansion quantity;

[0019] If the first expansion quantity is greater than the capacity threshold of the computing cluster, the operation of determining the first expansion quantity based on the fourth load data and the second indicator data is repeated until the number of times the operation is repeated is greater than the number of times the computing cluster is determined to be abnormal.

[0020] If the second indicator data is not obtained, it is determined that the computing cluster is malfunctioning.

[0021] This application provides a capacity adjustment device for a computing cluster, the computing cluster including multiple computing nodes. The device is used to perform at least one round of capacity expansion for the computing cluster in response to an anomaly, wherein each round of capacity expansion includes:

[0022] The expansion threshold determination module is used to determine the expansion threshold of the computing cluster in the current round based on the current load data of the computing cluster, wherein the expansion threshold is the maximum number of computing nodes included in the computing cluster after the expansion in the current round;

[0023] The expansion quantity determination module is used to determine the target expansion quantity of the computing cluster based on the expansion threshold and the preset expansion ratio for the computing cluster, wherein the target expansion quantity is the actual number of computing nodes included in the computing cluster after the current round of expansion.

[0024] The expansion module is used to expand the computing cluster according to the target expansion amount.

[0025] This application provides an electronic device, the electronic device comprising:

[0026] Memory is used to store executable instructions or computer programs.

[0027] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the capacity adjustment method for the computing cluster provided in the embodiments of this application.

[0028] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the capacity adjustment method for the computing cluster provided in this application.

[0029] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the capacity adjustment method for the computing cluster provided in this application.

[0030] The embodiments of this application have the following beneficial effects:

[0031] In the event of an anomaly in the computing cluster, a scaling threshold is determined based on the cluster's load data. The target scaling quantity is then calculated based on this threshold and a pre-defined scaling ratio for the cluster. This target scaling quantity represents the actual number of computing nodes included after scaling. The cluster is then scaled up according to this target quantity. This approach ensures an immediate response and initiation of scaling operations upon anomaly detection, guaranteeing a sufficient number of computing nodes to handle anomalies and thus ensuring stable business operations. The use of a multi-stage, gradual scaling approach—dynamically adjusting the scaling threshold based on load data in each round—rather than a one-time large-scale scaling operation, helps alleviate peak pressure on computing nodes and ensures the overall stability and reliability of the computing cluster. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the architecture of the computing cluster capacity adjustment system provided in the embodiments of this application;

[0033] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0034] Figure 3 This is a flowchart illustrating the capacity adjustment method for a computing cluster provided in an embodiment of this application. Figure 1 ;

[0035] Figure 4 This is a flowchart illustrating the capacity adjustment method for a computing cluster provided in an embodiment of this application. Figure 2 ;

[0036] Figure 5 This is a flowchart illustrating the capacity adjustment method for a computing cluster provided in an embodiment of this application. Figure 3 ;

[0037] Figure 6 This is a flowchart illustrating the capacity adjustment method for a computing cluster provided in an embodiment of this application. Figure 4 ;

[0038] Figure 7 This is a flowchart illustrating the capacity adjustment method for a computing cluster provided in an embodiment of this application. Figure 5 ;

[0039] Figure 8 This is a flowchart illustrating the capacity adjustment method for a computing cluster provided in an embodiment of this application. Figure 6 ;

[0040] Figure 9 This is a flowchart illustrating the fallback defense strategy provided in the embodiments of this application;

[0041] Figure 10 This is a schematic diagram of the heterogeneous computing system provided in the embodiments of this application;

[0042] Figure 11 This is a schematic diagram of the remedial capacity expansion strategy provided in the embodiments of this application;

[0043] Figure 12 This is a schematic diagram illustrating the principle of the capacity adjustment method for a computing cluster provided in the embodiments of this application;

[0044] Figure 13 This is a schematic diagram illustrating the configuration of thresholds and percentages provided in an embodiment of this application;

[0045] Figure 14 This is a flowchart illustrating the capacity adjustment method for a computing cluster provided in an embodiment of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0047] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0048] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0049] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0050] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0051] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0052] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0053] 1) Responding to: used to indicate the conditions or states on which the operation is performed depends. When the conditions or states on which it depends are met, one or more operations can be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0054] 2) Heterogeneous computing power: This refers to the ability to use different types of computing nodes to collaboratively complete computing tasks. Different types of computing units can include central processing units (CPUs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), etc.

[0055] 3) Replica: refers to the copying of a specific compute node, service instance, or data within a compute cluster.

[0056] 4) Heterogeneous containers: These refer to different types of containerized environments used in a computing cluster. These heterogeneous containers may be based on different operating systems, runtime environments, or architectures. The existence of heterogeneous containers enables the computing cluster to support a variety of different types of applications and services. This application does not limit the specific type of heterogeneous container; heterogeneous containers can be Docker containers, Windows containers, etc.

[0057] 5) Heterogeneous devices: These refer to different hardware devices included in a computing cluster. These heterogeneous devices may have different processing capabilities, memory capacities, network bandwidths, or other hardware characteristics. The existence of heterogeneous devices enables the computing cluster to operate in diverse hardware environments, adapting to different application scenarios and requirements. This application does not limit the specific type of heterogeneous device; heterogeneous devices can be CPUs, GPUs, FPGAs, etc.

[0058] 6) Computing Cluster: A computing cluster is a cluster of multiple computing nodes used to support business operations. For example, the business can be an inference service for a large model, which runs on the computing cluster. This application does not limit the specific architecture of the computing cluster; it can be a virtual cluster based on a cloud platform, a physical server cluster, etc.

[0059] 7) Compute Node: A compute node is an independent computing unit in a compute cluster, responsible for executing computing tasks to enable business operations. This application embodiment does not limit the specific hardware configuration of the compute node; the compute node can be a heterogeneous container or heterogeneous device, such as a server, virtual machine, or other hardware device.

[0060] 8) Expansion: This refers to increasing the number of computing nodes in a computing cluster to improve the overall computing and processing capabilities of the cluster. This application does not limit the specific method of expansion; expansion can be achieved by adding new physical servers or starting new virtual machines.

[0061] 9) Downsizing: This refers to reducing the number of computing nodes in a computing cluster to optimize resource utilization and reduce costs. This application does not limit the specific method of downsizing; it can be achieved by reducing the number of physical servers or shutting down virtual machines.

[0062] 10) Large models: These refer to machine learning or deep learning models with a large number of parameters and complex structures. Large models typically have stronger representation capabilities and higher accuracy, and are suitable for handling complex tasks such as natural language processing and computer vision.

[0063] 11) Artificial Intelligence Models: These are mathematical models trained using machine learning or deep learning algorithms, used to simulate and perform specific tasks. Artificial intelligence models can learn patterns from data and make predictions or decisions when encountering new data. They are widely used in natural language processing, computer vision, speech recognition, recommender systems, and other fields.

[0064] 12) Inference services: This refers to deploying trained models to a production environment to perform real-time or batch prediction, classification, regression, and other operations on new data. For example, inference services can be used for text classification, language translation, sentiment analysis, question answering, object detection, personalized recommendations, etc.

[0065] With the increasing scale of heterogeneous computing power and the high cost of stacking heterogeneous computing power, ensuring high stability of business services while reducing costs has become a critical issue. Especially for heterogeneous online inference services, if scaling down fails due to faulty load data or business metrics, preventing normal scaling up, it will affect online inference services and lead to low stability of the heterogeneous cluster supporting the business.

[0066] For scenarios where abnormal data is detected and the number of replicas cannot be increased during horizontal scaling of heterogeneous inference services, related technologies typically involve building detection alerts and sending them to operations and maintenance (O&M) personnel via telephone. O&M personnel then manually handle the anomalies. However, this traditional model is not automated from the discovery and issuance of alerts to the O&M personnel's response. Furthermore, delays can easily occur in the connecting links, further lengthening the timeliness of anomaly handling and impacting the service quality of heterogeneous inference. Moreover, the traditional detection and alert-driven anomaly handling method is unfriendly to O&M personnel with heterogeneous computing power, requiring them to respond to and handle anomalies constantly. This increases the manpower investment cost for heterogeneous O&M and the pressure on personnel following up on anomalies, failing to achieve the goal of freeing up manpower and improving resource utilization efficiency.

[0067] To address the aforementioned problems, embodiments of this application provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for adjusting the capacity of a computing cluster, which can improve the stability of the computing cluster. The exemplary application of the electronic device provided in this application embodiment is described below; the electronic device provided in this application embodiment can be implemented as a server. The exemplary application of the electronic device as a server will be described below.

[0068] The capacity adjustment method for computing clusters provided in this application can be applied to any scenario requiring adaptive capacity adjustment of the computing cluster, thereby improving the stability and reliability of the computing cluster. Specific application scenarios may include:

[0069] 1) Cloud Services: Cloud service providers dynamically adjust the size of computing clusters based on user demand and load conditions using automated capacity scaling methods. Servers can monitor the load data of the computing clusters supporting cloud services in real time. When an anomaly occurs in the computing cluster, user demand cannot be determined, and the original automated capacity scaling method fails. In this case, the server can scale up the computing cluster based on load data and a preset scaling ratio to ensure the normal operation of cloud services.

[0070] 2) E-commerce Platform: During promotional activities, e-commerce platforms may encounter malicious traffic attacks, causing a surge in computing cluster load and rendering conventional expansion ineffective. Servers can quickly identify abnormal traffic patterns through real-time monitoring data, determining that the computing cluster is malfunctioning. An emergency expansion process can then be initiated immediately, executing multiple rounds of expansion based on the computing cluster load data and a pre-defined expansion ratio. This rapidly adds a large number of computing nodes, ensuring the normal operation of the e-commerce platform's core transaction system.

[0071] 3) Video streaming platform: The computing cluster carries the live streaming service. If a sudden technical problem occurs during the live stream, causing the computing cluster to malfunction and unable to automatically scale up or down based on the live stream traffic, the server can initiate an emergency scaling process. By using the computing cluster's load data and a preset scaling ratio, multiple rounds of scaling are performed to quickly add a large number of computing nodes, ensuring the stable transmission of the live video stream and reducing latency and stuttering.

[0072] 4) Game Operation Scenario: In large-scale multiplayer online games or cloud games, computing nodes in the computing cluster can include heterogeneous computing resources such as CPUs, GPUs, and FPGAs to enable efficient inference services for the game's artificial intelligence models, such as inferring the behavior of non-player characters (NPCs). When the computing cluster experiences an anomaly, the server can perform multiple rounds of expansion on the computing cluster based on the cluster's load data and a preset expansion ratio, quickly adding a large number of computing nodes to ensure the normal operation of the efficient inference service of the artificial intelligence model and improve the smoothness of NPC-player interactions in the game.

[0073] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of a computing cluster capacity adjustment system provided in this application embodiment. To support the capacity adjustment application of a computing cluster, the computing cluster capacity adjustment system 100 includes at least a computing cluster 500 (which includes multiple computing nodes 500-1), a terminal 400, a network 300, and a server 200. The terminal 400 is connected to the server 200 through the network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of both.

[0074] See Figure 1Users can trigger the target service through the client on terminal 400. The client sends a service request for the target service to computing cluster 500. Computing cluster 500 responds to the service request, runs the target service, and begins operation. Server 200 obtains the load data of computing cluster 500 and the indicator data of the target service, and based on the load data and indicator data, determines whether computing cluster 500 needs to be expanded or reduced. If expansion is needed, the number of computing nodes 500-1 is increased; if reduction is needed, the number of computing nodes 500-1 is decreased. When the computing cluster experiences an anomaly, the server... In response to an anomaly in the computing cluster, server 200 performs at least one round of scaling up the computing cluster. In each round of scaling up, the following processes are performed: Server 200 determines the scaling up threshold for the computing cluster in the current round based on the current load data of the computing cluster. The scaling up threshold is the maximum number of computing nodes included in the computing cluster after the current round of scaling up. Server 200 determines the target scaling up number of the computing cluster based on the scaling up threshold and a preset scaling up ratio for the computing cluster. The target scaling up number is the actual number of computing nodes included in the computing cluster after the current round of scaling up. Server 200 scales up the computing cluster according to the target scaling up number.

[0075] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment.

[0076] The electronic device that implements the capacity adjustment method for the computing cluster provided in the embodiments of this application may be Figure 1 Server 200. See also Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Figure 2 The illustrated electronic device includes at least one processor 210, a memory 230, and at least one network interface 220. The various components of the electronic device are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 2 The general labeled all buses as Bus System 240.

[0077] The processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0078] The memory 230 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 230 may optionally include one or more storage devices physically located away from the processor 210.

[0079] The memory 230 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 230 described in this application embodiment is intended to include any suitable type of memory.

[0080] In some embodiments, memory 230 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0081] The operating system 231 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, for implementing various basic business functions and handling hardware-based tasks.

[0082] The network communication module 232 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 including Bluetooth, WiFi, and Universal Serial Bus (USB).

[0083] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A capacity adjustment device 255 for a computing cluster stored in memory 250 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a first determination module 2551, a second determination module 2552, and a capacity expansion module 2553. These modules are logically linked and can therefore be arbitrarily combined or further split according to their implemented functions. The functions of each module will be described below.

[0084] In other embodiments, the apparatus provided in this application can be implemented in hardware. As an example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the capacity adjustment method of the computing cluster provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0085] The following describes the capacity adjustment method for a computing cluster provided in the embodiments of this application. As mentioned above, the electronic device implementing the capacity adjustment method for a computing cluster in the embodiments of this application can be a terminal, a server, or a combination of both. Therefore, the executing entity of each step will not be described again below.

[0086] The capacity adjustment method for computing clusters provided in this application will be described in conjunction with exemplary applications and implementations of the servers provided in the embodiments of this application.

[0087] See Figure 3 , Figure 3 This is a flowchart illustrating the capacity adjustment method for a computing cluster provided in this application embodiment, which will be combined with... Figure 3 The steps shown are explained as follows: Figure 3 As shown, taking the server as the execution subject of the computing cluster capacity adjustment method as an example, in response to the computing cluster encountering an anomaly, at least one round of expansion is performed on the computing cluster, wherein the following steps 101 to 103 are performed in each round of expansion.

[0088] In step 101, the scaling threshold for the computing cluster in the current round is determined based on the current load data of the computing cluster.

[0089] The expansion threshold is the maximum number of computing nodes included in the computing cluster after the current round of expansion.

[0090] Here, the computing cluster includes multiple computing nodes used to support the target service. This application embodiment does not limit the specific hardware configuration of the computing nodes; the multiple computing nodes may include servers, virtual machines, and other hardware devices. The target service refers to the specific application or service that the computing cluster needs to process, such as big data processing, machine learning model training, web services, and large model inference services. The computing nodes run an image containing the processing logic of the target service. The image is a containerized application package containing the target service processing logic. The image encapsulates all the code, dependency libraries, and configuration files required by the target service, ensuring that each computing node can independently and consistently execute the target service processing logic. The computing cluster load data can be various performance data generated during the operation of the computing cluster, used to reflect the current workload of the computing cluster. The computing cluster load data includes the load data of each computing node in the computing cluster, including but not limited to CPU utilization, memory usage, and network bandwidth utilization.

[0091] Users can send service requests for a target service to the computing cluster through the client interface of the terminal or other interactive methods. After receiving the service request, the computing cluster distributes the service request to various computing nodes. After receiving the service request, the computing nodes can run the image containing the processing logic of the target service, obtain the running result, and return the running result to the terminal. In this embodiment, the target service is the inference service of a large model as an example for specific explanation. The large model is a language understanding model used for inference services for natural language processing tasks. Users can input a text through the client interface and request the language understanding model to generate an answer; the inference service is based on text-based answer generation. After receiving the service request for the target service, the computing cluster distributes the service request to various computing nodes. The computing nodes load the inference service image of the language understanding model, run the inference service image, generate an answer, and return it to the terminal.

[0092] In the above process, the computing cluster can automatically expand or shrink according to a preset capacity scaling process, optimizing computing resource allocation while ensuring the performance and stability of the target service. Expanding the computing cluster means increasing the number of computing nodes included in the cluster, while shrinking the cluster means reducing the number of computing nodes. However, when the computing cluster malfunctions, the preset capacity scaling process cannot be executed. In this case, to address potential performance degradation or operational interruptions due to insufficient computing nodes, it is necessary to quickly expand the computing cluster to the maximum capacity to support stable operation of the target service. However, directly expanding to the maximum capacity at once may cause the computing cluster to crash. Therefore, the expansion process is divided into multiple executions, with at least one round of expansion performed on the computing cluster. The preset capacity scaling process can monitor the load data of the computing cluster and the target service's metrics in real time, and expand or shrink the computing cluster based on the load data and metrics data. Metric data is used to characterize the operation of the target service, including but not limited to queue length, latency, response time, and success rate. An anomaly in the computing cluster refers to an abnormality in the real-time collection of load data and target business indicator data of the computing cluster, which causes the preset capacity scaling process to fail to execute normally.

[0093] In some embodiments, in response to an anomaly in the computing cluster, at least one round of scaling up can be performed on the computing cluster. In each round of scaling up, firstly, the current load data of the computing cluster is collected, and based on the current load data, the load data of the computing cluster in the previous round of scaling up, and the scaling up threshold of the computing cluster in the previous round, the scaling up threshold for the computing cluster in the current round is determined. It should be noted that in the first round of scaling up, a preset scaling up threshold can be directly determined as the scaling up threshold for the first round. The preset scaling up threshold can be set by the user based on historical scaling up experience of the computing cluster. This application embodiment does not limit the period between every two rounds and can be set according to actual needs. The period refers to the time interval between two rounds of scaling up.

[0094] For example, if the cycle is 5 minutes, then a round of scaling will be performed every 5 minutes. Exemplarily, the cycle can be adaptively adjusted based on the load jitter parameter, which is the rate at which load data changes over time, i.e., the slope of the load-time curve. A larger load jitter parameter indicates a faster rate of change in load data, meaning the load on the computing cluster changes more drastically and requires a shorter cycle; a smaller rate of change indicates a smoother load change on the computing cluster and requires a longer cycle.

[0095] In some embodiments, the load data includes multiple types of sub-load data. See also Figure 4In step 101, the expansion threshold of the computing cluster in the current round is determined based on the current load data of the computing cluster. This can be achieved through the following steps 1011 to 1014, which are explained in detail below.

[0096] In step 1011, if the current round is the first round, the preset expansion threshold is determined as the expansion threshold for the current round.

[0097] Here, various types of sub-load data may include, but are not limited to, sub-load data based on image download services, containerized central control sub-load data, and name service sub-load data. Sub-load data based on image download services is the load data generated when a new compute node joins the compute cluster or an existing compute node updates its image, downloading the latest image containing the processing logic of the target business. For example, when a compute node updates its image, it needs to download an image containing the latest business logic. Assuming this image is 500MB in size, the CPU utilization during the decompression and verification process might reach 80%. The sub-load data based on image download services represents this CPU utilization: 80%. Containerized central control sub-load data is the load data generated during the scheduling and management of compute nodes in the compute cluster. For example, the compute cluster can periodically (every minute) check the status of each compute node, assuming each check consumes 0.2 seconds of CPU time. If the compute cluster includes 100 compute nodes, this would require a total of 20 seconds of CPU time per minute, resulting in an increase in CPU utilization for managing compute nodes of approximately 3.33%. The containerized central control load data represents the CPU utilization: 3.33%. Name service load data refers to the load data generated during address resolution. For example, when a large model's inference service needs to access the storage system or other services, it relies on the name service to resolve the storage system's address, thus generating load data. For instance, when the target business is the model's inference service, the inference service frequently accesses the storage system or other services, requiring address resolution via the name service. Assuming each resolution request consumes 0.05 seconds of CPU time, and there are 100 resolution requests per second, then the name service's CPU utilization will increase by approximately 5%. Therefore, the name service load data represents the CPU utilization: 5%.

[0098] When a computing cluster experiences an anomaly, the first round of scaling up can be performed using a pre-set scaling threshold. That is, the preset scaling threshold is set as the maximum number of computing nodes the computing cluster can include after the first round of scaling up. This embodiment does not limit the value of the preset scaling threshold; it can be set based on historical data, experience, or expected load conditions. For example, assuming the preset scaling threshold is 20 computing nodes, the maximum number of computing nodes the computing cluster can include after the first round of scaling up is 20.

[0099] In step 1012, if the current round is the second round or any round after the second round, obtain the historical expansion threshold and historical load data of the computing cluster in the previous round.

[0100] Here, if the current round is the second round or any round thereafter, the expansion threshold for the current round can be determined based on the historical expansion threshold and historical load data of the computing cluster in the previous round, as well as the load data of the computing cluster in the current round. Therefore, the historical expansion threshold and historical load data of the computing cluster in the previous round can be obtained first. For example, if the current round is the third round, then the historical expansion threshold and historical load data of the computing cluster during the expansion process in the second round can be obtained.

[0101] If the previous round was the first round, the preset expansion threshold is set as the historical expansion threshold for that round. If the previous round was the second round or any subsequent round, the historical expansion threshold for the computing cluster in the previous round is determined based on the load data of the computing cluster in that round. The specific process of determining the historical expansion threshold for the computing cluster in the previous round based on the load data of the computing cluster in the previous round is similar to the specific process of determining the expansion threshold for the computing cluster in the current round based on the current load data of the computing cluster in step 101, and will not be described again here.

[0102] In step 1013, the sub-load data with the largest value in the load data is determined as the second load data, and the historical sub-load data with the largest value in the historical load data is determined as the third load data.

[0103] Here, the historical load data of the computing cluster also includes multiple types of historical sub-load data. The sub-load data with the largest value is selected from the current round of computing cluster load data as the second load data, and the historical sub-load data with the largest value is selected from the historical load data of the computing cluster during the previous round of expansion as the third load data.

[0104] For example, taking CPU utilization as an example, in the current load data, the CPU utilization of the sub-load data based on the image download service is 85%, the CPU utilization of the containerized central control sub-load data is 75%, and the CPU utilization of the name service sub-load data is 90%. Then the second load data is the CPU utilization "90%".

[0105] In step 1014, the expansion threshold is determined based on the second load data, the third load data, and the historical expansion threshold.

[0106] Here, the expansion threshold can be calculated directly based on the second load data, the third load data, and the historical expansion threshold. Alternatively, the initial number of compute nodes in the compute cluster before the previous expansion round can be obtained, and the difference between the historical expansion threshold and the initial number can be used as the historical expansion increment threshold. The historical expansion increment threshold is the maximum number of new compute nodes added to the compute cluster after the previous expansion round. Based on the second load data, the third load data, and the historical expansion increment threshold, the expansion increment threshold for the current round of compute cluster is determined. The number of compute nodes in the current round of compute cluster is obtained, and the number of compute nodes in the current round of compute cluster is summed with the expansion increment threshold to obtain the expansion threshold.

[0107] This application embodiment uses a preset expansion threshold in the first round, and determines the expansion threshold in subsequent rounds based on historical expansion thresholds and load data, thereby achieving dynamic adjustment of the expansion threshold in each round. This ensures that the computing cluster can add more computing nodes under high load conditions and avoid resource waste under low load conditions, ultimately improving the overall performance and stability of the computing cluster.

[0108] In this embodiment of the application, the expansion threshold is determined based on the second load data, the third load data, and the historical expansion threshold. This can be achieved in the following way: First, determine the first ratio of the third load data to the second load data; then, multiply the first ratio by the historical expansion threshold to determine the expansion threshold.

[0109] Here, the expansion threshold can satisfy the following formula (1).

[0110]

[0111] Where N1 is the expansion threshold of the current round, M is the third load data of the previous round, P is the second load data of the current round, and N0 is the historical expansion threshold of the previous round.

[0112] It should be noted that if the product of the first ratio and the historical expansion threshold is not a positive integer, the product can be rounded down. That is, the positive integer closest to and greater than the product of the first ratio and the historical expansion threshold is determined as the expansion threshold. For example, if the historical expansion threshold of the previous round was 20, the third load data of the previous round was a CPU utilization of 90%, and the second load data of the current round is a CPU utilization of 80%, then the expansion threshold of the current round = 90 / 80 × 20 = 22.5 ≈ 23.

[0113] The embodiments of this application are based on a dynamic adjustment mechanism that uses a first ratio of load data and historical expansion thresholds. This mechanism can accurately respond to load fluctuations, flexibly determine the optimal expansion threshold, and improve the resource utilization and stability of the computing cluster.

[0114] In step 102, the target expansion amount of the computing cluster is determined based on the expansion threshold and the preset expansion ratio for the computing cluster.

[0115] The target expansion number is the actual number of computing nodes included in the current round of expansion of the computing cluster.

[0116] Here, the preset expansion ratio for the computing cluster is a parameter used to control the magnitude of each expansion. For example, an expansion ratio of 20% means that each expansion increases the number of existing nodes by 20%. The preset expansion ratio for the computing cluster is related to the target service carried by the computing cluster. Operations personnel can set the expansion ratio corresponding to the target service based on the Service Level Agreement (SLA), as long as it ensures that the computing cluster can meet the SLA requirements for running the target service after each expansion, such as a response time of less than 1 second. If the number of computing clusters expanded based on the expansion ratio does not exceed the expansion threshold, then the number of computing clusters expanded based on the expansion ratio is used as the target expansion number; if the number of computing clusters expanded based on the expansion ratio exceeds the expansion threshold, then the expansion threshold is used as the target expansion number.

[0117] In some embodiments, see Figure 5 In step 102, the target expansion quantity of the computing cluster is determined based on the expansion threshold and the preset expansion ratio for the computing cluster. This can be achieved through the following steps 1021 to 1025, which are explained in detail below.

[0118] In step 1021, the number of second nodes of the computing cluster included in the current round is obtained.

[0119] Here, for each round of expansion, the number of compute nodes in the current round of the computing cluster is used as the second number of nodes. That is, the second number of nodes is the number of compute nodes in the computing cluster after the previous round of expansion. For example, if the current round is the 3rd round of expansion, and the number of compute nodes in the computing cluster increased to 30 after the 2nd round of expansion, then the second number of nodes in the current round is 30.

[0120] In step 1022, the expected expansion amount of the computing cluster is determined based on the expansion ratio and the number of second nodes.

[0121] Here, the product of the expansion ratio and the number of second nodes can be used to determine the number of nodes to be added in the current round. The number of nodes to be added is the number of new nodes needed in the computational cluster when expanding according to the expansion ratio in the current round. The sum of the number of nodes to be added and the number of second nodes is used to determine the expected expansion number of the computational cluster. The expected expansion number is the number of computational nodes included in the computational cluster after expanding according to the expansion ratio.

[0122] For example, the current expansion round is the third round, with an expansion ratio of 20%. The second number of compute nodes included in the compute cluster in this round is 30. Therefore, the number of nodes added = 30 × 20% = 6. The expected expansion number of the compute cluster = 6 + 30 = 36.

[0123] In step 1023, it is determined whether the expected expansion quantity is greater than the expansion threshold.

[0124] Here, it is determined whether the expected expansion amount is greater than the expansion threshold of the computing cluster in the current round. If the expected expansion amount is greater than the expansion threshold, the process jumps to step 1024; if the expected expansion amount is less than or equal to the expansion threshold, the process jumps to step 1025.

[0125] In step 1024, in response to the desired expansion amount being greater than the expansion threshold, the expansion threshold is determined as the target expansion amount.

[0126] For example, the current round of expansion is the 3rd round. The expected expansion number of the computing cluster is 36, and the expansion threshold is 35. Since the expected expansion number is greater than the expansion threshold, the target expansion number is determined to be 35.

[0127] In step 1025, in response to the desired expansion amount being less than or equal to the expansion threshold, the desired expansion amount is determined as the target expansion amount.

[0128] For example, the current round of expansion is the 3rd round. The expected expansion number of the computing cluster is 36, and the expansion threshold is 40. Since the expected expansion number is less than the expansion threshold, the target expansion number is determined to be 36.

[0129] This application embodiment accurately determines the expected expansion quantity by using the number of second nodes in the current round and the preset expansion ratio, and selects the minimum value between the expansion threshold and the expected expansion quantity as the target expansion quantity. This ensures that the target expansion quantity will not exceed a reasonable range, guaranteeing that the computing cluster can respond quickly to load changes while avoiding excessive expansion, thereby optimizing resource utilization and service performance and ensuring the stability of the computing cluster.

[0130] In some embodiments, see Figure 6 The pre-set expansion ratio for the computing cluster in step 102 can be determined through the following steps 201 to 204, which will be explained in detail below.

[0131] In step 201, the number of third nodes in the computing cluster included in the target time period is obtained.

[0132] Among them, the target time period is the time period with the largest number of requests for the target service among multiple historical time periods.

[0133] Here, this application embodiment does not limit the number of historical time periods, which can be selected according to actual needs. Each historical time period has the same length. For example, each day within a historical week can be selected as a historical time period. The time period with the largest number of requests for the target service refers to the time period with the largest number of service requests for the target service received by the computing cluster. The number of computing nodes included in the computing cluster within the target time period is used as the number of third nodes. If the number of computing nodes included in the computing cluster within the target time period is a dynamic value, that is, if there are multiple different numbers of computing nodes within the target time period, the maximum number of computing nodes included in the computing cluster within the target time period is used as the number of third nodes.

[0134] For example, suppose there are three historical time periods: Time Period A, Time Period B, and Time Period C. For the inference service of a large model, the computing cluster in Time Period A receives 1000 inference requests with 15 nodes; the computing cluster in Time Period B receives 1500 inference requests with 20 nodes; and the computing cluster in Time Period C receives 1200 inference requests with 18 nodes. Therefore, Time Period B receives the most inference requests, so Time Period B is chosen as the target time period, and the third time period has 20 nodes.

[0135] In step 202, the performance requirement data of the target service is obtained, and the node number threshold of the target service is determined based on the performance requirement data and the number of third nodes.

[0136] The node number threshold is the maximum number of computing nodes required for the target service to run stably in the computing cluster.

[0137] Here, stable operation of the target service means that the computing cluster can continuously and reliably provide services to the target service while meeting its specific requirements, without interruption or performance degradation due to insufficient computing resources, overload, or other abnormal conditions of the computing nodes. Performance requirements data refers to the various performance indicators needed for the stable operation of the target service in the computing cluster, including but not limited to response time (the maximum allowed response time for each service request), throughput (the maximum number of service requests that can be processed per unit time), and success rate (the percentage of service requests that are successfully processed). The actual response time of the target service running in the computing cluster should be less than or equal to the response time in the performance requirements data; the actual throughput should be greater than or equal to the response time-throughput in the performance requirements data; and the actual success rate should be greater than or equal to the success rate in the performance requirements data. Performance requirements data can be pre-defined by operations and maintenance personnel for the target service.

[0138] The node number threshold for the target service is the number of computing nodes required after multiple rounds of scaling up the computing cluster following a computing cluster failure. The minimum number of nodes needed to meet the performance requirements of the target service can be determined based on performance requirement data, and the sum of this minimum number and the third minimum number of nodes can be used as the node number threshold.

[0139] In some embodiments, see Figure 7 In step 202, the threshold for the number of nodes for the target service is determined based on the performance requirement data and the number of third nodes. This can be achieved through the following steps 2021 to 2023, which are explained in detail below.

[0140] In step 2021, the mapping table is queried based on the performance requirement data.

[0141] The mapping table includes the relationship between the data range and the number of fifth nodes. The data range is obtained by dividing the value range of the performance requirement data.

[0142] Here, the number of fifth nodes is the minimum number of computing nodes required to meet the performance requirements corresponding to the data interval. The performance requirement data is divided into multiple data intervals, and different data intervals correspond to different numbers of fifth nodes in the mapping table. This application does not limit the method and number of data intervals to be divided; these can be set based on historical experience.

[0143] For example, Table 1 is a preset mapping table, where the performance requirement data is the success rate, with a value range of 0 to 100. The success rate range [0, 100] is divided into multiple data intervals: [0, 10], [10, 50], [50, 80], and [80, 100]. The mapping table includes the association between each data interval and the number of fifth nodes. In the mapping table, the number of fifth nodes associated with the data interval [0, 10] is 1, the number of fifth nodes associated with the data interval [10, 50] is 3, the number of fifth nodes associated with the data interval [50, 80] is 10, and the number of fifth nodes associated with the data interval [80, 100] is 20.

[0144] Table 1 Mapping Table

[0145] Data range Number of fifth nodes [0,10] 1 [10,50] 3 [50,80] 10 [80,100] 20

[0146] In step 2022, if the target data range containing the performance requirement data is found by querying the mapping table, the number of fifth nodes associated with the target data range is determined.

[0147] For example, if the performance requirement data for the target business is a success rate of 95%, then the target data range where the performance requirement data is located, which can be found from the mapping table, is [80, 100]. The number of fifth nodes associated with the target data range [80, 100] is 20.

[0148] In step 2023, the sum of the number of third nodes and the number of fifth nodes associated with the target data interval is determined as the node number threshold.

[0149] Here, the number of third nodes refers to the number of computing nodes included in the computing cluster within the target time period, which is the time period among multiple historical time periods that receives the largest number of requests for the target service. The number of fifth nodes associated with the target data interval is the minimum number of computing nodes required to meet the performance requirements of the target service. Therefore, the sum of the number of third nodes and the number of fifth nodes associated with the target data interval is determined as the node number threshold.

[0150] In this embodiment, the total number of third nodes and the number of fifth nodes associated with the target data interval are determined as the node number threshold. This ensures that the node number threshold can meet the maximum number of target services during peak periods, support the performance requirements of the target services, and generate redundant nodes to prevent sudden situations and ensure that the computing cluster can support the stable operation of the target services.

[0151] In step 203, the difference between the node number threshold and the fourth node number is determined.

[0152] The fourth node number refers to the number of compute nodes included in the cluster before the first round of expansion.

[0153] Here, the number of fourth nodes is the initial number of nodes when the computing cluster encounters an anomaly. The difference between the node number threshold and the number of fourth nodes represents the number of additional nodes required to ensure the stable operation of the target service during multiple rounds of expansion.

[0154] In step 204, the second ratio of the difference to the preset number of expansions is determined as the preset expansion ratio for the computing cluster.

[0155] Here, this application embodiment does not limit the preset number of expansions, which can be set based on historical experience, etc. The preset number of expansions can be positively correlated with the total number of current nodes in the computing cluster, that is, positively correlated with the number of fourth nodes. For example, the positive correlation between the preset number of expansions and the number of fourth nodes can be represented by a linear function. Assume that the preset number of expansions and the number of fourth nodes satisfy the linear relationship E = kX + b, where E is the preset number of expansions, X is the number of fourth nodes, k is a proportionality coefficient, representing the expected increase in the number of expansions for each additional computing node, and b is the intercept of the linear function, representing the basic number of expansions that may still be performed based on historical experience or other factors even without any nodes (X = 0).

[0156] Based on historical high load conditions and the performance requirements of the target service, the embodiments of this application accurately determine and optimize the expansion ratio of the computing cluster, so that the computing cluster can maintain the stable operation of the target service during rapid expansion, thereby improving the reliability of the computing cluster.

[0157] In step 103, the computing cluster is expanded according to the target expansion amount.

[0158] Here, expanding a computing cluster refers to increasing the number of computing nodes included in the cluster. Expanding the computing cluster to the target expansion number involves adding multiple computing nodes, bringing the actual number of computing nodes in the cluster to the target expansion number.

[0159] In the event of an anomaly in the computing cluster, this embodiment determines the expansion threshold based on the cluster's load data and calculates the target expansion quantity based on the expansion threshold and a preset expansion ratio for the computing cluster. The target expansion quantity is the actual number of computing nodes included in the expanded computing cluster. Expanding the computing cluster according to the target expansion quantity ensures that the cluster has a sufficient number of computing nodes to cope with anomalies, thereby guaranteeing the stable operation of services. The adoption of a multi-stage, gradual expansion method based on dynamically adjusting the expansion threshold in each round of load data, rather than a one-time large-scale expansion, helps to reduce the peak pressure faced by computing nodes and ensures the stability and reliability of the entire computing cluster operation.

[0160] In some embodiments, the computing cluster is used to support the target service; where N rounds of expansion have already been performed, N is an integer greater than or equal to 1, see [link to relevant documentation]. Figure 8 The capacity adjustment method for computing clusters provided in this application embodiment further includes steps 301 to 305.

[0161] In step 301, it is determined whether the number of the first node in the computing cluster is less than the preset threshold for the number of nodes for the target service.

[0162] The first node count is the number of computing nodes included in the cluster after N rounds of expansion have been performed.

[0163] Here, if the number of the first node is less than the node number threshold, proceed to step 302; if the number of the first node is greater than or equal to the node number threshold, proceed to step 304.

[0164] In step 302, in response to the fact that the number of the first nodes in the computing cluster is less than the preset node number threshold for the target service, the current first load data of the computing cluster and the first indicator data of the target service are obtained.

[0165] The first indicator is used to characterize the operation of the target service after N rounds of capacity expansion have been performed.

[0166] Here, the first load data is the cluster load data calculated after N rounds of expansion have been performed. The first metric data may include, after N rounds of expansion, the queue length, latency, and success rate of the target service, etc.

[0167] In step 303, it is determined whether the first load data is less than or equal to a preset first load threshold, and whether the first indicator data conforms to a preset indicator data range.

[0168] Here, this application embodiment does not limit the value of the preset first load threshold. The first load threshold is a pre-set upper limit of load, used to determine whether the first load data of the current computing cluster is within an acceptable range. The first load threshold is usually determined based on historical data, performance test results, or business requirements. For example, for CPU utilization: setting the first load threshold to 80% means that when the CPU utilization exceeds 80%, the computing cluster is considered to be overloaded. The preset indicator data range refers to a set of reasonable ranges of performance indicators set in advance, used to evaluate whether the operating status of the target business (such as the inference service of a large model) is normal. For example, for latency, setting the indicator data range to within 1 second means that the latency of each target business request should not exceed 1 second.

[0169] If the first load data is less than or equal to the first load threshold and the first indicator data is within the indicator data range, proceed to step 304; if the first load data is greater than the preset first load threshold or the first indicator data is not within the preset indicator data range, proceed to step 305.

[0170] In step 304, the expansion operation for the computing cluster is terminated.

[0171] Here, the expansion operation for the computing cluster ends when the number of the first node is greater than or equal to the node number threshold. Alternatively, the expansion operation for the computing cluster ends when the first load data is less than or equal to the first load threshold and the first metric data falls within the metric data range.

[0172] In step 305, the computing cluster is expanded in round N+1.

[0173] Here, in response to the first load data being greater than the preset first load threshold, or the first indicator data not conforming to the preset indicator data range, the computing cluster is expanded in round N+1.

[0174] This application embodiment obtains load data and indicator data after each expansion until the load data and indicator data are within a preset range, or the number of computing nodes included in the computing cluster after expansion reaches a node number threshold. This ensures that the computing cluster can support the stable operation of the target business after the expansion is completed, and improves the reliability of the computing cluster.

[0175] In some embodiments, before performing at least one round of capacity expansion for the computing cluster in response to an anomaly, the capacity adjustment method for the computing cluster provided in this application can determine whether the computing cluster is experiencing an anomaly in the following ways: acquiring fourth load data of the computing cluster and second indicator data of the target service; determining that the computing cluster is experiencing an anomaly if the fourth load data is greater than a preset second load threshold, wherein the fourth load data is the load data of the computing cluster before performing at least one round of capacity expansion; determining a first expansion quantity based on the fourth load data and the second indicator data if it is determined that the computing cluster needs to be expanded based on the fourth load data and the second indicator data; repeatedly executing the operation of determining the first expansion quantity based on the fourth load data and the second indicator data if the first expansion quantity is greater than the capacity threshold of the computing cluster, until the number of repeated operations exceeds the number of times threshold, at which point it is determined that the computing cluster is experiencing an anomaly; and determining that the computing cluster is experiencing an anomaly if the second indicator data is not acquired.

[0176] Here, a pre-defined capacity scaling process is executed before the computing cluster experiences an anomaly. This process specifically includes: acquiring the fourth load data of the computing cluster prior to the anomaly, and the second indicator data of the target service prior to the anomaly. The second indicator data characterizes the operational status of the target service before the computing cluster undergoes at least one round of scaling. If acquiring the second indicator data fails, i.e., if the second indicator data is not obtained, it is determined that the computing cluster has experienced an anomaly.

[0177] If the second indicator data is successfully obtained, it is determined whether the fourth load data exceeds a preset second load threshold. The preset second load threshold is a pre-defined upper limit for the computing cluster's load, representing a load value that is almost impossible for the cluster to reach. It is used to determine whether the obtained computing cluster load data is reliable. If the fourth load data is greater than or equal to the second load threshold, it indicates that the fourth load data is unreliable, confirming an anomaly in the computing cluster. It should be noted that the second load threshold is greater than the first load threshold.

[0178] If the fourth load data is less than the second load threshold, it indicates that the fourth load data is reliable. Reliable fourth load data means that it is within a reasonable range of the computing cluster's load data. The need for scaling up or down the current computing cluster can be determined based on the fourth load data and the second indicator data. In one scenario, a higher value of the second indicator data indicates higher performance. In this case, if the fourth load data is greater than the first load threshold, or the second indicator data is less than a preset indicator data threshold, it is determined that the current computing cluster needs to be scaled up. Conversely, if the fourth load data is less than the first load threshold, or the second indicator data is greater than the indicator data threshold, it is determined that the current computing cluster needs to be scaled down. For example, if the second indicator data is a success rate of 70%, the preset indicator data threshold is 80%, the first load threshold is CPU utilization of 45%, and the fourth load data is CPU utilization of 60%, then the current computing cluster needs to be scaled up. In another scenario, a lower value of the second indicator data indicates higher performance, such as latency. In this case, if the fourth load data is greater than the first load threshold, or the second indicator data is greater than a preset indicator data threshold, it is determined that the current computing cluster needs to be scaled up.

[0179] If the computing cluster needs expansion based on the fourth load data and the second indicator data, a first expansion quantity can be determined based on these two data. The first expansion quantity is the actual number of computing nodes included after the computing cluster is expanded according to a preset capacity scaling process. It is then determined whether the first expansion quantity is greater than the computing cluster's capacity threshold, which is the upper limit on the number of computing nodes the computing cluster can accommodate. If the first expansion quantity is greater than the computing cluster's capacity threshold, it indicates that the first expansion quantity is impossible to achieve, the computing cluster is malfunctioning, and the operation of determining the first expansion quantity based on the fourth load data and the second indicator data is repeated. If the recalculated first expansion quantity is less than or equal to the computing cluster's capacity threshold, the computing cluster is expanded according to the first expansion quantity. Otherwise, the number of times this operation is repeated is recorded; when the number exceeds a preset threshold, it indicates that the computing cluster is malfunctioning.

[0180] In this embodiment of the application, if the second indicator data is lost, or the fourth load data exceeds the first load threshold, or the first expansion quantity is greater than the capacity threshold, it is determined that the computing cluster is abnormal. This can comprehensively detect abnormal situations in the computing cluster, so as to perform multiple rounds of expansion when abnormalities occur, thereby improving the stability of the computing cluster.

[0181] In some embodiments, if the first expansion quantity is less than or equal to the capacity threshold, the computing cluster is expanded according to the first expansion quantity.

[0182] Here, when the first expansion quantity is less than or equal to the capacity threshold, multiple computing nodes can be added to the computing cluster so that the actual number of computing nodes included in the computing cluster reaches the first expansion quantity.

[0183] In this embodiment of the application, when no abnormality occurs in the computing cluster, the computing cluster is expanded according to the expansion quantity calculated based on the actual load and business indicator data, so as to avoid over-expansion or under-expansion and optimize resource utilization.

[0184] In some embodiments, determining the first expansion quantity based on the fourth load data and the second indicator data can be achieved in the following manner: First, determine the second expansion quantity based on the fourth load data and a preset first load threshold; then, determine the third expansion quantity based on the second indicator data and a preset indicator data threshold; finally, determine the maximum value between the second expansion quantity and the third expansion quantity as the first expansion quantity.

[0185] Here, the second expansion quantity is the number of nodes required for the computing cluster to ensure that the fourth load data is less than or equal to the first load threshold. The third expansion quantity is the number of nodes required to achieve the second indicator data meeting the target business standard. The maximum value between the second and third expansion quantities is determined as the first expansion quantity, which satisfies both the load requirements of the computing cluster and the performance requirements of the target business.

[0186] The embodiments of this application ensure that the expansion decision takes into account both the current load and the business performance requirements, thereby accurately adjusting the resource allocation of the computing cluster, improving response speed and stability, optimizing resource utilization, and ensuring the efficient operation of the target business.

[0187] In this embodiment of the application, the determination of the second expansion quantity based on the fourth load data and the preset first load threshold can be achieved in the following way: First, determine the third ratio of the fourth load data and the first load threshold; then, multiply the third ratio by the number of computing nodes included in the computing cluster before the first round of expansion to determine the second expansion quantity.

[0188] For example, if the first load threshold is a CPU utilization of 45% and the fourth load threshold is a CPU utilization of 60%, then the third ratio = 60 / 45. Before the first round of expansion, the computing cluster included 45 compute nodes, so the second expansion amount = 60 / 45 × 45 = 60.

[0189] The third expansion quantity is determined based on the second indicator data and the preset indicator data threshold. This can be achieved in the following way: First, determine the fourth ratio between the second indicator data and the indicator data threshold; then, multiply the fourth ratio by the number of computing nodes included in the computing cluster before the first round of expansion to determine the third expansion quantity.

[0190] For example, if the second metric is a latency of 2 seconds and the preset threshold is 1 second, then the fourth ratio = 2 / 1. If the number of compute nodes in the cluster before the first round of expansion is 45, then the second expansion quantity = 2 / 1 × 45 = 90. It should be noted that the above method is used when a smaller second metric indicates higher performance. If a larger second metric indicates higher performance, the threshold and the ratio of the second metric can be determined, and the product of this ratio and the number of compute nodes in the cluster before the first round of expansion can be used to determine the third expansion quantity.

[0191] This application embodiment dynamically adjusts the expansion quantity based on actual load and target service metrics data to avoid over-expansion or under-expansion, thereby improving the stability of the computing cluster.

[0192] In some embodiments, if it is determined that the computing cluster needs to be expanded based on the fourth load data and the second indicator data, a second expansion quantity can also be determined based on the fourth load data and a preset first load threshold. A fifth expansion quantity is determined based on multiple preset ratios and preset indicator data ranges. The maximum value between the second and fifth expansion quantities is determined as the first expansion quantity.

[0193] In this embodiment, the values ​​of the multiple preset ratios are not limited, as long as the multiple preset ratios increase exponentially. For example, the multiple preset ratios are 1%, 2%, 4%, etc. Based on the multiple preset ratios and the preset index data range, the fifth expansion quantity is determined as follows: The multiple preset ratios are sequentially multiplied by the fourth node number to obtain the number of nodes added corresponding to each preset ratio, where the fourth node number is the number of computing nodes included in the computing cluster before the first round of expansion. The number of nodes added corresponding to each preset ratio is summed with the fourth node number to obtain the fourth expansion quantity corresponding to each preset ratio; the third index data of the target business is predicted after expanding the computing cluster according to the fourth expansion quantity; the first target preset ratio whose third index data conforms to the index data range is selected from the multiple preset ratios, and the fourth expansion quantity corresponding to the target preset ratio is determined as the fifth expansion quantity.

[0194] For example, if multiple preset ratios are 1%, 2%, and 4%, and the number of fourth nodes is 200, then the number of nodes added corresponding to each preset ratio are 200 × 1% = 2, 200 × 2% = 4, and 200 × 4% = 8, respectively. The number of fourth expansions corresponding to each preset ratio are 202, 204, and 208, respectively. Assuming that it is predicted that when the number of computing nodes in the computing cluster is increased to 204, the third indicator data of the target business will be 80% success rate; that when the number of computing nodes in the computing cluster is increased to 204, the third indicator data of the target business will be 85% success rate; and that when the number of computing nodes in the computing cluster is increased to 208, the third indicator data of the target business will be 87% success rate, and the indicator data range is a success rate of 85% or higher, then the first preset ratio where the third indicator data meets the indicator data range is 2%, and the fourth expansion quantity "204" is determined as the fifth expansion quantity.

[0195] This application embodiment predicts in sequence how the performance requirements of the target business can be met after the computing cluster is expanded by multiple preset ratios that increase exponentially. This achieves accurate determination of the number of nodes that need to be expanded, avoiding excessive expansion and resource waste.

[0196] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0197] With the rise of artificial intelligence models, the scale of heterogeneous computing power has also increased significantly. In the inference services supporting these models, to reduce the operational costs of heterogeneous computing power, replica scaling is implemented based on the peak and off-peak periods of service requests. Horizontal replica scaling is a crucial method for this. Horizontal replica scaling refers to adapting to load changes by increasing or decreasing the number of running replicas. This scaling can be based on the resource load of heterogeneous computing power or the detection data of service requests. For example, when the volume of service requests increases, the heterogeneous computing power system will automatically or manually add more replicas to distribute the load and ensure service performance is not affected. When the volume of service requests decreases, the heterogeneous computing power system will reduce unnecessary replicas, thereby freeing up resources and reducing costs. However, how to ensure that the inference services of the artificial intelligence models supported by the heterogeneous computing power system do not experience anomalies when the resource load of heterogeneous computing power or the detection data of service requests are abnormal is a pressing issue that needs to be addressed in replica relocation cost reduction (a strategy that optimizes resource utilization and reduces costs by dynamically adjusting the number and configuration of replicas).

[0198] To address the aforementioned issues, this application provides a method for adjusting the capacity of a computing cluster. This method is based on constructing a heterogeneous computing power defense and remediation strategy to improve resource efficiency. It approaches the issue from the perspective of disaster recovery and remediation, designing a fallback defense strategy to support the normal operation of the inference service of the artificial intelligence model. Figure 9 This is a flowchart illustrating the fallback defense strategy provided in an embodiment of this application. See also... Figure 9 When a user initiates the inference service of an artificial intelligence model, the heterogeneous computing power system acquires detection data. When the detection data is normal, the heterogeneous computing power system performs replica scaling based on the detection data; or, when the detection data is abnormal, the heterogeneous computing power system executes a defense and remediation strategy. The goal of the defense and remediation strategy is to restore the maximum number of replicas, thereby serving the heterogeneous computing power requests of the inference service.

[0199] Figure 10 This is a schematic diagram of the heterogeneous computing system provided in an embodiment of this application. See also... Figure 10 The heterogeneous computing system 600 includes a scaling module 601, a heterogeneous container module 602 (corresponding to the computing cluster in the above embodiments), and a product module 603. The heterogeneous container module 602 includes multiple heterogeneous containers 602-1 and / or heterogeneous devices 602-2 (corresponding to computing nodes in the above embodiments). The product module 603 includes an artificial intelligence model running on the heterogeneous computing system 600. This artificial intelligence model may include large models, visual models, speech models, game models, medical models, natural language processing (NLP) models, etc. The heterogeneous container module 602 is used to support inference services for large models, visual inference services for visual models, speech inference services for speech models, game inference services for game models, medical inference services for medical models, and NLP inference services for natural language processing models, etc.

[0200] In the scaling module 601, the replication of heterogeneous computing power (corresponding to the computing nodes in the above embodiments) is expanded based on the detection data. When the detection data is normal, the number of expanded replications (corresponding to the first expansion quantity in the above embodiments) is generated based on the detection data. The number of replications is the number of heterogeneous containers 602-1 and / or heterogeneous devices 602-2 in the heterogeneous container module 602. The number of expanded replications is sent to the production scheduling center, which performs normal replication expansion based on the number of expanded replications, increasing the number of heterogeneous containers 602-1 and / or heterogeneous devices 602-2 in the heterogeneous container module 602. However, when the detection data is abnormal, the number of expanded replications cannot be generated, and a fallback production scheduling strategy is activated to perform rapid expansion. In this embodiment, the fallback production scheduling strategy refers to rapidly increasing the number of replications to be expanded to the maximum replication limit of the service (corresponding to the node number threshold preset for the target service in the above embodiments) to ensure the normal operation of the service.

[0201] This application's embodiments design a defensive remedial strategy. Specifically, if an anomaly occurs in the horizontal scaling process of the heterogeneous inference service, the defense assumes that the auto-scaling function is no longer usable in the heterogeneous computing power system. The fallback method is to expand the number of heterogeneous computing power replicas to the maximum number required by the business to support normal service. During the scaling process, considering the sudden increase in pressure on the heterogeneous computing power system caused by simultaneous scaling of multiple services for replica retrieval and production creation, this application's embodiments employ different remedial scaling strategies based on the scale. The reason for using different scaling strategies is primarily to consider that a large number of concurrent scaling operations would cause excessive pressure on the infrastructure, such as high pressure on container image services and congestion in containerized production scheduling.

[0202] Figure 11 This is a schematic diagram of the remedial capacity expansion strategy provided in an embodiment of this application. See also... Figure 11First, when an anomaly occurs in the horizontal scaling process of the heterogeneous inference service, a remedial scaling strategy is triggered: replica scaling is performed. The number of replicas to be scaled up (corresponding to the expected scaling up number in the above embodiments) is calculated according to the percentage configured for the service (corresponding to the scaling ratio in the above embodiments). It is then determined whether the number of replicas exceeds a threshold (corresponding to the scaling threshold in the above embodiments). If it exceeds the threshold, a single scaling up operation is performed according to the threshold value; if it is within the threshold, scaling up is performed according to the replica count calculated as a percentage. After a single scaling up operation, the process of calculating the number of replicas to be scaled up according to the percentage configured for the service and determining whether the number of replicas exceeds the threshold is repeated cyclically to achieve multiple scaling up of the heterogeneous container modules. The heterogeneous computing power operation and maintenance personnel can determine the percentage of the service based on the number of devices in the service and the Service Level Agreement (SLA) of the committed service. The threshold can be dynamically updated based on the infrastructure load (obtained from the load data of the heterogeneous container modules).

[0203] The following is a detailed description of the capacity adjustment method for the computing cluster provided in the embodiments of this application.

[0204] The computing cluster capacity adjustment method provided in this application embodiment, when the horizontal scaling process of heterogeneous inference service is abnormal, in order to support the normal service of the business, expands the number of heterogeneous computing power replicas of the business to the maximum number of replicas required by the business, and makes cyclic decisions on the number of replicas for each scaling up during the scaling process.

[0205] A heterogeneous computing system can include multiple computing clusters, where each computing cluster is used to run inference services for different models. Figure 12 This is a schematic diagram illustrating the principle of the computing cluster capacity adjustment method provided in this application embodiment. See also... Figure 12 For a computing cluster 701, which includes multiple heterogeneous containers and multiple heterogeneous devices, both heterogeneous containers and devices can be understood as replicas (corresponding to computing nodes in the above embodiments). Load data of computing cluster 701 can be collected in real time and stored in database 702. The load data of computing cluster 701 includes the load data of each heterogeneous device and heterogeneous container. Simultaneously, the business center uploads the metric data of the inference service running on computing cluster 701 to database 702 in real time. Based on the stored load data and metric data, it determines whether to expand or shrink the cluster, and aggregates the scalable number of replicas to achieve normal expansion or shrinkage operations. Taking expansion as an example, based on the load data and metric data, the number of replicas for expansion (corresponding to the first expansion number in the above embodiments) is generated, and normal expansion of the computing cluster is performed according to the number of replicas for expansion, that is, increasing the number of replicas (increasing the number of heterogeneous devices or heterogeneous containers).

[0206] The process of generating the number of replicas for capacity expansion based on load data and metric data is as follows: A first number of replicas (corresponding to the second expansion quantity in the above embodiment) can be determined based on load data, and a second number of replicas (corresponding to the third expansion quantity in the above embodiment) can be determined based on service metric data. The maximum value between the first and second number of replicas is taken as the number of replicas for expansion. For load data: A first load threshold can be set. The first number of replicas is calculated based on the collected load data and the corresponding first load threshold. For example, if the CPU utilization threshold is 60%, the collected CPU utilization is 50%, and the original number of replicas is 10, then the first number of replicas = 10 × (60 / 50) = 12. For service metric data: A preset metric data threshold is set. A second number of replicas is calculated based on the collected metric data and the corresponding metric data threshold. For example, if the success rate threshold is 95%, and the collected success rate is 90%, then the second number of replicas = 10 + 10 × (95-90)% = 10.5 ≈ 11. Alternatively, if the latency threshold is 1 second and the collected latency data is 2 seconds, then the capacity is expanded by 1% (corresponding to the preset ratio in the above embodiment) to predict the time index after expansion. If the time index after expansion still does not reach 1 second, the capacity is expanded sequentially by exponential growth ratios such as 2% and 4%.

[0207] When the detected load data (corresponding to the fourth load data in the above embodiments) and indicator data (corresponding to the second indicator data in the above embodiments) are abnormal, the load data and indicator data cannot reflect the resource or request load status of the computing cluster, and a disaster recovery plan is determined to be executed. Abnormal load data and indicator data include one of the following: indicator data for the service request is missing; or, the detected load data exceeds a preset load data threshold (corresponding to the second load threshold in the above embodiments); or, based on the load data and indicator data, the number of replicas generated for expansion exceeds a replica number threshold (corresponding to the capacity threshold in the above embodiments).

[0208] The disaster recovery and mitigation solution constructed in this application embodiment enables the number of service replicas to be expanded to the maximum expected number of replicas in the event of a failure, thus ensuring service availability in terms of resource efficiency. The strategy center 703 can configure the percentage of each expansion in the disaster recovery process based on the service dimension, and supports dynamic adjustment of the threshold according to the load of the infrastructure that the heterogeneous containerization depends on. Figure 13 This is a schematic diagram illustrating the configuration of thresholds and percentages provided in an embodiment of this application. See also... Figure 13The strategy center 703 obtains load data from the database 702. The load data includes image download service load, container download service load, and name service load (corresponding to multiple types of sub-load data in the above embodiment). Image download service load is load data based on image download service, container download service load is central control load data generated by containerization, and name service load is name-based service load data for heterogeneous inference. The maximum value is selected from image download service load, container download service load, and name service load, and this maximum value is used as the load value of the infrastructure. The threshold N1 is dynamically output based on the load value of the infrastructure. The threshold N1 for the current period (corresponding to the scaling threshold of the computing cluster in the current round in the above embodiment) can be calculated using the above formula (1).

[0209] Where N1 is the threshold of the current period, N0 is the threshold of the previous period, M is the load value of the infrastructure in the previous period, and P is the load value of the infrastructure in the current period.

[0210] By comparing the infrastructure load values ​​M and P in adjacent periods, and multiplying them proportionally by the threshold N0 of the previous period, the threshold N1 for the current period is calculated and output. This threshold N is then used to support decisions on the number of replicas to be expanded in a single cycle. The period can be set by the user, such as 1 minute or 5 minutes. For different services, the Policy Center 703 can also obtain the percentage of single-cycle expansion configured by the user for different services. For example, the percentage for service A is Na%, and the percentage for service B is Nb%. Users can configure the percentage of single-cycle expansion at the service level. This percentage value is obtained by the service operations personnel based on the service's needs assessment and is a static configuration value that does not require dynamic changes.

[0211] After determining the threshold N and the percentage of a single service expansion, the strategy center 703 outputs the threshold (corresponding to the expansion threshold in the above embodiment) and sends the threshold and percentage to the judgment module 704. The judgment module 704 compares the threshold with the expected expansion replica count (corresponding to the expected expansion quantity in the above embodiment) based on the product of the percentage and the original replica count. If the expected expansion replica count is within the threshold, the replica count is calculated according to the percentage and used as the single expansion replica count (corresponding to the target expansion quantity in the above embodiment); or, if the expected expansion replica count exceeds the threshold, the threshold is used as the single expansion replica count. After confirming the single expansion replica count, abnormal expansion is performed on the computing cluster based on the single expansion replica count, resulting in the expanded computing cluster 705. The purpose of performing the above logical judgment based on the threshold and the percentage of the service is to ensure the availability of the infrastructure and avoid cumulative failures caused by large concurrent expansion. Each replication expansion requires a determination of the expansion value, and the threshold also changes dynamically. Therefore, the expansion threshold is volatile. The goal is to achieve the maximum expected number of replications for the business as quickly as possible while ensuring the availability of the infrastructure.

[0212] The maximum number of replicas required for a business can include the following two scenarios: For each business, operations and maintenance personnel can preset a maximum number of replicas for that business based on historical experience (corresponding to the preset node number threshold for the target business in the above embodiment); or, after each single abnormal expansion, the current indicator data of the business and the load data of the computing cluster are re-collected. When the current indicator data of the business (such as latency, queue length, success rate, etc.) meets the preset indicator range and the load data of the computing cluster also reaches the preset load threshold, the number of replicas after the expansion is determined to be the maximum number of replicas.

[0213] Figure 14 This is a flowchart illustrating the capacity adjustment method for a computing cluster provided in this application embodiment. The key aspect of the processing flow in this application embodiment is that when an anomaly occurs in the horizontal scaling process of the heterogeneous inference service, a fallback strategy is employed to expand the number of heterogeneous computing power replicas of the service to the maximum number of replicas required by the service, thereby supporting the normal service flow.

[0214] Step 1401, Begin.

[0215] Here, the computing cluster receives a relevant service request from the service (corresponding to the target service in the above embodiment) and begins to adjust the capacity of the computing cluster.

[0216] Step 1402: Detect the load of each heterogeneous container in the computing cluster.

[0217] Here, load data from various heterogeneous containers and devices in the computing cluster, as well as business metrics data, are collected.

[0218] Step 1403: Load information is reported to the database.

[0219] Here, load data and business metrics data are reported to the database and stored.

[0220] Step 1404: Aggregate the number of replicas required for expansion.

[0221] Here, the load data and business metrics data are aggregated to obtain the number of replicas for expansion (corresponding to the first expansion quantity in the above embodiment).

[0222] Step 1405: Determine if the number of replicas is output correctly.

[0223] Here, if the number of replicas is less than or equal to the set replica number threshold (corresponding to the capacity threshold of the computing cluster in the above embodiment), then the number of replicas is within the normal range, and the process jumps to step 1407; if the number of replicas is greater than the set replica number threshold, then the number of replicas is within the abnormal range, and the process jumps to step 1409.

[0224] Step 1406: Start the automated scaling of the task.

[0225] Here, steps 1406 and 1402 can be executed in parallel. In response to receiving relevant business requests, automatic scaling can be initiated. Initiating automatic scaling requires determining the number of replicas to be scaled up based on load data and business metrics data.

[0226] Step 1407: Determine the number of replicas for expansion.

[0227] Here, the number of replicas generated in step 1404 is determined as the number of expanded replicas (corresponding to the first expansion quantity in the above embodiment).

[0228] Step 1408: Perform normal capacity expansion operation.

[0229] Here, a normal expansion operation is performed on the computing cluster based on the number of expansion replicas, increasing the number of heterogeneous containers and heterogeneous devices in the computing cluster by the number of expansion replicas.

[0230] Step 1409, anomaly count.

[0231] Here, if the replica count output is abnormal, jump to step 1404 and retry step 1404 to aggregate the replica count of the expansion requirement. The number of times step 1404 is re-executed will be used as the exception count.

[0232] Step 1410: Determine whether the abnormal count exceeds the threshold.

[0233] Here, when the anomaly count exceeds the threshold, the detected data is determined to be abnormal, and an anomaly defense scaling operation is performed. Proceed to step 1411. Multiple anomaly checks are used to prevent misjudgments due to cluster instability. If the anomaly count does not exceed the threshold, proceed to step 1409.

[0234] Step 1411: Read the percentage and threshold of the business.

[0235] Here, when performing anomaly defense expansion, it is necessary to calculate the number of replicas for a single expansion (corresponding to the target expansion quantity in the above embodiments). First, determine the percentage of service configuration (corresponding to the expansion ratio in the above embodiments) and the threshold of the dynamically configured expansion upper limit (corresponding to the expansion threshold in the above embodiments). The percentage of service and the threshold can be read from the second database storing the percentage and the threshold.

[0236] Step 1412, compare the percentage and the threshold.

[0237] Here, the expected number of replicas (corresponding to the expected expansion number in the above embodiments) is calculated based on the percentage of the service configuration, and the expected number of replicas is compared with the threshold.

[0238] Step 1413: Determine the number of replicas involved in this abnormal expansion.

[0239] Here, the minimum value between the expected number of replicas and the threshold is selected as the number of replicas for this abnormal expansion (corresponding to the target expansion number in the above embodiment).

[0240] Step 1414: Perform the operation to increase the number of replicas for disaster recovery.

[0241] Here, the computing cluster is expanded according to the number of replicas in this abnormal expansion, and the number of heterogeneous containers and heterogeneous devices is increased according to the number of replicas in the abnormal expansion.

[0242] Step 1415: Obtain the load data of the infrastructure.

[0243] Here, after the computing cluster completes its expansion, the load data of the computing cluster is collected again, and the load data of the infrastructure is determined based on the load data of the computing cluster.

[0244] Step 1416: Perform the threshold calculation for capacity expansion.

[0245] Here, a new threshold (corresponding to the expansion threshold in the above embodiment) is obtained by calculating the expansion limit based on the infrastructure load.

[0246] Step 1417: Update the expansion threshold.

[0247] Here, it checks if the threshold has changed. If the threshold has changed, the new threshold is updated to match the expansion threshold. If the threshold has not changed, the original threshold is used to calculate the replica count for the next abnormal expansion.

[0248] Step 1418: Perform the data storage operation.

[0249] Here, the updated threshold is stored in the second database to calculate the number of replicas for the next abnormal expansion. Then, proceed to step 1411 to perform a cyclical calculation of the number of replicas for expansion until the load data and business metrics are within the normal range after expansion, or until the maximum number of replicas preset by the business is reached, at which point the expansion process stops.

[0250] Step 1419: Collect the expansion percentage values ​​for each service.

[0251] Here, step 1419 can be executed before the first abnormal capacity expansion, that is, before the first execution of step 1411. The capacity expansion percentage for each service can be configured by the operations and maintenance personnel. For example, the number of devices used by the service during peak periods can be determined based on historical data. Based on the SLA (Availability: determining the maximum acceptable downtime of the system; Response Time: determining the maximum acceptable response time for users; Recovery Time: determining the maximum acceptable recovery time of the system after a failure), the service level can be determined, and the corresponding redundant device quantity can be obtained. The total number of devices required for capacity expansion for the service = redundant device quantity + device quantity used by the service during peak periods - current device quantity. The total number of devices to be expanded / the set ideal number of expansions = the percentage of expansion. After collection, proceed to step 1418 to store the capacity expansion percentage values ​​for each service in the second database.

[0252] Step 1420: Configure the initial value of the expansion limit threshold.

[0253] Here, step 1420 can be executed before the first abnormal expansion, that is, before the first execution of step 1411. Step 1419 can be executed in parallel with step 1420. The user can set the initial value of the expansion limit threshold, that is, the threshold of the expansion limit during the first round of abnormal expansion. Jump to step 1418 to store the initial value of the expansion limit threshold in the second database.

[0254] In summary, this application's embodiments, from a disaster recovery perspective, design a fallback strategy. Specifically, when an anomaly occurs in the horizontal scaling process of the heterogeneous inference service, the defense assumes that the automatic scaling function is no longer usable in the system. The fallback method is to expand the number of heterogeneous computing power replicas to the maximum number required by the service, thereby supporting normal service. During the scaling process, the number of replicas for each scaling operation is considered, and a loop judgment is performed during each scaling operation. Advantages compared to traditional solutions: 1) This application's embodiments technically construct a fallback strategy for heterogeneous inference service scaling anomalies, eliminating the need for operations and maintenance personnel to follow up on alarm responses and processing, reducing the pressure on operations and maintenance personnel, and improving the efficiency of heterogeneous computing resources. 2) This application's embodiments employ a defensive remedial scaling strategy when heterogeneous inference service scaling anomalies. The purpose is to consider that the scope and severity of the impact cannot be assessed in abnormal situations, and from a resource service perspective, to quickly expand the number of service replicas to the expected upper limit of the service, serving as a fallback resource service. 3) In the remedial expansion process of this application embodiment, the logic of repeatedly judging the number of expansion replicas is adopted for each expansion, which avoids peak pressure on the infrastructure and thus ensures the overall stability of resource containerization production.

[0255] The following description continues to illustrate the exemplary structure of the computing cluster capacity adjustment device 255 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the capacity adjustment device 255 of the computing cluster in the memory 250 may include:

[0256] The first determining module 2551 is used to respond to an anomaly in the computing cluster by performing at least one round of expansion for the computing cluster. In each round of expansion, based on the current load data of the computing cluster, a expansion threshold for the computing cluster in the current round is determined. The expansion threshold is the maximum number of computing nodes included in the computing cluster after the current round of expansion.

[0257] The second determining module 2552 is used to determine the target expansion quantity of the computing cluster based on the expansion threshold and the preset expansion ratio for the computing cluster, wherein the target expansion quantity is the actual number of computing nodes included in the computing cluster after the current round of expansion.

[0258] The expansion module 2553 is used to expand the computing cluster according to the target expansion amount.

[0259] In some embodiments, the computing cluster is used to carry the target service; where N rounds of expansion have been performed, N is an integer greater than or equal to 1. The capacity adjustment device 255 of the computing cluster further includes a loop module, used to obtain the current first load data of the computing cluster and the first indicator data of the target service in response to the first number of nodes in the computing cluster being less than a preset node number threshold for the target service, wherein the first number of nodes is the number of computing nodes included in the computing cluster after N rounds of expansion have been performed, and the first indicator data is used to characterize the operation of the target service after N rounds of expansion have been performed; in response to the first load data being greater than the preset first load threshold, or the first indicator data not conforming to the preset indicator data range, the computing cluster is expanded in the N+1th round.

[0260] In some embodiments, the load data includes multiple types of sub-load data. The first determining module 2551 is further configured to: determine a preset expansion threshold as the expansion threshold for the current round if the current round is the first round; and obtain the historical expansion threshold and historical load data of the computing cluster in the previous round if the current round is the second round or any round thereafter; determine the sub-load data with the largest value in the load data as the second load data, and determine the historical sub-load data with the largest value in the historical load data as the third load data; and determine the expansion threshold based on the second load data, the third load data, and the historical expansion threshold.

[0261] In some embodiments, the first determining module 2551 is further configured to determine a first ratio of the third load data to the second load data; and to determine the expansion threshold by multiplying the first ratio by the historical expansion threshold.

[0262] In some embodiments, the second determining module 2552 is further configured to obtain the number of second nodes of the computing cluster included in the current round; determine the expected expansion number of the computing cluster based on the expansion ratio and the number of second nodes; determine the expected expansion number as the target expansion number in response to the expected expansion number being less than or equal to the expansion threshold; and determine the expansion threshold as the target expansion number in response to the expected expansion number being greater than the expansion threshold.

[0263] In some embodiments, the capacity adjustment device 255 of the computing cluster further includes a third determining module, configured to: obtain the number of third nodes of the computing nodes included in the computing cluster within a target time period, wherein the target time period is the time period in which the number of requests for the target service is the largest among multiple historical time periods; obtain performance requirement data of the target service, and determine a node number threshold for the target service based on the performance requirement data and the number of third nodes, wherein the node number threshold is the maximum number of computing nodes required to achieve stable operation of the target service in the computing cluster; determine the difference between the node number threshold and the number of fourth nodes, wherein the number of fourth nodes is the number of computing nodes included in the computing cluster before the first round of expansion; and determine the difference and a second ratio of a preset number of expansions as a preset expansion ratio for the computing cluster.

[0264] In some embodiments, the third determining module is further configured to query a mapping table based on performance requirement data, wherein the mapping table includes the relationship between data ranges and the number of fifth nodes, and the data ranges are obtained by dividing the value ranges of performance requirement data; if the target data range where the performance requirement data is located is found from the mapping table, the number of fifth nodes associated with the target data range is determined; and the sum of the number of third nodes and the number of fifth nodes associated with the target data range is determined as the node number threshold.

[0265] In some embodiments, the capacity adjustment device 255 of the computing cluster further includes a fourth determining module, configured to, in response to an anomaly in the computing cluster and before performing at least one round of capacity expansion for the computing cluster, acquire fourth load data of the computing cluster and second indicator data of the target service; determine that the computing cluster is abnormal if the fourth load data is greater than a preset second load threshold, wherein the fourth load data is the load data of the computing cluster before performing at least one round of capacity expansion; determine a first expansion quantity based on the fourth load data and the second indicator data if it is determined that the computing cluster needs to be expanded based on the fourth load data and the second indicator data; if the first expansion quantity is greater than the capacity threshold of the computing cluster, repeatedly execute the operation of determining the first expansion quantity based on the fourth load data and the second indicator data until the number of repeated operations exceeds a number threshold, and then determine that the computing cluster is abnormal; determine that the computing cluster is abnormal if the second indicator data is not acquired.

[0266] In some embodiments, the fourth determining module is further configured to expand the computing cluster according to the first expansion quantity if the first expansion quantity is less than or equal to the capacity threshold.

[0267] In some embodiments, the fourth determining module is further configured to determine a second expansion quantity based on the fourth load data and a preset first load threshold; determine a third expansion quantity based on the second indicator data and a preset indicator data threshold; and determine the maximum value of the second expansion quantity and the third expansion quantity as the first expansion quantity.

[0268] In some embodiments, the fourth determining module is further configured to: determine a third ratio of the fourth load data and the first load threshold; multiply the third ratio by the number of compute nodes included in the compute cluster before the first round of expansion to determine a second expansion quantity; determine a fourth ratio of the second indicator data and the indicator data threshold; and multiply the fourth ratio by the number of compute nodes included in the compute cluster before the first round of expansion to determine a third expansion quantity.

[0269] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the computing cluster capacity adjustment method described above in this application.

[0270] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the capacity adjustment method for a computing cluster provided in this application. For example, ... Figure 3 The method for adjusting the capacity of a computing cluster is shown.

[0271] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0272] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0273] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0274] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0275] In summary, the embodiments of this application can expand the number of computing nodes to the maximum number required by the business when anomalies occur in the horizontal scaling process of the computing cluster, thereby supporting normal business services, reducing the pressure on operation and maintenance personnel, improving the efficiency of heterogeneous computing resources, and reducing the risk of abnormal business requests. The process employs a loop-based logic to determine the number of nodes to be expanded in each expansion, avoiding peak pressure on the infrastructure and thus ensuring the overall stability of the computing cluster.

[0276] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for adjusting the capacity of a computing cluster, characterized in that, The computing cluster includes multiple computing nodes; the method includes: In response to an anomaly in the computing cluster, at least one round of scaling up is performed on the computing cluster, wherein the following processing is performed in each round of scaling up: Based on the current load data of the computing cluster, the expansion threshold of the computing cluster in the current round is determined, wherein the expansion threshold is the maximum number of computing nodes included in the computing cluster after the expansion in the current round; Based on the expansion threshold and the preset expansion ratio for the computing cluster, the target expansion quantity of the computing cluster is determined, wherein the target expansion quantity is the actual number of computing nodes included in the computing cluster after the current round of expansion. The computing cluster is expanded according to the target expansion amount.

2. The method according to claim 1, characterized in that, The computing cluster is used to support the target service; having already undergone N rounds of expansion (where N is an integer greater than or equal to 1), the method further includes: In response to the fact that the number of first nodes in the computing cluster is less than a preset threshold for the number of nodes for the target service, the first load data of the computing cluster and the first indicator data of the target service are obtained. The first number of nodes is the number of computing nodes included in the computing cluster after N rounds of expansion have been performed. The first indicator data is used to characterize the operation of the target service after N rounds of expansion have been performed. In response to the first load data being greater than a preset first load threshold, or the first indicator data not being within a preset indicator data range, the computing cluster is expanded in round N+1.

3. The method according to claim 1, characterized in that, The load data includes multiple types of sub-load data; Determining the scaling threshold for the computing cluster in the current round based on the current load data of the computing cluster includes: If the current round is the first round, the preset expansion threshold will be set as the expansion threshold for the current round. If the current round is the second round or any round after the second round, obtain the historical expansion threshold and historical load data of the computing cluster in the previous round; The sub-load data with the largest value in the load data is determined as the second load data, and the historical sub-load data with the largest value in the historical load data is determined as the third load data; The expansion threshold is determined based on the second load data, the third load data, and the historical expansion threshold.

4. The method according to claim 3, characterized in that, The step of determining the expansion threshold based on the second load data, the third load data, and the historical expansion threshold includes: Determine a first ratio between the third load data and the second load data; The product of the first ratio and the historical expansion threshold is determined as the expansion threshold.

5. The method according to claim 1, characterized in that, Determining the target expansion amount of the computing cluster based on the expansion threshold and the preset expansion ratio for the computing cluster includes: Obtain the number of second nodes of the computing nodes included in the computing cluster in the current round; Based on the expansion ratio and the number of second nodes, determine the expected expansion amount of the computing cluster; In response to the desired expansion amount being less than or equal to the expansion threshold, the desired expansion amount is determined as the target expansion amount; In response to the desired expansion amount being greater than the expansion threshold, the expansion threshold is determined as the target expansion amount.

6. The method according to claim 1, characterized in that, The computing cluster is used to support the target service; the method further includes: Obtain the number of third nodes of the computing nodes included in the computing cluster within a target time period, wherein the target time period is the time period with the largest number of requests for the target service among multiple historical time periods; The performance requirement data of the target service is obtained, and the node number threshold of the target service is determined based on the performance requirement data and the number of third nodes, wherein the node number threshold is the maximum number of computing nodes required to enable the target service to run stably in the computing cluster. Determine the difference between the node number threshold and the fourth node number, wherein the fourth node number is the number of computing nodes included in the computing cluster before the first round of expansion; The difference is compared with a second ratio of a preset number of expansions to determine the preset expansion ratio for the computing cluster.

7. The method according to claim 6, characterized in that, The step of determining the node number threshold for the target service based on the performance requirement data and the number of third nodes includes: The performance requirement data query mapping table includes the relationship between data ranges and the number of fifth nodes. The data ranges are obtained by dividing the value ranges of the performance requirement data. If the target data range containing the performance requirement data is found from the mapping table, determine the number of fifth nodes associated with the target data range; The sum of the number of the third node and the number of the fifth node associated with the target data interval is determined as the node number threshold.

8. The method according to any one of claims 1 to 7, characterized in that, The computing cluster is used to carry the target service; before performing at least one round of capacity expansion for the computing cluster in response to an anomaly, the method further includes: Obtain the fourth load data of the computing cluster and the second indicator data of the target service. If the fourth load data is greater than the preset second load threshold, determine that the computing cluster is abnormal. The fourth load data is the load data of the computing cluster before it performs the at least one round of expansion. If it is determined that the computing cluster needs to be expanded based on the fourth load data and the second indicator data, a first expansion quantity is determined based on the fourth load data and the second indicator data. If the first expansion quantity is greater than the capacity threshold of the computing cluster, the operation of determining the first expansion quantity based on the fourth load data and the second indicator data is repeated until the number of times the operation is repeated is greater than the number of times the computing cluster is determined to be abnormal. If the second indicator data is not obtained, it is determined that the computing cluster is malfunctioning.

9. The method according to claim 8, characterized in that, The method further includes: If the first expansion quantity is less than or equal to the capacity threshold, the computing cluster is expanded according to the first expansion quantity.

10. The method according to claim 8, characterized in that, Determining the first expansion quantity based on the fourth load data and the second indicator data includes: Based on the fourth load data and the preset first load threshold, the second expansion quantity is determined; Based on the second indicator data and the preset indicator data threshold, the third expansion quantity is determined; The maximum value between the second expansion quantity and the third expansion quantity is determined as the first expansion quantity.

11. The method according to claim 10, characterized in that, The step of determining the second expansion quantity based on the fourth load data and the preset first load threshold includes: Determine a third ratio between the fourth load data and the first load threshold; The product of the third ratio and the number of computing nodes included in the computing cluster before the first round of expansion is determined as the second expansion amount; The step of determining the third expansion quantity based on the second indicator data and a preset indicator data threshold includes: Determine a fourth ratio between the second indicator data and the threshold value of the indicator data; The third expansion quantity is determined by multiplying the fourth ratio by the number of computing nodes in the computing cluster before the first round of expansion.

12. A capacity adjustment device for a computing cluster, characterized in that, The computing cluster includes multiple computing nodes; the device includes: The first determining module is configured to respond to an anomaly in the computing cluster by performing at least one round of expansion on the computing cluster, wherein in each round of expansion, based on the current load data of the computing cluster, a expansion threshold for the computing cluster in the current round is determined, wherein the expansion threshold is the maximum number of computing nodes included in the computing cluster after the current round of expansion. The second determining module is used to determine the target expansion quantity of the computing cluster based on the expansion threshold and the preset expansion ratio for the computing cluster, wherein the target expansion quantity is the actual number of computing nodes included in the computing cluster after the current round of expansion. The expansion module is used to expand the computing cluster according to the target expansion amount.

13. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the capacity adjustment method of the computing cluster according to any one of claims 1 to 11.

14. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the capacity adjustment method of the computing cluster according to any one of claims 1 to 11.

15. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the capacity adjustment method of the computing cluster according to any one of claims 1 to 11.

Images