A cloud cluster automatic expansion method and system and a storage medium

Abstract

This invention relates to the field of information technology, and provides a method, system, and storage medium for automatic expansion of cloud clusters. The automatic expansion method includes: acquiring and parsing data within a first time window to determine at least one shard corresponding to each message on each node; forming a dynamic graph structure based on the at least one shard corresponding to each message on each node and the relationships between shards; using the dynamic graph structure as input to a TGNN model, and using the model to output the predicted traffic value and confidence level of each shard in a second time period; constructing expansion instructions indicating overloaded shards, the number of new nodes, and the traffic switching method, wherein overloaded shards are shards whose predicted traffic value is greater than an overload threshold and whose confidence level is greater than a trust threshold; creating and starting shadow nodes in parallel based on the number of new nodes, with shadow nodes prohibited from allocating shards; and migrating shard replicas of overloaded shards to shadow nodes in stages according to the traffic switching method, and then officially enabling the shadow nodes.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a method, system and storage medium for automatic expansion of cloud clusters. Background Technology

[0002] As product business continues to grow, the demands on distributed cloud cluster data storage and computing are also increasing. Existing cloud clusters, such as Elasticsearch clusters, are no longer large enough to meet the needs of rapid business growth. Furthermore, when the usage of the distributed cluster reaches its designed capacity limit, existing technologies necessitate scaling up the cloud cluster.

[0003] Current distributed cloud cluster expansion technologies determine whether expansion is needed based on the cluster's current load, and then manage and expand the cloud cluster manually. This "delayed triggering" expansion method causes the cloud cluster to face performance bottlenecks when traffic surges, failing to meet the real-time requirements of high-concurrency scenarios.

[0004] Manually scaling up a cloud cluster requires logging into the cluster to collect and observe the cluster software's operational status parameters, then manually configuring the new machines, including configuring network information and various cluster software settings, before manually adding the newly configured machines to the distributed cluster. This manual method of scaling up a cloud cluster is inefficient, time-consuming, inflexible, and requires significant manpower to resolve errors.

[0005] Furthermore, during the expansion process, the cloud cluster needs to reallocate shards and balance the load, that is, to achieve expansion by restarting the process and adding all shards. This process often makes the cloud cluster's services temporarily unavailable or significantly degrade in performance, thus causing an interruption.

[0006] In order to overcome the above-mentioned defects of existing technologies, there is an urgent need in this field for an automatic cloud cluster expansion technology that can solve the pain points of "delayed triggering and time-consuming interruption" in existing cloud cluster expansion, avoid the low efficiency of manual expansion, improve the efficiency of distributed cloud cluster expansion, and achieve horizontal expansion with zero business awareness. Summary of the Invention

[0007] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed descriptions that follow.

[0008] In order to overcome the above-mentioned defects in the existing technology, the present invention provides a method, system and storage medium for automatic expansion of cloud clusters.

[0009] Specifically, the cloud cluster automatic expansion method provided by the first aspect of the present invention includes the following steps: acquiring and parsing message traffic data and physical node topology data within a first time window to determine at least one shard corresponding to each message on each node; forming a dynamic graph structure based on the at least one shard corresponding to each message on each node and the relationship between the shards; using the dynamic graph structure as input to a TGNN model, and using the TGNN model to output the predicted traffic value of each shard in a second time period and the corresponding confidence level, wherein the confidence level is determined based on the uncertainty estimation, historical error estimation, and data integrity of the TGNN model; constructing expansion instructions indicating overloaded shards, the number of new nodes, and the traffic switching method, wherein the overloaded shard is the shard whose predicted traffic value is greater than an overload threshold and whose confidence level is greater than a trust threshold, and the number of new nodes is determined based on the predicted traffic value of the overloaded shard and the resource status of the node; creating and starting shadow nodes in parallel based on the number of new nodes, wherein the shadow nodes are prohibited from allocating shards; and migrating the shard replicas of the overloaded shards to the shadow nodes in stages according to the traffic switching method, and then making the shadow nodes active to complete the cloud cluster expansion.

[0010] Furthermore, in some embodiments of the present invention, the step of forming a dynamic graph structure based on at least one fragment on each node corresponding to each message and the relationship between the fragments includes: determining the vertices of the dynamic graph structure based on the index information of each message, the fragments, and the triples formed by the physical nodes where the fragments are located; and determining the edges of the dynamic graph structure based on the data flow relationship between the fragments.

[0011] Furthermore, in some embodiments of the present invention, the step of determining the vertices of the dynamic graph structure based on the index information of each message, the shard, and the triple formed by the physical node where the shard is located includes: determining the feature vector of the vertex based on the message traffic data and physical node topology data collected within the first time window, wherein the feature vector includes traffic features, node resource features, and topology features.

[0012] Furthermore, in some embodiments of the present invention, the TGNN model includes: a spatial convolutional layer, which uses a graph attention network or a graph convolutional network to aggregate first-order neighbor information of each vertex of the dynamic graph structure to capture local flow dependencies; and a temporal convolutional layer, which uses causal dilated convolution or temporal self-attention to capture long-term dependencies.

[0013] Further, in some embodiments of the present invention, the step of determining the confidence level includes: integrating a Dropout layer before each convolutional layer and fully connected layer of the TGNN model; using the Dropout layer to perform multiple forward propagation inferences on the input to obtain a probability distribution of predicted traffic values; obtaining the uncertainty variance of the TGNN model, which characterizes the uncertainty estimate of the TGNN model, based on the probability distribution of the predicted traffic values; characterizing the historical error estimate based on the historical prediction performance of the shard, wherein the historical prediction performance is determined based on the historical predicted traffic values ​​and historical actual traffic values ​​of the shard; evaluating the data quality of message traffic data and physical node topology data related to the shard within the first time window to characterize the data integrity; and determining the confidence level based on the uncertainty estimate of the TGNN model, the historical error estimate, and the data integrity.

[0014] Furthermore, in some embodiments of the present invention, the shadow node prevents the allocation of fragments to the shadow node by a triple lock, the triple lock including a configuration lock, a tag lock, and a weight lock.

[0015] Furthermore, in some embodiments of the present invention, the step of migrating the fragment replicas of the overloaded fragment to the shadow node in stages according to the traffic switching method includes: the shadow node determining the proportion of fragment replicas of the overloaded fragment to be accessed based on the fragment weight corresponding to each stage of the traffic switching method, wherein the fragment weight corresponding to each stage is increasing until the fragment weight is 100%.

[0016] Furthermore, in some embodiments of the present invention, the step of migrating the shard replicas of the overloaded shard to the shadow node in stages according to the traffic switching method includes the steps of: obtaining the health indicators of the shadow node in each stage, the health indicators including write round-trip latency, disk wait percentage and CPU preemption percentage; and reclaiming the shadow node in response to the health indicators of the shadow node indicating abnormal performance.

[0017] Furthermore, the cloud cluster automatic expansion system provided according to the second aspect of the present invention includes a memory and a processor. The memory stores computer instructions. The processor is connected to the memory and configured to execute the computer instructions stored in the memory to implement the cloud cluster automatic expansion method provided in any of the above embodiments.

[0018] Furthermore, the computer-readable storage medium provided according to the third aspect of the present invention stores computer instructions. When the computer instructions are executed by a processor, the automatic cloud cluster expansion method provided in any of the above embodiments is implemented.

[0019] The beneficial effects of the cloud cluster automatic expansion method and cloud cluster automatic expansion system provided by the present invention are as follows: The present invention can solve the pain points of "delayed triggering and time-consuming interruption" in the existing cloud cluster expansion, avoid the low efficiency of manual expansion, improve the efficiency of distributed cloud cluster expansion, and realize horizontal expansion with zero business awareness. Attached Figure Description

[0020] The above-described features and advantages of the present invention will be better understood after reading the following detailed description of embodiments of the present disclosure in conjunction with the accompanying drawings. In the drawings, components are not necessarily drawn to scale, and components having similar related characteristics or features may have the same or similar reference numerals.

[0021] Figure 1 The diagram illustrates application scenarios provided by some embodiments of the present invention;

[0022] Figure 2 A schematic diagram of an automatic cloud cluster scaling system provided according to some embodiments of the present invention is shown;

[0023] Figure 3 A flowchart of an automatic cloud cluster scaling method according to some embodiments of the present invention is shown;

[0024] Figure 4 A flowchart of an automatic cloud cluster scaling method according to a preferred embodiment of the present invention is shown.

[0025] Figure label:

[0026] 10: Server;

[0027] 20: Terminal;

[0028] 30: Internet;

[0029] 200: Cloud cluster automatic expansion system;

[0030] 210: Memory;

[0031] 211: Computer-readable storage medium;

[0032] 220: Processor;

[0033] S310~S360: Steps. Detailed Implementation

[0034] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Although the description of the present invention is presented in conjunction with preferred embodiments, this does not mean that the features of the invention are limited to these embodiments. On the contrary, the purpose of describing the invention in conjunction with embodiments is to cover other options or modifications that may be derived based on the claims of the present invention. To provide a thorough understanding of the invention, many specific details will be included in the following description. The invention may also be implemented without using these details. Furthermore, to avoid confusion or obscuring the focus of the invention, some specific details will be omitted in the description.

[0035] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0036] Furthermore, the terms "upper," "lower," "left," "right," "top," "bottom," "horizontal," and "vertical" used in the following description should be understood as the orientations shown in the relevant paragraphs and accompanying drawings. These relative terms are for illustrative purposes only and do not imply that the described apparatus must be manufactured or operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0037] It is understood that although terms such as "first," "second," and "third" may be used herein to describe various components, regions, layers, and / or parts, these components, regions, layers, and / or parts should not be limited by these terms, and these terms are only used to distinguish different components, regions, layers, and / or parts. Therefore, the first components, regions, layers, and / or parts discussed below may be referred to as second components, regions, layers, and / or parts without departing from some embodiments of the present invention.

[0038] As mentioned above, current distributed cloud cluster expansion technology determines whether expansion is needed based on the current cluster load, and then manages and expands the cloud cluster manually. This "delayed triggering" expansion method causes the cloud cluster to face performance bottlenecks when traffic surges, failing to meet the real-time requirements of high-concurrency scenarios.

[0039] Manually scaling up cloud clusters requires logging into the cluster to collect and observe the cluster software's operational status parameters, then manually configuring the new machines, including configuring network information and various cluster software settings, before manually adding the newly configured machines to the distributed cluster. This manual method of scaling up cloud clusters is inefficient, time-consuming, inflexible, and requires significant manpower to resolve errors.

[0040] Furthermore, during the expansion process, the cloud cluster needs to reallocate shards and balance the load, that is, to achieve expansion by restarting the process and adding all shards. This process often makes the cloud cluster's services temporarily unavailable or significantly degrade in performance, thus causing an interruption.

[0041] To overcome the aforementioned deficiencies in existing technologies, this invention provides a cloud cluster automatic expansion method, system, and storage medium that can solve the pain points of "delayed triggering and time-consuming interruption" in existing cloud cluster expansion, avoid the low efficiency of manual expansion, improve the efficiency of distributed cloud cluster expansion, and achieve horizontal expansion with zero business awareness.

[0042] The cloud cluster automatic expansion method provided by the first aspect of the present invention can be applied to any computer device with data processing and computing capabilities, and such computer device can be various types of terminals or servers.

[0043] Please refer to Figure 1 , Figure 1 The diagram illustrates application scenarios provided by some embodiments of the present invention.

[0044] exist Figure 1 In the illustrated embodiment, the application scenario may include at least one server 10 and a terminal 20, which are connected via a network 30. Here, the network 30 may be a wide area network (WAN), a local area network (LAN), or a combination of both. The terminal 20 and the server 10 are interconnected via a wired or wireless network.

[0045] In some embodiments, server 10 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal 20 can be a smartphone, tablet, laptop, desktop computer, set-top box, smart voice interaction device, smart home appliance, vehicle terminal, aircraft, and mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices, smart speakers, and smartwatches), but is not limited to these.

[0046] It should be understood that Figure 1 The server 10, terminal 20, and network 30 shown are merely illustrative. Depending on implementation needs, any number of servers, networks, and terminals can be included. It is understood that embodiments of this application may also allow multiple terminals 20 to access server 10 simultaneously.

[0047] Exemplary based on Figure 1 The implementation environment shown in this embodiment of the invention provides a cloud cluster automatic expansion system. The following description uses the example of the cloud cluster automatic expansion system being configured in server 10. It can be understood that the cloud cluster automatic expansion system can also be configured in terminal 20.

[0048] The cloud cluster automatic scaling method provided in the first aspect of the present invention can be implemented via the cloud cluster automatic scaling system provided in the second aspect of the present invention.

[0049] Please refer to Figure 2 , Figure 2 A schematic diagram of an automatic cloud cluster expansion system provided according to some embodiments of the present invention is shown.

[0050] like Figure 2 As shown, the cloud cluster automatic expansion system 200 may be configured with a memory 210 and a processor 220. The memory 210 includes, but is not limited to, the computer-readable storage medium 211 described in the third aspect of the present invention, which stores computer instructions thereon. The processor 220 is connected to the memory 210 and is configured to execute the computer instructions stored in the memory 210 to implement the cloud cluster automatic expansion method provided in the first aspect of the present invention.

[0051] The working principle of the above-mentioned automatic cloud cluster scaling system will be described below with reference to some examples of automatic cloud cluster scaling methods. The automatic cloud cluster scaling method provided by this invention achieves automatic scaling of cloud clusters through a three-segment parallel link of message traffic and physical node topology linkage traffic prediction, pre-setting shadow nodes, and weighted gray-scale online deployment, thereby improving the scaling efficiency of cloud clusters.

[0052] Those skilled in the art will understand that these embodiments of the automatic cloud cluster scaling methods are merely non-limiting implementations provided by the present invention, intended to clearly demonstrate the main concepts of the invention and provide specific solutions convenient for public implementation, rather than limiting all functions or operating methods of the automatic cloud cluster scaling system. Similarly, the automatic cloud cluster scaling system is also only one non-limiting implementation provided by the present invention, and does not limit the executing entities and execution order of the steps in these automatic cloud cluster scaling methods.

[0053] Please refer to Figure 3 , Figure 3 A flowchart of an automatic cloud cluster scaling method provided according to some embodiments of the present invention is shown.

[0054] like Figure 3 As shown, the cloud cluster automatic scaling method may include step S310: acquiring and parsing message traffic data and physical node topology data within the first time window to determine at least one shard on each node corresponding to each message.

[0055] The cloud cluster automatic scaling system can acquire information such as the number of messages written by each message middleware used for message transmission within a certain time window, the size of each message in bytes, and the rate of writing to the cluster, as well as resource indicator data on each node side in the cloud cluster, such as CPU utilization, disk wait rate, disk bandwidth, memory usage, GC pauses, and other dimensions.

[0056] A message is the smallest unit of data used to transmit business data or related instructions. Messages do not reside directly in nodes but are first assigned to shards. In some embodiments, message middleware can split a collection of messages of the same type (such as a topic in Kafka) into multiple shards. There is a many-to-many deployment relationship between shards and nodes; a node can host multiple shards, and a shard can be hosted by multiple nodes.

[0057] The cloud cluster automatic scaling system can parse messages within the first time window (e.g., 20 minutes) to obtain at least one shard corresponding to each message and determine the physical node where the shard corresponding to the message is located.

[0058] Afterwards, the cloud cluster automatic scaling system can execute step S320: forming a dynamic graph structure based on at least one shard on each node corresponding to each message and the relationship between the shards.

[0059] In some embodiments, the cloud cluster auto-scaling system can determine the vertices of the dynamic graph structure based on the message index information, shards, and the triples formed by the physical nodes where the shards are located, and determine the edges of the dynamic graph structure using the data flow relationships between shards.

[0060] Furthermore, the cloud cluster automatic scaling system can determine the feature vector of each vertex based on raw data such as message traffic data and physical node topology data collected within a first time window. In some embodiments, the cloud cluster automatic scaling system collects raw data within a first time window at a preset frequency (e.g., sampling once every 100ms). The time step size within the time window can be determined based on the preset frequency. The feature vector of each vertex can include traffic characteristics (e.g., messages per second), node resource characteristics (e.g., CPU utilization), and topology characteristics (e.g., average traffic of neighboring nodes) at each time step.

[0061] In this way, the cloud cluster automatic scaling system can link message traffic data and physical node topology data, and treat both message traffic and physical node topology as a graph that changes over time, thereby enabling cloud cluster traffic prediction at the shard level based on message traffic data and physical node topology data.

[0062] Please continue to refer to this. Figure 3 The cloud cluster automatic expansion system can execute step S330: using the dynamic graph structure as input to the TGNN model, and using the TGNN model to output the predicted traffic value of each shard in the second time period and the corresponding confidence level. The confidence level is determined based on the uncertainty estimation, historical error estimation and data integrity of the TGNN model.

[0063] The deep learning model TGNN (Temporal Graph Neural Network) can handle dynamic graph structure data that evolves over time. It captures the dynamic relationship evolution of nodes and edges in dynamic graphs by introducing time dimension modeling capabilities on the basis of traditional graph neural networks (GNN).

[0064] The cloud cluster automatic scaling system utilizes the TGNN model to achieve shard-level traffic prediction. In some embodiments, the TGNN model may include spatial convolutional layers and temporal convolutional layers. The spatial convolutional layer uses a graph attention network or a graph convolutional network to aggregate first-order neighbor information for each vertex of the dynamic graph structure to capture local traffic dependencies. The temporal convolutional layer uses causal dilated convolutions or temporal self-attention to capture long-term dependencies.

[0065] In some embodiments, the cloud cluster automatic expansion system provided by the present invention can use historical real traffic data and node monitoring data as samples to train and test the TGNN model during the offline training phase.

[0066] The cloud cluster automatic scaling system uses the dynamic graph structure of the current time step as the input of the TGNN model, and uses the TGNN model to output the predicted traffic value and corresponding confidence level of each shard in the second time period (e.g., 5 minutes).

[0067] The confidence score is not determined by a single formula, but rather through an evaluation framework that integrates multi-source information. It is based on the uncertainty estimate, historical error estimate, and data completeness of the TGNN model. This comprehensive approach, combining the uncertainty of the TGNN model itself, the historical behavior patterns of the predicted target, and the quality of the current data, yields a robust confidence score.

[0068] The automatic scaling system for cloud clusters can use Monte Carlo Dropout technology to estimate prediction uncertainty in the TGNN model.

[0069] In some embodiments, the cloud cluster automatic scaling system can integrate a Dropout layer before each convolutional and fully connected layer of the TGNN model. The Dropout layer performs multiple forward propagation inferences on the input, and the predicted traffic values ​​obtained from these multiple inferences constitute the probability distribution of future traffic values ​​predicted for the target segment. Based on this probability distribution of predicted traffic values, the uncertainty variance of the TGNN model, which characterizes the uncertainty estimate of the TGNN model, can be obtained. A larger uncertainty variance indicates greater discrepancy in the TGNN model's predictions for the target segment; higher uncertainty naturally leads to lower confidence.

[0070] The cloud cluster auto-scaling system can dynamically adjust the level of confidence in the next prediction based on the online learning concept of time series forecasting, using the recent prediction performance of a shard. Specifically, the cloud cluster auto-scaling system can characterize historical error estimation based on the historical prediction performance of a shard, which is determined by comparing the historical predicted traffic values ​​with the historical actual traffic values ​​of the shard.

[0071] The cloud cluster auto-scaling system can assess the data quality of message traffic data and physical node topology data related to a shard acquired within the first time window to characterize data integrity. Low-quality or missing data will inevitably lead to unreliable predictions from the TGNN model.

[0072] The confidence level is determined based on the uncertainty estimate, historical error estimate, and data completeness of the TGNN model described above. In some embodiments, the confidence level can be determined by weighted harmonics to integrate the three uncertainties and ensure that any uncertainty is reflected in the confidence level.

[0073] The cloud cluster automatic scaling system uses the aforementioned confidence calculation mechanism to quantify the uncertainty of each prediction by the TGNN model. The confidence level corresponding to each predicted traffic value output can provide a quantifiable and reliable confidence index for subsequent automated decisions (such as whether to trigger scaling or whether to initiate an abnormal rollback).

[0074] Afterwards, the cloud cluster automatic expansion system can execute step S340: construct expansion instructions that indicate overloaded shards, the number of new nodes, and the traffic switching method. Overloaded shards are shards whose predicted traffic value is greater than the overload threshold and whose confidence level is greater than the trust threshold. The number of new nodes is determined based on the predicted traffic value of the overloaded shards and the resource status of the nodes.

[0075] In other words, when the predicted traffic value of a certain shard is greater than the overload threshold and the confidence level is higher than the preset trust threshold, it indicates that the shard is about to be overloaded and the confidence level of the overload is very high, thus triggering a scaling up command. The overloaded shards indicated in the scaling up command include all overloaded shards that are about to be overloaded.

[0076] Here, the number of new nodes can be determined based on the overload status of overloaded shards as determined by predicted traffic values ​​and the resource status of the nodes. The traffic switching method can be preset or learned and optimized by the cloud cluster's automatic scaling system.

[0077] Next, the cloud cluster automatic scaling system can execute step S350: create and start shadow nodes in parallel based on the number of new nodes, and prohibit the allocation of shards to shadow nodes.

[0078] The cloud cluster automatic expansion system can be configured with a pre-set pipeline. The constructed expansion command is sent to the pre-set pipeline, and the pre-set pipeline is used to realize the creation, startup and warm-up of shadow nodes.

[0079] Pre-configured pipelines can create the required number of shadow nodes in parallel based on a pre-defined resource orchestration template. Upon startup, shadow nodes immediately register as nominal members with the master node. Shadow nodes do not allocate shards or participate in traffic writing; they only maintain the cluster topology heartbeat and complete new node warm-up, health checks, and version verification without interrupting business operations. The master node can be a node determined through voting to handle cluster-level operations.

[0080] To prevent shards from being assigned to shadow nodes, the cloud cluster auto-scaling system can enable triple locking when starting shadow nodes, and trigger an alarm and forcibly reclaim shadow nodes when any lock fails.

[0081] Triple locking can include configuration lock, tag lock, and weight lock.

[0082] The configuration lock can be a cluster-level parameter. This configuration lock is triggered and takes effect when the shadow node starts up, so as to globally freeze shard migration and prevent any node (including shadow nodes) from triggering shard rebalancing, thereby avoiding service interruption.

[0083] Tag locks can be shard-level parameters. During the startup of a shadow node, the tag lock is issued immediately (e.g., 1 second) after the master node writes the shadow node's metadata and publishes the new cluster state. This ensures that even if the cluster restarts, the shadow node is absolutely prohibited from being assigned any primary or replica shards before it becomes a master node.

[0084] Weighted locks can be index-level parameters. Before enabling canary traffic migration for shadow nodes, this weighted lock can prevent shards from being assigned to shadow nodes. For example, this weighted lock can force a limit of 0 shards per index on a shadow node.

[0085] After a shadow node starts up, it can send a request to the master node at a preset period (e.g., 1 second) to confirm that the shadow node's heartbeat is maintained. After starting up, the shadow node will not be elected as the master node, nor will it participate in the master node's voting. The master node will treat the shadow node as a node that only maintains a heartbeat and is connected to the cluster but is not assigned a shard for liveness testing, thereby achieving "zero-disturbance" health warm-up of the shadow node.

[0086] During the subsequent gray-scale traffic migration, the cloud cluster's automatic scaling system can assess the health indicators of shadow nodes. If a shadow node is healthy and the confidence level of the predicted traffic value corresponding to the overload situation of the shards allocated to that shadow node during the migration process consistently exceeds the trust threshold, the shadow node can be officially activated after the gray-scale traffic migration is completed. Since the activated shadow node itself has complete sharding capabilities, the activation of the shadow node can be completed instantaneously by removing the triple lock.

[0087] Here, the health metrics for shadow nodes can include write round-trip latency, disk wait percentage, and CPU preemption percentage.

[0088] In this way, the shadow node completes the entire closed loop of registration, health warm-up, and instantaneous activation without interrupting the current writes to the cluster, triggering shard rebalancing, or increasing the voting burden on the master node, thereby ensuring the achievement of the zero-interruption expansion goal of this invention.

[0089] The cloud cluster automatic expansion system can execute step S360: After migrating the overloaded shard replicas to the shadow node in stages according to the traffic switching method, the shadow node is turned into a normal node to complete the cloud cluster expansion.

[0090] The cloud cluster auto-scaling system can continue to enable canary traffic migration. Canary traffic migration can be achieved through a canary deployment pipeline. The canary deployment pipeline migrates the overloaded shard replicas to shadow nodes in stages according to the traffic switching method, and then the pre-configured pipeline turns the shadow nodes into active nodes to complete the cloud cluster scaling.

[0091] Shard allocation and shard migration are different. Shard allocation is a default and automatically executed basic operation in a cloud cluster, representing a deployment process from scratch. Shard migration, on the other hand, is a transfer process from one node to another within the cluster.

[0092] In some embodiments, the cloud cluster auto-scaling system can enable shadow nodes to determine the proportion of shard replicas of the overloaded shards to be accessed based on the shard weight corresponding to each stage. Here, the shard weight corresponding to each stage increases incrementally until the shard weight reaches 100%. The shard weight indicates the distribution ratio of shard replicas between new and old nodes. By controlling the distribution ratio of shard replicas of overloaded shards between new and old nodes, the cloud cluster auto-scaling system indirectly achieves the canary-scale switching of write traffic.

[0093] Preferably, the grayscale traffic migration can be divided into three stages. The first stage corresponds to a preset first weight (e.g., 10%). The second stage corresponds to a preset second weight (e.g., 50%). The first weight is less than the second weight. The third stage corresponds to the full weight (i.e., 100%).

[0094] When Phase 1 begins, shadow nodes access a portion of the overloaded shard replicas with the highest weight. When Phase 2 begins, shadow nodes continue to access the overloaded shard replicas with the second highest weight. When Phase 3 begins, shadow nodes access all overloaded shard replicas with all their weights, thus achieving 100% traffic takeover.

[0095] Here, traffic migration at each stage can be achieved through the Elasticsearch cluster rerouting API (cluster / reroute), and the migration pace can be controlled in conjunction with shard allocation filters.

[0096] In this way, the cloud cluster auto-scaling system can control the sharding distribution ratio of new and old nodes. After each stage of the gray-scale traffic migration is completed, the cloud cluster auto-scaling system automatically mounts the shadow nodes to the data disk and removes all temporary restriction rules (such as triple locks) of the shadow nodes through a pre-configured pipeline, officially marking them as full-function data nodes of the cluster, thereby completing the cloud cluster scaling.

[0097] Preferably, during the gray-scale traffic migration, the cloud cluster automatic scaling system can continuously collect health indicators for each shadow node according to a preset period. These health indicators include write round-trip latency, disk wait percentage, and CPU preemption percentage. When the health indicators of a shadow node indicate abnormal performance, the cloud cluster automatic scaling system can reset the sharding weight of the shadow node to 0% and reclaim the abnormal shadow node. Therefore, the cloud cluster automatic scaling system can solve the problem that traditional gray-scale scaling cannot immediately remove abnormal nodes.

[0098] The following is a specific, non-limiting preferred embodiment, which will be used to further illustrate the cloud cluster automatic expansion system proposed in this invention.

[0099] Please refer to Figure 4 , Figure 4 A flowchart of an automatic cloud cluster scaling method according to a preferred embodiment of the present invention is shown.

[0100] like Figure 4 As shown, the cloud cluster auto-scaling system can utilize a data acquisition layer to acquire message traffic data and physical node topology data at a sampling frequency of 100ms, forming a 20-minute rolling time window. Specifically, the data acquisition layer can use a lightweight, self-developed agent embedded in the Kafka-Broker message middleware to perform byte-level parsing of the target-index field in the header of each acquired message. Through parsing, the data acquisition layer can extract the index name to be written to Elasticsearch (ES) and include the index name, corresponding shard, message size, and timestamp.<index, shard, msg-size, timestamp> Quadruples are injected into the Broker in real time, for example, into the Broker's topic:traffic-graph, to avoid cross-process communication latency. Thus, the data acquisition layer uses Kafka to collect traffic and parse the index, shards, and message volume of each message in real time, providing accurate, low-latency raw data for subsequent predictive models.

[0101] In some embodiments, the inputs acquired by the data acquisition layer include Kafka message volume, the size of a single message in bytes, the write rate to the cluster, and data on the node side such as CPU utilization, disk wait rate, disk bandwidth, memory usage, and GC pauses. After standardization processing, the data acquisition layer outputs a standardized multidimensional vector as subsequent input.

[0102] like Figure 4 As shown, the cloud cluster automatic scaling system can use the TGNN prediction inference layer to predict the traffic of each shard for the next 5 minutes and determine the confidence level of the prediction, so as to detect shards that are about to be overloaded in advance.

[0103] In this preferred embodiment, the TGNN prediction inference layer can first construct a dynamic graph structure based on the output of the data acquisition layer.

[0104] Specifically, the construction of a dynamic graph structure includes vertex definition, edge definition, and dynamic graph structure representation.

[0105] Each vertex in the dynamic graph structure represents a triple (index, shard, node), that is, an index represents a shard on a physical node. The attributes of each vertex can include the number of messages written to that shard at the current time, the total message size, and the resource metrics of the node it belongs to (CPU, memory, disk I / O, etc.).

[0106] In a dynamic graph structure, each edge represents the data flow relationship between shards. For example, different shards with the same index may reside on the same node or different nodes. The edge weight can be the message size, and the edge direction can represent the data flow direction, such as from the producer shard to the consumer shard.

[0107] This forms a dynamic graph structure representation. The dynamic graph Gt can be a dynamic graph that changes over time. In this preferred embodiment, the cloud cluster automatic scaling system samples once every 100ms to form a time step. Within a 20-minute time window, that is, within a sliding window of length T=120 (i.e., 20 minutes), a time series graph sequence {Gt-T+1,...,Gt} is constructed.

[0108] Furthermore, the TGNN prediction and inference layer can determine the feature vector for each vertex at each time step based on raw data such as message traffic data and physical node topology data collected within a 20-minute sliding window. The feature vector for each vertex at each time step includes traffic features, node resource features, and topology features. Traffic features may include messages per second (msg / s), bytes per second (bytes / s), message size distribution (mean or variance), etc. Node resource features may include CPU utilization (%), memory utilization (%), disk I / O latency (ms), network bandwidth utilization (%), etc. Topology features may include the average traffic of neighboring nodes, the physical location (rack, availability zone) of the node where the shard is located, etc. Preferably, the TGNN prediction and inference layer can perform feature standardization to improve the stability of the TGNN model's training and inference.

[0109] In this preferred embodiment, the TGNN prediction inference layer includes a trained TGNN model.

[0110] Here, the TGNN model includes spatial convolutional layers and temporal convolutional layers. The spatial convolutional layers use a graph attention network or graph convolutional network to aggregate first-order neighbor information for each vertex of the dynamic graph structure to capture local flow dependencies. The temporal convolutional layers use causal dilated convolutions to capture long-term dependencies. This dilated convolution can effectively expand the receptive field, thereby capturing periodic patterns within a 20-minute sliding window.

[0111] The output of this TGNN model consists of two branches: a predicted flow value branch and a confidence branch. The predicted flow value branch outputs the predicted flow value Pshard for each segment within the next 5 minutes, and the confidence branch outputs the predicted confidence value Cshard, where Cshard ∈ [0,1]. The output of the confidence branch uses a sigmoid activation function to determine the stability of the numerical boundary.

[0112] During training, the loss function of the TGNN prediction and inference layer can be set. Mean squared error is used as the loss function for the predicted traffic value branch, and binary cross-entropy is used for the confidence branch. The label is whether the historical prediction error is below a threshold (e.g., 10%). The TGNN model is trained using real traffic and node monitoring data from the past three months, with each historical data point representing a sample every 30 seconds, totaling approximately 250,000 samples. The AdamW optimizer is selected during training, with a learning rate of 1e-4 (0.0001) and weight decay of 1e-5 (0.00001). The AdamW optimizer ensures that the TGNN model can efficiently and stably obtain an optimal solution, and the learning rate of 1e-4 achieves a good balance between training efficiency and stability.

[0113] The trained TGNN model can perform inference and prediction on the aforementioned real-time collected data. The TGNN model performs inference on the current dynamic graph structure Gt every 30 seconds, and outputs the predicted flow value Pshard(t+5min) and prediction confidence Cshard for each segment in the next 5 minutes.

[0114] The confidence level of the TGNN model is determined by the uncertainty estimate, historical error estimate, and data completeness of the TGNN model.

[0115] In this preferred embodiment, the uncertainty estimation of the TGNN model can be determined using the Monte Carlo Dropout technique. Specifically, a Dropout layer is integrated before each convolutional and fully connected layer of the TGNN model. This Dropout layer is used not only during training but also remains active during inference and prediction. When the TGNN model needs to predict the traffic at a given time point t, it performs N (e.g., N=50) forward propagation inferences on the same input data (i.e., the dynamic graph Gt). Due to the randomness of Dropout, each inference yields a slightly different prediction value Pshard_i (i from 1 to N). These N predictions constitute a probability distribution of the future traffic for that target slice.

[0116] Take the average of these N prediction results Pshard_i as the final predicted flow value Pshard, that is:

[0117] Pshard = (1 / N) × Σ(Pshard_i).

[0118] Then, calculate the uncertainty variance σ_model of the N prediction results Pshard_i. 2 As an estimate of the uncertainty in the TGNN model, the uncertainty variance σ_model 2 for:

[0119] σ_model 2 =(1 / N)×Σ(Pshard_i - Pshard) 2 .

[0120] Uncertainty variance σ_model 2 The larger the value, the greater the discrepancy in the TGNN model's prediction of the target segment, the higher the uncertainty, and naturally the lower the confidence level.

[0121] The TGNN prediction inference layer can dynamically adjust the level of confidence in the next prediction based on the online learning idea of ​​time series prediction, using the recent prediction performance of the segment.

[0122] The TGNN prediction inference layer can maintain a fixed-length historical window for each slice, for example, saving the predicted and actual traffic values ​​for the most recent M times (M=10).

[0123] After each actual flow rate value is generated (in this preferred embodiment, the actual flow rate value after 5 minutes), the TGNN prediction inference layer can calculate the absolute percentage error (APE) of that prediction:

[0124] APE = |(Ashard - Pshard) / Ashard|,

[0125] Here, Ashard represents the actual flow value.

[0126] Then, the TGNN prediction inference layer can calculate the mean absolute percentage error (MAPE) of the most recent M predictions:

[0127] MAPE_history = (1 / M)×Σ(APE).

[0128] The higher the historical MAPE (MAPE_history), the more difficult it is to predict the traffic pattern of that segment, or the more unstable the TGNN model's performance is on that segment. Therefore, a certain penalty should be imposed on the confidence level of the next prediction.

[0129] The cloud cluster auto-scaling system can assess the data quality of message traffic data and physical node topology data related to the shard acquired within a 20-minute time window to characterize data integrity.

[0130] Specifically, within the aforementioned 20-minute sliding time window for constructing the dynamic graph Gt, the missing rate of the data points in that shard is examined:

[0131] Missing_Rate = (Number_of_Missing_TimeSteps) / Total_TimeSteps,

[0132] Here, Number_of_Missing_TimeSteps represents the number of missing time steps, and Total_TimeSteps represents the total number of time steps.

[0133] Preferably, the TGNN prediction and inference layer can also check whether there are outliers (such as values ​​that are outside the reasonable range) in the collected node resource indicators (CPU, IO, etc.).

[0134] The higher the data missing rate or the more outliers, the lower the signal-to-noise ratio of the input features, and the lower the data completeness index in the prediction confidence.

[0135] The final confidence level Cshard is the sum of the three indicators mentioned above. In this preferred embodiment, the confidence level Cshard can be determined by weighted harmonic calculation of the three indicators to ensure that any uncertainty in any aspect is reflected in the confidence level Cshard. Specifically, the formula for calculating the confidence level Cshard can be:

[0136] Cshard = (w1×f(σ_model 2 ) + w2×(1 - MAPE_history) + w3×(1 -Missing_Rate)) / (w1 + w2 + w3),

[0137] Where, f(σ_model) 2 ) is a model that represents the variance of uncertainty σ_model 2 Functions mapped to the [0,1] interval, such as f(σ_model) 2 ) = exp(-β×σ_model 2 ), where β is the scaling parameter. Uncertainty variance σ_model 2 The larger the value of f(σ_model), the better. 2 The closer the function value of () is to 0, the better. Historical error estimates can be converted to confidence scores using (1 - MAPE_history). When the historical error is 0, this term is 1, indicating complete confidence; the larger the historical error, the closer this term is to 0. Missing rate is converted to a confidence score using (1 - Missing_Rate); the higher the missing rate, the closer this term is to 0. w1, w2, and w3 are adjustable weight parameters used to adjust the importance of the aforementioned confidence scores in the final confidence score Cshard. Typically, the weight w1 corresponding to the confidence score represented by the uncertainty estimate of the TGNN model is set to the highest.

[0138] Therefore, the TGNN prediction and inference layer can obtain a confidence level Cshard normalized to the [0,1] interval. In this preferred embodiment, the TGNN prediction and inference layer can set the confidence threshold to 0.85. Only when the confidence level Cshard > 0.85 will the TGNN prediction and inference layer consider the prediction reliable. Otherwise, the TGNN prediction and inference layer will consider the prediction unreliable, and the cloud cluster automatic scaling system can choose to continue observation to avoid unnecessary resource waste and potential risks caused by false alarms.

[0139] Thus, the TGNN predictive inference layer ensures that the entire cloud cluster's automatic scaling system is both "bold" (acting proactively based on predictions) and "cautious" (relying on high confidence levels for decision-making), thereby achieving truly intelligent, robust, and zero-disruption scaling. The TGNN model treats both "message traffic" and "physical node topology" as a graph that changes over time. It aggregates neighbor information through the graph structure and uses dilated convolution to capture historical trends over 20 minutes. Finally, it outputs the traffic prediction value Pshard and its confidence level Cshard for each shard for the next 5 minutes. These two predictions provide the basis for scaling / rollback decisions for shadow nodes, improving the efficiency of distributed cloud cluster scaling.

[0140] The cloud cluster auto-scaling system iterates through each shard in the cluster. When the cloud cluster auto-scaling system makes a prediction for a certain shard, in response to the confidence score Cshard output by the TGNN model being higher than 0.85 (Cshard>0.85) and the predicted traffic value Pshard exceeding the current capacity by more than 1.2 times (indicating overload of more than 20%), i.e., Cshard>0.85 and Pshard>1.2 × capacity_shard, the cloud cluster auto-scaling system immediately constructs a scaling instruction Q. capacity_shard is the current capacity of the shard.

[0141] The scaling instruction Q includes all the overloaded shards that are about to be overloaded, the number of new nodes needed to address the overload, and the traffic switching method. Thus, the cloud cluster automatic scaling system transforms the prediction results of the TGNN model into executable scaling parameters, specifying "where to scale, how much to scale, and how to scale."

[0142] The expansion command Q is sent to the pre-configured pipeline, and the shadow node is started within 30 seconds and the grayscale traffic migration begins.

[0143] In this preferred embodiment, the pre-built pipeline pulls up shadow nodes through the cloud interface (API) and starts lightweight instances in parallel, allowing each node device in the cloud cluster to "arrive first" and avoid waiting for expansion.

[0144] After receiving the expansion command Q, the pre-configured pipeline uses the cloud controller to create the required number of shadow nodes in parallel.

[0145] Once the shadow node starts up, it immediately registers itself as a nominal role with the master node. This nominal role does not allocate shards or participate in writes; it only maintains the cluster topology heartbeat, thereby completing the new node warm-up, health check, and version verification without interrupting business operations.

[0146] When a shadow node starts, parameter injection is performed first. The cloud controller generates a configuration file (an elasticsearch.yml fragment) based on the scaling command Q and writes it to the shadow node's local configuration directory within one second. Based on this, the shadow node is created and its functionality and permissions are set. Simultaneously, a temporary JVM system variable, Des.shadow, is injected into the shadow node and set to true, serving as an "invisible label" for subsequent internal distinctions, differentiating the shadow node from the original production node. This label exists only in the process memory and is lost upon restart, preventing it from being persisted to disk and causing dirty configurations.

[0147] Next, the shadow node sequentially uses its information and the current cluster state as parameters to perform verification of core security rules, thus achieving a name-based handshake. This verification mechanism provides underlying security for the fully automated scaling process, ensuring that no invalid or abnormally configured node can interfere with the stable operation of the production cluster.

[0148] During the naming handshake phase, the pre-configured pipeline can set up in-memory, non-persistent, instantaneous custom metadata for shadow nodes. This instantaneous custom metadata is stored only in server memory (in-memory state), without being persisted to disk or serialized to a file recording cluster status. Therefore, after the naming handshake, the shadow node allows the master node and data nodes in the cluster to perceive its existence, resource metrics (CPU / memory), and "named but not yet officially registered" status within milliseconds without being serialized to disk, ensuring transparent and interference-free business indexing. Furthermore, this instantaneous custom metadata records the shadow node's key status data (such as health metrics). This instantaneous custom metadata enables the cloud cluster auto-scaling system to access key statuses with microsecond-level latency, thereby supporting the cloud cluster auto-scaling system to complete zero-interruption business scaling within 30 seconds.

[0149] Then, to prevent the pre-configured pipeline from automatically deploying shards to shadow nodes in the background AllocationService, the pre-configured pipeline also enables a triple lock for shard allocation. If any lock fails, an alarm is triggered and the node is forcibly reclaimed, ensuring that shadow nodes are absolutely prohibited from being allocated any primary or replica shards before being officially recognized. The triple lock can include a configuration lock, a tag lock, and a weight lock.

[0150] Afterwards, the shadow node can send heartbeat requests to the master node every 1 second. The master node only treats the shadow node as a "client-data" type node that maintains a heartbeat, is connected to the cluster, but is not assigned a shard, and performs liveness probing. Therefore, when the business write traffic (such as the number of transactions per second, TPS) increases sharply, the shadow node will not become a bottleneck in the coordination layer, achieving "zero-disturbance" warm-up.

[0151] Once the gray-scale traffic migration is initiated, the pre-configured pipeline can assess the health metrics of shadow nodes. When a shadow node is healthy and the confidence level of the predicted traffic value for its shard overload during the migration process consistently exceeds the trust threshold, the pre-configured pipeline can convert the shadow node to a formal node after the gray-scale migration is complete. Specifically, the pre-configured pipeline can perform the following atomic operations within 1 second using the cloud controller to convert a shadow node to a formal node: remove the temporary JVM system variable `Des.shadow` and perform a rolling restart of the shadow node; issue commands to delete temporary restriction rules such as tag locks; replace the `shadow-` prefix in the node name with `formal-` and rejoin the cluster; update the status data in the instantaneous custom metadata to `formal_nodes` (formal nodes); and record the version number for subsequent scaling down or audit tracing.

[0152] Through the aforementioned pre-configured pipeline, the shadow node's startup, naming handshake, triple lock configuration, heartbeat maintenance, and official status operations are implemented. The shadow node completes the entire closed loop of naming registration, health warm-up, and instant official status without interrupting existing cluster writes, triggering shard rebalancing, or increasing the voting burden on the master node. Furthermore, by implementing parallel operations of shadow nodes and reducing the creation, startup, and official status time of shadow nodes, the time consumed by horizontal scaling is reduced from "hours" to "minutes," thereby ensuring the achievement of the "zero-interruption minute-level scaling" goal of the cloud cluster automatic scaling method of this invention.

[0153] After the shadow node is started, the cloud cluster automatic expansion system can continue to achieve shard-level gray-scale traffic migration through the gray-scale deployment pipeline.

[0154] Preferably, during the gray-scale traffic migration process, the cloud cluster automatic expansion system can use filtering rules (include._name) to limit the set of new nodes that shadow nodes are converted into new official nodes, and then use the single node shard limit rule (index.routing.allocation.total_shards_per_node) to prevent shadow node hotspots.

[0155] Please continue to refer to this. Figure 4After the weighted gray-scale rollout, shadow nodes are connected to the overloaded shard replicas in stages: 10%, 50%, and 100%. The switching between each stage can be done in 30-second increments. Utilizing the traffic switching method based on weighted switching to achieve gray-scale traffic migration ensures that any anomalies are only exposed on a small scale and can be quickly rolled back.

[0156] Specifically, the cloud cluster automatic scaling system determines the set of shard replicas S = {Shar} based on the overloaded shards indicated by the scaling instruction Q. Shar Shar }

[0157] In the first phase, the cloud cluster auto-scaling system calculates the number of replica shards for the initial migration based on a 10% weight. , will the first The replicas of each shard are migrated to the shadow node. At the same time, to prevent instantaneous overload, a shard limit can be set for the shadow node in the first phase to limit the number of shards that a single node can initially carry.

[0158] After the first phase of stable operation and passing a health check (lasting at least 30 seconds), the cloud cluster auto-scaling system enters the second phase. In the second phase, the cloud cluster auto-scaling system removes the shard limit and triggers a second round of migration based on a 50% weight, allowing shadow nodes to handle approximately 50% of the traffic.

[0159] After the second phase of stable operation and passing the health check (lasting at least 30 seconds), the cloud cluster automatic scaling system migrates all remaining shard replicas to the shadow nodes, enabling the shadow nodes to hold all shard replicas in set S and completely take over all its traffic.

[0160] During the gray-scale traffic migration, the cloud cluster's automatic scaling system can collect health metrics parameters for each node every 30 seconds. These health metrics parameters can include write round-trip time (RTT), disk wait time (IO-wait), and CPU steal time. When any of these metrics increases by 15% from the baseline, an abnormal performance can be identified, and a rollback operation can be initiated immediately.

[0161] like Figure 4As shown, the cloud cluster auto-scaling system can pull the shard weight of a shadow node with abnormal performance back to 0, that is, stop the shadow node from continuing to connect to the shard replica of the overloaded shard. Then, it calls the cloud interface to complete the reclamation of the abnormal node within 30 seconds. By deeply coupling anomaly detection and resource reclamation, it forms a "zero-interruption rollback" to solve the problem that traditional canary scaling cannot remove abnormal nodes in a timely manner. The cloud cluster auto-scaling system can also synchronously write the current shadow node into the in-memory hash table at the TGNN prediction and inference layer, and remove the node from the available vertex set in the next dynamic graph construction phase. In addition, the cloud cluster auto-scaling system can also set the weight coefficient of the shadow node to 0, so that it no longer participates in subsequent traffic prediction and allocation decisions, avoiding rescheduling.

[0162] After the three-stage verification is passed (i.e., no anomalies occur in any of the three stages), the shadow node is automatically mounted to the data disk and becomes a formal node, thus completing the cloud cluster expansion. Preferably, each new formal node can release excess resources to minimize continuous cloud costs. The cloud cluster automatic expansion system can record the metadata version of this shadow node conversion to a formal node, serving as a basis for subsequent scaling down or fault tracking.

[0163] Thus, the cloud cluster automatic expansion method provided by this invention achieves shard-level traffic prediction through the TGNN model, identifies bottlenecks 5 minutes in advance, parallelizes machine preparation actions by using shadow node pre-setting, and achieves zero-interruption traffic migration by using a three-stage weighted online deployment. The prediction pipeline, pre-setting pipeline, and gray-scale deployment pipeline are sequentially connected in time and closed-loop in real time in terms of data. Through the tight coupling of the three pipelines and the data closure loop, the time taken for horizontal expansion is fundamentally compressed from "hours" to "minutes", and the business is unaware of it and there is zero interruption.

[0164] In summary, the automatic cloud cluster expansion method provided by this invention can solve the pain points of "delayed triggering and time-consuming interruption" in existing cloud cluster expansion, avoid the low efficiency of manual expansion, improve the efficiency of distributed cloud cluster expansion, and achieve horizontal expansion with zero business awareness.

[0165] Although the methods described above are illustrated and depicted as a series of actions for the sake of simplicity, it should be understood and appreciated that these methods are not limited by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but which may be understood by those skilled in the art, according to one or more embodiments.

[0166] Those skilled in the art will understand that information, signals, and data can be represented using any of a variety of different techniques and skills. For example, the data, instructions, commands, information, signals, bits, symbols, and chips described throughout the above description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof.

[0167] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.

[0168] The various illustrative logic modules and circuits described in conjunction with the embodiments disclosed herein may be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in alternatives, it may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.

[0169] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read and write information to / from the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0170] In one or more exemplary embodiments, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functionality may be stored or transmitted as one or more instructions or code on or through a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, encompassing any medium that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible to a computer. Any connection is also legitimately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. As used in this article, disk and disc include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0171] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for automatic scaling up of a cloud cluster, characterized in that, Including the following steps: Acquire and parse message traffic data and physical node topology data within the first time window to determine at least one fragment on each node corresponding to each message; A dynamic graph structure is formed based on at least one fragment on each node corresponding to each message and the relationship between the fragments; Using the dynamic graph structure as input to the TGNN model, the TGNN model outputs the predicted flow value of each segment in the second time period and the corresponding confidence level. The confidence level is determined based on the uncertainty estimation, historical error estimation and data completeness of the TGNN model. Construct expansion instructions that indicate overloaded shards, the number of new nodes, and the traffic switching method. The overloaded shard is the shard whose predicted traffic value is greater than the overload threshold and whose confidence level is greater than the trust threshold. The number of new nodes is determined based on the predicted traffic value of the overloaded shard and the resource status of the nodes. Shadow nodes are created and started in parallel based on the number of newly added nodes, and the shadow nodes are prohibited from being allocated shards; as well as After migrating the shard replicas of the overloaded shard to the shadow node in stages according to the traffic switching method, the shadow node is then turned into a legitimate node to complete the cloud cluster expansion.

2. The cloud cluster automatic scaling method as described in claim 1, characterized in that, The step of forming a dynamic graph structure based on at least one fragment on each node corresponding to each message and the relationship between the fragments includes: The vertices of the dynamic graph structure are determined based on the index information of each message, the fragments, and the triples formed by the physical nodes where the fragments reside; and The edges of the dynamic graph structure are determined based on the data flow relationships between the fragments.

3. The cloud cluster automatic scaling method as described in claim 2, characterized in that, The step of determining the vertices of the dynamic graph structure based on the index information of each message, the fragment, and the triple formed by the physical node where the fragment is located includes: Based on the message traffic data and physical node topology data collected within the first time window, the feature vector of the vertex is determined, and the feature vector includes traffic features, node resource features and topology features.

4. The cloud cluster automatic scaling method as described in claim 1, characterized in that, The TGNN model includes: A spatial convolutional layer, which uses a graph attention network or a graph convolutional network to aggregate first-order neighbor information for each vertex of the dynamic graph structure to capture local flow dependencies; and Temporal convolutional layers, which use causal dilated convolutions or temporal self-attention to capture long-term dependencies.

5. The cloud cluster automatic scaling method as described in claim 1, characterized in that, The steps for determining the confidence level include: A Dropout layer is integrated before each convolutional and fully connected layer of the TGNN model. The Dropout layer is used to perform multiple forward propagation inferences on the input to obtain the probability distribution of the predicted flow value. The uncertainty variance of the TGNN model is obtained based on the probability distribution of the predicted flow value, which characterizes the uncertainty estimate of the TGNN model. The historical error estimate is characterized by the historical prediction performance of the shard, and the historical prediction performance is determined based on the historical predicted traffic value and the historical actual traffic value of the shard. The data quality of message traffic data and physical node topology data related to the shard within the first time window is evaluated to characterize the data integrity; and The confidence level is determined based on the uncertainty estimation of the TGNN model, the historical error estimation, and the data completeness.

6. The cloud cluster automatic scaling method as described in claim 1, characterized in that, The shadow node prevents the allocation of fragments to the shadow node through a triple lock, which includes a configuration lock, a tag lock, and a weight lock.

7. The cloud cluster automatic scaling method as described in claim 1, characterized in that, The step of migrating the shard replicas of the overloaded shard to the shadow node in stages according to the traffic switching method includes: The shadow node determines the proportion of shard replicas of the overloaded shard to be accessed based on the shard weight corresponding to each stage of the traffic switching method. The shard weight corresponding to each stage increases until the shard weight is 100%.

8. The cloud cluster automatic scaling method as described in claim 7, characterized in that, The step of migrating the shard replicas of the overloaded shard to the shadow node in stages according to the traffic switching method includes the following steps: Obtain the health metrics of the shadow nodes in each stage, including write round-trip latency, disk wait percentage, and CPU preemption percentage; and In response to a health metric indicating abnormal performance of the shadow node, the shadow node is reclaimed.

9. A cloud cluster automatic expansion system, characterized in that, include: Memory, on which computer instructions are stored; as well as A processor, connected to the memory, and configured to execute computer instructions stored on the memory to implement the cloud cluster automatic scaling method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, the automatic scaling method for cloud clusters as described in any one of claims 1 to 8 is implemented.

Images