A method, apparatus, device, medium, and product for coordinating node scheduling

Abstract

This application provides a method, apparatus, device, medium, and product for coordinating node scheduling, which can be applied to the fields of distributed technology, artificial intelligence technology, and fintech. The method includes: determining the overall load for a set of coordinating nodes; using any coordinating node in the set of coordinating nodes to coordinate distributed transactions; if the determined overall load is greater than a current load expansion threshold, expanding the current set of coordinating nodes and lowering the current load reduction threshold; if the determined overall load is less than the current load reduction threshold, reducing the current set of coordinating nodes and raising the current load expansion threshold.

Description

Technical Field

[0001] This application relates to the fields of distributed technology, artificial intelligence technology, and fintech, and specifically to a method, apparatus, device, medium, and product for coordinating node scheduling. Background Technology

[0002] In the processing of distributed transactions, coordinating nodes can be used to coordinate the handling of distributed transactions. Based on the status of the sub-transactions within a distributed transaction, they determine whether all transactions should be committed successfully or rolled back, thereby improving the consistency of the distributed transaction. Currently, multiple coordinating nodes are typically used to process different distributed transactions in parallel, improving the efficiency of distributed transaction processing.

[0003] However, with a fixed number of coordinating nodes, the adaptability to constantly changing load conditions is often low. When the number of distributed transactions is large, the coordinating nodes are under high load and prone to failure; when the number of distributed transactions is small, there is a lot of resource waste in the coordinating nodes. Summary of the Invention

[0004] In view of the above problems, embodiments of this application provide a coordination node scheduling method, apparatus, device, medium and product, which can improve the adaptability of coordination nodes to different load conditions.

[0005] According to a first aspect of this application, a method for coordinating node scheduling is provided, comprising: determining the overall load for a set of coordinating nodes; using any coordinating node in the set of coordinating nodes to coordinate distributed transactions; if the determined overall load is greater than a current load expansion threshold, expanding the nodes in the current set of coordinating nodes and lowering the current load reduction threshold; if the determined overall load is less than the current load reduction threshold, reducing the nodes in the current set of coordinating nodes and raising the current load expansion threshold.

[0006] According to an embodiment of this application, expanding the node capacity of the current set of coordination nodes includes: adding a first number of coordination nodes to the current set of coordination nodes; the first number is determined based on the absolute value of a first difference between the determined overall load and the current load expansion threshold; the first number is positively correlated with the absolute value of the first difference. Shrinking the node capacity of the current set of coordination nodes includes: reducing a second number of coordination nodes in the current set of coordination nodes; the second number is determined based on the absolute value of a second difference between the current load shrinkage threshold and the determined overall load; the second number is positively correlated with the absolute value of the second difference.

[0007] According to an embodiment of this application, the method for determining the overall load includes: determining the current actual overall load of the current set of coordination nodes; determining the future predicted overall load of the current set of coordination nodes; and determining the overall load based on the determined current actual overall load and the future predicted overall load.

[0008] According to an embodiment of this application, determining the future predicted overall load of the current coordination node set includes: predicting and determining the initial predicted overall load of the coordination node set based on the actual overall load time series of the coordination node set in different historical periods; and determining the future predicted overall load of the coordination node set by combining the determined initial predicted overall loads.

[0009] According to an embodiment of this application, the actual overall load is determined based on at least one of the following: the number of distributed transactions processed by the coordination node set per unit time; the resource utilization rate of the coordination node set; the statistical value of the distributed transaction response time of the coordination node set; and the number of distributed transactions waiting to be processed in the coordination node set.

[0010] According to an embodiment of this application, the step of predicting and determining the initial predicted overall load of the coordination node set based on the actual overall load time series sequence of the coordination node set in different historical periods includes: determining the corresponding historical smoothed load sequence based on the actual overall load time series sequence of the coordination node set in different historical periods; predicting and determining the initial predicted overall load of the coordination node set based on different historical smoothed load sequences; the historical smoothed load at a specified historical moment in any historical smoothed load sequence is determined based on the weighted sum of different actual overall loads within a fixed duration before the specified historical moment in the corresponding actual overall load time series sequence, wherein the duration between the actual overall load corresponding historical moment and the specified historical moment is negatively correlated with the weight of the actual overall load.

[0011] According to embodiments of this application, reducing the current load scaling threshold includes: reducing the current load scaling threshold based on the number of coordinating nodes in the current coordinating node set after node scaling; if the determined overall load is greater than the current load scaling threshold, the method further includes: adjusting the current load scaling threshold based on the number of coordinating nodes in the current coordinating node set after node scaling; increasing the current load scaling threshold includes: increasing the current load scaling threshold based on the number of coordinating nodes in the current coordinating node set after node scaling; if the determined overall load is less than the current load scaling threshold, the method further includes: adjusting the current load scaling threshold based on the number of coordinating nodes in the current coordinating node set after node scaling.

[0012] According to an embodiment of this application, the node scaling down for the current set of coordination nodes includes: scaling down coordination nodes in the current set of coordination nodes that meet preset scaling down conditions; the preset scaling down conditions include at least one of the following: the node load is lower than a preset node load threshold; the number of distributed transactions processed by the node per unit time is less than a preset transaction number threshold; the rollback rate of the distributed transactions processed by the node is higher than a preset rollback rate threshold; the statistical value of the distributed transaction response time of the node is higher than a preset duration threshold; and the node stability is lower than a preset stability threshold.

[0013] According to an embodiment of this application, the method is applied to a control node; the control node is used to allocate received distributed transactions to coordination nodes in the current coordination node set for processing; determining the overall load for the coordination node set includes: predicting the number of future distributed transactions to be received based on a historical distributed transaction time sequence; the historical distributed transaction time sequence includes the number of distributed transactions received by the control node at different historical times; and determining the overall load for the current coordination node set based on the predicted number of future distributed transactions.

[0014] According to a second aspect of this application, a coordination node scheduling device is provided, comprising: an overall load module, configured to determine an overall load for a set of coordination nodes; any coordination node in the set of coordination nodes is used to coordinate the processing of distributed transactions; an expansion module, configured to expand the node capacity of the current set of coordination nodes and reduce the current load reduction threshold when the determined overall load is greater than the current load expansion threshold; and a reduction module, configured to reduce the node capacity of the current set of coordination nodes and increase the current load expansion threshold when the determined overall load is less than the current load reduction threshold.

[0015] According to a third aspect of this application, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0016] According to a fourth aspect of this application, a computer-readable storage medium is also provided, on which a computer program or instructions are stored, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0017] According to a fifth aspect of this application, a computer program product is also provided, comprising a computer program or instructions that, when executed by a processor, implement the steps of the above-described method. Attached Figure Description

[0018] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0019] Figure 1 This diagram illustrates an application scenario of a coordination node scheduling method according to an embodiment of this application.

[0020] Figure 2 A flowchart illustrating a coordination node scheduling method according to an embodiment of this application is shown schematically.

[0021] Figure 3 This schematic diagram illustrates a structural block diagram of a coordination node scheduling device according to an embodiment of the present application;

[0022] Figure 4 A block diagram schematically illustrates an electronic device suitable for implementing a coordinated node scheduling method according to an embodiment of this application. Detailed Implementation

[0023] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0024] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0025] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0026] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0027] In the processing of distributed transactions, coordinating nodes are used to coordinate the handling of distributed transactions. Based on the status of sub-transactions within a distributed transaction, they determine whether all transactions should be committed successfully or rolled back, thereby improving the consistency of the distributed transaction. Currently, multiple coordinating nodes are typically used to process different distributed transactions in parallel to improve the efficiency of distributed transaction processing. However, with a fixed number of coordinating nodes, the adaptability to constantly changing load conditions is often low. When the number of distributed transactions is large, the coordinating nodes experience high load and are prone to failure; when the number of distributed transactions is small, there is significant resource waste in the coordinating nodes.

[0028] To address the aforementioned technical problems, embodiments of this application provide a method for coordinating node scheduling.

[0029] In this method, the coordination node set can be scaled up or down based on its overall load, such as the total load or average load of each coordination node. Specifically, load scaling up and down thresholds can be set, with the scaling up threshold being higher than the scaling down threshold. When the overall load of the coordination node set exceeds the scaling up threshold, node scaling up is performed to add more coordination nodes and distribute the load. When the overall load of the coordination node set is less than the scaling down threshold, node scaling down is performed to reduce the number of coordination nodes, thereby improving resource utilization and reducing resource waste.

[0030] Therefore, the above method can expand or shrink the capacity by flexibly adapting to the overall load situation, improve the adaptability of the coordinating nodes to different load situations, improve the adaptability of the number of coordinating nodes in the coordinating node set to different load situations, and improve the flexibility of the coordinating node set.

[0031] Furthermore, the fixed load expansion and contraction thresholds in the above method may not be suitable for situations with large load fluctuations. For example, based on fixed load expansion and contraction thresholds, if large load fluctuations occur, frequently exceeding the load expansion threshold and falling below the load contraction threshold, it is easy to frequently result in fluctuations in the coordination node, such as expanding and then contracting or contracting and then expanding, leading to a waste of scheduling resources of the coordination node.

[0032] Based on the situation considered in the above analysis, the load expansion threshold and load reduction threshold can be further dynamically adjusted in the above method to adapt to different load conditions, improve the adaptability of the number of coordination nodes in the coordination node set to the load conditions, and improve the flexibility of the coordination node set.

[0033] The above method does not limit the specific way to adjust the load expansion threshold and load reduction threshold.

[0034] Optionally, after scaling up a node, the load scaling-down threshold can be reduced accordingly. This ensures that the overall load on the set of coordinated nodes after scaling up must be less than the reduced load scaling-down threshold before the corresponding node scaling-down can be triggered. This reduces the frequency of scaling up and then scaling down, and minimizes the waste of coordinated node scheduling resources.

[0035] Optionally, after scaling down a node, the load expansion threshold can be increased accordingly. This ensures that the overall load on the set of coordinated nodes after scaling down must be greater than the increased load expansion threshold before corresponding node expansion can be triggered. This reduces the frequency of scaling down and then expanding, and minimizes the waste of coordinated node scheduling resources.

[0036] Of course, other methods can also be used to adjust the thresholds. For example, the load scaling-up threshold and load shrinking threshold can be adjusted accordingly based on the current number of coordinating nodes in the coordinating node set. Specifically, the correspondence between the number of coordinating nodes and the thresholds can be preset. In particular, different ranges of the number of coordinating nodes can correspond to different standard load scaling-up thresholds, or they can correspond to different standard load shrinking thresholds.

[0037] Therefore, after scaling up nodes, the load scaling down threshold can be lowered or adjusted. Specifically, the load scaling up threshold can be adjusted to the standard load scaling up threshold corresponding to the range of current coordinating nodes in the coordinating node set after scaling up. Conversely, after scaling down nodes, the load scaling down threshold can be raised or adjusted. Again, the load scaling down threshold can be adjusted to the standard load scaling down threshold corresponding to the range of current coordinating nodes in the coordinating node set after scaling down.

[0038] The above method does not limit the way the overall load of the coordination node set is determined. The overall load can be used to determine whether scaling up or down is necessary; therefore, improving the accuracy of the overall load can correspondingly improve the accuracy of scaling up or down.

[0039] Optionally, multiple metrics from the coordination node set can be directly integrated to determine the overall load situation. These metrics include, for example, the distributed transaction throughput of the coordination node set, the computing resource utilization of the coordination node set, the average distributed transaction response time of the coordination node set, and the number of distributed transactions waiting to be processed in the coordination node set. Determining the overall load situation through multi-faceted metric data can improve the accuracy and comprehensiveness of the overall load situation.

[0040] Optionally, historical load time-series data can be combined to predict the future overall load of the coordination node set, thus determining the overall load situation. Predicting the future overall load situation can improve the accuracy of the overall load forecast and allow for early determination of scaling up and down requirements. Specific prediction methods can combine various approaches, such as prediction based on the rate of change of load indicators, model prediction (e.g., sequence prediction models), and prediction based on the periodic variation characteristics of the load. Specifically, predictions can be made separately for historical load time-series sequences of different historical periods and durations, and then the results of these predictions can be combined to obtain the future overall load situation, thereby improving the accuracy of the predicted future overall load situation.

[0041] For example, based on the historical one-hour overall load change time series, the first future overall load can be predicted; based on the historical 10-day overall load change time series, the second future overall load can be predicted; and based on the historical 100-day overall load change time series, the third future overall load can be predicted. By making predictions based on overall load change time series of different historical periods and durations, multiple dimensions of overall load change characteristics (including load periodicity characteristics) can be determined, improving the comprehensiveness and accuracy of load prediction. In particular, for distributed transactions, which typically include many periodically triggered distributed transactions, load periodicity characteristics can be extracted, thereby improving the accuracy of load prediction.

[0042] Alternatively, the overall load situation can be determined by combining the current actual overall load situation with the predicted future overall load situation. Determining the overall load situation from both the current actual load and the future load perspective can improve the accuracy and comprehensiveness of the overall load situation.

[0043] In the above method, when expanding or shrinking a node, the number of coordinating nodes adjusted can be determined based on the difference between the overall load and the corresponding threshold, thereby improving the accuracy of node expansion or shrinkage. For example, if the overall load significantly exceeds the load expansion threshold, the number of coordinating nodes can be increased accordingly; if the overall load significantly falls below the load shrinkage threshold, the number of coordinating nodes can be decreased accordingly.

[0044] In the above method, when scaling down a node, coordinating nodes can be randomly selected for deletion or removal, or specific coordinating nodes can be selected for deletion or removal. For example, the specific situation of each coordinating node in the current set can be analyzed, and coordinating nodes with poor stability and frequent failures, or those that are slow in processing distributed transactions, can be deleted to improve the overall distributed transaction processing efficiency of the coordinating node set.

[0045] Of course, when expanding the number of nodes, better coordinating nodes can be selected and added to the set of coordinating nodes.

[0046] It should be noted that the coordination node scheduling method and apparatus provided in the embodiments of this application can be used in the fields of distributed technology, artificial intelligence technology, and fintech. For example, for distributed transactions and coordination nodes (coordinators) in financial institutions such as banks, the coordination node scheduling method provided in the embodiments of this application can be used for corresponding scaling up and down to improve the adaptability of the coordination node set to the current overall load. The coordination node scheduling method and apparatus provided in the embodiments of this application can also be used in any field other than fintech, and the application field of the coordination node scheduling method and apparatus provided in the embodiments of this application is not limited.

[0047] As used in this paper, the term "model" refers to a model that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs using multiple layers of processing units. A neural network model is an example of a deep learning-based model. In this paper, "model" may also be referred to as a "machine learning model," "learning model," "machine learning network," or "learning network," and these terms are used interchangeably.

[0048] It's important to note that the term "neural network" can refer to a machine learning network based on deep learning. A neural network processes input and provides corresponding output, typically consisting of an input layer, an output layer, and one or more hidden layers between them. Neural networks used in deep learning applications often include many hidden layers, increasing the network's depth. The layers of a neural network are connected sequentially, so that the output of the previous layer serves as the input to the next layer. The input layer receives the input to the neural network, while the output layer's output becomes the final output. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each processing the input from the layer above.

[0049] It should be understood that machine learning generally includes three phases: training, testing, and application (also known as inference). In the training phase, a given model is trained using a large amount of training data, iteratively updating parameter values ​​until the model can consistently generate inferences that meet the expected goals from the training data. Through training, the model can be considered to have learned the relationship between inputs and outputs (also known as the input-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing phase, test inputs are applied to the trained model to test whether it can provide the correct output, thus determining the model's performance. In the application phase, the model can be used to process actual inputs based on the trained parameter values ​​to determine the corresponding output.

[0050] Figure 1 The diagram illustrates an application scenario of a coordination node scheduling method according to an embodiment of this application. Figure 1 As shown, application scenario 100 according to an embodiment of this application may include a first coordinating node 101, a second coordinating node 102, a third coordinating node 103, a network 104, and a control node 105. The network 104 serves as a medium for providing communication links between the first coordinating node 101, the second coordinating node 102, the third coordinating node 103, and the control node 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0051] The first coordinating node 101, the second coordinating node 102, and the third coordinating node 103 can be coordinators or coordinating devices used to handle distributed transactions. They can be core components in a distributed system that ensure the atomicity of distributed transactions and can be used to ensure that all sub-transactions in a distributed transaction are successfully committed or rolled back, thus maintaining data consistency in a distributed system.

[0052] Control node 105 can be used to control multiple coordinating nodes, determine the load status of each coordinating node, and perform node scaling up or down accordingly. It can also be used to route and distribute distributed transactions to each coordinating node. Specifically, control node 105 can be a server or a service-oriented application.

[0053] It should be noted that the coordination node scheduling method provided in this application embodiment can generally be executed by the control node 105. Accordingly, the coordination node scheduling device provided in this application embodiment can generally be set in the control node 105.

[0054] It should be understood that Figure 1 The number of coordinating nodes, network nodes, and control nodes shown is merely illustrative. Depending on implementation needs, any number of coordinating nodes, network nodes, and control nodes can be included.

[0055] Figure 2 A flowchart illustrating a coordination node scheduling method according to an embodiment of this application is shown schematically. Figure 2 As shown, a coordination node scheduling method 200 according to an embodiment of this application may include steps S210 to S230.

[0056] This application does not limit the execution subject of a coordination node scheduling method; it can be applied to any electronic device or software application. Optionally, a coordination node scheduling method can be applied to a control node, which can be used to control the coordination node or a set of coordination nodes.

[0057] In step S210, the overall load is determined for the set of coordination nodes; any coordination node in the set of coordination nodes is used to coordinate the processing of distributed transactions.

[0058] In step S220, if the determined overall load is greater than the current load expansion threshold, the current set of coordination nodes is expanded, and the current load reduction threshold is lowered.

[0059] In step S230, if the determined overall load is less than the current load scaling-down threshold, the current set of coordination nodes is scaled down, and the current load scaling-up threshold is increased.

[0060] This method allows for flexible scaling up or down of the coordination node set to adapt to the overall load, improving the adaptability of coordination nodes to different overall loads and increasing the adaptability of the number of coordination nodes in the set to different overall loads, thus enhancing the flexibility of the coordination node set. Furthermore, by adjusting thresholds after node scaling up or down, this method can reduce the frequent occurrence of scaling up followed by scaling down or vice versa, minimizing wasted scheduling resources for coordination nodes.

[0061] In explaining this method, we can deeply integrate the requirements of distributed transaction processing scenarios to schedule coordination nodes and improve the adaptability of the coordination node set to changing load conditions.

[0062] The embodiments of this application do not limit the coordinating node. Optionally, the coordinating node can be a distributed transaction coordinator, which can be used to coordinate the processing of distributed transactions. In a specific example, the coordinating node can monitor the status of each sub-transaction in any distributed transaction. If it determines that each sub-transaction has been successfully committed, it determines that the distributed transaction has been successfully committed. If it determines that any sub-transaction has failed to commit or has timed out, it can roll back each sub-transaction. If it determines that the distributed transaction has failed to commit, it can roll back, thereby improving the consistency of distributed transactions.

[0063] The embodiments of this application do not limit the set of coordinating nodes. Optionally, one or more coordinating nodes can be added to the set of coordinating nodes, allowing different distributed transactions to be processed in parallel by different coordinating nodes in the set, thereby improving the processing efficiency of distributed transactions. For the set of coordinating nodes, the number of nodes can be increased or decreased according to the overall load situation to adapt to the overall load. It is understood that different coordinating nodes in the set can communicate with the control node separately, and the control node can perform node expansion or contraction to determine the overall load situation of the set of coordinating nodes. Alternatively, the control node can assign distributed transactions to the coordinating nodes in the set. Optionally, the set of coordinating nodes can be defined as the set of coordinating nodes controlled by the control node.

[0064] The embodiments of this application do not limit the control node. Optionally, the control node can be a control device or a control server, which can be used to control the coordination nodes in the coordination node set, determine the overall load of the coordination node set to determine whether to perform node expansion or reduction, and perform node expansion or reduction for the coordination node set. That is, to add a new coordination node to the coordination node set and establish a communication connection between the control node and the new coordination node to achieve node expansion, or to delete, take offline, or disconnect the communication connection between the coordination node and the control node in the coordination node set to achieve node reduction. Optionally, the control node can also receive distributed transactions and distribute the received distributed transactions to the coordination nodes in the coordination node set. Specifically, load balancing and other strategies can be used to distribute distributed transactions.

[0065] The embodiments of this application do not limit the overall load. Optionally, for the overall load of the coordination node set, the overall load can be determined by comprehensively considering the load information of each coordination node in the coordination node set. Specifically, the overall load can be determined by the load statistics of each coordination node in the coordination node set (such as the total load, the weighted sum of loads, the average load, the weighted average load, etc.).

[0066] The embodiments of this application do not limit the specific form of the load. Specifically, it can be characterized by the load amount or the proportion of the load amount to the maximum load amount.

[0067] The embodiments of this application do not limit the specific method for determining the load of the coordinating node. The load of the coordinating node can be determined by comprehensively considering one or more indicators of the coordinating node. Examples of coordinating node indicators include: resource utilization (computation resource utilization, storage resource utilization, and communication resource utilization); distributed transaction response time statistics (average response time); distributed transaction throughput (the number of distributed transactions processed by the coordinating node per unit time); and the number of distributed transactions waiting to be processed in the coordinating node. Of course, the indicators of the coordinating node can also be combined into indicators for the entire set of coordinating nodes to determine the load of the set. Examples include: resource utilization (computation resource utilization, storage resource utilization, and communication resource utilization) of the set of coordinating nodes; distributed transaction response time statistics (average response time) of the set of coordinating nodes; distributed transaction throughput (the number of distributed transactions processed by the set of coordinating nodes per unit time); and the number of distributed transactions waiting to be processed in the set of coordinating nodes. It is understood that the overall load can be determined directly by combining the above indicators, or additional processing can be performed on the above indicators to determine the overall load.

[0068] Optionally, the actual load of the coordination node set can be determined by combining the above indicators. Alternatively, the future load of the coordination node set can be predicted by combining the historical time-series changes of indicators. This allows for a comprehensive determination of the overall load of the coordination node set by combining the actual and future loads, improving the accuracy and comprehensiveness of the overall load assessment. It is understood that by predicting the future load change trend of the coordination node set, adjustments to node expansion or contraction can be made in advance to adapt to future load changes, improving the adaptability of the coordination node set to load changes. For a more detailed explanation, please refer to the explanations in other embodiments.

[0069] The embodiments of this application do not limit the load scaling threshold and the load shrinking threshold. Optionally, the load scaling threshold and the load shrinking threshold can be in the form of load amount or in the form of a percentage of the maximum load amount. In a specific example, the load scaling threshold can be 90% of the maximum load amount, 85% of the maximum utilization of computing resources, or 500 distributed transactions processed per unit time.

[0070] It is understood that as node scaling up and down are executed, the number of coordinating nodes in the coordinating node set can change accordingly, and the load scaling up and down thresholds can be dynamically adjusted accordingly. The embodiments of this application do not limit the adjustment method of the load scaling up and down thresholds. In a specific example, if the number of coordinating nodes in the coordinating node set is greater than 400, the load scaling up threshold can be adjusted to 95% of the maximum load, and the load down threshold can be adjusted to 20% of the maximum load; if the number of coordinating nodes in the coordinating node set is less than 10, the load scaling up threshold can be adjusted to 85% of the maximum load, and the load down threshold can be adjusted to 10% of the maximum load. For a more detailed explanation, please refer to the explanations of other embodiments.

[0071] The embodiments of this application are not limited to node scaling up or down. The number of nodes to be adjusted for scaling up or down can be determined based on the actual overall load. When scaling down, less efficient or less effective coordinating nodes can also be selected for scaling down, thereby improving the overall distributed transaction processing efficiency and performance of the coordinating node set. For a more detailed explanation, please refer to the explanations in other embodiments.

[0072] The embodiments of this application do not limit the execution order of steps S210 to S230. Optionally, steps S220 and S230 can be executed in parallel or sequentially, depending on the conditions that are actually met.

[0073] The following section provides a detailed explanation of how the overall load is determined.

[0074] In one optional embodiment, the overall load of the coordination node set can characterize the overall load situation of the coordination node set, and be used to subsequently determine whether node scaling up or down is necessary. Optionally, the current actual load of the coordination node set can be directly determined as the overall load to characterize the current actual load situation of the coordination node set. Furthermore, the future load situation of the coordination node set can be predicted to characterize the future load increase / decrease trend of the coordination node set. Determining the predicted future overall load as the overall load facilitates the determination of whether node scaling up or down is necessary based on the predicted future load increase / decrease trend. Currently, the overall load can also be determined by combining the current actual load and the predicted future load, comprehensively determining the overall load situation from both the current actual load situation and the predicted future load increase / decrease trend, thereby improving the accuracy and comprehensiveness of the overall load.

[0075] Therefore, optionally, the method for determining the overall load includes: determining the current actual overall load of the current set of coordination nodes; determining the future predicted overall load of the current set of coordination nodes; and determining the overall load based on the determined current actual overall load and the future predicted overall load. This embodiment can determine the overall load by combining the current actual overall load and the predicted future predicted overall load, comprehensively determining the overall load from both the current actual load situation and the future predicted load increase / decrease trend, thereby improving the accuracy and comprehensiveness of the overall load.

[0076] In this application, the embodiments are not limited to predicting the overall load in the future, but can specifically be the overall load of the predicted set of coordination nodes in a future specified time window.

[0077] The embodiments of this application are not limited to the method of combining the current actual overall load and the future predicted overall load. Optionally, the overall load can be determined by the weighted sum of the current actual overall load and the future predicted overall load, or the future load increase or decrease trend can be determined based on the future predicted overall load (which can be in the form of a load time series), and the current actual overall load can be adjusted based on the determined future load increase or decrease trend to obtain the overall load.

[0078] The embodiments of this application do not limit the method for determining the actual overall load. Optionally, the actual overall load is determined based on at least one of the following: (1) the number of distributed transactions processed by the coordination node set per unit time; (2) the resource utilization rate of the coordination node set; (3) the statistical value of the distributed transaction response time of the coordination node set; and (4) the number of distributed transactions waiting to be processed in the coordination node set. This embodiment can determine the actual overall load based on multiple dimensions of indicator information, which can improve the accuracy and comprehensiveness of the actual overall load.

[0079] The actual overall load is positively correlated with each of the four pieces of information mentioned above. It can be understood that the more distributed transactions the coordination node set processes per unit of time, the higher the actual overall load; conversely, the more distributed transactions waiting to be processed in the coordination node set, the higher the actual overall load.

[0080] Specifically, the actual overall load can be determined by weighting at least two of the four pieces of information mentioned above, where the weights for each piece of information can be positive numbers; or by determining one of the four pieces of information as the actual overall load. Correspondingly, the current actual overall load can be determined based on at least one of the four pieces of information currently present in the set of coordinating nodes, or by determining the real-time overall load of the set of coordinating nodes.

[0081] The resource utilization rate of the coordination node set can specifically include the utilization rates of various resources such as computing resources, storage resources, and communication resources, which can facilitate the determination of the load status of the coordination node set from the perspective of multiple resources.

[0082] In addition, the number of distributed transactions processed by the coordination node set per unit time can be the sum of the number of distributed transactions processed by each coordination node in the coordination node set per unit time; the resource utilization rate of the coordination node set can be the average resource utilization rate of each coordination node in the coordination node set, or the ratio of the total resources occupied by each coordination node to the total resources of the coordination node set; the statistical value of the distributed transaction response time of the coordination node set can be the average or weighted average of the distributed transaction response time of each coordination node in the coordination node set; and the number of distributed transactions waiting to be processed in the coordination node set can be the sum of the number of distributed transactions waiting to be processed in each coordination node of the coordination node set.

[0083] The embodiments of this application do not limit the method of determining the overall load forecast in the future. Optionally, the overall load trend in the future can be predicted based on the historical load changes of the coordination node set, or the overall load situation in the future can be predicted, based on neural network models or deep learning models; or the overall load situation in the near future can be determined by combining the historical load change rate in a recent short period of time.

[0084] In one optional embodiment, the historical overall load change sequences of different durations in different historical periods can be combined, predicted separately, and then the multiple prediction results can be aggregated to obtain the final prediction result, which can improve the accuracy of predicting the future overall load.

[0085] Therefore, optionally, determining the future predicted overall load of the current set of coordination nodes can specifically include: predicting the initial predicted overall load of the set of coordination nodes based on the actual overall load time series sequences of the set of coordination nodes in different historical periods; and combining the determined initial predicted overall loads to determine the future predicted overall load of the set of coordination nodes. This embodiment can improve the accuracy and comprehensiveness of the future predicted overall load by predicting the future overall load based on the actual overall load change time series sequences of different historical periods and combining the different prediction results.

[0086] The embodiments of this application are not limited to different historical time periods. In a specific example, predictions can be made based on the actual overall load changes of the coordination node set in different historical time periods, such as the previous 5 minutes, the previous day, the previous 30 days, and the previous 100 days. The actual overall load for different historical time periods can be determined according to the actual overall load determination method described in the above embodiments.

[0087] The embodiments of this application do not limit the specific prediction method. Optionally, different prediction methods can be used for different historical periods. For example, for the actual overall load time series of the 5 minutes before the current moment, the overall load change rate can be analyzed for prediction; for the actual overall load time series of the 100 days before the current moment, a deep learning model or neural network model can be combined to extract the periodic characteristics of load changes, and prediction can be made based on the periodic characteristics. It is understood that distributed transactions usually include many periodically triggered distributed transactions, such as periodically checking the security of the distributed system and periodically synchronizing the time information of the distributed system. Furthermore, the number of distributed transactions can also exhibit periodic changes; for example, the number of distributed transactions during the day is usually greater than the number at night, and the number of distributed transactions on holidays is usually less than the number on weekdays, etc. Therefore, by extracting the periodic change characteristics of the load in the time series of the actual overall load changes, the accuracy of load prediction can be improved.

[0088] The embodiments of this application do not limit the method of integrating various initial predicted overall loads. Optionally, the weighted sum of various initial predicted overall loads can be used to determine the future predicted overall load. Optionally, the weights of different initial predicted overall loads can be determined according to information such as the length of the corresponding historical period or the chronological order of the corresponding historical periods. For example, the weight of the initial predicted overall load can be positively correlated with the length of the corresponding historical period; the weight of the initial predicted overall load can be negatively correlated with the duration between the corresponding historical period and the current time.

[0089] In one optional embodiment, the actual overall load time series can be smoothed first to obtain a historical smoothed load series, reducing the impact of noise data in the actual overall load time series. Then, prediction based on the historical smoothed load series can improve the accuracy of predicting the future overall load. This embodiment does not limit the specific method of smoothing; smoothing can be implemented using a sliding window or based on the entire time series.

[0090] Therefore, optionally, the initial predicted overall load of the coordination node set can be predicted and determined based on the actual overall load time series sequences of the coordination node set in different historical periods. Specifically, this can include: determining the corresponding historical smoothed load sequences based on the actual overall load time series sequences of the coordination node set in different historical periods; and predicting and determining the initial predicted overall load of the coordination node set based on different historical smoothed load sequences. The historical smoothed load at a specified historical moment in any historical smoothed load sequence can be determined based on the weighted sum of different actual overall loads within a fixed duration before the specified historical moment in the corresponding actual overall load time series sequence. The duration between the actual overall load at the corresponding historical moment and the specified historical moment can be negatively correlated with the weight of the actual overall load. This embodiment, by smoothing the actual overall load time series sequences of historical periods, can reduce the impact of noise data in the actual overall load time series sequences and improve the accuracy of predicting future overall load.

[0091] Understandably, a sliding window approach can be used to smooth the overall load time series for any historical period. The specified historical time can be any historical time. Different actual overall loads within the sliding window size prior to the specified historical time can be weighted and summed to determine the smoothed historical load for that specified historical time. This allows for sequential adjustment of the sliding window to determine the smoothed historical load for different historical times, resulting in the smoothed historical load series for the corresponding historical period, thus reducing the impact of noisy data. The fixed duration can be the size of the sliding window.

[0092] Furthermore, after determining the current actual overall load, a smoothing process can be applied to the current actual overall load to determine the overall load. Specifically, this can be done by combining the current actual overall load with the actual overall load at a fixed time prior to the current moment to determine the current smoothed overall load, and then using the current smoothed overall load and the future predicted overall load to determine the final overall load. Smoothing can reduce the impact of noisy data and improve the accuracy of the overall load.

[0093] In one alternative embodiment, the overall load can also be determined based on the total number of distributed transactions received by the coordination node set, that is, the number of distributed transactions that the coordination node set needs to process. Alternatively, the overall load can be determined by combining the predicted number of distributed transactions that the coordination node set will receive in the future.

[0094] Optionally, the above method can be applied to a control node; the control node can be used to allocate received distributed transactions to coordination nodes in the current coordination node set for processing; determining the overall load for the coordination node set can specifically include: predicting the number of future distributed transactions to be received based on historical distributed transaction time series; the historical distributed transaction time series includes the number of distributed transactions received by the control node at different historical times; and determining the overall load for the current coordination node set based on the predicted number of future distributed transactions. This embodiment can determine the overall load based on the predicted number of future distributed transactions received, which can improve the accuracy of the overall load.

[0095] The embodiments of this application are not limited to historical distributed transaction time sequence. Historical distributed transaction time sequence can be used to characterize the change sequence of the number of historical distributed transactions received by the control node, and can be used to predict the number of distributed transactions that the control node will receive in the future, so as to determine the number of distributed transactions that the coordination node set will need to process in the future, and further determine the overall load.

[0096] The embodiments of this application do not limit the method for predicting the number of future distributed transactions received. Optionally, the number of future distributed transactions can be predicted based on historical distributed transaction time-series sequences, using deep learning models or neural network models, etc.

[0097] The embodiments of this application do not limit the specific method of determining the overall load based on the number of future distributed transactions. Optionally, the overall load can be determined by determining the predicted overall load based on the number of future distributed transactions, or the overall load can be determined by combining the current actual overall load, the predicted overall load, and the number of future distributed transactions. The number of future distributed transactions may be positively correlated with the overall load; the number of future distributed transactions may be positively correlated with the predicted overall load. It is understood that the predicted number of future distributed transactions can be used as one of the bases for determining the overall load, and the overall load can be comprehensively determined in conjunction with the explanations of other embodiments.

[0098] The above embodiments explain how to determine the overall load. The following sections further explain node scaling up and node scaling down.

[0099] In one optional embodiment, the number of nodes adjusted when scaling up or down can be a fixed number of nodes or can be dynamically adjusted according to the overall load.

[0100] Optionally, expanding the current set of coordination nodes can specifically include: adding a first number of coordination nodes to the current set of coordination nodes; the first number is determined based on the absolute value of a first difference between the determined overall load and the current load expansion threshold; the first number is positively correlated with the absolute value of the first difference. Shrinking the current set of coordination nodes can specifically include: reducing a second number of coordination nodes to the current set of coordination nodes; the second number is determined based on the absolute value of a second difference between the current load shrinkage threshold and the determined overall load; the second number is positively correlated with the absolute value of the second difference. This embodiment can dynamically adjust the number of nodes for expansion or shrinkage according to the degree of difference between the overall load and the threshold, which can improve the accuracy of scheduling coordination nodes and improve the adaptability of the coordination node set to the current load situation.

[0101] Understandably, the larger the absolute value of the first difference between the overall load and the current load scaling threshold, the more the overall load exceeds the capacity of the current set of coordinating nodes. This allows for the addition of more coordinating nodes to share the load and reduce the load on individual nodes. Conversely, the larger the absolute value of the second difference between the current load scaling threshold and the determined overall load, the lower the overall load and the more wasted resources in the current set of coordinating nodes. This allows for the reduction of more coordinating nodes to improve resource utilization.

[0102] The embodiments of this application do not limit the method of selecting nodes when scaling down. Optionally, nodes with lower efficiency in handling distributed transactions or nodes with poor stability can be selected for node scaling down to improve the overall transaction processing efficiency and stability of the coordination node set.

[0103] Therefore, optionally, node scaling down can be performed on the current set of coordination nodes. Specifically, this can include scaling down coordination nodes in the current set of coordination nodes that meet preset scaling down conditions. The preset scaling down conditions can include at least one of the following: (1) the node load is lower than a preset node load threshold; (2) the number of distributed transactions processed by the node per unit time is less than a preset transaction number threshold; (3) the rollback rate of the distributed transactions processed by the node is higher than a preset rollback rate threshold; (4) the statistical value of the distributed transaction response time of the node is higher than a preset duration threshold; (5) the node stability is lower than a preset stability threshold. This embodiment can select coordination nodes with poor performance through multiple preset scaling down conditions and perform node scaling down, which can conveniently improve the overall distributed transaction processing efficiency and processing effect of the coordination node set. It can be understood that, specifically, it can be selected from coordination nodes that meet the preset scaling down conditions, and the selected coordination nodes can be deleted or taken offline from the coordination node set to achieve node scaling down. The embodiments of this application do not limit the specific preset scaling down conditions, and the above embodiments are used for illustrative purposes.

[0104] The embodiments of this application do not limit the method of determining node load. For details, please refer to the explanations of other embodiments, where the load of a single coordinating node is determined based on its metric information. For example, the node load of a single coordinating node can be determined based on its current actual load and its future predicted load. The current actual load of a single coordinating node can be determined based on at least one of the following: (1) the number of distributed transactions processed by the coordinating node per unit time; (2) the resource utilization rate of the coordinating node; (3) the statistical value of the distributed transaction response time of the coordinating node; and (4) the number of distributed transactions waiting to be processed in the coordinating node. The future predicted load of a single coordinating node can be predicted and determined based on the historical load time series of the single coordinating node. For a detailed explanation, please refer to the explanations of other embodiments, where the set of coordinating nodes in other embodiments can be replaced with a single coordinating node for predicting and determining node load.

[0105] The embodiments of this application do not limit the distributed transaction rollback rate. Optionally, distributed transaction rollback can be a situation where the entire distributed transaction is rolled back when a sub-transaction fails or times out. The distributed transaction rollback rate processed by a node can specifically be the percentage of the number of distributed transactions that are triggered for rollback by a single coordinating node in the total number of distributed transactions processed by that coordinating node.

[0106] The embodiments of this application do not limit the distributed transaction response time. Optionally, the distributed transaction response time can be the time taken for a single coordinating node to process a single distributed transaction from start to finish, which can characterize the efficiency of the coordinating node in processing a single distributed transaction. Accordingly, the statistical value of the distributed transaction response time can be the average or weighted average determined among the response times of a single coordinating node processing different distributed transactions. The statistical value of the distributed transaction response time can more comprehensively and accurately characterize the efficiency of a single coordinating node in processing distributed transactions.

[0107] The embodiments of this application do not limit the method for determining node stability. Specifically, the stability of the coordinating node can be determined based on its historical failures, communication disconnections, and historical anomalies. The fewer failures and anomalies a coordinating node experiences, the higher its stability.

[0108] Correspondingly, when expanding the node set, coordinating nodes with better distributed processing performance can be selected and added to the coordinating node set. Optionally, expanding the node set for the current coordinating node set can specifically include: selecting nodes from coordinating nodes that meet preset expansion conditions and adding them to the current coordinating node set. The preset expansion conditions can include at least one of the following: (1) the success rate of historical distributed transactions processed by the node is higher than a preset success rate threshold; (2) the stability of the node is higher than a preset stability threshold; (3) the resource quantity of the node is greater than a preset resource quantity threshold. The embodiments of this application do not limit the specific preset expansion conditions, and the above embodiments are used for illustrative purposes.

[0109] The following section further explains the dynamic adjustment of load reduction and expansion thresholds.

[0110] The embodiments of this application do not limit the specific methods of adjusting the load shrinkage threshold and the load expansion threshold.

[0111] Optionally, if the determined overall load exceeds the current load expansion threshold, the current load reduction threshold can be lowered. Specifically, it can be lowered by a fixed value, or dynamically based on the number of coordinating nodes in the coordinating node set or the absolute value of the difference between the overall load and the load expansion threshold. It is understood that lowering the load reduction threshold can reduce the frequency of frequent expansion followed by reduction, thus reducing the waste of coordinating node scheduling resources.

[0112] Optionally, the more coordinating nodes in the coordinating node set after node expansion, the greater the reduction in the load scaling-down threshold that triggers node scaling-down, thus reducing the frequency of expansion followed by scaling-down. Therefore, optionally, the number of coordinating nodes in the current coordinating node set after node expansion can be positively correlated with the reduction in the load scaling-down threshold, or negatively correlated with the reduced load scaling-down threshold. The absolute value of the first difference between the overall load and the load expansion threshold can be positively correlated with the reduction in the load scaling-down threshold, or negatively correlated with the reduced load scaling-down threshold. The first increase during node expansion can be positively correlated with the reduction in the load scaling-down threshold, or negatively correlated with the reduced load scaling-down threshold.

[0113] Optionally, if the determined overall load exceeds the current load expansion threshold, the load expansion threshold can be adjusted accordingly. Specifically, a pre-defined correspondence between the range of coordinated nodes in the coordinated node set and the standard load expansion threshold can be established. This allows the current load expansion threshold to be adjusted based on the standard load expansion threshold corresponding to the range of coordinated nodes in the current coordinated node set after node expansion. Specifically, the standard load expansion threshold corresponding to the range of coordinated nodes in the current coordinated node set after node expansion can be determined as the current load expansion threshold.

[0114] Of course, other methods can also be used to adjust the load scaling threshold. Specifically, if the overall load is greater than the current load scaling threshold, the current load scaling threshold can be increased based on the number of coordinating nodes in the current coordinating node set after node scaling. As the number of coordinating nodes in the coordinating node set gradually increases after node scaling, the overall load-bearing capacity of the coordinating node set can gradually increase. This allows for increasing the current load scaling threshold and also reduces the frequency of scaling triggered due to load fluctuations.

[0115] Optionally, if the determined overall load is less than the current load scaling-down threshold, the current load scaling-up threshold can be increased. Specifically, it can be increased by a fixed value, or dynamically based on the number of coordinating nodes in the coordinating node set or the absolute value of the difference between the overall load and the load scaling-down threshold. It is understood that increasing the load scaling-up threshold can reduce the frequency of scaling down followed by scaling up, thus reducing the waste of coordinating node scheduling resources.

[0116] Optionally, the fewer the number of coordinating nodes in the coordinating node set after node scaling down, the greater the increase in the load expansion threshold that triggers node scaling up can be, thereby reducing the frequent occurrence of scaling down first and then scaling up. Therefore, optionally, the number of coordinating nodes in the current coordinating node set after node scaling down can be negatively correlated with the increase in the load expansion threshold. The absolute value of the second difference between the overall load and the load scaling down threshold can be positively correlated with the increase in the load expansion threshold. The second reduction during node scaling down can be positively correlated with the increase in the load expansion threshold.

[0117] Optionally, if the determined overall load is less than the current load scaling threshold, the load scaling threshold can be adjusted accordingly. Specifically, a pre-defined correspondence between the range of coordinated nodes in the coordinated node set and the standard load scaling threshold can be established. This allows the current load scaling threshold to be adjusted based on the standard load scaling threshold corresponding to the range of coordinated nodes in the current coordinated node set after node scaling. Specifically, the standard load scaling threshold corresponding to the range of coordinated nodes in the current coordinated node set after node scaling can be determined as the current load scaling threshold.

[0118] Of course, other methods can also be used to adjust the load scaling threshold. Specifically, if the overall load is less than the current load scaling threshold, the current load scaling threshold can be lowered based on the number of coordinating nodes in the current coordinating node set after node scaling. As the number of coordinating nodes in the coordinating node set gradually decreases after node scaling, the overall resource volume of the coordinating node set decreases, which can lower the current load scaling threshold and reduce the frequency of scaling down due to load fluctuations.

[0119] Therefore, optionally, lowering the current load scaling threshold can specifically include: reducing the current load scaling threshold based on the number of coordinating nodes in the current coordinating node set after node scaling. If the determined overall load is greater than the current load scaling threshold, the above method flow can further include: adjusting the current load scaling threshold based on the number of coordinating nodes in the current coordinating node set after node scaling. Raising the current load scaling threshold can specifically include: raising the current load scaling threshold based on the number of coordinating nodes in the current coordinating node set after node scaling; if the determined overall load is less than the current load scaling threshold, the above method flow can further include: adjusting the current load scaling threshold based on the number of coordinating nodes in the current coordinating node set after node scaling. This embodiment can dynamically adjust the load scaling threshold and load scaling threshold by combining the number of nodes in the coordinating node set, which can improve the flexibility and accuracy of the load scaling threshold and load scaling threshold.

[0120] For ease of understanding, this application also provides an application embodiment.

[0121] First, let's explain the concepts involved in this embodiment.

[0122] Distributed transaction coordinator (corresponding to the coordination node in the above method embodiments): The distributed transaction coordinator is the core component in a distributed system that ensures the atomicity of distributed transactions. It guarantees that each sub-transaction either all commits successfully or all rolls back, maintaining data consistency in the distributed system.

[0123] Elastic scaling up and down: refers to the system's ability to dynamically adjust resource allocation based on real-time load. Scaling up automatically increases the number of coordinating nodes to improve processing capacity under high load, while scaling down reduces the number of nodes to save resources under low load.

[0124] Transaction throughput: The number of distributed transactions processed per unit of time.

[0125] Network latency: the response time of distributed transactions.

[0126] Processor utilization: The utilization rate of computing resources of the coordinating node.

[0127] Queue depth: The backlog of distributed transactions waiting to be processed.

[0128] This embodiment proposes a dynamic scaling architecture for a distributed transaction coordination system based on intelligent decision-making. The architecture consists of three core components: a load perception module, a dual-track prediction module, and an intelligent decision engine. It achieves precise matching of resources and load through closed-loop control in three stages: real-time detection, advance prediction, and dynamic decision-making.

[0129] (a) Deep detection mechanism of load sensing module.

[0130] The load awareness module adopts a distributed probe cluster deployment architecture. Within each transaction coordination node, it accurately collects four core indicators with a sampling period of 100 milliseconds: transaction throughput is counted in real time as the number of distributed transaction requests processed per second, accurate to the single digit; network latency is calculated as the end-to-end response time of distributed transaction requests, with an accuracy of microseconds; processor utilization is obtained by reading processor performance counters to obtain core-level computing load; and transaction queue depth is measured in real time as the backlog of distributed transactions to be processed, using atomic counters to ensure concurrency accuracy.

[0131] The raw data collection process inevitably involves transient noise interference (such as operating system thread scheduling jitter, network packet retransmission, etc.), and directly using this raw data may lead to misjudgments by the decision engine. To reduce the impact of such interference, this embodiment uses a sliding window weighted filtering algorithm to smooth the raw data. This algorithm maintains a fixed time window of 10 (corresponding to a 1-second time span). When each newly collected data point enters the window, a weight coefficient is assigned according to the time decay principle: the newest data point is assigned the highest weight of 0.15, the second newest point 0.14, and so on until the oldest data point has a weight of 0.04. This weight allocation model is designed based on the principle of exponential decay, ensuring that the system remains highly sensitive to recent state changes (data within the last 500 milliseconds accounts for 68% of the weight), while retaining the reference value of historical data.

[0132] (ii) A dual-track prediction mechanism of periodic pattern prediction and real-time feature prediction.

[0133] This mechanism uses the four-dimensional smoothed data output by the load perception module as the input source to construct a periodic pattern prediction module and a real-time feature prediction module.

[0134] The periodic prediction module extracts the inherent periodic characteristics of the load based on filtered time-series data (including transaction throughput, network latency, processor utilization, and transaction queue depth) from the same period (±10-minute window) over the past 30 days. By leveraging these inherent periodic characteristics, it can address scenarios of traffic surges or drops.

[0135] The real-time feature prediction module makes predictions based on filtered time-series data from the past 5 minutes. Specifically, it calculates the rate of change of the indicators based on the time-series sequence of the indicators over the past 5 minutes, and calculates the weighted acceleration accordingly. In particular, it introduces time decay weights (the closer to the current time, the greater the weight) to calculate the weighted acceleration.

[0136] The acceleration of the four metrics (transaction throughput, network latency, processor utilization, and transaction queue depth) is assigned a weight of 0.35, 0.3, 0.25, and 0.1, respectively.

[0137] (III) Optimized control algorithm of intelligent decision engine.

[0138] The intelligent decision engine determines whether to scale up or down based on the following three dimensions: 1. Real-time analysis of the four load metrics mentioned above to determine if the current configuration can withstand the pressure; 2. Prediction results from the periodic prediction module; 3. Prediction results from the real-time feature prediction module. The data from these three dimensions can be normalized into a unified load score.

[0139] To reduce frequent expansion and contraction oscillations in the system under critical load conditions, an innovative dual-threshold hysteresis triggering mechanism was implemented. The core of this mechanism lies in setting two key thresholds—an expansion threshold (e.g., 0.85) and a contraction threshold (e.g., 0.25)—and introducing a hysteresis interval of 0.05. State-aware dynamic threshold adjustment: If the system has previously performed an expansion operation, the actual trigger threshold for contraction will decrease from 0.25 to 0.20 (i.e., 0.25-0.05), meaning the system needs to withstand a lower load pressure before contraction is allowed; conversely, if contraction was previously performed, the actual trigger threshold for expansion will increase from 0.85 to 0.90 (0.85+0.05), requiring the system to reach a higher load level before triggering expansion.

[0140] Correspondingly, if the system has previously performed an expansion operation, the actual trigger threshold for scaling down will decrease from 0.25 to 0.20 (i.e., 0.25-0.05), while the actual trigger threshold for expansion can be adjusted to 0.85. If the system has previously performed a scaling down operation, the actual trigger threshold for expansion will increase from 0.85 to 0.90 (0.85+0.05), while the actual trigger threshold for scaling down can be adjusted to 0.25.

[0141] Understandably, if there are frequent instances of scaling up and then scaling down, or vice versa, it can easily lead to a waste of node scheduling resources. This waste can be reduced by adjusting the threshold accordingly.

[0142] For capacity expansion calculations, an overload proportional elasticity algorithm is used. This algorithm determines the number of nodes to expand or shrink based on the quantification of the difference between the load score and the threshold. This gradual expansion / shrinkage strategy avoids resource waste while effectively handling sudden traffic surges.

[0143] The dynamic scaling architecture proposed in this embodiment significantly improves the stability and resource utilization efficiency of the distributed system through the collaborative optimization of three core modules. The load awareness module employs a lightweight monitoring agent to collect system operating status in real time and filter out interfering data, ensuring the accuracy and stability of decision-making. The dual-track prediction mechanism module provides indicators for predicting whether to perform scaling operations in advance by comprehensively analyzing historical and current data. The intelligent decision engine accurately judges scaling needs through a dynamic threshold mechanism and elastic algorithms, avoiding frequent switching at load critical points and allocating resources on demand to reduce redundancy overhead. The entire architecture responds quickly to load changes through closed-loop control of "monitoring, decision-making, and execution," significantly reducing resource waste and operational costs. The system maintains stable operation even under high concurrency scenarios and is suitable for distributed scenarios with stringent real-time and reliability requirements.

[0144] Based on the above method embodiments, embodiments of this application also provide a coordination node scheduling device. The following will be combined with... Figure 3The device is described in detail.

[0145] Figure 3 The diagram illustrates a structural block diagram of a coordination node scheduling device according to an embodiment of this application.

[0146] like Figure 3 As shown, the coordination node scheduling device 300 provided in this embodiment includes: an overall load module 310, an expansion module 320, and a reduction module 330.

[0147] The overall load module 310 is used to determine the overall load for the set of coordinating nodes; any coordinating node in the set of coordinating nodes is used to coordinate the processing of distributed transactions. In one embodiment, the overall load module 310 can be used to execute the steps S210 and related steps described above, which will not be repeated here.

[0148] The expansion module 320 is used to expand the number of nodes in the current set of coordinated nodes and reduce the current load reduction threshold when the determined overall load exceeds the current load expansion threshold. In one embodiment, the expansion module 320 can be used to execute the steps S220 and related steps described above, which will not be repeated here.

[0149] The scaling-down module 330 is used to scale down the current set of coordinated nodes and increase the current load expansion threshold when the determined overall load is less than the current load scaling-down threshold. In one embodiment, the scaling-down module 330 can be used to execute the steps S230 and related steps described above, which will not be repeated here.

[0150] According to an embodiment of this application, the expansion module 320 is used to: add a first number of coordinating nodes to the current set of coordinating nodes; the first number is determined based on the absolute value of a first difference between the determined overall load and the current load expansion threshold; the first number is positively correlated with the absolute value of the first difference; the reduction module 330 is used to: reduce a second number of coordinating nodes to the current set of coordinating nodes; the second number is determined based on the absolute value of a second difference between the current load reduction threshold and the determined overall load; the second number is positively correlated with the absolute value of the second difference.

[0151] According to an embodiment of this application, the overall load module 310 is used to perform an overall load determination method, including: determining the current actual overall load of the current coordination node set; determining the future predicted overall load of the current coordination node set; and determining the overall load based on the determined current actual overall load and the future predicted overall load.

[0152] According to an embodiment of this application, the overall load module 310 is used to: predict and determine the initial predicted overall load of the coordination node set based on the actual overall load time sequence of the coordination node set in different historical periods; and determine the future predicted overall load of the coordination node set by combining the determined initial predicted overall loads.

[0153] According to an embodiment of this application, the actual overall load is determined based on at least one of the following: the number of distributed transactions processed by the coordination node set per unit time; the resource utilization rate of the coordination node set; the statistical value of the distributed transaction response time of the coordination node set; and the number of distributed transactions waiting to be processed in the coordination node set.

[0154] According to an embodiment of this application, the overall load module 310 is used to: determine the corresponding historical smoothed load sequence based on the actual overall load time sequence of the coordination node set in different historical periods; predict and determine the initial predicted overall load of the coordination node set based on the different historical smoothed load sequences; the historical smoothed load at a specified historical moment in any historical smoothed load sequence is determined based on the weighted sum of different actual overall loads within a fixed duration before the specified historical moment in the corresponding actual overall load time sequence, wherein the duration between the actual overall load corresponding to the historical moment and the specified historical moment is negatively correlated with the weight of the actual overall load.

[0155] According to an embodiment of this application, the expansion module 320 is used to: reduce the current load reduction threshold based on the number of coordinating nodes in the current coordinating node set after node expansion; and adjust the current load expansion threshold based on the number of coordinating nodes in the current coordinating node set after node expansion when the determined overall load is greater than the current load expansion threshold. The reduction module 330 is used to: increase the current load expansion threshold based on the number of coordinating nodes in the current coordinating node set after node reduction; and adjust the current load reduction threshold based on the number of coordinating nodes in the current coordinating node set after node reduction when the determined overall load is less than the current load reduction threshold.

[0156] According to an embodiment of this application, the scaling-down module 330 is used to: scale down the nodes in the current set of coordination nodes that meet the preset scaling-down conditions; the preset scaling-down conditions include at least one of the following: the node load is lower than a preset node load threshold; the number of distributed transactions processed by the node per unit time is less than a preset transaction number threshold; the rollback rate of the distributed transactions processed by the node is higher than a preset rollback rate threshold; the statistical value of the distributed transaction response time of the node is higher than a preset duration threshold; and the node stability is lower than a preset stability threshold.

[0157] According to an embodiment of this application, the above-described apparatus can be applied to a control node; the control node is used to allocate received distributed transactions to coordination nodes in the current coordination node set for processing; the overall load module 310 is used to: predict the number of future distributed transactions to be received based on the historical distributed transaction time sequence; the historical distributed transaction time sequence includes the number of distributed transactions received by the control node at different historical times; and determine the overall load for the current coordination node set based on the predicted number of future distributed transactions.

[0158] According to embodiments of this application, any plurality of modules among the overall load module 310, expansion module 320, and reduction module 330 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this application, at least one of the overall load module 310, expansion module 320, and reduction module 330 can be at least partially implemented as a hardware circuit, such as a field-programmable gate array, a programmable logic array, a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit, or implemented by any other reasonable means of integrating or packaging the circuit, or implemented by any one of software, hardware, and firmware, or by a suitable combination of any of these three implementation methods. Alternatively, at least one of the overall load module 310, expansion module 320, and reduction module 330 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0159] The explanation of the above device embodiments can be found in the explanation of other embodiments, and any step in the above method embodiments can be executed by the corresponding module in the above device embodiments.

[0160] Figure 4 A block diagram schematically illustrates an electronic device suitable for implementing a coordinated node scheduling method according to an embodiment of this application.

[0161] like Figure 4 As shown, an electronic device 1200 according to an embodiment of this application includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory 1202 or a program loaded from a storage portion 1208 into a random access memory 1203. The processor 1201 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a dedicated microprocessor. The processor 1201 may also include onboard memory for caching purposes. The processor 1201 may include a single processing unit or multiple processing units for executing different steps of the method flow according to an embodiment of this application.

[0162] Random access memory 1203 stores various programs and data required for the operation of electronic device 1200. Processor 1201, read-only memory 1202, and random access memory 1203 are interconnected via bus 1204. Processor 1201 executes various steps of the method flow according to embodiments of this application by executing programs in read-only memory 1202 and / or random access memory 1203. It should be noted that the programs may also be stored in one or more memories other than read-only memory 1202 and random access memory 1203. Processor 1201 may also execute various steps of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0163] According to embodiments of this application, the electronic device 1200 may further include an input / output interface 1205, which is also connected to the bus 1204. The electronic device 1200 may also include one or more of the following components connected to the input / output interface 1205: an input section 1206 including a keyboard, mouse, etc.; an output section 1207 including a cathode ray tube, liquid crystal display, etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card, such as a local area network card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the input / output interface 1205 as needed. A removable medium 1211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1210 as needed so that computer programs read from it can be installed into the storage section 1208 as needed.

[0164] Embodiments of this application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0165] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include the read-only memory 1202, and / or random access memory 1203, and / or one or more memories other than read-only memory 1202 and random access memory 1203 described above.

[0166] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0167] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 1209, and / or installed from the removable medium 1211. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0168] In embodiments of this application, the computer program can be downloaded and installed from a network via communication section 1209, and / or installed from removable medium 1211. When the computer program is executed by processor 1201, it performs the functions defined in the system of this application embodiment. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0169] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0170] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0171] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A method for coordinating node scheduling, characterized in that, The method includes: The overall load is determined for the set of coordination nodes; each coordination node in the set of coordination nodes is used to coordinate the processing of distributed transactions. If the determined overall load is greater than the current load expansion threshold, expand the current set of coordination nodes and reduce the current load reduction threshold. If the overall load is less than the current load scaling-down threshold, scale down the current set of coordinating nodes and increase the current load scaling-up threshold.

2. The method according to claim 1, characterized in that, The node expansion for the current set of coordination nodes includes: adding a first number of coordination nodes to the current set of coordination nodes; the first number is determined based on the absolute value of a first difference between the determined overall load and the current load expansion threshold; the first number is positively correlated with the absolute value of the first difference. The node scaling down for the current set of coordination nodes includes: reducing the number of coordination nodes in the current set of coordination nodes by a second number; the second number is determined based on the absolute value of a second difference between the current load scaling down threshold and the determined overall load; the second number is positively correlated with the absolute value of the second difference.

3. The method according to claim 1, characterized in that, The method for determining the overall load includes: Determine the current actual overall load of the current set of coordination nodes; Determine the future forecast of the overall load of the current set of coordinating nodes; The overall load is determined based on the current actual overall load and the future projected overall load.

4. The method according to claim 3, characterized in that, Determining the future predicted overall load of the current set of coordination nodes includes: The initial predicted overall load of the coordination node set is determined based on the actual overall load time series of the coordination node set in different historical periods. Based on the combined initial predicted overall loads, the future predicted overall load of the coordination node set is determined.

5. The method according to claim 3 or 4, characterized in that, The actual overall load is determined based on at least one of the following: The number of distributed transactions processed by the coordination node set per unit time; the resource utilization rate of the coordination node set; the statistical value of the distributed transaction response time of the coordination node set; the number of distributed transactions waiting to be processed in the coordination node set.

6. The method according to claim 4, characterized in that, The step of predicting and determining the initial predicted overall load of the coordination node set based on the actual overall load time series of the coordination node set in different historical periods includes: Based on the actual overall load time sequence of the coordination node set in different historical periods, the corresponding historical smoothed load sequence is determined. The initial predicted overall load of the coordination node set is determined based on different historical smoothed load sequences. The historical smoothed load at a specified historical moment in any historical smoothed load sequence is determined by the weighted sum of different actual overall loads within a fixed duration before the specified historical moment in the corresponding actual overall load time series. The duration between the actual overall load corresponding to the historical moment and the specified historical moment is negatively correlated with the weight of the actual overall load.

7. The method according to claim 1, characterized in that, The reduction of the current load scaling threshold includes: reducing the current load scaling threshold based on the number of coordinating nodes in the current coordinating node set after node scaling. If the determined overall load is greater than the current load expansion threshold, the method further includes: adjusting the current load expansion threshold according to the number of coordinating nodes in the current coordinating node set after node expansion. The method of increasing the current load scaling threshold includes: increasing the current load scaling threshold based on the number of coordinating nodes in the current coordinating node set after node scaling down. If the determined overall load is less than the current load scaling-down threshold, the method further includes: adjusting the current load scaling-down threshold based on the number of coordinating nodes in the current coordinating node set after node scaling-down.

8. The method according to claim 1, characterized in that, The process of scaling down the current set of coordination nodes includes: For the coordination nodes in the current set of coordination nodes that meet the preset scaling-down conditions, perform node scaling-down. The preset reduction conditions include at least one of the following: The node load is lower than the preset node load threshold; The number of distributed transactions processed by a node per unit of time is less than a preset threshold for the number of transactions. The distributed transaction rollback rate processed by the node is higher than the preset rollback rate threshold; The distributed transaction response time of the node is higher than the preset time threshold. The node stability is lower than the preset stability threshold.

9. The method according to claim 1, characterized in that, The method is applied to a control node; the control node is used to allocate received distributed transactions to coordination nodes in the current set of coordination nodes for processing. The determination of the overall load for the set of coordinating nodes includes: Based on the historical distributed transaction time sequence, predict the number of future distributed transactions to be received; the historical distributed transaction time sequence includes the number of distributed transactions received by the control node at different historical times; Determine the overall load on the current set of coordinating nodes based on the predicted number of future distributed transactions.

10. A coordination node scheduling device, characterized in that, The device includes: The overall load module is used to determine the overall load for the set of coordination nodes; any one of the coordination nodes in the set of coordination nodes is used to coordinate the processing of distributed transactions. The expansion module is used to expand the number of nodes in the current set of coordinated nodes and reduce the current load reduction threshold when the overall load is greater than the current load expansion threshold. The scaling-down module is used to scale down the current set of coordinated nodes and increase the current load expansion threshold when the overall load is less than the current load scaling-down threshold.

11. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 9.

Images