Load migration method and system for balancing energy efficiency of server cluster

Abstract

The application provides a server cluster energy efficiency balanced load migration method and system, and relates to the technical field of distributed computing. The application obtains power consumption data of distributed cache nodes in a server cluster, identifies high-power consumption and low-power consumption nodes; simultaneously obtains access frequency data of cache data fragments, performs cross-node unified sorting to generate a global access heat result; based on the heat sorting and the power consumption data, determines the fragment with the highest access heat in the high-power consumption node as a to-be-migrated fragment, selects a low-power consumption node with sufficient resources as a target node, forms a migration decision; updates cluster metadata mapping relationships, establishes a data transmission channel, and asynchronously replicates data blocks in batches and performs verification; after completion, switches client routing and points requests to the target node. The application can realize energy efficiency balance of the server cluster, effectively reduce overall power consumption, and improve resource utilization.

Description

Technical Field

[0001] This application relates to the field of distributed computing technology, and in particular to a load migration method and system for energy-efficient balancing of server clusters. Background Technology

[0002] In modern distributed computing environments, server clusters are widely used to support caching systems that handle high concurrency, such as cloud computing platforms and big data applications. Distributed cache nodes in these systems need to process massive amounts of data requests. With rising energy costs and increasing demands for green computing, achieving balanced energy efficiency across server clusters has become a critical technological requirement. This involves reducing energy consumption of high-power nodes through intelligent load migration while ensuring full utilization of low-power node resources. This improves the overall system's energy efficiency ratio and resource utilization, preventing overheating, performance bottlenecks, or additional energy consumption caused by uneven load distribution.

[0003] Currently, a common existing solution to address the aforementioned technical requirements is a dynamic migration strategy based on local load thresholds. This solution monitors load metrics such as CPU utilization and memory usage of each distributed cache node, sets predefined thresholds, and automatically migrates a portion of the cached data on that node to a node with a lower load when the node's load exceeds the threshold. This method relies on the load data within each node, uses a simple heuristic algorithm for migration decisions, requires no manual intervention, and achieves a certain degree of automated energy efficiency management.

[0004] However, this existing approach has significant drawbacks: it makes decisions based solely on local load metrics, ignoring the global access patterns of cached data shards, such as access frequency and popularity distribution. This can lead to the migration process failing to prioritize high-access-frequency data, resulting in hot data remaining concentrated on a few nodes after migration, thus failing to effectively reduce overall power consumption. Furthermore, the lack of unified sorting across nodes can introduce unnecessary latency and resource conflicts during the migration process, exacerbating the energy efficiency imbalance problem. Summary of the Invention

[0005] The purpose of this application is to provide a load migration method and system for energy-efficient balancing of server clusters, so as to solve the problem that the dynamic migration strategy based on local load thresholds in the prior art ignores the global access mode, resulting in poor energy efficiency balancing effect.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides a load migration method for energy-efficient balancing of server clusters, comprising:

[0007] Obtain power consumption data of each distributed cache node in the server cluster, and identify high-power nodes and low-power nodes based on the power consumption data;

[0008] At the same time, the access frequency data of each cached data shard on the distributed cache node is obtained, and the access frequency data of all cached data shards is uniformly sorted across nodes to generate a global access popularity ranking result.

[0009] Based on the global access popularity ranking result and the power consumption data, the cache data fragment with the highest global access popularity ranking among the high power consumption nodes is determined as the fragment to be migrated, while a low power consumption node with sufficient resource capacity is selected as the target node to form a migration decision.

[0010] Based on the migration decision, the mapping relationship between the cached data shards and the distributed cache nodes in the updated cluster metadata is updated. A data transmission channel between the high-power node and the target node is established based on the updated mapping relationship. The data blocks of the shards to be migrated are asynchronously copied to the target node in batches within the data transmission channel through the inter-node data synchronization mechanism. Data verification is performed after each batch is completed.

[0011] After data verification is completed, the client routing configuration is switched according to the updated mapping relationship, and subsequent access requests are directed to the target node, thereby achieving data migration and energy efficiency balance of the server cluster.

[0012] Optionally, the step of determining the cache data shard with the highest global access popularity among the high-power nodes as the shard to be migrated based on the global access popularity ranking result and the power consumption data, and simultaneously selecting a low-power node with sufficient resource capacity as the target node, to form a migration decision, includes:

[0013] Filter the cached data fragments located on the high-power nodes from the global access popularity ranking results;

[0014] Select the cache data shard with the highest global access popularity from the filtered cache data shards as the shard to be migrated;

[0015] Assess the current resource usage of each of the low-power nodes, select low-power nodes with resource utilization below a preset threshold and sufficient storage space as target nodes, and generate a migration decision that includes the shard identifier to be migrated, the source node identifier, and the target node identifier.

[0016] Optionally, the step of updating the mapping relationship between the cached data shards and the distributed cache nodes in the cluster metadata according to the migration decision, establishing a data transmission channel between the high-power node and the target node based on the updated mapping relationship, asynchronously copying the data blocks of the shards to be migrated to the target node in batches within the data transmission channel through an inter-node data synchronization mechanism, and performing data verification after each batch is completed includes:

[0017] Based on the information in the migration decision, an update instruction is sent to the cluster metadata database through the metadata service interface to modify the node identifier corresponding to the shard to be migrated in the mapping relationship between the cached data shard and the node to the identifier of the target node.

[0018] Based on the updated mapping relationship, a dedicated data transmission link is established between the high-power node and the target node;

[0019] The fragment to be migrated is divided into multiple data block units, and each data block unit is transmitted sequentially through the data transmission link.

[0020] After each data block unit is transmitted, the integrity of the transmitted data block unit is verified.

[0021] Optionally, establishing a dedicated data transmission link between the high-power node and the target node based on the updated mapping relationship includes:

[0022] Based on the updated mapping relationship, the network topology information of the high-power node and the target node is parsed to obtain the network address and connection parameters of the node.

[0023] Based on the network address and connection parameters, an end-to-end dedicated physical transmission channel is established between the high-power node and the target node;

[0024] Configure independent network bandwidth resources for the dedicated physical transmission channel and set transmission protocol parameters, including transmission timeout time and retry mechanism;

[0025] Enabling transmission encryption and data compression mechanisms on the dedicated physical transmission channel ensures the security and efficiency of data transmission.

[0026] Optionally, after data verification is completed, the step of switching the client routing configuration according to the updated mapping relationship and directing subsequent access requests to the target node to achieve data migration and energy efficiency balance of the server cluster includes:

[0027] After all data block units have been transferred and their integrity verified, it is confirmed that the fragment to be migrated has been completely replicated to the target node;

[0028] A route update command is sent to all client nodes in the cluster, enabling clients to send subsequent access requests to the migrated shards to the target node according to the updated mapping relationship, thereby achieving a balance between data migration and energy efficiency of the server cluster.

[0029] Optionally, the step of simultaneously obtaining the access frequency data of each cached data shard on the distributed cache node, and performing a unified cross-node sorting based on the access frequency data of all cached data shards to generate a global access popularity ranking result includes:

[0030] The access statistics module on the distributed cache node records the number of times each cached data fragment is accessed within a unit of time.

[0031] The access count data of the cached data fragments recorded on each of the distributed cache nodes are integrated into the cluster management unit;

[0032] In the cluster management unit, all cached data shards are sorted from highest to lowest number of accesses, and a global access popularity ranking result for the cached data shards is generated based on the ranking result.

[0033] Optionally, obtaining power consumption data of each distributed cache node in the server cluster, and identifying high-power nodes and low-power nodes based on the power consumption data, includes:

[0034] The power consumption data of each node is collected at fixed time intervals by a power consumption monitoring unit deployed on the distributed cache nodes.

[0035] The power consumption data of each distributed cache node is collected over multiple consecutive collection periods to calculate the average power consumption level of each node.

[0036] The average power consumption level of each node is compared with the average power consumption level of the entire cluster. A first group of nodes with a power consumption level higher than the average power consumption level of the entire cluster and a second group of nodes with a power consumption level lower than the average power consumption level of the entire cluster are identified. The first group of nodes is marked as high power consumption nodes and the second group of nodes is marked as low power consumption nodes.

[0037] Secondly, this application provides a load migration system for energy-efficient balancing of server clusters, comprising:

[0038] The identification module is used to obtain power consumption data of each distributed cache node in the server cluster, and identify high-power nodes and low-power nodes based on the power consumption data.

[0039] The generation module is used to simultaneously obtain the access frequency data of each cached data shard on the distributed cache nodes, perform cross-node unified sorting based on the access frequency data of all cached data shards, and generate a global access popularity ranking result.

[0040] The forming module is used to determine the cache data fragment with the highest global access popularity ranking among the high-power nodes as the fragment to be migrated based on the global access popularity ranking result and the power consumption data, and at the same time select a low-power node with sufficient resource capacity as the target node to form a migration decision.

[0041] The verification module is used to update the mapping relationship between the cached data shards and the distributed cache nodes in the cluster metadata according to the migration decision, establish a data transmission channel between the high-power node and the target node based on the updated mapping relationship, asynchronously copy the data blocks of the shards to be migrated to the target node in batches in the data transmission channel through the inter-node data synchronization mechanism, and perform data verification after each batch is completed.

[0042] The implementation module is used to switch the client routing configuration according to the updated mapping relationship after the data verification is completed, and direct subsequent access requests to the target node, thereby realizing data migration and energy efficiency balance of the server cluster.

[0043] Thirdly, this application provides an electronic device, comprising:

[0044] Memory, used to store computer programs;

[0045] A processor, used to execute the computer program to implement the load migration method for energy-efficient balancing of server clusters as described in the first aspect above.

[0046] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the load migration method for energy-efficient balancing of server clusters as described in the first aspect above.

[0047] The server cluster energy efficiency balancing load migration method provided in this application obtains power consumption data of each distributed cache node to identify high and low power nodes, and obtains the access frequency of each cache shard and sorts it across nodes to generate a global heat result. This provides accurate basic data on node power consumption and data heat for migration decisions. By determining the highest heat shard among high power nodes as the shard to be migrated and selecting low power nodes as target nodes based on global heat and power data, the migration decision is both targeted and feasible. By updating the data mapping relationship according to the decision and establishing a transmission channel, the shards to be migrated are transmitted asynchronously in batches and verified, ensuring data integrity without interrupting business. After verification, the client routing is switched to the target node, avoiding service interruption and achieving cluster energy efficiency balancing.

[0048] Furthermore, cache shards on high-power nodes are screened from the global heat sequence, and the highest heat shard is selected as the shard to be migrated. Low-power node resources are evaluated to select qualified nodes, and migration decisions with relevant identifiers are generated. This minimizes the load on high-power nodes, avoids overload of target nodes, ensures cluster stability, and provides reliable decision support for energy efficiency balance. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 A flowchart illustrating a load migration method for energy efficiency balancing of a server cluster, provided in an embodiment of this application;

[0051] Figure 2 A flowchart illustrating a load migration method for energy efficiency balancing of a server cluster, as provided in this application embodiment;

[0052] Figure 3 A flowchart illustrating a load migration method for energy efficiency balancing of a server cluster, as provided in this application embodiment;

[0053] Figure 4 This is a schematic diagram of a load migration system for energy-efficient balancing of a server cluster, provided in an embodiment of this application. Detailed Implementation

[0054] In existing technologies, load migration methods based on node local power consumption thresholds and access frequencies can trigger migration operations by monitoring node power consumption and internal access frequencies. However, because their decision-making process is limited to data sorting within a single node and lacks a global assessment of the overall cluster access patterns, low-power nodes can easily become new access hotspots after migration, exacerbating energy consumption or imbalance problems. Furthermore, the synchronous data replication mechanism used in this approach can block node resources during migration, introducing performance bottlenecks and reducing cluster response speed and availability, thus failing to achieve true energy efficiency balance.

[0055] To overcome these shortcomings, this application proposes a load migration method for energy-efficient balancing of server clusters. This method comprehensively collects power consumption and access frequency data from all nodes, performs cross-node unified sorting to generate a global access heat result, ensuring that migration decisions are based on overall cluster optimization and avoiding the generation of new hotspots. Simultaneously, it employs asynchronous data replication and batch verification mechanisms to reduce the impact on cluster performance and improve migration efficiency. In this way, the proposed solution not only effectively solves the problems of decision localization and performance bottlenecks in existing technologies, but also achieves balanced energy consumption and coordinated optimization of data access performance in server clusters.

[0056] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] The core of this application is to provide a load migration method for energy-efficient balancing of server clusters, and a flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes:

[0058] S101. Obtain the power consumption data of each distributed cache node in the server cluster, and identify high-power nodes and low-power nodes based on the power consumption data.

[0059] Optionally, S101 may specifically include the following steps:

[0060] S1011. By using the power consumption monitoring unit deployed on the distributed cache node, power consumption data of each node is collected at fixed time intervals.

[0061] S1012. Collect power consumption data of each distributed cache node in multiple consecutive collection cycles, and calculate the average power consumption level of each node.

[0062] S1013. Compare the average power consumption level of each node with the average power consumption level of the entire cluster, identify a first group of nodes that are higher than the average power consumption level of the entire cluster and a second group of nodes that are lower than the average power consumption level of the entire cluster, and mark the first group of nodes as high power consumption nodes and the second group of nodes as low power consumption nodes.

[0063] In the above scheme, the power consumption monitoring unit is a hardware sensor or software agent deployed on each distributed cache node to measure and record the node's power consumption data; the fixed time interval refers to the preset periodic data acquisition frequency, such as once every 5 minutes or every 15 minutes; the acquisition period refers to the time unit for power consumption data statistics; the average power consumption level refers to the arithmetic mean of the power consumption data of a node over multiple consecutive acquisition periods; the overall average power consumption level of the cluster refers to the sum of the average power consumption of all nodes; high power consumption nodes are nodes whose average power consumption is significantly higher than the overall cluster level; low power consumption nodes are nodes whose average power consumption is significantly lower than the overall cluster level.

[0064] In this embodiment of the application, firstly, the power consumption monitoring unit deployed on each distributed cache node by S1011 collects the power consumption data of the distributed cache node at preset time intervals (such as every 5 minutes). The power consumption monitoring unit obtains accurate power consumption readings through the IPMI interface or the intelligent power management interface, and adds a timestamp to the collected data and stores it in the local cache.

[0065] Then, through S1012, the cluster management unit periodically pulls historical power consumption data from each distributed cache node, performs statistical analysis on the power consumption data of each distributed cache node in multiple consecutive collection periods (such as the most recent 24 periods), and uses a moving average algorithm to calculate the average power consumption level of each distributed cache node.

[0066] Next, using S1013, the average power consumption level of all distributed cache nodes is used as the overall benchmark level of the cluster. The standard deviation analysis method is used to compare the average power consumption of each distributed cache node with the overall benchmark of the cluster: nodes that are higher than the overall benchmark by a certain threshold are classified into the first group (high power consumption nodes), and nodes that are lower than the overall benchmark by a certain threshold are classified into the second group (low power consumption nodes). Finally, the corresponding power consumption level label is added to the identified nodes, and the classification results are stored in the cluster state database.

[0067] In practical applications, in a cluster containing 5 nodes, the power consumption monitoring unit collects data every 10 minutes. After 24 hours of monitoring, the average power consumption of node A is calculated to be 180W, node B 170W, node C 160W, node D 150W, and node E 140W. The overall average power consumption of the cluster is 160W. Nodes A and B (above 160W) are marked as high-power nodes, nodes D and E (below 160W) are marked as low-power nodes, and node C, as the baseline node, is not specially marked.

[0068] The overall solution of S101 described above achieves accurate perception of the energy consumption status of the server cluster through a systematic power consumption monitoring and data analysis mechanism. Based on the high-power and low-power nodes identified by statistical analysis methods, it provides accurate decision-making basis for subsequent data migration and energy efficiency optimization. This enables the system to implement targeted load reduction measures for high-power nodes while making full use of the remaining processing power of low-power nodes, thereby achieving energy efficiency optimization and operating cost reduction for the entire cluster, while ensuring the stability and reliability of system performance.

[0069] S102. Simultaneously obtain the access frequency data of each cached data shard on the distributed cache node, perform cross-node unified sorting based on the access frequency data of all cached data shards, and generate a global access popularity sorting result.

[0070] Optionally, S102 may specifically include the following steps:

[0071] S1021. The access statistics module on the distributed cache node records the number of times each cached data fragment is accessed within a unit of time.

[0072] S1022. Integrate the access count data of the cache data fragments recorded on each of the distributed cache nodes into the cluster management unit;

[0073] S1023. In the cluster management unit, all the cached data shards are sorted from high to low according to the number of times they are accessed, and a global access popularity ranking result of the cached data shards is generated based on the ranking result.

[0074] In the above scheme, the access statistics module is a data collection component deployed on each distributed cache node, used to record and count the access frequency information of each cache data shard; the unit time refers to a preset statistical time window, such as a fixed duration of 1 minute or 1 hour; the number of accesses refers to the total number of read or write operations on the cache data shard within the time window; the cluster management unit is the control center component responsible for centralized management and coordination of the entire distributed cache system; the global access popularity ranking result refers to the data shard popularity ranking list generated after uniformly sorting the access statistics data of all nodes.

[0075] In the embodiments of this application, such as Figure 2 As shown, firstly, the access statistics module running on each distributed cache node through S1021 continuously records the number of times each cache data shard is accessed in a sliding time window manner. The access statistics module adopts a mechanism combining counters and timestamps. Each time an access request is received, the counter of the corresponding shard is incremented by 1, and the counter is reset and the statistical value is recorded at the end of each statistical period.

[0076] Next, through S1022, the distributed cache node periodically reports the access count data recorded within the statistical period to the cluster management unit via a distributed message queue or RPC call. The cluster management unit aggregates this distributed data using the eventual consistency principle, summing the access counts of the same shard on different nodes to obtain the global access count for each shard.

[0077] Then, using S1023 in the cluster management unit, quicksort or mergesort algorithms are used to sort all cached data shards in descending order according to the calculated global access count. After sorting, a global access popularity ranking result containing shard identifier, access count and ranking position is generated. This result will serve as an important basis for subsequent migration decisions.

[0078] In a practical application, in a three-node distributed caching cluster, the access statistics module on node A records that shard X was accessed 50 times within one minute, node B records 30 times, and node C records 20 times. This data is reported to the cluster management node via a Kafka message queue. The management node calculates the total number of accesses to shard X as 100. After similarly tracking other shards, all shards are sorted by access count, ultimately generating a global access popularity ranking.

[0079] The above-mentioned S102 overall solution achieves accurate statistics on the access popularity of cached data across the entire cluster by combining distributed data acquisition and centralized data processing; it uses a unified sorting algorithm to generate a global popularity ranking, providing reliable data support for data migration decisions; and it enables the system to intelligently identify hot data based on actual access patterns, providing a scientific basis for subsequent energy efficiency optimization and load balancing strategy formulation, thereby effectively improving the resource utilization efficiency and performance of the entire caching system.

[0080] S103. Based on the global access popularity ranking result and the power consumption data, the cache data fragment with the highest global access popularity ranking among the high power consumption nodes is determined as the fragment to be migrated, and at the same time, a low power consumption node with sufficient resource capacity is selected as the target node to form a migration decision.

[0081] Optionally, S103 may specifically include the following steps:

[0082] S1031. Select cached data fragments located on the high-power nodes from the global access popularity ranking results;

[0083] S1032. Select the cache data fragment with the highest global access popularity ranking from the filtered cache data fragments as the fragment to be migrated;

[0084] S1033. Evaluate the current resource usage of each of the low-power nodes, select a low-power node with a resource utilization rate lower than a preset threshold and sufficient storage space as the target node, and generate a migration decision including the fragment identifier to be migrated, the source node identifier, and the target node identifier.

[0085] In the above scheme, the global access popularity ranking result refers to the ranking list generated by uniformly sorting the access frequency of all cached data shards in the cluster across nodes; high power consumption nodes refer to server nodes whose energy consumption level is significantly higher than the cluster average after power consumption data analysis; cached data shards are the smallest logical units of data storage in a distributed caching system; shards to be migrated refer to data shards selected to be transferred from the current node to other nodes; resource usage includes multiple resource indicators such as node CPU utilization, memory utilization, remaining storage space, and network bandwidth utilization; the preset threshold is the upper limit of resource utilization pre-set according to cluster resource configuration and performance requirements; migration decision is a structured data object containing the unique identifier of the shard to be migrated, source node address information, and target node address information.

[0086] In this embodiment of the application, firstly, based on the global access popularity ranking result in S1031, a data filtering algorithm is used to select all cache data fragments located on high-power nodes from the global access popularity ranking result to form a set of candidate fragments.

[0087] Then, in step S1032, the maximum value search algorithm is used to select the data shard with the highest global access popularity from the set of candidate shards, and it is determined as the shard to be migrated.

[0088] Finally, resource usage data of each low-power node is collected through the resource monitoring system via S1033. A multi-dimensional evaluation algorithm is used to comprehensively calculate the resource utilization score of each node. The resource utilization score is compared with a preset threshold to filter out a set of candidate nodes with resource utilization below the threshold and sufficient storage space. Then, an optimal selection algorithm is used to select the node with the best overall resource status from the set of candidate nodes as the target node, and a migration decision containing complete migration information is generated. The migration decision includes information such as the unique identifier of the shard to be migrated, the network address of the source node, and the network address of the target node.

[0089] In a practical application, within a distributed caching cluster, global access popularity ranking shows data shards P, Q, and R as the top three, with P and Q located on high-power node X. First, shards P and Q on high-power nodes are selected, and then the higher-ranked shard P is chosen as the shard to migrate. Simultaneously, low-power nodes Y and Z are detected with resource utilization rates of 30% and 25% respectively, both with sufficient storage space. Finally, node Z, with lower resource utilization, is selected as the target node. A migration decision file containing the shard P identifier, node X address, and node Z address is generated.

[0090] The above-mentioned S103 overall solution, through a precise screening and evaluation mechanism, ensures the accuracy and effectiveness of migration decisions. It can effectively reduce the load pressure on high-power nodes and avoid placing an excessive burden on low-power nodes. It realizes intelligent optimization of server cluster energy consumption and reasonable allocation of resources, while ensuring the stability of system operation and the performance of data access.

[0091] S104. Update the mapping relationship between the cached data shards and the distributed cache nodes in the cluster metadata according to the migration decision. Establish a data transmission channel between the high-power node and the target node based on the updated mapping relationship. Asynchronously copy the data blocks of the shards to be migrated to the target node in batches in the data transmission channel through the inter-node data synchronization mechanism. Perform data verification after each batch is completed.

[0092] Optionally, S104 may specifically include the following steps:

[0093] S1041. Based on the information in the migration decision, send an update instruction to the cluster metadata database through the metadata service interface to modify the node identifier corresponding to the shard to be migrated in the mapping relationship between the cached data shard and the node to the identifier of the target node.

[0094] S1042. Based on the updated mapping relationship, establish a dedicated data transmission link between the high-power node and the target node;

[0095] The S1042 includes:

[0096] Based on the information in the migration decision, an update instruction is sent to the cluster metadata database through the metadata service interface to modify the node identifier corresponding to the shard to be migrated in the mapping relationship between the cached data shard and the node to the identifier of the target node.

[0097] Based on the updated mapping relationship, a dedicated data transmission link is established between the high-power node and the target node;

[0098] The fragment to be migrated is divided into multiple data block units, and each data block unit is transmitted sequentially through the data transmission link.

[0099] After each data block unit is transmitted, the integrity of the transmitted data block unit is verified.

[0100] S1043. Divide the fragment to be migrated into multiple data block units, and transmit each data block unit sequentially through the data transmission link;

[0101] S1044. After each data block unit is transmitted, the integrity of the transmitted data block unit is verified.

[0102] In the above scheme, the metadata service interface is a set of standardized programming interfaces provided by the cluster metadata management system to implement CRUD operations on metadata; the cluster metadata database is a distributed database that stores the mapping relationship between all data shards and nodes; the node identifier is the encoded information used to uniquely identify each distributed cache node; the data transmission link is a dedicated network communication channel established between the source node and the target node; the data block unit is the basic unit of data transmission after the data to be migrated is divided into shards of a fixed size; and integrity verification is the process of confirming the accuracy of data transmission through checksum comparison or hash value calculation.

[0103] In the embodiments of this application, such as Figure 3 As shown, firstly, according to the identifier of the shard to be migrated and the identifier of the target node contained in the migration decision, an update instruction is sent to the cluster metadata database by calling the RESTFUL API interface of the metadata service in S1041. The node identifier corresponding to the shard to be migrated in the mapping relationship between the cached data shard and the node is modified to the identifier of the target node, thereby forming a new mapping relationship.

[0104] Secondly, in step S1042, the network topology information of the high-power node and the target node is parsed based on the updated mapping relationship to obtain the network address and connection parameters of the node. According to the network address and connection parameters, a dedicated point-to-point data transmission link is established between the high-power node and the target node. Independent network bandwidth resources are configured for the data transmission link, and transmission protocol parameters, including transmission timeout time and retry mechanism, are set. Transmission encryption mechanism and data compression mechanism are enabled on the data transmission link to ensure the security and efficiency of data transmission.

[0105] Next, the data to be migrated is divided into multiple consecutive data block units according to a preset block size (e.g., 4MB) by S1043, and these data blocks are sent to the target node sequentially through the established data transmission link via an asynchronous transmission mechanism.

[0106] Finally, after each data block is transmitted via S1044, the source node calculates the CRC32 checksum of the data block and sends it to the target node. The target node performs the same calculation on the received data block and compares the checksum. If they match, the source node confirms that the data block has been transmitted completely; otherwise, it requests a retransmission of the data block.

[0107] In practical applications, in a distributed caching system, when data shard M needs to be migrated from node A to node B according to migration decisions, the mapping relationship of shard M is first updated from node A to node B through the metadata API. Then, a dedicated SSL encrypted transmission channel is established between nodes A and B. Shard M is divided into 10 data blocks and transmitted sequentially through this channel. After each block is transmitted, a CRC32 check is performed. For example, when transmitting the 5th data block, the checksum is calculated as X. If the target node receives the block and calculates the checksum as X, then the transmission of the block is confirmed to be successful.

[0108] The overall solution of S104 described above ensures the orderliness and reliability of data transmission by updating metadata first and then establishing a dedicated transmission channel; it adopts a batch asynchronous transmission and verification mechanism, which significantly reduces the impact of large-scale data migration on system performance; and the integrity verification mechanism ensures the accuracy of data migration and avoids the risk of data corruption or loss, thereby achieving an efficient and secure data migration process while maintaining the continuity and stability of cluster services.

[0109] S105. After the data verification is completed, the client routing configuration is switched according to the updated mapping relationship, and subsequent access requests are directed to the target node to achieve data migration and energy efficiency balance of the server cluster.

[0110] Optionally, S105 may specifically include the following steps:

[0111] S1051. After all data block units have been transmitted and their integrity verified, confirm that the fragment to be migrated has been completely copied to the target node;

[0112] S1052. Send a route update command to all client nodes in the cluster, so that the clients send subsequent access requests to the migrated shards to the target node according to the updated mapping relationship, thereby realizing data migration and energy efficiency balance of the server cluster.

[0113] In the above scheme, integrity verification refers to the process of confirming the integrity and accuracy of data through methods such as checksum comparison, hash value calculation, or data block sequence number verification; the route update command is a configuration update command that includes the latest mapping relationship between data fragments and nodes; client route configuration refers to the routing table or configuration information used by the client to determine the target node for data access; energy efficiency balancing refers to optimizing the load distribution of the server cluster through data migration to achieve reasonable allocation and reduction of overall energy consumption.

[0114] In this embodiment, after all data block units have been transmitted and passed integrity verification in S1051, the source node generates a migration completion confirmation signal, and the target node generates a data reception integrity report at the same time. Through the two-way confirmation mechanism, it is ensured that all data of the fragment to be migrated has been completely and accurately copied to the target node. The system then updates the migration status record and marks the fragment as "migration complete".

[0115] Next, via S1052, a route update instruction is broadcast to all client nodes through the cluster configuration management service. The route update instruction is distributed using a publish-subscribe model and includes the latest mapping relationship between data shard identifiers and target node addresses. After receiving the update instruction, the client updates its local route cache and points the access path of the migrated data shard to the new target node. At the same time, to ensure a smooth transition, the system sets a short gray-scale switching period. During this period, the old and new nodes are double-verified. After confirming that all clients have successfully updated their route configurations, the switching process is finally completed, thereby achieving a balance between data migration and the energy efficiency of the server cluster.

[0116] In practical applications, within a distributed storage system, after data shard X is migrated from node A to node B, the system confirms that all data blocks have passed MD5 checksum verification. It then sends a routing update broadcast to all clients, informing them that "shard X is now located on node B." Upon receiving the instruction, client C updates its local routing table, sending subsequent access requests to shard X to node B. Once the system detects that all clients have completed their updates, the migration process is officially complete.

[0117] The S105 solution described above ensures the integrity and accuracy of data migration through a robust verification and confirmation mechanism; it guarantees the consistency of client configurations by employing a centralized routing update and distribution mechanism; the smooth switching process avoids the risk of service interruption; and it ultimately achieves reasonable redirection of data access traffic, effectively reducing the load pressure on high-power nodes and improving the resource utilization of low-power nodes, thereby achieving an optimized balance of overall energy efficiency for the server cluster while ensuring the continuity and stability of system services.

[0118] The following is a complete example for steps 101-105. First, the power consumption of the distributed cache nodes in the server cluster is monitored and classified. In a cluster containing 5 nodes (nodes A, B, C, D, and E), each node is equipped with a power consumption monitoring unit in the form of a hardware sensor, which collects power consumption data at fixed time intervals of 10 minutes. After continuous monitoring for 24 hours, the power consumption data of each node for 24 consecutive collection cycles is counted, and the average power consumption is calculated using a moving average algorithm: Node A is 180W, Node B is 170W, Node C is 160W, Node D is 150W, and Node E is 140W. Then, the overall average power consumption of the cluster is calculated to be 160W. Through standard deviation analysis, nodes A and B with average power consumption higher than the overall cluster level are marked as high-power nodes, and nodes D and E with average power consumption lower than the overall cluster level are marked as low-power nodes. Node C serves as the baseline node and is not specially marked. The classification results are stored in the cluster state database.

[0119] Secondly, the access frequency of cached data shards is statistically analyzed, and a global access popularity ranking is generated. In the cache cluster with the three nodes (nodes A, B, and C), each node deploys an access statistics module. Using a counter and timestamp mechanism, the access count of each cached data shard is recorded every minute – node A records shard X as accessed 50 times, node B records 30 times, and node C records 20 times. Each node periodically reports the statistical data to the cluster management unit via a Kafka message queue. The management unit aggregates the data according to the eventual consistency principle, calculating the total global access count of shard X as 100 times. Then, a quicksort algorithm is used to sort all cached data shards from highest to lowest global access count, generating a global access popularity list containing shard identifiers, access counts, and rankings.

[0120] Next, migration decisions are made based on global access popularity and power consumption data. From the global access popularity list, cached data fragments located on high-power nodes (nodes A and B) are selected. Assuming that among the top three most popular fragments P, Q, and R, P and Q are located on node A, a maximum value search algorithm is used to select fragment P, which has the higher ranking, as the fragment to be migrated. At the same time, the CPU utilization, memory usage, and storage space data of low-power nodes D and E are collected through the resource monitoring system. It is found that the resource utilization of node D is 30% and that of node E is 25%, and both meet the storage space requirements. Finally, node E, which has a lower resource utilization, is selected as the target node, and a migration decision file containing the fragment P identifier, the source node A address, and the target node E address is generated.

[0121] Next, data migration and verification are performed based on the migration decision. The RESTful API interface of the metadata service is invoked to send an update command to the cluster metadata database, changing the mapping relationship between shard P and nodes from node A to node E. Based on the updated mapping relationship, a dedicated SSL-encrypted data transmission link is established between nodes A and E via TCP protocol. Shard P is divided into 10 data blocks of 4MB each and sent sequentially to node E via asynchronous transmission. After each data block is transmitted, node A calculates its CRC32 checksum and sends it to node E. Node E compares its calculated checksum with its own; if they match, the transmission is confirmed as complete; otherwise, a retransmission is requested, until all data blocks pass verification.

[0122] Finally, the client routing configuration is switched to achieve energy efficiency balance in the cluster. After all data blocks of shard P have passed verification, node A generates a migration completion confirmation signal, and node E generates a data reception integrity report, marking shard P as "migration complete" through a two-way confirmation mechanism. Subsequently, the cluster configuration management service adopts a publish-subscribe model to broadcast routing update instructions to all clients, informing them that "shard P is currently located at node E". After receiving the instructions, the clients update their local routing cache and direct access requests for shard P to node E. The system sets a short gray-scale switching period, and after detecting that all clients have completed the routing update, the migration process officially ends. At this time, the load pressure on the high-power node A is significantly reduced, the resources of the low-power node E are fully utilized, the entire cluster achieves energy efficiency balance, and the service is uninterrupted throughout the process.

[0123] Figure 4 This is a schematic diagram illustrating a specific implementation of a server cluster energy-efficient load migration system provided in this application embodiment, with reference to... Figure 4 The system may include:

[0124] The identification module 41 is used to obtain power consumption data of each distributed cache node in the server cluster, and identify high-power nodes and low-power nodes based on the power consumption data.

[0125] The generation module 42 is used to simultaneously obtain the access frequency data of each cached data shard on the distributed cache node, perform cross-node unified sorting based on the access frequency data of all cached data shards, and generate a global access popularity ranking result.

[0126] The forming module 43 is used to determine the cache data fragment with the highest global access popularity ranking among the high-power nodes as the fragment to be migrated based on the global access popularity ranking result and the power consumption data, and at the same time select a low-power node with sufficient resource capacity as the target node to form a migration decision.

[0127] The verification module 44 is used to update the mapping relationship between the cached data shards and the distributed cache nodes in the cluster metadata according to the migration decision, establish a data transmission channel between the high-power node and the target node based on the updated mapping relationship, asynchronously copy the data blocks of the shards to be migrated to the target node in batches in the data transmission channel through the inter-node data synchronization mechanism, and perform data verification after each batch is completed.

[0128] The implementation module 45 is used to switch the client routing configuration according to the updated mapping relationship after the data verification is completed, and direct subsequent access requests to the target node, thereby realizing data migration and energy efficiency balance of the server cluster.

[0129] The server cluster energy efficiency balancing load migration system of this application embodiment is used to implement the aforementioned server cluster energy efficiency balancing load migration method. Therefore, the specific implementation of the server cluster energy efficiency balancing load migration system can be found in the embodiment section of the server cluster energy efficiency balancing load migration method above. The specific implementation can be referred to the description of the corresponding embodiment, and will not be repeated here.

[0130] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the load migration method for energy-efficient balancing of the server cluster described above.

[0131] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described server cluster energy efficiency balancing load migration methods.

[0132] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0133] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described server cluster energy efficiency balancing load migration method embodiments.

[0134] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0135] The above provides a detailed description of a server cluster energy-efficient load migration method and system provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A load migration method for energy efficiency balancing in server clusters, characterized in that, include: Obtain power consumption data of each distributed cache node in the server cluster, and identify high-power nodes and low-power nodes based on the power consumption data; At the same time, the access frequency data of each cached data shard on the distributed cache node is obtained, and the access frequency data of all cached data shards is uniformly sorted across nodes to generate a global access popularity ranking result. Based on the global access popularity ranking result and the power consumption data, the cache data fragment with the highest global access popularity ranking among the high power consumption nodes is determined as the fragment to be migrated, while a low power consumption node with sufficient resource capacity is selected as the target node to form a migration decision. Based on the migration decision, the mapping relationship between the cached data shards and the distributed cache nodes in the updated cluster metadata is updated. A data transmission channel between the high-power node and the target node is established based on the updated mapping relationship. The data blocks of the shards to be migrated are asynchronously copied to the target node in batches within the data transmission channel through the inter-node data synchronization mechanism. Data verification is performed after each batch is completed. After data verification is completed, the client routing configuration is switched according to the updated mapping relationship, and subsequent access requests are directed to the target node, thereby achieving data migration and energy efficiency balance of the server cluster. The step of obtaining power consumption data for each distributed cache node in the server cluster, and identifying high-power and low-power nodes based on the power consumption data, includes: The power consumption data of each node is collected at fixed time intervals by a power consumption monitoring unit deployed on the distributed cache nodes. The power consumption data of each distributed cache node is collected over multiple consecutive collection periods to calculate the average power consumption level of each node. The average power consumption level of each node is compared with the average power consumption level of the entire cluster. A first group of nodes with a power consumption level higher than the average power consumption level of the entire cluster and a second group of nodes with a power consumption level lower than the average power consumption level of the entire cluster are identified. The first group of nodes is marked as high power consumption nodes and the second group of nodes is marked as low power consumption nodes.

2. The method as described in claim 1, characterized in that, Based on the global access popularity ranking result and the power consumption data, the cache data fragment with the highest global access popularity ranking among the high-power nodes is determined as the fragment to be migrated. Simultaneously, a low-power node with sufficient resource capacity is selected as the target node, forming a migration decision, including: Filter the cached data fragments located on the high-power nodes from the global access popularity ranking results; Select the cache data shard with the highest global access popularity from the filtered cache data shards as the shard to be migrated; Assess the current resource usage of each low-power node, select a low-power node with a resource utilization rate below a preset threshold and sufficient storage space as the target node, and generate a migration decision that includes the shard identifier to be migrated, the source node identifier, and the target node identifier.

3. The method as described in claim 1, characterized in that, Based on the migration decision, the mapping relationship between the cached data shards and the distributed cache nodes in the updated cluster metadata is updated. A data transmission channel is established between the high-power node and the target node based on the updated mapping relationship. Through an inter-node data synchronization mechanism, the data blocks of the shards to be migrated are asynchronously copied to the target node in batches within the data transmission channel. Data verification is performed after each batch is completed, including: Based on the information in the migration decision, an update instruction is sent to the cluster metadata database through the metadata service interface to modify the node identifier corresponding to the shard to be migrated in the mapping relationship between the cached data shard and the node to the identifier of the target node. Based on the updated mapping relationship, a dedicated data transmission link is established between the high-power node and the target node; The fragment to be migrated is divided into multiple data block units, and each data block unit is transmitted sequentially through the data transmission link. After each data block unit is transmitted, the integrity of the transmitted data block unit is verified.

4. The method as described in claim 3, characterized in that, Based on the updated mapping relationship, a dedicated data transmission link is established between the high-power node and the target node, including: Based on the updated mapping relationship, the network topology information of the high-power node and the target node is parsed to obtain the network address and connection parameters of the node. Based on the network address and connection parameters, an end-to-end dedicated physical transmission channel is established between the high-power node and the target node; Configure independent network bandwidth resources for the dedicated physical transmission channel and set transmission protocol parameters, including transmission timeout time and retry mechanism; Enabling transmission encryption and data compression mechanisms on the dedicated physical transmission channel ensures the security and efficiency of data transmission.

5. The method as described in claim 1, characterized in that, After data verification is complete, the client routing configuration is switched according to the updated mapping relationship, directing subsequent access requests to the target node. This achieves data migration and energy efficiency balancing of the server cluster, including: After all data block units have been transferred and their integrity verified, it is confirmed that the fragment to be migrated has been completely replicated to the target node; A route update command is sent to all client nodes in the cluster, enabling clients to send subsequent access requests to the migrated shards to the target node according to the updated mapping relationship, thereby achieving a balance between data migration and energy efficiency of the server cluster.

6. The method as described in claim 1, characterized in that, Simultaneously, the access frequency data of each cached data shard on the distributed cache nodes is obtained, and a unified cross-node sorting is performed based on the access frequency data of all cached data shards to generate a global access popularity ranking result, including: The access statistics module on the distributed cache node records the number of times each cached data fragment is accessed within a unit of time. The access count data of the cached data fragments recorded on each of the distributed cache nodes are integrated into the cluster management unit; In the cluster management unit, all cached data shards are sorted from highest to lowest number of accesses, and a global access popularity ranking result for the cached data shards is generated based on the ranking result.

7. A load migration system for energy-efficient balancing of server clusters, characterized in that, include: The identification module is used to obtain power consumption data of each distributed cache node in the server cluster, and identify high-power nodes and low-power nodes based on the power consumption data. The generation module is used to simultaneously obtain the access frequency data of each cached data shard on the distributed cache nodes, perform cross-node unified sorting based on the access frequency data of all cached data shards, and generate a global access popularity ranking result. The forming module is used to determine the cache data fragment with the highest global access popularity ranking among the high-power nodes as the fragment to be migrated based on the global access popularity ranking result and the power consumption data, and at the same time select a low-power node with sufficient resource capacity as the target node to form a migration decision. The verification module is used to update the mapping relationship between the cached data shards and the distributed cache nodes in the cluster metadata according to the migration decision, establish a data transmission channel between the high-power node and the target node based on the updated mapping relationship, asynchronously copy the data blocks of the shards to be migrated to the target node in batches in the data transmission channel through the inter-node data synchronization mechanism, and perform data verification after each batch is completed. The implementation module is used to switch the client routing configuration according to the updated mapping relationship after the data verification is completed, and direct subsequent access requests to the target node, thereby achieving data migration and energy efficiency balance of the server cluster. The step of obtaining power consumption data for each distributed cache node in the server cluster, and identifying high-power and low-power nodes based on the power consumption data, includes: The power consumption data of each node is collected at fixed time intervals by a power consumption monitoring unit deployed on the distributed cache nodes. The power consumption data of each distributed cache node is collected over multiple consecutive collection periods to calculate the average power consumption level of each node. The average power consumption level of each node is compared with the average power consumption level of the entire cluster. A first group of nodes with a power consumption level higher than the average power consumption level of the entire cluster and a second group of nodes with a power consumption level lower than the average power consumption level of the entire cluster are identified. The first group of nodes is marked as high power consumption nodes and the second group of nodes is marked as low power consumption nodes.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program, implement the steps of the load migration method for energy-efficient balancing of a server cluster according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the load migration method for energy-efficient balancing of a server cluster according to any one of claims 1 to 6.

Images