Cluster server management method based on edge computing

Abstract

The invention discloses a cluster server management method based on edge computing. The cluster server management method comprises the following steps: S1, deploying a center node and a plurality of edge nodes in a cluster server architecture; s2, the central node collects global task information, edge node resource state information and communication link state information of the cluster server; s3, generating a plurality of task allocation schemes and selecting the scheme with the highest fitness as an initial scheme; s4, monitoring the task execution state and the resource use condition in real time; s5, migrating part of tasks to other available edge nodes; s6, when detecting that the edge node or the communication link has a fault, triggering a fault-tolerant mechanism by the center node; s7, adjusting a resource allocation strategy in advance, and optimizing a task allocation scheme; s8, when the task load is increased, part of nodes are restarted to meet resource requirements; and S9, after task execution is completed, forming a collaborative management closed loop of the edge node and the center node. The operation efficiency, the reliability and the energy-saving effect of the system are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cluster servers, and in particular to a cluster server management method based on edge computing. Background Art

[0002] With the rapid development of edge computing technology, the management method of cluster servers is changing from the traditional centralized architecture to the distributed architecture. As an important infrastructure for processing massive tasks and scheduling complex resources, cluster servers are widely used in cloud computing, Internet of Things, and big data analysis. Traditional centralized cluster server management methods face many challenges, and performance bottlenecks are particularly prominent in scenarios where task loads change dynamically.

[0003] At present, most cluster server management systems still use a centralized task scheduling and resource management architecture. Under this architecture, all task scheduling decisions are made by the central node, which is responsible for collecting resource status information and task execution information from all edge nodes and formulating task allocation plans based on a global view. Although this approach can achieve unified task scheduling and global resource management, it has the following significant defects in actual applications:

[0004] Single-point bottleneck and high latency problems: When the amount of tasks increases dramatically, the computing and communication load of the central node may increase significantly, leading to a single-point bottleneck problem. At the same time, since all task scheduling and resource allocation must be completed through the central node, the high load of the central node will directly cause task allocation delays, thereby affecting the response speed of the entire system. In addition, in scenarios with high network communication delays, centralized management is difficult to meet low-latency task processing requirements.

[0005] Low resource utilization: In the traditional cluster management architecture, the task allocation strategy of the central node is often based on static or predefined rules, lacking the ability to flexibly adapt to dynamic resource status and real-time task loads, resulting in low resource utilization of some edge nodes, while other nodes may be overloaded, making it difficult to optimize the overall resource utilization efficiency.

[0006] Low fault recovery efficiency: When an edge node or communication link fails, the traditional centralized architecture requires the central node to re-plan the task allocation plan. This process usually takes a long time and may cause task execution to be interrupted. At the same time, the existing architecture lacks an effective distributed fault-tolerant mechanism and task migration strategy, making it difficult to quickly restore the normal operation of the system. Summary of the invention

[0007] One object of the present invention is to propose a cluster server management method based on edge computing, which significantly improves the operating efficiency, reliability and energy-saving effect of the system.

[0008] A cluster server management method based on edge computing according to an embodiment of the present invention includes the following steps:

[0009] S1. Deploy a central node and multiple edge nodes in a cluster server architecture;

[0010] S2, the central node collects the global task information of the cluster server, the resource status information of the edge node and the communication link status information;

[0011] S3, optimize and calculate the global task information, edge node resource status information and communication link status information through the sparrow search algorithm to obtain the initial task allocation plan;

[0012] S4. Send the initial task allocation plan to each edge node. Each edge node performs task processing according to the received task allocation plan and monitors its task execution status and resource usage in real time.

[0013] S5. According to the task execution status and resource usage of the edge nodes, the distribution of tasks among the edge nodes is dynamically adjusted in combination with the global search and local optimization capabilities of the sparrow search algorithm. When the resource utilization of an edge node exceeds the preset threshold, some tasks are migrated to other available edge nodes.

[0014] S6. When a failure of an edge node or a communication link is detected, the central node triggers the fault-tolerance mechanism, reuses the sparrow search algorithm to generate a task redistribution plan, migrates the affected tasks to other available edge nodes or backup nodes, and ensures the continuous execution of some functions through the autonomy of the edge nodes;

[0015] S7. Combine the task execution history data and real-time monitoring information, predict the future task load trend through the sparrow search algorithm, adjust the resource allocation strategy in advance, and optimize the task allocation plan;

[0016] S8. According to the resource usage of edge nodes in the cluster server, the operation mode of edge nodes is adjusted regularly. When the nodes are in a low-load state, some nodes are shut down to save energy. When the task load increases, some nodes are restarted to meet resource requirements.

[0017] S9. After completing the task execution, the edge node uploads the task execution result to the central node. The central node updates the global resource status table and optimizes the subsequent task allocation plan, forming a collaborative management closed loop between the edge node and the central node.

[0018] Optionally, the S1 specifically includes the following contents:

[0019] S11. Deploy a central node and multiple edge nodes in a cluster server architecture. The central node and the edge nodes are connected via a communication link. The central node is used to perform global task scheduling and resource status aggregation operations, and the edge nodes are used to perform distributed task processing and resource status monitoring.

[0020] S12. The central node has the global task scheduling capability and generates a task allocation plan based on the global task information T and the edge node resource status information R. The global task information T includes the resource requirements of the task. and expected completion time The edge node resource status information R includes the available computing resources of the edge node Storage resources and online resources Summarize the task completion results and update the global resource status table RS;

[0021] S13, the edge node has independent task execution capability, receives and parses the task allocation plan D issued by the central node, and executes the task allocation strategy d according to the task allocation plan. i , schedule the local resources of edge nodes to complete the task;

[0022] S14. The edge node has the ability to monitor resource status, monitor the resource usage of the edge node and generate a resource status matrix:

[0023]

[0024] Among them, m ij represents the utilization rate of the j-th resource of edge node i;

[0025] Upload the resource status matrix M to the central node regularly to update the global resource status table RS;

[0026] S15. Edge nodes have the ability to schedule some tasks. When tasks are overloaded, they dynamically reorder local tasks based on local resource usage status and task priority. They collaborate among edge nodes to transfer some tasks to other edge nodes with lower loads. The set of transferred tasks is T. migrate .

[0027] Optionally, the S3 specifically includes the following contents:

[0028] S31, input the collected global task information T, edge node resource status information R and communication link status information C into the optimization module of the sparrow search algorithm, initialize the number of individuals in the population to N, and each individual in the population X i Represents a task allocation scheme, which is formed by matching the global task information T and the edge node resource status information R;

[0029] S32. Calculate each task allocation scheme X i The fitness value F(X i ), the fitness function is composed of the task completion time T(X i ), load balancing degree L(X i ) and energy consumption E(X i ) Comprehensive decision:

[0030] F(X i )=α·T(X i )+β·L(X i )+γ·E(X i );

[0031] Among them, α, β, γ are weight coefficients, T(X i ) represents the task completion time, which is defined as:

[0032]

[0033] Among them, max represents the worst performance of task completion time in the system;

[0034] L(X i ) indicates the node load balance, which is defined as:

[0035]

[0036] Among them, T j represents the task set of node j;

[0037] E(X i ) represents energy consumption, which is defined as the consumption of node computing resources and communication link bandwidth during task execution:

[0038]

[0039] Among them, P cpu is the unit computing resource energy consumption, P link is the energy consumption per unit bandwidth, Indicates the current bandwidth of each link;

[0040] S33, screening the individuals in the population based on the fitness values, sorting them according to the fitness values, and retaining the individuals with fitness values ​​higher than the preset value;

[0041] S34, perform global search on the explorer population, adjust the position of the population individuals based on the fitness value, perform local optimization operations on the follower population, fine-tune the task allocation plan based on the fitness value and the resource status of the edge node, perform a final screening of all population individuals based on the fitness value, and select the task allocation plan X with the highest fitness value optimal As the initial task allocation plan, the optimized initial task allocation plan Xoptimal Sent to each edge node.

[0042] Optionally, the S34 specifically includes the following contents:

[0043] S341, in the population of the sparrow search algorithm, according to the fitness value F(X i ) Sort the individuals in the population and select the individual with the highest fitness value and the individual with the lowest fitness value

[0044] S342: According to the load index l of the edge node ej Dynamically adjust the search radius r of each population individual i :

[0045]

[0046] Among them, r max is the maximum search radius, l ej is the load index of edge node j, l max is the maximum load index in the system;

[0047] S343, the population individuals are layered according to the search radius, wherein the individuals with a search radius greater than a preset value perform global search, and the individuals with a search radius less than a preset value perform local optimization:

[0048] Task allocation scheme for global search layer X i Adjust its position to explore the entire solution space:

[0049]

[0050] in, represents the position of the global search individual i in the tth generation, q is the global search step length, and are the individuals with the highest and lowest fitness values, respectively. rand is a random number used to increase the randomness of the search;

[0051] Task allocation scheme for local optimization layer X i , perform local search based on the resource status of the current node combined with the fitness value:

[0052]

[0053] in, represents the position of the local optimization individual i in the tth generation, and l is the local search step length;

[0054] S344, update the individuals after global search and local optimization respectively as and And merge into the updated complete population location set Recalculate the fitness value of each individual in the updated population And update the individual with the highest fitness value according to the fitness value sorting and the lowest individual At the same time, dynamically adjust the next generation search radius set R t+1 ;

[0055] S345. Under the hierarchical search mechanism of the dynamic detection radius, repeat the iterative operation of global search and local optimization until any of the following conditions is met:

[0056] Maximum number of iterations T max Reach the preset value;

[0057] The optimal fitness value R t+1 If it remains unchanged in multiple consecutive iterations, it indicates that the algorithm has converged;

[0058] S346. After the iteration is completed, select the population individual X with the highest fitness value optimal As the final task allocation plan.

[0059] Optionally, the S5 specifically includes the following contents:

[0060] S51, the initial task allocation plan X optimal Send it to each edge node, and each edge node will allocate tasks according to the global task information T in the allocation plan. edge Execute task processing;

[0061] S52: The edge node monitors the task execution status and resource usage in real time when executing the task, and generates a task execution status set E. edge and resource usage set U edge :

[0062] Task execution status ei ∈E edge Including tasks ei The current execution progress and remaining completion time of

[0063] Resource usage ej ∈U edge Including computing resource utilization, storage resource utilization, and network bandwidth utilization of edge node j;

[0064] S53, according to the resource usage of the edge node, set U edge and the task execution state set E edge Calculate the resource utilization matrix M of the current edge node edge ;

[0065] S54, when the resource utilization matrix M edge When the resource utilization of any edge node exceeds the preset threshold, the task migration module is triggered to select part of the task set T from the overloaded edge node. migrate_e ={t me1 ,t me2 ,…,t meq} to migrate, select the task based on the following confirmation:

[0066]

[0067] in, is the task priority, is the remaining resources of the target edge node k;

[0068] S55. Redistribute the task set T by combining the global search capability of the sparrow search algorithm migrate_e To the edge node with lower resource utilization, update the task allocation plan X after migration migrate_e and resource utilization matrix M′ edge After the migration is completed, the task migration results are synchronized to the central node in real time, and the global resource status table RS and the task execution status set E are updated. edge .

[0069] Optionally, the S8 specifically includes the following contents:

[0070] S81. Set U according to the resource usage of edge nodes edge and the task execution state set E edge Calculate the comprehensive load index set of edge nodes:

[0071] L edge = {l e1 ,l e2 ,…,l en};

[0072] Among them, the comprehensive load index l ej By calculating resource utilization Storage resource utilization Network bandwidth utilization and the task queue length q ej Overall decision:

[0073]

[0074] Among them, α1, β1, γ1 and δ1 are weight coefficients, reflecting the influence weight of each resource indicator on the load index;

[0075] S82, according to the comprehensive load index set L edge Set the operation mode of the edge node, including:

[0076] If l ej ≤θ low , where θ low is a low load threshold, the edge node j is set to low power consumption mode or shutdown mode;

[0077] If l ej ≥θ high , where θ high is a high load threshold, the edge node j is set to high performance mode;

[0078] If θ low <l ej <θ high , then edge node j maintains the current mode;

[0079] S83. In low power mode or shutdown mode, the edge node retains basic monitoring functions, updates its resource usage and load index in real time and uploads them to the central node, forming a low-load edge node B. low ;

[0080] S84, when the central node detects that the task load increases, it comprehensively analyzes the global task information T and the resource status set U global Activate some edge nodes B that are in a low-load state low , and redistribute the task load, redistribute the set of tasks T reassign ={t r1 ,t r2 ,…,t rp}:

[0081]

[0082] in, is the task resource requirement, is the remaining available resources of the target node, is the expected completion time of the task, q ej is the task queue length of the target node;

[0083] S85. After the activated edge node executes the task load, it updates its resource usage U′ in real time edge and the comprehensive load index set L′ edge , synchronized to the central node to optimize global resource management.

[0084] The beneficial effects of the present invention are:

[0085] (1) The present invention realizes dynamic optimization of task allocation and resource utilization by combining the global search and local optimization capabilities of the sparrow search algorithm. At the same time, a hierarchical search mechanism is proposed to globally and locally adjust the population according to the fitness value. By dynamically adjusting the search radius, step size and optimization target, the global exploration capability and local convergence efficiency of the algorithm are significantly improved. Compared with the traditional static rule task allocation method, the present invention can dynamically adjust the task allocation scheme according to the real-time task load and the resource status of the edge node, avoiding the problem of central node resource overload or edge node resource waste, and significantly improving resource utilization and system response speed.

[0086] (2) The present invention introduces an edge node load index calculation model, and constructs a load index set by integrating the computing resources, storage resources, network bandwidth and task queue length of the comprehensive node. Combined with the dynamic operation mode adjustment strategy, it can manage the sleep of low-load nodes and quickly activate standby nodes under high-load conditions, effectively reducing energy consumption and improving task processing capabilities.

[0087] (3) The present invention optimizes the allocation of task sets based on task migration and fault-tolerant mechanisms combined with the sparrow search algorithm, effectively solving the problems of edge node resource overload and link failure. During the task migration process, the task execution status and resource utilization of edge nodes are monitored in real time, and tasks are allocated to the optimal nodes using dynamic migration strategies, ensuring the system's ability to continue operating under fault or high-load conditions. Compared with existing centralized fault-tolerant methods, the distributed migration mechanism of the present invention can respond quickly to network anomalies, greatly reducing the task interruption rate, while improving the robustness and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0089] Figure 1 This is a flow chart of a cluster server management method based on edge computing proposed by the present invention. DETAILED DESCRIPTION

[0090] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0091] refer to Figure 1 , a cluster server management method based on edge computing, comprising the following steps:

[0092] S1. Deploy a central node and multiple edge nodes in a cluster server architecture;

[0093] S2, the central node collects the global task information of the cluster server, the resource status information of the edge node and the communication link status information;

[0094] S3, optimize and calculate the global task information, edge node resource status information and communication link status information through the sparrow search algorithm to obtain the initial task allocation plan;

[0095] S4. Send the initial task allocation plan to each edge node. Each edge node performs task processing according to the received task allocation plan and monitors its task execution status and resource usage in real time.

[0096] S5. According to the task execution status and resource usage of the edge nodes, the distribution of tasks among the edge nodes is dynamically adjusted in combination with the global search and local optimization capabilities of the sparrow search algorithm. When the resource utilization of an edge node exceeds the preset threshold, some tasks are migrated to other available edge nodes.

[0097] S6. When a failure of an edge node or communication link is detected, the central node triggers the fault-tolerance mechanism and reuses the sparrow search algorithm to generate a task redistribution plan to migrate the affected tasks to other available edge nodes or backup nodes. At the same time, the autonomy of the edge nodes is used to ensure the continuous execution of some functions.

[0098] S7. Combine the task execution history data and real-time monitoring information, predict the future task load trend through the sparrow search algorithm, adjust the resource allocation strategy in advance, and optimize the task allocation plan;

[0099] S8. According to the resource usage of edge nodes in the cluster server, the operation mode of edge nodes is adjusted regularly. When the nodes are in a low-load state, some nodes are shut down to save energy. When the task load increases, some nodes are restarted to meet resource requirements.

[0100] S9. After completing the task execution, the edge node uploads the task execution result to the central node. The central node updates the global resource status table and optimizes the subsequent task allocation plan, forming a collaborative management closed loop between the edge node and the central node.

[0101] In this implementation, S1 specifically includes the following contents:

[0102] S11. Deploy a central node and multiple edge nodes in a cluster server architecture. The central node and the edge nodes are connected via a communication link. The central node is used to perform global task scheduling and resource status aggregation operations, and the edge nodes are used to perform distributed task processing and resource status monitoring.

[0103] S12. The central node has the global task scheduling capability and generates a task allocation plan based on the global task information T and the edge node resource status information R. The global task information T includes the resource requirements of the task. and expected completion time The edge node resource status information R includes the available computing resources of the edge node Storage resources and online resources Summarize the task completion results and update the global resource status table RS;

[0104] S13, the edge node has independent task execution capability, receives and parses the task allocation plan D issued by the central node, and executes the task allocation strategy d according to the task allocation plan. i , schedule the local resources of edge nodes to complete the task;

[0105] S14. The edge node has the ability to monitor resource status, monitor the resource usage of the edge node and generate a resource status matrix:

[0106]

[0107] Among them, m ij represents the utilization rate of the j-th resource of edge node i;

[0108] Upload the resource status matrix M to the central node regularly to update the global resource status table RS;

[0109] S15. Edge nodes have the ability to schedule some tasks. When tasks are overloaded, they dynamically reorder local tasks based on local resource usage status and task priority. They collaborate among edge nodes to transfer some tasks to other edge nodes with lower loads. The set of transferred tasks is T. migrate .

[0110] In this implementation, S3 specifically includes the following contents:

[0111] S31, input the collected global task information T, edge node resource status information R and communication link status information C into the optimization module of the sparrow search algorithm, initialize the number of individuals in the population to N, and each individual in the population X i Represents a task allocation scheme, which is formed by matching the global task information T and the edge node resource status information R;

[0112] S32. Calculate each task allocation scheme X i The fitness value F(X i ), the fitness function is composed of the task completion time T(X i ), load balancing degree L(X i ) and energy consumption E(X i) Comprehensive decision:

[0113] F(X i )=α·T(X i )+β·L(X i )+γ·E(X i );

[0114] Among them, α, β, γ are weight coefficients, T(X i ) represents the task completion time, which is defined as:

[0115]

[0116] Among them, max represents the worst performance of task completion time in the system;

[0117] L(X i ) indicates the node load balance, which is defined as:

[0118]

[0119] Among them, T j represents the task set of node j;

[0120] E(X i ) represents energy consumption, which is defined as the consumption of node computing resources and communication link bandwidth during task execution:

[0121]

[0122] Among them, P cpu is the unit computing resource energy consumption, P link is the energy consumption per unit bandwidth, Indicates the current bandwidth of each link;

[0123] S33, screening the individuals in the population based on the fitness values, sorting them according to the fitness values, and retaining the individuals with fitness values ​​higher than the preset value;

[0124] S34, perform global search on the explorer population, adjust the position of the population individuals based on the fitness value, perform local optimization operations on the follower population, fine-tune the task allocation plan based on the fitness value and the resource status of the edge node, perform a final screening of all population individuals based on the fitness value, and select the task allocation plan X with the highest fitness value optimal As the initial task allocation plan, the optimized initial task allocation plan X optimal Sent to each edge node.

[0125] In this implementation, S34 specifically includes the following contents:

[0126] S341, in the population of the sparrow search algorithm, according to the fitness value F(X i ) Sort the individuals in the population and select the individual with the highest fitness value and the individual with the lowest fitness value

[0127] S342: According to the load index l of the edge node ej Dynamically adjust the search radius r of each population individual i :

[0128]

[0129] Among them, r max is the maximum search radius, l ej is the load index of edge node j, l max is the maximum load index in the system;

[0130] S343, the population individuals are layered according to the search radius, wherein the individuals with a search radius greater than a preset value perform global search, and the individuals with a search radius less than a preset value perform local optimization:

[0131] Task allocation scheme for global search layer X i Adjust its position to explore the entire solution space:

[0132]

[0133] in, represents the position of the global search individual i in the tth generation, q is the global search step length, and are the individuals with the highest and lowest fitness values, respectively. rand is a random number used to increase the randomness of the search;

[0134] Task allocation scheme for local optimization layer X i , perform local search based on the resource status of the current node combined with the fitness value:

[0135]

[0136] in, represents the position of the local optimization individual i in the tth generation, and l is the local search step length;

[0137] S344, update the individuals after global search and local optimization respectively as and And merge into the updated complete population location set Recalculate the fitness value of each individual in the updated population And update the individual with the highest fitness value according to the fitness value sorting and the lowest individual At the same time, dynamically adjust the next generation search radius set R t+1 ;

[0138] S345. Under the hierarchical search mechanism of the dynamic detection radius, repeat the iterative operation of global search and local optimization until any of the following conditions is met:

[0139] Maximum number of iterations T max Reach the preset value;

[0140] The optimal fitness value R t+1 If it remains unchanged in multiple consecutive iterations, it indicates that the algorithm has converged;

[0141] S346. After the iteration is completed, select the population individual X with the highest fitness value optimal As the final task allocation plan.

[0142] In this implementation, S5 specifically includes the following contents:

[0143] S51, the initial task allocation plan X optimal Send it to each edge node, and each edge node will allocate tasks according to the global task information T in the allocation plan. edge Execute task processing;

[0144] S52: The edge node monitors the task execution status and resource usage in real time when executing the task, and generates a task execution status set E. edge and resource usage set U edge :

[0145] Task execution status ei ∈E edge Including tasks ei The current execution progress and remaining completion time of

[0146] Resource usage ej ∈U edge Including computing resource utilization, storage resource utilization, and network bandwidth utilization of edge node j;

[0147] S53, according to the resource usage of the edge node, set U edge and the task execution state set E edge Calculate the resource utilization matrix M of the current edge node edge ;

[0148] S54, when the resource utilization matrix M edge When the resource utilization of any edge node exceeds the preset threshold, the task migration module is triggered to select part of the task set T from the overloaded edge node. migrate_e ={tme1 ,t me2 ,…,t meq} to migrate, select the task based on the following confirmation:

[0149]

[0150] in, is the task priority, is the remaining resources of the target edge node k;

[0151] S55. Redistribute the task set T by combining the global search capability of the sparrow search algorithm migrate_e To the edge node with lower resource utilization, update the task allocation plan X after migration migrate_e and resource utilization matrix M′ edge After the migration is completed, the task migration results are synchronized to the central node in real time, and the global resource status table RS and the task execution status set E are updated. edge .

[0152] In this implementation, S8 specifically includes the following contents:

[0153] S81. Set U according to the resource usage of edge nodes edge and the task execution state set E edge Calculate the comprehensive load index set of edge nodes:

[0154] L edge = {l e1 ,l e2 ,…,l en};

[0155] Among them, the comprehensive load index l ej By calculating resource utilization Storage resource utilization Network bandwidth utilization and the task queue length q ej Overall decision:

[0156]

[0157] Among them, α1, β1, γ1 and δ1 are weight coefficients, reflecting the influence weight of each resource indicator on the load index;

[0158] S82, according to the comprehensive load index set L edge Set the operation mode of the edge node, including:

[0159] If l ej ≤θ low , where θ lowis a low load threshold, the edge node j is set to low power consumption mode or shutdown mode;

[0160] If l ej ≥θ high , where θ high is a high load threshold, the edge node j is set to high performance mode;

[0161] If θ low <l ej <θ high , then edge node j maintains the current mode;

[0162] S83. In low power mode or shutdown mode, the edge node retains basic monitoring functions, updates its resource usage and load index in real time and uploads them to the central node, forming a low-load edge node B. low ;

[0163] S84, when the central node detects that the task load increases, it comprehensively analyzes the global task information T and the resource status set U global Activate some edge nodes B that are in a low-load state low , and redistribute the task load, redistribute the set of tasks T reassign ={t r1 ,t r2 ,…,t rp}:

[0164]

[0165] in, is the task resource requirement, is the remaining available resources of the target node, is the expected completion time of the task, q ej is the task queue length of the target node;

[0166] S85. After the activated edge node executes the task load, it updates its resource usage U′ in real time edge and the comprehensive load index set L′ edge , synchronized to the central node to optimize global resource management.

[0167] Embodiment 1:

[0168] In order to verify the feasibility and effectiveness of the cluster server management method based on edge computing combined with the sparrow search algorithm of the present invention, this embodiment takes the Double Eleven promotion activity of a large e-commerce platform in a certain place as the background, simulates the actual application scenario of its cluster server management, and analyzes the task scheduling efficiency, resource utilization and system fault recovery capability. The entire application process of the present invention is described in detail and specific data comparison is provided.

[0169] During the Double Eleven promotion period, the number of user visits to the e-commerce platform surged. The high-frequency tasks of order processing, payment requests, and recommendation services placed extremely high demands on the platform's cluster servers. In order to cope with sudden traffic bursts, the platform deployed a cluster server system, including 1 central node and 100 edge nodes. The central node is used for global task scheduling and resource status aggregation, and the edge nodes are distributed in data centers across the country, responsible for processing user requests, executing recommendation algorithms, and managing order data.

[0170] During the peak period of the event, the platform servers needed to process more than 100 million orders, 500 million product browsing requests, and 300 million recommendation algorithm calls every day. The traditional centralized management method caused significant delays in task scheduling due to excessive computing and communication pressure on the central node. Some users even experienced page loading failures. Past system records showed that the average task delay during the Double Eleven peak period was as high as 500 milliseconds, and the response time of some tasks exceeded 1 second. Resource utilization was unevenly distributed, with more than 30% of edge nodes having resource utilization rates of less than 50%, while some nodes had resource utilization rates of more than 90% and overload problems.

[0171] In order to optimize the management of cluster servers, the platform introduces the method of the present invention, which combines edge computing and sparrow search algorithm to comprehensively upgrade cluster servers in terms of task allocation, resource optimization, dynamic load management and fault recovery.

[0172] Before the promotion begins, the platform first deploys central nodes and edge nodes, collects the initial resource status of all edge nodes, including the availability of CPU, memory, and bandwidth, and uploads them to the central node to form a global resource status table. At the same time, the platform simulates the load distribution of user access and predicts the task volume and resource requirements in different periods based on historical data. The sparrow search algorithm generates an initial task allocation plan by optimizing the fitness value in real time, and allocates user requests to edge nodes with closer geographical locations and lower resource utilization, thereby reducing the communication pressure on the central node.

[0173] During the peak period of promotional activities, the system monitors the resource usage and task execution status of each edge node in real time. An edge node A is located in East China, with an initial load index of 30, a CPU utilization of 20%, a memory utilization of 30%, and a bandwidth utilization of 40%. When the number of user visits suddenly surges, the load index of node A rises to 85, and the resource utilization is close to the threshold. At this time, the system triggers the dynamic task migration mechanism to migrate some low-priority tasks to backup nodes B and C with lower load indexes. The selection of migration tasks is optimized through the fitness value and load index calculation formula, and the efficiency of task allocation is significantly improved.

[0174] During a promotion, an edge node D located in a data center at location A went down due to a power failure, and the order task originally planned to be processed was not completed. The system detected that the resource status of node D was abnormal and automatically triggered the fault-tolerant mechanism. The system reallocated the tasks through the sparrow search algorithm. The reallocated tasks were preferentially assigned to nodes E and F with lower loads, ensuring the stability of the system.

[0175] After the promotion ended, the system generated a detailed resource usage report based on the task execution history data. The report showed that through the method of the present invention, the resource utilization of each edge node reached a better balance, the task delay was significantly reduced, and the energy consumption was effectively optimized.

[0176] In order to verify that the method of the present invention is superior to the traditional method, a comparative experiment was conducted on cluster servers of the same platform using the traditional method and the method of the present invention. The experiment lasted for 24 hours. The simulated tasks included order processing, recommended services, and user requests. The data results are as follows:

[0177] Table 1 Performance comparison data of the traditional method and the method of the present invention in the experiment

[0178] index Traditional methods Method of the present invention Average task latency (ms) 500 220 Maximum task delay (ms) 1200 400 Edge node resource utilization 30% (too low) or 90% (too high) 60%-80% (balanced) Average system energy consumption (kWh) 4200 3300 Fault recovery time (seconds) 300 60 Task interruption rate 5％ 0.8％

[0179] Experimental data fully demonstrate the significant advantages of the present invention in dynamic task scheduling, resource optimization and fault recovery. Through the above-mentioned embodiments, the method of the present invention realizes efficient cluster server management, effectively solves the shortcomings of traditional methods in task delay, resource utilization and fault recovery time, and provides reliable technical support for high-load business scenarios of e-commerce platforms.

[0180] The present invention realizes dynamic optimization of task allocation and resource utilization by combining the global search and local optimization capabilities of the sparrow search algorithm, and proposes a hierarchical search mechanism, which globally and locally adjusts the population according to the fitness value, and significantly improves the global exploration capability and local convergence efficiency of the algorithm by dynamically adjusting the search radius, step size and optimization target. Compared with the traditional static rule task allocation method, the present invention can dynamically adjust the task allocation scheme according to the real-time task load and the resource status of the edge node, avoiding the problem of central node resource overload or edge node resource waste, and significantly improving resource utilization and system response speed.

[0181] The present invention introduces an edge node load index calculation model, constructs a load index set by integrating the computing resources, storage resources, network bandwidth and task queue length of the comprehensive node, and combines it with a dynamic operation mode adjustment strategy to perform sleep management on low-load nodes and quickly activate standby nodes under high-load conditions, effectively reducing energy consumption and improving task processing capabilities.

[0182] The present invention optimizes the allocation of task sets based on task migration and fault-tolerant mechanisms combined with a sparrow search algorithm, effectively solving the problems of edge node resource overload and link failure. During the task migration process, the task execution status and resource utilization of edge nodes are monitored in real time, and tasks are allocated to the optimal node using a dynamic migration strategy, ensuring the system's ability to continue operating under fault or high-load conditions. Compared with existing centralized fault-tolerant methods, the distributed migration mechanism of the present invention can respond quickly to network anomalies, greatly reducing the task interruption rate, while improving the robustness and reliability of the system.

[0183] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A cluster server management method based on edge computing, characterized in that: The steps include: S1. Deploy a central node and multiple edge nodes in a cluster server architecture; S2, the central node collects the global task information of the cluster server, the resource status information of the edge node and the communication link status information; S3, optimize and calculate the global task information, edge node resource status information and communication link status information through the sparrow search algorithm to obtain the initial task allocation plan; S4. Send the initial task allocation plan to each edge node. Each edge node performs task processing according to the received task allocation plan and monitors its task execution status and resource usage in real time. S5. According to the task execution status and resource usage of the edge nodes, the distribution of tasks among the edge nodes is dynamically adjusted in combination with the global search and local optimization capabilities of the sparrow search algorithm. When the resource utilization of an edge node exceeds the preset threshold, some tasks are migrated to other available edge nodes. S6. When a failure of an edge node or a communication link is detected, the central node triggers the fault-tolerance mechanism, reuses the sparrow search algorithm to generate a task redistribution plan, migrates the affected tasks to other available edge nodes or backup nodes, and ensures the continuous execution of some functions through the autonomy of the edge nodes; S7. Combine the task execution history data and real-time monitoring information, predict the future task load trend through the sparrow search algorithm, adjust the resource allocation strategy in advance, and optimize the task allocation plan; S8. According to the resource usage of edge nodes in the cluster server, the operation mode of edge nodes is adjusted regularly. When the nodes are in a low-load state, some nodes are shut down to save energy. When the task load increases, some nodes are restarted to meet resource requirements. S9. After completing the task execution, the edge node uploads the task execution result to the central node. The central node updates the global resource status table and optimizes the subsequent task allocation plan, forming a collaborative management closed loop between the edge node and the central node.

2. A cluster server management method based on edge computing according to claim 1, characterized in that: The S1 specifically includes the following contents: S11. Deploy a central node and multiple edge nodes in a cluster server architecture. The central node and the edge nodes are connected via a communication link. The central node is used to perform global task scheduling and resource status aggregation operations, and the edge nodes are used to perform distributed task processing and resource status monitoring. S12. The central node has the global task scheduling capability and generates a task allocation plan based on the global task information T and the edge node resource status information R. The global task information T includes the resource requirements of the task. and expected completion time The edge node resource status information R includes the available computing resources of the edge node Storage resources and online resources Summarize the task completion results and update the global resource status table RS; S13, the edge node has independent task execution capability, receives and parses the task allocation plan D issued by the central node, and executes the task allocation strategy d according to the task allocation plan. i , schedule the local resources of edge nodes to complete the task; S14. The edge node has the ability to monitor resource status, monitor the resource usage of the edge node and generate a resource status matrix: Among them, m ij represents the utilization rate of the j-th resource of edge node i; Upload the resource status matrix M to the central node regularly to update the global resource status table RS; S15. Edge nodes have the ability to schedule some tasks. When tasks are overloaded, they dynamically reorder local tasks based on local resource usage status and task priority. They collaborate among edge nodes to transfer some tasks to other edge nodes with lower loads. The set of transferred tasks is T. migrate .

3. The cluster server management method based on edge computing according to claim 1, characterized in that: The S3 specifically Includes the following: S31, input the collected global task information T, edge node resource status information R and communication link status information C into the optimization module of the sparrow search algorithm, initialize the number of individuals in the population to N, and each individual in the population X i Represents a task allocation scheme, which is formed by matching the global task information T and the edge node resource status information R; S32. Calculate each task allocation scheme X i The fitness value F(X i ), the fitness function is the task completion time T(X i ), load balancing degree L(X i ) and energy consumption E(X i ) Comprehensive decision: F(X i )=α·T(X i )+β·L(X i )+γ·E(X i ); Among them, α, β, γ are weight coefficients, T(X i ) represents the task completion time, which is defined as: Among them, max represents the worst performance of task completion time in the system; L(X i ) indicates the node load balance, which is defined as: Among them, T j represents the task set of node j; E(X i ) represents energy consumption, which is defined as the consumption of node computing resources and communication link bandwidth during task execution: Among them, P cpu is the unit computing resource energy consumption, P link is the energy consumption per unit bandwidth, Indicates the current bandwidth of each link; S33, screening the individuals in the population based on the fitness values, sorting them according to the fitness values, and retaining the individuals with fitness values ​​higher than the preset value; S34, perform global search on the explorer population, adjust the position of the population individuals based on the fitness value, perform local optimization operations on the follower population, fine-tune the task allocation plan based on the fitness value and the resource status of the edge node, perform a final screening of all population individuals based on the fitness value, and select the task allocation plan X with the highest fitness value optimal As the initial task allocation plan, the optimized initial task allocation plan X optimal Sent to each edge node.

4. The cluster server management method based on edge computing according to claim 3 is characterized in that: The S34 specifically includes the following contents: S341, in the population of the sparrow search algorithm, according to the fitness value F(X i ) Sort the individuals in the population and select the individual with the highest fitness value and the individual with the lowest fitness value S342: According to the load index l of the edge node ej Dynamically adjust the search radius r of each population individual i : Among them, r max is the maximum search radius, l ej is the load index of edge node j, l max is the maximum load index in the system; S343, the population individuals are layered according to the search radius, wherein the individuals with a search radius greater than a preset value perform global search, and the individuals with a search radius less than a preset value perform local optimization: Task allocation scheme for global search layer X i Adjust its position to explore the entire solution space: in, represents the position of the global search individual i in the tth generation, q is the global search step length, and are the individuals with the highest and lowest fitness values, respectively. rand is a random number used to increase the randomness of the search; Task allocation scheme for local optimization layer X i , perform local search based on the resource status of the current node combined with the fitness value: in, represents the position of the local optimization individual i in the tth generation, and l is the local search step length; S344, update the individuals after global search and local optimization respectively as and And merge into the updated complete population location set Recalculate the fitness value of each individual in the updated population And update the individual with the highest fitness value according to the fitness value sorting and the lowest individual At the same time, dynamically adjust the next generation search radius set R t+1 ; S345. Under the hierarchical search mechanism of the dynamic detection radius, repeat the iterative operation of global search and local optimization until any of the following conditions is met: Maximum number of iterations T max Reach the preset value; The optimal fitness value R t+1 If it remains unchanged in multiple consecutive iterations, it indicates that the algorithm has converged; S346. After the iteration is completed, select the population individual X with the highest fitness value optimal As the final task allocation plan.

5. The cluster server management method based on edge computing according to claim 3 is characterized in that: The S5 specifically includes the following contents: S51, the initial task allocation plan X optimal Send it to each edge node, and each edge node will allocate tasks according to the global task information T in the allocation plan. edge Execute task processing; S52: The edge node monitors the task execution status and resource usage in real time when executing the task, and generates a task execution status set E. edge and resource usage set U edge : Task execution status ei ∈E edge Including tasks ei The current execution progress and remaining completion time of Resource usage ej ∈U edge Including computing resource utilization, storage resource utilization, and network bandwidth utilization of edge node j; S53, according to the resource usage of the edge node, set U edge and the task execution state set E edge Calculate the resource utilization matrix M of the current edge node edge ; S54, when the resource utilization matrix M edge When the resource utilization of any edge node exceeds the preset threshold, the task migration module is triggered to select part of the task set T from the overloaded edge node. migrate_e ={t me1 ,t me2 ,…,t meq } to migrate, select the task based on the following confirmation: in, is the task priority, is the remaining resources of the target edge node k; S55. Redistribute the task set T by combining the global search capability of the sparrow search algorithm migrate_e To the edge node with lower resource utilization, update the task allocation plan X after migration migrate_e and resource utilization matrix M′ edge After the migration is completed, the task migration results are synchronized to the central node in real time, and the global resource status table RS and the task execution status set E are updated. edge .

6. A cluster server management method based on edge computing according to claim 3, characterized in that: The S8 specifically includes the following contents: S81. Set U according to the resource usage of edge nodes edge and the task execution state set E edge Calculate the comprehensive load index set of edge nodes: L edge ＝{l e1 ,l e2 ,…,l en}； Among them, the comprehensive load index l ej By calculating resource utilization Storage resource utilization Network bandwidth utilization and the task queue length q ej Overall decision: Among them, α1, β1, γ1 and δ1 are weight coefficients, reflecting the influence weight of each resource indicator on the load index; S82, according to the comprehensive load index set L edge Set the operation mode of the edge node, including: If l ej ≤θ low , where θ low is a low load threshold, the edge node j is set to low power consumption mode or shutdown mode; If l ej ≥θ high , where θ high is a high load threshold, the edge node j is set to high performance mode; If θ low <l ej <θ high , then edge node j maintains the current mode; S83. In low power mode or shutdown mode, the edge node retains basic monitoring functions, updates its resource usage and load index in real time and uploads them to the central node, forming a low-load edge node B. low ; S84, when the central node detects that the task load increases, it comprehensively analyzes the global task information T and the resource status set U global Activate some edge nodes B that are in a low-load state low , and redistribute the task load, redistribute the set of tasks T reassign ={t r1 ,t r2 ,…,t rp }: in, is the task resource requirement, is the remaining available resources of the target node, is the expected completion time of the task, q ej is the task queue length of the target node; S85. After the activated edge node executes the task load, it updates its resource usage U′ in real time edge and the comprehensive load index set L′ edge , synchronized to the central node to optimize global resource management.

Images