Server power management and scheduling method and system based on deep reinforcement learning

Abstract

The present invention relates to the field of server power technology, and more specifically, to a server power management and scheduling method and system based on deep reinforcement learning. The method comprises the following steps: obtaining real-time load data, power consumption data, and power-consuming device demand data of a server cluster; constructing a server state representation using a topological embedding algorithm based on the real-time load data; constructing an action space using group theory based on the server state representation; constructing a Q function approximation based on the action space using Lie group theory; adopting a transformation optimization strategy based on the Q function approximation; designing an exploration strategy using a Riemann-Zeta function based on the optimization strategy; generating a server power management and scheduling plan based on the exploration strategy, and outputting the plan. The method uses a topological embedding algorithm to construct a server state representation. This method can not only effectively reduce the dimension of the state space, but also preserve the topological relationship between states.

Description

Technical Field

[0001] The present invention relates to the field of server power technology, and more specifically, to a server power management and scheduling method and system based on deep reinforcement learning. Background Art

[0002] With the rapid development of information technology and the increasing scale and complexity of data centers, server power management and scheduling have become a critical issue that needs to be addressed. Traditional server power management methods rely primarily on fixed threshold policies or simple load balancing algorithms. These methods often perform poorly when faced with dynamically changing workloads and struggle to strike a good balance between energy efficiency and service quality.

[0003] In recent years, with advances in artificial intelligence (AI), researchers have begun applying machine learning methods to server power management. Among these, reinforcement learning-based approaches have garnered widespread attention due to their ability to adaptively learn optimal policies. However, existing reinforcement learning methods still face numerous challenges when dealing with high-dimensional state spaces and complex action spaces. For example, simple Q-learning algorithms often struggle to converge when state spaces are large, while deep Q-networks (DQNs), while improving learning capabilities, perform poorly when faced with continuous action spaces.

[0004] In addition, existing methods generally have the following problems: first, the representation of server status is often oversimplified, making it difficult to capture the intrinsic relationships and topological structures between states; second, the design of the action space lacks a theoretical basis, making it difficult to ensure the effectiveness and completeness of exploration; third, the approximation method of the Q function usually adopts a simple neural network structure, which makes it difficult to ensure stability and generalization ability in complex environments; finally, existing exploration strategies often adopt a simple ε-greedy method, which makes it difficult to achieve a good balance between exploration and exploitation.

[0005] These issues often lead to poor performance of existing methods in practical applications, especially in large-scale, highly dynamic data center environments, where it is difficult to achieve multi-objective optimization of energy efficiency, service quality, and system stability. Therefore, new server power management and scheduling methods are urgently needed that can effectively handle high-dimensional state spaces, design reasonable action spaces, improve the approximation accuracy of Q functions, and implement intelligent exploration strategies. Summary of the Invention

[0006] This paper aims to address the aforementioned issues in existing technologies by proposing a server power management and scheduling method and system based on deep reinforcement learning. This method innovatively combines advanced mathematical tools such as topology, group theory, and Lie group theory to construct a new deep reinforcement learning framework that effectively handles the complex state and action spaces inherent in server power management and scheduling.

[0007] The present invention provides a server power management and scheduling method based on deep reinforcement learning, including:

[0008] The acquisition steps include:

[0009] Obtain real-time load data, power consumption data, and power equipment demand data of the server cluster;

[0010] Processing steps include:

[0011] Based on the real-time load data, construct a server status representation using a topology embedding algorithm;

[0012] constructing an action space using group theory based on the server state representation;

[0013] Based on the action space, the Q function approximation is constructed using Lie group theory;

[0014] According to the Q function approximation, using Transform optimization strategies;

[0015] Based on the optimization strategy, an exploration strategy is designed using the Riemann-Zeta function;

[0016] Output steps include:

[0017] Based on the exploration strategy, a server power management and scheduling plan is generated and output.

[0018] Preferably, the obtaining step specifically includes:

[0019] Collecting load indicators of the server cluster in real time through a server monitoring system;

[0020] Acquire power consumption data of the server cluster from a power management module;

[0021] Receive power equipment demand data input by users.

[0022] Preferably, the constructing the server status representation using the topology embedding algorithm specifically includes:

[0023] Define server status collection and manifolds

[0024] Construct a homeomorphic mapping φ: Remove server status from the manifold Mapping to Euclidean space R d ;

[0025] Get the representation x of the server state in Euclidean space i ∈Rd .

[0026] Preferably, the constructing the action space by using group theory specifically includes:

[0027] Define action group G = {g j |g j :R→R d ,j=1,2,…,m}, where each element g j represents a transformation in Euclidean space;

[0028] Define a group action · such that g j ·x i =y i , where y i To perform action g j The new state after.

[0029] Preferably, the method of constructing a Q function approximation using Lie group theory specifically includes:

[0030] Define the Q function Q:R d ×G→R;

[0031] The Q function is parameterized using the special orthogonal group SO(n) so that Q(x,g)=tr(R T WR);

[0032] Adopt the matrix exponential function update rule:

[0033] As a preference, the use The transformation optimization strategies specifically include:

[0034] Define the policy function π:R D →G;

[0035] use Transformation parameterization strategy: Where ad-bc=1;

[0036] Optimize the objective function Where x follows the state distribution under policy π.

[0037] Preferably, the Riemann-Zeta function is used to design an exploration strategy, which specifically includes:

[0038] Defining exploration probability Where ζ(s) is the Riemann-Zeta function;

[0039] Construct exploration strategy π exp (x), executes policy π(x) with probability 1-p(x), and performs a random action with probability p(x).

[0040] As an option, it also includes:

[0041] Adjusting the power supply of the server cluster based on the server power management and scheduling solution;

[0042] Monitor the performance and energy consumption of the adjusted server cluster;

[0043] Based on the monitoring results, the deep reinforcement learning model is updated.

[0044] As an option, it also includes:

[0045] Set energy consumption thresholds and performance indicators;

[0046] When the energy consumption of the server cluster exceeds the energy consumption threshold or the performance is lower than the performance index, the processing step is triggered to be re-executed.

[0047] A server power management and scheduling system based on deep reinforcement learning that executes the method includes:

[0048] The acquisition module is used to obtain the real-time load data, power consumption data and power equipment demand data of the server cluster;

[0049] Processing module for:

[0050] Based on the real-time load data, construct a server status representation using a topology embedding algorithm;

[0051] constructing an action space using group theory based on the server state representation;

[0052] Based on the action space, the Q function approximation is constructed using Lie group theory;

[0053] According to the Q function approximation, using Transform optimization strategies;

[0054] Based on the optimization strategy, an exploration strategy is designed using the Riemann-Zeta function;

[0055] The output module is used to generate a server power management and scheduling plan based on the exploration strategy and output the plan.

[0056] The beneficial effects of the present invention are mainly reflected in the following aspects:

[0057] First, this invention uses a topology embedding algorithm to construct a representation of server states. This approach not only effectively reduces the dimensionality of the state space but also preserves the topological relationships between states. This enables the system to better understand and predict the overall state of the server cluster, enabling more accurate management decisions. For example, when the load suddenly increases, the system can quickly identify the state changes of the relevant servers and make corresponding adjustments, avoiding the delayed response problem common in traditional methods.

[0058] Secondly, this invention utilizes group theory to construct an action space. This innovative design provides the system with a rich and structured selection of actions. Compared to traditional discrete action spaces, this approach enables more precise control over server power management policies. For example, the system can precisely adjust the power of individual servers rather than simply switching them on and off, thereby maximizing energy efficiency while maintaining performance.

[0059] Thirdly, this invention uses Lie group theory to construct a Q-function approximation, significantly improving its expressiveness and learning stability. Compared to traditional neural network architectures, Lie group-based Q-functions better capture the inherent structure of state-action-value functions, resulting in stronger generalization capabilities in complex environments. This means the system can more quickly adapt to new load patterns and maintain stable performance over the long term.

[0060] In addition, the present invention adopts Transforming optimization strategies. This nonlinear transformation enables the system to express complex strategies within a limited parameter space. This not only improves the expressiveness of the strategies but also enhances the convergence of the optimization process. In practical applications, this means the system can more quickly find near-optimal management strategies and adapt to a wider range of server management scenarios.

[0061] Finally, this paper utilizes the Riemann-Zeta function to design an exploration strategy. This innovative approach provides the system with an intelligent, adaptive exploration mechanism. Compared to the traditional ε-greedy method, the Riemann-Zeta function-based exploration strategy can dynamically adjust the exploration probability based on the uncertainty of the current state, thereby achieving a better balance between exploration and exploitation. This enables the system to converge to a stable management strategy more quickly while maintaining its learning ability.

[0062] In summary, this invention utilizes a series of innovative mathematical tools and algorithmic designs to construct an efficient, stable, and adaptable server power management and scheduling system. These innovations are not simply a summation of parts, but rather form an organic whole, with each component synergizing and complementing each other. For example, the topologically embedded state representation provides more meaningful input for the action space constructed using group theory, while the Lie group-based Q-function approximation enables learning in this complex state-action space. The transformation optimization strategy and the exploration strategy of Riemann-Zeta function design further improve the learning efficiency and adaptability of the system in this complex space.

[0063] This holistic and innovative design enables this invention to demonstrate significant advantages in practical applications, significantly improving data center energy efficiency while simultaneously ensuring service quality and system stability. This provides a new solution for the intelligent and green management of modern large-scale data centers, with significant theoretical and practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 Flow chart of the method of the present invention.

[0065] Figure 2 This is a logic block diagram of the acquisition module of the present invention.

[0066] Figure 3 It is a logic block diagram of the processing module of the present invention.

[0067] Figure 4 It is a logic block diagram of the output module of the present invention. DETAILED DESCRIPTION

[0068] Please refer to Figure 1-4 This paper provides a server power management and scheduling method and system based on deep reinforcement learning. This method innovatively combines deep reinforcement learning technology with advanced mathematical theory to achieve intelligent and refined control of server cluster power management.

[0069] First, the method of the present invention includes an acquisition step, a processing step, and an output step. In the acquisition step, the system obtains real-time load data, power consumption data, and power device demand data from the server cluster. This data forms the basis for subsequent processing and provides the necessary input information for the deep reinforcement learning model.

[0070] In the processing step, this paper uses a series of innovative algorithms to process the acquired data. First, a topological embedding algorithm is used to construct a server state representation based on real-time load data. The purpose of this step is to map the complex server state into a more manageable mathematical space. Specifically, a homeomorphic mapping φ is defined:

[0071] φ:

[0072] in, is the manifold where the server state is located, R d is a d-dimensional Euclidean space. Through this mapping, each server state s i Represented as a point x in Euclidean space i :

[0073] φ(s i )=x i ,

[0074] This representation method not only preserves the topological structure of the server state, but also makes subsequent mathematical processing more convenient. The topological embedding algorithm is used to map the complex server state into a more tractable mathematical space. Through this mapping, the complex state space is simplified into a form that is easy to calculate. For example, in R d In this space, it is easy to calculate metrics such as distance and similarity between states, which is crucial for subsequent decision-making. For example, the distance between CPU utilization and memory usage can be calculated to evaluate server load and better allocate resources.

[0075] The server monitoring system collects the load indicators of the server cluster in real time (such as CPU usage, memory usage, network traffic, etc.). After preprocessing, the data is used to learn the homeomorphism mapping φ through a neural network (such as MLP or CNN) to map the original state to the Euclidean space R d .

[0076] Group theory is used to construct an action space, which allows different management operations to have intrinsic relationships. Next, the present invention uses group theory to construct an action space based on the server state representation. The innovation of this step lies in applying abstract algebraic structures to practical server management problems. An action group G is defined:

[0077] G={g j |g j :R d →R d ,j=1,2,…,m},

[0078] Among them, each g jRepresents a possible management action, such as adjusting the server's power supply or reallocating tasks. Group actions are defined as:

[0079] g j ·x i =y i ,

[0080] Here, y i Indicates execution of action g j The advantage of this representation method is that it can capture the intrinsic relationship between different management actions, so that the deep reinforcement learning model can better understand and select appropriate actions.

[0081] This definition describes how actions change the system state, allowing the system to simulate and predict the effects of different management decisions. For example, the operation of adjusting the CPU frequency can be modeled as a transformation g j , which transforms the current state x i Transition to new state y i This helps the system optimize resource allocation and improve energy efficiency. Data acquisition and processing:

[0082] The power management module obtains the power consumption data of the server cluster (such as the real-time power consumption, voltage, and current of each server). This data is used to train the model to identify which actions (such as reducing the CPU frequency) can reduce energy consumption while maintaining performance.

[0083] Lie group theory is used to construct an approximation of the Q function, estimating the long-term reward of taking an action in a given state. After constructing the state representation and action space, this paper uses Lie group theory to construct an approximation of the Q function. The Q function is a core concept in deep reinforcement learning, which estimates the long-term reward of taking an action in a given state. The special orthogonal group SO(n) is innovatively used to parameterize the Q function:

[0084] Q(xg)=tr(R T WR),

[0085] Where R∈SO(n), W is the weight matrix, and tr represents the trace of the matrix. This representation method not only captures the complex relationship between state and action, but also ensures the smoothness and invariance of the Q function.

[0086] The update rule of the Q function uses a matrix exponential function:

[0087]

[0088] Here, exp is the matrix exponential function and ∈ is the learning rate. This update method can perform gradient descent on the manifold of Lie groups, thus ensuring the effectiveness of the update.

[0089] This parameterization ensures the rotational invariance of the Q function and allows for flexible control of the impact of different dimensions on the Q value by adjusting the weight matrix W. For example, by adjusting W, the system can prioritize energy consumption optimization for high-load servers, thereby improving overall energy efficiency.

[0090] Obtain historical operating data of the server cluster, including load changes, energy consumption changes, etc. Use this data to train the Q function model and continuously optimize the weight matrix W to adapt to different server load patterns.

[0091] Through the above steps, the method of the present invention can effectively learn and optimize server power management policies. In practical applications, this method can dynamically adjust power supply according to the real-time status and load of the server, thereby maximizing energy efficiency while ensuring service quality.

[0092] Preferably, in one embodiment of the present invention, the acquisition step can be further refined. Specifically, the server cluster's load metrics are collected in real time through the server monitoring system. These metrics may include CPU usage, memory usage, network traffic, etc. Simultaneously, the server cluster's power consumption data is acquired from the power management module. This data may include real-time power consumption, voltage, current, and other information for each server. Furthermore, the system also receives user-entered power demand data for electrical devices, which may include information such as expected quality of service levels and peak business hours.

[0093] This refined acquisition process provides more comprehensive and accurate data support for subsequent processing. For example, by analyzing the time series of CPU usage, the system can predict future load trends; by comparing the energy consumption data of different servers, it can identify less energy-efficient devices; and by considering user demand data, it can optimize energy use while ensuring service quality.

[0094] Through this comprehensive and detailed data acquisition, the method of the present invention provides a rich set of environmental information to the deep reinforcement learning model, enabling more intelligent and efficient power management decisions. This not only improves the overall energy efficiency of the system but also enables flexible adjustments based on actual needs to meet the service requirements of different scenarios.

[0095] The method of the present invention adopts an innovative topology embedding algorithm when constructing the server state representation. Specifically, the algorithm first defines the server state set and manifolds Manifold It can be understood as an abstract mathematical space that is used to represent all possible server states. Next, the algorithm constructs a homeomorphism φ that transforms the server states from the manifold Mapping to Euclidean space R d The core of this step is to maintain the topological relationship between states while simplifying the complex state space into a more tractable Euclidean space.

[0096] Preferably, in one embodiment of the present invention, the homeomorphism mapping φ can be implemented by a neural network. For example, a multi-layer perceptron (MLP) or a convolutional neural network (CNN) can be used to learn this mapping relationship. Such a design enables the system to automatically learn the most suitable state representation without the need for manual feature design. Through this mapping, each server state s i We can get its representation x in Euclidean space i :

[0097] φ(s i )=x i ,

[0098] The advantage of this representation method is that it not only preserves the topological structure of the original state space, but also transforms the state into a form that is convenient for mathematical processing and machine learning. In the action space, the distance, similarity and other indicators between states can be easily calculated, which is crucial for the subsequent decision-making process. The method of the present invention cleverly uses the concept of group theory when constructing the action space. Specifically, the method defines an action group G = {g j |g j :R d →R d ,j=1,2,…,m}, where each element g j Represents transformations in Euclidean space. These transformations can be mapped to actual server management operations, such as adjusting CPU frequency, changing memory allocation, or adjusting network bandwidth.

[0099] Preferably, in one embodiment of the present invention, the action group G can be designed as a Lie group. This allows the continuous nature of Lie groups to achieve smooth changes in the action space. For example, the special orthogonal group SO(n) or the special linear group SL(n) can be used to represent the actions. This design enables the system to make fine adjustments in a continuous action space, rather than being limited to discrete action selection.

[0100] The method also defines a group action · such that g j ·x i =y i , where y i To perform action g jThe new state after the action. This definition is very intuitive, describing how the action changes the system state. Through group action, the effects of different management decisions can be easily simulated and predicted, which provides a theoretical basis for subsequent decision optimization. In the process of constructing the Q function, the method of the present invention innovatively applies Lie group theory. First, the method defines the Q function Q:R d ×G→R, this function maps states and actions to a real-valued value representing the expected long-term reward of taking an action in a given state.

[0101] In a preferred embodiment of the present invention, a special orthogonal group SO(n) is used to parameterize the Q function so that Q(x, g) = tr(R T WR). Here, R∈SO(n), W is the weight matrix, and tr represents the matrix trace. This parameterization method has several significant advantages: first, it ensures the rotation invariance of the Q function, which is particularly useful when dealing with high-dimensional state spaces; second, by adjusting the weight matrix W, the influence of different dimensions on the Q value can be flexibly controlled. The method uses the matrix exponential function as the update rule of the Q function:

[0102]

[0103] Here, exp is the matrix exponential function, and ∈ is the learning rate. This update method ensures that the updated R matrix remains in the SO(n) group, thereby ensuring the consistency of the Q function parameterization. Furthermore, the use of the matrix exponential function allows the gradient update to be performed in the tangent space of the Lie group, which is theoretically more reasonable and also leads to better convergence properties.

[0104] Transformation is used to parameterize the strategy, so that the system can adapt to complex server management scenarios. The method of the present invention introduces the Transformation is a powerful mathematical tool. First, the method defines the policy function π:R d →G, which maps states to actions. Innovatively, the method uses Transformation to parameterize the strategy:

[0105]

[0106] Preferably, in one embodiment of the present invention, the characteristics of the policy can be controlled by adjusting the parameters a, b, c, and d. For example, by increasing |a| and |d|, the policy can be made more sensitive in certain regions of the state space; by adjusting b and c, the overall tendency of the policy can be changed. This flexible parameterization approach enables the system to adapt to various complex server management scenarios.

[0107] The transformation can express complex nonlinear strategies within a limited parameter space while maintaining good mathematical properties (such as conformality and bijectivity). For example, during peak server load, by adjusting the parameters a, b, c, and d, the system can be made more sensitive to load changes, allowing timely adjustments to optimize energy consumption.

[0108] Collect user-inputted data on power demand (e.g., expected service quality, peak hours, etc.). Use this data to train the policy function π and adjust it based on actual operating conditions to maximize the expected long-term return J(π).

[0109] The optimization objective function of the method is defined as:

[0110]

[0111] Here, x follows the state distribution under policy π. This objective function measures the expected long-term reward under the current policy. By maximizing this objective function, the system can continuously improve its power management strategy to achieve a better balance between energy efficiency and performance.

[0112] The Riemann-Zeta function is used to design exploration strategies, adjust exploration probability, and balance the relationship between utilization and exploration. The method of the present invention adopts the innovative Riemann-Zeta function in the design of exploration strategy. Specifically, the method defines the exploration probability Where ζ(s) is the Riemann-Zeta function. The uniqueness of this design is that it uses the characteristics of the Riemann-Zeta function to adjust the exploration probability.

[0113] Monitor the performance and energy consumption of the adjusted server cluster and collect multi-dimensional monitoring data (such as service response time, request processing capacity, CPU utilization, etc.). Based on this data, dynamically adjust the exploration probability p(x), enabling the system to flexibly switch between exploration and exploitation strategies in different states.

[0114] Preferably, in one embodiment of the present invention, the parameter s can be adjusted according to the specific needs of the system. When the s value is small, the exploration probability distribution is more uniform, which is conducive to extensive exploration in the entire state space; when the s value is large, the exploration probability will be concentrated in certain areas of the state space, which is conducive to in-depth exploration in known potential high-return areas. For example, in the initial stage of the system, a smaller s value (such as s = 2) can be selected to encourage extensive exploration; as learning proceeds, the s value can be gradually increased (such as s = 3 or 4) to concentrate on exploring potential areas. By adjusting the parameter s, a trade-off can be made between extensive exploration and concentrated exploration. This ensures that the system can find the optimal strategy at different stages.

[0115] Based on the above exploration probability, the method constructs the exploration strategy π exp (x). The strategy executes the current optimal strategy π(x) with probability 1-p(x) and performs random actions with probability p(x). This design cleverly balances the relationship between exploitation and exploration. Near states with known higher rewards, the system tends to execute the current optimal strategy; while near states with unknown or lower rewards, the system is more likely to try random actions to discover potential better strategies. The method of the present invention not only focuses on the generation of strategies, but also pays attention to feedback and adjustments after the execution of strategies. Specifically, the method includes the step of adjusting the power supply of the server cluster based on the generated server power management and scheduling scheme. This step closely combines the theoretical model with actual operations to ensure the practical feasibility of the strategy.

[0116] Preferably, in one embodiment of the present invention, power supply adjustments can be implemented at multiple levels. For example, at the macro level, the total power consumption of the entire server cluster can be adjusted; at the meso level, power distribution between different racks or zones can be adjusted; and at the micro level, parameters such as the CPU frequency and memory frequency of individual servers can be fine-tuned. This multi-level adjustment strategy enables the system to optimize energy usage at different granularities.

[0117] Next, the method also includes the step of monitoring the performance and energy consumption of the adjusted server cluster. This step provides real-time feedback to the system, enabling it to promptly evaluate the effectiveness of the adjustment policy. Preferably, monitoring metrics may include, but are not limited to, service response time, request processing capacity, CPU utilization, memory usage, network throughput, and power efficiency (performance per watt). This multi-dimensional monitoring data provides a rich basis for subsequent policy optimization.

[0118] Based on the monitoring results, the method further includes the step of updating the deep reinforcement learning model. This step demonstrates the adaptability and continuous learning capabilities of the method. By continuously feeding actual operational data into the model, the system can dynamically adjust its decision-making strategy to adapt to changing server loads and environmental conditions.

[0119] The method of the present invention also incorporates an intelligent triggering mechanism to ensure the system always operates at optimal levels. Specifically, the method includes the steps of setting energy consumption thresholds and performance indicators. These thresholds and indicators serve as reference benchmarks for the system's operating status.

[0120] Preferably, in one embodiment of the present invention, the energy consumption threshold can be set at 80% of the server's rated power, while the performance indicator can be set at a service response time of no more than 100 milliseconds. These specific values ​​can be adjusted based on actual application scenarios and requirements. For example, for high-performance computing tasks, a higher energy consumption threshold may be allowed in exchange for better performance; while for conventional web services, greater emphasis may be placed on energy consumption control.

[0121] The method further stipulates that when the server cluster's energy consumption exceeds the energy consumption threshold or its performance falls below the performance indicator, re-execution of the processing steps is triggered. This design ensures that the system can promptly respond to environmental changes. For example, if a sudden increase in load causes a decrease in service quality, the system will automatically reassess the current state and generate a new management policy. Similarly, if energy consumption increases abnormally, the system will quickly adjust to optimize energy utilization.

[0122] Through this dynamic adjustment and self-optimization mechanism, the method of the present invention can always maintain an efficient and stable operating state in a complex and changeable server operating environment, and achieve an optimal balance between energy consumption and performance.

[0123] The present invention also provides a server power management and scheduling system based on deep reinforcement learning. The design of this system corresponds to the above method and aims to achieve efficient and intelligent server power management.

[0124] The system includes an acquisition module 1, which is responsible for acquiring real-time load data, power consumption data, and power-consuming device demand data from the server cluster. The design of acquisition module 1 fully considers the diversity and real-time nature of the data, providing comprehensive and timely information support for subsequent processing. Preferably, acquisition module 1 can collect data in real time through a distributed sensor network to ensure data accuracy and timeliness. For example, a load monitor can be deployed on each server to collect real-time metrics such as CPU utilization and memory usage. Simultaneously, energy consumption monitoring devices can be installed in the power supply units to record real-time power consumption data.

[0125] Processing Module 2 is the core component of the system, responsible for performing a series of complex data processing and decision optimization tasks. First, Processing Module 2 constructs a representation of the server state based on real-time load data using a topological embedding algorithm. This step is performed by the State Representation Submodule 21, which converts raw data into an abstract mathematical representation. Preferably, State Representation Submodule 21 can employ a deep neural network architecture, such as an autoencoder or graph neural network, to capture the inherent structure and characteristics of the server state.

[0126] Next, the action space construction submodule 22 in processing module 2 constructs the action space using group theory based on the server state representation. This innovative design enables the system to mathematically rigorously define and manipulate possible management actions. Preferably, the action space construction submodule 22 can use a differentiable parameterization method to represent the action group, thereby supporting end-to-end gradient optimization.

[0127] Based on the constructed action space, the Q-function approximation submodule 23 uses Lie group theory to construct a Q-function approximation. This step is the core of deep reinforcement learning. Through continuous learning and updating, the Q-function approximation submodule 23 estimates the long-term rewards of different state-action pairs. Preferably, the Q-function approximation submodule 23 can adopt a deep Q-network (DQN) or its variants, such as dual DQN or prioritized experience replay DQN, to improve learning efficiency and stability.

[0128] The strategy optimization submodule 24 uses the Q function approximation Transform optimization strategy. This unique optimization method enables the system to express complex nonlinear strategies within a limited parameter space. Preferably, the policy optimization submodule 24 can be combined with a policy gradient method, such as the REINFORCE algorithm or the dominant actor-critic (A2C) algorithm, to further improve the effectiveness of policy optimization.

[0129] The exploration strategy design submodule 25 designs an exploration strategy based on the optimization strategy and utilizes the Riemann-Zeta function. This innovative design provides the system with a flexible exploration mechanism that strikes a balance between leveraging known information and exploring unknown possibilities. Preferably, the exploration strategy design submodule 25 can dynamically adjust the parameters of the Riemann-Zeta function to adapt to the exploration needs at different stages.

[0130] Output Module 3 is responsible for generating and outputting a server power management and scheduling plan based on the exploration strategy. Output Module 3 goes beyond simple data output and also includes visualization and interpretation of the plan. Preferably, Output Module 3 can generate a detailed scheduling report, including information such as power configuration recommendations for each server, expected energy savings, and performance impact assessments. Furthermore, Output Module 3 can provide an interactive graphical interface, allowing administrators to fine-tune the output plan based on actual needs.

[0131] Through this modular design, the system of the present invention realizes full-process automation from data acquisition to strategy generation. Each module has been carefully designed to not only independently complete its specific functions, but also seamlessly cooperate with other modules to form an efficient and intelligent server power management and scheduling system. This system can not only significantly improve the energy efficiency of the data center, but also ensure the quality of service, providing strong technical support for the operation of modern data centers. In order to verify the superiority of the server power management and scheduling method based on deep reinforcement learning and the system thereof of the present invention, the present invention conducted a series of experimental comparisons. The following will introduce in detail the settings, detection methods and result analysis of the embodiments and comparative examples.

[0132] Example 1 adopts the method of the present invention, including topological embedded state representation, group theory constructed action space, Lie group theory constructed Q function approximation, The transformation optimization strategy and the exploration strategy based on Riemann-Zeta function are applied to a medium-sized data center consisting of 100 servers with dual-core Intel Xeon processors and 128GB memory.

[0133] Comparative Example 1 uses a traditional fixed threshold strategy for server power management. This strategy decides whether to power on or off a server based on a preset CPU utilization threshold, without considering dynamic load changes and long-term optimization.

[0134] Comparative Example 2 adopts a simple deep Q learning (DQN) method and uses a multilayer perceptron as the Q network, but does not adopt the special mathematical structure and optimization strategy of the present invention.

[0135] The test lasted 30 days, simulating various load scenarios, including daily fluctuations, sudden high loads, and low load periods. Key metrics tested included power usage effectiveness (PUE), quality of service (QoS, measured by average response time), system stability (measured by standard deviation), and convergence speed (measured by the time required to reach a stable strategy).

[0136] The detection method is as follows:

[0137] 1. PUE: Calculated by dividing the total energy consumption of the data center by the energy consumption of the IT equipment.

[0138] 2.QoS: Record and calculate the average response time of all requests.

[0139] 3. System stability: Calculate the standard deviation of daily PUE over a 30-day period.

[0140] 4. Convergence speed: The number of days required for the algorithm to reach a stable strategy (PUE variation of less than 1% for 7 consecutive days) is recorded.

[0141] The test results are shown in Table 1:

[0142] Table 1. Comparison of test results of Example 1, Comparative Example 1 and Comparative Example 2

[0143] index Example 1 Comparative Example 1 Comparative Example 2 PUE 1.15 1.42 1.28 QoS (ms) 75 120 95 Stability (σ_PUE) 0.02 0.08 0.05 Convergence speed (days) 5 N / A 12

[0144] From the test results in Table 1, it can be seen that the method of the present invention (Example 1) shows significant advantages in all indicators. The specific analysis is as follows:

[0145] 1. Energy Efficiency: The PUE of Example 1 reached 1.15, which was 19.0% lower than that of Comparative Example 1 and 10.2% lower than that of Comparative Example 2. This shows that the method of the present invention can more accurately regulate the server power supply and reduce unnecessary energy waste.

[0146] 2. Service Quality: The average response time of Example 1 is 75ms, which is 37.5% and 21.1% shorter than that of Comparative Example 1 and Comparative Example 2, respectively. This shows that the method of the present invention not only saves energy but also ensures better service quality, thanks to its accurate load prediction and reasonable resource allocation.

[0147] 3. System Stability: The PUE standard deviation of Example 1 is only 0.02, much lower than that of the other two methods. This shows that the method of the present invention can maintain stable performance without large fluctuations under complex and changing load conditions.

[0148] 4. Convergence Speed: Example 1 reached a stable strategy in just 5 days, while Comparative Example 2 required 12 days. Since Comparative Example 1 is a fixed strategy with no learning process, this metric is not applicable. This demonstrates that the method of the present invention has a higher learning efficiency and can adapt to environmental changes more quickly.

[0149] These results fully demonstrate the superiority of the proposed method. Its core innovations include topologically embedded state representation, which enables the system to better understand the internal structure of server states; the action space constructed by group theory provides a richer selection of management strategies; and the Q function approximation constructed by Lie group theory improves learning stability. The transformation optimization strategy and the exploration strategy based on the Riemann-Zeta function have significantly improved the optimization effect and learning efficiency.

[0150] Of particular note, the proposed method maintains excellent service quality while ensuring high energy efficiency, resolving the energy-performance trade-off often faced by traditional methods. Furthermore, its rapid convergence speed means the system can quickly adapt to new environments, which is particularly important in dynamically changing data center scenarios.

[0151] In summary, the method of the present invention has demonstrated excellent performance in practical applications and provides strong technical support for the intelligent and efficient management of modern data centers.

[0152] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, replacement, or improvement made within the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A server power management and scheduling method based on deep reinforcement learning, characterized in that: include: The acquisition steps include: Obtain real-time load data, power consumption data, and power equipment demand data of the server cluster; Processing steps include: Based on the real-time load data, construct a server status representation using a topology embedding algorithm; constructing an action space using group theory based on the server state representation; Based on the action space, the Q function approximation is constructed using Lie group theory; According to the Q function approximation, using Transform optimization strategies; Based on the optimization strategy, an exploration strategy is designed using the Riemann-Zeta function; Output steps include: Generate a server power management and scheduling plan based on the exploration strategy and output the plan; The use of Lie group theory to construct the Q function approximation specifically includes: Define the Q function Q:R d ×G→R; The Q function is parameterized using the special orthogonal group SO(n) so that Q(x,g)=tr(R T WR); Adopt the matrix exponential function update rule: The Riemann-Zeta function is used to design an exploration strategy, which specifically includes: Defining exploration probability Where ζ(s) is the Riemann-Zeta function; Construct exploration strategy π exp (x), executes policy π(x) with probability 1-p(x), and performs a random action with probability p(x).

2. The method according to claim 1, characterized in that The obtaining step specifically includes: Collecting load indicators of the server cluster in real time through a server monitoring system; Acquire power consumption data of the server cluster from a power management module; Receive power equipment demand data input by users.

3. The method according to claim 1, characterized in that The use of the topology embedding algorithm to construct the server state representation specifically includes: Define server status collection and manifolds Constructing homeomorphic mappings Remove server status from the manifold Mapping to Euclidean space R d ; Get the representation x of the server state in Euclidean space i ∈R d .

4. The method according to claim 1, wherein The use of group theory to construct the action space specifically includes: Define action group G = {g j |g j :R→R d ,j=1,2,…,m}, where each element g j represents a transformation in Euclidean space; Define a group action · such that g j ·x i =y i , where y i To perform action g j The new state after.

5. The method according to claim 1, wherein The adoption The transformation optimization strategies specifically include: Define the policy function π:R d →G; use Transformation parameterization strategy: Where ad-bc=1; Optimize the objective function Where x follows the state distribution under policy π.

6. The method according to claim 1, wherein Also includes: Adjusting the power supply of the server cluster based on the server power management and scheduling solution; Monitor the performance and energy consumption of the adjusted server cluster; Based on the monitoring results, the deep reinforcement learning model is updated.

7. The method according to claim 1, characterized in that Also includes: Set energy consumption thresholds and performance indicators; When the energy consumption of the server cluster exceeds the energy consumption threshold or the performance is lower than the performance index, the processing step is triggered to be re-executed.

8. A server power management and scheduling system based on deep reinforcement learning that executes the method according to any one of claims 1 to 7, characterized in that: include: The acquisition module is used to obtain the real-time load data, power consumption data and power equipment demand data of the server cluster; Processing module for: Based on the real-time load data, construct a server status representation using a topology embedding algorithm; constructing an action space using group theory based on the server state representation; Based on the action space, the Q function approximation is constructed using Lie group theory; According to the Q function approximation, using Transform optimization strategies; Based on the optimization strategy, an exploration strategy is designed using the Riemann-Zeta function; The output module is used to generate a server power management and scheduling plan based on the exploration strategy and output the plan.

Images