A self-adaptive frequency adjustment method and system for SPARK cluster, a terminal and a storage medium

Abstract

The application relates to the technical field of distributed computing, and discloses a SPARK cluster-oriented adaptive frequency adjustment method and system, a terminal and a storage medium. The method comprises the following steps: constructing a state vector of a SPARK-on-YARN cluster; generating an action vector based on an actor-critic network model of a proximal policy optimization algorithm; converting the action vector into a CPU frequency regulation instruction and delivering the CPU frequency regulation instruction to a working node; calculating an instant reward according to the adjusted state, and storing interaction data through an experience replay pool; and updating the network model when the data volume reaches a threshold. The application breaks away from the dependence on preset rules, can adaptively respond to cluster load changes and different job types, can guarantee job performance and service quality, can maintain cluster operation stability, and can realize intelligent balance optimization of energy efficiency and performance through a multi-dimensional state perception and an online learning mechanism while significantly reducing energy consumption.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed computing, and particularly relates to a SPARK cluster-oriented adaptive frequency adjustment method and system, a terminal and a storage medium. BACKGROUND

[0002] In the field of big data and distributed computing, Apache SPARK (a large-scale data processing engine) has become a core computing framework for processing massive data. Its in-memory computing feature significantly improves data processing efficiency, but also brings significant energy consumption growth. SPARK-on-YARN (a deployment mode that runs the Apache SPARK computing framework as an application on the YARN resource management system) as its mainstream deployment mode, hands over resource management to YARN (Yet Another Resource Negotiator), which improves resource utilization but also makes cluster-level energy consumption optimization more complex.

[0003] Currently, the energy consumption optimization of SPARK clusters is mainly based on heuristic rules for task scheduling and resource allocation. This method relies on expert experience to preset rules, such as assigning different types of nodes or setting static CPU frequencies according to the CPU-intensive or I / O-intensive characteristics of the job. This method is simple to implement, but lacks flexibility, making it difficult to adapt to dynamic and changing production environment loads, and performs poorly when faced with unknown or mixed jobs.

[0004] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0005] The main purpose of the present application is to provide a SPARK cluster-oriented adaptive frequency adjustment method and system, a terminal and a storage medium, which aims to solve the problem in the prior art that the SPARK cluster energy consumption optimization method relies on fixed rules and cannot adapt to unknown job types and dynamically changing cluster loads.

[0006] The purpose of the present application can be achieved by the following technical solutions:

[0007] A SPARK cluster-oriented adaptive frequency adjustment method, comprising the following steps:

[0008] Collecting node-level state information of each working node in the SPARK-on-YARN cluster and job-level state information of the current running job, and constructing a first state vector according to the node-level state information and the job-level state information;

[0009] Based on the first state vector, an action vector is generated using a preset actor-critic network model based on a proximal policy optimization algorithm;

[0010] The action vector is converted into a CPU frequency adjustment instruction, and the CPU frequency adjustment instruction is sent to the target working node to control the target working node to adjust the CPU frequency.

[0011] The second state vector of the cluster operation status after frequency adjustment is obtained, the instant reward is calculated based on the second state vector, and the interaction data is stored in the experience replay pool. When the amount of data in the experience replay pool reaches the threshold, the actor-critic network model is updated.

[0012] The interaction data includes the first state vector, the action vector, the instant reward, and the second state vector.

[0013] Furthermore, the pre-training process of the actor-critic network model is as follows:

[0014] Using historical workload logs or training data generated by simulators, the actor-critic network architecture is initially trained to obtain an actor-critic network model with an initial strategy for node CPU frequency regulation.

[0015] Furthermore, the step of generating action vectors based on the first state vector using a preset actor-critic network model based on a proximal policy optimization algorithm includes:

[0016] The first state vector is input into the actor network in the actor-critic network model to obtain the probability distribution of DVFS actions of each node;

[0017] The probability distribution is filtered for legality based on the action mask to remove illegal frequency options;

[0018] The filtered probability distribution is sampled to obtain the action vector.

[0019] Furthermore, the step of obtaining the second state vector of the cluster's operating state after frequency adjustment, and calculating the instantaneous reward based on the second state vector, includes:

[0020] After a fixed time interval, obtain the second state vector of the cluster's operating status after frequency adjustment;

[0021] The immediate reward for this action is calculated based on the second state vector and the multi-objective reward function.

[0022] Furthermore, the multi-objective reward function includes an energy consumption term, a default penalty term, and a frequency stability term.

[0023] Further, the updating the actor-critic network model when the amount of data in the experience replay pool reaches a threshold value comprises:

[0024] triggering a policy updating process when the accumulated amount of data samples in the experience replay pool reaches a predetermined threshold value;

[0025] calculating an advantage function and a policy ratio, and iteratively updating parameters of the actor-critic network model according to the advantage function and the policy ratio under the constraint of a clipping mechanism.

[0026] Further, the node-level state information comprises CPU utilization, memory utilization and CPU frequency, and the job-level state information comprises job type, stage progress and remaining execution time prediction value.

[0027] In addition, to achieve the above-mentioned purposes, the present application also provides a SPARK cluster-oriented adaptive frequency adjustment system, wherein the SPARK cluster-oriented adaptive frequency adjustment system comprises:

[0028] a monitoring layer configured to collect node-level state information of each working node in a SPARK-on-YARN cluster and job-level state information of a currently running job, and construct a first state vector according to the node-level state information and the job-level state information;

[0029] a decision layer configured to generate an action vector through a preset actor-critic network model based on a proximal policy optimization algorithm according to the first state vector;

[0030] an execution layer configured to convert the action vector into a CPU frequency regulation instruction, and issue the CPU frequency regulation instruction to a target working node to control the target working node to adjust the CPU frequency;

[0031] a learning layer configured to obtain a second state vector of a cluster running state after frequency adjustment, calculate an immediate reward according to the second state vector, and store interaction data into an experience replay pool, and update the actor-critic network model when the amount of data in the experience replay pool reaches a threshold value;

[0032] wherein the interaction data comprises the first state vector, the action vector, the immediate reward and the second state vector.

[0033] In addition, to achieve the above object, the present application also provides a terminal, wherein the terminal comprises a memory, a processor, and a SPARK cluster-oriented adaptive frequency adjustment program stored in the memory and executable on the processor, and the SPARK cluster-oriented adaptive frequency adjustment program implements the steps of the SPARK cluster-oriented adaptive frequency adjustment method when executed by the processor.

[0034] In addition, to achieve the above object, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a SPARK cluster-oriented adaptive frequency adjustment program, and the SPARK cluster-oriented adaptive frequency adjustment program implements the steps of the SPARK cluster-oriented adaptive frequency adjustment method when executed by a processor.

[0035] The present application has the following beneficial effects: through the actor-critic network model based on the proximal policy optimization algorithm, the dependence on preset rules and fixed thresholds is eliminated, real-time responses to cluster load fluctuations and job characteristic changes are achieved, and automatic adaptation to different types of SPARK job loads is achieved; through continuous environmental interaction and online experience learning evolution, the system performance is continuously optimized and improved over time. The node-level state information and the job-level state information are synchronously collected, and through the construction of a composite state vector, the deep feature association between the resource level and the job level is achieved, providing a comprehensive and accurate environmental state description for intelligent decision-making. The interactive data is stored in the experience replay pool to improve the sample utilization efficiency; the model is updated when the data amount reaches a threshold, ensuring the stability and efficiency of the learning process; the system can learn from historical experience and continuously optimize the control strategy. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is a flowchart of a preferred embodiment of the SPARK cluster-oriented adaptive frequency adjustment method of the present application;

[0037] Figure 2 is a flowchart of a double-layer circulation interaction process of the PPOAgent in the SPARK cluster energy consumption optimization of the present application;

[0038] Figure 3 is a structure diagram of a preferred embodiment of the SPARK cluster-oriented adaptive frequency adjustment system of the present application;

[0039] Figure 4 is a structure diagram of a preferred embodiment of the terminal of the present application. DETAILED DESCRIPTION

[0040] The application provides a SPARK cluster-oriented adaptive frequency adjustment method and system, a terminal and a storage medium. To make the purpose, technical solutions and effects of the application clearer and more explicit, the application is further described in detail below with reference to the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application and do not limit the application.

[0041] Those skilled in the art can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which the application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood as having meanings consistent with those in the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as such.

[0042] In addition, if the description of "first", "second" and the like is involved in the embodiments of the application, the description of "first", "second" and the like is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can be explicitly or implicitly included at least one of the features. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor within the protection scope required by the application.

[0043] The application provides a SPARK cluster-oriented adaptive frequency adjustment method, system, terminal and storage medium, aiming to solve the problem that the SPARK cluster energy consumption optimization method in the prior art relies on fixed rules and cannot adapt to unknown job types and dynamically changing cluster load. The SPARK cluster-oriented adaptive frequency adjustment system is referred to as PPOAgent or agent hereinafter.

[0044] The SPARK cluster-oriented adaptive frequency adjustment method (inner scheduling loop) of the preferred embodiment of the application is as shown in Figure 1 The SPARK cluster-oriented adaptive frequency adjustment method comprises the following steps:

[0045] S10, collecting node-level state information of each work node in the SPARK-on-YARN cluster and job-level state information of the current running job, and constructing a first state vector according to the node-level state information and the job-level state information.

[0046] Specifically,

[0047] The node-level state information includes: CPU utilization, memory utilization, current CPU frequency, and real-time power consumption data collected through IPMI interface or rapld tool of each node; the job-level state information includes: application ID, application type, current completed task proportion, remaining execution time based on historical execution progress prediction, and preset service level agreement SLA deadline.

[0048] The node-level state information includes: CPU utilization, memory utilization, current CPU frequency, and real-time power consumption data collected through IPMI interface or rapld tool of each node; the job-level state information includes: application ID, application type, current completed task proportion, remaining execution time based on historical execution progress prediction, and preset service level agreement SLA deadline.

[0049] The application type automatic identification method includes: CPU-intensive job, I / O-intensive job and mixed job.

[0050] The node-level state information includes: CPU utilization, memory utilization, current CPU frequency, and real-time power consumption data collected through IPMI interface or rapld tool of each node; the job-level state information includes: application ID, application type, current completed task proportion, remaining execution time based on historical execution progress prediction, and preset service level agreement SLA deadline.

[0051] The expression of the first state vector is:

[0052] ;

[0053] Wherein, represents the first state vector, is a feature vector, and each element in it represents a dimension describing the current system condition; and respectively represent the CPU and memory utilization of node at time point ; represents the CPU frequency of node at time point ; represents the application type of the th SPARK application; represents the th SPARK application​ In time The remaining execution time.

[0054] S20, generating an action vector by a preset actor-critic network model based on a proximal policy optimization algorithm according to the first state vector.

[0055] Specifically, it comprises:

[0056] S21, inputting the first state vector into an actor network in the actor-critic network model to obtain a probability distribution of each node DVFS action;

[0057] In the above steps, the actor network (Actor Network) is the core component of the actor-critic model, which receives the first state vector (including node-level indicators such as CPU utilization, memory utilization, current frequency, and job-level indicators such as job type, remaining time, etc.). Through neural network forward propagation, the actor network outputs the original probability distribution of DVFS actions (frequency reduction, maintenance, frequency increase) on each work node. This is based on the policy learning mechanism of the PPO algorithm, in which the actor network parameters are optimized by gradient ascent to maximize the long-term cumulative reward. The probability distribution generation embodies the policy function approximation of deep reinforcement learning, which maps high-dimensional states to interpretable action strategies.

[0058] S22, filtering the probability distribution based on an action mask to remove illegal frequency options;

[0059] In the above steps, the action mask (Action Mask) is an external safety mechanism that dynamically generates a binary mask (0 represents an illegal action and 1 represents a legal action) based on hardware constraints (such as CPU frequency physical upper and lower limits) and the current system state. The mask is multiplied element by element with the probability distribution, forcing the probability of illegal actions to be zero, and the result is renormalized. This ensures that the action conforms to the physical limit and prevents frequency options from causing hardware damage or system instability.

[0060] S23, sampling the filtered probability distribution to obtain an action vector.

[0061] Wherein, the definition of the action vector is:

[0062] ;

[0063] Wherein, represents the action vector at time point , represents frequency reduction, 0 represents maintenance, represents frequency increase, ​The preset frequency adjustment step is 0.1 GHz to 2.7 GHz, and the final frequency is constrained in the physical frequency range supported by the processor, and the embodiment takes 0.1 GHz.

[0064] In the above step, the filtered probability distribution is subjected to Categorical Sampling, and an action is randomly selected for each node independently. Sampling introduces exploration, allowing the agent to try different actions during the training phase to discover better strategies, while switching to greedy selection (selecting the action with the highest probability) during deployment to stabilize performance. The sampling mechanism balances exploration and utilization, and is a key step for the convergence of the optimal policy in reinforcement learning.

[0065] It should be noted that the Proximal Policy Optimization (PPO) algorithm is an improved reinforcement learning algorithm based on policy gradient, which solves the problems of step selection difficulty and training instability in traditional policy gradient algorithms by limiting the amplitude of policy update. The algorithm limits the update amplitude of the policy network by clipping the objective function to ensure the stability of the training.

[0066] It should be emphasized that DVFS (Dynamic Voltage and Frequency Scaling) is a basic and powerful hardware energy-saving technology. It realizes energy efficiency optimization by dynamically coordinating the reduction of processor operating voltage and frequency.

[0067] Further, the pre-training process of the actor-critic network model is: using historical workload logs or training data generated by a simulator, the actor-critic network architecture is preliminarily trained to obtain an actor-critic network model with an initial strategy of node CPU frequency regulation.

[0068] In other words, the actor-critic network model includes an offline pre-training phase before deployment and implementation: using historical workload logs or training data generated by a simulator, the actor-critic network architecture is preliminarily trained to obtain an actor-critic network model with an initial strategy of node CPU frequency regulation, and then fine-tuned through online learning.

[0069] Need to explain, actor-critic network (Actor-Critic Network, A2C) is a kind of mixed architecture combining 'policy learning' and 'value function learning', in which the actor network is responsible for taking action, and the critic network is used to evaluate whether the action is good. The critic network provides low-variance advantage estimation for the actor by approximating the value function, which significantly reduces the high-variance problem of pure policy gradient; The clipping and KL constraint mechanism of PPO applies a controllable step to each update of the actor, avoiding training collapse caused by policy mutation. At the same time, the high-dimensional representation ability of deep neural network enables the critic to accurately approximate the complex state-value mapping, so as to minimize the estimation error.

[0070] S30, the action vector is converted into a CPU frequency regulation instruction, and the CPU frequency regulation instruction is sent to the target work node to control the target work node to adjust the CPU frequency.

[0071] Specifically:

[0072] The action vector is parsed into specific operating system level CPU frequency regulation instructions by the control module, and is sent to the target work node for execution using the SSH protocol, completing the dynamic adjustment of the CPU working frequency.

[0073] S40, a second state vector of the frequency adjusted cluster running state is obtained, the immediate reward is calculated according to the second state vector, and the interaction data is stored in the experience replay pool. When the amount of data in the experience replay pool reaches a threshold, the actor-critic network model is updated.

[0074] Among them, the interaction data includes the first state vector , the action vector , the immediate reward And the second state vector . The form of the interaction data is four-tuple .

[0075] Need to explain, the second state vector and the first state vector have the same construction method.

[0076] Step S40 specifically includes:

[0077] S41, after a fixed time interval, the second state vector of the frequency adjusted cluster running state is obtained; according to the second state vector, the immediate reward of this action is calculated according to the multi-objective reward function, and the interaction data is stored in the experience replay pool;

[0078] Furthermore, the multi-objective reward function includes an energy consumption term, a default penalty term, and a frequency stability term, and the calculation formula for the multi-objective reward function is as follows:

[0079] ;

[0080] in, Indicates an immediate reward. This represents the energy consumption item. Indicates penalties for breach of contract. Represents the frequency stability term. gamma , lambda , mu These are the weighting coefficients for energy consumption, SLA default, and stability, respectively. In this embodiment, the weighting coefficients are statically configured and set as follows: lambda =0.5, gamma =0.4, mu =0.1.

[0081] The formula for calculating energy consumption is:

[0082] ;

[0083] in, This represents the energy consumption item. Set fixed parameters for the nodes (hardware-dependent). For the node in the current time step The cube of the frequency, This is the cube of the frequency of the next time step.

[0084] It should be noted that the energy consumption term measures the energy consumption change of each node based on the DVFS cubic power consumption model. The expression for the cubic power consumption model is as follows: The agent can directly obtain positive rewards by reducing the frequency.

[0085] It should be noted that, in order to accurately describe the energy consumption of the SPARK cluster, this invention models energy consumption as the sum of task execution energy consumption and idle energy consumption:

[0086] ;

[0087] in, It represents the sum of energy consumption during task execution and energy consumption during idle time. This represents the basic energy consumption of a node under low load. This indicates the additional energy consumption during task execution.

[0088] According to the dynamic power consumption model of CMOS circuits, the dynamic power consumption of a CPU can be expressed as:

[0089] ;

[0090] in This indicates the CPU's dynamic power consumption. C For switched capacitors, V Operating voltage f This refers to the CPU frequency. Since voltage and frequency typically have an approximately linear relationship, it can be further simplified to:

[0091] ;

[0092] Therefore, a slight reduction in CPU frequency can lead to a cubic-level decrease in power consumption, which is the theoretical basis for DVFS's energy efficiency. The total energy consumption of the cluster can be expressed by the following formula:

[0093] ;

[0094] in, This represents the total energy consumption of the cluster. N Indicates the number of worker nodes. Representing nodes respectively i idle power consumption and Representing nodes respectively i At the point of time t CPU and memory utilization and These are weighting factors related to the type of work.

[0095] Therefore, the energy consumption of the SPARK cluster of this invention can be calculated using the cubic power consumption model.

[0096] In the energy consumption model of this invention, parameters α i and β i This is used to reflect the sensitivity of different job types to computing and storage resources. Since SPARK jobs can generally be classified into CPU-intensive, memory I / O-intensive, and hybrid types at the operator level, this invention adopts a static configuration method, directly assigning values ​​based on the job type label when the job is submitted, without the need for online training or dynamic estimation.

[0097] Specifically:

[0098] CPU-intensive tasks (such as iterative machine learning tasks in Spark MLlib, PageRank, KMeans, etc.): are sensitive to CPU resource consumption, and energy consumption is highly correlated with CPU frequency; static settings α i Larger (ranging from 0.7 to 0.9), β i Smaller (ranging from 0.1 to 0.3).

[0099] Memory I / O intensive jobs (e.g. SPARK SQL queries, TPC-DS like OLAP analysis tasks): mainly affected by memory bandwidth and data exchange efficiency, CPU frequency upgrade brings limited performance improvement; static setting β i Large (take 0.6 to 0.8), α i Small (take 0.2 to 0.4).

[0100] Mixed jobs (e.g. Sort, WordCount, GraphX partial tasks): both CPU and memory resources have high occupancy; static setting α i And β i The weight is close (take 0.4 to 0.6).

[0101] The above weight values are determined based on the running characteristics of typical SPARK benchmark jobs (HiBench, TPC-DS, MLlib, etc.). When deploying the cluster, the corresponding α i , β i PPOAgent directly calls these parameters when building the energy consumption model, without additional online learning overhead.

[0102] In this embodiment, the energy consumption term directly measures the energy consumption change brought by frequency adjustment according to the power consumption model under DVFS.

[0103] In each time step, it drives the agent to tend to choose the frequency configuration that can reduce the overall energy consumption of the cluster.

[0104] The default penalty term is used to ensure that the SLA constraint of the latest deadline is strictly guaranteed, and is defined as follows:

[0105] ;

[0106] Where, represents the default penalty term, and respectively represent the execution time and the remaining estimated execution time of the task after t of time has passed in the current time step; represents the SLA constraint execution time of the job; M represents a large penalty value.

[0107] In this embodiment, the default penalty term determines whether the job remaining time exceeds the SLA deadline through an indicator function, and triggers a high-weight penalty if violated. Throughout the control process, it acts as a "hard constraint" to prevent the agent from causing task delays for the sake of energy saving.

[0108] The calculation formula of the frequency stability term is:

[0109]

[0110] wherein, represents the frequency stability term, is the frequency of the node i in the current time step, is the frequency of the node i in the next time step, N is the number of worker nodes.

[0111] In this embodiment, the frequency stability term imposes a penalty on excessive frequent adjustments according to the frequency adjustment amplitude of adjacent time steps. It limits the jitter of the policy output, making the frequency adjustment more smooth and avoiding negative impacts on the underlying hardware and task execution.

[0112] From the above analysis of the multi-objective reward function, it can be seen that in each control time step, the agent first obtains "energy-saving incentives" through the energy consumption term, while its behavior is constrained by the SLA penalty term to meet the time requirement, and then suppresses excessive frequent frequency switching through the stability term. The three work together in the reward function to form a closed-loop incentive signal that prioritizes energy saving, time requirement, and stable operation.

[0113] S42, periodically randomly sample a small batch of data from the experience replay pool, and iteratively update the parameters of the actor-critic network model based on the clipping objective function and generalized advantage estimation of the proximal policy optimization algorithm, to realize online optimization and adaptive learning of decision-making ability. Specifically:

[0114] When the data samples in the experience replay pool accumulate to a predetermined threshold, the policy update process is triggered;

[0115] Calculate the advantage function and the policy ratio, and iteratively update the parameters of the actor-critic network model based on the advantage function and the policy ratio under the constraint of the clipping mechanism.

[0116] In the above scheme, the proximal policy optimization algorithm uses a clipping objective function to limit the policy update amplitude and ensure training stability.

[0117] Further, the calculation formula of the clipping objective function of the proximal policy optimization algorithm is:

[0118] ​ ;

[0119] wherein, represents a clipping target function of the parameters of the current actor-critic network model, represents an expected value of the sampled time step t ; represents comparing the unclipped target and the clipped target, and taking the smaller one as the final target; represents a new-old policy probability ratio; theta represents the parameters of the current policy network (Actor); represents an advantage function; is a clipping function, which limits the new-old policy probability ratio between , and if the new-old policy probability ratio is too large or too small, it will be truncated to ensure that the policy update will not deviate too much from the old policy, thereby avoiding training shocks or collapse; is a clipping threshold. Through the above target function, the agent can ensure exploration while avoiding frequent and drastic changes in policy, thereby ensuring the stability of training.

[0120] The calculation formula of the new-old policy probability ratio is:

[0121] ;

[0122] wherein, represents a new-old policy probability ratio, represents the probability of the agent selecting action theta in state under the current policy parameter ; represents the probability of the agent selecting action in state under the old policy parameter .

[0123] The calculation formula of the advantage function is:

[0124] ;

[0125] wherein, represents an advantage function, represents the expected return after taking action in state and following policy , represents the baseline value in state .

[0126] It is emphasized that the PPOAgent of the present application takes SPARK-on-YARN cluster as a running environment, and through a closed loop mode of "monitoring-decision-making-executing-feedback-learning", the CPU frequency of each working node is adaptively adjusted at a fixed time step, so as to reduce the total energy consumption under the premise of meeting the job SLA. The overall working process is divided into two levels of outer scheduling cycle and inner control cycle: the outer scheduling cycle is promoted in units of Application, and the inner control cycle is interacted with the environment and the strategy is updated in units of time step.

[0127] wherein, Figure 2 The double-loop interaction mechanism of PPOAgent in SPARK cluster energy consumption optimization is shown. The mechanism includes an outer scheduling cycle (in units of Application) and an inner scheduling cycle (in units of time step), which work together to ensure energy consumption optimization and SLA guarantee at different levels. The outer scheduling cycle includes: when the job enters the YARN queue, the ResourceManager allocates AM or Container for it and starts the Executor; PPOAgent iteratively processes the queue in units of Application, and evaluates the overall effect of the current strategy according to the cumulative return after each job is completed, to provide a better initial strategy for subsequent jobs. The inner scheduling cycle is: during the execution of a single SPARK job, the system performs closed-loop control at a fixed step Δt.

[0128] Specifically, the process of the double-loop interaction mechanism is as follows:

[0129] I. Outer scheduling cycle

[0130] 1. Job submission: when a user submits a SPARK job, the YARN scheduler allocates a container for it and starts the Executor.

[0131] 2. Optimization start: PPOAgent enters the outer loop and takes the job as the optimization object.

[0132] 3. Enter the inner loop: during the job running, the system performs the decision-making and learning process of the inner loop according to the fixed time step.

[0133] 4. Loop exit: when the job is completed, the outer loop ends, and the system waits for the next Application submission and enters the next round of optimization.

[0134] The role of the outer loop is to ensure that each job can be optimized and controlled by PPOAgent, and to form an overall energy efficiency evaluation after the job is completed.

[0135] II. Inner scheduling cycle

[0136] During the execution of a single SPARK job, the system performs an inner loop at a fixed time step Δt. Each inner loop includes the entire process from steps S10 to S40 above. After step S40 ends, it is determined whether the current job is completed. If the job is not completed, it returns to step S10 and continues the interaction of the next time step. If the job is completed, it exits the inner loop and returns to the outer loop.

[0137] The inner loop's function is to dynamically sense and adjust the CPU frequency during job execution, achieving the goals of energy saving, ensuring SLA, and reducing jitter.

[0138] In summary, the outer loop ensures optimization across different applications, guaranteeing the method's versatility at the cluster level. The inner loop enables fine-grained, dynamic adjustment at the time step level, improving energy efficiency and ensuring SLA. This combination of two loops allows PPOAgent to continuously learn and optimize its strategies across multiple job scenarios, considering both global and local factors.

[0139] The beneficial effects of the adaptive frequency adjustment method for Spark clusters described in this invention are as follows: By using an actor-critic network model based on Proximal Policy Optimization (PPO), it eliminates the dependence on preset rules and fixed thresholds, enabling real-time response to cluster load fluctuations and job characteristic changes, and automatically adapting to different types of Spark job loads; through continuous environmental interaction and online experience learning evolution, system performance is continuously optimized and improved over time. Node-level and job-level state information are collected synchronously, and by constructing a composite state vector, deep feature association between the resource and job levels is achieved, providing a comprehensive and accurate environmental state description for intelligent decision-making. Interaction data is stored in an experience replay pool, improving sample utilization efficiency; model updates are triggered when the data volume reaches a threshold, ensuring the stability and efficiency of the learning process; the system can learn from historical experience and continuously optimize the control strategy. Constraint mechanisms in the reward function ensure that all jobs are completed on time; while reducing energy consumption, user experience and service quality are not affected; and drastic frequency fluctuations are avoided, maintaining cluster operational stability.

[0140] Furthermore, such as Figure 3 As shown, based on the above-described adaptive frequency adjustment method for Spark clusters, this invention also provides an adaptive frequency adjustment system for Spark clusters, wherein the adaptive frequency adjustment system for Spark clusters includes:

[0141] The monitoring layer is used to collect node-level status information of each worker node in the SPARK-on-YARN cluster and job-level status information of the currently running job, and to construct a first state vector based on the node-level status information and the job-level status information.

[0142] a decision layer configured to generate an action vector by a preset actor-critic network model based on a proximal policy optimization algorithm according to the first state vector;

[0143] an execution layer configured to convert the action vector into a CPU frequency regulation instruction and send the CPU frequency regulation instruction to a target working node to control the target working node to adjust the CPU frequency;

[0144] a learning layer configured to obtain a second state vector of a cluster running state after frequency adjustment, calculate an immediate reward according to the second state vector, and store interaction data into an experience replay pool, and update the actor-critic network model when the amount of data in the experience replay pool reaches a threshold;

[0145] In the SPARK cluster-oriented adaptive frequency regulation system, a complete intelligent control closed loop is formed by the running process: first, a user submits a SPARK job through a YARN resource manager, a ResourceManager allocates a container for the job and starts an Executor to execute the job, and the job continuously generates node-level and job-level running data during the running process; then, a monitoring module starts to periodically collect node-level indicators such as CPU utilization, memory utilization, current CPU frequency and real-time energy consumption of each working node, and obtains job-level information such as job type, stage progress and remaining execution time through an API provided by a SPARKHistory Server and a YARN, and fuses and encapsulates these multi-source information into a first state vector and transmits the first state vector to a PPOAgent intelligent decision center; after the PPOAgent receives the state vector, an Actor network of the PPOAgent generates a probability distribution of DVFS adjustment actions of each node based on a current policy, filters illegal frequency options exceeding a hardware range by combining an action mask mechanism, and finally samples a specific action vector, i.e., a frequency adjustment scheme to be executed by each node; after the control module receives the action vector, the control module analyzes the action vector into a frequency regulation command such as CPUpower executable by an operating system, and sends the frequency regulation command to each working node, and the node adjusts the CPU frequency immediately after receiving the command, so that the SPARK job continues to run under the new frequency configuration; after a fixed time interval Δt, the monitoring module collects a second state vector of the cluster state, the system calculates an immediate reward according to the energy consumption change, the SLA violation state and the frequency switching amplitude, and feeds back a complete interaction experience tuple to the PPOAgent; finally, the PPOAgent stores the experience data into a replay buffer, and when the sample accumulation reaches a predetermined threshold, a policy updating process based on a PPO clipping target function is triggered, the network parameters are updated under the constraint of the clipping mechanism by calculating an advantage function and a policy ratio, so that the optimized policy can be applied to the subsequent control cycle, thereby realizing the continuous self-optimization of the system.

[0146] Further, as shown in Figure 4 Based on the above-mentioned adaptive frequency adjustment method and system for SPARK cluster, the application further provides a terminal, which comprises a processor 10, a memory 20 and a display 30. Figure 4 Only some components of the terminal are shown, but it should be understood that all the shown components are not required, and more or less components can be alternatively implemented.

[0147] The memory 20 can be an internal storage unit of the terminal in some embodiments, such as a hard disk or a memory of the terminal. The memory 20 can also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the terminal. Further, the memory 20 can include both the internal storage unit and the external storage device of the terminal. The memory 20 is used to store application software and various data installed on the terminal, such as program codes of the terminal, etc. The memory 20 can also be used to temporarily store data that has been output or will be output. In an embodiment, the memory 20 stores an adaptive frequency adjustment program 40 for SPARK cluster, which can be executed by the processor 10 to implement the adaptive frequency adjustment method for SPARK cluster in the application.

[0148] The processor 10 can be a Central Processing Unit (CPU), a microprocessor or other data processing chip in some embodiments, which is used to run program codes or process data stored in the memory 20, such as to execute the adaptive frequency adjustment method for SPARK cluster, etc.

[0149] The display 30 can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. in some embodiments. The display 30 is used to display information of the terminal and to display visualized user interfaces. The processor 10, the memory 20 and the display 30 of the terminal communicate with each other through a system bus.

[0150] In an embodiment, the processor 10 implements the steps of the adaptive frequency adjustment method for SPARK cluster as described above when executing the adaptive frequency adjustment program 40 for SPARK cluster in the memory 20.

[0151] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a SPARK cluster-oriented adaptive frequency adjustment program, and the SPARK cluster-oriented adaptive frequency adjustment program, when executed by a processor, implements the steps of the SPARK cluster-oriented adaptive frequency adjustment method.

[0152] In summary, the application provides a SPARK cluster-oriented adaptive frequency adjustment method, system, terminal and computer readable storage medium, which deeply integrates DRL and DVFS, can accurately regulate frequency according to real-time load, and can reduce cluster energy consumption by 32% at most and significantly improve average energy efficiency. The SLA hard constraint penalty mechanism is built in the reward function to ensure that the optimization process always takes the premise of not violating the job time limit, and the experiment shows that the task on-time completion rate is improved by 5-10%. The PPO algorithm is used for online learning, which can adapt to different types of SPARK jobs (CPU-intensive, IO-intensive, and mixed) and dynamically changing cluster load without relying on pre-labeled historical data or fixed heuristic rules. Through the stability term in the reward function and the clipping mechanism of the PPO algorithm itself, the smoothness of the policy output is ensured, which effectively avoids the dramatic fluctuation of the frequency, prolongs the hardware life and reduces the system overhead. The entire scheme is completely implemented based on software, and is monitored and controlled by calling standard operating system interfaces (such as CPU power) and cluster standard APIs (such as REST API), without modifying the hardware or kernel, and is easy to deploy and apply in existing big data platforms.

[0153] It should be noted that in this paper, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or terminal. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or terminal including the element.

[0154] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware (such as a processor, a controller, etc.) to complete, and the program can be stored in a computer readable computer readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The computer readable storage medium can be a memory, a disk, an optical disk, etc.

[0155] It should be understood that the application is not limited to the examples described above, which can be modified or transformed by a person of ordinary skill in the art in light of the above description, all these modifications and transformations being intended to belong to the scope of protection of the application as defined by the claims hereafter.

Claims

1. An adaptive frequency adjustment method for Spark clusters, characterized in that, Includes the following steps: Collect node-level status information of each worker node in the SPARK-on-YARN cluster and job-level status information of the currently running job, and construct a first state vector based on the node-level status information and the job-level status information; Based on the first state vector, an action vector is generated using a preset actor-critic network model based on a proximal policy optimization algorithm; The action vector is converted into a CPU frequency adjustment instruction, and the CPU frequency adjustment instruction is sent to the target working node to control the target working node to adjust the CPU frequency. The second state vector of the cluster operation status after frequency adjustment is obtained, the instant reward is calculated based on the second state vector, and the interaction data is stored in the experience replay pool. When the amount of data in the experience replay pool reaches the threshold, the actor-critic network model is updated. The interaction data includes the first state vector, the action vector, the instant reward, and the second state vector. The step of generating action vectors based on the first state vector using a preset actor-critic network model based on a proximal policy optimization algorithm includes: The first state vector is input into the actor network in the actor-critic network model to obtain the probability distribution of DVFS actions of each node; The probability distribution is filtered for legality based on the action mask to remove illegal frequency options; The filtered probability distribution is sampled to obtain the action vector; The action vector is defined as follows: ； in, Indicates a point in time The action vector, 0 indicates frequency reduction, and 0 indicates maintaining the frequency. Indicates upsampling. The preset frequency adjustment step size ranges from 0.1 GHz to 2.7 GHz.

2. The adaptive frequency adjustment method for Spark clusters according to claim 1, characterized in that, The pre-training process of the actor-critic network model is as follows: Using historical workload logs or training data generated by simulators, the actor-critic network architecture is initially trained to obtain an actor-critic network model with an initial strategy for node CPU frequency regulation.

3. The adaptive frequency adjustment method for Spark clusters according to claim 1, characterized in that, The process of obtaining the second state vector of the cluster's operating state after frequency adjustment and calculating the instantaneous reward based on the second state vector includes: After a fixed time interval, obtain the second state vector of the cluster's operating status after frequency adjustment; The immediate reward for this action is calculated based on the second state vector and the multi-objective reward function.

4. The adaptive frequency adjustment method for Spark clusters according to claim 3, characterized in that, The multi-objective reward function includes an energy consumption term, a default penalty term, and a frequency stability term.

5. The adaptive frequency adjustment method for Spark clusters according to claim 1, characterized in that, The step of updating the actor-critic network model when the amount of data in the experience replay pool reaches a threshold includes: When the accumulated data samples in the experience replay pool reach a predetermined threshold, the policy update process is triggered. The advantage function and policy ratio are calculated, and the parameters of the actor-critic network model are iteratively updated based on the advantage function and policy ratio under the constraint of the pruning mechanism.

6. The adaptive frequency adjustment method for Spark clusters according to claim 1, characterized in that, The node-level status information includes CPU utilization, memory utilization, and CPU frequency, while the job-level status information includes job type, stage progress, and predicted remaining execution time.

7. An adaptive frequency adjustment system for Spark clusters, characterized in that, The adaptive frequency adjustment system for Spark clusters includes: The monitoring layer is used to collect node-level status information of each worker node in the SPARK-on-YARN cluster and job-level status information of the currently running job, and to construct a first state vector based on the node-level status information and the job-level status information. The decision layer is used to generate action vectors based on the first state vector using a preset actor-critic network model based on a proximal policy optimization algorithm. The execution layer is used to convert the action vector into CPU frequency adjustment instructions and send the CPU frequency adjustment instructions to the target worker node to control the target worker node to adjust the CPU frequency. The learning layer is used to obtain the second state vector of the cluster running state after frequency adjustment, calculate the instant reward based on the second state vector, and store the interaction data in the experience replay pool. When the amount of data in the experience replay pool reaches the threshold, the actor-critic network model is updated. The interaction data includes the first state vector, the action vector, the instant reward, and the second state vector.

8. A terminal, characterized in that, The terminal includes: a memory, a processor, and an adaptive frequency adjustment program for a Spark cluster stored in the memory and executable on the processor. When the adaptive frequency adjustment program for a Spark cluster is executed by the processor, it implements the steps of the adaptive frequency adjustment method for a Spark cluster as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an adaptive frequency adjustment program for a Spark cluster, which, when executed by a processor, implements the steps of the adaptive frequency adjustment method for a Spark cluster as described in any one of claims 1-6.

Images