A neural network optimization method for server cluster load prediction

Abstract

The application discloses a kind of neural network optimization methods for server cluster load prediction, it is related to load prediction and server cluster management technical field, this method includes: real-time acquisition hardware performance counter data of multiple computing units in server cluster and pre-processing;Hardware performance bottleneck is judged based on the data, when existing bottleneck, dynamically trigger the structure of load prediction neural network model Sparse processing;Subgraph splitting is carried out to the model after sparsification, and multiple logical subgraphs with different computing characteristics are obtained;According to the real-time hardware state of cluster and the characteristics of subgraph, the subgraph is scheduled to the corresponding heterogeneous computing unit and executes in parallel;Online calibration mechanism is added to maintain model prediction accuracy, the method is realized by four core modules, using distributed architecture and event-driven communication mechanism, the application realizes the dynamic adaptation of load prediction model and bottom hardware state, improves energy efficiency and resource utilization, and guarantees prediction accuracy.

Description

Technical Field

[0001] This invention relates to the field of load prediction and server cluster management technology, specifically a neural network optimization method for server cluster load prediction. Background Technology

[0002] With the rapid development of cloud computing and big data technologies, server clusters, as the core infrastructure supporting various internet services and enterprise applications, are facing increasingly prominent issues regarding resource utilization efficiency and energy consumption. Statistics show that the server utilization rate of most cloud computing data centers has long been at a low level. Load forecasting technology, by analyzing resource usage data and predicting future demand, is of great significance for optimizing resource scheduling and improving energy efficiency. Currently, deep learning-based load forecasting models are gradually replacing traditional statistical methods, better handling the uncertainty and nonlinear characteristics of cloud load data. However, existing methods often treat the hardware environment as a static black box, focusing only on improving model accuracy while neglecting the dynamic interaction between the neural network runtime and the underlying hardware state. This makes it difficult for the model to fully adapt to heterogeneous computing resources in actual deployment, limiting further improvements in overall efficiency.

[0003] While existing patents attempt to optimize load prediction and resource allocation from different perspectives, none have solved the technical problem of algorithm-hardware collaborative adaptation. For example, a load migration method, apparatus, and electronic device (patent publication number CN120803708A) predicts node load using historical time-series data and migrates data shards based on the prediction results to achieve load balancing. This method focuses on resource scheduling strategies but does not address the structural optimization of the load prediction model itself. Furthermore, its prediction module is independent of the hardware operating environment and cannot dynamically adjust the model's computational granularity based on real-time hardware bottlenecks. Another patent (patent publication number CN118860667B) presents a machine learning-based adaptive server load balancing method. Although it utilizes a machine learning model to predict load and guide resource allocation, its technical means are still limited to software-level data feature extraction and hash mapping scheduling. This method lacks awareness of hardware performance counters such as CPU cache hit rate and memory bandwidth, and cannot adaptively trigger model lightweighting when hardware resources are scarce, leading to increased prediction latency and decreased energy efficiency under high load scenarios.

[0004] To address the aforementioned shortcomings, the technical problem this invention aims to solve is the poor dynamic adaptation between the computational efficiency and hardware resources of existing load prediction models. Specifically, when static neural network models run in heterogeneous server clusters, they cannot adjust the computational structure and resource allocation strategies in real time according to hardware bottlenecks, resulting in low hardware execution efficiency, unsatisfactory energy efficiency, and difficulty in achieving all-time resource demand balance while ensuring prediction accuracy.

[0005] In summary, existing load forecasting technologies are mostly limited to software algorithm optimization or independent hardware metric monitoring, failing to form a closed-loop optimization system that links algorithms with hardware status. Therefore, there is an urgent need for an integrated method that can deeply integrate real-time hardware status awareness, dynamic sparsity of neural networks, and heterogeneous resource scheduling to overcome the bottleneck between model computation efficiency and hardware resource adaptation, and provide a technical foundation for achieving efficient load forecasting and resource management of server clusters. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a neural network optimization method for server cluster load prediction. By collecting server cluster hardware performance data in real time, dynamically judging hardware bottlenecks and triggering the sparsification of the load prediction neural network structure, combined with subgraph splitting and heterogeneous computing unit scheduling, and supplemented by online calibration, the model and hardware can be dynamically adapted, thereby improving hardware execution efficiency and cluster resource utilization.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a neural network optimization method for server cluster load prediction, comprising the following steps: Step 1: Collect hardware performance counter data from multiple computing units in the server cluster in real time; Step 2: Determine whether there is a hardware performance bottleneck based on the hardware performance counter data; Step 3: When the aforementioned hardware performance bottleneck exists, dynamically trigger the structural sparsity processing of the load prediction neural network model; Step 4: Perform subgraph splitting on the load prediction neural network model after the structure has been sparsified to obtain multiple subgraphs; Step 5: Based on the real-time hardware status of the server cluster, schedule the multiple subgraphs to the corresponding heterogeneous computing units for execution; The hardware performance counter data reflects the internal operating status of the computing unit. The structural sparsity processing reduces the computational complexity and resource consumption of the neural network model by reducing the number of neuron connections. The subgraph splitting divides the model into multiple independent subgraphs based on the computation graph structure and operator dependencies of the neural network model. The scheduling process allocates the subgraphs to different types of computing units based on the real-time hardware status to achieve parallel execution.

[0008] Furthermore, step one specifically includes the following steps: Continuously monitor hardware performance events of the computing unit through the operating system-level interface; The collected hardware performance counter data includes multiple indicators such as cache hit rate, memory bandwidth utilization, branch prediction error rate, and instruction cycle value. The collected hardware performance counter data is preprocessed, including data cleaning and aggregation. The data cleaning is used to remove outliers and noisy data; The aggregation process is used to calculate a short-term moving average to smooth out instantaneous fluctuations; The preprocessed hardware performance counter data is compared with a preset security threshold. The preset security threshold is dynamically adjusted based on historical hardware performance data; The frequency of data collection for the hardware performance counter is dynamically adjusted according to the server cluster load status. The data collection process covers all active computing units in the server cluster, including the central processing unit, graphics processing unit, and neural network processor.

[0009] Furthermore, step two specifically includes the following steps: Establish a rule base for judging hardware performance bottlenecks; The rule base contains mapping relationships between various hardware performance counter metrics and performance bottlenecks; The mapping relationship is established based on a machine learning model or predefined rules; The judgment process includes calculating the short-term trend of the hardware performance counter metrics; When the hardware performance counter index exceeds the preset safety threshold multiple times consecutively, it is determined that there is a hardware performance bottleneck. The types of hardware performance bottlenecks include cache resource bottlenecks, memory bandwidth bottlenecks, and computing resource bottlenecks. Generate a corresponding model optimization trigger signal for each type of hardware performance bottleneck; The model optimization trigger signal includes the target computing unit identifier and bottleneck severity information; The model optimization trigger signal is used to drive the subsequent structural sparsification process.

[0010] Furthermore, step three specifically includes the following steps: Upon receiving the model optimization trigger signal, the structure sparsification process is initiated. The structural sparsification process includes analyzing the importance of parameters in the neural network model; The importance of the parameter is calculated using a hybrid importance score; The hybrid importance score combines the parameter norm and the parameter's contribution to the hardware bottleneck layer; The structure sparsification process employs a structured pruning method. The structured pruning removes neurons or filters that contribute less to the neural network model; The degree of pruning is dynamically adjusted based on the severity of the hardware performance bottleneck. During the structural sparsification process, neuron connections in the neural network model whose mixed importance score is higher than a preset threshold are preserved. The neuronal connections are identified through gradient information or activation intensity; The structure is sparsified to generate an optimized neural network model version.

[0011] Furthermore, step four specifically includes the following steps: The computational graph structure of the neural network model after the structure sparsification process is analyzed; The computation graph structure includes operator types and dependencies between operators; The computation graph structure is divided into multiple logical subgraphs according to the operator type; Logical subgraph grouping is based on operator computational characteristics and resource requirements; The computational characteristics include computationally intensive, memory access intensive, and control intensive. The subgraph splitting process ensures that the operators within each subgraph have high cohesion; Dependencies between subgraphs are represented by data flow edges; Generate metadata for each subgraph describing its computational characteristics and resource requirements; The metadata is used for subsequent scheduling decisions.

[0012] Furthermore, step five specifically includes the following steps: Maintain dynamic resource profiles for the server cluster; The dynamic resource profile records the real-time load status and performance indicators of each computing unit; The real-time load status is obtained through the hardware performance counter data and system load data; The scheduling decision is based on the matching of the metadata in the subgraph and the dynamic resource profile; The matching process is based on the computational characteristics and computational unit expertise described in the subgraph; The computing unit's expertise was evaluated using historical execution efficiency data; The scheduling process prioritizes allocating the computationally intensive subgraph to the graphics processor. The control-intensive subgraph is preferentially allocated to the central processing unit; The memory access-intensive subgraph is assigned to a high-bandwidth memory computing unit; The execution effect of the subgraph is monitored after scheduling and verified using the hardware performance counter data.

[0013] Furthermore, the method also includes step six, an online calibration mechanism: The online calibration mechanism is activated after the structure sparsification process or subgraph scheduling. The online calibration mechanism uses the latest load data to fine-tune the optimized neural network model; The fine-tuning process employs a mini-batch gradient descent algorithm. The mini-batch gradient descent algorithm uses recent load data within a sliding window; The fine-tuning objective is to maintain the prediction accuracy of the neural network model; The fine-tuning process only updates some parameters of the neural network model; The selection of some parameters is based on parameter sensitivity analysis; The parameter sensitivity is calculated using gradient magnitude or error contribution. The execution frequency of the online calibration mechanism is adjusted according to the rate of change of model accuracy.

[0014] Furthermore, the method is implemented collaboratively by a hardware performance monitoring module, a dynamic optimization decision-making module, a model reconstruction module, and a runtime scheduling module; The hardware performance monitoring module is responsible for collecting and processing the hardware performance counter data; The dynamic optimization decision module generates the model optimization trigger signal based on the hardware performance counter data; The model reconstruction module performs the structural sparsification process and subgraph splitting; The runtime scheduling module handles subgraph scheduling and resource allocation; The modules communicate with each other through an event-driven mechanism; The event-driven mechanism ensures data consistency and processing sequence between modules; Each module is implemented using a distributed architecture to support server cluster environments; The distributed architecture achieves decoupling between modules through message middleware.

[0015] Furthermore, in this invention, a "subgraph" refers to a logically independent functional unit obtained by splitting a neural network computation graph; it can also be called a "logical subgraph." The splitting is based on, but is not limited to, operator type, computational characteristics, and data dependencies, unless otherwise stated.

[0016] Compared with existing technologies, this neural network optimization method for server cluster load prediction has the following advantages: I. This invention collects hardware performance counter data from each computing unit in a server cluster in real time, establishes a hardware performance bottleneck judgment rule base based on this data, and accurately identifies the bottleneck type and severity. When a bottleneck exists, it dynamically triggers the structural sparsification of the load prediction neural network model, and then performs subgraph splitting based on the model's computation graph structure and operator characteristics. Combined with the real-time hardware status of the cluster, the subgraph is scheduled to be executed by the appropriate heterogeneous computing unit. This achieves dynamic linkage between the load prediction model and the underlying hardware status, enabling the model to adaptively adjust the computation structure and resource allocation strategy according to the hardware bottleneck, significantly improving hardware execution efficiency and energy efficiency, and ensuring the model runs stably and efficiently in a heterogeneous server cluster.

[0017] Second, by setting up an online calibration mechanism, this invention uses recent load data to fine-tune the model after sparsification of the model structure or subgraph scheduling, effectively offsetting the accuracy loss that may occur during model optimization and ensuring the long-term stability of load prediction accuracy. At the same time, through the distributed deployment of the four core modules and the event-driven communication mechanism, efficient collaboration of data acquisition, bottleneck identification, model optimization and resource scheduling is achieved, ensuring data consistency and processing sequence among modules.

[0018] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0020] Fig. 1 This is a hardware perception and model dynamic optimization architecture diagram of the present invention; Fig. 2 This is a schematic diagram illustrating the heterogeneous resource collaborative scheduling principle of the present invention; Fig. 3 This is a flowchart of the steps of the present invention. Detailed Implementation

[0021] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0022] Example 1 like Figs. 1-3 As shown, this embodiment discloses a basic implementation scheme for a neural network optimization method for server cluster load prediction. It is applicable to heterogeneous server clusters containing a central processing unit (CPU), a graphics processing unit (GPU), and a neural network processor (NPU). This cluster can be deployed in scenarios such as cloud computing data centers and enterprise private clouds to support load prediction and resource scheduling for various applications such as internet services and big data analytics. This embodiment achieves dynamic adaptation between the load prediction model and the hardware environment through a closed-loop mechanism of hardware performance awareness, dynamic model optimization, and heterogeneous resource collaborative scheduling. The specific implementation process of each step is described in detail below.

[0023] In this embodiment, the hardware performance monitoring module undertakes the data acquisition and preprocessing tasks. It adopts a distributed deployment approach, deploying a acquisition agent on each active computing unit in the cluster, and continuously monitoring hardware performance events through an operating system-level interface.

[0024] Data Acquisition Interface and Coverage: In a Linux system environment, the acquisition agent accesses hardware performance counters through the perf tool interface and PCIe device driver interface; in a Windows system environment, data acquisition is achieved through the PerformanceMonitorAPI and dedicated interfaces provided by hardware manufacturers. The acquisition process covers all active computing units within the cluster, including x86 architecture CPUs, NVIDIA series GPUs, and Cambricon series NPUs, ensuring that no cluster hardware status is missed during monitoring.

[0025] Data Collection Metrics Definition: The collected hardware performance counter data includes four core metrics. The physical meaning and collection method of each metric are as follows: Cache hit rate: Reflects the data access hit rate of each level of CPU cache (L1, L2, L3), and is calculated by the ratio of cache read / write event counter to total access count counter; Memory bandwidth utilization: Characterizes the level of data transmission activity on the memory bus, and is determined by the ratio of the memory read / write throughput counter to the theoretical maximum memory bandwidth; Branch prediction error rate: Records the proportion of CPU branch instruction prediction failures, calculated from the branch prediction error event counter and the total branch instruction counter; Instruction Cycle Value (CPI): This reflects the efficiency of instruction execution and is obtained by comparing the CPU clock cycle counter with the number of instructions executed.

[0026] Dynamic sampling frequency adjustment: The sampling frequency is adaptively adjusted based on the current load status of the server cluster, employing a load intensity feedback mechanism: when the overall CPU utilization of the cluster is below 30% (low load), the sampling frequency is set to 5 seconds / sampling; when the CPU utilization is between 30% and 70% (medium load), the sampling frequency is adjusted to 2 seconds / sampling; and when the CPU utilization is above 70% (high load), the sampling frequency is increased to 1 second / sampling. This adjustment mechanism ensures real-time data transmission under high load while avoiding unnecessary resource consumption under low load.

[0027] The data preprocessing procedure in this embodiment is implemented as follows: Data cleaning: Outliers are removed using the 3σ principle for the collected data sequences of each indicator. Calculate the mean and standard deviation , will satisfy The value is identified as an outlier and replaced with the valid data from the previous moment of the indicator; at the same time, the noisy data is filtered by median filtering, with the window size set to 5, that is, the median of 5 consecutive collection points is taken as the cleaned data at the current moment.

[0028] Aggregation processing: Short-term moving average method is used to smooth instantaneous fluctuations. The calculation formula is as follows:

[0029] in, for Aggregated data at different times For the first Data after constant cleaning This is the size of the sliding window, with a value of 10 (corresponding to a 20-second time window under medium load conditions, balancing response speed and smoothness).

[0030] Specifically, in the formula Represents the current collection cycle number. Represents the historical data collection period number, with values ​​ranging from... to , corresponding to continuous Cleaned data from each collection cycle ; In this embodiment, the sliding window size is... The value needs to be dynamically matched with the data acquisition frequency and load status—when the cluster is under low load (CPU utilization is below 30%, acquisition frequency is 5 seconds / time), Set to 8 (corresponding to a 40-second time window), under medium load conditions (CPU utilization 30%-70%, sampling frequency 2 seconds / time), Set to 10 (corresponding to a 20-second time window), under high load conditions (CPU utilization higher than 70%, sampling frequency 1 second / time), Setting it to 15 (corresponding to a 15-second time window) ensures smooth data flow while avoiding trend response delays caused by excessively long time windows under high load scenarios. It should be noted that... The aggregated data results have the same physical meaning as the original collected indicators (such as cache hit rate, memory bandwidth utilization, etc.), and only the interference of instantaneous noise on subsequent bottleneck judgment is reduced by averaging.

[0031] Preset security threshold dynamically adjusted: The preset security threshold is dynamically updated based on historical hardware performance data using an exponential moving average method. The calculation formula is as follows:

[0032] in, for Preset safety threshold at any time The threshold value at the previous time step. This is the aggregated data from the previous moment. This is a smoothing coefficient, set to 0.8 (balancing the stability of historical data with the sensitivity of real-time data). Initial threshold. Determined based on the statistical average of historical data from 24 hours of unloaded cluster operation.

[0033] Specifically, The value is 0.8, which was determined through multiple sets of comparative experiments: when At that time, the threshold is greatly affected by real-time data fluctuations (fluctuation range of about 15%), which can easily lead to misjudgment of bottlenecks; when At that time, the threshold update was significantly delayed (response delay of about 2 seconds), failing to adapt to changes in hardware status in a timely manner; while At this time, the threshold fluctuation range is controlled within 8%, and the response delay is ≤1 second, which can balance the stability of historical data and the sensitivity of real-time data. Initial threshold The determination of the baseline must meet the "idle load" requirement: First, the server cluster is placed in an idle state (defined as a stable state with CPU utilization ≤5%, memory bandwidth utilization ≤10%, and no user tasks running). Hardware performance data is continuously collected for 24 hours, during which more than 3 consecutive outliers are removed (judged according to the 3σ principle). The arithmetic mean of the remaining valid data is then taken, and the result is the baseline. This ensures that the initial threshold reflects the baseline level for normal hardware operation.

[0034] In this embodiment, the specific implementation of hardware performance bottleneck determination is as follows: The dynamic optimization decision module uses preprocessed hardware performance counter data and a preset rule base to accurately identify performance bottlenecks and generate trigger signals.

[0035] Bottleneck identification rule base construction: The rule base contains the mapping relationship between hardware performance counter metrics and performance bottlenecks. This mapping relationship is generated through training a Gradient Boosting Tree (GBDT) model. The training data consists of the cluster's hardware performance data from the past 6 months and manually labeled bottleneck event data. The rule base specifically includes three types of bottleneck identification logic: Cache resource bottleneck: triggered when the cache hit rate falls below the preset safety threshold multiple times consecutively, and the short-term trend shows a downward trend; Memory bandwidth bottleneck: Triggered when memory bandwidth utilization exceeds the preset safety threshold multiple times consecutively, and the short-term trend shows an upward trend; Computational resource bottleneck: Triggered when the instruction cycle value exceeds the preset safety threshold multiple times consecutively, and the branch prediction error rate exceeds the preset safety threshold (the branch prediction error rate threshold is generated separately during training and is fixed at 15%).

[0036] Short-term trend calculation: A linear regression model is used to calculate the short-term trend of the hardware performance counter index, taking the 10 most recent aggregated data points. ,in Fitting the linear equation The slope This is the short-term trend coefficient:

[0037] in the formula The trend slope Represents the periodicity of the most recent 10 aggregated data points ( Corresponding to the earliest data point, (For the latest data point, the period number increases sequentially according to the collection time.) This is the aggregated data for the corresponding period. For ease of understanding, a specific calculation example is given here: Assume the collected 10 aggregated memory bandwidth utilization data (unit: %) are 85, 88, 90, 87, 82, 78, 75, 72, 70, and 68, corresponding to the period numbers... If the value is between 1 and 10, then the first step is to calculate... , , , The second step is to substitute the values ​​into the formula to calculate the numerator: denominator: The third step yields the slope. (Due to differences in the example data, adjustments need to be made based on the magnitude of the indicators in actual calculations. This applies to conventional hardware indicators.) The absolute value is usually small, such as cache hit rate. Generally within ±0.05). The trend judgment criteria are: when (k < -0.02), it is judged as a downward trend; when (k > 0.02), it is judged as an upward trend; otherwise, it is a stable trend. This threshold is verified by historical bottleneck event data and can effectively distinguish between normal fluctuations and abnormal trends.

[0038] In this embodiment, the bottleneck determination condition and trigger signal generation are implemented as follows: The core criteria for bottleneck identification are: the indicator exceeds the preset safety threshold three times consecutively and meets the short-term trend requirements of the corresponding bottleneck type. When the criteria are met, a model optimization trigger signal is generated. The signal uses a structured data format and includes three core fields: target computing unit identifier (such as CPU UUID, GPU device number), bottleneck type (cache / memory bandwidth / computing resources), and bottleneck severity (mild / moderate / severe). The severity is determined based on the proportion of the indicator exceeding the threshold: 10%-20% is mild, 20%-40% is moderate, and above 40% is severe.

[0039] In this embodiment, the specific implementation of the structural sparsity processing of the load prediction neural network model is as follows: After receiving the optimization trigger signal, the model reconstruction module initiates structural sparsity processing, which reduces the computational complexity of the model through structured pruning while ensuring that the prediction accuracy remains basically stable.

[0040] Basic structure of the load prediction neural network model: The load prediction neural network model used in this embodiment is an improved LSTM network with an input layer dimension of 128 (corresponding to 128-dimensional preprocessed load-related features), containing 3 hidden layers (each with 256, 128, and 64 neurons respectively), and an output layer dimension of 1 (predicting the average CPU utilization of the cluster in the next 5 minutes). The model parameters include the weight matrices and bias vectors of each layer, with a total of approximately 800,000 parameters.

[0041] Hybrid Importance Score Calculation: Parameter importance is quantified using a hybrid importance score, which combines the parameter norm with the parameter's contribution to the hardware bottleneck layer. The calculation formula is as follows:

[0042] in, For parameters The mixed importance score ranges from [0,1]. This is the balance coefficient, with a value of 0.5 (representing both the importance of the balance parameter itself and its suitability for the bottleneck). For parameters The L2 norm reflects the overall importance of the parameters. loss function For parameters The gradient represents the degree of influence of parameters on the model prediction error. In this embodiment... The mean squared error loss function is used, and its expression is: ,in For the sample size, For the first The actual load values ​​of each sample (such as CPU utilization and memory usage). For the model to the first Predicted loading values ​​for each sample, gradient The calculation is implemented through the backpropagation algorithm, specifically as follows: . This is the maximum absolute value of the gradient of all parameters, used to normalize the gradient contribution to the [0,1] interval, avoiding scoring bias due to differences in parameter magnitudes. For example, if the absolute value of the gradient of a parameter is 0.8 and the maximum value is 1.0, then the gradient contribution of that parameter is 0.8. If the maximum value is 0.5, then the gradient contribution is 1.0 (i.e., truncated to 1.0), ensuring a consistent scoring range.

[0043] Structured pruning implementation process: A channel-level structured pruning method is used, processing layer by layer according to the hidden layer: Calculate the mixed importance score of all parameters in the current layer. ; The pruning ratio is determined based on the severity of the bottleneck: 10% for mild bottlenecks, 20% for moderate bottlenecks, and 30% for severe bottlenecks. Sort the parameters in ascending order of their scores, and remove the channel (neuron or filter) containing the corresponding proportional parameter with the lowest score. The pruned model is reconnected to ensure the integrity of the network structure.

[0044] Preset threshold determination: During the structured pruning process, neural connections with mixed importance scores higher than a preset threshold are retained. This threshold is determined by validation set accuracy calibration: with the constraint that the prediction accuracy loss of the model on the validation set after pruning does not exceed 3%, the optimal threshold is obtained by searching using a binary search method. In this embodiment, the preset threshold range is 0.2-0.4 (dynamically adjusted with the pruning ratio; the higher the pruning ratio, the lower the threshold).

[0045] Model optimization after pruning: After pruning, the model is quickly fine-tuned using mini-batch gradient descent with a batch size of 32, a learning rate of 0.001, and 10 iterations. Only 10% of the retained parameters are updated (based on parameter sensitivity selection) to ensure that the model accuracy is quickly restored to more than 97% of that before pruning.

[0046] In this embodiment, the specific implementation of the subgraph splitting process is as follows: After structural sparsification, the model reconstruction module splits the optimized model into subgraphs, generating multiple logical subgraphs that can be executed in parallel.

[0047] Computational graph structure analysis: The computational graph structure of the optimized model is analyzed using TensorFlow's GraphDef tool to identify the types of operators (including convolution operators, fully connected operators, activation operators, pooling operators, etc.) and the dependencies between operators (represented by data flow edges). For example, fully connected operators depend on the output data of the previous layer, and activation operators depend on the output data of fully connected operators.

[0048] Operator classification and subgraph partitioning: Based on the computational characteristics of operators, all operators are divided into three categories: Computationally intensive operators: fully connected operators and convolution operators, whose computational cost is positively correlated with the number of parameters, and floating-point operations account for more than 80% of the total computation. Memory-intensive operators: pooling operators and data reshaping operators, accounting for over 70% of data read and write operations; Control-intensive operators: activation operators and dropout operators, which contain many conditional judgments and logical control operations.

[0049] Based on the principles of operator classification and high cohesion with low coupling, the Louvain community detection algorithm is used for subgraph partitioning: Each operator is treated as an independent node, and the dependencies between operators are treated as edge weights. Calculate modularity The value measures the quality of subgraph partitioning. The formula for calculating the value is:

[0050] in the formula Modularity (value range [-1, 1]) The closer to 1, the better the partition quality. For operators with operators The edge weights between operators are determined based on the strength of the data dependency between them: if the operator Must rely on operators Only the output of [the specified condition] can be executed (strong dependency). If the operator Only partially dependent operators The output (weak dependency), then If there is no direct dependency, then For example, the convolution operator and subsequent activation operators are strongly dependent. Two fully connected operators with no data interaction . For operators The degree (i.e., with the operator) The sum of the weights of all connected edges. ), The sum of the weights of all edges ( ), Operators , Subgraph identifier For indicator functions (if) The value is 1 if the subgraph partitioning is active, and 0 otherwise. The stopping criterion for subgraph partitioning iteration is: when the subgraph partitioning iteration stops after three consecutive iterations... The partitioning stops when the value increase is ≤0.001, ensuring that the subgraph cohesion requirement is met—for example, in the first iteration. The second time The third time The fourth time Since the increase in the third and fourth iterations is 0, the stopping condition is met, and the division is stopped.

[0051] Iterative optimization of modularity Value, when The partitioning stops when the value no longer increases, and 3-5 logical subgraphs are generated in the end (in this embodiment, 4 subgraphs are generated, namely 2 computationally intensive, 1 memory access intensive, and 1 control intensive).

[0052] Subgraph metadata generation: Generate metadata for each subgraph, including the following core information: subgraph type (computation / memory access / control intensive), computational cost (number of floating-point operations, FLOPs), memory requirement (maximum memory usage, MB), dependent subgraph ID (if a dependency exists), and execution priority (levels 1-5, with computationally intensive subgraphs having the highest priority of level 5).

[0053] In this embodiment, the specific implementation of subgraph heterogeneous scheduling is as follows: The runtime scheduling module uses cluster dynamic resource profiles and subgraph metadata to achieve optimal allocation and execution monitoring of subgraphs.

[0054] Dynamic resource profile maintenance: The dynamic resource profile records the real-time status of each computing unit and adopts a timed update mechanism (the update cycle is consistent with the data collection frequency), including the following indicators: Real-time load status: CPU utilization, GPU memory utilization, NPU computing unit utilization; Performance metrics: CPU clock frequency (GHz), GPU floating-point performance (TFLOPs), memory bandwidth (GB / s); Specialty Ability Score: Calculated based on historical execution data, with a score range of [0,1]. Computation-intensive task score = historical GPU speedup / average cluster speedup; control-intensive task score = historical CPU execution efficiency / average cluster execution efficiency; memory access-intensive task score = high-bandwidth memory computing unit data transfer rate / average cluster transfer rate.

[0055] Scheduling decision algorithm: An improved Hungarian algorithm is used to achieve optimal matching between subgraphs and computational units. The objective function is to minimize the total execution delay, and the constraint is that the resource requirements of the subgraph do not exceed the remaining resources of the computational units. The scheduling decision process is as follows: Constructing a cost matrix ,in For subgraph The estimated execution latency allocated to computing unit j is obtained by fitting the subgraph computation amount, computing unit performance indicators and historical execution data; The Hungarian algorithm is used to find the minimum weight matching of the cost matrix to obtain the initial scheduling scheme; Initial scheme adjusted based on computational unit expertise score: if subgraph For computationally intensive tasks, if the GPU's specialty capability score is 0.3 or higher than the CPU's, the task will be preferentially assigned to the GPU. Similarly, control-intensive subgraphs will be preferentially assigned to the CPU, and memory access-intensive subgraphs will be preferentially assigned to high-bandwidth memory computing units (such as GPUs with HBM).

[0056] Scheduling Execution and Monitoring: After the scheduling plan is executed, the execution status of the subgraph is monitored in real time. Collect subgraph execution latency (time from start to finish) and resource utilization (average load of computing units during execution). If the execution delay of a subgraph exceeds 20% of the estimated delay, it is marked as a scheduling anomaly and will be reallocated to other computing units in the next scheduling cycle. Subgraph execution efficiency is calculated every 5 scheduling cycles, and the computational unit's specialty capability score is updated.

[0057] In this embodiment, the online calibration mechanism is implemented as follows: Online calibration is initiated after structural sparsification or subgraph scheduling to ensure stable model prediction accuracy.

[0058] Calibration trigger conditions: Calibration is triggered when any of the following conditions are met: After the structure sparsification process is completed (this will be triggered after pruning); After three consecutive scheduling attempts, the model's prediction accuracy decreased by more than 3%. Significant changes have occurred in the cluster hardware status (such as adding / removing computing units).

[0059] Mini-batch gradient descent fine-tuning: The mini-batch gradient descent algorithm is used for model fine-tuning, and the parameter update formula is as follows:

[0060] in the formula For the updated model parameters, For the current parameter, For learning rate, For batch sample sets, For a single sample For parameters The loss gradient. Specifically, the batch sample set. The selection is based on a "sliding window" strategy: the sliding window contains the most recent 200 server cluster load data points (each data point contains 128-dimensional features, corresponding to hardware performance counter data and historical load data), and 64 data points are randomly selected from the window as batch samples during each fine-tuning (i.e., The learning rate is 64, a setting that ensures the timeliness and diversity of the samples and avoids overfitting. The value is 0.0005, and its selection is based on the results of multiple comparative experiments: when At that time, the model converges slowly (requiring 50 iterations to restore accuracy); when When the model is prone to oscillation (accuracy fluctuation ≥ 4%), At this point, the model can restore its accuracy to over 97% of its pre-pruning level within 20 iterations, with a fluctuation range of ≤1%, hence this value is chosen. It should be noted that in this embodiment... Using a fixed value, without dynamic adjustment with the number of iterations—because the model parameters are close to their optimal values ​​during the fine-tuning phase, a fixed small learning rate can avoid excessive parameter fluctuations; if the model pruning ratio exceeds 30% (heavy pruning), then... The value was adjusted to 0.0003 to further reduce the risk of oscillations. Furthermore, the gradient calculation in the formula must be consistent with the model training phase, i.e., solved using the backpropagation algorithm, ensuring that the gradient direction is consistent with the direction of loss descent.

[0061] Parameter sensitivity analysis and selection: Fine-tuning updates only some parameters, filtering update targets based on parameter sensitivity: Calculate parameter sensitivity (Gradient magnitude); Sort the parameters in descending order of sensitivity and select the top 20% for updating (focus on updating the core parameters that have a significant impact on model accuracy). The number of fine-tuning iterations is 20. During the iteration process, the accuracy of the validation set is monitored, and the process stops when the accuracy is improved to the level before pruning or no longer improves.

[0062] Calibration frequency adjustment: The online calibration execution frequency is dynamically adjusted based on the rate of change of model accuracy. Precision change rate ( For current accuracy, (Accuracy of the previous cycle); when At this time, the calibration frequency is once every 10 prediction cycles; when At this time, the calibration frequency is once every 5 prediction cycles; when At that time, the calibration frequency is once every 2 prediction cycles.

[0063] In this embodiment, the specific implementation of module collaboration is as follows: This method utilizes four core modules that work collaboratively, communicating with each other via an event-driven mechanism, and is deployed based on a distributed architecture. Hardware performance monitoring module: Each computing unit deploys a data acquisition agent, which sends preprocessed data to the dynamic optimization decision module through a message middleware (Kafka). The communication protocol is HTTP / 2 to ensure low latency data transmission. Dynamic optimization decision module: Deployed on the cluster control node, it receives monitoring data and generates trigger signals, which are sent to the model reconstruction module through the event bus. The trigger signals are serialized in JSON format. Model reconstruction module: Deployed on high-performance computing nodes, after completing sparsification and subgraph splitting, it sends the subgraph metadata to the runtime scheduling module and receives accuracy feedback from online calibration. Runtime scheduling module: Deployed on the cluster scheduling node, it performs scheduling based on resource profiles and metadata, and feeds back the execution status to other modules to form a closed loop.

[0064] Data consistency between modules is guaranteed through distributed locks (implemented in Redis), and processing sequence is synchronized through event timestamps to ensure that each step is executed in order without data conflicts or time sequence errors. This embodiment requires no hardware modification; it can adapt to existing heterogeneous server clusters through software-level algorithm optimization and module collaboration, exhibiting good compatibility and scalability.

[0065] Example 2 like Figs. 1-3As shown, this embodiment is an optimized implementation of a neural network optimization method for server cluster load prediction. Based on Embodiment 1, it optimizes hardware data acquisition compatibility, model sparsity accuracy preservation, subgraph scheduling dynamic adaptability, and online calibration efficiency. It is suitable for hybrid heterogeneous server clusters including x86 architecture CPUs, AMD series GPUs, Horizon series NPUs, and ARM architecture edge computing units, and can be deployed in edge cloud nodes, hybrid cloud data centers, and other scenarios with higher requirements for hardware adaptability and real-time prediction. This embodiment further improves the collaborative efficiency between the model and the hardware environment through multi-source data acquisition and fusion, hierarchical structured pruning, reinforcement learning scheduling decisions, and adaptive calibration mechanisms, while ensuring stable prediction accuracy. The specific implementation process of each step is described in detail below.

[0066] In this embodiment, the hardware performance monitoring module adopts a multi-source acquisition and fusion scheme, combining hardware manufacturer-specific tools with the operating system's native interface to achieve comprehensive monitoring across architecture computing units, while optimizing the data preprocessing process to improve the accuracy of bottleneck identification.

[0067] Specifically, in this embodiment, data acquisition covers all active computing units within the cluster, including x86 architecture CPUs, AMD series GPUs, Horizon series NPUs, and ARM architecture edge computing units. Different acquisition interfaces are used for different hardware types: for x86 architecture CPUs, data acquisition is conducted collaboratively through the Intel VTune tool interface and the Linux system's perf tool interface to ensure accurate acquisition of core metrics such as cache hit rate and branch prediction error rate; for AMD series GPUs, the rocprof interface in the ROCm tool stack is used to collect metrics such as memory bandwidth utilization and computing unit utilization; for Horizon series NPUs, the manufacturer-provided BPUSDK interface is used to obtain specific metrics such as instruction cycle value and data transmission latency; for ARM architecture edge computing units, hardware performance events are collected through the Linux system's sysfs interface and hwmon subsystem to adapt to the acquisition requirements of low-power edge scenarios.

[0068] Based on Example 1, the collected hardware performance counter data includes three additional metrics: GPU memory read / write latency, NPU computing unit utilization, and ARM architecture unit instruction execution throughput, further enriching the dimensions of hardware status description. Specifically, GPU memory read / write latency is calculated as the ratio of memory access event counters to data transfer volume, reflecting memory data interaction efficiency; NPU computing unit utilization is determined as the ratio of occupied computing cores to the total number of cores, characterizing the NPU resource busyness; and ARM architecture unit instruction execution throughput is obtained by counting the number of instructions completed per unit time, reflecting the execution efficiency of the edge computing unit.

[0069] The dynamic adjustment of data acquisition frequency adopts a load gradient feedback mechanism, which is more flexible than the fixed threshold adjustment in Implementation Example 1. Specifically, it calculates the gradient of changes in the overall cluster load. When the absolute value of the load gradient is less than 5% / minute, the current acquisition frequency is maintained; when the load gradient is greater than or equal to 5% / minute but less than 10% / minute, the acquisition frequency is increased by one level; when the load gradient is greater than or equal to 10% / minute, the acquisition frequency is increased by two levels. The initial acquisition frequency is set as follows: 4 seconds / time for low load (CPU utilization below 30%), 1.5 seconds / time for medium load (CPU utilization 30%-70%), and 0.8 seconds / time for high load (CPU utilization above 70%). This ensures the real-time performance of data in high dynamic load scenarios while avoiding resource consumption caused by frequent acquisition.

[0070] The data preprocessing workflow is optimized as follows: In the data cleaning stage, the Isolation Forest algorithm replaces the 3σ principle to detect outliers in the collected data. Multiple isolated trees are constructed to quantify the isolation level of data points. Data with isolation scores exceeding a preset threshold are identified as outliers and replaced with the median within the sliding window of this indicator. Compared to the 3σ principle, this approach is more suitable for non-normally distributed hardware performance data. Simultaneously, Gaussian filtering is used to filter noisy data. A Gaussian kernel function with a standard deviation of 0.8 is used to smooth the data, reducing the impact of instantaneous fluctuations on subsequent judgments. In the aggregation stage, a weighted moving average method is used. The 10 most recent data collection points are assigned different weights, with higher weights for points closer to the current time. The weight coefficients are distributed in an arithmetic progression (first term 0.2, last term 0.01). The aggregated data is obtained through weighted summation, preserving short-term trend characteristics while reducing the interference of random fluctuations.

[0071] The dynamic adjustment of the preset safety threshold adopts a two-factor update mechanism, combining historical data statistics and real-time load levels. Specifically, a load level coefficient is introduced into the threshold update formula: 0.9 for low load, 1.0 for medium load, and 1.1 for high load. This coefficient balances the threshold sensitivity under different load scenarios. The initial threshold is determined by fusing historical data from 48 hours of cluster idle operation with benchmark data from similar clusters, ensuring the rationality of the initial threshold setting.

[0072] In this embodiment, the dynamic optimization decision module introduces a load correlation analysis mechanism based on the rule base of Embodiment 1, constructs a multi-dimensional bottleneck judgment rule base, improves the accuracy of bottleneck identification, and optimizes the information dimension of the trigger signal to support subsequent fine-grained optimization.

[0073] It is understandable that relying on a single indicator in related technologies can easily lead to misjudgment of bottlenecks. This embodiment enhances the reliability of bottleneck judgment by establishing a correlation model between hardware performance indicators and cluster load. The mapping relationship of the bottleneck judgment rule base is generated through training a random forest model. The training data includes hardware performance data, load data, and manually labeled bottleneck event data of the cluster over the past 12 months. Five-fold cross-validation is used during model training to ensure generalization ability. The rule base includes judgment logic for four types of bottlenecks. In addition to the three bottleneck types in Example 1, a data transmission bottleneck is added: The criteria for determining a cache resource bottleneck are: the cache hit rate is repeatedly lower than a preset safety threshold, with a short-term downward trend, and the cluster data read / write request volume is higher than the average; the criteria for determining a memory bandwidth bottleneck are: the memory bandwidth utilization is repeatedly higher than a preset safety threshold, with a short-term upward trend, and the memory queue length exceeds the threshold; the criteria for determining a computing resource bottleneck are: the instruction cycle value is repeatedly higher than a preset safety threshold, the branch prediction error rate is higher than 15%, and the computing task queue backlog exceeds the threshold; and the criteria for determining a data transmission bottleneck are: the GPU memory read / write latency and the NPU data transmission latency are repeatedly higher than preset safety thresholds, with a short-term upward trend, and the data transmission volume accounts for more than 40% of the total cluster load.

[0074] Short-term trend calculation uses the exponentially weighted moving average method instead of the linear regression model. It takes the 15 most recent aggregated data points and determines the trend direction by calculating the slope of the exponentially weighted moving average. Specifically, it calculates the first-order and second-order exponentially weighted moving averages for the data series. The ratio of the difference between the two to the first-order exponentially weighted moving average is the trend coefficient. A trend coefficient less than -0.03 indicates a downward trend, greater than 0.03 indicates an upward trend, and otherwise, it indicates a stationary trend. Compared to the linear regression model, this method can more quickly capture short-term fluctuations in the data.

[0075] The bottleneck determination criteria have been optimized to require the indicator to exceed a preset safety threshold four times consecutively, while also meeting the short-term trend requirements and load correlation conditions for the corresponding bottleneck type. The generated model optimization trigger signal, based on Example 1, adds two fields: bottleneck duration and associated load type. The bottleneck duration is calculated by the number of consecutive collection cycles exceeding the threshold, while the associated load type is determined by analyzing the type of tasks currently running in the cluster (such as computing tasks, data transmission tasks, storage tasks, etc.). This provides a more accurate decision-making basis for subsequent structure sparsity processing and subgraph scheduling.

[0076] In this embodiment, the model reconstruction module adopts a hierarchical structured pruning scheme and combines an attention mechanism to optimize parameter importance assessment. This further reduces the computational complexity of the model while preserving the prediction accuracy to the greatest extent and adapting to the differences in hardware resources of hybrid heterogeneous clusters.

[0077] The load prediction neural network model used in this embodiment is an improved GRU network. The input layer has a dimension of 160 (containing 128 dimensions of load-related features and 32 dimensions of hardware state features), four hidden layers (each with 320, 192, 128, and 64 neurons respectively), and an output layer with a dimension of 2 (predicting the average cluster load for the next 5 and 10 minutes respectively). The total number of parameters is approximately 1.2 million. This model adds a hardware state adaptation layer to the traditional GRU network, dynamically adjusting the weights of hardware features through an attention mechanism to improve the model's sensitivity to changes in hardware state.

[0078] Parameter importance assessment employs a fusion of hybrid importance scoring and attention weighting. The hybrid importance score is calculated in the same manner as in Example 1, but with the addition of an attention weight coefficient. This coefficient is obtained from the output of the model's hardware state adaptation layer and reflects the parameter's sensitivity to changes in hardware state. The final parameter importance score is the product of the hybrid importance score and the attention weight coefficient, ranging from 0 to 1, considering both the parameter's contribution to the model's prediction accuracy and its sensitivity to hardware state adaptation.

[0079] Structured pruning employs a layered pruning strategy, differentiating pruning ratios based on the functional characteristics of hidden layers: hidden layers closer to the input layer (layers 1 and 2) are pruned less than those closer to the output layer (layers 3 and 4), because parameters near the input layer are more responsible for feature extraction, and excessive pruning can easily lead to feature loss. Specifically, under mild bottlenecks, the pruning ratio for layers 1 and 2 is 8%, and for layers 3 and 4 it is 12%; under moderate bottlenecks, the pruning ratio for layers 1 and 2 is 15%, and for layers 3 and 4 it is 25%; under severe bottlenecks, the pruning ratio for layers 1 and 2 is 22%, and for layers 3 and 4 it is 35%. The pruning method uses inter-layer structured pruning, removing neurons and their connections whose importance scores are below a preset threshold, while preserving key pathways between layers to ensure the structural integrity of the pruned model. The preset thresholds are determined through a two-layer calibration: first, based on the constraint that the accuracy loss on the validation set does not exceed 2%, the initial thresholds for each layer are determined; then, the initial thresholds are fine-tuned by simulating the model's performance under different hardware bottleneck scenarios, ensuring that the thresholds balance computational complexity and prediction accuracy under different bottleneck types. Rapid fine-tuning of the pruned model employs momentum gradient descent with a batch size of 48, a learning rate of 0.0008, 15 iterations, and updates to 15% of the retained parameters (based on a combination of parameter importance scores and attention weights), ensuring that the model accuracy quickly recovers to over 98% of its pre-pruning level.

[0080] In this embodiment, the model reconstruction module adopts an adaptive subgraph splitting algorithm, which combines the priority of operator resource requirements to optimize the subgraph partitioning strategy, generate logical subgraphs that are more suitable for the capabilities of heterogeneous computing units, and improve the adaptation efficiency of subsequent scheduling.

[0081] The computation graph structure analysis employs the PyTorch torch.jit module in collaboration with the ONNX parsing tool. This not only identifies operator types and dependencies between operators but also extracts the resource requirements of each operator, including computational load, peak memory usage, data transfer volume, and instruction complexity, providing richer data for subgraph partitioning. Based on the three categories in Example 1, operator classification is further subdivided into four types: lightweight computationally intensive, heavily computationally intensive, low-latency memory-intensive, and high-throughput memory-intensive. Lightweight computationally intensive operators have a floating-point operation ratio of 60%-80%, heavily computationally intensive operators have a floating-point operation ratio exceeding 80%, low-latency memory-intensive operators require data read / write latency below 10 milliseconds, and high-throughput memory-intensive operators require a data transfer rate exceeding 10 GB / s.

[0082] Subgraph splitting employs an adaptive hierarchical clustering algorithm, replacing the Louvain community detection algorithm in Implementation Example 1. The specific process is as follows: First, each operator is treated as an independent node, and the similarity between nodes is calculated based on operator type, resource requirement characteristics, and dependencies. Then, hierarchical clustering is used to aggregate the nodes, dynamically adjusting the clustering threshold to maximize subgraph cohesion and minimize inter-subgraph dependencies. Finally, based on the number and type of cluster computing units, the clustering results are divided into 4-6 logical subgraphs, ensuring that the resource requirements of each subgraph match the capabilities of at least one computing unit. For example, computationally intensive operators are aggregated into independent subgraphs, while low-latency memory-access-intensive operators and control-intensive operators are aggregated into subgraphs based on dependencies.

[0083] The subgraph metadata adds resource requirement priority and latency sensitivity level fields to the existing implementation. Resource requirement priority is divided into three levels: high, medium, and low. Subgraphs with heavy computational intensity and low latency memory access intensity have high priority, subgraphs with light computational intensity and high throughput memory access intensity have medium priority, and control-intensive subgraphs have low priority. Latency sensitivity level is divided into three levels: sensitive, moderate, and insensitive. Subgraphs on the predicted critical path have a latency sensitivity level of sensitive, subgraphs on non-critical paths with short execution times have an insensitive level, and the rest have a moderate level, providing a basis for subsequent priority allocation in scheduling.

[0084] In this embodiment, the runtime scheduling module adopts a reinforcement learning scheduling decision algorithm, combined with a real-time update mechanism for dynamic resource profiles, to achieve accurate matching between subgraphs and computing units. At the same time, it optimizes the scheduling feedback adjustment mechanism to improve scheduling stability under complex load scenarios.

[0085] The maintenance of dynamic resource profiles employs a combination of real-time and batch updates. The real-time update mechanism targets computational units corresponding to high-priority subgraphs, updating their load status and performance metrics after each subgraph execution. The batch update mechanism targets other computational units, with the update cycle consistent with the data collection frequency. The dynamic resource profile adds two new metrics for computational units: load volatility coefficient and task adaptation success rate. The load volatility coefficient is calculated by determining the standard deviation of the load status over the past 10 periods, reflecting the load stability of the computational unit. The task adaptation success rate is determined through historical execution data statistics, specifically the percentage of times a certain type of subgraph achieves the expected execution efficiency on that computational unit, demonstrating the computational unit's adaptability to specific types of subgraphs.

[0086] The scheduling decision employs a reinforcement learning algorithm based on DQN (Deep Q-Network) instead of the improved Hungarian algorithm in Implementation Example 1. Specifically, the subgraph scheduling problem is modeled as a Markov decision process. The state space includes the resource requirements of the subgraph, latency sensitivity level, real-time load of the computing units, performance metrics, load fluctuation coefficient, and task adaptation success rate. The action space represents the matching combination of subgraphs and computing units. The reward function is designed as a weighted sum of the execution latency achievement rate, resource utilization achievement rate, and task adaptation success rate, with higher weight coefficients for subgraphs with higher latency sensitivity. The reinforcement learning model is optimized through a combination of offline training and online iteration. Offline training uses scheduling data from the cluster over the past six months, while online iteration updates the model parameters every 10 scheduling cycles to ensure the scheduling strategy can adapt to dynamic changes in cluster load.

[0087] During scheduling, execution priority is allocated based on the resource demand priority and latency sensitivity level of the subgraph: high-priority subgraphs are given priority to occupy idle resources in computing units, and latency-sensitive subgraphs are prioritized for scheduling to computing units with a load fluctuation coefficient of less than 0.1 to ensure stable execution latency. The post-scheduling monitoring mechanism is optimized to combine real-time monitoring and periodic evaluation: real-time monitoring of the execution progress and resource usage of subgraphs; if the execution latency exceeds 15% of the estimated latency or the resource usage exceeds 20% of the pre-allocated amount, scheduling adjustment is immediately triggered, and the subgraph is migrated to other idle computing units; every 3 scheduling cycles, the execution efficiency of all subgraphs is periodically evaluated, the execution latency compliance rate and resource utilization compliance rate are statistically analyzed, and the task adaptation success rate of computing units is updated to provide feedback for subsequent scheduling decisions.

[0088] In this embodiment, the online calibration mechanism adopts an adaptive learning rate fine-tuning algorithm to optimize the parameter update strategy and calibration triggering conditions, thereby improving calibration efficiency and reducing resource consumption during the calibration process while ensuring stable prediction accuracy.

[0089] The calibration trigger condition now includes a new accuracy fluctuation trend judgment: when the fluctuation of the model's prediction accuracy exceeds 2% within 5 consecutive prediction periods and shows a downward trend, online calibration will be initiated even if other trigger conditions are not met; at the same time, the trigger mechanism after the structure sparsification process has been optimized so that calibration is only triggered when the pruning ratio exceeds 20%, avoiding unnecessary calibration consumption after light pruning.

[0090] The model fine-tuning employs the Adam adaptive learning rate algorithm, replacing the mini-batch gradient descent algorithm in Implementation Example 1. This algorithm dynamically adjusts the learning rate for each parameter by calculating the first and second moment estimates of the gradient. Compared to gradient descent with a fixed learning rate, it converges faster and effectively avoids local optima. During fine-tuning, the batch size is dynamically adjusted based on the current cluster's memory resources: when cluster memory utilization is below 40%, the batch size is set to 96; when memory utilization is between 40% and 70%, the batch size is set to 64; and when memory utilization is above 70%, the batch size is set to 32, ensuring that the fine-tuning process does not affect the operation of other tasks on the cluster.

[0091] The parameter update strategy employs a hierarchical update mechanism, selecting update targets based on the functional characteristics and sensitivity of the layer in which the parameters reside: the sensitivity weight coefficient for parameters in the input layer and hardware state adaptation layer is 1.2, the weight coefficient for parameters in the hidden layer is 1.0, and the weight coefficient for parameters in the output layer is 0.8. After calculating the sensitivity score of each parameter, they are sorted in descending order of score, and the top 25% of parameters are selected for updating, prioritizing parameters that have a significant impact on model accuracy and are related to hardware state adaptation. An adaptive stopping mechanism is used for the number of fine-tuning iterations. Fine-tuning is immediately stopped when the accuracy improvement on the validation set is less than 0.5% after three consecutive iterations, or when the accuracy returns to the pre-pruning level, to avoid resource waste caused by excessive iteration.

[0092] The dynamic adjustment of calibration frequency introduces an accuracy recovery rate metric, which is the ratio of the accuracy improvement after calibration to the calibration time. When the accuracy recovery rate is higher than 0.5% / minute, the calibration frequency remains at the current level; when the accuracy recovery rate is between 0.3% and 0.5% / minute, the calibration frequency is increased by one level; when the accuracy recovery rate is lower than 0.3% / minute, the calibration frequency is increased by two levels, and the initial value of the learning rate of the fine-tuning algorithm is adjusted to ensure that the calibration efficiency meets the requirements.

[0093] The collaborative architecture of the four core modules in this embodiment is optimized based on the first embodiment. It adopts a hybrid message middleware and a distributed consensus protocol to improve the reliability and flexibility of communication between modules and adapt to the distributed deployment requirements of hybrid heterogeneous clusters. The hardware performance monitoring module's data acquisition agent adopts a lightweight design, optimizing resource usage for ARM architecture edge computing units. The agent's memory usage is kept below 50MB, and CPU utilization is less than 5%. Inter-module communication utilizes a hybrid message middleware mix of RabbitMQ and Kafka: high-frequency data transmission between the hardware performance monitoring module and the dynamic optimization decision-making module uses Kafka to ensure high throughput; control signals and metadata transmission between the dynamic optimization decision-making module, model reconstruction module, and runtime scheduling module use RabbitMQ to ensure reliable and timely message transmission. The communication protocol uniformly adopts gRPC, which further reduces transmission latency and improves data serialization and deserialization efficiency compared to HTTP / 2.

[0094] The dynamic optimization decision-making module is deployed on the cluster's master control node, employing a master-slave backup architecture. The master node is responsible for generating model optimization trigger signals, while slave nodes synchronize data in real time and monitor the master node's status. When the master node fails, the slave nodes switch over within 10 seconds, ensuring uninterrupted bottleneck identification and trigger signal generation. The model reconstruction module uses a distributed deployment approach, allocating structural sparsity processing and subgraph splitting tasks to multiple high-performance computing nodes for parallel execution. Distributed locks (based on ZooKeeper) ensure mutual exclusion and data consistency in task execution, resulting in processing efficiency improvements of over 30% compared to Implementation Example 1.

[0095] The runtime scheduling module has added a scheduling conflict resolution mechanism. When multiple high-priority subgraphs simultaneously request resources from the same computing unit, they are sorted based on their latency sensitivity level and task urgency. Subgraphs with high latency sensitivity and high task urgency are given priority for resource allocation, while the remaining subgraphs are temporarily queued or assigned to backup computing units. An event-driven mechanism between modules introduces event priorities: bottleneck trigger events and scheduling adjustment events have the highest priority, data acquisition events and calibration feedback events have medium priority, and state synchronization events have the lowest priority, ensuring timely handling of critical events.

[0096] This embodiment requires no modification to existing hardware equipment, is compatible with heterogeneous computing units of various architectures, has stronger compatibility and practicality, and can be widely used in load prediction and resource optimization scenarios of various hybrid heterogeneous server clusters.

[0097] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A neural network optimization method for server cluster load prediction, characterized in that, Includes the following steps: Step 1: Collect hardware performance counter data from multiple computing units in the server cluster in real time; Step 2: Determine whether there is a hardware performance bottleneck based on the hardware performance counter data; Step 3: When the aforementioned hardware performance bottleneck exists, dynamically trigger the structural sparsity processing of the load prediction neural network model; Step 4: Perform subgraph splitting on the load prediction neural network model after the structure has been sparsified to obtain multiple subgraphs; Step 5: Based on the real-time hardware status of the server cluster, schedule the multiple subgraphs to the corresponding heterogeneous computing units for execution; The hardware performance counter data reflects the internal operating status of the computing unit. The structural sparsity processing reduces the computational complexity and resource consumption of the neural network model by reducing the number of neuron connections. The subgraph splitting divides the model into multiple independent subgraphs based on the computation graph structure and operator dependencies of the neural network model. The scheduling process allocates the subgraphs to different types of computing units based on the real-time hardware status to achieve parallel execution.

2. The neural network optimization method for server cluster load prediction according to claim 1, characterized in that, Step one specifically includes the following steps: Continuously monitor hardware performance events of the computing unit through the operating system-level interface; The collected hardware performance counter data includes multiple indicators such as cache hit rate, memory bandwidth utilization, branch prediction error rate, and instruction cycle value. The collected hardware performance counter data is preprocessed, including data cleaning and aggregation. The data cleaning is used to remove outliers and noisy data; The aggregation process is used to calculate a short-term moving average to smooth out instantaneous fluctuations; The preprocessed hardware performance counter data is compared with a preset security threshold. The preset security threshold is dynamically adjusted based on historical hardware performance data; The frequency of data collection for the hardware performance counter is dynamically adjusted according to the server cluster load status. The data collection process covers all active computing units in the server cluster, including the central processing unit, graphics processing unit, and neural network processor.

3. The neural network optimization method for server cluster load prediction according to claim 1, characterized in that, Step two specifically includes the following steps: Establish a rule base for judging hardware performance bottlenecks; The rule base contains mapping relationships between various hardware performance counter metrics and performance bottlenecks; The mapping relationship is established based on a machine learning model or predefined rules; The judgment process includes calculating the short-term trend of the hardware performance counter metrics; When the hardware performance counter index exceeds the preset safety threshold multiple times consecutively, it is determined that there is a hardware performance bottleneck. The types of hardware performance bottlenecks include cache resource bottlenecks, memory bandwidth bottlenecks, and computing resource bottlenecks. Generate a corresponding model optimization trigger signal for each type of hardware performance bottleneck; The model optimization trigger signal includes the target computing unit identifier and bottleneck severity information; The model optimization trigger signal is used to drive the subsequent structural sparsification process.

4. The neural network optimization method for server cluster load prediction according to claim 1, characterized in that, Step three specifically includes the following steps: Upon receiving the model optimization trigger signal, the structure sparsification process is initiated. The structural sparsification process includes analyzing the importance of parameters in the neural network model; The importance of the parameter is calculated using a hybrid importance score; The hybrid importance score combines the parameter norm and the parameter's contribution to the hardware bottleneck layer; The structure sparsification process employs a structured pruning method. The structured pruning removes neurons or filters that contribute less to the neural network model; The degree of pruning is dynamically adjusted based on the severity of the hardware performance bottleneck. During the structural sparsification process, neuron connections in the neural network model whose mixed importance score is higher than a preset threshold are preserved. The neuronal connections are identified through gradient information or activation intensity; The structure is sparsified to generate an optimized neural network model version.

5. The neural network optimization method for server cluster load prediction according to claim 1, characterized in that, Step four specifically includes the following steps: The computational graph structure of the neural network model after the structure sparsification process is analyzed; The computation graph structure includes operator types and dependencies between operators; The computation graph structure is divided into multiple logical subgraphs according to the operator type; Logical subgraph grouping is based on operator computational characteristics and resource requirements; The computational characteristics include computationally intensive, memory access intensive, and control intensive. The subgraph splitting process ensures that the operators within each subgraph have high cohesion; Dependencies between subgraphs are represented by data flow edges; Generate metadata for each subgraph describing its computational characteristics and resource requirements; The metadata is used for subsequent scheduling decisions.

6. The neural network optimization method for server cluster load prediction according to claim 1, characterized in that, Step five specifically includes the following steps: Maintain dynamic resource profiles for the server cluster; The dynamic resource profile records the real-time load status and performance indicators of each computing unit; The real-time load status is obtained through the hardware performance counter data and system load data; The scheduling decision is based on the matching of the metadata in the subgraph and the dynamic resource profile; The matching process is based on the computational characteristics and computational unit expertise described in the subgraph; The computing unit's expertise was evaluated using historical execution efficiency data; The scheduling process prioritizes allocating the computationally intensive subgraph to the graphics processor. The control-intensive subgraph is preferentially allocated to the central processing unit; The memory access-intensive subgraph is assigned to a high-bandwidth memory computing unit; The execution effect of the subgraph is monitored after scheduling and verified using the hardware performance counter data.

7. The neural network optimization method for server cluster load prediction according to claim 1, characterized in that, The method also includes step six, an online calibration mechanism: The online calibration mechanism is activated after the structure sparsification process or subgraph scheduling. The online calibration mechanism uses the latest load data to fine-tune the optimized neural network model; The fine-tuning process employs a mini-batch gradient descent algorithm. The mini-batch gradient descent algorithm uses recent load data within a sliding window; The fine-tuning objective is to maintain the prediction accuracy of the neural network model; The fine-tuning process only updates some parameters of the neural network model; The selection of some parameters is based on parameter sensitivity analysis; The parameter sensitivity is calculated using gradient magnitude or error contribution. The execution frequency of the online calibration mechanism is adjusted according to the rate of change of model accuracy.

8. A neural network optimization method for server cluster load prediction according to any one of claims 1 to 7, characterized in that, The method is implemented collaboratively by a hardware performance monitoring module, a dynamic optimization decision-making module, a model reconstruction module, and a runtime scheduling module. The hardware performance monitoring module is responsible for collecting and processing the hardware performance counter data; The dynamic optimization decision module generates the model optimization trigger signal based on the hardware performance counter data; The model reconstruction module performs the structural sparsification process and subgraph splitting; The runtime scheduling module handles subgraph scheduling and resource allocation; The modules communicate with each other through an event-driven mechanism; The event-driven mechanism ensures data consistency and processing sequence between modules; Each module is implemented using a distributed architecture to support server cluster environments; The distributed architecture achieves decoupling between modules through message middleware.

Images