Distributed component dynamic resource allocation method based on multi-objective optimization

Abstract

The invention discloses a distributed component dynamic resource allocation method based on multi-objective optimization, which is characterized in that a PPO algorithm is introduced into a distributed system, dynamic adjustment is carried out aiming at a plurality of optimization objectives to optimize the overall configuration of resources, and the system firstly collects the real-time state, the task demand and the resource use condition of a distributed component; and then training an intelligent agent by using a PPO algorithm to gradually optimize a resource allocation strategy according to environment feedback, and finally realizing long-term optimization of a resource scheduling process by continuously interacting with the environment and continuously adjusting the strategy through the PPO algorithm. According to the method, the PPO algorithm in reinforcement learning is combined, efficient resource allocation of the distributed components in the complex dynamic environment is achieved, different from an existing rule driving or static optimization method, the allocation strategy can be adjusted in a self-adaptive mode according to task requirements, resource use conditions and system loads which change in real time, the resource utilization rate is increased, and the resource utilization rate is increased. And the system burden is reduced, and efficient operation of the system under variable conditions is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of distributed computing and resource management, and in particular relates to a distributed component dynamic resource allocation method based on multi-objective optimization. Background Art

[0002] With the rapid development of distributed computing and cloud computing technologies, resource management is becoming increasingly important in large-scale distributed systems. Existing resource allocation methods are mostly based on static configurations or preset rules, often relying on manual intervention and periodic updates. These methods are unable to cope with changes in system load, fluctuations in task requirements, and the dynamic adjustment requirements in complex environments, resulting in poor resource utilization and unstable system performance. In large-scale distributed systems, with the continuous growth of computing demands and data transmission volumes, how to efficiently and dynamically allocate system resources to ensure high availability, load balancing, and energy efficiency has become a pressing challenge.

[0003] In recent years, the rapid development of artificial intelligence and reinforcement learning technologies, particularly adaptive decision-making methods based on deep reinforcement learning (such as the PPO algorithm), has found applications in areas such as resource scheduling and dynamic load balancing. These methods can continuously optimize decision-making strategies through interaction with the environment, gradually improving the accuracy and adaptability of resource allocation. However, existing reinforcement learning methods still face challenges in multi-objective resource allocation optimization. In particular, how to simultaneously balance multiple objectives such as load balancing, response time, and energy efficiency to ensure the overall effectiveness of the system remains an unresolved issue in highly dynamic environments. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides a distributed component dynamic resource allocation method based on multi-objective optimization, which realizes efficient dynamic allocation and adaptive adjustment of distributed system resources by combining reinforcement learning with multi-objective optimization technology.

[0005] The technical solution adopted by the present invention is: a distributed component dynamic resource allocation method based on multi-objective optimization, the specific steps are as follows:

[0006] S1. Deploy resource monitoring devices in the distributed system to collect status information, resource usage, and task requirements of each component in real time, and store the data in a unified data management center;

[0007] First, based on the operating environment characteristics and resource requirements of distributed components, high-precision resource monitoring equipment is installed on each computing node in the distributed system. The selected monitoring equipment includes: computing resource monitoring module, network traffic monitoring module and task scheduling monitoring module.

[0008] Among them, the computing resource monitoring module collects key indicators in the distributed system in real time, that is, the status information of each component, including: CPU utilization, memory usage, network bandwidth, storage usage, disk read and write rates, and task queue length; the network traffic monitoring module collects high-speed network traffic data in real time; the task scheduling monitoring module monitors the resource usage dynamics of each component and changes in task requirements, including: CPU limits, memory usage, and task running time.

[0009] All real-time monitoring data is efficiently transmitted via the gRPC protocol and transmitted to the distributed data management center via a unified API interface. A time series database is used for persistent storage. In terms of storage architecture, short-term data is stored in the distributed data management center, while long-term data is stored in the time series database, which is efficiently compressed using the Zstandard algorithm.

[0010] Among them, the distributed data management center includes: Kafka, Redis; the time series database includes: InfluxDB; short-term data refers to data within 24 hours, and long-term data refers to data exceeding 24 hours.

[0011] The distributed system incorporates an adaptive data sampling strategy. When CPU load is low (<30%), the data sampling interval is adjusted to 500ms; when the load is within the normal range of 30%-70%, the sampling interval is adjusted to 200ms; and when the load is high (>70%), the sampling interval is shortened to 100ms. When the computing resource monitoring module detects an anomaly, the system automatically triggers an alarm mechanism, sends a WebHook notification to the operations team, and combines it with an automatic scaling strategy to dynamically allocate additional resources to the computing task. All abnormal data is stored in Elasticsearch.

[0012] The abnormal situations include:

[0013] 1) CPU abnormality: When the CPU usage exceeds 95% continuously, it is considered abnormal;

[0014] 2) Memory abnormality: When the memory usage reaches or exceeds 90%, accompanied by frequent memory overflow and increased swap operations, it is considered abnormal;

[0015] 3) Disk abnormality: When the read and write rates of the storage device fluctuate abnormally, or the disk I / O latency increases significantly, it is considered an abnormality;

[0016] 4) Network anomaly: When the network delay continuously exceeds the set threshold, the packet loss rate increases significantly, or the bandwidth utilization suddenly reaches an abnormally high level, it is determined to be an anomaly;

[0017] 5) Task scheduling anomaly: When a task is delayed in the task queue for a long time, the task response time is abnormally extended, or the frequency of task scheduling failure increases, it is considered an anomaly;

[0018] 6) Other service anomalies: When the error frequency of key APIs, service connections, or cross-component communications increases sharply or the response time exceeds the standard, an alarm will be issued as an abnormal situation.

[0019] S2. Preprocess the real-time data collected in step S1 to ensure data quality and consistency so that the data meets the input requirements of the PPO algorithm;

[0020] The preprocessing includes: data cleaning, data normalization, data denoising, and data consistency verification.

[0021] S3. Build and train a PPO agent model based on the PPO algorithm. The PPO agent model learns to optimize resource allocation strategies under multi-objective constraints through interaction with the system environment, and adjusts the allocation strategy based on task requirements, load changes, and system status.

[0022] The PPO agent model is a reinforcement learning model, including a policy network and a value network. The multi-objective constraints include load balancing, task response time, and energy efficiency.

[0023] S4. Integrate the PPO agent model trained in step S3 into the resource management layer of the distributed system, receive real-time status feedback from each distributed component, and dynamically allocate system resources according to the strategy generated by the model to ensure optimal performance under different loads;

[0024] S5. Combined with the multi-objective optimization algorithm, that is, optimizing the resource allocation strategy under multi-objective constraints, considering multiple objectives in the resource allocation process, adjusting the weight of each objective based on the PPO agent model to ensure the balance of various objectives and maximize the overall efficiency of the system;

[0025] Among them, the multi-objective optimization algorithms include: NSGA-II and MOEA / D.

[0026] S6. During system operation, through real-time monitoring and performance evaluation, continuously track resource allocation effects, provide feedback on system performance, and use the PPO algorithm to incrementally learn and optimize the optimal resource allocation strategy under multi-objective constraints;

[0027] S7. Regularly analyze the system's performance, update and expand the data set based on new task requirements and environmental changes, and further train the PPO model.

[0028] Furthermore, the step S2 is specifically as follows:

[0029] S21, data cleaning;

[0030] First, the original real-time monitoring data collected in step S1 is preliminarily screened to remove redundant, abnormal or missing data points. That is, for data with duplicate records, a hash value-based deduplication method is used to process them. For abnormal data, the Z-score method is used to perform statistical analysis on various monitoring indicators. If the Z-score value of a data point exceeds the set threshold, it is marked as abnormal data and repaired by interpolation based on the trend of the previous and next timestamps. For missing data, if the missing ratio is lower than p%, linear interpolation or Lagrange interpolation is used to fill it. If the missing ratio is higher than 3p%, the data sample is directly discarded.

[0031] The threshold setting includes: the 3σ principle and the value of p is determined according to the actual situation.

[0032] S22, data normalization;

[0033] The multidimensional heterogeneous data obtained after data cleaning in step S21 is normalized and mapped to the interval [0, 1] to eliminate the dimensional differences between different data dimensions.

[0034] The multi-dimensional heterogeneous data includes: number of CPU cores and memory GB units.

[0035] For numerical data, two normalization methods are used: minimum-maximum normalization and zero-mean normalization. The appropriate normalization method is automatically selected based on the distribution of the monitored data.

[0036] The calculation expression of minimum-maximum normalization is as follows:

[0037]

[0038] Among them, X represents the original real-time monitoring data, X min and X max Represent the minimum and maximum values of the feature respectively, and the normalized data X ′ The value range is [0,1].

[0039] For data that conforms to the normal distribution, zero mean normalization is used, and the calculation expression is as follows:

[0040]

[0041] Among them, μ represents the mean of the data, σ represents the standard deviation of the data, and the mean of the normalized data is 0 and the standard deviation is 1.

[0042] S23, data denoising;

[0043] Gaussian filtering and wavelet denoising methods are used to smooth instantaneous fluctuations and extract stable resource usage trend characteristics.

[0044] First, Gaussian filtering is used to smooth the data. The calculation expression is as follows:

[0045]

[0046] Among them, G(i,j) represents the weight of the filter, σ represents the standard deviation, and x, y represent the pixel position offset.

[0047] Then, for time series data, the wavelet denoising method is used to remove high-frequency noise through three steps of decomposition, threshold denoising and reconstruction to improve the smoothness of the data.

[0048] First, select the appropriate wavelet basis and decomposition layer number J, perform discrete wavelet transform on the signal, and decompose it into approximate components A in different frequency bands. j and detail component D j , the expression is as follows:

[0049]

[0050] Among them, x[n] represents the discretized data, φ j,k represents the scaling function; ψ j,k represents the wavelet function.

[0051] When denoising in the wavelet domain, only the detail coefficient D j Perform threshold processing and set the detail coefficient obtained by wavelet decomposition of the jth layer as D j ={d j,k :k=1,2,…}, select a threshold λ j , and D j Use hard threshold processing to form the denoised coefficient The expression is as follows:

[0052]

[0053] After completing the threshold processing of the detail coefficients, A is transformed into J and Merge step by step to get the reconstructed signal The reconstruction process expression is as follows:

[0054]

[0055] Among them, A J Preserve the main energy and contour of the signal; Represents high-frequency detail information after noise removal.

[0056] S24, data consistency check;

[0057] First, the timestamp comparison method is used to check whether the timestamps of each data point are synchronized. If the time deviation of the data exceeds the set threshold, time interpolation correction is performed.

[0058] Then, logical rule verification is used to check whether the data complies with physical constraints. If logically abnormal data is found, it is marked as abnormal data and processed in combination with data correction strategies.

[0059] The physical constraints include: CPU usage should not exceed 100%, and memory usage should not be higher than total memory.

[0060] Furthermore, the step S3 is specifically as follows:

[0061] S31. Construct a PPO agent model based on the PPO algorithm;

[0062] The PPO agent model is constructed based on the PPO algorithm and adopts the Actor-Critic architecture, including a policy network and a value network. The network layer adopts the LSTM structure to capture the temporal characteristics of load changes.

[0063] Among them, the policy network Actor is used to generate resource allocation decisions. The input is the current resource status characteristics of the system, and the output is the allocation strategy of each resource. The network structure adopts two fully connected layers, each layer includes 128 neurons, and uses the ReLU activation function; the value network Critic is used to evaluate the value function of the current strategy and guide the learning process of the agent. The structure is consistent with the policy network, but the output is a single state value. During the optimization process, the Adam optimizer is used for gradient update, and the initial learning rate is set to 3×10 -4 , and combined with gradient clipping technology to control the update amplitude of the gradient.

[0064] S32, define multi-objective constraints;

[0065] During resource allocation, multiple objectives are optimized, including load balancing, task response time, and energy efficiency. All optimization objectives are integrated into the reward function of the PPO algorithm. The final reward function R is expressed as follows:

[0066] R=w1·R balance +w2·R response +w3·R energy

[0067] Among them, w1, w2, and w3 represent load balancing R balance , task response time R response and energy efficiency R energy The weight of the target is dynamically adjusted according to actual needs.

[0068] S33, Environmental Modeling and Simulation;

[0069] A distributed system simulation environment is constructed based on real system logs. A Poisson distribution is used to simulate resource state changes, task demand dynamics, system load distribution, random node failure times, and spatiotemporal variability in network latency across components in the distributed system. The simulation environment is constructed using the Gym framework. The state space includes CPU utilization, memory usage, and task queue length, while the action space is defined as the allocation ratio of each resource. Task requests in the environment are generated according to a Poisson distribution, and load variations follow a normal distribution. The PPO agent model interacts with the system in the simulation environment, observing the current state, taking actions, and optimizing its own strategy based on a reward function.

[0070] S34, training the PPO agent model;

[0071] The training process uses the PPO algorithm to optimize the policy network and value network parameters θ through multiple rounds of iteration. The advantage function is used to evaluate the agent's behavior benefits. The calculation expression is as follows:

[0072] A(s,a)=Q(s,a)-V(s)

[0073] Among them, s represents the state, a represents the action, A(s,a) represents the advantage function, Q(s,a) represents the state-action value function, and V(s) represents the state value function.

[0074] Then the policy cutting method is used to limit the policy update range. The expression is as follows:

[0075]

[0076] Among them, L CLIP Represents the objective function in PPO, and the network parameters θ are updated by maximizing the objective function. θ represents the expectation, r(θ) represents the ratio of the current policy to the old policy, A represents the advantage function, and ∈ represents the pruning hyperparameter. During training, mini-batch SGD is used for optimization. The time steps and batch size of each update round are set according to the actual situation.

[0077] During the training of PPO, the reward function is used to determine whether the strategy has converged. After each iteration, the average reward change of the agent model in the environment is recorded. If the reward continues to fluctuate around a certain value after multiple iterations and does not rise significantly, the strategy is considered to have converged and the iteration is stopped.

[0078] S35, performance evaluation and optimization;

[0079] During training, the performance of the agent model is regularly evaluated using Pareto frontier analysis. Evaluation metrics include resource allocation efficiency, goal achievement rate, and policy stability.

[0080] Among them, resource allocation efficiency is measured by calculating the average reward value of the intelligent agent model under different load conditions, the goal achievement rate is calculated by comparing the degree of closeness between the allocation strategy of the intelligent agent model and the theoretical optimal solution, and the strategy stability is judged by measuring the strategy variance of the intelligent agent model under different initial states.

[0081] Based on the evaluation results, Bayesian optimization is used to dynamically adjust the weights of the reward function and optimize the model parameters to further enhance the agent's decision-making ability. After training is complete, the agent model is saved and used in the actual resource allocation process.

[0082] Furthermore, the step S4 is specifically as follows:

[0083] S41, model deployment and interface adaptation;

[0084] The PPO agent model trained in step S3 is packaged as a lightweight inference service. During deployment, the model is packaged using TensorFlow Serving, optimized for inference efficiency, and converted to the ONNX format. The inference service is packaged in a Docker container, with each container occupying no more than 80% of the memory. It is deployed as a sidecar container in the Kubernetes cluster, and horizontal autoscaling adjusts the number of pods to keep response latency within a reasonable range. Regarding interface adaptation, a unified resource management interface is established using the gRPC protocol to receive real-time status feedback from each distributed component. Several resource status feedback requests are processed per second, including CPU utilization, memory usage, network bandwidth, task queue length, and latency metrics. All input data is converted to a standardized tensor format to match the input requirements of the PPO agent model. The input data format is a l×4-dimensional tensor, with each row containing CPU utilization, memory usage, network bandwidth, and task queue length. All data is normalized to the range [0, 1].

[0085] Wherein, l represents the data length.

[0086] S42, Strategic Reasoning and Dynamic Decision-making;

[0087] After model deployment is complete in step S41, the PPO agent model performs policy inference based on real-time status data and generates a multidimensional resource allocation decision vector, including the required computing resource allocation ratios for each component, network bandwidth priority adjustment factors, and task migration recommendations. The model inference takes no more than 20 milliseconds, and the output of the PPO agent model is a 1×4-dimensional resource allocation vector, with each row representing the allocated CPU core ratio, memory allocation ratio, network bandwidth allocation ratio, and a binary value indicating whether the task should be migrated.

[0088] The elastic scaling ratio is ±20% of the resource quota, and the target nodes for task migration are the three candidate nodes with the lowest load.

[0089] The decision frequency of the PPO agent model is dynamically adjusted according to the current system load, increasing the inference frequency when the load fluctuates violently and reducing the frequency in a stable state.

[0090] S43, resource scheduling instruction distribution;

[0091] The inference results of the PPO agent model are converted into executable resource scheduling instructions, and the instructions are broadcast through the publish-subscribe model. After receiving the instructions, the node agent calls the local resource manager to perform quota adjustments. In the conversion rules, when the CPU allocation ratio output by the PPO agent model exceeds 0.9, the system triggers Pod automatic expansion to adjust the CPU limit. When the task migration value is 1, the system generates a task rescheduling instruction to migrate the current task from the high-load node to a computing node with a CPU utilization rate of less than 50%. When the network bandwidth allocation ratio exceeds 0.8, the bandwidth priority is adjusted to increase the network priority of the task to the highest level. All resource scheduling instructions are transmitted through a distributed message queue, and the maximum throughput of the message queue is 50,000 messages per second.

[0092] S44, real-time distribution effect verification;

[0093] After the resource allocation strategy is executed in step S43, its effectiveness is monitored in real time. A / B testing is used to compare the performance of the new strategy with the baseline strategy. The computing resource monitoring module collects task metrics, including average response time, load balancing, and resource utilization improvement. The average task response time is calculated by averaging the execution time of all tasks using the following expression:

[0094]

[0095] Among them, T avg represents the average response time, N represents the total number of tasks, T i Indicates the execution time of the i-th task.

[0096] The load balancing degree is measured by calculating the standard deviation of resource utilization of each computing node. The calculation expression is as follows:

[0097]

[0098] Among them, σ load Indicates the load balancing degree, M indicates the total number of computing nodes, U j represents the CPU utilization of the jth computing node, U avg Indicates the average CPU utilization of all compute nodes.

[0099] If the indicator does not reach the expected optimization threshold, that is, the load balance is less than 10%, the system triggers the online fine-tuning mechanism of the PPO agent model, recalculates the policy gradient based on the incremental data and updates the model parameters. The calculation expression of the policy gradient update is as follows:

[0100]

[0101] Among them, θ represents the current policy parameters, θ′ represents the updated policy parameters, represents the gradient of the policy optimization objective function, and the learning rate is set to 0.0003.

[0102] If both resource utilization and load balancing meet the requirements, the model parameters are retained and the verification ends.

[0103] S45, fault tolerance and rollback mechanism;

[0104] If resource utilization and load balancing don't meet requirements, the system introduces a fault-tolerance and rollback mechanism and automatically adjusts model parameters. Upon detecting the abnormal event mentioned in step S1, the system suspends the current policy execution and switches to a threshold-based static resource allocation strategy, adjusting CPU limits and lowering the priority of non-currently executing tasks and their associated tasks. If more than three task scheduling failures are detected, the system automatically rolls back to the previous PPO model version and records the abnormal data, which is then stored in the Elasticsearch abnormality log database.

[0105] Among them, abnormal data is injected into the PPO training set and used to enhance the robustness of the model in the next round of training. And after each abnormality occurs, the system executes the experience replay mechanism to retrain the PPO model.

[0106] Furthermore, the step S5 is specifically as follows:

[0107] In terms of load balancing, the objective function is set to achieve uniform distribution of loads among components in the system. That is, the system measures the load balancing situation by calculating the standard deviation of the CPU utilization of each computing node, and adjusts the PPO reward function when the balance degree exceeds 0.15 to increase the weight of the load balancing target.

[0108] In terms of response time, the goal of task response time optimization is to minimize the execution delay of the task. When the average response time of the task exceeds 200ms, the PPO model prioritizes scheduling resources on low-load nodes and dynamically adjusts the task queue priority.

[0109] In terms of energy efficiency optimization, the goal of minimizing power consumption is introduced. That is, the system measures the optimization effect by calculating the energy consumption ratio of unit tasks. When the energy consumption ratio exceeds 0.5W / ms, the CPU frequency of low-priority tasks is reduced, and task merging technology is used to reduce computing resource usage.

[0110] The reward function of the PPO agent model is dynamically adjusted based on load balancing, task response time, and energy efficiency. The weight of each objective is adaptively adjusted according to the system state during operation. When the deviation of a certain objective exceeds a set threshold, the system automatically increases the weight of that objective.

[0111] Furthermore, the step S6 is specifically as follows:

[0112] The computing resource monitoring module collects the CPU utilization, memory occupancy, task response time, and task queue length of the computing node every 100ms and records key performance indicators. All monitoring data is transmitted to the distributed data management center through the message queue, stored in the database, and the short-term effect of the resource allocation strategy is evaluated through sliding window statistics. If the key performance indicator is lower than 15% of the training indicator, the incremental learning mechanism of the PPO model is triggered. During the incremental learning process, the system first filters out abnormal data within the past 10 minutes and recalculates the policy gradient based on this data. The learning rate is set to 0.0003. The updated PPO model is immediately deployed to the inference server and applied to the next round of resource allocation decisions. The system uses an experience replay mechanism to randomly extract data from the past period of time for comparative learning based on actual conditions.

[0113] Furthermore, the step S7 is specifically as follows:

[0114] Regularly analyze historical performance data, changes in task requirements, and dynamic changes in the system operating environment to identify resource allocation bottlenecks. Expand the training set by injecting simulated load data, and reuse existing model parameters in combination with transfer learning technology to accelerate the convergence speed of the PPO intelligent agent model in new scenarios.

[0115] Among them, identifying resource allocation bottlenecks includes: long-term overload of specific nodes; simulated load data includes: peak traffic and sudden tasks.

[0116] The system conducts a comprehensive performance evaluation every 24 hours, using metrics including average task response time, load balancing, and energy efficiency optimization. All data is stored in a performance analysis database and aggregated over time. When the system detects a performance degradation trend, it automatically triggers retraining of the PPO agent model. During training, the system constructs a new training set based on the most recent week's monitoring data and performs policy optimization. The number of time steps and batch size included in each training round is set based on actual conditions. The optimization goal is to minimize task response time and maximize energy efficiency, with the policy update range controlled between 0.8 and 1.2.

[0117] After training is complete, the new model is tested and automatically deployed to the inference server if it meets performance requirements, replacing the old model. Furthermore, after each training session, the dataset is expanded, mixing past data with current data based on actual conditions to optimize the dynamic resource allocation strategy for distributed components.

[0118] Beneficial effects of the present invention: The method of the present invention introduces the PPO algorithm in a distributed system and dynamically adjusts multiple optimization objectives to optimize the overall configuration of resources. The system first collects the real-time status, task requirements and resource usage of distributed components, and then uses the PPO algorithm to train the intelligent agent so that it can gradually optimize the resource allocation strategy based on environmental feedback. The PPO algorithm continuously adjusts the strategy through continuous interaction with the environment, and ultimately achieves long-term optimization of the resource scheduling process. The method of the present invention combines the PPO algorithm in reinforcement learning to achieve efficient resource allocation of distributed components in complex dynamic environments. Unlike existing rule-driven or static optimization methods, it can adaptively adjust the allocation strategy according to real-time changing task requirements, resource usage and system load, improve resource utilization, reduce system burden, and ensure efficient operation of the system under changing conditions. The method of the present invention combines reinforcement learning and multi-objective optimization algorithms, and dynamically adjusts the resource allocation strategy through real-time monitoring of the status of each component and resource usage in the distributed system to achieve optimization of system performance. The method of the present invention improves the computing power and resource utilization of the system through adaptive decision-making mechanisms and real-time optimization strategies. Through collaborative hardware and software design, combined with lightweight model deployment and efficient data transmission mechanisms, a complete closed-loop optimization process from data collection and intelligent analysis to dynamic allocation is achieved. The system can be widely applied in cloud computing, large-scale data centers, high-performance computing clusters, and other scenarios, providing key technical support for intelligent resource management, effectively reducing resource waste and operation and maintenance costs, and improving overall system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0119] Figure 1 This is a flow chart of a distributed component dynamic resource allocation method based on multi-objective optimization of the present invention.

[0120] Figure 2 This is a data collection flow chart in an embodiment of the present invention.

[0121] Figure 3 This is a flow chart of agent training in an embodiment of the present invention.

[0122] Figure 4 This is a flowchart of model deployment and decision-making in an embodiment of the present invention.

[0123] Figure 5 This is a flowchart of system monitoring and incremental optimization in an embodiment of the present invention. DETAILED DESCRIPTION

[0124] The method of the present invention is further described below with reference to the accompanying drawings and examples.

[0125] like Figure 1 As shown in FIG, a flow chart of a distributed component dynamic resource allocation method based on multi-objective optimization of the present invention is shown, and the specific steps are as follows:

[0126] S1. Deploy resource monitoring devices in the distributed system to collect real-time status information, resource usage, and task requirements of each component (providing high-quality data support for subsequent resource allocation) and store the data in a unified data management center;

[0127] like Figure 2 As shown in the figure, first, based on the operating environment characteristics and resource requirements of the distributed components, high-precision resource monitoring equipment is installed on each computing node in the distributed system to ensure comprehensive and real-time data collection. The selected monitoring equipment includes: computing resource monitoring module, network traffic monitoring module, and task scheduling monitoring module.

[0128] Among them, the computing resource monitoring module of this embodiment adopts Prometheus Node Exporter to collect key indicators in the distributed system in real time, that is, the status information of each component, including: CPU utilization, memory usage, network bandwidth, storage usage, disk read and write rate, and task queue length, and process it through PromQL query; the network traffic monitoring module of this embodiment relies on Wireshark and InfluxDB, combined with the Intel X550 10GbE network card, to collect high-speed network traffic data in real time; the task scheduling monitoring module of this embodiment is based on Kubernetes's kube-state-metrics to monitor the resource usage dynamics of each component and changes in task requirements, including: CPU limit, memory usage, and task running time.

[0129] All real-time monitoring data is efficiently transmitted via the gRPC protocol, ensuring low latency and high throughput. It is then transferred to a distributed data management center via a unified API interface and persistently stored using a time series database. In this embodiment, the storage architecture stores short-term data in a Redis Cluster to ensure low latency access, while long-term data is stored in InfluxDB and efficiently compressed using the Zstandard algorithm to reduce storage space.

[0130] Among them, the distributed data management center includes: Kafka, Redis; the time series database includes: InfluxDB; short-term data refers to data within 24 hours, and long-term data refers to data exceeding 24 hours.

[0131] To optimize data collection flexibility, the distributed system introduces an adaptive data sampling strategy. When the CPU load is low (<30%), the data sampling interval is adjusted to 500ms; when the load is within the normal range of 30%-70%, the sampling interval is adjusted to 200ms; when the load is high (>70%), the sampling interval is shortened to 100ms to ensure data accuracy under high load conditions. When the computing resource monitoring module detects an anomaly, the system automatically triggers an alarm mechanism, sends a WebHook notification to the operations team, and combines it with the Auto Scaling strategy to dynamically allocate additional resources to the computing task to prevent system performance degradation. At the same time, all abnormal data is stored in Elasticsearch for subsequent analysis.

[0132] The abnormal situations include:

[0133] 1) CPU abnormality: When the CPU usage exceeds 95% continuously (for example, for more than 1 minute), it is considered abnormal;

[0134] 2) Memory abnormality: When the memory usage reaches or exceeds 90%, accompanied by frequent memory overflow and increased swap operations, it is considered abnormal;

[0135] 3) Disk abnormality: When the read and write rates of the storage device fluctuate abnormally, or the disk I / O latency increases significantly (exceeds the preset latency threshold), it is considered an abnormality;

[0136] 4) Network anomaly: When the network delay continuously exceeds the set threshold, the packet loss rate increases significantly, or the bandwidth utilization suddenly reaches an abnormally high level, it is determined to be an anomaly;

[0137] 5) Task scheduling anomaly: When a task is delayed in the task queue for a long time, the task response time is abnormally extended, or the frequency of task scheduling failure increases, it is considered an anomaly;

[0138] 6) Other service anomalies: When the error frequency of key APIs, service connections, or cross-component communications increases sharply or the response time exceeds the standard, an alarm will be issued as an abnormal situation.

[0139] S2. Preprocess the real-time data collected in step S1 to ensure data quality and consistency so that the data meets the input requirements of the PPO algorithm;

[0140] The preprocessing includes: data cleaning, data normalization, data denoising, and data consistency verification.

[0141] S3. Build and train a PPO agent model based on the PPO algorithm. The PPO agent model learns to optimize resource allocation strategies under multi-objective constraints through interaction with the system environment, and adjusts the allocation strategy based on task requirements, load changes, and system status.

[0142] The PPO agent model is a reinforcement learning model, including a policy network and a value network. The multi-objective constraints include load balancing, task response time, and energy efficiency.

[0143] S4. Integrate the PPO agent model trained in step S3 into the resource management layer of the distributed system, receive real-time status feedback from each distributed component, and dynamically allocate system resources according to the strategy generated by the model to ensure optimal performance under different loads;

[0144] S5. Combined with the multi-objective optimization algorithm, that is, optimizing the resource allocation strategy under multi-objective constraints, considering multiple objectives in the resource allocation process, adjusting the weight of each objective based on the PPO agent model to ensure the balance of various objectives and maximize the overall efficiency of the system;

[0145] Among them, the multi-objective optimization algorithms include: NSGA-II and MOEA / D.

[0146] S6. During system operation, through real-time monitoring and performance evaluation, we continuously track the resource allocation effect, provide feedback on system performance, and use the PPO algorithm to incrementally learn and optimize the optimal resource allocation strategy under multi-objective constraints, continuously improving the accuracy and adaptability of the resource allocation strategy;

[0147] S7. Regularly analyze the system's performance, update and expand the data set based on new task requirements and environmental changes, and further train the PPO model to ensure that the resource allocation of the distributed system in complex and dynamic environments remains efficient and stable.

[0148] In this embodiment, step S2 is specifically as follows:

[0149] S21, data cleaning;

[0150] First, the raw real-time monitoring data collected in step S1 is preliminarily screened to remove redundant, abnormal, or missing data points to ensure data integrity and accuracy. Specifically, for data with duplicate records, a hash-based deduplication method is used to prevent duplicate data from affecting the training process. For abnormal data, the Z-score method is used to perform statistical analysis on various monitoring indicators. If the Z-score value of a data point exceeds the set threshold, it is marked as abnormal data and repaired using interpolation based on the trend of the previous and next timestamps. For missing data, if the missing ratio is less than p%, linear interpolation or Lagrange interpolation is used to fill the missing data. If the missing ratio is greater than 3p%, the data sample is directly discarded to ensure the overall quality of the dataset.

[0151] The threshold setting includes: the 3σ principle and the value of p is determined according to the actual situation.

[0152] S22, data normalization;

[0153] Since different monitoring indicators have different numerical ranges and dimensions, the multidimensional heterogeneous data obtained after data cleaning in step S21 is normalized and mapped to the interval [0, 1] to eliminate the dimensional differences between different data dimensions.

[0154] The multi-dimensional heterogeneous data includes: number of CPU cores and memory GB units.

[0155] For numerical data, we use two methods: Min-Max Scaling and Z-score Normalization. Based on the distribution of the monitored data, we automatically select the appropriate normalization method to ensure that the data can better adapt to the input requirements of the PPO algorithm.

[0156] The calculation expression of minimum-maximum normalization is as follows:

[0157]

[0158] Among them, X represents the original real-time monitoring data, X min and X max Represent the minimum and maximum values of the feature respectively, and the normalized data X ′ The value range is [0,1].

[0159] For data that conforms to the normal distribution, zero mean normalization is used, and the calculation expression is as follows:

[0160]

[0161] Among them, μ represents the mean of the data, σ represents the standard deviation of the data, and the mean of the normalized data is 0 and the standard deviation is 1.

[0162] S23, data denoising;

[0163] During data collection, factors such as sensor errors and network jitter may affect the data, resulting in noise. Therefore, it is necessary to use a suitable denoising algorithm to smooth the data and extract representative and stable feature information. This example uses Gaussian filtering and wavelet denoising to smooth instantaneous fluctuations and extract stable resource usage trend features.

[0164] First, Gaussian filtering is used to smooth the data. The calculation expression is as follows:

[0165]

[0166] Where G(i,j) represents the filter weight, σ represents the standard deviation, and x and y represent the pixel position offset. Gaussian filtering can effectively remove random noise while maintaining the overall trend of the data.

[0167] Then, for time series data, the wavelet denoising method is used to remove high-frequency noise through three steps of decomposition, threshold denoising and reconstruction to improve the smoothness of the data.

[0168] First, select the appropriate wavelet basis and decomposition layer number J, perform discrete wavelet transform on the signal, and decompose it into approximate components A in different frequency bands. j and detail component D j , the expression is as follows:

[0169]

[0170] Among them, x[n] represents the discretized data, φ j,k represents the scaling function; ψ j,k represents the wavelet function.

[0171] When denoising in the wavelet domain, only the detail coefficient D j Perform threshold processing and set the detail coefficient obtained by wavelet decomposition of the jth layer as D j ={d j,k :k=1,2,…}, select a threshold λ j , and D j Use hard threshold processing to form the denoised coefficient The expression is as follows:

[0172]

[0173] After completing the threshold processing of the detail coefficients, A is transformed into J and Merge step by step to get the reconstructed signal The reconstruction process expression is as follows:

[0174]

[0175] Among them, A J Preserve the main energy and contour of the signal; Represents high-frequency detail information after noise removal.

[0176] S24, data consistency check;

[0177] Since data comes from multiple different components and sensors, consistency checks are required to ensure the temporal and logical consistency of the data.

[0178] First, a timestamp alignment method is used to check whether the timestamps of each data point are synchronized. If the time deviation of the data exceeds a set threshold (10 ms in this embodiment), time interpolation correction is performed.

[0179] Rule-based validation is then used to check whether the data complies with physical constraints. If logically abnormal data is found, it is marked as abnormal data and processed in combination with data correction strategies.

[0180] The physical constraints include: CPU usage should not exceed 100%, and memory usage should not be higher than total memory.

[0181] like Figure 3 As shown, in this embodiment, the step S3 is specifically as follows:

[0182] S31. Construct a PPO agent model based on the PPO (Proximal Policy Optimization) algorithm;

[0183] The PPO agent model is constructed based on the PPO algorithm and adopts the Actor-Critic architecture, including a policy network and a value network. The network layer adopts the LSTM structure to capture the temporal characteristics of load changes.

[0184] Among them, the policy network Actor is used to generate resource allocation decisions. The input is the current resource status characteristics of the system, and the output is the allocation strategy of each resource. The network structure adopts two fully connected layers, each layer includes 128 neurons, and uses the ReLU activation function; the value network Critic is used to evaluate the value function of the current strategy and guide the learning process of the agent. The structure is consistent with the policy network, but the output is a single state value. During the optimization process, the Adam optimizer is used for gradient update, and the initial learning rate is set to 3×10 -4 , and combined with gradient clipping technology to control the gradient update amplitude to prevent gradient explosion or disappearance.

[0185] S32, define multi-objective constraints;

[0186] During resource allocation, multiple objectives are optimized, including load balancing, task response time, and energy efficiency. The load balancing objective is measured by calculating the standard deviation of resource utilization across each compute node. The task response time objective is evaluated based on the average task completion time, while the energy efficiency objective is defined by the energy consumption per unit compute task. All optimization objectives are integrated into the reward function of the PPO algorithm to guide the agent in learning a reasonable resource allocation strategy. The final reward function R is expressed as follows:

[0187] R=w1·R balance +w2·R response +w3·R energy

[0188] Among them, w1, w2, and w3 represent load balancing R balance , task response time R response and energy efficiency R energy The weights of the goals are dynamically adjusted according to actual needs to ensure a balance between the goals.

[0189] S33, Environmental Modeling and Simulation;

[0190] Before training the agent, a simulation environment was constructed. This distributed system simulation environment was constructed based on real system logs. A Poisson distribution was used to simulate resource state changes, task demand dynamics, system load distribution, random node failure times, and spatiotemporal variability in network latency across components in the distributed system. The simulation environment was constructed using the Gym framework. The state space included CPU utilization, memory usage, and task queue length, while the action space was defined as the allocation ratio of each resource. Task requests in the environment were generated according to a Poisson distribution, and load variations followed a normal distribution. The PPO agent model interacted with the system in the simulation environment, observing the current state, taking actions, and optimizing its own strategy based on a reward function.

[0191] S34, training the PPO agent model;

[0192] The training process uses the PPO algorithm to optimize the policy network and value network parameters θ through multiple rounds of iteration. To improve training stability, the advantage function is used to evaluate the agent's behavior benefits. The calculation expression is as follows:

[0193] A(s,a)=Q(s,a)-V(s)

[0194] Among them, s represents the state, a represents the action, A(s,a) represents the advantage function, Q(s,a) represents the state-action value function, and V(s) represents the state value function.

[0195] In order to prevent instability caused by excessive policy updates, the Clipped SurrogateObjective method is used to limit the policy update range. The expression is as follows:

[0196]

[0197] Among them, L CLIP Represents the objective function in PPO, and the network parameters θ are updated by maximizing the objective function. Denotes the expectation, r(θ) denotes the ratio of the current policy to the old policy, A denotes the advantage function, and ∈ denotes the clipping hyperparameter, which is set to 0.2 to ensure update stability. During training, each update round consists of 2048 time steps, and mini-batch SGD is used for optimization with a batch size of 64.

[0198] During the training of PPO, the reward function is used to determine whether the strategy has converged. After each iteration, the average reward change of the agent model in the environment is recorded. If the reward continues to fluctuate around a certain value after multiple iterations and does not rise significantly, the strategy is considered to have converged and the iteration is stopped.

[0199] S35, performance evaluation and optimization;

[0200] During training, the agent model's performance is regularly evaluated through Pareto frontier analysis to ensure the effectiveness of its strategy. Evaluation metrics include resource allocation efficiency, goal achievement rate, and strategy stability.

[0201] Among them, resource allocation efficiency is measured by calculating the average reward value of the intelligent agent model under different load conditions, the goal achievement rate is calculated by comparing the degree of closeness between the allocation strategy of the intelligent agent model and the theoretical optimal solution, and the strategy stability is judged by measuring the strategy variance of the intelligent agent model under different initial states.

[0202] Based on the evaluation results, Bayesian optimization is used to dynamically adjust the weights of the reward function and optimize the model parameters to further enhance the agent's decision-making ability. After training is complete, the agent model is saved and used in the actual resource allocation process.

[0203] like Figure 4 As shown, in this embodiment, the step S4 is specifically as follows:

[0204] S41, model deployment and interface adaptation;

[0205] The PPO agent model trained in step S3 is encapsulated as a lightweight inference service to support efficient calls in a distributed system environment. During the deployment process, TensorFlow Serving is used to encapsulate the model, optimize its inference efficiency, and convert it into ONNX format so that it can run on different computing architectures. The inference service is encapsulated using a Docker container, with each container occupying no more than 512MB of memory. It is deployed to the sidecar container of the Kubernetes cluster, and the number of Pods is adjusted through horizontal automatic scaling (HPA) to control the response delay within 50ms to ensure inference capabilities under high load conditions. In terms of interface adaptation, the gRPC protocol is used to establish a unified resource management interface (such as REST API or message queue) to receive real-time status feedback data from each distributed component and process 2,000 resource status feedback requests per second. The feedback data includes: CPU occupancy, memory usage, network bandwidth, task queue length, and latency indicators. All input data is converted into a standardized tensor format to match the input requirements of the PPO agent model. The input data format is a l×4 dimensional tensor. Each row of data includes: CPU utilization, memory usage, network bandwidth, and task queue length. All data are normalized to [0, 1].

[0206] Wherein, l represents the data length.

[0207] S42, Strategic Reasoning and Dynamic Decision-making;

[0208] Based on step S41, after the model deployment is completed, the PPO agent model performs policy reasoning based on real-time status data and generates a multi-dimensional resource allocation decision vector, including: the computing resource allocation ratio required by each component (number of CPU cores, memory quota), network bandwidth priority adjustment coefficient, and task migration suggestions (such as scheduling high-latency tasks to low-load nodes). The calculation time of model reasoning does not exceed 20ms to ensure low-latency response. The output of the PPO agent model is a l×4-dimensional resource allocation vector, and each row of data represents the allocated CPU core ratio, memory allocation ratio, network bandwidth allocation ratio, and a binary value of whether the task is migrated.

[0209] The elastic scaling ratio is ±20% of the resource quota, and the target nodes for task migration are the three candidate nodes with the lowest load.

[0210] The decision frequency of the PPO agent model is dynamically adjusted according to the current system load. The inference frequency is increased when the load fluctuates violently, and the frequency is reduced in a stable state. In this embodiment, when the CPU utilization rate changes by more than 15%, the inference frequency is adjusted to 100ms / time. When the system load is in a stable state, that is, the CPU utilization rate changes by less than 2%, the inference frequency is reduced to 500ms / time to reduce system overhead.

[0211] S43, resource scheduling instruction distribution;

[0212] The PPO agent model's inference results are converted into executable resource scheduling instructions and broadcasted via a publish-subscribe model. After receiving the instructions, the node agent calls the local resource manager to perform quota adjustments. In the conversion rules, when the CPU allocation ratio output by the PPO agent model exceeds 0.9, the system triggers Pod Autoscaling (HPA - Horizontal PodAutoscaler), adjusting the CPU limit of the compute instance to 2000m (2 CPU cores). When the task migration value is 1, the system generates a task rescheduling instruction (Re-scheduling Instruction), migrating the current task from the high-load node to a compute node with a CPU utilization below 50%. When the network bandwidth allocation ratio exceeds 0.8, the bandwidth priority (QoSPolicy Update) is adjusted to raise the task's network priority to the highest level (Class 5). All resource scheduling instructions are transmitted via the Kafka distributed message queue, with a maximum throughput of 50,000 messages per second, ensuring efficient instruction transmission.

[0213] S44, real-time distribution effect verification;

[0214] After the resource allocation strategy is executed in step S43, its effectiveness is monitored in real time. A / B testing is used to compare the performance of the new strategy with the baseline strategy. The computing resource monitoring module collects task metrics, including average response time, load balancing, and resource utilization improvement. The average task response time is calculated by averaging the execution time of all tasks using the following expression:

[0215]

[0216] Among them, T avg represents the average response time, N represents the total number of tasks, T i Indicates the execution time of the i-th task.

[0217] The load balancing degree is measured by calculating the standard deviation of resource utilization of each computing node. The calculation expression is as follows:

[0218]

[0219] Among them, σ load Indicates the load balancing degree, M indicates the total number of computing nodes, U j represents the CPU utilization of the jth computing node, U avg Indicates the average CPU utilization of all compute nodes.

[0220] If the indicator does not reach the expected optimization threshold, that is, the load balance degree is less than 10% in this embodiment, the system triggers the online fine-tuning mechanism of the PPO agent model, recalculates the policy gradient based on the incremental data and updates the model parameters. The calculation expression of the policy gradient update is as follows:

[0221]

[0222] Among them, θ represents the current policy parameters, θ′ represents the updated policy parameters, represents the gradient of the policy optimization objective function, and the learning rate is set to 0.0003. The updated PPO model is applied in the next round of inference to optimize the resource allocation strategy.

[0223] If both resource utilization and load balancing meet the requirements, the model parameters are retained and the verification ends.

[0224] S45, fault tolerance and rollback mechanism;

[0225] During the dynamic resource allocation process, abnormal situations such as computing node crashes, task overloads, or resource over-allocation may occur. If the resource utilization and load balancing do not meet the requirements (the effect is not as expected), the system introduces a fault tolerance and rollback mechanism and automatically adjusts the model parameters. When the abnormal event involved in step S1 is detected, the system suspends the execution of the current policy and switches to the backup rule engine to temporarily take over resource allocation, that is, a static resource allocation strategy based on the threshold, which adjusts the CPU limit of all tasks to 1000m (1 CPU core) and reduces the priority of non-currently executed tasks and their associated tasks. When it is detected that the number of task scheduling failures exceeds 3 times, the system automatically rolls back to the previous PPO model version and records the abnormal data, and the abnormal data is stored in the Elasticsearch abnormal log database. The abnormal data is stored in the Elasticsearch abnormal log database, and the log format is as follows:

[0226] {"timestamp":1700000000000,"node_id":"node-05","cpu_usage":98.5,"memory_usage":85.2,"error_type":"CPU Overload","action":"Rollback to previousmodel"}

[0227] Abnormal data is injected into the PPO training set and used to enhance the model's robustness during the next round of training. After each abnormality occurs, the system executes an experience replay mechanism to retrain the PPO model, improving its stability in high-load scenarios.

[0228] In this embodiment, step S5 is specifically as follows:

[0229] In terms of load balancing, an objective function is set to achieve uniform distribution of loads among components in the system to avoid overload or idle resources. That is, the system measures the load balancing situation by calculating the standard deviation of the CPU utilization of each computing node, and adjusts the PPO reward function when the balance degree exceeds 0.15 to increase the weight of the load balancing objective, so that the intelligent agent tends to balance resource allocation.

[0230] In terms of response time, the goal of task response time optimization is to minimize the execution delay of the task. When the average response time of the task exceeds 200ms, the PPO model prioritizes scheduling resources on low-load nodes and dynamically adjusts the task queue priority to reduce task execution delay and ensure the real-time performance of the system.

[0231] In terms of energy efficiency optimization, the goal of minimizing power consumption is introduced. That is, the system measures the optimization effect by calculating the energy consumption ratio of unit tasks. When the energy consumption ratio exceeds 0.5W / ms, the CPU frequency of low-priority tasks is reduced, and task merging technology is used to reduce computing resource usage, reduce system energy consumption, and extend the service life of hardware.

[0232] The reward function of the PPO agent model is dynamically adjusted based on load balancing, task response time, and energy efficiency. The weight of each objective is adaptively adjusted according to the system state during operation to maintain a balance between different objectives, maximize the overall efficiency of the system, and adapt to the performance requirements under different loads. When the deviation of a certain objective exceeds the set threshold, the system automatically increases the weight of the objective to ensure the balance of each optimization objective.

[0233] The resource allocation strategy maintains optimal performance under different loads, enabling the distributed system to achieve a reasonable balance between load balancing, low latency and high energy efficiency.

[0234] like Figure 5As shown, in this embodiment, step S6 is specifically as follows:

[0235] The computing resource monitoring module collects CPU utilization, memory usage, task response time, and task queue length of the computing node every 100ms and records key performance indicators. All monitoring data is transmitted to the distributed data management center via the Kafka message queue and stored in the InfluxDB database. The short-term effectiveness of the resource allocation strategy is evaluated using sliding window statistics. If the key performance indicator falls below 15% of the training indicator, the incremental learning mechanism of the PPO model is triggered. During incremental learning, the system first filters out abnormal data within the past 10 minutes and recalculates the policy gradient based on this data, with a learning rate set to 0.0003. The updated PPO model is immediately deployed to the inference server and applied to the next round of resource allocation decisions. To prevent policy drift caused by incremental learning, the system uses an experience replay mechanism to randomly sample data from the past week for comparative learning to ensure model stability.

[0236] Through real-time monitoring, data feedback, and incremental learning mechanisms, the PPO model can continuously optimize its own strategies when the system environment changes dynamically, improve the accuracy of resource allocation, and adapt to operational requirements under different load conditions.

[0237] In this embodiment, step S7 is specifically as follows:

[0238] Regularly analyze historical performance data, changes in task requirements, and dynamic changes in the system operating environment to identify resource allocation bottlenecks. Expand the training set by injecting simulated load data, and reuse existing model parameters in combination with transfer learning technology to accelerate the convergence speed of the PPO intelligent agent model in new scenarios.

[0239] Among them, identifying resource allocation bottlenecks includes: long-term overload of specific nodes; simulated load data includes: peak traffic and sudden tasks.

[0240] The system conducts a comprehensive performance evaluation every 24 hours. The evaluation indicators include: average task response time, load balancing and energy efficiency optimization. All data are stored in the performance analysis database and aggregated according to the time dimension. When the system detects a performance degradation trend (such as the task response time increases by more than 10% compared to the previous day or the load balancing degree is lower than the set threshold of 0.15), it automatically triggers the retraining of the PPO agent model. During the training process, the system constructs a new training set based on the monitoring data of the past week and performs strategy optimization. Each round of training contains 10,000 time steps and a batch size of 128. The optimization goal is to minimize the task response time and maximize energy efficiency. The strategy update range is controlled between 0.8 and 1.2 to ensure the stability of the strategy adjustment.

[0241] After training is complete, the new model is tested and automatically deployed to the inference server if it meets performance requirements, replacing the old model. To enhance the model's generalization capabilities, the dataset is expanded after each training session. In this example, data from the past 30 days is mixed with current data to prevent the model from overfitting to recent data and optimize the dynamic resource allocation strategy for distributed components.

[0242] Through regular performance analysis, data updates, and retraining of the PPO model, the system can continuously optimize resource allocation strategies to ensure efficient and stable operation in various complex and dynamic environments.

[0243] In summary, the method of the present invention significantly improves the dynamics and global optimization capabilities of distributed system resource allocation by combining the PPO algorithm with multi-objective optimization technology. First, the dynamic strategy generation mechanism based on reinforcement learning can respond to load fluctuations in real time, minimizing task delays and energy consumption while ensuring load balancing; secondly, through lightweight deployment and online incremental learning, the system can continuously optimize resource allocation strategies without interrupting services, overcoming the shortcomings of poor adaptability of existing static rules; in addition, the dynamic adjustment mechanism of multi-objective weights and the fault-tolerant rollback design ensure the high reliability and stability of the system in complex scenarios (such as node failures and sudden traffic). Compared with existing methods, the method of the present invention has significant advantages in allocation efficiency, multi-objective collaborative optimization and long-term operation and maintenance costs, and is particularly suitable for dynamic distributed environments such as cloud computing and edge computing.

[0244] The foregoing description is merely a specific embodiment of the present invention, and the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be readily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A distributed component dynamic resource allocation method based on multi-objective optimization, the specific steps are as follows: S1. Deploy resource monitoring devices in the distributed system to collect status information, resource usage, and task requirements of each component in real time, and store the data in a unified data management center; First, based on the operating environment characteristics and resource requirements of distributed components, high-precision resource monitoring equipment is installed on each computing node in the distributed system. The selected monitoring equipment includes: Computing resource monitoring module, network traffic monitoring module and task scheduling monitoring module; Among them, the computing resource monitoring module collects key indicators in the distributed system in real time, that is, the status information of each component, including: CPU utilization, memory usage, network bandwidth, storage usage, disk read and write rates, and task queue length; the network traffic monitoring module collects high-speed network traffic data in real time; the task scheduling monitoring module monitors the resource usage dynamics of each component and changes in task requirements, including: CPU limits, memory usage, and task running time; All real-time monitoring data is efficiently transmitted via the gRPC protocol and transmitted to the distributed data management center through a unified API interface. A time series database is used for persistent storage. In terms of storage architecture, short-term data is stored in the distributed data management center, while long-term data is stored in the time series database and efficiently compressed using the Zstandard algorithm. The distributed data management center includes Kafka and Redis; the time series database includes InfluxDB; short-term data refers to data within 24 hours, and long-term data refers to data exceeding 24 hours; The distributed system introduces an adaptive data sampling strategy. When the CPU load is low (i.e., <30%), the data sampling interval is adjusted to 500ms; when the load is in the normal range of 30%-70%, the sampling interval is adjusted to 200ms; when the load is high (i.e., >70%), the sampling interval is shortened to 100ms. When the computing resource monitoring module detects an anomaly, the system automatically triggers an alarm mechanism, sends a WebHook notification to the operations team, and combines it with an automatic scaling strategy to dynamically allocate additional resources for computing tasks. All abnormal data will be stored in Elasticsearch. The abnormal situations include: 1) CPU abnormality: When the CPU usage exceeds 95% continuously, it is considered abnormal; 2) Memory abnormality: When the memory usage reaches or exceeds 90%, accompanied by frequent memory overflow and increased swap operations, it is considered abnormal; 3) Disk abnormality: When the read and write rates of the storage device fluctuate abnormally, or the disk I / O latency increases significantly, it is considered an abnormality; 4) Network anomaly: When the network delay continuously exceeds the set threshold, the packet loss rate increases significantly, or the bandwidth utilization suddenly reaches an abnormally high level, it is determined to be an anomaly; 5) Task scheduling anomaly: When a task is delayed in the task queue for a long time, the task response time is abnormally extended, or the frequency of task scheduling failure increases, it is considered an anomaly; 6) Other service anomalies: When the error frequency of key APIs, service connections, or cross-component communications increases sharply or the response time exceeds the standard, an alarm will be issued as an abnormal situation; S2. Preprocess the real-time data collected in step S1 to ensure data quality and consistency so that the data meets the input requirements of the PPO algorithm; The preprocessing includes: data cleaning, data normalization, data denoising, and data consistency verification; S3. Build and train a PPO agent model based on the PPO algorithm. The PPO agent model learns to optimize resource allocation strategies under multi-objective constraints through interaction with the system environment, and adjusts the allocation strategy based on task requirements, load changes, and system status. The PPO agent model is a reinforcement learning model, including a policy network and a value network; the multi-objective constraints include load balancing, task response time, and energy efficiency; S4. Integrate the PPO agent model trained in step S3 into the resource management layer of the distributed system, receive real-time status feedback from each distributed component, and dynamically allocate system resources according to the strategy generated by the model to ensure optimal performance under different loads; S5. Combined with the multi-objective optimization algorithm, that is, optimizing the resource allocation strategy under multi-objective constraints, considering multiple objectives in the resource allocation process, adjusting the weight of each objective based on the PPO agent model to ensure the balance of various objectives and maximize the overall efficiency of the system; The multi-objective optimization algorithms include: NSGA-II, MOEA / D; S6. During system operation, through real-time monitoring and performance evaluation, continuously track resource allocation effects, provide feedback on system performance, and use the PPO algorithm to incrementally learn and optimize the optimal resource allocation strategy under multi-objective constraints; S7. Regularly analyze the system's performance, update and expand the data set based on new task requirements and environmental changes, and further train the PPO model.

2. A distributed component dynamic resource allocation method based on multi-objective optimization according to claim 1, characterized in that: The step S2 is specifically as follows: S21, data cleaning; First, the original real-time monitoring data collected in step S1 is preliminarily screened to remove redundant, abnormal or missing data points. That is, for data with duplicate records, a hash value-based deduplication method is used to process them. For abnormal data, the Z-score method is used to perform statistical analysis on various monitoring indicators. If the Z-score value of a data point exceeds the set threshold, it is marked as abnormal data and repaired using interpolation based on the trend of the previous and next timestamps. For missing data, if the missing ratio is less than p%, linear interpolation or Lagrange interpolation is used to fill it. If the missing ratio is higher than 3p%, the data sample is directly discarded. The threshold setting includes: 3σ principle; p is set according to the actual situation; S22, data normalization; Normalize the multidimensional heterogeneous data obtained after data cleaning in step S21 and map it to the interval [0, 1] to eliminate the dimensional differences between different data dimensions; The multi-dimensional heterogeneous data includes: number of CPU cores, memory GB unit; For numerical data, two methods, minimum-maximum normalization and zero-mean normalization, are used; the appropriate normalization method is automatically selected based on the distribution of the monitoring data; The calculation expression of minimum-maximum normalization is as follows: Among them, X represents the original real-time monitoring data, X min and X max Represent the minimum and maximum values of the feature respectively, and the normalized data X ′ The value range of is [0,1]; For data that conforms to the normal distribution, zero mean normalization is used, and the calculation expression is as follows: Among them, μ represents the mean of the data, σ represents the standard deviation of the data, and the mean of the normalized data is 0 and the standard deviation is 1; S23, data denoising; Gaussian filtering and wavelet denoising methods are used to smooth instantaneous fluctuations and extract stable resource usage trend characteristics; First, Gaussian filtering is used to smooth the data. The calculation expression is as follows: Where G(i,j) represents the weight of the filter, σ represents the standard deviation, and x, y represent the pixel position offset; Then, for time series data, wavelet denoising method is used to remove high-frequency noise through three steps of decomposition, threshold denoising and reconstruction to improve the smoothness of the data; First, select the appropriate wavelet basis and decomposition layer number J, perform discrete wavelet transform on the signal, and decompose it into approximate components A in different frequency bands. j and detail component D j , the expression is as follows: Among them, x[n] represents the discretized data, φ j,k represents the scaling function; ψ j,k represents the wavelet function; When denoising in the wavelet domain, only the detail coefficient D j Perform threshold processing and set the detail coefficient obtained by wavelet decomposition of the jth layer as D j ={d j,k :k=1,2,…}, select a threshold λ j , and D j Use hard threshold processing to form the denoised coefficient The expression is as follows: After completing the threshold processing of the detail coefficients, A is transformed into J and Merge step by step to get the reconstructed signal The reconstruction process expression is as follows: Among them, A J Preserve the main energy and contour of the signal; Represents high-frequency detail information after noise removal; S24, data consistency check; First, the timestamp comparison method is used to check whether the timestamps of each data point are synchronized. If the time deviation of the data exceeds the set threshold, time interpolation correction is performed; Then, logical rule verification is used to check whether the data complies with physical constraints. If any logically abnormal data is found, it is marked as abnormal data and processed in combination with data correction strategies; The physical constraints include: CPU usage should not exceed 100%, and memory usage should not be higher than total memory.

3. The distributed component dynamic resource allocation method based on multi-objective optimization according to claim 1 is characterized in that: The step S3 is specifically as follows: S31. Construct a PPO agent model based on the PPO algorithm; The PPO agent model is built based on the PPO algorithm and adopts the Actor-Critic architecture, including a policy network and a value network. The network layer adopts the LSTM structure to capture the temporal characteristics of load changes. Among them, the policy network Actor is used to generate resource allocation decisions. The input is the current resource status characteristics of the system, and the output is the allocation strategy of each resource. The network structure adopts two fully connected layers, each layer includes 128 neurons, and uses the ReLU activation function; the value network Critic is used to evaluate the value function of the current strategy and guide the learning process of the agent. The structure is consistent with the policy network, but the output is a single state value. During the optimization process, the Adam optimizer is used for gradient update, and the initial learning rate is set to 3×10 -4 , and combined with gradient clipping technology to control the gradient update amplitude; S32, define multi-objective constraints; During the resource allocation process, multiple objectives are optimized, including load balancing, task response time, and energy efficiency. All optimization objectives are integrated into the reward function of the PPO algorithm. The final reward function R is expressed as follows: R=w1·R balance +w2·R response +w3·R energy Among them, w1, w2, and w3 represent load balancing R balance , task response time R response and energy efficiency R energy The weight of the goal is dynamically adjusted according to actual needs; S33, Environmental Modeling and Simulation; A distributed system simulation environment is constructed based on real system logs. The Poisson distribution is used to simulate resource state changes, task demand dynamics, system load distribution, random node failure times, and spatiotemporal variability in network latency across components in the distributed system. The simulation environment is constructed using the Gym framework. The state space includes CPU utilization, memory occupancy, and task queue length, while the action space is defined as the allocation ratio of each resource. Task requests in the environment are generated according to a Poisson distribution, and load changes follow a normal distribution. The PPO agent model interacts with the system in the simulation environment, observing the current state, taking actions, and optimizing its own strategy based on a reward function. S34, training the PPO agent model; The training process uses the PPO algorithm to optimize the policy network and value network parameters θ through multiple rounds of iterations. The advantage function is used to evaluate the behavior benefits of the agent. The calculation expression is as follows: A(s,a)=Q(s,a)-V(s) Where s represents the state, a represents the action, A(s,a) represents the advantage function, Q(s,a) represents the state-action value function, and V(s) represents the state value function; Then the policy cutting method is used to limit the policy update range. The expression is as follows: Among them, L CLIP Represents the objective function in PPO, and the network parameters θ are updated by maximizing the objective function. represents the expectation, r(θ) represents the ratio of the current strategy to the old strategy, A represents the advantage function, and ∈ represents the clipping hyperparameter. During the training process, Mini-batch SGD is used for optimization, and the time step and batch size included in each update round are set according to the actual situation. During the training of PPO, the reward function is used to determine whether the strategy has converged. After each iteration, the average reward change of the agent model in the environment is recorded. If the reward fluctuates steadily around a certain value after multiple iterations and does not rise significantly, the strategy is considered to have converged and the iteration is stopped. S35, performance evaluation and optimization; During training, the performance of the agent model is regularly evaluated using Pareto frontier analysis. Evaluation metrics include resource allocation efficiency, goal achievement rate, and strategy stability. The resource allocation efficiency is measured by calculating the average reward value of the agent model under different load conditions. The goal achievement rate is calculated by comparing the closeness of the agent model's allocation strategy to the theoretical optimal solution. The strategy stability is determined by measuring the strategy variance of the agent model under different initial states. Based on the evaluation results, Bayesian optimization is used to dynamically adjust the weight of the reward function and optimize the model parameters to further enhance the decision-making ability of the agent. After training is completed, the agent model will be saved and used in the actual resource allocation process.

4. The distributed component dynamic resource allocation method based on multi-objective optimization according to claim 1 is characterized in that: The step S4 is specifically as follows: S41, model deployment and interface adaptation; The PPO agent model trained in step S3 is encapsulated as a lightweight inference service. During the deployment process, TensorFlow Serving is used to encapsulate the model, optimize its inference efficiency, and convert it into ONNX format. The inference service is encapsulated in a Docker container, with each container occupying no more than 80% of the memory. It is deployed to the sidecar container of the Kubernetes cluster, and the number of Pods is adjusted through horizontal automatic expansion to control the response delay within a reasonable range. In terms of interface adaptation, the gRPC protocol is used to establish a unified resource management interface to receive real-time status feedback data from each distributed component and process several resource status feedback requests per second. The feedback data includes: CPU occupancy, memory usage, network bandwidth, task queue length, and latency indicators. All input data is converted into a standardized tensor format to match the input requirements of the PPO agent model. The input data format is a l×4 dimensional tensor. Each row of data includes: CPU utilization, memory occupancy, network bandwidth, and task queue length. All data is normalized to [0,1]. Wherein, l represents the data length; S42, Strategic Reasoning and Dynamic Decision-making; Based on step S41, after the model deployment is completed, the PPO agent model performs policy reasoning based on real-time status data and generates a multi-dimensional resource allocation decision vector, including: the computing resource allocation ratio required by each component, the network bandwidth priority adjustment coefficient, and the task migration recommendation; the computational time of the model reasoning does not exceed 20ms, and the output of the PPO agent model is a l×4-dimensional resource allocation vector, in which each row of data represents the allocated CPU core ratio, memory allocation ratio, network bandwidth allocation ratio, and a binary value indicating whether the task should be migrated; The elastic scaling ratio is ±20% of the resource quota, and the target nodes for task migration are the three candidate nodes with the lowest load. The decision frequency of the PPO agent model is dynamically adjusted according to the current system load, increasing the inference frequency when the load fluctuates violently and reducing the frequency in a stable state; S43, resource scheduling instruction distribution; The PPO agent model inference results are converted into executable resource scheduling instructions, and the instructions are broadcast through the publish-subscribe model. After receiving the instructions, the node agent calls the local resource manager to perform quota adjustment. In the conversion rules, when the CPU allocation ratio output by the PPO agent model exceeds 0.9, the system triggers Pod automatic expansion to adjust the CPU limit. When the task migration value is 1, the system generates a task rescheduling instruction to migrate the current task from the high-load node to a computing node with a CPU utilization rate of less than 50%. When the network bandwidth allocation ratio exceeds 0.8, the bandwidth priority is adjusted to increase the network priority of the task to the highest level. All resource scheduling instructions are transmitted through a distributed message queue, and the maximum throughput of the message queue is 50,000 messages per second. S44, real-time distribution effect verification; Based on step S43, after the resource allocation strategy is executed, its effect is monitored in real time, and the performance difference between the new strategy and the baseline strategy is compared using an A / B testing mechanism. The computing resource monitoring module collects task indicators, including: average response time, load balancing, and resource utilization improvement. The average task response time is calculated by calculating the average execution time of all tasks, and the calculation expression is as follows: Among them, T avg represents the average response time, N represents the total number of tasks, T i represents the execution time of the i-th task; The load balancing degree is measured by calculating the standard deviation of resource utilization of each computing node. The calculation expression is as follows: Among them, σ load Indicates the load balancing degree, M indicates the total number of computing nodes, U j represents the CPU utilization of the jth computing node, U avg Indicates the average CPU utilization of all computing nodes; If the indicator does not reach the expected optimization threshold, that is, the load balance is lower than 10%, the system triggers the online fine-tuning mechanism of the PPO agent model, recalculates the policy gradient based on the incremental data, and updates the model parameters; the calculation expression of the policy gradient update is as follows: Among them, θ represents the current policy parameters, θ′ represents the updated policy parameters, represents the gradient of the policy optimization objective function, and the learning rate is set to 0.0003; If both resource utilization and load balancing meet the requirements, the model parameters are retained and the verification ends; S45, fault tolerance and rollback mechanism; If resource utilization and load balancing do not meet the requirements, the system introduces a fault-tolerance and rollback mechanism and automatically adjusts the model parameters. When an abnormal event involved in step S1 is detected, the system suspends the execution of the current policy and switches to a threshold-based static resource allocation policy, adjusts the CPU limit, and reduces the priority of non-currently executed tasks and their associated tasks. When the number of task scheduling failures exceeds three, the system automatically rolls back to the previous PPO model version and records the abnormal data, which is then stored in the Elasticsearch abnormality log database. Among them, abnormal data is injected into the PPO training set and used to enhance the robustness of the model in the next round of training; and after each abnormality occurs, the system executes the experience replay mechanism to retrain the PPO model.

5. The method for dynamic resource allocation of distributed components based on multi-objective optimization according to claim 1, characterized in that: The step S5 is specifically as follows: In terms of load balancing, the objective function is set to achieve uniform distribution of loads among components in the system. That is, the system measures the load balancing situation by calculating the standard deviation of the CPU utilization of each computing node, and adjusts the PPO reward function when the balance degree exceeds 0.15 to increase the weight of the load balancing objective. In terms of response time, the goal of task response time optimization is to minimize the execution delay of tasks. When the average response time of a task exceeds 200ms, the PPO model prioritizes resources on low-load nodes and dynamically adjusts the task queue priority. In terms of energy efficiency optimization, the goal of minimizing power consumption is introduced. That is, the system measures the optimization effect by calculating the energy consumption ratio of each unit task. When the energy consumption ratio exceeds 0.5W / ms, the CPU frequency of low-priority tasks is reduced, and task merging technology is used to reduce computing resource usage. The reward function of the PPO agent model is dynamically adjusted based on load balancing, task response time, and energy efficiency. The weight of each target is adaptively adjusted according to the system state during operation. When the deviation of a certain target exceeds the set threshold, the system automatically increases the weight of the target.

6. The method for dynamic resource allocation of distributed components based on multi-objective optimization according to claim 1, characterized in that: The step S6 is specifically as follows: The computing resource monitoring module collects CPU utilization, memory usage, task response time, and task queue length of computing nodes every 100ms and records key performance indicators. All monitoring data is transmitted to the distributed data management center via message queues and stored in a database. The short-term effectiveness of resource allocation strategies is evaluated using sliding window statistics. If the key performance indicator falls below 15% of the training indicator, the incremental learning mechanism of the PPO model is triggered. During the incremental learning process, the system first filters out abnormal data within the past 10 minutes and recalculates the policy gradient based on this data, with the learning rate set to 0.0003; the updated PPO model is immediately deployed to the inference server and applied to the next round of resource allocation decisions. The system also uses an experience replay mechanism to randomly extract data from the past period for comparative learning based on actual conditions.

7. The method for dynamic resource allocation of distributed components based on multi-objective optimization according to claim 1, characterized in that: The step S7 is specifically as follows: Regularly analyze historical performance data, changes in task requirements, and dynamic changes in the system operating environment to identify resource allocation bottlenecks. Expand the training set by injecting simulated load data and reuse existing model parameters with transfer learning technology to accelerate the convergence of the PPO agent model in new scenarios. Among them, identifying resource allocation bottlenecks includes: long-term overload of specific nodes; simulated load data includes: peak traffic, sudden tasks; The system conducts a comprehensive performance evaluation every 24 hours. Evaluation metrics include average task response time, load balancing, and energy efficiency optimization. All data is stored in a performance analysis database and aggregated over time. When the system detects a performance degradation trend, it automatically triggers retraining of the PPO agent model. During training, the system constructs a new training set based on the most recent week's monitoring data and performs policy optimization. The time steps and batch size included in each training round are set according to actual conditions. The optimization goal is to minimize task response time and maximize energy efficiency. The policy update range is controlled between 0.8 and 1.

2. After training is completed, the new model is tested and automatically deployed to the inference server after meeting performance requirements, replacing the old model. The data set is expanded after each training session, and data from the past period is mixed with current data based on actual conditions to optimize the dynamic resource allocation strategy of distributed components.

Images