Server load state evaluation method based on dynamic evaluation algorithm

Abstract

The invention relates to the technical field of server load management and dynamic resource scheduling, and discloses a server load state evaluation method based on a dynamic evaluation algorithm, which comprises the following steps: data acquisition: acquiring performance index data, log data and network flow data from a multi-modal data source of a server; data preprocessing: performing time alignment, data cleaning, feature extraction and normalization processing on the acquired data; and self-supervised learning modeling: extracting feature representation from the multi-modal data through a self-supervised pre-training task, wherein the self-supervised pre-training task comprises a time sequence covering prediction task. Through multi-modal data fusion and self-supervised learning, the dynamic characteristics of the server load are accurately extracted, real-time load classification, anomaly detection and trend prediction are realized, the evaluation precision and real-time performance are significantly improved, and at the same time, a resource scheduling strategy of rule driving and reinforcement learning is combined, the server resource allocation efficiency is optimized, and the overload risk is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of server load management and dynamic resource scheduling, and in particular to a server load state evaluation method based on a dynamic evaluation algorithm. Background Art

[0002] With the rapid development of cloud computing, edge computing, and distributed systems, server load status evaluation has gradually become one of the important technologies to ensure system stability and efficiency. Existing load status evaluation methods mainly rely on single-modal data analysis, such as evaluation based only on performance indicators (CPU utilization, memory occupancy, etc.) or log data. These methods use fixed rules or simple statistical analysis techniques to meet the basic needs in static load scenarios. However, in complex dynamic load environments, the server load status is often affected by the comprehensive influence of multimodal data, and the limitations of existing technologies are gradually emerging.

[0003] Specifically, the existing technology has the following problems: first, the ability to integrate multimodal data is insufficient. Traditional methods cannot effectively integrate multimodal information such as performance indicators, log data, and network traffic, resulting in the evaluation results being unable to fully reflect the actual operating status of the server; second, the real-time and prediction capabilities are limited. Due to the lack of modeling of dynamic load change characteristics, existing methods often lag behind actual load changes and make it difficult to perceive potential risks in a timely manner; third, resource scheduling is not smart enough. Existing technologies rely on preset static rules to adjust resource allocation and lack an adaptive optimization mechanism, which can easily lead to resource waste or overload risks, thereby reducing the overall efficiency and stability of the system. These problems seriously restrict the application effect of existing server load assessment and resource management technologies in complex dynamic environments. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a server load status evaluation method based on a dynamic evaluation algorithm, which solves the problems in the prior art that multimodal data cannot be efficiently integrated, load status evaluation lacks real-time performance, and resource scheduling lacks an intelligent optimization mechanism.

[0005] To achieve the above objectives, the present invention is implemented by the following technical scheme: a server load status evaluation method based on a dynamic evaluation algorithm comprises the following steps: Data collection: Collect performance indicator data, log data, and network traffic data from multimodal data sources on the server; Data preprocessing: time alignment, data cleaning, feature extraction and normalization of the collected data; Self-supervised learning modeling: extracting feature representations from multimodal data through self-supervised pre-training tasks, including time series mask prediction tasks, multimodal contrastive learning tasks, and contextual relationship prediction tasks; Dynamic load state assessment: Based on the feature representation, load state classification, abnormal load detection and load trend prediction are performed; Intelligent resource scheduling: Dynamically adjust server resource allocation strategies based on evaluation results.

[0006] Preferably, the performance indicator data includes but is not limited to CPU utilization, memory occupancy, disk I / O utilization and network bandwidth.

[0007] Preferably, the data preprocessing includes: Align the time steps of different modal data by timestamps to form a joint time series; Use interpolation to fill missing values ​​and remove outliers in performance indicator data; Analyze key event fields in log data and convert logs into feature vectors using natural language processing technology; Dimensionality reduction of network traffic data using principal component analysis.

[0008] Preferably, the self-supervised pre-training task specifically includes: Time series masking prediction task: by randomly masking some time step data in the time series, the model is required to predict the value of the masked data; Multimodal contrastive learning task: construct positive and negative sample pairs, where the positive sample is a combination of multimodal data at the same time step, and the negative sample is a combination of data at different time steps; Contextual relationship prediction task: Use the model to predict the correlation between performance indicators and log data, as well as the causal relationship between performance indicators.

[0009] Preferably, the time series cover prediction task is optimized by the following formula: , in, is the actual value, is the predicted value, and mask is the set of masked positions.

[0010] Preferably, the multimodal contrastive learning task is optimized by the following contrastive loss function: , in, Represents the similarity between feature vectors.

[0011] Preferably, the dynamic load state assessment includes: Load status classification: classify the load status into low load, medium load, high load and overload; Anomaly detection: Detect abnormal load status based on high-dimensional feature representations based on isolation forest or density-based clustering algorithms; Trend prediction: Use LSTM or Transformer-based time series models to predict load change trends in future time steps.

[0012] Preferably, the classifier used for load state classification is a fully connected neural network or a logistic regression classifier, and the input of the classifier is a high-dimensional feature representation generated by a self-supervised learning model. .

[0013] Preferably, the abnormal load state detection is performed by calculating a high-dimensional feature representation Cluster center with normal state The degree of deviation is realized, and the detection formula is: , in, represents the Euclidean distance between the feature representation and the cluster center, is the standard deviation of the normal state.

[0014] Preferably, the intelligent resource scheduling is implemented by a reinforcement learning algorithm, specifically including: Status: current load status of the server; Action: Adjust resource allocation; Reward function: Combine resource utilization and load balancing to optimize the reward strategy and improve the overall performance of the system.

[0015] The present invention provides a server load status evaluation method based on a dynamic evaluation algorithm. It has the following beneficial effects: 1. The present invention collects and processes multimodal data such as performance indicators, log data, and network traffic, and uses a self-supervised learning algorithm to achieve time synchronization and feature fusion of multimodal data. In particular, by introducing time series modeling and inter-modal correlation modeling, the dynamic change characteristics of the server's operating status are effectively captured. Compared with traditional single-modal methods, the present invention significantly improves the accuracy and robustness of server load status assessment, especially in complex and variable load scenarios.

[0016] 2. The present invention not only supports accurate classification and anomaly detection of the current load status of the server, but also perceives future load changes in advance through the trend prediction module, significantly improving the prediction ability of the system. Thanks to the model's ability to model short-term and long-term dependencies, the present invention can identify potential overload risks in advance and provide a reliable basis for resource scheduling, thereby effectively avoiding performance bottlenecks caused by delayed resource allocation.

[0017] 3. The present invention combines load classification results and prediction information, and realizes intelligent dynamic scheduling of server resources through the dual strategies of rule-driven and reinforcement learning. This scheduling mechanism can not only quickly adapt to load changes, but also continuously improve resource allocation efficiency through adaptive optimization. Compared with fixed resource allocation schemes, the present invention significantly reduces resource waste under variable load scenarios, while enhancing the stability of the system and task processing performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a diagram of the method steps of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the specification of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Please see attached Figure 1 The embodiment of the present invention provides a server load status evaluation method based on a dynamic evaluation algorithm, comprising: Specific implementation of step S1 In this embodiment, step S1 involves collecting multimodal data generated during the operation of the server to support subsequent load status evaluation. The focus of this step is to obtain the performance indicator data, log data and network traffic data of the server in real time and comprehensively, and ensure the time synchronization and collection integrity of the data.

[0021] As an option, this embodiment uses a variety of tools to collect multimodal data from the server, including performance monitoring tools (such as Prometheus), log management tools (such as ELK Stack), and traffic analysis tools (such as Wireshark). It should be noted that the tools used can be selected according to actual needs, but they should all meet the requirements of high-frequency collection and multimodal data support.

[0022] Specifically, performance indicator data is one of the core collection contents of this step. Performance indicator data includes but is not limited to the following: CPU utilization: collect the core utilization of the server CPU and record its occupancy percentage; Memory usage: includes the usage of physical memory and virtual memory, and records the total usage percentage, available memory, and cache memory; Disk I / O usage: records the rate of disk read and write operations (such as MB / s), as well as the average latency of each operation; Network bandwidth: monitor the server's inbound and outbound traffic and record the real-time bandwidth usage ratio.

[0023] In one possible implementation, performance indicator data is sampled once per second and stored in real time through a time series database (such as Prometheus's TSDB) to form continuous time series samples to support subsequent feature analysis and modeling.

[0024] Log data is another important collection content, mainly including the following types: System event log: records key events related to server status changes, such as task start, task completion, or abnormal termination; Error log: including error information such as application crash, memory overflow, network connection interruption, etc. Operation status log: records the status change information generated during the server operation, including resource scheduling logs, thread creation information, etc.

[0025] In this embodiment, the collection of log data is implemented through ELK Stack, which specifically includes the parsing and standardization of logs by Logstash, the structured storage of logs by Elasticsearch, and the visualization support of logs by Kibana. It should be noted that log data is usually unstructured text data, and basic formatting processing is required during the collection process to facilitate subsequent feature extraction.

[0026] The collection of network traffic data in this embodiment mainly includes the following key contents: Bandwidth usage: records the inbound and outbound data traffic of the server and records the changes of real-time traffic in the form of time series; Packet loss rate: collects the proportion of data packets lost during data transmission, reflecting the stability of network communication; Network delay: Monitors the communication delay of each data packet in the network and provides a dynamic representation of network performance.

[0027] For example, the collection of network traffic data can be achieved through tools such as Wireshark, and the specific process includes steps such as packet capture and analysis of network card traffic, data decoding, and statistical calculation. It is understandable that traffic collection needs to be synchronized with performance indicators and log data to ensure the consistency of time series.

[0028] In a possible implementation, in order to achieve synchronous collection of multimodal data, a unified timestamp is added to all data in this embodiment. Specifically, the timestamp is recorded at the millisecond level, and each piece of data corresponds to a unique time point to ensure time alignment during subsequent data fusion.

[0029] In the present invention, the timestamp of the collected data is not only used for time alignment, but also for real-time evaluation. Specifically, by analyzing the time interval of data collection, it is possible to further verify whether the collection frequency meets the preset requirements, thereby improving the reliability of the data.

[0030] As an extended technical content, in some embodiments, specific dimensions for collection may be added for performance indicators, log data, and network traffic data: Expanded dimensions of performance indicators: Based on this embodiment, additional key indicators such as the number of threads, context switching rate, and hardware temperature may be collected to further enrich the dynamic performance of the load state.

[0031] Semantic analysis of log data: For the collected log information, natural language processing (NLP) technology can be combined to further extract implicit semantic features in the log, such as the causal relationship between abnormal events.

[0032] Data enhancement of network traffic: In network traffic collection, in addition to bandwidth, packet loss rate and latency, traffic protocol distribution and number of connection sessions can also be collected to support more fine-grained traffic behavior analysis.

[0033] It should be noted that the present invention gives priority to the integrity and synchronization of data in the data collection step to ensure that the collected multimodal data can be directly processed as a time series input model in the subsequent steps. In addition, a distributed database is used in this embodiment to store the collected data to meet the real-time storage and query requirements of massive data. Specifically, data storage optimizes storage efficiency through a partitioning mechanism and uses column storage to improve query performance.

[0034] The design of this embodiment fully discloses the specific implementation of server multimodal data collection, especially providing clear technical details in terms of performance indicators, log and network traffic data collection dimensions, collection tools, data synchronization, etc. In this way, the present invention lays a solid data foundation for the subsequent evaluation of server load status.

[0035] Specific implementation of step S2 In this embodiment, step S2 involves preprocessing the multimodal data collected by the server to improve data quality, enhance data consistency, and provide efficient and reliable data input for subsequent self-supervised learning modeling and dynamic load assessment. This step includes time alignment, data cleaning, feature extraction and dimensionality reduction, and data normalization, and each link is optimized and designed in combination with the technical features of the present invention.

[0036] As an option, when the multimodal data is time-aligned in the present invention, the time synchronization of different modal data is ensured by a unified timestamp mechanism. Specifically, during the collection process, the performance indicator data, log data, and network flow data are all marked with millisecond timestamps to mark their generation time. In a possible implementation, the log data and network flow data are aligned to the time step of the performance indicator data (such as 1 second) by interpolating the low-frequency data. The interpolation method can be linear interpolation or spline interpolation, and the specific choice depends on the changing trend of the time series data.

[0037] It should be noted that the time alignment operation not only solves the problem of different sampling frequencies of multimodal data, but also lays the foundation for subsequent multimodal feature fusion. In some embodiments, in order to further improve the accuracy of time alignment, a correction mechanism for clock drift can be introduced, especially in a distributed system, where the time deviation of each node may cause data synchronization problems.

[0038] The present invention has carried out detailed optimization design for data cleaning. Specifically, for performance indicator data, the cleaning steps include: Eliminate outliers, such as data with CPU utilization exceeding 100%; Use interpolation to fill missing values. The interpolation algorithm can choose linear interpolation or mean filling according to the data distribution characteristics.

[0039] For log data, the cleaning steps include: Parse the JSON or text format of the logs and extract key fields related to server load, such as task launch, warning events, system recovery, etc. Filter irrelevant information, such as debug logs or redundant information fields, to ensure that only log content related to the server status is retained.

[0040] For network traffic data, the cleaning steps include: Smoothing high-frequency noise, such as calculating a moving average of traffic through a sliding window; Abnormal peaks caused by short-term burst traffic are eliminated to avoid misleading the subsequent modeling process.

[0041] It should be noted that the goal of data cleaning is to remove noise and irrelevant information in the data to the maximum extent possible while retaining the core features that are highly relevant to the server load status.

[0042] In the feature extraction and dimensionality reduction stage, the present invention designs a targeted processing method based on the characteristics of multimodal data. Specifically, for log data, natural language processing (NLP) technology is used to convert unstructured text data into numerical features. Exemplarily, Word2Vec or TF-IDF methods can be used to generate dense embedding vectors to capture the semantic information of log content. In some embodiments, for specific server log templates, template matching technology can also be used to further optimize the log feature extraction process.

[0043] For network traffic data, principal component analysis (PCA) is used to reduce the dimensionality of high-dimensional traffic features, thereby extracting key features that best reflect the network status. For example, for original traffic data (such as bandwidth, delay, packet loss rate, etc.), PCA can project it into a lower-dimensional space while retaining 95% of the data variance.

[0044] It should be noted that the goal of feature extraction and dimensionality reduction is not only to reduce the data dimension, but also to retain key information that is decisive for the load state. In order to verify the effectiveness of the extracted features, the present embodiment performs a correlation analysis on the features after dimensionality reduction to ensure that the correlation between the features is effectively weakened.

[0045] In the data normalization stage, the present embodiment adopts the Min-Max normalization method to uniformly map the performance indicator data, log data, and network traffic data to the [0,1] interval. Specifically, the normalization formula is: , in, Represents the original data, and Represents the minimum and maximum values ​​of the data respectively.

[0046] As an option, the normalization operation can adjust the normalization range based on the characteristics of different modal data. For example, for network delay data, a normalization range of [0,0.5] can be selected to highlight the impact of high delay on server status. It should be noted that the purpose of normalization is to eliminate the differences in the numerical range of different modal data to ensure that the data has equal importance in subsequent modeling.

[0047] It is understandable that the data preprocessing stage is not only to clean and optimize the original data, but also to strictly control the data quality. In some embodiments, the present invention can also add data enhancement technology in the preprocessing stage, for example: Smoothing performance indicator data to enhance the stability of short-term load fluctuations; Generate synthetic samples for log data and increase the sample size of low-frequency event logs through data expansion; Sharding technology is used for network traffic data to split long time series into multiple short sequences to improve the real-time performance of subsequent modeling.

[0048] The preprocessing steps of the present invention are specially designed for the characteristics of multimodal data of the server, and comprehensively cover the key links of time alignment, data cleaning, feature extraction and dimensionality reduction, and data normalization. In this way, data quality and consistency are ensured, providing efficient data input for subsequent self-supervised learning modeling. At the same time, through the optimized design of feature extraction and dimensionality reduction, the accuracy and robustness of load state assessment are significantly improved.

[0049] Specific implementation of step S3 In this embodiment, step S3 involves extracting features from multimodal data using a self-supervised learning model to generate a deep feature representation of the dynamic server load state. By designing reasonable pre-training tasks and efficient model structures, it is ensured that features related to the load state can be extracted from multimodal data such as performance indicators, log data, and network traffic.

[0050] As an option, this embodiment designs a multi-task joint self-supervised learning architecture to make full use of the time series characteristics of multimodal data and the correlation between modalities. Specifically, the model consists of an input layer, a feature extraction layer, and an output layer. The functions and details of each layer are designed as follows: In the input layer, the model receives preprocessed multimodal data in a time window in the following format: , in, Represents a time window T Performance indicator time series within, is the log feature sequence in the corresponding time window, is the network traffic feature sequence. It should be noted that the time window T It is usually set to 10 seconds or longer to capture short-term load fluctuations and medium-term dependencies.

[0051] In the feature extraction layer, the present invention combines the methods of time series modeling and inter-modal feature interaction, and uses the Transformer module and graph neural network (GNN) for feature extraction.

[0052] Specifically, the Transformer module is used in time series modeling to capture the global and local dependencies of the time series through a multi-head self-attention mechanism. For example, the input data is encoded by multiple layers of Transformer to generate intermediate feature representations: , in Represents the feature representation after time series modeling.

[0053] In one possible implementation, in order to capture the dynamic correlation between performance indicators, log data, and network traffic data, the present invention further uses a graph neural network (GNN) to establish an interactive relationship diagram between multiple modalities. Specifically, a multimodal graph is constructed in which a node represents the characteristics of each modality, and the node characteristics are updated through a message passing mechanism: , in It is a joint representation of multimodal features for subsequent tasks.

[0054] It should be noted that the Transformer module focuses on time series modeling, while GNN focuses on interaction modeling between modes. The combination of the two can simultaneously capture the temporal dynamics of the load state and the complex dependencies between modes.

[0055] In order to train the self-supervised learning model, the present invention designs a variety of pre-training tasks, including time series cover prediction tasks, multimodal contrastive learning tasks and contextual relationship prediction tasks.

[0056] Specifically, in the time series masking prediction task, by randomly masking some time steps of data, the model is required to predict the masked data values, thereby learning the short-term changes and long-term trends of the time series. As an option, the loss function of this task is defined as: , in, represents the predicted value, is the true value, and mask is the set of masked time steps.

[0057] In the multimodal contrastive learning task, this embodiment constructs positive and negative sample pairs to learn the correlation between multimodal features. Specifically, the positive sample is a combination of multimodal features at the same time step, and the negative sample is a random feature combination at different time steps. In order to optimize the similarity between modalities, this task uses the loss function of contrastive learning: , in, represents cosine similarity, and N is the number of sample pairs.

[0058] In the contextual relationship prediction task, the present invention uses the model to predict the causal relationship between performance indicators and the impact of logs on performance indicators. For example, the model predicts whether certain performance indicators (such as CPU utilization) exceed a certain threshold and whether they are triggered by specific log events through classification or regression. It should be noted that this task further strengthens the model's understanding of the internal dependencies of the load state.

[0059] The present invention forms a multi-task joint optimization goal by jointly optimizing the loss functions of the above three pre-training tasks: , in , and are hyperparameters used to control the weight of each task in the overall optimization. In one possible implementation, these hyperparameters can be tuned through grid search or random search methods to obtain the best model performance.

[0060] In the output layer, the model generates a high-dimensional feature representation ,This feature representation integrates the interactive information of time series features and multimodal features, and has high generalization ability, and can be directly used in subsequent load state evaluation tasks.

[0061] In some embodiments, in order to further improve the training efficiency and feature quality of the model, the following extended technical contents may be introduced: Add dynamic masking ratio to time series masking prediction tasks to adapt to time series data with different load fluctuation ranges; Introducing more negative sample pairs in the multimodal contrastive learning task to enhance the model’s robustness to noise; Causal dependency modeling based on graph convolutional network (GCN) is added to the contextual relationship prediction task to further explore the complex relationship between log events and performance indicators.

[0062] The self-supervised learning modeling step of this embodiment focuses on extracting deep features related to server load status from multimodal data, and realizes joint modeling of temporal dynamics and inter-modal dependencies through reasonable task design and model architecture. These designs provide comprehensive and efficient feature support for subsequent load status evaluation, and significantly improve the applicability and robustness of the model in complex dynamic scenarios.

[0063] Specific implementation of step S4 In this embodiment, step S4 involves evaluating the dynamic load state of the server based on the high-dimensional feature representation extracted by the aforementioned self-supervised learning model. This step includes load state classification, anomaly detection, and trend prediction, aiming to comprehensively analyze the current and future load changes of the server and provide a reliable basis for intelligent resource scheduling.

[0064] As an option, this embodiment divides the load status into four categories: low load, medium load, high load and overload. The classification of load status is based on the performance indicators, log characteristics and network traffic characteristics of the server. Specifically, the present invention uses a lightweight classifier to represent the high-dimensional features generated by self-supervised learning. The classifier can choose a logistic regression model, a support vector machine, or a fully connected neural network, where the neural network can further optimize the classification performance.

[0065] In one possible implementation, the input features is the time window T Aggregated multimodal features, the classifier is supervised trained according to the labels of different load categories. It should be noted that the goal of load status classification is to clarify the degree of usage of server resources in order to distinguish between normal operation and abnormal operation status.

[0066] In the anomaly detection phase, this embodiment uses unsupervised learning methods to identify abnormal patterns in load data. Specifically, the high-dimensional feature representation is clustered and analyzed by using an isolation forest or density-based clustering algorithm (such as DBSCAN) to determine whether the load state deviates from the normal range.

[0067] As an option, the Isolation Forest Model scores the degree of abnormality of each feature by constructing multiple decision trees and outputs the detection results in the form of anomaly scores. The setting of anomaly scores can be based on specific thresholds, for example, samples below a certain score range are considered normal load status, and samples above the threshold are marked as abnormal.

[0068] Specifically, short-term fluctuations in network traffic, sudden increases in CPU usage, frequent error warnings in log events, etc. may all be marked as abnormal patterns. In some embodiments, the root cause of the abnormality can be further confirmed by combining the correlation between performance indicator features and log events, thereby providing clues for the diagnosis of server abnormal events.

[0069] In this embodiment, the future load changes of the server are also analyzed by trend prediction. As an option, the trend prediction task is based on historical features. , use the time series model to predict the feature values ​​of future time windows.

[0070] Specifically, the prediction model can use the long short-term memory network (LSTM) or the Transformer model to predict short-term load fluctuations and medium- and long-term load trends, respectively. For example, the LSTM model can capture the continuous change characteristics of CPU usage and predict the peak usage at a specific time point in the future; while the Transformer model uses a global attention mechanism to analyze the dependency between log events and traffic data, thereby predicting future network bandwidth requirements.

[0071] The goal of trend prediction of the present invention is to perceive potential load problems in advance, such as predicting future peak loads, so as to provide input basis for intelligent scheduling. In some embodiments, the prediction results can also be used to evaluate the rate of change of load trends, such as a rapidly rising load may indicate that the system needs to expand resources immediately.

[0072] In a possible implementation, in order to further optimize the evaluation performance, this embodiment also introduces a dynamic correction mechanism for model performance. Specifically, the evaluation results of the model are corrected in real time based on the latest collected data to ensure the robustness and real-time performance of classification, detection and prediction tasks.

[0073] The load state assessment in this embodiment not only focuses on the classification and detection of the current load state, but also gives the system predictability through the trend prediction module, improving the adaptability to complex dynamic load environments. At the same time, by combining load classification with anomaly detection results, it is possible to more accurately identify the transition state between normal operation and abnormal operation of the server, thereby providing support for the formulation of subsequent resource scheduling strategies.

[0074] In some embodiments, the present invention can also be extended to apply to the joint load status evaluation of multiple servers, for example, by analyzing the load balancing status of each node in the server cluster, further optimizing the resource allocation strategy of the entire cluster.

[0075] The design of load state classification, anomaly detection and trend prediction of the present invention fully utilizes the temporal dynamic characteristics of multimodal data and the deep feature representation of the self-supervised learning model. The above steps together constitute the core content of load state evaluation, and its design goal is to provide efficient technical support for the intelligent operation and resource scheduling of the server. Through the reasonable implementation of these steps, the present invention can significantly improve the stability and resource utilization of the server in a complex operating environment.

[0076] Specific implementation of step S5 In this embodiment, step S5 involves intelligent scheduling of server resources based on the load status evaluation results. By dynamically adjusting the server's computing resources, memory allocation, and task distribution strategies, the system is ensured to maintain efficient operation under variable load environments and to maximize resource utilization. This step formulates a resource scheduling strategy based on load status classification, anomaly detection, and trend prediction results, and dynamically optimizes resource allocation in combination with a scheduling feedback mechanism.

[0077] As an option, this embodiment divides the resource scheduling process of the server into two parts: a rule-driven static scheduling strategy and a dynamic scheduling strategy based on reinforcement learning. Specifically, the rule-driven scheduling strategy adjusts resource allocation according to preset rules based on clear load classification results, while the dynamic scheduling strategy optimizes scheduling behavior based on real-time load status and resource utilization through reinforcement learning algorithms.

[0078] In one possible implementation, the rule-driven scheduling strategy is based on the following logic: When the server is assessed to be in a low-load state, some pre-allocated resources are released, such as reducing the number of active CPU cores or lowering the amount of memory allocation to save system resources; When the server is in a medium load state, maintain the current resource configuration to avoid unnecessary resource adjustments; When the server is under high load, increase available resources, such as dynamically expanding the number of CPU cores or increasing memory allocation; When the server is overloaded, the load balancing mechanism is triggered to migrate some tasks to other server nodes to avoid performance bottlenecks.

[0079] It should be noted that the rule-driven scheduling strategy is suitable for scenarios where the load status does not change much in a short period of time. Its design goal is to achieve basic optimization of resources with the lowest computing overhead.

[0080] In the dynamic scheduling strategy part, this embodiment uses a reinforcement learning algorithm to achieve continuous optimization of resource allocation. Specifically, the server load status is regarded as the "state" of the system, resource adjustment operations (such as adding or releasing resources) are regarded as "actions", and indicators such as resource utilization, load balancing degree and task response time together constitute the "reward function" of the scheduling strategy.

[0081] Exemplarily, the scheduling algorithm may adopt a deep reinforcement learning method, such as a deep Q network (DQN) or a proximal policy optimization algorithm (PPO). In one possible implementation, the training process of the reinforcement learning model is as follows: First, the system state is initialized using the load state classification results; Then, based on the current load status, the optimal action is selected to adjust server resources; After executing the action, the new state of the system is evaluated in real time, and the scheduling effect is calculated based on the reward function; Finally, the accuracy and adaptability of the scheduling strategy are improved by iteratively updating the reinforcement learning model.

[0082] It should be noted that the reinforcement learning algorithm is particularly suitable for scenarios where the load state changes frequently or has large uncertainties. Its scheduling goal is to achieve dual optimization of resource utilization efficiency and system stability.

[0083] This embodiment also designs a scheduling feedback mechanism to ensure the effectiveness and real-time nature of resource adjustment. Specifically, after scheduling is completed, the system will re-analyze the load status of the server through the load status evaluation module to verify whether the resource adjustment has achieved the expected effect. As an option, if the adjustment effect does not reach the set threshold, for example, the resource utilization rate has not been significantly improved or the load is still in an overloaded state, the adaptive optimization process of the scheduling strategy is triggered to further improve resource allocation.

[0084] In a possible implementation, the adaptive optimization process includes the following steps: Collect the load status data of recent multiple dispatches and analyze the relationship between dispatch operations and load changes; Dynamically adjust the parameters of the scheduling strategy based on the analysis results, such as the weight of the reward function in reinforcement learning; Retrain the scheduling model when necessary to improve its adaptability to new load patterns.

[0085] It should be noted that the resource scheduling of this embodiment is not only applicable to a single server, but can also be extended to a server cluster environment. For example, by introducing a joint evaluation of the cluster load status in the load balancer, the task distribution strategy is further optimized to achieve optimal resource utilization of the entire cluster. In some embodiments, containerization technology (such as Kubernetes) can also be combined to dynamically adjust container resource quotas to support more fine-grained resource scheduling.

[0086] In the extended technical content, this embodiment can further optimize the scheduling implementation method, for example, by introducing the following extended functions: Use historical load status data to train prediction models, sense future load peaks before scheduling, and expand resources in advance; For abnormal load conditions, such as sudden growth in network traffic or long-term saturation of CPU usage, an early warning mechanism can be set to notify operation and maintenance personnel to intervene before resource scheduling; For real-time applications with low latency requirements, resources can be allocated to critical tasks first, while releasing resources occupied by non-critical tasks through background scheduling.

[0087] The intelligent resource scheduling strategy in this embodiment takes dynamic evaluation results as the core, combines the dual advantages of rule-driven and reinforcement learning, and realizes accurate resource allocation for variable load environments. The above scheduling logic has been reasonably designed to ensure the stability of the system and significantly improve resource utilization, thereby providing strong technical support for the efficient operation of the server.

[0088] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A server load status evaluation method based on a dynamic evaluation algorithm, characterized in that: The following steps are involved: Data collection: Collect performance indicator data, log data, and network traffic data from multimodal data sources on the server; Data preprocessing: time alignment, data cleaning, feature extraction and normalization of the collected data; Self-supervised learning modeling: extracting feature representations from multimodal data through self-supervised pre-training tasks, including time series mask prediction tasks, multimodal contrastive learning tasks, and contextual relationship prediction tasks; Dynamic load state assessment: Based on the feature representation, load state classification, abnormal load detection and load trend prediction are performed; Intelligent resource scheduling: Dynamically adjust server resource allocation strategies based on evaluation results.

2. The server load status evaluation method based on the dynamic evaluation algorithm according to claim 1 is characterized in that: The performance indicator data includes but is not limited to CPU utilization, memory occupancy, disk I / O utilization and network bandwidth.

3. The server load status evaluation method based on the dynamic evaluation algorithm according to claim 1 is characterized in that: The data preprocessing includes: Align the time steps of different modal data by timestamps to form a joint time series; Use interpolation to fill missing values ​​and remove outliers in performance indicator data; Analyze key event fields in log data and convert logs into feature vectors using natural language processing technology; Dimensionality reduction of network traffic data using principal component analysis.

4. The server load status evaluation method based on the dynamic evaluation algorithm according to claim 1 is characterized in that: The self-supervised pre-training task specifically includes: Time series masking prediction task: by randomly masking some time step data in the time series, the model is required to predict the value of the masked data; Multimodal contrastive learning task: construct positive and negative sample pairs, where the positive sample is a combination of multimodal data at the same time step, and the negative sample is a combination of data at different time steps; Contextual relationship prediction task: Use the model to predict the correlation between performance indicators and log data, as well as the causal relationship between performance indicators.

5. The server load status evaluation method based on the dynamic evaluation algorithm according to claim 4 is characterized in that: The time series cover prediction task is optimized by the following formula: ， in, is the actual value, is the predicted value, and mask is the set of masked positions.

6. The server load status evaluation method based on a dynamic evaluation algorithm according to claim 4 is characterized in that: The multimodal contrastive learning task is optimized by the following contrastive loss function: ， in, Represents the similarity between feature vectors.

7. The server load status evaluation method based on a dynamic evaluation algorithm according to claim 1, characterized in that: The dynamic load status assessment includes: Load status classification: classify the load status into low load, medium load, high load and overload; Anomaly detection: Detect abnormal load status based on high-dimensional feature representations based on isolation forest or density-based clustering algorithms; Trend prediction: Use LSTM or Transformer-based time series models to predict load change trends in future time steps.

8. The server load status evaluation method based on the dynamic evaluation algorithm according to claim 7 is characterized in that: The classifier used for load state classification is a fully connected neural network or a logistic regression classifier, and the input of the classifier is a high-dimensional feature representation generated by a self-supervised learning model. .

9. The server load status evaluation method based on a dynamic evaluation algorithm according to claim 7, characterized in that: The abnormal load state detection is achieved by calculating high-dimensional feature representation Cluster center with normal state The degree of deviation is realized, and the detection formula is: ， in, represents the Euclidean distance between the feature representation and the cluster center, is the standard deviation of the normal state.

10. The server load status evaluation method based on a dynamic evaluation algorithm according to claim 1, characterized in that: The intelligent resource scheduling is implemented through a reinforcement learning algorithm, specifically including: Status: current load status of the server; Action: Adjust resource allocation; Reward function: Combine resource utilization and load balancing to optimize the reward strategy and improve the overall performance of the system.

Images