Server resource scheduling system integrating AI and edge computing

Abstract

The invention relates to the technical field of server resource scheduling, in particular to a server resource scheduling system integrating AI and edge computing. The server resource scheduling system integrating the AI and the edge computing comprises a computing node module, a resource calling module, a task traversal module, a demand prediction module, a resource allocation module and a load scheduling module, the computing node module is used for monitoring the real-time running state of a server computing node and performing distributed node architecture fitting, and the load scheduling module is used for scheduling the load of the server computing node; constructing a distributed server node network; the resource calling module is used for acquiring a server running log; performing server resource calling behavior analysis on the server running log, performing resource calling time sequence characteristic evolution, and generating a server resource calling time sequence characteristic; and the task traversal module is used for carrying out unprocessed task traversal identification on the server running log and constructing an unprocessed task state mapping sequence. According to the invention, efficient and load-balanced server resource scheduling is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of server resource scheduling, and in particular to a server resource scheduling system integrating AI and edge computing. Background Art

[0002] With the rapid development of cloud computing and big data technologies, the scale and complexity of data centers and enterprise IT infrastructure continue to increase. As the core component supporting modern information applications, servers undertake a large number of computing and storage tasks. The effective scheduling and management of server resources have become a key factor affecting system performance and service quality. In real applications, server resource scheduling faces many challenges. First, with the widespread application of large-scale virtualization and containerization technologies, resource demand has become highly dynamic. Second, the requirements for service quality and user experience are increasing. The system needs to adjust resource allocation according to real-time load conditions to ensure efficient service response.

[0003] Traditional server resource scheduling methods usually rely on centralized scheduling systems, which mainly allocate resources based on preset rules or static policies. This approach is incapable of coping with rapidly changing workloads and dynamic environments, and can easily lead to resource waste or uneven resource load distribution. Therefore, in order to address the limitations of traditional server resource scheduling methods, a more intelligent server resource scheduling system is needed. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a server resource scheduling system integrating AI and edge computing to solve at least one of the above technical problems.

[0005] To achieve the above objectives, the present invention provides a server resource integrating AI and edge computing. The server resource scheduling system integrating AI and edge computing includes a computing node module, a resource calling module, a task traversal module, a demand prediction module, a resource allocation module and a load scheduling module: A computing node module is used to monitor the real-time operation status of the server computing nodes, perform distributed node architecture fitting, and build a distributed server node network; The resource call module is used to obtain the server operation log; analyze the server resource call behavior of the server operation log, and perform resource call timing feature evolution to generate server resource call timing features; The task traversal module is used to traverse and identify unprocessed tasks in the server operation log, mine task status features, and build an unprocessed task status mapping sequence; The demand prediction module is used to perform dynamic task processing demand prediction on the unprocessed task state mapping sequence according to the server resource call timing characteristics to obtain dynamic task processing demand prediction data; The resource allocation module is used to match the dynamic task processing demand forecast data with available resources according to the distributed server node network, and to perform dynamic computing resource allocation, thereby obtaining a dynamic computing resource allocation strategy; The load scheduling module is used to evolve task resource allocation based on dynamic computing resource allocation strategy, and then perform adaptive load scheduling optimization to build a load scheduling optimization model.

[0006] The present invention can help discover potential problems and ensure system stability by monitoring the operating status of server computing nodes in real time. Building a distributed server node network can help improve the scalability and fault tolerance of the system and optimize resource utilization. Analyzing the resource call behavior in the server operation log can reveal the system resource utilization and provide a basis for performance optimization. Generating resource call timing characteristics can help understand the evolution of resource call patterns and provide data support for subsequent task processing demand forecasting. Identifying unprocessed tasks and mining task status characteristics can help optimize task scheduling and resource allocation and improve system efficiency. Building an unprocessed task status mapping sequence can help understand the task processing process and provide a basis for subsequent demand forecasting. Predicting dynamic task processing needs based on sequence characteristics can allocate resources in advance and optimize system response speed. Obtaining dynamic task processing demand prediction data is helpful to reasonably plan resource allocation strategies and improve the overall efficiency of the system. Resource matching and dynamic computing resource allocation based on the distributed server node network can achieve rational use of resources and improve the overall performance of the system. Obtaining dynamic computing resource allocation strategies can help adjust resource allocation according to real-time needs and optimize system load balancing. Task resource allocation evolution and load scheduling optimization based on dynamic computing resource allocation strategies can achieve system load balancing and improve system stability and performance. Building a load scheduling optimization model is helpful to continuously improve the system's load scheduling strategy and improve the system's adaptability and efficiency.

[0007] Preferably, the computing node module is used to monitor the real-time operation status of the server computing node, and to fit the distributed node architecture to build a distributed server node network, specifically for: Identify all server computing nodes; Performing terminal device type identification on the server computing node, marking the central device computing node and the edge device computing node; Perform real-time operation status monitoring on the central device computing nodes and the edge device computing nodes to obtain the central node operation status parameters and the edge node operation status parameters; Perform node load balancing feature analysis on the central node operation status parameters and the edge node operation status parameters to obtain the central node load balancing feature and the edge node load balancing feature; According to the load balancing characteristics of the central node and the edge node, the distributed node architecture is fitted to the terminal device computing nodes and the edge device computing nodes to build a distributed server node network.

[0008] The present invention can establish an overall view by identifying all server computing nodes, which is helpful for comprehensive monitoring and management of server resources. Marking central device computing nodes and edge device computing nodes is helpful for distinguishing the functions and positioning of different nodes, providing a basis for subsequent load balancing. Real-time monitoring of the operating status of central device and edge device computing nodes can timely discover problems and take measures to maintain system stability. Acquiring the operating status parameters of central nodes and edge nodes is helpful for quantifying the operating conditions of nodes and providing data support for load balancing feature analysis. Analyzing the load balancing characteristics of central nodes and edge nodes can understand the load distribution between nodes and provide a basis for resource scheduling. Extracting load balancing characteristics is helpful for evaluating the performance of nodes and providing a reference for subsequent architecture fitting and resource allocation. Distributed node architecture fitting based on the load balancing characteristics of central nodes and edge nodes can achieve reasonable resource allocation and load balancing, improve the overall performance of the system, and build a distributed server node network to optimize the data processing process, improve the response speed and efficiency of the system, thereby optimizing resource load scheduling.

[0009] Preferably, the specific steps of performing node load balancing feature analysis on the central node operation status parameters and the edge node operation status parameters to obtain the central node load balancing feature and the edge node load balancing feature are: The CPU utilization of each node is calculated for the central node operating status parameters and the edge node operating status parameters to obtain the CPU utilization of each node; Perform utilization fluctuation analysis on the CPU utilization of each node to generate node utilization fluctuation characteristics; Perform real-time memory usage calculation on the central node operating status parameters and edge node operating status parameters to obtain the memory usage value of each node; Analyze the operating status parameters of the central node and the network throughput of the edge nodes, and extract the network throughput of each node; Analyze the current load characteristics of nodes by considering the fluctuation characteristics of node utilization, the memory usage of each node, and the network throughput of each node, so as to obtain the current load characteristics of each node. Identify the node type based on the current load characteristics of each node, and extract the load characteristics of each central node and each edge node; Perform load difference identification on the load characteristics of each central node and each edge node, and extract the load difference data between the central node and the edge node; The node load balancing characteristics are analyzed based on the load difference data of the central node and the edge node to obtain the load balancing characteristics of the central node and the edge node.

[0010] The present invention helps to understand the computing resource consumption of the node by calculating the CPU utilization of each node, and provides an important indicator for load balancing. Analyzing the fluctuation characteristics of CPU utilization can reveal the stability and volatility of the node load, and provide a reference for load scheduling optimization. Real-time calculation of the memory occupancy value of each node helps to monitor the memory resource usage of the node and provide a basis for resource allocation. Analysis of the network throughput of the node can evaluate the communication efficiency between nodes and provide data support for network topology optimization and load balancing. Comprehensive consideration of the CPU utilization, memory occupancy and network throughput characteristics of the node can fully understand the current load status of the node and provide a basis for load balancing decisions. Identifying the node type can distinguish between the central node and the edge node, which is helpful to formulate targeted load balancing strategies and optimize the resource scheduling effect. Extracting the load difference data between the central node and the edge node can help discover the load imbalance between the nodes and provide clues for further optimizing the load distribution. Load balancing feature analysis based on the load difference data can adjust the resource allocation strategy, achieve load balancing between nodes, and improve the overall system performance.

[0011] Preferably, the resource calling module is used to obtain the server operation log; analyze the server resource calling behavior of the server operation log, and perform resource calling timing feature evolution to generate server resource calling timing features, which is specifically used for: Obtain server operation logs; analyze server resource call behavior on the server operation logs and extract server resource call behavior data; Divide the server resource call behavior data into time series windows to obtain resource call behavior data of multiple time windows; The resource call behavior data of multiple time windows are counted for scheduling times to obtain the scheduling times of each time window; The scheduling frequency is calculated based on the number of scheduling times in each time window to generate the resource calling frequency of each window; The resource call frequency of each window is used to evolve the resource call timing characteristics and generate the server resource call timing characteristics.

[0012] The present invention can record various operations and events during server operation by obtaining server operation logs, providing a data basis for subsequent analysis. Extracting server resource call behavior data is helpful to understand the specific usage of server resources and provide a basis for resource scheduling optimization. Dividing resource call behavior data by time series windows can time serialize resource call behavior, which is helpful to discover the law and trend of resource calls. Processing resource call behavior data of multiple time windows can provide a data basis for subsequent frequency statistics and feature analysis. Counting the number of scheduling times in each time window can understand the frequency of resource scheduling and provide a basis for the formulation of resource scheduling strategies. Calculating the scheduling frequency can quantify the frequency change of resource calls and help discover the regularity and change trend of resource scheduling. The evolution of resource call time series characteristics can reveal the evolution law of resource call behavior and provide a reference for future optimization of resource scheduling. Generating server resource call time series characteristics is helpful to fully understand the scheduling of server resources and provide data support for load scheduling optimization.

[0013] Preferably, the task traversal module is used to traverse and identify unprocessed tasks in the server operation log, and mine task status features to construct an unprocessed task status mapping sequence, specifically for: Traverse and identify unprocessed tasks in the server operation logs and extract unprocessed tasks from multiple servers; Calculate the remaining time for task processing of unprocessed tasks on multiple servers and extract the timestamps of unprocessed tasks; Calculate the task resource usage of unprocessed tasks on the server and extract task resource usage data; Perform task status feature mining on the unprocessed task timestamps and task resource occupancy data to generate each unprocessed task status feature; Performing task sequence encoding on multiple unprocessed tasks of the servers to obtain an unprocessed task encoding sequence; According to each unprocessed task state feature, the unprocessed task encoding sequence is mapped to the corresponding task state, and an unprocessed task state mapping sequence is constructed.

[0014] The present invention can help understand the workload to be processed in the system by identifying the unprocessed tasks of the server, and provide basic data for resource scheduling and optimization. Extracting the unprocessed tasks of multiple servers is helpful to fully understand the distribution of tasks in the system and provide data support for subsequent processing. Calculating the remaining processing time of the task can evaluate the timeliness of task completion and help to reasonably arrange resources and task scheduling. Extracting the timestamp of the unprocessed task can record the initiation time of the task and provide a time clue for the task state change analysis. Calculating the resource occupancy of the task is helpful to evaluate the consumption of system resources by the task and provide a basis for resource scheduling optimization. Through task state feature mining, the state evolution law of the task can be deeply understood, and a reference basis can be provided for task scheduling and optimization. Sequential encoding of the unprocessed tasks can convert the task state into a data form that can be processed by a computer, which facilitates subsequent analysis and processing. Constructing an unprocessed task state mapping sequence can map the task state feature to the corresponding coding sequence, which is helpful for the visualization and analysis of the task state.

[0015] Preferably, the demand prediction module is used to perform dynamic task processing demand prediction on the unprocessed task state mapping sequence according to the server resource call timing characteristics to obtain dynamic task processing demand prediction data, specifically for: Perform multi-time point resource call prediction on the server resource call timing characteristics, thereby generating resource call prediction data at multiple time points; Perform resource call time series change analysis on resource call prediction data at multiple time points to obtain resource call time series change data; Perform task-by-task resource requirement analysis on the unprocessed task status mapping sequence to obtain resource requirement data for each task; Based on the resource call timing change data, dynamic task processing demand prediction is performed on the resource demand data of each task to obtain dynamic task processing demand prediction data.

[0016] The present invention can help the system make resource allocation decisions in advance and optimize resource utilization through resource call prediction. Analyzing resource call timing changes helps to understand the trend and change law of resource calls, and provide a basis for subsequent resource scheduling. Task-by-task resource demand analysis can deeply understand the resource demand of each task, and provide guidance for task scheduling and resource allocation. Dynamic task processing demand prediction based on resource call timing change data can predict the resource demand change of tasks according to the resource call situation, and help the system make timely resource scheduling and allocation.

[0017] Preferably, the specific steps of performing multi-time point resource call prediction on the server resource call timing characteristics to generate resource call prediction data at multiple time points are: Perform multi-time-point segmentation processing on the server resource call timing characteristics to generate resource call timing data at multiple time points; Extract timestamps of resource call timing data at multiple time points; Performing multi-time point timing fitting according to the timestamp to obtain a multi-time point resource call timing curve; Perform deep call feature learning on resource call timing curves at multiple time points to generate server resource call rules; Based on the server resource call rules, short-term resource call simulation is performed on the server resource call timing characteristics to generate resource call simulation data; Define the forecast time node; Resource call prediction is performed on the resource call simulation data according to the prediction time node to generate resource call prediction data at multiple time points.

[0018] The present invention can divide resource call data by time points through multi-time point resource call time series data processing, which is helpful to analyze resource usage in different time periods. The trends and rules of resource calls at multiple time points can be found through time series fitting, providing basic data for subsequent deep learning. Hidden patterns and rules in resource call data can be mined through deep call feature learning, providing a deeper understanding for resource scheduling optimization. Generating server resource call rules helps the system understand the characteristics and changing trends of resource calls, and provides guidance for resource allocation and scheduling strategies. Short-term resource call simulation can simulate the actual situation of resource calls and evaluate the effectiveness of resource allocation strategies. Resource call prediction can predict future resource needs and help the system make reasonable resource allocation and scheduling decisions.

[0019] Preferably, it is used to match the dynamic task processing demand forecast data with available resources according to the distributed server node network, and to perform dynamic computing resource allocation, thereby obtaining a dynamic computing resource allocation strategy, specifically for: Identifying the callable computing nodes on the distributed server node network to mark the callable computing nodes; Calculate the idle resources of the callable computing nodes and extract the idle resource data of each computing node; Based on the idle resource data of each computing node, the dynamic task processing demand forecast data is matched with available resources to obtain task processing resource matching data; Based on the task processing resource matching data, the distributed server node network is dynamically allocated computing resources to obtain a dynamic computing resource allocation strategy.

[0020] The present invention can determine which nodes can be used for task processing by identifying the callable computing nodes, thereby improving resource utilization. Extracting the idle resource data of each computing node helps to understand the situation of available resources in the system and provide a basis for task allocation. Dynamic task processing demand prediction based on the idle resource data of the computing nodes can find the best computing node for task allocation according to the actual resource situation. Generating task processing resource matching data can ensure that the task obtains sufficient resource support, improve task processing efficiency and system performance, and dynamically allocate computing resources based on task processing resource matching data can reasonably allocate resources according to actual needs and improve the overall performance of the system. The dynamic computing resource allocation strategy can flexibly adjust resource allocation according to the current task situation and computing node resource status, optimize server resource load scheduling, and improve system stability and efficiency.

[0021] Preferably, the load scheduling module is used to perform task resource allocation evolution based on a dynamic computing resource allocation strategy, and then perform adaptive load scheduling optimization to build a load scheduling optimization model, specifically for: Based on the dynamic computing resource allocation strategy, the distributed server node network is subjected to task resource allocation evolution to generate task resource allocation evolution data; the computing node load analysis is performed on the task resource allocation simulation data to obtain the load value of each computing node; Performing a node load peak analysis on the load value of each computing node to generate a node load peak value; Based on the node load peak, the dynamic computing resource allocation strategy is adaptively optimized for load scheduling and a load scheduling optimization model is constructed.

[0022] The present invention can understand the changing trend of task allocation by analyzing the evolution data of task resource allocation, and provide historical data support for load scheduling. The calculation node load analysis is helpful to evaluate the load situation of each node and provide a reference for load scheduling optimization. The highest point of the node load can be determined through node load peak analysis to help the system predict the peak load period. The adaptive load scheduling optimization based on the node load peak can adjust the resource allocation strategy according to the node load situation to avoid excessive or low load, and improve the load balance and performance stability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a structural schematic diagram of a server resource scheduling system integrating AI and edge computing according to the present invention. DETAILED DESCRIPTION

[0024] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0025] The example of this application provides a server resource scheduling system integrating AI and edge computing. The execution subject of the server resource scheduling system integrating AI and edge computing includes but is not limited to: mechanical equipment, data processing platform, cloud server node, network upload device, etc. equipped with the system can be regarded as the general computing node of this application, and the data processing platform includes but is not limited to: at least one of audio and image management system, information management system, and cloud data management system.

[0026] See also Figure 1 The present invention provides a server resource scheduling system integrating AI and edge computing, wherein the server resource scheduling system integrating AI and edge computing includes a computing node module, a resource calling module, a task traversal module, a demand prediction module, a resource allocation module and a load scheduling module: A computing node module is used to monitor the real-time operation status of the server computing nodes, perform distributed node architecture fitting, and build a distributed server node network; The resource call module is used to obtain the server operation log; analyze the server resource call behavior of the server operation log, and perform resource call timing feature evolution to generate server resource call timing features; The task traversal module is used to traverse and identify unprocessed tasks in the server operation log, mine task status features, and build an unprocessed task status mapping sequence; The demand prediction module is used to perform dynamic task processing demand prediction on the unprocessed task state mapping sequence according to the server resource call timing characteristics to obtain dynamic task processing demand prediction data; The resource allocation module is used to match the dynamic task processing demand forecast data with available resources according to the distributed server node network, and to perform dynamic computing resource allocation, thereby obtaining a dynamic computing resource allocation strategy; The load scheduling module is used to evolve task resource allocation based on dynamic computing resource allocation strategy, and then perform adaptive load scheduling optimization to build a load scheduling optimization model.

[0027] The present invention can help discover potential problems and ensure system stability by monitoring the operating status of server computing nodes in real time. Building a distributed server node network can help improve the scalability and fault tolerance of the system and optimize resource utilization. Analyzing the resource call behavior in the server operation log can reveal the system resource utilization and provide a basis for performance optimization. Generating resource call timing characteristics can help understand the evolution of resource call patterns and provide data support for subsequent task processing demand forecasting. Identifying unprocessed tasks and mining task status characteristics can help optimize task scheduling and resource allocation and improve system efficiency. Building an unprocessed task status mapping sequence can help understand the task processing process and provide a basis for subsequent demand forecasting. Predicting dynamic task processing needs based on sequence characteristics can allocate resources in advance and optimize system response speed. Obtaining dynamic task processing demand prediction data is helpful to reasonably plan resource allocation strategies and improve the overall efficiency of the system. Resource matching and dynamic computing resource allocation based on the distributed server node network can achieve rational use of resources and improve the overall performance of the system. Obtaining dynamic computing resource allocation strategies can help adjust resource allocation according to real-time needs and optimize system load balancing. Task resource allocation evolution and load scheduling optimization based on dynamic computing resource allocation strategies can achieve system load balancing and improve system stability and performance. Building a load scheduling optimization model is helpful to continuously improve the system's load scheduling strategy and improve the system's adaptability and efficiency.

[0028] In the embodiment of the present invention, refer to Figure 1 , is a schematic diagram of the steps of a server resource scheduling system integrating AI and edge computing of the present invention. In this example, the server resource scheduling system integrating AI and edge computing includes a computing node module, a resource calling module, a task traversal module, a demand prediction module, a resource allocation module and a load scheduling module: A computing node module is used to monitor the real-time operation status of the server computing nodes, perform distributed node architecture fitting, and build a distributed server node network; In this embodiment, the key performance indicators (KPIs) that need to be monitored are determined, such as CPU usage, memory usage, network bandwidth, disk I / O, etc., and thresholds are set so that an alarm is triggered when resource usage exceeds a certain limit. Common monitoring tools include Prometheus, Nagios, Zabbix, and Grafana, which can provide real-time monitoring, data collection, and visualization functions. For example, Prometheus can collect and store monitoring data through its time series database, and Grafana can be used for data visualization. A monitoring agent (such as Node Exporter) to collect the running status data of the nodes, configure the monitoring agent to ensure that it can periodically collect and send data to the monitoring server, configure the monitoring tool to regularly collect the running status data of each node at a set time interval (such as every minute), ensure that the data of all nodes can be effectively collected and stored in the monitoring system, clean the collected data, remove duplicate values, missing values ​​and abnormal values, ensure the accuracy and completeness of the data, perform necessary standardization processing for different monitoring indicators for subsequent analysis, select a suitable model for distributed node architecture fitting according to the characteristics of the monitoring data, and use machine learning methods (such as cluster analysis, regression model) or statistical methods (such as time series analysis). For example, you can use the k-means clustering algorithm to classify nodes and identify nodes with similar performance characteristics. The processed monitoring data (such as CPU The training and fitting model is based on the data usage rate, memory usage, etc.) as input, and the model is trained and fitted. The model performance is evaluated by adjusting the model parameters and using the training set (such as historical data) to ensure that it can accurately reflect the operating status of the node. The accuracy and robustness of the model are evaluated by methods such as cross-validation. If the model performance does not meet the requirements, the model parameters need to be optimized or a different fitting method needs to be selected. According to the fitting results, the structure of the distributed server node network is designed, including the connection relationship between nodes, data flow direction and load balancing strategy, and the role of each node (such as master node, slave node, load balancing node) is determined to optimize resource utilization and task processing efficiency. The network architecture is deployed to ensure that all nodes can communicate effectively. Virtual networks (such as Docker networks, Kubernetes services) or physical networks can be used, and network switches and routers are configured to ensure that data packets can flow smoothly between nodes. Monitoring tools are used to monitor the operating status of the distributed server node network in real time to ensure its stability and reliability. Network performance data is regularly analyzed, the load and response time of the nodes are evaluated, and the network configuration is adjusted in time to cope with changing workloads.

[0029] The resource call module is used to obtain the server operation log; analyze the server resource call behavior of the server operation log, and perform resource call timing feature evolution to generate server resource call timing features; In this embodiment, the log type to be analyzed is determined, such as system logs ( / var / log / syslog), application logs (such as Nginx, Apache logs), monitoring logs, etc., and the path and format of the log file are ensured to be known, usually in text format or JSON format. Tools such as Filebeat, Fluentd or Logstash can be used. These tools can collect logs from different sources and send them to a centralized log management system (such as Elasticsearch). The log collection tool is configured to collect log files regularly, and the collected logs are stored in a queryable database such as Elasticsearch, MongoDB or a relational database (such as MySQL). It is ensured that the log data can be quickly retrieved by timestamp for subsequent analysis. According to the format of the log (such as JSON, CSV, Plain Text), the log data is converted into a structured format using a suitable parsing method. The Python pandas library or other data processing tools can be used for parsing. The log data is cleaned to remove unnecessary fields, duplicate entries and null values ​​to ensure the accuracy and completeness of the data. The timestamps are uniformly formatted to ensure that the timestamps of all records are in a consistent format (such as ISO 8601), filter out log entries related to resource calls according to needs, such as call records related to CPU, memory, I / O operations, etc., use statistical analysis methods to identify the behavior patterns of server resource calls, which may include call frequency, call duration, resource consumption, etc. Cluster analysis (such as K-means) can be used to identify different call patterns, and use data visualization tools (such as Grafana, Tableau, Matplotlib) to visualize resource call behaviors to help quickly identify trends and anomalies. The visualization content includes the changing trend of the number of calls, the time distribution of resource usage, etc. Based on the cleaned log data, construct the time series data of resource calls, which includes recording the resource usage at each time point (such as CPU The timestamps should be consistent to ensure that the data can be arranged in chronological order, and feature extraction should be performed on time series data, including: Trend: observe the long-term trend of resource usage (increasing, decreasing or stable), Seasonality: identify periodic changes in resource usage (such as peak and trough periods), Sudden behavior: analyze abnormal fluctuations and emergencies in resource usage, apply time series analysis models (such as ARIMA, LSTM) to perform evolutionary analysis on resource call time series features, predict future resource usage trends, record key features and change points identified during the evolution process, and integrate the extracted time series features into a structured data set, which can be a table containing timestamps, resource usage, call frequency and other information, to ensure that the integrated data can be used for subsequent analysis and modeling.Save the generated resource call timing characteristics in a suitable format (such as CSV, JSON) for subsequent use. The data can be stored in a data warehouse or data lake for further analysis.

[0030] The task traversal module is used to traverse and identify unprocessed tasks in the server operation log, mine task status features, and build an unprocessed task status mapping sequence; In this embodiment, the log types that need to be analyzed are determined, such as application logs, task scheduling logs, etc., to ensure that relevant information about unprocessed tasks can be obtained, and tools such as Filebeat, Fluentd, or Logstash are used to centralize log data to an analyzable location (such as Elasticsearch or a database), and the collected logs are stored in a queryable database to ensure that log records can be quickly accessed and retrieved. According to the log format (such as JSON, CSV, PlainText), appropriate methods are used to convert log data into a structured format for subsequent analysis, clean up log data, remove unnecessary fields, duplicate entries, and null values, ensure data accuracy and completeness, and filter out log entries related to task status, such as "task created", "task running", and "task failed". , etc., read and process the log data line by line, find the relevant information of the unprocessed tasks, which usually includes the task ID, task creation time, task status, etc., determine what kind of tasks are considered "unprocessed", such as tasks that are not completed or unsuccessful within a specific time, identify the unprocessed tasks by status labels (such as "Created", "Running", "Failed"), create a data structure (such as a dictionary, list or DataFrame) to record the ID, status, creation time and other information of the unprocessed tasks for subsequent analysis, analyze the state changes of the unprocessed tasks, extract the state transition features, such as the transition from "Created" to "Running" or "Failed", count the frequency and duration of each state, identify the pattern of state changes, and use time series analysis methods to observe the trend of state changes. Use tools (such as Matplotlib, Seaborn or Grafana) to visualize the state features to help identify the pattern and abnormality of state changes. According to the state changes of unprocessed tasks, construct a state mapping sequence, each sequence records the task ID and the timestamp of its state change, for example: [(task_id, timestamp, "Created"), (task_id, timestamp, "Running") ...] Arrange the state mapping sequence into a structured format (such as a list or DataFrame) for subsequent analysis and storage.

[0031] The demand prediction module is used to perform dynamic task processing demand prediction on the unprocessed task state mapping sequence according to the server resource call timing characteristics to obtain dynamic task processing demand prediction data; In this embodiment, the server resource call timing characteristics (such as CPU usage, memory usage, etc.) are merged with the unprocessed task status mapping sequence to form a comprehensive data set. The data set should include timestamps, resource usage characteristics and status information of unprocessed tasks, handle missing values, outliers and duplicate data, ensure data quality, ensure that the timestamp format is consistent, and sort the data for subsequent analysis. Feature selection methods (such as LASSO regression and random forest importance analysis) are used to identify features that have a greater impact on task processing demand prediction, ensuring that the selected features can fully reflect changes in task processing demand. Appropriate machine learning models are selected according to the characteristics of the data and the prediction target. Common models include: linear regression: suitable for simple prediction of linear relationships, decision trees or random forests: suitable for processing complex nonlinear relationships, time series models: such as ARIMA or LSTM, suitable for processing time-dependent data, and the data set is divided into a training set (for training the model) and a test set (for evaluating model performance). Usually 70% of the data is used for training and 30% for testing. The data is used for testing, the selected model is trained using the training set, and the model parameters are adjusted to improve the prediction accuracy. For example, for random forest, the number of trees, maximum depth and other parameters can be adjusted. The prediction performance of the model is evaluated using the test set. Indicators such as root mean square error (RMSE) and mean absolute error (MAE) are used. The stability and generalization ability of the model are further verified through methods such as cross-validation. The predicted dynamic task processing demand results are saved in a database or file for subsequent analysis and scheduling decisions. The rationality and feasibility of the prediction results are checked, and the accuracy and deviation of the prediction are analyzed. The trend of the prediction results is displayed through visualization tools (such as Matplotlib and Seaborn) for easy understanding and decision-making. According to the predicted dynamic task processing requirements, the server resource allocation strategy is adjusted to ensure that resources can meet the expected task requirements. Preventive measures can be formulated, such as allocating resources in advance and dynamically adjusting task scheduling strategies.

[0032] The resource allocation module is used to match the dynamic task processing demand forecast data with available resources according to the distributed server node network, and to perform dynamic computing resource allocation, thereby obtaining a dynamic computing resource allocation strategy; In this embodiment, the predicted data is formatted into a table or structured format, including fields such as task ID, resource requirements, and priority, to ensure that the data can be effectively read by subsequent processing tools. The status information of the distributed server nodes is obtained in real time through monitoring tools (such as Prometheus and Grafana), including the CPU usage, memory usage, network bandwidth, and load of the nodes. According to the collected node status, the available resources of each node are evaluated, and the available CPU, memory, and other information of each node are recorded for subsequent matching. The resource matching criteria, such as task priority and resource requirements, are determined, and tasks with high requirements are preferentially matched to nodes with sufficient available resources. Greedy algorithms, minimum cost flow algorithms, or Hungarian algorithms can be used for resource matching. These algorithms can help achieve efficient resource allocation. The predicted task demand data is traversed and matched with the available resources of the distributed server nodes. The corresponding relationship between each task and the matching node is recorded. If the resource demand of a task fails to match a suitable node within a time window, it should be recorded as a task to be allocated for subsequent processing. According to the resource matching results, a dynamic computing resource allocation strategy is formulated, which includes allocating specific computing resources (such as CPU Core, memory capacity) to each matching task, consider the task priority and node load, ensure the rationality of resource allocation, assign tasks to corresponding computing nodes, update node status information, record resource usage after allocation, monitor the real-time effect of resource allocation, ensure the smooth execution of tasks, record the generated dynamic computing resource allocation strategy into the database, including task ID, allocation node, allocated resource amount, allocation time and other information, regularly evaluate the effect of resource allocation strategy, analyze indicators such as task execution success rate and resource utilization efficiency, optimize resource allocation strategy based on monitoring data and historical allocation results to cope with future dynamic task requirements.

[0033] The load scheduling module is used to evolve task resource allocation based on dynamic computing resource allocation strategy, and then perform adaptive load scheduling optimization to build a load scheduling optimization model.

[0034] In this embodiment, detailed information of the tasks to be processed is collected, including the task ID, the required amount of resources (CPU, memory, I / O), priority, etc. This information can be obtained through the task scheduling system or monitoring tools. According to the collected task requirements, the previously defined dynamic computing resource allocation strategy is applied in real time to decide how to allocate resources to each task, taking into account the priority of the task and the current load of the node, to ensure that high-priority tasks can obtain resources first. After each resource allocation, detailed information of the resource allocation is recorded, including the allocation time, task ID, allocated node, amount of resources, etc., to form historical data of resource allocation evolution, which is convenient for subsequent analysis and optimization. Monitoring tools (such as Prometheus and Grafana) are used to monitor the load of the computing nodes in real time, including CPU, memory, network usage, etc., and thresholds are set. When the node load exceeds a certain critical value, the adaptive scheduling mechanism is triggered. According to the monitored load conditions, the adaptive load is defined. Load scheduling rules, for example, if the load of a node exceeds 80%, new tasks are assigned to nodes with lower loads. Load balancing algorithms (such as polling, least connections, weighted polling, etc.) can also be used to optimize resource allocation. According to the load monitoring results and the defined scheduling rules, resource allocation can be dynamically adjusted. For example, tasks can be migrated from high-load nodes to low-load nodes, or the allocation strategy of new tasks can be adjusted. Scheduling decisions and results are recorded for subsequent analysis. Historical resource allocation data, load conditions, task execution results, etc. are collected as the basic data set for building the optimization model. Key features are extracted from historical data, such as the number of tasks, resource usage, node load, scheduling delay, etc., to ensure that the features can accurately reflect the influencing factors of task scheduling. The selected model is trained using the prepared data set, and model parameters are optimized to improve prediction accuracy. Cross-validation can be used to evaluate the performance of the model to ensure the generalization ability of the model. RMSE, MAE, R² The prediction performance of the model is evaluated by indicators such as , to ensure that the model can accurately reflect the needs of load scheduling. According to the evaluation results, the model is continuously optimized, and the feature selection and model parameters are adjusted to improve the scheduling efficiency and accuracy. The trained model is incorporated into the load scheduling module to predict and optimize task resource allocation in real time, monitor the actual effect of the model, and ensure that it remains effective in a dynamic environment.

[0035] In this embodiment, the computing node module is used to monitor the real-time operation status of the server computing node, and to fit the distributed node architecture to build a distributed server node network, specifically for: Identify all server computing nodes; Performing terminal device type identification on the server computing node, marking the central device computing node and the edge device computing node; Perform real-time operation status monitoring on the central device computing nodes and the edge device computing nodes to obtain the central node operation status parameters and the edge node operation status parameters; Perform node load balancing feature analysis on the central node operation status parameters and the edge node operation status parameters to obtain the central node load balancing feature and the edge node load balancing feature; According to the load balancing characteristics of the central node and the edge node, the distributed node architecture is fitted to the terminal device computing nodes and the edge device computing nodes to build a distributed server node network.

[0036] In this embodiment, a network scanning tool (such as Nmap or Angry IP Scanner) is used to scan the network, identify all active server computing nodes, collect basic information such as the node's IP address, MAC address, and host name, and determine the node's device type by analyzing the node's open ports and services (such as HTTP, SSH, FTP, etc.). SNMP (Simple Network Management Protocol) is used to obtain detailed information about the node, such as the device manufacturer, model, and function. According to the identified device type, the server node is marked as a central device computing node (such as a data center server) and an edge device computing node (such as an IoT device, an edge gateway, etc.). The classification results are stored in a database to form a list of central nodes and edge nodes. A suitable monitoring tool (such as Zabbix, Prometheus, or Nagios) is selected for real-time operation status monitoring to determine the operation status parameters that need to be monitored, such as CPU Utilization rate, memory utilization rate, network bandwidth, disk I / O, etc., configure monitoring tools to ensure that the operating status parameters of the central node and edge nodes can be collected in real time, set data storage strategies to ensure long-term preservation and access to monitoring data, clean the collected operating status parameters, remove invalid data and outliers, use statistical analysis methods (such as mean, standard deviation) and machine learning algorithms (such as cluster analysis) to analyze load balancing characteristics, calculate the load balancing characteristics of the central node and edge nodes, such as load distribution, response time, throughput, etc., use visualization tools (such as Matplotlib, Tableau) to generate load balancing feature graphs to help understand the load of the nodes, design a suitable distributed node architecture based on the load balancing characteristics to ensure efficient use of computing resources, use optimization algorithms (such as genetic algorithms, particle swarm optimization) to fit the distributed node architecture to achieve optimal resource allocation and load balancing, build a distributed server node network based on the fitting results, ensure efficient communication and data flow between the central device and the edge device, and record the network topology, including the connection relationship of each node, bandwidth allocation and other information.

[0037] In this embodiment, the specific steps of performing node load balancing feature analysis on the central node operation status parameters and the edge node operation status parameters to obtain the central node load balancing features and the edge node load balancing features are: The CPU utilization of each node is calculated for the central node operating status parameters and the edge node operating status parameters to obtain the CPU utilization of each node; Perform utilization fluctuation analysis on the CPU utilization of each node to generate node utilization fluctuation characteristics; Perform real-time memory usage calculation on the central node operating status parameters and edge node operating status parameters to obtain the memory usage value of each node; Analyze the operating status parameters of the central node and the network throughput of the edge nodes, and extract the network throughput of each node; Analyze the current load characteristics of nodes by considering the fluctuation characteristics of node utilization, the memory usage of each node, and the network throughput of each node, so as to obtain the current load characteristics of each node. Identify the node type based on the current load characteristics of each node, and extract the load characteristics of each central node and each edge node; Perform load difference identification on the load characteristics of each central node and each edge node, and extract the load difference data between the central node and the edge node; The node load balancing characteristics are analyzed based on the load difference data of the central node and the edge node to obtain the load balancing characteristics of the central node and the edge node.

[0038] In this embodiment, a monitoring tool (such as Prometheus, Zabbix or Grafana) is used to configure node monitoring to ensure that CPU usage data can be collected. CPU utilization = (total time / total usage time) × 100%. For each node, the CPU usage time (user state, kernel state, etc.) within the monitoring period is collected to calculate the CPU utilization of each node. The fluctuation characteristics of the CPU utilization of each node are calculated, including the mean, standard deviation, maximum value, minimum value, etc. The utilization fluctuation pattern is identified through time series analysis. The utilization fluctuation characteristics of each node (such as fluctuation amplitude, frequency, etc.) are recorded in the database to provide a basis for subsequent analysis. The monitoring tool is configured to set the monitoring parameters of memory usage (such as total memory, used memory, free memory), calculate the memory usage value of each node, and record the results. The network monitoring tool (such as Wireshark, ntop) is used to monitor the network traffic, and the necessary parameters (such as bandwidth, traffic) are configured. The network throughput can be calculated by the following formula: Network throughput = time / amount of data transmitted (such as Mbps). The CPU The utilization fluctuation characteristics, memory usage values, and network throughput are integrated into the current load characteristics. Data analysis tools (such as Pandas and NumPy) are used to analyze the integrated data, and the current load characteristics of each node, including CPU, memory, and network load, are extracted. The node type is identified based on the node's load characteristics (such as CPU, memory, and network traffic pattern) to determine whether it is a central node or an edge node. The load characteristics of each central node and edge node are extracted and recorded to ensure the integrity of the characteristics. Statistical methods such as t-tests or variance analysis are used to compare the load characteristics of central nodes and edge nodes, identify load differences, and record the identified load difference data in the database for subsequent analysis. Cluster analysis, regression analysis, and other methods are used to analyze the load balancing characteristics. Based on the load difference data between central nodes and edge nodes, the load balancing characteristics of central nodes and edge nodes are extracted.

[0039] In this embodiment, the resource calling module is used to obtain the server operation log; analyze the server resource calling behavior of the server operation log, and perform resource calling timing feature evolution to generate server resource calling timing features, which are specifically used for: Obtain server operation logs; analyze server resource call behavior on the server operation logs and extract server resource call behavior data; Divide the server resource call behavior data into time series windows to obtain resource call behavior data of multiple time windows; The resource call behavior data of multiple time windows are counted for scheduling times to obtain the scheduling times of each time window; The scheduling frequency is calculated based on the number of scheduling times in each time window to generate the resource calling frequency of each window; The resource call frequency of each window is used to evolve the resource call timing characteristics and generate the server resource call timing characteristics.

[0040] In this embodiment, the types of logs to be collected are determined, including: operating system logs: such as syslog and dmesg, which record system events and errors; application logs: detailed information of application runtime, such as logs of Web servers (Apache, Nginx) or databases (MySQL, PostgreSQL); security logs: recording security-related events, such as user logins and permission changes; log collection tools (such as rsyslog or Filebeat) are used to configure log sources and target storage locations, ensure that the tools can read log files in real time or periodically, and transmit them to a central storage system (such as ELK Stack or Hadoop); set a log rotation mechanism to prevent log files from being too large and affecting server performance; use log parsing tools (such as Logstash and Fluentd) to parse the operation logs, extract key fields, including timestamps, resource types (CPU, memory, disk, network, etc.), call duration or request size, define parsing rules to ensure data accuracy and consistency; extract server resource call behavior data from the parsed logs to form a structured data set, record detailed information of each call, and select a suitable time window size (such as 1 minute, 5 minutes, or 10 minutes) according to the characteristics of server resource usage and analysis requirements. Minutes), the window size needs to take into account the accuracy of the analysis and the data processing capability. The resource call behavior data within the entire time range is divided into multiple time windows to ensure the data integrity in each time window. Sliding windows (for example, windows are updated at regular intervals) or fixed windows (fixed time periods for each window) can be used for division. The resource call behaviors in each time window are counted, and the number of scheduling times in each window (that is, the total number of resource calls) is counted. This can be achieved through a simple counter. The number of scheduling times in each time window and its corresponding time window information are recorded in the database or data framework for subsequent analysis and query. The scheduling frequency can be calculated by the following formula: Scheduling frequency = Window size (time) / number of scheduling times. Apply this formula to each time window to calculate the resource call frequency, which reflects the active level of resource use in each window. Perform time series analysis on the resource call frequency of each window to extract time series features, including: Average frequency: the average resource call frequency in the window. Volatility: the degree of change in frequency, which may be calculated by standard deviation or coefficient of variation. Trend: analyze the trend of frequency changes over time and identify increasing or decreasing patterns. Periodicity: check whether there is a periodic pattern (for example, daily peak hours). Record the extracted time series features (such as frequency change trends and fluctuations) in the database for subsequent analysis and report generation.

[0041] In this embodiment, the task traversal module is used to traverse and identify unprocessed tasks in the server operation log, and to mine task status features to construct an unprocessed task status mapping sequence, which is specifically used for: Traverse and identify unprocessed tasks in the server operation logs and extract unprocessed tasks from multiple servers; Calculate the remaining time for task processing of unprocessed tasks on multiple servers and extract the timestamps of unprocessed tasks; Calculate the task resource usage of unprocessed tasks on the server and extract task resource usage data; Perform task status feature mining on the unprocessed task timestamps and task resource occupancy data to generate each unprocessed task status feature; Performing task sequence encoding on multiple unprocessed tasks of the servers to obtain an unprocessed task encoding sequence; According to each unprocessed task state feature, the unprocessed task encoding sequence is mapped to the corresponding task state, and an unprocessed task state mapping sequence is constructed.

[0042] In this embodiment, the log types to be collected are determined, mainly including: operating system logs: such as syslog or journalctl, which record system events and error information; application logs: such as Web server logs (such as Apache, Nginx) and database logs (such as MySQL, PostgreSQL), which record the running status of the application and the processing of requests; task scheduling logs: monitor the logs generated by the task scheduler (such as Cron, Celery), record the start, completion and failure of the task, use parsing tools (such as Logstash, Fluentd, or custom Python scripts) to parse the log files according to the predefined format, and pay attention to the following fields during parsing: task ID: uniquely identifies each task; task status: such as "not started", "in progress", "failed", "completed", etc.; timestamp: record the creation time, start time and end time of the task; extract these fields through regular expressions or log analysis libraries (such as pandas), and store the filtered unprocessed task information (including task ID, timestamp and status) in a database (such as MySQL, PostgreSQL) or a data frame (such as Pandas DataFrame), form a list of unprocessed tasks, for each unprocessed task, determine its estimated completion time, this time may be directly provided in the log, or need to be estimated based on the task type and historical data. If there is no direct estimated completion time, you can use the following methods to estimate: Historical average time: Analyze the historical data of similar tasks and calculate their average processing time, dynamic estimation: Dynamically adjust according to the current system load, resource usage and task complexity, record the calculated remaining time (in seconds or minutes) of the unprocessed task with the timestamp in the database, and form a record containing the task ID, remaining time and timestamp, use monitoring tools (such as Prometheus, Grafana, or system monitoring commands such as top, htop) to obtain the resource usage of the current unprocessed task. Key resources include: CPU usage: CPU time or percentage used by each task, memory usage: amount of memory used by each task, I / O usage: number and speed of disk read and write operations, associate the monitored resource usage with the corresponding task ID, and use calculation scripts (such as Python scripts combined with psutil The database) counts the resource usage of each unprocessed task and stores the results in the database to determine the key status characteristics of the unprocessed tasks, such as: remaining time: the time difference from the current time to the expected completion time of the task, current resource usage: the real-time value of CPU, memory and I / O usage, task priority: if the priority is defined in the task management system, record the priority of the task, historical status: the previous success rate and number of failures of the task, etc.Use statistical analysis tools (such as pandas) to extract features from each unprocessed task and generate feature vectors for each task. Cluster analysis techniques (such as K-means) can be used to classify tasks and extract relevant features. Assign a unique code to each unprocessed task. The following methods can be used: Directly use the task ID: simple and direct, easy to reference later. Hash coding: Use a hash function (such as SHA256) to hash the task ID or task features to generate a fixed-length code. Define mapping rules based on the state features of each unprocessed task, associate each task in the coding sequence with the corresponding state features, and generate an unprocessed task state mapping sequence by combining the unprocessed task coding sequence with the corresponding state features. This sequence will be used for subsequent data analysis and decision support to help identify task processing priorities and resource allocation. ,

[0043] In this embodiment, the demand prediction module is used to perform dynamic task processing demand prediction on the unprocessed task state mapping sequence according to the server resource call timing characteristics to obtain dynamic task processing demand prediction data, which is specifically used for: Perform multi-time point resource call prediction on the server resource call timing characteristics, thereby generating resource call prediction data at multiple time points; Perform resource call time series change analysis on resource call prediction data at multiple time points to obtain resource call time series change data; Perform task-by-task resource requirement analysis on the unprocessed task status mapping sequence to obtain resource requirement data for each task; Based on the resource call timing change data, dynamic task processing demand prediction is performed on the resource demand data of each task to obtain dynamic task processing demand prediction data.

[0044] In this embodiment, historical resource call data is collected: historical resource call time series data is extracted from the monitoring system or log collection tool. This data should include timestamps, CPU usage, memory usage, network bandwidth, etc. Missing values ​​and outliers are cleaned to ensure data integrity. Standardization or normalization is performed as needed to facilitate model training. Additional features such as hour, day, week, season, etc. are extracted from timestamps to capture potential periodic behaviors. Sliding window features: additional features such as the past N are created using sliding window technology. The average value, maximum value, etc. of each time point can help the model capture time dependence. Model selection: Select a suitable prediction model, such as ARIMA: Suitable for stationary time series, LSTM: A deep learning model suitable for capturing long-term and short-term dependencies, Prophet: Suitable for time series with strong seasonal and trend changes. Use the training set (usually the first 80%-90% of the historical data) to train the selected model, and use the test set (the rest) to evaluate the model performance. Evaluate through indicators such as the root mean square error (RMSE). Analyze the changes in the predicted data at multiple time points, and use the percentage of change between adjacent time points: rate of change = (prediction value at the current time point − prediction value at the previous time point) / prediction value at the previous time point × 100%. By drawing a time series graph, observe the trend line of the predicted value, and identify rising, falling or stable patterns. The rate of change, trend direction, and fluctuation amplitude are recorded as a new data set to facilitate subsequent analysis and decision-making. The generated unprocessed task status mapping sequence should contain the status information of each task, such as not started, in progress, paused, etc. Define resource requirement characteristics for each task, including: CPU requirement: The CPU required for each task Percentage, Memory requirement: the amount of memory required for task execution (MB), I / O requirement: the amount of disk I / O operations required for the task, Traverse the sequence of unprocessed task state mappings, Based on historical data or resource usage of similar tasks, extract the resource requirement data of each task, which can be analyzed by statistical analysis methods, such as calculating the average resource requirement of historical similar tasks, Integrate the resource call timing change data and the resource requirement data of unprocessed tasks to form a comprehensive data set, This data set should include time series characteristics, resource change rate and task requirement characteristics, According to the characteristics of the integrated data, select a suitable dynamic demand prediction model, such as: Regression model: such as linear regression, random forest regression, Deep learning model: such as LSTM, suitable for capturing dynamic changes in time series data, Use the integrated data set to train the prediction model, evaluate the model performance, and ensure that the model can effectively capture the relationship between resource demand and call changes, Based on the trained model, predict the future task execution and generate dynamic task processing demand prediction data, This data should include resource demand predictions for each task at different time points.

[0045] In this embodiment, the specific steps of performing multi-time point resource call prediction on the server resource call time series characteristics to generate resource call prediction data at multiple time points are: Perform multi-time-point segmentation processing on the server resource call timing characteristics to generate resource call timing data at multiple time points; Extract timestamps of resource call timing data at multiple time points; Performing multi-time point timing fitting according to the timestamp to obtain a multi-time point resource call timing curve; Perform deep call feature learning on resource call timing curves at multiple time points to generate server resource call rules; Based on the server resource call rules, short-term resource call simulation is performed on the server resource call timing characteristics to generate resource call simulation data; Define the forecast time node; Resource call prediction is performed on the resource call simulation data according to the prediction time node to generate resource call prediction data at multiple time points.

[0046] In this embodiment, according to the needs, the time series data is divided into multiple time periods (for example, every minute, every hour, or every fixed time window). The sliding window or fixed window method can be used. The data of each time period will be used as the basis for subsequent analysis. The resource call data of multiple time points after division are stored in a database or a data frame to form a structured time series data set. The timestamp of each time period is extracted from the divided time series data to ensure one-to-one correspondence with the resource call data. The timestamp usually includes date and time information for subsequent analysis. The extracted timestamp is stored together with the corresponding resource call data to form a complete time series data set containing the timestamp. A suitable fitting model is selected according to the data characteristics, such as: linear regression model: suitable for relatively simple linear relationships, polynomial regression: suitable for more complex relationships, time series prediction models: such as ARIMA and SARIMA, used to process trends and seasonality in time series data, the extracted timestamps and corresponding resource call data are fitted, and the selected model is used to train the data to obtain the resource call time series curve, and the fitting effect is evaluated. Common indicators include root mean square error (RMSE) and determination coefficient (R²). Determine the features to be extracted, such as: when the call frequency peaks At this moment, the call pattern (such as periodicity and trend) can be learned by using deep learning models (such as LSTM and GRU) to capture the complex patterns in time series data. Prepare training sets and validation sets, use historical resource call data for model training, train the deep learning model, use the back propagation algorithm to optimize weights and biases, and improve the learning effect of the model through multiple iterations. According to the extracted resource call rules, select the appropriate simulation method. The following methods can be used: Monte Carlo simulation: Generate multiple possible call scenarios through random sampling, model-based simulation: Use the previously established time series model to generate short-term resource call data, generate future short-term resource call simulation data based on the extracted rules and historical data, record the simulation results, define the prediction time nodes according to business needs and prediction cycles, such as specific time points every 5 minutes, every hour or every day, combine the generated resource call simulation data with the prediction time nodes for subsequent prediction analysis, select the appropriate prediction model (such as SARIMA and LSTM) for short-term prediction according to the characteristics of the simulation data, use the trained model to predict the resource call simulation data according to the defined prediction time nodes, and generate resource call prediction data at multiple time points.

[0047] In this embodiment, the resource allocation module is used to match the dynamic task processing demand prediction data with available resources according to the distributed server node network, and to perform dynamic computing resource allocation, thereby obtaining a dynamic computing resource allocation strategy, which is specifically used for: Identifying the callable computing nodes on the distributed server node network to mark the callable computing nodes; Calculate the idle resources of the callable computing nodes and extract the idle resource data of each computing node; Based on the idle resource data of each computing node, the dynamic task processing demand forecast data is matched with available resources to obtain task processing resource matching data; Based on the task processing resource matching data, the distributed server node network is dynamically allocated computing resources to obtain a dynamic computing resource allocation strategy.

[0048] In this embodiment, node status information is obtained from a monitoring platform of a distributed system (such as Kubernetes, Apache Mesos, or OpenStack). This information usually includes the health status, IP address, CPU and memory usage of the node, etc. The real-time status of the node is queried using an API or command line tool to ensure that the latest node information can be obtained. The criteria for callable computing nodes are determined. For example, the node must be in an "online" state and the current CPU usage is lower than a set threshold (such as 70%). Additional criteria can be defined, such as memory usage, load balancing status, etc., to ensure that the node has sufficient resources to process new tasks. The collected node information is traversed and computing nodes that meet the callable criteria are marked. This step usually records the node ID, status, and current resource usage for subsequent resource calculation and matching. System monitoring tools (such as Prometheus, Nagios, or Zabbix) are used to obtain real-time resource usage data of callable nodes. These tools provide a wealth of monitoring indicators that can help us evaluate the resource usage of each node. In particular, CPU The idle resource data of each node is organized into a structured format (such as a table or database) to facilitate subsequent analysis and decision-making. The resource requirement information of the tasks to be processed is obtained from the task management system or scheduler. This information usually includes the CPU required for the task. The number of cores, memory size, and other resource requirements ensure that the task demand information is up to date and is classified according to the priority of the task. Determine how to match the task requirements with the idle resources of the callable computing nodes. Simple conditional judgments (such as whether there are enough idle resources) or more complex algorithms (such as greedy algorithms and minimum cost flow algorithms) can be used for matching. Traverse the resource requirements of each task and find the callable computing nodes that meet the requirements. Record the correspondence between each task and its matching node for subsequent resource allocation. This step requires ensuring that each task is effectively matched to the appropriate computing node. Based on the priority of the task and the load of the callable node, a dynamic resource allocation strategy is designed. Tasks with high priority should be allocated to nodes with more idle resources as much as possible without affecting the overall performance of the system. Consider using a load balancing algorithm to ensure even distribution of resources and avoid overloading some nodes while other nodes are idle. According to the resource matching results, dynamically allocate tasks to appropriate computing nodes, monitor the allocation and execution status of tasks, ensure that each task can run efficiently on the most suitable node, and use monitoring tools to continuously observe the execution of tasks and the resource usage of computing nodes.Monitoring indicators include CPU and memory usage, task execution time, etc. If it is found that some nodes have uneven resource usage or task processing delays, the system should be able to quickly re-evaluate the resource allocation strategy and reallocate tasks when necessary to ensure the efficient operation of the system.

[0049] In this embodiment, the load scheduling module is used to perform task resource allocation evolution based on the dynamic computing resource allocation strategy, and then perform adaptive load scheduling optimization to build a load scheduling optimization model, which is specifically used for: Based on the dynamic computing resource allocation strategy, the distributed server node network is allocated with task resources, and task resource allocation evolution data is generated; Perform computing node load analysis on the task resource allocation simulation data to obtain the load value of each computing node; Performing a node load peak analysis on the load value of each computing node to generate a node load peak value; Based on the node load peak, the dynamic computing resource allocation strategy is adaptively optimized for load scheduling and a load scheduling optimization model is constructed.

[0050] In this embodiment, the previously designed dynamic computing resource allocation strategy is used to perform real-time task allocation by monitoring the current task requirements and the idle resources of the computing nodes. During the allocation process, the priority of the task, the load of the node, and the availability of resources are considered to flexibly adjust the resource allocation. After each resource allocation, the allocation of each task is recorded, including the task ID, the allocated computing node ID, the amount of allocated resources (such as CPU and memory), etc. This information is stored in a database or log system to form a structured data format for subsequent analysis. The real-time load data of each computing node is collected from a monitoring system (such as Prometheus, Grafana, or a custom monitoring tool), which includes CPU usage, memory usage, I / O operations, etc., to ensure that the timestamp information of the data is accurate for subsequent analysis. The load value of each node is calculated based on the collected data. The load value can be obtained by the following formula: Load value = CPU usage + memory usage + I / O Waiting time (weighted according to specific circumstances) records the load value of each node to form a load analysis data set. Analyze the load value of each computing node within a certain time window to identify the peak value of the load. Statistical methods (such as moving average, maximum value) can be used to identify the peak value. Set a suitable time window (such as 5 The maximum load value is calculated within this window, and the load peak of each node is recorded to form a node load peak data set. This data set will be used for subsequent load scheduling optimization. Based on the node load peak data, an adaptive load scheduling optimization model is designed. The model can use machine learning algorithms (such as regression models, decision trees, random forests, etc.) to predict load changes and resource requirements. The model input can include historical load data, task characteristics, node status, etc. The output is the best resource allocation strategy for each node. The optimization model is trained using historical data sets (including task resource allocation evolution data and node load data). The model is cross-validated to evaluate its accuracy and generalization ability, and the model parameters are adjusted to improve prediction accuracy and scheduling efficiency. In the actual operation process, the optimization model is used to adjust the resource allocation strategy in real time. When it is monitored that the load of a node is close to the peak, the system should automatically adjust the task allocation, reduce the number of tasks on nodes with excessively high loads, or migrate tasks to nodes with lower loads, monitor the scheduling effect, and continuously adjust and optimize the model based on feedback.

[0051] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0052] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A server resource scheduling system integrating AI and edge computing, characterized in that: The server resource scheduling system integrating AI and edge computing includes a computing node module, a resource calling module, a task traversal module, a demand prediction module, a resource allocation module and a load scheduling module: A computing node module is used to monitor the real-time operation status of the server computing nodes, perform distributed node architecture fitting, and build a distributed server node network; Resource call module, used to obtain server operation logs; Analyze the server resource call behavior of the server operation log, and perform resource call timing feature evolution to generate server resource call timing features; The task traversal module is used to traverse and identify unprocessed tasks in the server operation log, mine task status features, and build an unprocessed task status mapping sequence; The demand prediction module is used to perform dynamic task processing demand prediction on the unprocessed task state mapping sequence according to the server resource call timing characteristics to obtain dynamic task processing demand prediction data; The resource allocation module is used to match the dynamic task processing demand forecast data with available resources according to the distributed server node network, and to perform dynamic computing resource allocation, thereby obtaining a dynamic computing resource allocation strategy; The load scheduling module is used to evolve task resource allocation based on dynamic computing resource allocation strategy, and then perform adaptive load scheduling optimization to build a load scheduling optimization model.

2. The server resource scheduling system integrating AI and edge computing according to claim 1, characterized in that: The computing node module is used to monitor the real-time operation status of the server computing node, perform distributed node architecture fitting, and build a distributed server node network, specifically for: Identify all server computing nodes; Performing terminal device type identification on the server computing node, marking the central device computing node and the edge device computing node; Perform real-time operation status monitoring on the central device computing nodes and the edge device computing nodes to obtain the central node operation status parameters and the edge node operation status parameters; Perform node load balancing feature analysis on the central node operation status parameters and the edge node operation status parameters to obtain the central node load balancing feature and the edge node load balancing feature; According to the load balancing characteristics of the central node and the edge node, the distributed node architecture is fitted to the terminal device computing nodes and the edge device computing nodes to build a distributed server node network.

3. The server resource scheduling system integrating AI and edge computing according to claim 2, characterized in that: The specific steps of performing node load balancing feature analysis on the central node operation status parameters and the edge node operation status parameters to obtain the central node load balancing feature and the edge node load balancing feature are as follows: The CPU utilization of each node is calculated for the central node operation status parameters and the edge node operation status parameters to obtain the CPU utilization of each node; Perform utilization fluctuation analysis on the CPU utilization of each node to generate node utilization fluctuation characteristics; Perform real-time memory usage calculation on the central node operating status parameters and edge node operating status parameters to obtain the memory usage value of each node; Analyze the operating status parameters of the central node and the network throughput of the edge nodes, and extract the network throughput of each node; Analyze the current load characteristics of nodes by considering the fluctuation characteristics of node utilization, the memory usage of each node, and the network throughput of each node, so as to obtain the current load characteristics of each node. Perform node type identification on the current load characteristics of each node, extract the load characteristics of each central node and each edge node; Perform load difference identification on the load characteristics of each central node and each edge node, and extract the load difference data between the central node and the edge node; The node load balancing characteristics are analyzed based on the load difference data of the central node and the edge node to obtain the load balancing characteristics of the central node and the edge node.

4. The server resource scheduling system integrating AI and edge computing according to claim 1, characterized in that: The resource calling module is used to obtain the server operation log; analyze the server resource calling behavior of the server operation log, and perform resource calling time series feature evolution to generate server resource calling time series features, which are specifically used for: Obtain server operation logs; analyze server resource call behavior on server operation logs and extract server resource call behavior data; Divide the server resource call behavior data into time series windows to obtain resource call behavior data of multiple time windows; The resource call behavior data of multiple time windows are counted for scheduling times to obtain the scheduling times of each time window; The scheduling frequency is calculated according to the number of scheduling times in each time window to generate the resource calling frequency of each window; The resource call frequency of each window is used to evolve the resource call timing characteristics and generate the server resource call timing characteristics.

5. The server resource scheduling system integrating AI and edge computing according to claim 1, characterized in that: The task traversal module is used to traverse and identify unprocessed tasks in the server operation log, mine task status features, and build an unprocessed task status mapping sequence. It is specifically used for: Traverse and identify unprocessed tasks in the server operation logs and extract unprocessed tasks from multiple servers; Calculate the remaining time for task processing of unprocessed tasks on multiple servers and extract the timestamps of unprocessed tasks; Calculate the task resource usage of unprocessed tasks on the server and extract task resource usage data; Perform task status feature mining on the unprocessed task timestamps and task resource occupancy data to generate each unprocessed task status feature; Performing task sequence encoding on multiple unprocessed tasks of the servers to obtain an unprocessed task encoding sequence; According to each unprocessed task state feature, the unprocessed task encoding sequence is mapped to the corresponding task state, and an unprocessed task state mapping sequence is constructed.

6. The server resource scheduling system integrating AI and edge computing according to claim 1, characterized in that: The demand prediction module is used to perform dynamic task processing demand prediction on the unprocessed task state mapping sequence according to the server resource call timing characteristics to obtain dynamic task processing demand prediction data, which is specifically used for: Perform multi-time point resource call prediction on the server resource call timing characteristics, thereby generating resource call prediction data at multiple time points; Perform resource call time series change analysis on resource call prediction data at multiple time points to obtain resource call time series change data; Perform task-by-task resource requirement analysis on the unprocessed task status mapping sequence to obtain resource requirement data for each task; Based on the resource call timing change data, dynamic task processing demand prediction is performed on the resource demand data of each task to obtain dynamic task processing demand prediction data.

7. The server resource scheduling system integrating AI and edge computing according to claim 6, characterized in that: The specific steps of performing multi-time point resource call prediction on the server resource call time series characteristics to generate resource call prediction data at multiple time points are: Perform multi-time-point segmentation processing on the server resource call timing characteristics to generate resource call timing data at multiple time points; Extract timestamps of resource call timing data at multiple time points; Performing multi-time point timing fitting according to the timestamp to obtain a multi-time point resource call timing curve; Perform deep call feature learning on resource call timing curves at multiple time points to generate server resource call rules; Based on the server resource call rules, short-term resource call simulation is performed on the server resource call timing characteristics to generate resource call simulation data; Define the forecast time node; Resource call prediction is performed on the resource call simulation data according to the prediction time node to generate resource call prediction data at multiple time points.

8. The server resource scheduling system integrating AI and edge computing according to claim 1, characterized in that: The resource allocation module is used to match the dynamic task processing demand forecast data with available resources according to the distributed server node network, and to dynamically allocate computing resources, thereby obtaining a dynamic computing resource allocation strategy, which is specifically used for: Identifying the callable computing nodes on the distributed server node network to mark the callable computing nodes; Calculate the idle resources of the callable computing nodes and extract the idle resource data of each computing node; Based on the idle resource data of each computing node, the dynamic task processing demand forecast data is matched with available resources to obtain task processing resource matching data; Based on the task processing resource matching data, the distributed server node network is dynamically allocated computing resources to obtain a dynamic computing resource allocation strategy.

9. The server resource scheduling system integrating AI and edge computing according to claim 1, characterized in that: The load scheduling module is used to perform task resource allocation evolution based on the dynamic computing resource allocation strategy, and then perform adaptive load scheduling optimization to build a load scheduling optimization model. It is specifically used for: Based on the dynamic computing resource allocation strategy, the distributed server node network is allocated with task resources, and task resource allocation evolution data is generated; Perform computing node load analysis on the task resource allocation simulation data to obtain the load value of each computing node; Performing a node load peak analysis on the load value of each computing node to generate a node load peak value; Based on the node load peak, the dynamic computing resource allocation strategy is adaptively optimized for load scheduling and a load scheduling optimization model is constructed.

Images