Intelligent scheduling method and system for heterogeneous equipment system

Through edge computing and distributed learning technology, data preprocessing and model training are carried out in heterogeneous device systems, combined with rapid local optimization and periodic global optimization strategies, the problem of inefficient resource scheduling in heterogeneous device systems is solved, and resources are realized The efficient utilization and continuous optimization of scheduling strategies have improved the overall performance of the system and data privacy security.

CN118939438APending Publication Date: 2024-11-12HANGZHOU DONGBEI INTELLIGENT COMPUTING TECH CO LTD

Patent Information

Application Number
CN202411429048.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-14
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In heterogeneous equipment systems, existing resource scheduling methods are difficult to effectively collect and process real-time state data, build prediction models that adapt to the dynamic changes of the system, and realize efficient resource utilization and dynamic adjustment of scheduling strategies, resulting in uneven resource allocation and task processing efficiency low.

Method used

Data preprocessing is performed using edge computing technology, and predictive model training is carried out in parallel on multiple edge nodes using distributed learning technology. Combining fast local optimization and periodic global optimization strategies, resource allocation schemes are generated and performed through execution and monitoring modules. Continuous optimization and adjustment.

Benefits of technology

It improves the system response speed and overall efficiency, ensures the best use of resources, enhances the accuracy and timeliness of the prediction model, realizes continuous optimization of scheduling strategies, and ensures data privacy and security.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939438A_ABST
    Figure CN118939438A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of computer system resource scheduling, and particularly discloses an intelligent scheduling method and system for a heterogeneous device system.The method comprises the steps that running state data deployed on heterogeneous devices are obtained, and the collected data are preprocessed through the edge computing technology; carrying out prediction model training in parallel on a plurality of edge nodes by utilizing a distributed learning technology; calculating and generating a locally optimal resource allocation scheme through rapid local optimization and periodic global optimization; the periodic global optimization strategy adopts a deep reinforcement learning algorithm, the state and performance of the heterogeneous equipment system are evaluated regularly, and a resource allocation strategy is adjusted; the improved execution and monitoring module is used for implementing a scheduling scheme, continuously monitoring an execution effect and dynamically adjusting a resource allocation and scheduling strategy through a feedback mechanism; through the contents, the efficiency and the accuracy of scheduling decision making are remarkably improved, and continuous optimization and high-performance performance of the system are ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of resource scheduling in computer systems, and in particular to an intelligent scheduling method and system for heterogeneous device systems. Background Art

[0002] With the development of information technology, heterogeneous device systems have been widely used in many fields due to their flexibility and scalability. Heterogeneous device systems are usually composed of multiple types of hardware devices, such as high-performance servers, edge computing nodes, IoT devices, and mobile terminals. These devices have significant differences in processing power, storage capacity, and network bandwidth, so they face many challenges in resource scheduling.

[0003] Traditional resource scheduling methods often adopt centralized scheduling strategies, which have limitations when dealing with large-scale, highly heterogeneous equipment systems, such as high scheduling delays and heavy computing burdens on central nodes. In addition, with the increase in the number of devices and the diversification of task requirements, a single global optimization strategy is difficult to take into account the needs of local optimization, resulting in uneven resource allocation and inefficient task processing.

[0004] In recent years, the application of edge computing technology and distributed learning technology has provided new ideas for solving the above problems. Edge computing can perform preliminary processing close to the data source, reduce data transmission delays and reduce the computing burden of the central node. Distributed learning technology allows model training on each edge node, and then summarizes the model parameters for global optimization, which not only protects data privacy but also improves the efficiency of model training. However, existing resource scheduling methods still face the following major problems: (1) How to effectively collect and process real-time status data from different heterogeneous devices to support rapid scheduling decisions; (2) How to build a prediction model that can adapt to dynamic changes in the system and support real-time updates to improve the accuracy and timeliness of scheduling decisions; (3) How to achieve efficient resource utilization while ensuring computing efficiency, especially when dealing with large-scale heterogeneous device systems; (4) How to dynamically adjust scheduling strategies based on real-time feedback to cope with changes in system load and task requirements.

[0005] Therefore, there is an urgent need for an intelligent scheduling method and system that can effectively solve the above problems, so as to improve the resource scheduling efficiency and task processing capability in heterogeneous device systems. Summary of the invention

[0006] The present invention provides an intelligent scheduling method and system for a heterogeneous device system to solve the problem of how to achieve efficient resource scheduling and task allocation in a heterogeneous device system to improve the overall performance and resource utilization of the system.

[0007] In order to solve the above problems, the present invention provides an intelligent scheduling method and system for a heterogeneous device system, including: An intelligent scheduling method for a heterogeneous equipment system, comprising: Obtain the operating status data deployed on heterogeneous devices and pre-process the collected data using edge computing technology; Based on the preprocessed data, training the prediction model in parallel on multiple edge nodes using distributed learning technology; Under the guidance of the trained prediction model, the local optimal resource allocation plan is calculated and generated through rapid local optimization and periodic global optimization. The rapid local optimization strategy adopts a dynamic programming algorithm based on the Bellman equation; the periodic global optimization strategy adopts a deep reinforcement learning algorithm to regularly evaluate the status and performance of the heterogeneous device system and adjust the resource allocation strategy; According to the generated resource allocation plan, the execution and monitoring module is improved, the scheduling plan is implemented and the execution effect is continuously monitored, and the resource allocation and scheduling strategy is dynamically adjusted through the feedback mechanism.

[0008] As a preferred solution, the intelligent scheduling method for the heterogeneous equipment system further includes: Through sensors and monitoring tools deployed on heterogeneous devices, the operating status data of the devices is continuously collected, including CPU utilization, memory usage, network traffic, storage usage, temperature, and power consumption indicators; Preprocess the received operating status data, including data cleaning, format conversion, anomaly detection and data compression; Using edge computing technology, the data is summarized into summary information and input into the central scheduling system for use in global optimization strategies.

[0009] As a preferred solution, the intelligent scheduling method for the heterogeneous equipment system further includes: Using distributed learning technology, the model training process is dispersed to multiple edge nodes. Each node uses local data to train the model, and then the model parameters are aggregated to the central node. The trained prediction model receives the preprocessed input data, makes real-time predictions, and monitors the actual effectiveness of the prediction results; Choose incremental learning to update the model and test it on an independent validation set.

[0010] As a preferred solution, the intelligent scheduling method for the heterogeneous equipment system further includes: Performing rapid local optimization and periodic global optimization on the test data; Dynamically adjust resource allocation strategies based on prediction results and current system status, including task allocation, load balancing, and priority sorting; Use the variational method in functional analysis to solve the functional extremum problem.

[0011] As a preferred solution, the intelligent scheduling method for the heterogeneous equipment system further includes: Continuously monitor the execution of scheduling decisions, collect execution results and system feedback, including task completion time, resource utilization efficiency, and system performance indicators, to evaluate the effectiveness of scheduling strategies; Based on monitoring data, identify deficiencies or bottlenecks in scheduling strategies and dynamically adjust optimization parameters, including resource reservations, task priorities, and algorithm parameters; Based on the results of environmental perception, the module dynamically adjusts the scheduling strategy.

[0012] As a preferred solution, the preprocessing module performs standardization or normalization on the data to convert all values ​​to the same scale.

[0013] As a preferred solution, the edge nodes use local data to train the model, and then aggregate the model parameters to the central node for aggregation.

[0014] As a preferred solution, the fast local optimization strategy is used to analyze the device status and task requirements in real time to calculate the local optimal resource allocation solution, and perform task allocation, load balancing and priority sorting during task scheduling.

[0015] As a preferred solution, the scheduling solution monitors the execution of the resource allocation solution and makes feedback adjustments according to the actual operation of the system to optimize the scheduling efficiency.

[0016] As a preferred solution, the scheduling scheme is dynamically adjusted based on the task completion time, resource utilization efficiency and system performance indicators collected by the monitoring module to ensure continuous optimization of the system.

[0017] The present invention proposes an intelligent scheduling method and system for a heterogeneous equipment system, which has the following beneficial effects: (1) Improved the response speed and overall efficiency of the system: Preliminary data processing and real-time status perception are performed through edge computing technology, which reduces the computing burden of the central node and reduces data transmission delay, thereby improving the response speed and overall efficiency of the system.

[0018] (2) Improved training efficiency and accuracy of the prediction model: Distributed learning technology is used to disperse the model training process to multiple edge nodes, which not only improves the efficiency of model training, but also ensures the accuracy of the prediction model through a real-time update mechanism, and can make timely adjustments even in the face of changes in system status.

[0019] (3) Optimal utilization of resources: By combining fast local optimization and periodic global optimization strategies, a balance between computational efficiency and optimization effect is achieved, ensuring efficient utilization of resources, improving the timeliness of task completion and the overall utilization of system resources.

[0020] (4) Ensure continuous optimization of scheduling strategies: By improving the execution and monitoring modules and adding an adaptive adjustment mechanism, the scheduling strategy can be dynamically adjusted based on real-time feedback, ensuring continuous optimization of system performance and making scheduling decisions more flexible and efficient.

[0021] (5) Simplified data processing process: The preprocessing module cleans the raw data, converts the format, detects anomalies, and compresses the data, ensuring the consistency and integrity of the data quality and reducing the data processing burden of the central node.

[0022] (6) Guaranteed data privacy and security: During the distributed learning process, edge nodes only upload model parameters instead of transmitting original data, thus protecting data privacy. At the same time, data encryption and compression technology are used to further enhance the security of data transmission.

[0023] (7) Improved prediction accuracy and timeliness: By periodically or event-driven updating of the prediction model, the accuracy and timeliness of the prediction results are ensured, making scheduling decisions more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of an intelligent scheduling method for a heterogeneous device system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] Reference Figure 1 , is a flow chart of an intelligent scheduling method and system for a heterogeneous device system according to an embodiment of the present invention. The intelligent scheduling method and system for a heterogeneous device system can at least include steps S100-S400: S100, combined with the data acquisition and preprocessing modules, uses edge computing to perform preliminary data processing and real-time status perception to reduce the computing burden of the central node.

[0027] S200, upgrade the prediction and modeling module, adopt distributed learning technology to improve model training efficiency and prediction accuracy, and support real-time updating of the model.

[0028] S300 integrates the intelligent scheduling decision module and adopts a hybrid optimization strategy, combining fast local optimization and periodic global optimization to achieve a balance between computing efficiency and optimization effect.

[0029] S400, improve the execution and monitoring modules, add an adaptive adjustment mechanism, dynamically adjust the scheduling strategy based on real-time feedback, and ensure continuous optimization of system performance.

[0030] An intelligent scheduling method and system for a heterogeneous device system provided in an embodiment of the present invention are mainly used to solve the technical problem solved by the present invention, which is "how to achieve efficient resource scheduling and task allocation in a heterogeneous device system to improve the overall performance and resource utilization of the system."

[0031] The present invention relates to the technical field of computer system resource scheduling, and specifically discloses an intelligent scheduling method and system for a heterogeneous device system, the method comprising obtaining operating status data deployed on heterogeneous devices, preprocessing the collected data using edge computing technology; training prediction models in parallel on multiple edge nodes using distributed learning technology; performing rapid local optimization and periodic global optimization on training data, and running a dynamic adjustment resource allocation strategy; improving the execution and monitoring module according to the scheduling scheme generated by the intelligent scheduling decision module, implementing resource allocation decisions, and continuously monitoring the execution effect. Through these contents, the present invention significantly improves the efficiency and accuracy of scheduling decisions, ensuring continuous optimization and high performance of the system.

[0032] S100: Combine the data collection and preprocessing modules, use edge computing to perform preliminary data processing and real-time status perception, and reduce the computing burden of the central node. Step S100 includes steps S110-S130: S110, through the sensors and monitoring tools deployed on heterogeneous devices, continuously collect the operating status data of the devices, including CPU utilization, memory usage, network traffic, storage usage, temperature, and power consumption indicators. Specifically: High-precision sensors and specially designed monitoring software are pre-installed on various heterogeneous devices, such as high-performance servers, edge computing nodes, IoT devices, and mobile terminals. These sensors can capture the physical status of the device, such as temperature and power consumption, while the monitoring software is responsible for monitoring the logical operating status of the device, such as CPU utilization, memory usage, network traffic, and storage usage.

[0033] Continuously collect data at set intervals (e.g., every second or every minute). The collected data points cover the key performance indicators (KPIs) of the device, such as CPU load, RAM usage, hard disk I / O rate, and network bandwidth usage.

[0034] S120: pre-process the received raw data, including data cleaning, format conversion, anomaly detection and data compression. Specifically: The module first denoises the received raw data to remove outliers or noise data caused by device failure, network fluctuations, or sensor distortion. For example, if the CPU utilization record at a certain moment is negative or exceeds a reasonable range, these data points will be marked as abnormal and removed from the data set. There may be missing values ​​in the data due to network delays or intermittent device failures. The preprocessing module uses interpolation methods or statistical methods based on neighboring data points to estimate missing values ​​to ensure the integrity of the data set.

[0035] The preprocessing module standardizes or normalizes the data, converting all values ​​to the same scale, such as converting the CPU utilization of all devices to a value between 0 and 1, to facilitate subsequent comparison and analysis. This solves the problem that the format and dimension of the data may not be uniform because the data comes from different heterogeneous devices. For non-numeric data, such as device type or task status, the preprocessing module converts it into a numerical code to facilitate processing by machine learning algorithms.

[0036] The module uses statistical methods such as Z-score or IQR (interquartile range) to identify data points that deviate from the normal distribution. Based on historical data and pattern recognition algorithms, the preprocessing module can identify abnormal patterns in the data, such as an abnormal surge in CPU utilization in a short period of time, which may indicate system overload or attack behavior.

[0037] Through principal component analysis (PCA) or other dimensionality reduction techniques, the module can identify the most important features in the data set, remove redundant information, reduce the dimension of the data set, and reduce the processing burden of the central node. Using compression algorithms such as Huffman coding and LZ77 / LZ78 (such as gzip), the module can reduce the volume of data transmission and speed up data transmission.

[0038] S130, using edge computing technology, the module performs preliminary data processing and analysis, reduces data transmission delays, and improves the system's response speed and overall efficiency. Specifically: Deploy edge computing nodes near heterogeneous devices or groups of devices. These nodes are equipped with computing resources such as CPU, memory, and storage to perform preliminary data processing and analysis tasks. Edge nodes collect real-time data from devices within their coverage area, such as CPU utilization, memory usage, network traffic, storage usage, temperature, and power consumption indicators.

[0039] After receiving the data, the edge node immediately performs data preprocessing, including denoising, format conversion, anomaly detection, and data compression operations to ensure data quality and consistency. The preprocessed data is then used for local analysis, such as predicting upcoming load patterns through machine learning models, or identifying whether the current device status is within the normal range. Based on the results of local analysis, the edge node can respond quickly and perform predictions and preliminary scheduling decisions. For example, if it detects that a device is about to be overloaded, the edge node can immediately transfer part of the task to other devices to balance the load.

[0040] The edge nodes summarize the processed and analyzed data into summary information, which contains key insights and decision-making recommendations rather than the original large amount of data. The summary data is then efficiently transmitted to the central scheduling system for use in global optimization strategies. The edge nodes maintain two-way communication with the central scheduling system, which can not only upload local analysis results, but also receive update instructions from the central scheduling to achieve dynamic adjustment and optimization.

[0041] S200, upgrade the prediction and modeling module, adopt distributed learning technology, improve model training efficiency and prediction accuracy, and support real-time update of the model. Step S200 at least includes steps S210-S230: S210, using distributed learning technology, the model training process is distributed to multiple edge nodes, each node can use local data for model training, and then aggregate the model parameters to the central node for aggregation. After the edge node completes the preliminary data processing and analysis in S130, the distributed model training process will be started, specifically: After completing data preprocessing and local analysis (S130), the edge node uses the local data set to perform further training on the basis of the initialized global model. The data used by the edge node includes not only the real-time operation data of the local device, but also the analysis summary results generated in S130. These data optimize the model parameters through the local training process, thereby improving the prediction accuracy of the model.

[0042] After local training is completed, the edge nodes do not upload the original data, but upload the updated model parameters (weights and biases) to the central node. The central node aggregates the model parameters from each edge node and merges them to update the global model using a federated averaging algorithm or other aggregation strategies.

[0043] The updated global model parameters are distributed to all edge nodes again and enter the next round of training. The cycle stops when the global model reaches the predetermined performance target or the training rounds reach the upper limit. The process ensures the timeliness and adaptability of the model, enabling it to effectively cope with dynamic changes in the system.

[0044] The trained model is deployed to the intelligent scheduling decision module and applied to resource allocation decisions in real time, while receiving continuous feedback from edge nodes and central systems to continuously optimize the scheduling strategy.

[0045] S220. Based on the trained model, the module can predict the key indicators of system load, task execution time, and resource requirements in real time, providing forward-looking information for scheduling decisions. The prediction results can guide the pre-allocation of resources, reduce waiting time, and improve system response speed. Specifically: The module receives data from the intelligent perception and edge preprocessing modules in real time, including CPU utilization, memory usage, network traffic, storage usage, temperature, power consumption indicators, as well as the current status of the device and task queue information. After receiving the data, it is preprocessed, including standardization, format conversion and feature engineering, to ensure that the data format is consistent with the data format used during model training so that the model can be used directly.

[0046] Based on the feature importance determined during model training, the module selects the most relevant features as input to the model. This may include current system load, recent task execution history, and device status. The input data is organized into the format required by the model, usually in vector or matrix form, to facilitate the model to make predictions.

[0047] The trained prediction model receives the preprocessed input data and makes real-time predictions. The prediction results include but are not limited to the future load forecast of the system, the time and resource estimation required for the tasks to be executed, and the possible resource demand forecast. The prediction results output by the model are interpreted and transformed into a form that is easy to understand and apply. For example, the system load forecast may be converted into the expected CPU and memory requirements in the next few hours. This prediction information is immediately passed to the hybrid intelligent scheduling decision module as part of its decision-making process to guide the pre-allocation of resources, reduce waiting time, and improve system responsiveness.

[0048] The actual effectiveness of the prediction results is monitored and evaluated, and compared with the actual situation to check the accuracy of the prediction. If the prediction results deviate significantly from the actual system state, the module triggers model retraining or parameter fine-tuning to improve the accuracy of future predictions.

[0049] S230: As the system status changes and new data flows in, the module can automatically update the prediction model to adapt to the dynamic changes in the environment. Model updates can be periodic or event-driven to ensure the accuracy and timeliness of the prediction. Specifically: The module continuously receives real-time data from the intelligent perception and edge preprocessing modules, including CPU utilization, memory usage, network traffic, storage usage, temperature, and power consumption, which constitute the input for model updates. The data is preprocessed immediately after receipt, including data cleaning, standardization, and feature extraction steps to ensure data quality and format consistency.

[0050] The model conducts self-evaluation periodically or when certain events occur (such as when the prediction error exceeds a threshold or when the system state changes significantly) to check how well the prediction matches the actual system state. If the evaluation results show that the model performance has degraded or is no longer applicable to the current system state, the module will trigger the model update process.

[0051] Depending on the specific design of the model and the current amount of data, the module can choose to update the model in an incremental learning manner, that is, fine-tune the original model using new data to save computing resources. Alternatively, if the system state changes significantly or the model architecture needs to be adjusted, the module may choose to completely retrain the model, using the latest data set to train from scratch to ensure the timeliness and accuracy of the model.

[0052] If the federated learning distributed learning framework is used, the model update process will involve distributing the update task to multiple edge nodes, each node uses local data to train or fine-tune the model, and then uploads the updated model parameters to the central node. The central node collects the parameter updates of all edge nodes, aggregates the parameters using the federated averaging algorithm, and generates a globally updated model.

[0053] The updated model needs to be tested on an independent validation set to ensure that its generalization ability and prediction accuracy meet the requirements. After passing the validation, the updated model will replace the old model and become the latest version in the real-time prediction task, continuing to support intelligent scheduling decisions.

[0054] S300, integrating the intelligent scheduling decision module, adopting a hybrid optimization strategy, combining fast local optimization and periodic global optimization, to achieve a balance between computing efficiency and optimization effect. Step S300 at least includes steps S310-S330: S310: In the scheduling decision process after the model is updated, a hybrid optimization strategy is adopted, combining fast local optimization and periodic global optimization to balance computing efficiency and optimization effect, specifically: Under the system state predicted by the model, rapid local optimization is immediately performed. Using the real-time data and prediction results provided by the model, the module can quickly calculate the local optimal resource allocation plan to cope with the current system load changes. Furthermore, the module regularly executes the global optimization strategy, and globally optimizes the resource allocation of the entire system through deep reinforcement learning and functional analysis methods. The coordination of the local optimization and global optimization can dynamically adjust the resource allocation strategy according to the latest model prediction results, thereby improving the balance between the overall efficiency and computational complexity of the system. When executing the hybrid optimization strategy, the module continuously monitors the execution of the system and collects feedback data, including task completion time, resource utilization efficiency, system performance indicators, etc. Based on these feedback data, the module can further fine-tune the optimization strategy to ensure continuous improvement of system performance.

[0055] 1. Original algorithm formula designed Assume that there are N heterogeneous devices in the system, and the current state of each device i is represented by a set of state variables, including CPU utilization , memory usage , network bandwidth usage At the same time, we have M tasks to be executed, and the attributes of each task j include the required CPU time , memory required , the required network bandwidth Our goal is to maximize the overall system resource utilization while ensuring that task completion time is minimized.

[0056] ① In order to achieve fast local optimization, we use a new equation to guide immediate scheduling decisions. Let V(s) be the value function of the device in state s, A(s) be the set of actions taken in state s, R(s, ) To take action The immediate reward obtained after , γ is the discount factor.

[0057]

[0058] where s' is the action taken The next state that the system reaches is the immediate reward R(s, ) is a weighted sum based on the device resource utilization, V(s') represents the device taking action The value function of the new state s' arrived later, for example:

[0059] Here , , It is the weight coefficient, which represents the contribution of CPU, memory, and network bandwidth utilization to the instant reward.

[0060] ② In order to achieve periodic global optimization, we use a new algorithm to update the Q table. ) means taking action in state s Expected long-term reward:

[0061] in, is the learning rate, s' is the action After the status. represents one of the actions taken in the new state s'. Q(s', ) means taking action in state s' expected long-term rewards.

[0062] ③ In order to more accurately predict task requirements and equipment performance, we use a multivariate regression model, which inputs include the historical status data of the equipment and task attributes, and outputs the predicted task execution time and resource requirements. Let X be the input feature matrix, Y be the output vector, and β be the regression coefficient, then the multivariate linear regression model is:

[0063] Among them, ϵ is the error term, which represents the influencing factors that the model fails to capture.

[0064] (ii) The operation process of this link: In the hybrid intelligent scheduling decision module, we first use a fast local optimization strategy to respond to immediate changes in device status and adjust resource allocation. Subsequently, in the periodic global optimization stage, we use new algorithms to optimize the long-term trends of equipment and tasks, update the Q table, and guide longer-term resource allocation strategies. At the same time, through multivariate regression analysis, we predict future task requirements and equipment performance to provide a more accurate basis for scheduling decisions. Through the combination of the above algorithms, we achieve rapid response to local changes while regularly optimizing global resources to achieve a balance between overall efficiency and computational complexity.

[0065] S320: Based on the prediction results and the current system status, the module dynamically adjusts the resource allocation strategy, such as task allocation, load balancing, and priority sorting, to achieve optimal resource utilization. Specifically: 1. Designing an original algorithm formula ① First, we build an original multivariate linear regression model to predict the resource requirements of each task i, including CPU time ,Memory and network bandwidth The model input includes task attributes, such as task type , task size , and system state variables, such as the current time t. The model output is the resource demand forecast.

[0066]

[0067] in, , , is the intercept term, , , is the coefficient of the task type, , , is the coefficient of task size, , , is the coefficient of the current time, , , is the error term.

[0068] ② We build a new model to represent the resource availability of device j, including the CPU surplus 、Memory remaining and network bandwidth remaining Model inputs include device status, such as current CPU usage , memory usage , network bandwidth usage , and the maximum resource of the device , , .

[0069]

[0070] ③ We define the resource allocation optimization problem, the goal is to minimize the total task delay time T while satisfying resource constraints. We use a new algorithm to solve this problem.

[0071]

[0072] in, , , is the processing speed of task i, m is the total number of devices, and n is the total number of tasks. Subject to refers to constraints, that is, a series of restrictions that must be met in the optimization problem; minimize refers to minimization, that is, finding a set of variable values ​​that minimizes the objective function.

[0073] ④ Construct a new function L and use partial derivatives to find the optimal solution.

[0074]

[0075] in, , , The Lagrange multipliers corresponding to the three constraints of CPU time, memory and network bandwidth are used to ensure that the objective function of the optimization problem reaches the optimal value while satisfying all constraints when constructing the Lagrangian function. By setting the partial derivative of the new function with respect to each variable equal to zero, we can solve the optimal resource allocation solution.

[0076] 2. Resource allocation operation process ① Collect task attributes and system status data, and use the resource demand prediction model to predict the resource requirements of each task.

[0077] ②Monitor device status and calculate resource availability for each device.

[0078] ③Construct the resource allocation optimization problem, including the objective function and constraints.

[0079] ④ Use the new multiplier method to solve the resource allocation optimization problem and obtain the optimal resource allocation plan.

[0080] ⑤ According to the optimal plan, allocate resources to tasks and execute scheduling decisions.

[0081] ⑥ Monitor system status and task execution, and adjust prediction models and optimization problems based on real-time data to adapt to system changes.

[0082] S330: Once the scheduling decision is generated, the module will send instructions to the corresponding devices or execution units to coordinate the use of resources and ensure the smooth execution of the scheduling decision. 1. Algorithm formula design Consider the state of device j represented by the complex Indicates that Represents the physical resource status of the device (such as CPU, memory, network), represents the virtual resource status of the device (such as virtualization efficiency, concurrent processing capability). Similarly, the demand of task i is represented by the plural Indicates that represents the physical resource requirements of the task, Represents the virtual resource requirements of the task.

[0083] ① Define the functional F as the mapping between device status and task requirements, expressed as:

[0084] Where Ω is an interval defined on time τ, f is an integrable function defined on Ω, and dμ(τ) is a measure representing the weight distribution of time.

[0085] ② Find the device state that minimizes the functional F and mission requirements The pairing is:

[0086] ③ To ensure that device resources are not over-allocated, add constraints:

[0087] This means that the total amount of the device's resources is greater than or equal to the task's requirements.

[0088] ④Use the variational method in functional analysis to solve the extreme value of functional F, and introduce the Lagrange multiplier λ to meet the constraints:

[0089] ⑤ Dynamically adjust device status during decision execution and mission requirements , to adapt to the changes in the system state and keep the functional F minimized.

[0090] 2. Decision-making and Implementation Process ① Calculate the initial state of all devices j and the initial requirements of task i .

[0091] ② Initialize the functional F according to the device status and task requirements.

[0092] ③ Use the variational method to solve the functional extremum problem and obtain the pairing of equipment state and task requirements that minimizes the functional F.

[0093] ④ According to the optimal solution, assign appropriate equipment j to each task i to ensure reasonable resource allocation.

[0094] ⑤ During task execution, the execution unit continuously monitors changes in device status and task requirements and collects feedback data.

[0095] ⑥Adjust equipment status according to feedback data and mission requirements , and re-solve the functional extreme value to adapt to the changes in the system state.

[0096] S400, improve the execution and monitoring module, add an adaptive adjustment mechanism, dynamically adjust the scheduling strategy according to real-time feedback, and ensure continuous optimization of system performance. Step S400 at least includes steps S410-S430: S410, the module continuously monitors the execution of scheduling decisions, collects execution results and system feedback, including task completion time, resource utilization efficiency, and system performance indicators, to evaluate the effectiveness of scheduling strategies. Specifically: ① After the execution monitoring module receives the decision execution instruction from the intelligent scheduling decision module, it immediately starts the real-time data collection mechanism. This includes but is not limited to collecting the following input data and indicators related to our invention patent: Task completion time: record the timestamp of each task from the start to the completion to evaluate the impact of the scheduling strategy on the overall task execution efficiency; Resource utilization efficiency: monitor the resource consumption of device j when processing task i, such as CPU utilization , memory usage and network bandwidth usage , where t represents time; System performance indicators: collect key indicators that reflect the overall health and performance of the system, such as system response time τ, throughput θ, and system load Λ.

[0097] ② The monitoring module uses the collected data to conduct real-time analysis and evaluation: Efficiency analysis: By comparing the actual completion time of the task with the expected completion time, the timeliness of the scheduling decision is evaluated; Resource consumption evaluation: Based on the equipment status and mission requirements Based on the matching results, analyze whether resource usage has reached the optimal configuration to avoid resource waste or shortage; System performance monitoring: monitor the changing trends of system performance indicators, identify potential bottlenecks and performance degradation points, and take optimization measures in advance.

[0098] ③ The monitoring results are fed back to the intelligent scheduling decision module in real time for dynamic adjustment of scheduling strategies. The specific operations are as follows: Immediate feedback: When it is detected that the resource utilization efficiency is lower than the preset threshold or the system performance indicators are abnormal, the decision module is immediately notified to intervene; Long-term trend analysis: Regularly summarize the execution monitoring data, analyze the long-term effect of the scheduling strategy, and provide data support for strategy optimization; Strategy adjustment suggestions: Based on the feedback from the execution monitoring, make suggestions for strategy adjustment, such as modifying the parameter settings of the functional F, or updating the matching rules between the equipment status and task requirements.

[0099] ④The execution monitoring module is a component of the continuous learning and self-improvement of the intelligent scheduling system. Through continuous monitoring, analysis and feedback, the system can gradually optimize the scheduling strategy, better balance the goals of global optimization and local optimization, and improve the overall resource utilization efficiency and task execution efficiency.

[0100] 420. Based on the monitoring data, the module can identify deficiencies or bottlenecks in the scheduling strategy and dynamically adjust optimization parameters, such as resource reservations, task priorities, and algorithm parameters, to continuously optimize system performance. Specifically: ① The adaptive adjustment module first receives and analyzes real-time data from the execution monitoring module, including but not limited to task completion time, resource utilization efficiency, and key information of system performance indicators. By comparing the actual execution results with the expected goals, the module identifies deficiencies or bottlenecks in the scheduling strategy, such as uneven resource allocation and task execution delays exceeding expectations.

[0101] ② Based on the balance principle between global optimization and local optimization described in the invention patent, the module evaluates the effectiveness of the current scheduling strategy. For example, if it is found that the completion time of some high-priority tasks is much longer than expected, it may be due to insufficient resource reservation or unreasonable priority allocation; if the system performance indicators show inefficient resource utilization, it may be due to improper algorithm parameter settings or the need to adjust the resource allocation strategy.

[0102] ③ After identifying the problem, the adaptive adjustment module will dynamically adjust the optimization parameters to correct the deficiencies in the scheduling strategy. This includes but is not limited to: Resource reservation adjustment: Based on the current load status of device j and the requirements of task i , adjust the resource reservation to ensure that key tasks receive sufficient resource support; re-order task priorities: re-evaluate task priorities based on the urgency of the task and resource requirements to ensure that important tasks are executed first and reduce waiting time; fine-tuning of algorithm parameters: optimize the parameters of the functional F, such as weight coefficients and penalty terms, to improve the performance of the scheduling algorithm, enhance the global optimization capability and control the computational complexity.

[0103] ④ The operation of the adaptive adjustment module is a continuous optimization cycle. The results of each adjustment will be monitored again to verify the effect of the adjustment and serve as the basis for the next round of optimization. The module will continue to learn and adapt to changes in the system environment to ensure that the scheduling strategy is dynamically optimized as the system status and task requirements change.

[0104] ⑤ During the adjustment process, the module pays special attention to maintaining a balance between global optimization and local optimization. On the one hand, by optimizing resource allocation and task scheduling strategies, the module seeks to maximize the overall system performance; on the other hand, it considers computing resource limitations and real-time response requirements to avoid efficiency reduction caused by overly complex calculations.

[0105] ⑥ The adjusted strategy and its execution results will be fed back to the intelligent scheduling decision module as a basis for subsequent decisions. If the new strategy significantly improves system performance or solves previous bottlenecks, these adjustments will be saved and become the standard for future scheduling decisions.

[0106] S430, the module continuously optimizes the scheduling strategy, and through learning and adaptation, continuously improves the efficiency and effect of resource scheduling, ensuring that the system can maintain or improve its performance level when facing dynamically changing loads and environments. Specifically: ① The performance optimization module conducts in-depth analysis based on real-time data from the execution monitoring module, including task completion time, resource utilization efficiency, and system performance indicators. These data not only reflect the effect of the current scheduling strategy, but also provide detailed information about the system status and resource usage, which is the basis for module learning and optimization.

[0107] ② The module continuously monitors changes in the system environment, including the frequency, type, resource requirements, and system load of task requests. By analyzing these dynamic changes, the performance optimization module can identify system performance bottlenecks or potential optimization opportunities. For example, when a certain type of computationally intensive task is detected to occur frequently, the module can predict and reserve the corresponding resources to reduce waiting time and improve response speed.

[0108] ③ Based on the results of environmental perception, the module dynamically adjusts the scheduling strategy to meet the performance requirements in different scenarios. The adjustment may include: Resource reservation and allocation: according to the resource requirements of the task and the current status of the device j , dynamically adjust the resource reservation to ensure that resource allocation meets task requirements and avoids waste; task priority adjustment: re-order task priorities according to the urgency of the task and resource requirements to ensure that high-priority tasks can obtain resources in time and be executed quickly; algorithm parameter optimization: according to system performance indicators and task execution results, adjust the parameters in the functional F, such as weight coefficients and penalty terms, to optimize the performance of the scheduling algorithm.

[0109] ④ The performance optimization module continuously learns and adapts to the dynamic characteristics of the system through machine learning algorithms. It not only adjusts parameters based on historical data, but also predicts future workload patterns, so as to make scheduling decisions in advance and reduce response time. The learning process of the module is an iterative process, and each scheduling decision and its results are incorporated into the learning model to continuously improve the accuracy and efficiency of decision-making.

[0110] ⑤ During the optimization process, the module pays special attention to balancing the relationship between global optimization and local optimization. By comprehensively considering global performance indicators and local resource utilization efficiency, the module can achieve efficient execution of local tasks without sacrificing the overall performance of the system. This balance is crucial to avoid excessive consumption of computing resources and improve the overall responsiveness of the system.

[0111] ⑥ The optimized scheduling strategy will be applied to the system, and its execution results and system performance indicators will be continuously monitored and evaluated. If the new strategy significantly improves system performance or solves pre-existing problems, these optimizations will be saved and applied in future scheduling decisions, forming a closed-loop optimization process.

[0112] The key innovative features of the present invention include: (1) Edge computing and data preprocessing: Edge computing nodes are deployed near heterogeneous devices to collect, preprocess, and perform preliminary analysis on data in real time to reduce the computing pressure on the central nodes.

[0113] (2) Distributed learning and model updating: Distributed learning technology is used for model training to ensure data privacy while improving model training efficiency. It also supports real-time model updates to adapt to changes in system status.

[0114] (3) Hybrid optimization strategy: Combining fast local optimization with periodic global optimization to ensure rapid response to local changes and regular global resource optimization.

[0115] (4) Adaptive scheduling strategy adjustment: Dynamically adjust the scheduling strategy based on real-time feedback to continuously optimize system performance and ensure the efficiency of resource allocation and task scheduling.

[0116] The present invention proposes an intelligent scheduling method and system for a heterogeneous equipment system, which has the following beneficial effects: (1) Improved the response speed and overall efficiency of the system: Preliminary data processing and real-time status perception are performed through edge computing technology, which reduces the computing burden of the central node and reduces data transmission delay, thereby improving the response speed and overall efficiency of the system.

[0117] (2) Improved training efficiency and accuracy of the prediction model: Distributed learning technology is used to disperse the model training process to multiple edge nodes, which not only improves the efficiency of model training, but also ensures the accuracy of the prediction model through a real-time update mechanism, and can make timely adjustments even in the face of changes in system status.

[0118] (3) Optimal utilization of resources: By combining fast local optimization and periodic global optimization strategies, a balance between computational efficiency and optimization effect is achieved, ensuring efficient utilization of resources, improving the timeliness of task completion and the overall utilization of system resources.

[0119] (4) Ensure continuous optimization of scheduling strategies: By improving the execution and monitoring modules and adding an adaptive adjustment mechanism, the scheduling strategy can be dynamically adjusted based on real-time feedback, ensuring continuous optimization of system performance and making scheduling decisions more flexible and efficient.

[0120] (5) Simplified data processing process: The preprocessing module cleans the raw data, converts the format, detects anomalies, and compresses the data, ensuring the consistency and integrity of the data quality and reducing the data processing burden of the central node.

[0121] (6) Guaranteed data privacy and security: During the distributed learning process, edge nodes only upload model parameters instead of transmitting original data, thus protecting data privacy. At the same time, data encryption and compression technology are used to further enhance the security of data transmission.

[0122] (7) Improved prediction accuracy and timeliness: By periodically or event-driven updating of the prediction model, the accuracy and timeliness of the prediction results are ensured, making scheduling decisions more accurate.

[0123] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An intelligent scheduling method for a heterogeneous device system, characterized in that: include: S100, obtaining operation status data deployed on heterogeneous devices, and preprocessing the collected data using edge computing technology; S200, based on the data preprocessed by S100, uses distributed learning technology to train the prediction model in parallel on multiple edge nodes; S300, under the guidance of the prediction model trained in S200, calculate and generate a local optimal resource allocation plan through rapid local optimization and periodic global optimization; S400. According to the locally optimal resource allocation plan generated in S300, the execution and monitoring modules are improved, the scheduling plan is implemented and the execution effect is continuously monitored, and the resource allocation and scheduling strategies are dynamically adjusted through the feedback mechanism.

2. The method according to claim 1, characterized in that Step S100 includes steps S110-S130: S110, through sensors and monitoring tools deployed on heterogeneous devices, continuously collect device operating status data, including CPU utilization, memory usage, network traffic, storage usage, temperature, and power consumption indicators; S120, preprocessing the received operating status data, including data cleaning, format conversion, anomaly detection and data compression; S130: Using edge computing technology, the data processed by S120 is summarized into summary information and input to the central scheduling system for use in the global optimization strategy.

3. The method according to claim 2, characterized in that Step S200 includes steps S210-S230: S210, using distributed learning technology, disperse the model training process to multiple edge nodes, each node uses the local data of S130 to train the model, and then summarizes the model parameters to the central node for aggregation; S220, the trained prediction model receives the preprocessed input data, performs real-time prediction, and monitors the actual effectiveness of the prediction results; S230. Select incremental learning to update the model and test it on an independent validation set.

4. The method according to claim 3, characterized in that Step S300 includes steps S310-S330: S310, performing rapid local optimization and periodic global optimization on the data tested in S230; S320, according to the prediction results and the current system status, dynamically adjust the resource allocation strategy, including task allocation, load balancing, and priority sorting; S330. Use the variational method in functional analysis to solve the functional extreme value problem.

5. The method according to claim 4, characterized in that Step S400 includes steps S410-S430: S410, continuously monitor the execution of scheduling decisions, collect execution results and system feedback, including task completion time, resource utilization efficiency, and system performance indicators, to evaluate the effectiveness of scheduling strategies; S420, based on the monitoring data, identifying deficiencies or bottlenecks in the scheduling strategy, and dynamically adjusting optimization parameters, including resource reservations, task priorities, and algorithm parameters; S430: Based on the result of environmental perception, dynamically adjust the scheduling strategy of the module.

6. The method according to claim 2, characterized in that In step S120, the preprocessing module standardizes or normalizes the data to convert all values ​​to the same scale.

7. The method according to claim 3, characterized in that In step S210, the edge node uses local data to train the model, and then summarizes the model parameters to the central node for aggregation.

8. The method according to claim 4, characterized in that The fast local optimization strategy in step S300 is used to analyze the device status and task requirements in real time to calculate the local optimal resource allocation solution, and perform task allocation, load balancing and priority sorting during task scheduling.

9. The method according to claim 5, characterized in that The scheduling scheme in step S400 monitors the execution of the resource allocation scheme in S300 and makes feedback adjustments according to the actual operation of the system to optimize the scheduling efficiency.

10. The method according to claim 5, characterized in that The step S400 further includes: dynamically adjusting the scheduling scheme based on the task completion time, resource utilization efficiency and system performance indicators collected by the monitoring module to ensure continuous optimization of the system.

Citation Information

Patent Citations

  • Method and system for improving computing power efficiency

    CN118550711A

  • Heterogeneous hardware computing power scheduling method and device, equipment and medium

    CN118626263A

  • Cloud resource load scheduling method based on Q-learning

    CN118642855A

Cited By

  • Resource dynamic analysis and sample load scheduling optimization method oriented to edge distributed training

    CN119149247A

  • Multi-source trusted data access method and system, electronic equipment and storage medium

    CN119248526A

  • Linux kernel scheduler parameter optimization system and method

    CN119271382A

  • Edge node scheduling system, edge node scheduling method and edge node

    CN119363753A

  • Building equipment intelligent scheduling method based on AI algorithm

    CN119417189A