Resource scheduling management system and method based on cloud platform

Abstract

The invention relates to the technical field of cloud data management, and particularly discloses a resource scheduling management system and method based on a cloud platform, and the method comprises the steps: obtaining heterogeneous resources from cloud platform resources, carrying out the information collection of the heterogeneous resources through a heterogeneous data interface, carrying out the same-type division of the heterogeneous resources, and obtaining heterogeneous resource nodes, the method comprises the steps of obtaining historical resource scheduling data of equipment, performing resource scheduling feature extraction and time sequence analysis, constructing a resource scheduling feature vector, performing similarity matching, determining equipment scheduling heterogeneous resource nodes, establishing a dynamic load prediction algorithm, performing fitting analysis on a load trend in combination with a sliding window, predicting load change and generating a scheduling early warning signal. And in combination with a resource priority strategy, a resource allocation scheme is obtained, task migration is performed on high-load heterogeneous resource nodes, and the migration tasks are allocated to the same type of heterogeneous resource nodes with low loads, so that optimization of resource allocation is facilitated, and the resource utilization rate of the cloud platform is improved.

Description

Technical Field

[0001] This invention relates to the field of cloud data management technology, specifically to a cloud platform-based resource scheduling and management system and method. Background Technology

[0002] Reasonable scheduling of computing, storage, and network resources during the operation of industrial cloud platforms is crucial to ensuring business continuity and improving resource utilization. Existing scheduling solutions typically rely directly on historical resource data, failing to accurately identify the core resource needs of business equipment. Furthermore, the fragmented and inconsistent formats of heterogeneous resource information reduce the accuracy of matching resources with equipment requirements, easily leading to potential resource mismatches and making it difficult to adapt to actual needs. This often results in scheduling discrepancies or redundant scheduling.

[0003] In existing technologies, actual industrial cloud platforms contain various types and specifications of computing, storage, and network resources. These resources differ in their technical forms and support capabilities, and resource information is fragmented and inconsistent in format. This makes it difficult to accurately identify the core resource requirements of equipment when matching equipment resource needs. Furthermore, the platform fails to anticipate load change trends in advance, making it impossible to detect the overload risk of resource nodes in a timely manner. It also fails to prioritize tasks based on their importance and resource usage, resulting in chaotic scheduling and an inability to quickly alleviate node pressure. Ultimately, this leads to low resource utilization efficiency of the cloud platform, failing to meet the needs of efficient and stable business operations.

[0004] Therefore, the present invention provides a cloud platform-based resource scheduling and management system and method. Summary of the Invention

[0005] The purpose of this invention is to provide a cloud platform-based resource scheduling and management system and method to solve the aforementioned background problems.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] A cloud platform-based resource scheduling and management system includes:

[0008] Resource partitioning module: Obtain heterogeneous resources from cloud platform resources, collect information on heterogeneous resources through heterogeneous data interface, partition heterogeneous resources of the same type, and obtain heterogeneous resource nodes;

[0009] The module for confirming the configuration of the device is as follows: it acquires historical resource scheduling data of the device, performs resource scheduling feature extraction and time series analysis on the historical scheduling data, constructs resource scheduling feature vectors, performs similarity matching on the resource scheduling feature vectors in heterogeneous resource nodes, and determines the heterogeneous resource nodes for device scheduling.

[0010] Predictive Analysis Module: Based on heterogeneous resource nodes of device scheduling, a dynamic load prediction algorithm is established to monitor the resource utilization, response latency and task queue length of each heterogeneous resource node on the cloud platform in real time. Combined with sliding window, load trend fitting analysis is performed to predict load changes and generate scheduling early warning signals.

[0011] Adjustment and migration module: Based on the scheduling warning signal and combined with the resource priority strategy, obtain the resource allocation plan, migrate the tasks of high-load heterogeneous resource nodes, and allocate the migration tasks to low-load heterogeneous resource nodes of the same type.

[0012] Furthermore, heterogeneous resources are divided into similar types to obtain heterogeneous resource nodes:

[0013] The interface collects information on heterogeneous resources in the cloud platform and obtains resource type data information for each heterogeneous resource.

[0014] The collected heterogeneous resource information is categorized according to resource type, and heterogeneous resources with the same resource type are grouped into the same category to obtain heterogeneous resource nodes;

[0015] Obtain the resource status data corresponding to each scheduled resource within each type of resource node, and obtain the feature vector of each scheduled resource.

[0016] Furthermore, resource scheduling features are extracted and time-series analysis is performed:

[0017] Acquire historical status data of the device during each resource scheduling, summarize and organize it to obtain the device resource scheduling status sequence;

[0018] Extract the resource type of each resource in the equipment resource scheduling status sequence, perform statistical analysis, obtain the number of times each resource type is scheduled in the historical resource scheduling process, obtain the number of times the same type of resource is scheduled, and obtain the total number of times resource scheduling is performed;

[0019] The ratio of the number of times the same type of resource is scheduled to the total number of resource scheduling is used to obtain the scheduling frequency ratio of each resource type.

[0020] Perform time series analysis on each resource of the same type to obtain the scheduling time of the most recent resource scheduling for each resource of the same type. Calculate the time interval between the current time and the time interval by interpolation to obtain the scheduling time difference.

[0021] The scheduling time differences corresponding to all resource types are added together to obtain the comprehensive scheduling time difference value;

[0022] The scheduling time difference of each resource type is compared with the overall scheduling time difference to obtain the scheduling time difference ratio of each resource.

[0023] Furthermore, construct the resource scheduling feature vector:

[0024] The scheduling frequency ratio is compared with the scheduling time difference ratio to obtain the scheduling coefficient for each resource type.

[0025] The obtained scheduling coefficients are compared, and the resource type with the largest scheduling coefficient is determined as the core resource type of the current device. The historical scheduling data corresponding to the core resource type is extracted as the resource scheduling feature vector.

[0026] Furthermore, similarity matching is performed on the resource scheduling feature vectors across heterogeneous resource nodes:

[0027] Based on the resource scheduling feature vector, the core resource type is matched with the resource type in the heterogeneous resource node, and heterogeneous resource nodes with the same resource type are selected as candidate heterogeneous resource nodes.

[0028] The similarity between the resource scheduling feature vector and the feature vector of each scheduled resource in the candidate heterogeneous resource node is calculated by using cosine similarity to obtain the similarity value.

[0029] If the similarity value is greater than or equal to the similarity threshold, it means that the candidate heterogeneous resource node meets the scheduling matching condition, and the candidate heterogeneous resource node is used as the device scheduling heterogeneous resource node.

[0030] Furthermore, a fitting analysis of the load trend is performed using a sliding window:

[0031] Obtain real-time performance metrics of each scheduled resource within the heterogeneous resource node of the device scheduling;

[0032] The collected real-time performance metrics are aggregated and calculated according to resource type to obtain the overall load status data of each heterogeneous resource node;

[0033] Using a sliding window, historical load status data of heterogeneous resource nodes in device scheduling is extracted and summarized in time series.

[0034] At each sampling time, the historical load status data sequence of the heterogeneous resource node within the current sliding window is extracted. The historical load status data sequence within each sampling period is summarized to obtain the load trend sequence.

[0035] Furthermore, generate scheduling early warning signals:

[0036] Based on the load trend sequence, the load index for the next sampling period is predicted using time series forecasting methods to obtain the predicted load value.

[0037] If the predicted load value is greater than or equal to the load threshold, it indicates that the corresponding heterogeneous resource node is at risk of overload, and a high load warning signal is generated.

[0038] If the predicted load value is less than the low load threshold, it indicates that the corresponding heterogeneous resource nodes have idle resources, and a low load warning signal is generated.

[0039] Furthermore, obtain the resource allocation plan:

[0040] Identify the types of heterogeneous resource nodes that trigger the alert, and filter out scheduling resource nodes of the same type whose current load is below the low load threshold;

[0041] The tasks running on the high-load heterogeneous resource nodes that triggered the warning are sorted according to task priority, resource utilization, etc., to obtain a list of tasks to be migrated.

[0042] Based on the list of tasks to be migrated, and combined with the remaining resources of the selected low-load heterogeneous resource nodes of the same type, a target migration node is assigned to each task to be migrated, thus obtaining a resource allocation scheme.

[0043] Furthermore, the migration tasks will be assigned to low-load heterogeneous resource nodes of the same type:

[0044] The running status of the migration task is monitored. If the load on the target node is found to be abnormally high, the rollback mechanism is triggered to migrate the task back to the original node or other low-load heterogeneous resource nodes of the same type.

[0045] The task to be migrated is moved from a high-load heterogeneous resource node to a target low-load heterogeneous resource node of the same type, and the resources occupied by the migration task are released from the original high-load node.

[0046] Furthermore, a cloud platform-based resource scheduling and management method is characterized by the following steps:

[0047] Obtain heterogeneous resources from cloud platform resources, collect information on heterogeneous resources through heterogeneous data interfaces, classify heterogeneous resources of the same type, and obtain heterogeneous resource nodes;

[0048] Obtain historical resource scheduling data of equipment, extract resource scheduling features and perform time series analysis on the historical scheduling data, construct resource scheduling feature vectors, and perform similarity matching on the resource scheduling feature vectors in heterogeneous resource nodes to determine the heterogeneous resource nodes for equipment scheduling.

[0049] Based on the scheduling of heterogeneous resource nodes, a dynamic load prediction algorithm is established to monitor the resource utilization, response latency and task queue length of each heterogeneous resource node on the cloud platform in real time. The load trend is fitted and analyzed by combining a sliding window to predict load changes and generate scheduling early warning signals.

[0050] Based on the scheduling warning signal and combined with the resource priority strategy, a resource allocation plan is obtained. Tasks are migrated to high-load heterogeneous resource nodes, and the migration tasks are allocated to low-load heterogeneous resource nodes of the same type.

[0051] The beneficial effects of this invention are as follows:

[0052] 1. This invention collects information such as the type and specifications of heterogeneous resources through a heterogeneous data interface, divides them into heterogeneous resource nodes according to resource type, and extracts the feature vectors of the scheduling resources within the nodes to complete the standardized integration of heterogeneous resources; then, it acquires historical resource scheduling data of the equipment, extracts scheduling features and constructs resource scheduling feature vectors, and determines the appropriate heterogeneous resource nodes for equipment scheduling through similarity matching. This is conducive to accurately matching the core resource requirements of equipment scheduling, providing a standardized resource foundation for subsequent scheduling processes, and improving the accuracy of resource scheduling.

[0053] 2. This invention analyzes load trends by monitoring resource node utilization, response latency, task queue length, and other indicators in real time, combined with sliding window and prediction algorithms, to generate overload or low load scheduling early warning signals. Based on the early warning signals and resource priority strategies, tasks on high-load nodes are sorted by priority and resource occupancy and migrated to low-load nodes of the same type. This achieves early detection and warning of load, avoids node overload or resource idleness, dynamically optimizes resource allocation schemes, and improves the resource utilization and business response speed of the cloud platform. Attached Figure Description

[0054] The invention will now be further described with reference to the accompanying drawings.

[0055] Figure 1 This is a flowchart of a cloud platform-based resource scheduling and management system according to an embodiment of the present invention;

[0056] Figure 2 This is a logic judgment diagram of a cloud platform-based resource scheduling and management system as described in an embodiment of the present invention;

[0057] Figure 3 This is a flowchart illustrating the steps of a cloud platform-based resource scheduling and management method according to an embodiment of the present invention. Detailed Implementation

[0058] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0059] Example 1

[0060] Please see Figure 1 - Figure 2As shown in the embodiment of the present invention, a cloud platform-based resource scheduling and management system addresses the problems of fragmented and inconsistent formats of heterogeneous resource information in cloud platforms, which lead to difficulties in resource organization; unclear core requirements during device resource scheduling, resulting in resource type mismatches; lack of load prediction for heterogeneous resource nodes; and chaotic scheduling priorities under high load conditions. Through standardized integration of heterogeneous resources, scheduling feature mining and matching, dynamic load prediction and early warning, and resource priority scheduling adjustment, the system achieves accurate and efficient scheduling of heterogeneous resources. Specifically, it includes the following modules:

[0061] Resource partitioning module: Obtain heterogeneous resources from cloud platform resources, collect information on heterogeneous resources through heterogeneous data interface, partition heterogeneous resources of the same type, and obtain heterogeneous resource nodes;

[0062] Specifically, heterogeneous resources refer to the general term for various resources in a cloud platform that differ in type, hardware specifications, and technical form, but can work together to provide core support for business tasks such as computing, storage, and network transmission. These include computing hardware such as different models of CPU servers and ARM architecture edge computing boxes, storage hardware such as mechanical hard disk arrays, solid-state drives (SSDs), and high-speed NVMe storage devices, as well as network hardware such as switches and routers with different bandwidths.

[0063] Information is collected from heterogeneous resources, and these resources are then categorized into similar types. The process is as follows:

[0064] Use heterogeneous data interfaces to collect information from heterogeneous resources in the cloud platform and obtain resource type data information for each heterogeneous resource;

[0065] It should be noted that the heterogeneous data interface refers to a standardized data interaction interface that adapts to the data interaction needs of different types of heterogeneous resources in the cloud platform. It is compatible with multiple communication protocols (such as RESTful, SSH, SNMP, etc.) and hardware vendor data formats. Its function is to break down communication barriers between different heterogeneous resources, realize a unified data acquisition entry point for various heterogeneous resources such as computing, storage, and network, and support efficient capture, format conversion, and preliminary integration of raw data.

[0066] The collected heterogeneous resource information is categorized according to resource type, and heterogeneous resources with the same resource type are grouped into the same category to obtain heterogeneous resource nodes;

[0067] It should be noted that heterogeneous resource nodes are basic scheduling units formed by aggregating heterogeneous resources of the same type, and have the dual characteristics of multiple nodes of the same type and multiple scheduling resources on a single node. For example, computing resources in a cloud platform can be subdivided according to deployment clusters, regions or hardware specifications to form cluster A computing nodes, edge area computing nodes, etc. Each heterogeneous resource node contains multiple scheduling resources of the same type.

[0068] Based on heterogeneous resource nodes, the resource status data corresponding to each scheduled resource in each type of resource node is obtained, and the feature vector of each scheduled resource is obtained, including CPU utilization, memory usage, network bandwidth utilization, etc.

[0069] For example, all computing resources are classified as computing nodes, all storage resources as storage nodes, and all network resources as network nodes.

[0070] It should be noted that the role of this module is to sort out and standardize the heterogeneous resources of the cloud platform, laying a solid resource foundation for the entire heterogeneous resource scheduling system. Specifically, firstly, it achieves unified information collection of scattered heterogeneous resources through heterogeneous data interfaces, breaking down information barriers between different types of resources, solving the problems of fragmented and inconsistent formats of heterogeneous resource information, and forming a comprehensive and standardized resource information dataset; secondly, it forms heterogeneous resource nodes by classifying and dividing them according to resource type, aggregating scattered heterogeneous resources into basic scheduling units with clear functions and accurate identification, and clearly defining the functional boundaries of various resources; thirdly, it outputs heterogeneous resource nodes, providing accurate resource objects and reliable data support for subsequent modules such as resource scheduling feature matching, load prediction monitoring, resource allocation adjustment, and task migration, ensuring that subsequent scheduling processes can be carried out in an orderly manner.

[0071] The module for confirming the configuration of the device is as follows: it acquires historical resource scheduling data of the device, performs resource scheduling feature extraction and time series analysis on the historical scheduling data, constructs resource scheduling feature vectors, performs similarity matching on the resource scheduling feature vectors in heterogeneous resource nodes, and determines the heterogeneous resource nodes for device scheduling.

[0072] Specifically, historical resource scheduling data refers to the resource usage records generated by various business devices (or virtual devices) during the historical resource scheduling process during the operation of the cloud platform. It mainly includes time-series indicators such as CPU utilization, memory usage, network bandwidth usage, and storage access frequency. This data is usually stored in monitoring logs or resource management systems in time-series form.

[0073] The process of extracting resource scheduling features from historical scheduling data and performing time-series analysis to construct resource scheduling feature vectors is as follows:

[0074] Acquire historical status data of the device during each resource scheduling, summarize and organize it to obtain the device resource scheduling status sequence;

[0075] Extract the resource type of each resource in the equipment resource scheduling status sequence, perform statistical analysis, obtain the number of times each resource type is scheduled in the historical resource scheduling process, obtain the number of times the same type of resource is scheduled, and obtain the total number of times resource scheduling is performed;

[0076] The ratio of the number of times the same type of resource is scheduled to the total number of resource scheduling is used to obtain the scheduling frequency ratio of each resource type.

[0077] Perform time series analysis on each resource of the same type to obtain the scheduling time of the most recent resource scheduling for each resource of the same type. Calculate the time interval between the current time and the time interval by interpolation to obtain the scheduling time difference.

[0078] The scheduling time differences corresponding to all resource types are added together to obtain the comprehensive scheduling time difference value;

[0079] The scheduling time difference of each resource type is compared with the overall scheduling time difference to obtain the scheduling time difference ratio of each resource.

[0080] The scheduling frequency ratio is compared with the scheduling time difference ratio to obtain the scheduling coefficient for each resource type.

[0081] It should be noted that the scheduling coefficient reflects the activity level of each resource type in historical scheduling. The larger the scheduling coefficient, the more frequently the corresponding resource type is scheduled in historical scheduling and the closer the most recent scheduling time is to the current moment, indicating that the scheduling demand of the resource type corresponding to the current device is high.

[0082] The obtained scheduling coefficients are compared, and the resource type with the largest scheduling coefficient is determined as the core resource type of the current device. The historical scheduling data corresponding to the core resource type is extracted as the resource scheduling feature vector.

[0083] It should be noted that historical scheduling data refers to data such as CPU utilization, memory usage, and network bandwidth usage related to core resource types during previous scheduling processes.

[0084] Based on the resource scheduling feature vector, the core resource type is matched with the resource type in the heterogeneous resource node, and heterogeneous resource nodes with the same resource type are selected as candidate heterogeneous resource nodes.

[0085] Based on candidate heterogeneous resource nodes, the similarity between the resource scheduling feature vector and the feature vector of each scheduled resource in the candidate heterogeneous resource node is calculated by cosine similarity to obtain the similarity value.

[0086] Compare the similarity value with a similarity threshold;

[0087] If the similarity value is greater than or equal to the similarity threshold, it means that the candidate heterogeneous resource node meets the scheduling matching condition, and the candidate heterogeneous resource node is used as the device scheduling heterogeneous resource node.

[0088] If the similarity value is less than the similarity threshold, it means that the candidate heterogeneous resource nodes do not meet the scheduling matching requirements;

[0089] It should be noted that the function of this module is to accurately locate the core needs of equipment in resource scheduling by deeply mining and analyzing the historical resource scheduling data of the equipment, and match the most suitable heterogeneous resource nodes. Through resource scheduling feature extraction and time series analysis, a resource scheduling feature vector that can reflect the historical scheduling behavior and resource usage preferences of the equipment is constructed. Similarity matching is performed on heterogeneous resource nodes to select the node that best matches the resource scheduling features of the equipment from many candidate nodes, ensuring that the equipment can be scheduled to the heterogeneous resources that are most suitable for its current business needs, which helps to improve the accuracy and efficiency of resource scheduling.

[0090] Example 2

[0091] like Figure 1 - Figure 2 As shown in the embodiment of the present invention, a cloud platform-based resource scheduling and management system includes the following modules:

[0092] Predictive Analysis Module: Based on heterogeneous resource nodes of device scheduling, a dynamic load prediction algorithm is established to monitor the resource utilization, response latency and task queue length of each heterogeneous resource node on the cloud platform in real time. Combined with sliding window, load trend fitting analysis is performed to predict load changes and generate scheduling early warning signals.

[0093] Specifically, for heterogeneous resource nodes in device scheduling, real-time performance metrics of each scheduling resource within the heterogeneous resource node are obtained, including:

[0094] Resource utilization: including CPU utilization, memory usage, storage I / O utilization, network bandwidth utilization, etc.

[0095] Response latency includes task processing latency, storage access latency, network round-trip latency, etc.

[0096] Task queue length: refers to the number of tasks or the length of the data packet queue currently waiting to be processed on this resource node;

[0097] The collected real-time performance metrics are aggregated and calculated according to resource type to obtain the overall load status data of each heterogeneous resource node;

[0098] It should be noted that the overall load status data reflects the comprehensive load level of the node at the current moment and is used for trend analysis and prediction of load changes. Aggregation calculation refers to the statistical operation of summing or averaging the real-time performance indicators of all scheduled resources under the same resource type to obtain the overall performance data of the resource type in heterogeneous resource nodes. For example, for computing resources, the average CPU utilization of all computing scheduled resources is obtained by summing the CPU utilization of computing resources in heterogeneous resource nodes. Similarly, the overall data of indicators such as memory usage is obtained. For response latency, the task processing latency, storage access latency, network round-trip latency, etc. of all scheduled resources are statistically calculated according to resource type to obtain the average response latency of each type of resource. The task queue length is obtained by summing the number of waiting tasks or the data packet queue length of all scheduled resources under the same resource type.

[0099] By combining a sliding window to fit and analyze load trends, load changes are predicted and scheduling early warning signals are generated. The process is as follows:

[0100] Using a sliding window, historical load status data of heterogeneous resource nodes in device scheduling is extracted and summarized in time series.

[0101] Set the length and step size of the sliding window. For example, the window length can be set to 10 sampling periods and the step size can be set to 1 sampling period.

[0102] At each sampling time, the historical load status data sequence of the heterogeneous resource node within the current sliding window is extracted, and the historical load status data sequence within each sampling period is summarized to obtain the load trend sequence.

[0103] It should be noted that the sampling time refers to the time point at which load status data is collected periodically, while the sampling period refers to the time interval between two adjacent sampling times.

[0104] Based on the load trend sequence, time series forecasting methods (such as LSTM) are used to predict the load index for the next sampling period, and the predicted load value is obtained.

[0105] Compare the predicted load value with the preset load threshold;

[0106] If the predicted load value is greater than or equal to the load threshold, it indicates that the corresponding heterogeneous resource node is at risk of overload, and a high load warning signal is generated.

[0107] If the predicted load value is less than the low load threshold, it indicates that there are idle resources on the corresponding heterogeneous resource nodes, and a low load warning signal is generated.

[0108] If the predicted load value is at the low load threshold and the load threshold, it means that the corresponding heterogeneous resource node can process the current task.

[0109] It should be noted that the dispatching early warning signals are divided into two categories: overload early warning and low load early warning. Overload early warning is used to prompt the dispatching system to perform load evacuation or resource expansion in advance, while low load early warning indicates that resource integration or energy-saving dispatch can be carried out.

[0110] For example, for computing heterogeneous resource nodes, if the CPU utilization sequence in the sliding window shows a continuous upward trend, and the predicted load value output indicates that the CPU utilization will exceed the load threshold in the next cycle, a scheduling warning signal will be generated, and the associated devices and tasks will be added to the priority adjustment queue.

[0111] It should be noted that the function of this module is to realize real-time perception and prediction of the load status of heterogeneous resource nodes, providing a basis for decision-making for dynamic resource scheduling. Specifically, firstly, by monitoring multi-dimensional load indicators in real time, the operating status of nodes can be fully grasped, avoiding misjudgment based on a single indicator; secondly, by using sliding windows and prediction algorithms, load change patterns can be identified; and thirdly, by using an early warning signal mechanism, the scheduling process can be triggered in advance, effectively preventing node overload or resource waste and improving the overall resource utilization efficiency of the cloud platform.

[0112] Adjustment and migration module: Based on the scheduling warning signal and combined with the resource priority strategy, obtain the resource allocation plan, migrate the high-load heterogeneous resource nodes, and allocate the migration tasks to the low-load heterogeneous resource nodes of the same type.

[0113] Specifically, based on the high load warning signal, the heterogeneous resource node type that triggers the warning (such as compute nodes, storage nodes or network nodes) is identified, and scheduling resource nodes of the same type with current loads below the low load threshold are selected.

[0114] The tasks running on the high-load heterogeneous resource nodes that triggered the warning are sorted according to task priority, resource utilization, etc., to obtain a list of tasks to be migrated.

[0115] Task priorities can be set comprehensively based on factors such as the importance, urgency, and business type of the task. For example, critical business tasks have higher priority than ordinary business tasks, and tasks with high real-time requirements have higher priority than tasks with low real-time requirements.

[0116] Resource utilization rate analyzes the task's usage of resources such as CPU, memory, and network bandwidth. Tasks that consume more resources are migrated first to quickly reduce the load pressure on high-load nodes.

[0117] Based on the list of tasks to be migrated, and combined with the remaining resources of the selected low-load heterogeneous resource nodes of the same type, a target migration node is assigned to each task to be migrated, and a resource allocation scheme is obtained.

[0118] For example, during the allocation process, it is necessary to ensure that the target migration node has sufficient resources to carry the migration task, so as to avoid the task migration failure or target node overload due to insufficient resources. At the same time, the impact of task migration on network bandwidth should be considered, and a path with low network latency and sufficient bandwidth should be selected for task migration to reduce migration time and impact on business.

[0119] During the migration process, the running status of the migration task is monitored. If the load on the target node is found to be abnormally high, a rollback mechanism is triggered to migrate the task back to the original node or other low-load heterogeneous resource nodes of the same type.

[0120] It should be noted that the rollback mechanism is designed to ensure the reliability and stability of task migration. When the target node experiences an abnormally high load due to unforeseen factors (such as sudden traffic or hardware failure) and is unable to continue to carry the migration task, the rollback process is triggered to migrate the task back to the original high-load node (if the original node's load has recovered to a safe range) or to reselect other eligible backup nodes for a second migration, so as to avoid service interruption due to task migration failure.

[0121] Migrate the task to be migrated from a high-load heterogeneous resource node to a target low-load heterogeneous resource node of the same type, and release the resources occupied by the migration task from the original high-load node;

[0122] It should be noted that this module is designed to dynamically obtain resource allocation schemes based on load prediction results and resource priority strategies, thereby achieving reasonable resource flow and efficient utilization. Through task migration and resource integration, it can effectively avoid overload or idle resources on heterogeneous resource nodes, improve the overall performance and resource utilization of the cloud platform, and at the same time, a reasonable task migration strategy can also reduce task execution time, improve business response speed and user experience.

[0123] The technical solution of this invention is as follows:

[0124] This invention collects information such as the type and specifications of heterogeneous resources through a heterogeneous data interface, divides them into heterogeneous resource nodes according to resource type, and extracts feature vectors of scheduled resources within the nodes to complete the standardized integration of heterogeneous resources. It then acquires historical resource scheduling data from devices, extracts scheduling features, and constructs resource scheduling feature vectors. Through similarity matching, it determines suitable heterogeneous resource nodes for device scheduling, which facilitates accurate matching of core resource requirements for device scheduling, provides a standardized resource foundation for subsequent scheduling processes, and improves the accuracy of resource scheduling. Based on real-time monitoring of resource node resource utilization, response latency, task queue length, and other indicators, it analyzes load trends using sliding windows and prediction algorithms to generate overload or low-load scheduling early warning signals. Based on these early warning signals and a resource priority strategy, tasks from high-load nodes are sorted by priority and resource occupancy and migrated to low-load nodes of the same type. This achieves early detection and warning of load, helps avoid node overload or resource idleness, dynamically optimizes resource allocation schemes, and improves the resource utilization and business response speed of the cloud platform.

[0125] Example 3

[0126] like Figure 3 As shown in the figure, the cloud platform-based resource scheduling and management method described in this embodiment of the invention includes the following steps:

[0127] Step 1: Obtain heterogeneous resources from the cloud platform resources, collect information on heterogeneous resources through the heterogeneous data interface, classify heterogeneous resources of the same type, and obtain heterogeneous resource nodes;

[0128] Step 2: Obtain historical resource scheduling data of the equipment, extract resource scheduling features and perform time series analysis on the historical scheduling data, construct resource scheduling feature vectors, and perform similarity matching on the resource scheduling feature vectors in heterogeneous resource nodes to determine the heterogeneous resource nodes for equipment scheduling.

[0129] Step 3: Based on the scheduling of heterogeneous resource nodes, establish a dynamic load prediction algorithm, monitor the resource utilization, response latency and task queue length of each heterogeneous resource node on the cloud platform in real time, and combine the sliding window to fit and analyze the load trend, predict load changes and generate scheduling early warning signals.

[0130] Step 4: Based on the scheduling warning signal and the resource priority strategy, obtain the resource allocation plan, migrate the high-load heterogeneous resource nodes, and allocate the migration tasks to the low-load heterogeneous resource nodes of the same type.

[0131] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the present invention should still fall within the scope of the present invention.

Claims

1. A cloud platform-based resource scheduling and management system, characterized in that: Includes the following modules: Resource partitioning module: Obtain heterogeneous resources from cloud platform resources, collect information on heterogeneous resources through heterogeneous data interface, partition heterogeneous resources of the same type, and obtain heterogeneous resource nodes; The module for confirming the configuration of the device is as follows: it acquires historical resource scheduling data of the device, performs resource scheduling feature extraction and time series analysis on the historical scheduling data, constructs resource scheduling feature vectors, performs similarity matching on the resource scheduling feature vectors in heterogeneous resource nodes, and determines the heterogeneous resource nodes for device scheduling. Predictive Analysis Module: Based on heterogeneous resource nodes of device scheduling, a dynamic load prediction algorithm is established to monitor the resource utilization, response latency and task queue length of each heterogeneous resource node on the cloud platform in real time. Combined with sliding window, load trend fitting analysis is performed to predict load changes and generate scheduling early warning signals. Adjustment and migration module: Based on the scheduling warning signal and combined with the resource priority strategy, obtain the resource allocation plan, migrate the tasks of high-load heterogeneous resource nodes, and allocate the migration tasks to low-load heterogeneous resource nodes of the same type.

2. The cloud platform-based resource scheduling and management system according to claim 1, characterized in that: By partitioning heterogeneous resources into similar categories, heterogeneous resource nodes are obtained: The interface collects information on heterogeneous resources in the cloud platform and obtains resource type data information for each heterogeneous resource. The collected heterogeneous resource information is categorized according to resource type, and heterogeneous resources with the same resource type are grouped into the same category to obtain heterogeneous resource nodes; Obtain the resource status data corresponding to each scheduled resource within each type of resource node, and obtain the feature vector of each scheduled resource.

3. The cloud platform-based resource scheduling and management system according to claim 1, characterized in that: Resource scheduling feature extraction and time series analysis: Acquire historical status data of the device during each resource scheduling, summarize and organize it to obtain the device resource scheduling status sequence; Extract the resource type of each resource in the equipment resource scheduling status sequence, perform statistical analysis, obtain the number of times each resource type is scheduled in the historical resource scheduling process, obtain the number of times the same type of resource is scheduled, and obtain the total number of times resource scheduling is performed; The ratio of the number of times the same type of resource is scheduled to the total number of resource scheduling is used to obtain the scheduling frequency ratio of each resource type. Perform time series analysis on each resource of the same type to obtain the scheduling time of the most recent resource scheduling for each resource of the same type. Calculate the time interval between the current time and the time interval by interpolation to obtain the scheduling time difference. The scheduling time differences corresponding to all resource types are added together to obtain the comprehensive scheduling time difference value; The scheduling time difference of each resource type is compared with the overall scheduling time difference to obtain the scheduling time difference ratio of each resource.

4. The cloud platform-based resource scheduling and management system according to claim 1, characterized in that: Constructing resource scheduling feature vectors: The scheduling frequency ratio is compared with the scheduling time difference ratio to obtain the scheduling coefficient for each resource type. The obtained scheduling coefficients are compared, and the resource type with the largest scheduling coefficient is determined as the core resource type of the current device. The historical scheduling data corresponding to the core resource type is extracted as the resource scheduling feature vector.

5. A cloud platform-based resource scheduling and management system according to claim 1, characterized in that: Perform similarity matching on resource scheduling feature vectors across heterogeneous resource nodes: Based on the resource scheduling feature vector, the core resource type is matched with the resource type in the heterogeneous resource node, and heterogeneous resource nodes with the same resource type are selected as candidate heterogeneous resource nodes. The similarity between the resource scheduling feature vector and the feature vector of each scheduled resource in the candidate heterogeneous resource node is calculated by using cosine similarity to obtain the similarity value; If the similarity value is greater than or equal to the similarity threshold, it means that the candidate heterogeneous resource node meets the scheduling matching condition, and the candidate heterogeneous resource node is used as the device scheduling heterogeneous resource node.

6. A cloud platform-based resource scheduling and management system according to claim 1, characterized in that: Fitting analysis of load trends using a sliding window: Obtain real-time performance metrics of each scheduled resource within the heterogeneous resource node of the device scheduling; The collected real-time performance metrics are aggregated and calculated according to resource type to obtain the overall load status data of each heterogeneous resource node; Using a sliding window, historical load status data of heterogeneous resource nodes in device scheduling is extracted and summarized in time series. At each sampling time, the historical load status data sequence of the heterogeneous resource node within the current sliding window is extracted. The historical load status data sequence within each sampling period is summarized to obtain the load trend sequence.

7. A cloud platform-based resource scheduling and management system according to claim 1, characterized in that: Generate scheduling early warning signals: Based on the load trend sequence, the load index for the next sampling period is predicted using time series forecasting methods to obtain the predicted load value. If the predicted load value is greater than or equal to the load threshold, it indicates that the corresponding heterogeneous resource node is at risk of overload, and a high load warning signal is generated. If the predicted load value is less than the low load threshold, it indicates that the corresponding heterogeneous resource nodes have idle resources, and a low load warning signal is generated.

8. A cloud platform-based resource scheduling and management system according to claim 1, characterized in that: Obtain resource allocation plan: Identify the types of heterogeneous resource nodes that trigger the alert, and filter out scheduling resource nodes of the same type whose current load is below the low load threshold; The tasks running on the high-load heterogeneous resource nodes that triggered the warning are sorted according to task priority, resource utilization, etc., to obtain a list of tasks to be migrated. Based on the list of tasks to be migrated, and combined with the remaining resources of the selected low-load heterogeneous resource nodes of the same type, a target migration node is assigned to each task to be migrated, thus obtaining a resource allocation scheme.

9. A cloud platform-based resource scheduling and management system according to claim 1, characterized in that: Assign migration tasks to heterogeneous resource nodes of the same type with low load: The running status of the migration task is monitored. If the load on the target node is found to be abnormally high, the rollback mechanism is triggered to migrate the task back to the original node or other low-load heterogeneous resource nodes of the same type. The task to be migrated is moved from a high-load heterogeneous resource node to a target low-load heterogeneous resource node of the same type, and the resources occupied by the migration task are released from the original high-load node.

10. A cloud platform-based resource scheduling and management method, characterized in that: Includes the following steps: Obtain heterogeneous resources from cloud platform resources, collect information on heterogeneous resources through heterogeneous data interfaces, classify heterogeneous resources of the same type, and obtain heterogeneous resource nodes; Obtain historical resource scheduling data of equipment, extract resource scheduling features and perform time series analysis on the historical scheduling data, construct resource scheduling feature vectors, and perform similarity matching on the resource scheduling feature vectors in heterogeneous resource nodes to determine the heterogeneous resource nodes for equipment scheduling. Based on the scheduling of heterogeneous resource nodes, a dynamic load prediction algorithm is established to monitor the resource utilization, response latency and task queue length of each heterogeneous resource node on the cloud platform in real time. The load trend is fitted and analyzed by combining a sliding window to predict load changes and generate scheduling early warning signals. Based on the scheduling warning signal and combined with the resource priority strategy, a resource allocation plan is obtained. Tasks are migrated to high-load heterogeneous resource nodes, and the migration tasks are allocated to low-load heterogeneous resource nodes of the same type.

Images