Cross-cloud resource elastic scheduling method and system for management system development

Abstract

The application discloses a cross-cloud resource elastic scheduling method and system for management system development, relates to the technical field of resource scheduling, and solves the technical problems of cross-cloud edge resource static configuration and dynamic business demand mismatch, task scheduling and resource elastic expansion independent of each other, difficulty in collaborative optimization, lack of cross-cloud unified perception of container stretching mechanism, and rigid instance adjustment. Through a real-time load vector, short-time load accurate prediction is completed, and combined with service level agreement constraints, overall resource demand is accurately calculated. The existing resource quota, predicted resource demand and edge node cooperative cache ratio are included in the unified decision variable, a multi-objective optimization model is built, and balanced and reasonable allocation between resource domains is realized. Based on the optimization result, the expected number of resource instances is determined, and the PID controller is used to smoothly regulate the number of container and virtual machine instances. An improved differential polling scheduling mechanism is deployed at the network outlet, and the scheduling granularity is dynamically adjusted in combination with the business message characteristics and the resource load state.

Description

Technical Field

[0001] This invention belongs to the field of resource scheduling, specifically a cross-cloud resource elastic scheduling method and system developed for management systems. Background Technology

[0002] As enterprises deepen their digital transformation, management systems (such as enterprise resource planning systems, customer relationship management systems, and supply chain management systems) need to handle ever-increasing business data and complex computing tasks. To ensure the privacy and compliance of business data, many enterprises adopt localized deployment and training of intelligent agents to prevent business data from leaving the domain. A cross-cloud resource elastic scheduling solution developed for management systems is used to collaboratively manage the scheduling of computing resources across local data centers, multiple public cloud platforms, and edge nodes.

[0003] Existing cross-cloud resource elastic scheduling solutions typically employ locally deployed static hardware resources, which cannot cope with the tidal effects of business access. The lack of coordination between existing scheduling algorithms and scaling mechanisms leads to new instances failing to receive tasks in a timely manner during scaling up, and tasks being incorrectly assigned to instances about to be released during scaling down, making it difficult to balance multiple objectives such as response time, cost, and energy efficiency. The autoscaling of existing container orchestration platforms is usually limited to a single cluster, unable to synchronously add or remove instances based on the overall load across multiple cloud platforms and edge nodes, resulting in low resource utilization efficiency and high operational pressure. Traditional differential round-robin algorithms use a fixed replenishment granularity, which can easily generate sudden traffic spikes and uneven scheduling when there are significant differences in packet length across different networks in a cross-cloud environment. Furthermore, the deficit counter may flip, causing severe inaccuracies in the ratio. Summary of the Invention

[0004] This invention aims to solve at least one of the technical problems existing in the prior art. To this end, this invention proposes a cross-cloud resource elastic scheduling method and system for management system development, which is used to solve the technical problems of mismatch between static configuration of cross-cloud edge resources and dynamic business needs, independent task scheduling and resource elastic expansion, difficulty in co-optimization, lack of unified cross-cloud awareness of container scaling mechanism, and rigid instance adjustment.

[0005] To address the aforementioned problems, the first aspect of this invention provides a cross-cloud resource elastic scheduling method and system for management system development, comprising the following steps: The local data center, cloud platform and edge nodes are defined as resource domains. The multi-dimensional feature vector of each resource domain is extracted, and the active discovery and state synchronization of cross-LAN resource domains are carried out through a distributed registry. Collect real-time load vectors of resource domains, predict the load for the next time period, and calculate the total resource requirements based on the predicted load and service level agreement for the next time period. Using the current resource allocation and total resource demand of all resource domains, as well as the collaborative caching ratio between adjacent edge nodes, as decision variables, a multi-objective optimization model is constructed to optimize the resource allocation of resource domains. Based on the resource allocation obtained from multi-objective optimization, the expected number of instances in each resource domain is calculated. A proportional-integral-derivative controller is used to smoothly adjust the number of instances. The final scaling is analyzed, and automatic scaling across cloud containers or virtual machines is executed based on the native API of each resource domain. At the same time, a comprehensive elasticity factor is calculated for each edge node, and a resource borrowing strategy from the nearest cloud resource domain is analyzed based on the comprehensive elasticity factor. An improved differential round-robin algorithm is used for packet scheduling at the network egress point to dynamically adjust the replenishment granularity of each scheduling, while an adaptive deficit compensation mechanism is established.

[0006] Optionally, in one example of the above aspects, the local data center, cloud platform, and edge nodes are defined as resource domains. A multi-dimensional feature vector is extracted for each resource domain, and proactive discovery and state synchronization across local area network resource domains are performed through a distributed registry, including the following steps: Local data centers, cloud platforms and edge nodes are defined as resource domains. Logical resource pools are built between resource domains where data transmission occurs, and multi-dimensional feature vectors of each resource domain are extracted. Deploy a lightweight agent in each resource domain to collect local multidimensional feature vectors and encapsulate them into standardized messages; Lightweight agents deployed in resource domains periodically calculate local multidimensional feature vectors, using a centralized registry or a decentralized hash table as a distributed registry. When the change of feature value in the multidimensional feature vector exceeds the preset threshold, the incremental update is immediately reported to the registry to actively discover and synchronize the status of cross-LAN resource domains.

[0007] Optionally, in one example of the above aspects, acquiring the real-time load vector of the resource domain, predicting the load for the next time period, and calculating the total resource requirement based on the predicted load and service level agreement for the next time period includes the following steps: Collect real-time load vectors from the resource domain and define time intervals. The load feature vector; Based on the historical load feature vector of the resource domain, a hybrid model combining ARIMA-NN autoregressive integral moving average and neural network is used to predict the next time period. The load vector; According to the predicted next period Calculate the total resource requirements based on the load vector and SLA (Service Level Agreement). .

[0008] Optionally, in one example of the above aspects, a multi-objective optimization model is constructed using the current resource allocation and total resource demand of all resource domains, as well as the collaborative caching ratio between adjacent edge nodes, as decision variables to optimize the resource allocation of resource domains, including the following steps: The decision variables for setting the resource domain include: resource domain At any moment Current resource allocation Resource domain At any moment Total resource demand Resource domain To adjacent edge nodes Traffic ratio of collaborative caching ; A multi-objective optimization model is constructed and solved in a distributed manner using the alternating direction multiplier method. Resource domain is solved in a distributed manner using the Alternating Direction Multiplier Method (ADMM). At any moment Total resource demand This yields the optimized resource allocation for the resource domain.

[0009] Optionally, in one example of the above aspects, calculating the expected number of instances for each resource domain based on the resource allocation obtained from multi-objective optimization includes the following steps: Based on the resource allocation obtained from multi-objective optimization, analyze the expected number of instances.

[0010] Optionally, in one example of the above aspects, a proportional-integral-derivative controller is used to smoothly adjust the number of instances, analyze the final scaling amount, and perform automatic scaling across cloud containers or virtual machines based on the native APIs of each resource domain, including the following steps: The proportional-integral-derivative (PID) controller is used to smoothly adjust the number of instances, taking into account the current number of instances with the available resource allocation. Analyze the final expansion / contraction amount; Analyze the final expansion / contraction amount and set the cooldown time. It performs automatic scaling across cloud containers or virtual machines based on the native APIs of each resource domain.

[0011] Optionally, in one example of the above aspects, a comprehensive resilience factor is calculated for each edge node, and a resource borrowing strategy from the nearest cloud resource domain is analyzed based on the comprehensive resilience factor, including the following steps: For resource domains For adjacent edge nodes, analyze the resource demand elasticity factor according to different task types, including: compute-intensive, I / O-intensive, and latency-sensitive. For resource domains When adjacent edge nodes encounter sudden loads, priority is given to requesting cooperation from the nearest edge node, and the emergency resilience factor is analyzed; The resource demand elasticity factor and the emergency elasticity factor are normalized, and the comprehensive elasticity factor is obtained by weighted summation of the normalized resource demand elasticity factor and the emergency elasticity factor. The overall elasticity factor is divided into four intervals, and a corresponding resource borrowing strategy is set for each interval.

[0012] Optionally, in one example of the above aspects, an improved differential round-robin algorithm is used for packet scheduling at the network egress, dynamically adjusting the replenishment granularity of each scheduling, while establishing an adaptive deficit compensation mechanism, including the following steps: At the network exit point where the edge node connects to the cloud resource domain, an improved differential round-robin scheduling method is adopted to distribute and schedule cross-domain resource call messages. Combined with the calculated edge node comprehensive elasticity factor, a dynamic granularity adjustment algorithm is established. Simultaneously, an adaptive deficit compensation mechanism is introduced: after each round of scheduling, the deficit counter... Updated based on actual sent volume, when At that time, an additional compensation round is triggered.

[0013] According to another aspect of this disclosure, a cross-cloud resource elastic scheduling system for management system development is provided. This system adopts the cross-cloud resource elastic scheduling method for management system development as described above to realize cross-cloud resource elastic scheduling for management system development.

[0014] Compared with the prior art, the beneficial effects of the present invention are: This invention relies on real-time load vectors to accurately predict short-term load and combines service level agreement constraints to precisely calculate overall resource demand. It abandons the traditional method of relying on experience to estimate resource demand, providing data-driven basis for resource allocation and reservation. This avoids both insufficient resource reservation leading to business disruptions and excessive reservation causing resource waste. It incorporates existing resource quotas, predicted resource demand, and edge node collaborative caching ratios into unified decision variables, building a multi-objective optimization model to achieve balanced and reasonable allocation of resources across domains.

[0015] This invention determines the expected number of resource instances based on optimization results and uses a PID controller to smoothly regulate the number of container and virtual machine instances, eliminating the business fluctuations caused by sudden increases and decreases in traditional scaling strategies. This achieves smooth and stable elastic scaling of resources, ensuring uninterrupted operation of services such as cost data analysis and construction progress monitoring.

[0016] This invention deploys an improved differential round-robin scheduling mechanism at the network egress point. It dynamically adjusts the scheduling granularity based on service packet characteristics and resource load status, matching different traffic transmission needs across cloud resource calls and improving overall packet forwarding throughput efficiency. An adaptive deficit compensation mechanism is also established to automatically correct traffic allocation deviations across multiple interconnected links, maintaining long-term load balance at the egress network. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0019] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figure 1 The first aspect of this invention provides a cross-cloud resource elastic scheduling method and system for management system development, comprising the following steps: The local data center, cloud platform and edge nodes are defined as resource domains. The multi-dimensional feature vector of each resource domain is extracted, and the active discovery and state synchronization of cross-LAN resource domains are carried out through a distributed registry. Collect real-time load vectors of resource domains, predict the load for the next time period, and calculate the total resource requirements based on the predicted load and service level agreement for the next time period. Using the current resource allocation and total resource demand of all resource domains, as well as the collaborative caching ratio between adjacent edge nodes, as decision variables, a multi-objective optimization model is constructed to optimize the resource allocation of resource domains. Based on the resource allocation obtained from multi-objective optimization, the expected number of instances in each resource domain is calculated. A proportional-integral-derivative controller is used to smoothly adjust the number of instances. The final scaling is analyzed, and automatic scaling across cloud containers or virtual machines is executed based on the native API of each resource domain. At the same time, a comprehensive elasticity factor is calculated for each edge node, and a resource borrowing strategy from the nearest cloud resource domain is analyzed based on the comprehensive elasticity factor. An improved differential round-robin algorithm is used for packet scheduling at the network egress point to dynamically adjust the replenishment granularity of each scheduling, while an adaptive deficit compensation mechanism is established.

[0021] Specifically, in this embodiment, resources across the entire domain are standardized and categorized through local data centers, cloud platforms, and edge nodes. Multi-dimensional feature vectors are used to quantitatively represent the status of various resource domains. A distributed registry is used to achieve automatic discovery and real-time status synchronization of cross-LAN resources. This breaks down resource silos of different architectures and network segments, realizes unified view management of resources across the entire network, and provides a data foundation that is fully perceptible for the scheduling system.

[0022] By relying on real-time load vectors to accurately predict short-term loads and combining service level agreement constraints to accurately calculate overall resource demand, the traditional method of relying on experience to estimate resource demand is abandoned. This allows resource application and reservation to be based on data, which avoids business disruptions caused by insufficient resource reservation and prevents resource waste caused by excessive reservation.

[0023] By incorporating existing resource quotas, predicted resource demand, and edge node collaborative caching ratios into unified decision variables, a multi-objective optimization model is built. This model balances resource utilization, business response latency, and edge local caching collaboration efficiency, outputting a globally optimal resource allocation scheme to achieve balanced and reasonable resource distribution across resource domains. It also enhances the proximity collaboration capabilities of edge nodes, alleviating core computing power pressure through optimized caching ratios.

[0024] Based on the optimization results, the expected number of resource instances is determined. A PID controller is used to smoothly regulate the number of container and virtual machine instances, eliminating the business fluctuations caused by sudden increases and decreases in traditional scaling strategies. This achieves smooth and stable elastic scaling of resources, ensuring uninterrupted operation of services such as cost data analysis and construction progress monitoring. A cloud-based resource borrowing strategy is formulated based on the comprehensive elasticity factor of edge nodes.

[0025] An improved differential round-robin scheduling mechanism is deployed at the network egress point. This mechanism dynamically adjusts the scheduling granularity based on business packet characteristics and resource load status, matching different traffic transmission needs across cloud resource calls and improving overall packet forwarding throughput efficiency. An adaptive deficit compensation mechanism is also established to automatically correct traffic allocation deviations across multiple interconnected links, maintaining long-term load balance at the egress network.

[0026] In one embodiment of the present invention, the local data center, cloud platform, and edge nodes are defined as resource domains. A multi-dimensional feature vector is extracted from each resource domain, and a distributed registry is used for proactive discovery and state synchronization of cross-LAN resource domains, including the following steps: Local data centers, cloud platforms and edge nodes are defined as resource domains. Logical resource pools are built between resource domains where data transmission occurs, and multi-dimensional feature vectors of each resource domain are extracted. In this embodiment, the multidimensional feature vector is: in, The number of available CPU cores; For available GPU memory or computing power (e.g., 4*A10080G); This refers to the available memory capacity. The average network latency from the resource domain to the service node; This refers to the cost of resource usage (cost per unit of computing power / unit of time). Energy consumption in the resource domain; Data privacy compliance level is pre-assigned according to compliance policy; The resource domain type is cloud / edge / datacenter; Deploy a lightweight agent in each resource domain to collect local multidimensional feature vectors and encapsulate them into standardized messages; Lightweight agents deployed in resource domains periodically calculate local multidimensional feature vectors, using a centralized registry or a decentralized hash table as a distributed registry. When the change of feature value in the multidimensional feature vector exceeds the preset threshold, the incremental update is immediately reported to the registry to actively discover and synchronize the status of cross-LAN resource domains.

[0027] In this embodiment, a centralized registry or a decentralized hash table is used; the registry maintains a global resource state matrix. When a user initiates a task, the scheduler queries the registry to obtain the multi-dimensional feature vectors of all domains.

[0028] In one embodiment of the present invention, the real-time load vector of the resource domain is collected, the load for the next time period is predicted, and the total resource requirement is calculated based on the predicted load and service level agreement for the next time period, including the following steps: Collect real-time load vectors from the resource domain and define time intervals. Load feature vector ,in, Request delivery rate (requests per second); This is the length of the task queue; The number of available CPU cores; Available GPU memory or computing power; This refers to the available memory capacity. Based on the historical load feature vector of the resource domain, a hybrid model combining ARIMA-NN autoregressive integral moving average and neural network is used to predict the next time period. The load vector is given by the formula: in, To predict the next time period using the ARIMA model Request delivery rate; To predict the next time period using an LSTM model The load vector; For the next period of prediction The load vector; For adaptive fusion weights, By updating through online gradient descent, the prediction error threshold triggers retraining to avoid overfitting; According to the predicted next period Calculate the total resource requirements based on the load vector and SLA (Service Level Agreement). for: in, This represents the average computational load per task. Processing capacity per unit of resource; To allow for maximum utilization, in this embodiment, 0.8 is used to avoid overload.

[0029] In one embodiment of the present invention, a multi-objective optimization model is constructed using the current resource allocation and total resource demand of all resource domains, as well as the collaborative caching ratio between adjacent edge nodes, as decision variables to optimize the resource allocation of resource domains, including the following steps: The decision variables for setting the resource domain include: resource domain At any moment Current resource allocation Resource domain At any moment Total resource demand Resource domain To adjacent edge nodes Traffic ratio of collaborative caching In this embodiment, the resource domain At any moment Current resource allocation; Construct a multi-objective optimization model and use the alternating direction multiplier method for distributed solution: The constraints are: in, Let i be the unit resource cost of resource domain i; When the resource allocation exceeds the quota The penalty coefficient at that time is used to curb the overuse of expensive resources; This refers to the free or low-cost resource quota for resource domain i; exceeding this quota will trigger penalties. Let Q(t) be the unit resource energy consumption of resource domain i; Q(t) is the length of the system task queue at time t. For the predicted request delivery rate, For prediction window; This represents the processing rate per unit of resource in resource domain i, such as the number of tasks that each vCPU can process per second. The network round-trip delay from resource domain i to neighboring edge node j; The maximum latency is used as a normalized reference, such as the maximum value of all possible latency in the system, to scale the benefits of collaboration to a uniform scale. Data privacy compliance level; , These are adjustable weighting coefficients, which control the influence of privacy benefits and edge collaboration benefits on the objective function, respectively. This represents the edge node adjacent to resource domain i. This represents resource domain i within the resource pool, calculated by the load prediction model. This represents the upper limit of available resources for resource domain i at time t (limited by hardware configuration or cloud platform quota). Let i be the maximum cache capacity of edge node i; Let be the size of the k-th data block. It is a 0-1 variable, indicating whether data block k is cached in resource domain i at time t; Resource domain is solved in a distributed manner using the Alternating Direction Multiplier Method (ADMM). At any moment Total resource demand This yields the optimized resource allocation for the resource domain.

[0030] In one embodiment of the present invention, the expected number of instances in each resource domain is calculated based on the resource allocation amount obtained from multi-objective optimization, including the following steps: Based on the resource allocation obtained from multi-objective optimization, the expected number of instances is analyzed using the following formula: in, For the expected number of instances, This represents the resource capacity available for each instance in resource domain i. For example, if resource domain i uses a specification of 2 vCPUs per container, then... =2 (vCPU / instance). This value is determined by the type and configuration of the resource domain. Different domains can have different instance specifications. For example, an edge node may have only 0.5 vCPUs per instance, while a cloud node may have 8 vCPUs per instance.

[0031] In one embodiment of the present invention, a proportional-integral-derivative controller is used to smoothly adjust the number of instances, analyze the final scaling amount, and perform automatic scaling across cloud containers or virtual machines based on the native APIs of each resource domain, including the following steps: The proportional-integral-derivative (PID) controller is used to smoothly adjust the number of instances, taking into account the current number of instances with the available resource allocation. The final expansion / contraction amount is analyzed using the following formula: in, The recommended adjustment calculated for the PID controller; positive values ​​indicate that an instance needs to be increased, and negative values ​​indicate that an instance needs to be decreased. It is a continuous value and has not yet been rounded. The deviation at time t is the difference between the expected number of instances and the current number of instances. When e(t) > 0, it means that there are not enough instances and the capacity needs to be expanded; when e(t) < 0, it means that there are too many instances and the capacity needs to be reduced. This is the proportionality coefficient. This determines the immediate strength of the response to the current deviation; The larger the value, the faster the system responds to deviations, but this may lead to overshoot and oscillations. The smaller the value, the smoother the response, but it may not be able to eliminate the bias in time; The integral coefficient is... Used to eliminate steady-state error; the integral term accumulates the deviations of all past moments. If there is a persistent small deviation, such as the number of instances being slightly lower than the expected value for a long time, the integral term will gradually increase, driving the system to eventually eliminate the deviation. However, an excessively large integral term can also cause overshoot. These are the differential coefficients. The derivative term reacts in advance based on the rate of change (i.e., trend) of the deviation; when the deviation is rapidly increasing, the derivative term has a counteracting effect to suppress overshoot; when the deviation is rapidly decreasing, the derivative term can weaken the control action in advance to reduce oscillation; the derivative term helps to improve the stability of the system. It represents the integral of the deviation over time, that is, the sum (cumulative value) of the deviations at all past moments. Let i be the expected number of instances in resource domain i; This is the derivative of the deviation with respect to time, i.e., the rate of change of the deviation.

[0032] Final stretch And set the cooldown time. Based on the native APIs of each resource domain, it performs automatic scaling across cloud containers or virtual machines, where the round(⋅) function rounds the real number adjustment to the nearest integer. Since the number of instances must be an integer, it cannot add 0.3 instances. It can also use rounding up or rounding down, but usually rounding to the nearest zero is used. As a cooldown period, the system needs to wait after each extension / retraction operation is completed. The next scaling operation will only be performed after a few seconds; this can prevent frequent additions and subtractions of instances due to small fluctuations in load; it is usually set according to the characteristics of the resource domain, for example, 30 seconds for container environments and 2 minutes for virtual machine environments.

[0033] In one embodiment of the present invention, a comprehensive resilience factor is calculated for each edge node, and a resource borrowing strategy from the nearest cloud resource domain is analyzed based on the comprehensive resilience factor, including the following steps: For resource domains For adjacent edge nodes, analyze the resource demand elasticity factor according to different task types. Task types include: compute-intensive, I / O-intensive, and latency-sensitive. The formula is: in, This is the resource demand elasticity factor; For the next period of prediction The Load vector of the task class; For the first Baseline resource consumption for this type of task; The weighted coefficient for resource diversity is derived through regression analysis of historical operational data. In this embodiment, this formula replaces single-dimensional resource estimation and accurately identifies the proportion of CPU, memory, GPU, and network requirements for different services.

[0034] For resource domains When adjacent edge nodes encounter sudden loads, they prioritize requesting cooperation from the nearest edge node and analyze the emergency resilience factor, as shown in the formula: in, For emergency resilience factors; This represents the length of the current node's task queue. This represents the maximum queue capacity of the node. This represents the current CPU utilization. The resource demand elasticity factor and the emergency elasticity factor are normalized, and the comprehensive elasticity factor is obtained by weighted summation of the normalized resource demand elasticity factor and the emergency elasticity factor. The overall elasticity factor is divided into four intervals, and a corresponding resource borrowing strategy is set for each interval.

[0035] In one embodiment of the present invention, a corresponding resource borrowing strategy is set for each interval, including: If 0.5 ≤ comprehensive elasticity factor < 1.0, the resource borrowing strategy is: no cloud borrowing; If 0.5 ≤ comprehensive elasticity factor < 1.0, the resource borrowing strategy is: pre-allocate cloud resources; If 1.0 ≤ comprehensive elasticity factor < 1.5, the resource borrowing strategy is: first allocate edge nodes, then allocate cloud resources according to demand; If the overall elasticity factor is ≥1.5, the resource borrowing strategy is: directly borrow cloud resources in an emergency.

[0036] Another implementation of this embodiment is that if the comprehensive elasticity factor is <0.5, the resource borrowing strategy is: no cloud borrowing, only local scheduling of edge nodes is maintained; If 0.5 ≤ comprehensive elasticity factor < 1.0, the resource borrowing strategy is: pre-allocate a small amount of spare resources from the nearest cloud resource domain, increase the resource quota by 10%-20%, and use it only for non-latency-sensitive tasks; If 1.0 ≤ comprehensive elasticity factor < 1.5, the resource borrowing strategy is as follows: first try collaboration between edge nodes; if the queue continues to grow, allocate resources from the nearest cloud resource domain according to demand, increase the resource quota to 50% of the demand, and keep the cost weight unchanged. If the overall elasticity factor is ≥1.5, the resource borrowing strategy is as follows: skip the edge collaboration step, borrow resources directly from the nearest cloud resource domain, reduce the cost weight to 0.5 times the original, prioritize the protection of latency-sensitive tasks, and increase the resource quota to 100% of the demand.

[0037] In one embodiment of the present invention, an improved differential round-robin algorithm is used for packet scheduling at the network egress point, dynamically adjusting the supplementary granularity of each scheduling, and establishing an adaptive deficit compensation mechanism, including the following steps: At the network egress point where the edge node connects to the cloud resource domain, an improved differential round-robin scheduling method is adopted for cross-domain resource call message distribution and scheduling. Combined with the calculated edge node comprehensive elasticity factor, a dynamic granularity adjustment algorithm is established, with the following formula: in, To supplement the granularity of dynamic scheduling; Maximum transmission unit; For the most recent at time t Average length of each message; Let be the standard deviation of the message length at time t; Let be the comprehensive elasticity factor at time t; This is an adjustable coefficient. ; and The weighting coefficients are, in order, the weighting coefficients for the standard deviation of message length and the weighting coefficients for the comprehensive elasticity factor; among them, + =1, in this embodiment =1.3, =0.4, =0.6; Simultaneously, an adaptive deficit compensation mechanism is introduced: after each round of scheduling, the deficit counter... Updated based on actual sent volume, when At that time, an additional compensation round is triggered. In this example, to trigger the threshold coefficient, =0.6, It adds granularity to dynamic scheduling and avoids counter flipping; it enables smooth traffic and high accuracy to be maintained even in different network environments across clouds (such as some clouds that mainly use small packets).

[0038] In this embodiment, under high-load emergency cloud resource retrieval scenarios, priority is given to ensuring the normal forwarding of core resource scheduling messages, and the convergence pace of deficit compensation is reasonably slowed down. After the node load decreases and the resource borrowing demand stabilizes, the traffic compensation convergence is completed smoothly. This ensures the rapid passage of cloud resource borrowing messages under high-pressure scenarios and maintains a balanced traffic distribution among multiple interconnected links in the long term. It solves problems such as the rigid granularity of traditional polling scheduling, long-term traffic distribution imbalance, and high-load scheduling lag, and achieves efficient, fair, and adaptive scheduling of resource call messages between the edge and the cloud.

[0039] In another embodiment of the present invention, a cross-cloud resource elastic scheduling system for management system development is provided. This system adopts the cross-cloud resource elastic scheduling method for management system development as described above to realize cross-cloud resource elastic scheduling for management system development.

[0040] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A cross-cloud resource elastic scheduling method for management system development, characterized in that, Includes the following steps: The local data center, cloud platform and edge nodes are defined as resource domains. The multi-dimensional feature vector of each resource domain is extracted, and the active discovery and state synchronization of cross-LAN resource domains are carried out through a distributed registry. Collect real-time load vectors of resource domains, predict the load for the next time period, and calculate the total resource requirements based on the predicted load and service level agreement for the next time period. Using the current resource allocation and total resource demand of all resource domains, as well as the collaborative caching ratio between adjacent edge nodes, as decision variables, a multi-objective optimization model is constructed to optimize the resource allocation of resource domains. Based on the resource allocation obtained from multi-objective optimization, the expected number of instances in each resource domain is calculated. A proportional-integral-derivative controller is used to smoothly adjust the number of instances. The final scaling is analyzed, and automatic scaling across cloud containers or virtual machines is executed based on the native API of each resource domain. At the same time, a comprehensive elasticity factor is calculated for each edge node, and a resource borrowing strategy from the nearest cloud resource domain is analyzed based on the comprehensive elasticity factor. An improved differential round-robin algorithm is used for packet scheduling at the network egress point, dynamically adjusting the replenishment granularity for each scheduling, while establishing an adaptive deficit compensation mechanism.

2. The cross-cloud resource elastic scheduling method and system for management system development according to claim 1, characterized in that, The local data center, cloud platform, and edge nodes are defined as resource domains. A multi-dimensional feature vector is extracted for each resource domain, and proactive discovery and state synchronization of cross-LAN resource domains are performed through a distributed registry, including the following steps: Local data centers, cloud platforms and edge nodes are defined as resource domains. Logical resource pools are built between resource domains where data transmission occurs, and multi-dimensional feature vectors of each resource domain are extracted. Deploy a lightweight agent in each resource domain to collect local multidimensional feature vectors and encapsulate them into standardized messages; Lightweight agents deployed in resource domains periodically calculate local multidimensional feature vectors, using a centralized registry or a decentralized hash table as a distributed registry. When the change of feature value in the multidimensional feature vector exceeds the preset threshold, the incremental update is immediately reported to the registry to actively discover and synchronize the status of cross-LAN resource domains.

3. The cross-cloud resource elastic scheduling method and system for management system development according to claim 1, characterized in that, Collect real-time load vectors of resource domains, predict the load for the next time period, and calculate the total resource requirements based on the predicted load and service level agreement for the next time period, including the following steps: Collect real-time load vectors from the resource domain and define time intervals. Load feature vector ,in, For request arrival rate; This is the length of the task queue; The number of available CPU cores; Available GPU memory or computing power; This refers to the available memory capacity. Based on the historical load feature vector of the resource domain, a hybrid model combining ARIMA-NN autoregressive integral moving average and neural network is used to predict the next time period. The load vector is given by the formula: in, To predict the next time period using the ARIMA model Request delivery rate; To predict the next time period using an LSTM model The load vector; For the next period of prediction The load vector; For adaptive fusion weights; According to the predicted next period Calculate the total resource requirements based on the load vector and SLA (Service Level Agreement). for: in, This represents the average computational load per task. Processing capacity per unit of resource; To allow for maximum utilization.

4. The cross-cloud resource elastic scheduling method and system for management system development according to claim 1, characterized in that, Using the current resource allocation and total resource demand of all resource domains, as well as the collaborative caching ratio between adjacent edge nodes, as decision variables, a multi-objective optimization model is constructed to optimize the resource allocation of resource domains, including the following steps: The decision variables for setting the resource domain include: resource domain At any moment Current resource allocation Resource domain At any moment Total resource demand Resource domain To adjacent edge nodes Traffic ratio of collaborative caching ; Construct a multi-objective optimization model and use the alternating direction multiplier method for distributed solution: The constraints are: in, Let i be the unit resource cost of resource domain i; When the resource allocation exceeds the quota The penalty coefficient at that time is used to curb the overuse of expensive resources; Free or low-cost resource quotas for resource domain i; Let Q(t) be the unit resource energy consumption of resource domain i; Q(t) is the length of the system task queue at time t. For the predicted request delivery rate, For prediction window; The processing rate per unit resource in resource domain i; The network round-trip delay from resource domain i to neighboring edge node j; The normalized reference maximum delay; Data privacy compliance level; , These are adjustable weighting coefficients; This represents the edge node adjacent to resource domain i. Represents resource domain i within the resource pool; Let be the upper limit of available resources in resource domain i at time t; Let i be the maximum cache capacity of edge node i; The size of the k-th data block; It is a 0-1 variable, indicating whether data block k is cached in resource domain i at time t; Resource domain is solved in a distributed manner using the Alternating Direction Multiplier Method (ADMM). At any moment Total resource demand This yields the optimized resource allocation for the resource domain.

5. The cross-cloud resource elastic scheduling method and system for management system development according to claim 1, characterized in that, Based on the resource allocation obtained from multi-objective optimization, the expected number of instances in each resource domain is calculated, including the following steps: Based on the resource allocation obtained from multi-objective optimization, the expected number of instances is analyzed using the following formula: in, For the expected number of instances, The resource capacity that can be provided for each instance in resource domain i.

6. The cross-cloud resource elastic scheduling method and system for management system development according to claim 1, characterized in that, The system employs a proportional-integral-derivative (PID) controller to smoothly adjust the number of instances, analyzes the final scaling amount, and performs automatic scaling across cloud containers or virtual machines based on the native APIs of each resource domain. This includes the following steps: The proportional-integral-derivative controller is used to smoothly adjust the number of instances, taking into account the current number of instances with the available resources. The final expansion / contraction amount is analyzed using the following formula: in, Recommended adjustment amount calculated for the PID controller; The deviation at time t; This is the proportionality coefficient; The integral coefficient; These are the differential coefficients; This represents the integral of the deviation over time. Let i be the expected number of instances in resource domain i; Final stretch And set the cooldown time. It performs automatic scaling across cloud containers or virtual machines based on the native APIs of each resource domain, where round(⋅) is the rounding function.

7. The cross-cloud resource elastic scheduling method and system for management system development according to claim 1, characterized in that, Calculate the comprehensive resilience factor for each edge node, and analyze the resource borrowing strategy from the nearest cloud resource domain based on the comprehensive resilience factor, including the following steps: For resource domains For adjacent edge nodes, analyze the resource demand elasticity factor according to different task types. Task types include: compute-intensive, I / O-intensive, and latency-sensitive. The formula is: in, This is the resource demand elasticity factor. For the next period of prediction The Load vector of the task class; For the first Baseline resource consumption for this type of task; Weighting coefficient for resource diversity; For resource domains When adjacent edge nodes encounter sudden loads, they prioritize requesting cooperation from the nearest edge node and analyze the emergency resilience factor, as shown in the formula: in, For emergency resilience factors; This represents the length of the current node's task queue. This represents the maximum queue capacity of the node. This represents the current CPU utilization rate. The resource demand elasticity factor and the emergency elasticity factor are normalized, and the comprehensive elasticity factor is obtained by weighted summation of the normalized resource demand elasticity factor and the emergency elasticity factor. The overall elasticity factor is divided into four intervals, and a corresponding resource borrowing strategy is set for each interval.

8. The cross-cloud resource elastic scheduling method and system for management system development according to claim 7, characterized in that, Set corresponding resource borrowing policies for each interval, including: If 0.5 ≤ comprehensive elasticity factor < 1.0, the resource borrowing strategy is: no cloud borrowing; If 0.5 ≤ comprehensive elasticity factor < 1.0, the resource borrowing strategy is: pre-allocate cloud resources; If 1.0 ≤ comprehensive elasticity factor < 1.5, the resource borrowing strategy is: first allocate edge nodes, then allocate cloud resources according to demand; If the overall elasticity factor is ≥1.5, the resource borrowing strategy is: directly borrow cloud resources in an emergency.

9. The cross-cloud resource elastic scheduling method and system for management system development according to claim 1, characterized in that, An improved differential round-robin algorithm is used for packet scheduling at the network egress point, dynamically adjusting the replenishment granularity for each scheduling session, and establishing an adaptive deficit compensation mechanism, including the following steps: At the network egress point where the edge node connects to the cloud resource domain, an improved differential round-robin scheduling method is adopted for cross-domain resource call message distribution and scheduling. Combined with the calculated edge node comprehensive elasticity factor, a dynamic granularity adjustment algorithm is established, with the following formula: in, To supplement the granularity of dynamic scheduling; Maximum transmission unit; For the most recent at time t Average length of each message; Let be the standard deviation of the message length at time t; Let be the comprehensive elasticity factor at time t; This is an adjustable coefficient. ; and The weighting coefficients are, in order, the weighting coefficients for the standard deviation of message length and the weighting coefficients for the comprehensive elasticity factor; Simultaneously, an adaptive deficit compensation mechanism is introduced: after each round of scheduling, the deficit counter... Updated based on actual sent volume, when At that time, an additional compensation round is triggered. This is the trigger threshold coefficient.

10. A cross-cloud resource elastic scheduling system developed for management systems, characterized in that: The system adopts the cross-cloud resource elastic scheduling method for management system development as described in any one of claims 1-9 to realize cross-cloud resource elastic scheduling for management system development.

Images