Multi-modal network resource scheduling method and system based on resource blocks

Abstract

The invention relates to the technical field of network resource management, in particular to a multi-modal network resource scheduling method and system based on a resource block, and the method comprises the steps: building a multi-dimensional service demand model, quantifying each resource demand in the multi-dimensional service demand model, and forming a quantitative index; sensing physical resources, abstracting the physical resources, and dividing the abstracted physical resources into resource blocks; acquiring a service request, performing resource block allocation by adopting a comprehensive allocation strategy, and performing dynamic adjustment in an operation process; the target of the comprehensive allocation strategy is resource block load balance, and the matching degree of the service demand in the service request and the resource block attribute is optimal; according to the scheme, through analysis modeling and reasonable scheduling calculation, storage, network and other resources of the service request, demand matching of the resource blocks and the service request is ensured, the number and capacity of the resource blocks can be dynamically adjusted to adapt to instantaneous change of the service, and full utilization of bottom layer physical resources can be ensured in the change process of the resource blocks.

Description

Technical Field

[0001] The present invention relates to the technical field of network resource management, and in particular to a multi-modal network resource scheduling method based on resource blocks. Background Art

[0002] With the rapid development of technologies such as cloud computing, edge computing, and 5G networks, modern networks and computing systems are becoming more and more complex, and user needs are becoming increasingly diverse. At the same time, the limitations of physical resources and the complexity of network topology make it an urgent problem to efficiently schedule and manage these diverse computing and network resources.

[0003] Traditional scheduling methods are mainly based on static resource allocation models, which perform well when facing stable business loads. However, in modern network environments, this method is difficult to cope with instantly changing business needs. The unpredictability of business loads (such as the burstiness of user requests, the complexity of computing tasks, etc.) makes it impossible for traditional static scheduling methods to adapt to business needs in a timely manner, resulting in problems such as over-allocation of resources or low resource utilization.

[0004] In recent years, with the maturity of virtualization technology, the introduction of containerization technology and network function virtualization (NFV) technology has gradually improved the flexibility and management efficiency of computing and storage resources. Virtualization technology abstracts physical resources into multiple logical resource pools to achieve on-demand allocation and elastic expansion of resources. Containerization technology (such as Docker) enables applications to be quickly deployed and effectively isolated through lightweight virtualization. However, there are still great challenges in the abstraction and unified scheduling of network resources.

[0005] The scheduling and management of network resources have become particularly critical, especially in the era of computing-network integration, where computing resources and network resources are closely coupled. To achieve efficient business execution, not only dynamic scheduling of computing and storage resources is required, but network resources must also be able to be flexibly allocated to ensure rapid data transmission between different nodes and meet the latency and bandwidth requirements of the service.

[0006] To address these issues, researchers proposed the concept of resource blocks in their research. Resource blocks are a new type of resource management and scheduling unit that uniformly encapsulates and abstracts computing, storage, network and other resources so that these resources can be managed and scheduled as a whole. Resource blocks not only solve the problem of fragmented scheduling of different resource types, but can also dynamically adjust their internal resource composition according to business needs, thereby achieving elastic expansion and efficient utilization of resources.

[0007] The resource block-based construction method provides an infrastructure for solving the problem of network resource coordination. However, the construction of resource blocks alone cannot fully meet the dynamic changes and diversification of modern business needs. To achieve efficient business processing in a complex network environment, the key lies in how to efficiently match business requests with resource blocks, and how to ensure efficient operation of the business through dynamic adjustment and allocation of resource blocks.

[0008] Therefore, how to implement network resource scheduling based on resource blocks in a dynamic and complex network environment so that computing, storage, and network resources can work together has become an important direction for the development of current network resource management technology. Summary of the invention

[0009] The present invention aims to provide a multimodal network resource scheduling method based on resource blocks. By analyzing and modeling business requests, computing, storage, network and other resources are reasonably scheduled to ensure that resource blocks match the needs of business requests. The number and capacity of resource blocks can be dynamically adjusted to adapt to instantaneous changes in business, and the full utilization of underlying physical resources can be ensured during the change of resource blocks.

[0010] The present invention provides the following basic solution: a multi-modal network resource scheduling method based on resource blocks, including the following contents:

[0011] S1. Establish a multi-dimensional business demand model and quantify each resource requirement in the multi-dimensional business demand model to form quantitative indicators;

[0012] S2, perceive physical resources, abstract the physical resources, and divide the abstracted physical resources into resource blocks;

[0013] S3. Obtain the service request, adopt a comprehensive allocation strategy, allocate resource blocks, and dynamically adjust them during operation; the goal of the comprehensive allocation strategy is to balance the load of resource blocks, and the matching degree between the service requirements in the service request and the resource block attributes is optimal;

[0014] S4. Construct a dynamic change model that characterizes the change of resource status of the resource block when processing services, and manage and update the resource block according to the dynamic change model;

[0015] S5. Use management strategies to manage resource blocks and make adaptive adjustments based on dynamic changes in load conditions and business needs.

[0016] Furthermore, the multi-dimensional business requirements in the multi-dimensional business requirements model include: computing resource requirements, storage resource requirements, network resource requirements, task execution time and scheduling priority;

[0017] Computing resource requirements, including the number of CPU cores, number and type of GPUs, and parallelism of computing tasks required by the business;

[0018] Storage resource requirements, including: storage space size, storage type, and data access speed requirements required by the business;

[0019] Network resource requirements, including bandwidth, latency, and jitter tolerance required by the business;

[0020] Task execution time and scheduling priority, including: business execution time requirements, task scheduling urgency and priority;

[0021] The physical resources include computing resources, storage resources and network resources;

[0022] Through resource monitoring and management tools, the real-time status of physical resources on each node can be perceived;

[0023] Abstract physical resources: Through virtualization technology, physical resources are abstracted into logical resource pools;

[0024] The abstracted physical resources are divided into resource blocks. Based on the perception and abstraction of physical resources, the physical resources are divided into resource blocks. A resource block is a logical unit that aggregates and encapsulates multiple physical resources. Each resource block corresponds to all resources on a physical node, or aggregates resources across multiple physical nodes. Each resource block has a set of attribute parameters to describe its resource characteristics.

[0025] Furthermore, the matching degree between the business requirements and the resource block attributes includes:

[0026] Establish a classification model for business requirements and quantize business attributes:

[0027] The business requirement attribute vector of business request i is D i =[d i,1 ,d i,2 ,…,d i,n ],in:

[0028] d i,1 Represents computing resource requirements (such as CPU or GPU);

[0029] d i,2 Represents storage resource requirements;

[0030] d i,3 Represents the network bandwidth requirement;

[0031] …

[0032] d i,n Represents other resource requirements;

[0033] The attribute vector of resource block j is R j =[r j,1 ,r j,2 ,…,r j,n ], indicating the supply capacity of various resources in the resource block, where:

[0034] r j,1 Represents the computing capacity of the resource block;

[0035] r j,2 Represents storage capacity;

[0036] r j,3 Represents network bandwidth;

[0037] …

[0038] r j,n Represents other resource attributes;

[0039] Set the matching function to measure the matching degree between the service request and the resource block, and use the vector inner product to measure the matching degree:

[0040]

[0041] Among them, w k is the weight coefficient of each resource dimension, and the weights of different resource types are adjusted according to the importance of business requirements; the goal is to allocate business requests to the resource blocks that best match their resource demand attributes, maximizing the matching degree M i,j .

[0042] Furthermore, the load balancing of the resource blocks includes:

[0043] Set the load function to the resource utilization of the resource block. If the resource utilization of each resource block j is L j,k , the overall load can be expressed as the weighted average of all resource utilizations in the resource block:

[0044]

[0045] in

[0046]

[0047] The goal of resource block load balancing is to balance the load of all resource blocks by allocating service requests and to minimize the maximum load L. j With minimum load L j , or directly minimize the variance of the resource block:

[0048]

[0049] in, is the average load of all resource blocks.

[0050] Further, the S3 includes:

[0051] Initial allocation, based on the matching function M i,j , allocate the resource block j with the highest matching degree to service request i, the steps are:

[0052] For all service requests i and all resource blocks j, calculate the matching degree M i,j .

[0053] For each service request i, select the resource block j with the greatest matching degree, that is: j* = argmax j M i,j ;

[0054] After the initial allocation is completed, the remaining resource capacity of the resource block is updated;

[0055] Load balancing adjustment steps are:

[0056] Calculate the load L of each resource block j , and determine the resource block j with the highest current load max and the resource block j with the lowest load min ;

[0057] In j max Select several business requests and migrate them to j min ;

[0058] The adjusted migration process can be performed iteratively until the loads of all resource blocks are balanced within the preset allowable range;

[0059] Dynamic adjustment mechanism, the steps are:

[0060] Dynamic expansion and reduction of resource blocks: When the number of business requests exceeds the processing capacity of existing resource blocks, new resource blocks are dynamically added and the resource block capacity is expanded in real time according to business needs. At the same time, when the business load decreases, the resource blocks are released.

[0061] Online monitoring and migration: Through real-time monitoring of resource blocks, if there is a resource block overload, load balancing adjustment will be performed.

[0062] Furthermore, the dynamic change model includes:

[0063] Resource Block Resource Model:

[0064] R j =(r j,1 ,r j,2 ,…,r j,n );

[0065] Business resource demand model:

[0066] D i =(d i,1 ,d i,2 ,…,d i,n )

[0067] When service i is allocated to resource block R j When the resource block is on, the resource consumption is:

[0068]

[0069] Among them, r j ' ,k Indicates that during the execution of service i, resource block R j The remaining resources on the k-th resource.

[0070] Further, the management and update of resource blocks according to the dynamic change model includes:

[0071] When service i is completed, resource block R j The resources occupied by the business will be released, thus returning to the previous state, as shown:

[0072]

[0073] At time t, resource block R j The resource usage is R j (t)=(r j,1 (t),r j,2 (t),…,r j,n (t));

[0074] When a new service request i arrives and is allocated to resource block R j When , the resource usage changes at time t+1 as follows:

[0075]

[0076] When service request i arrives at time t i After completion, its resource requirements are released and the state of the resource block is restored to:

[0077]

[0078] Each resource block introduces a residual resource vector Represents the remaining available resources of the resource block at time t:

[0079]

[0080] After service request i is allocated, the remaining resources are updated to:

[0081]

[0082] When service request i arrives at time t i After completion, its resource requirements are released and the remaining resources are restored to:

[0083]

[0084] Furthermore, the management strategy includes: expansion and reduction strategies of resource blocks;

[0085] When the remaining resources of a resource block are insufficient to meet new service requests, the resource block is automatically expanded; conversely, when the utilization rate of a resource block is low, the resource block is dynamically reduced;

[0086] Scaling out and scaling down policies are based on preset thresholds, including:

[0087] If the load of a resource block is L j (t) is greater than or equal to the preset threshold, then resource expansion is triggered:

[0088]

[0089] If the load of a resource block is less than the preset threshold, the resource is reduced:

[0090] L j (t)<θ shrink .

[0091] Furthermore, the management strategy also includes: a migration and balancing strategy for resource blocks;

[0092] When the load of a resource block is too high, some service requests are migrated to other resource blocks to rebalance resources.

[0093] Triggered by real-time monitoring of resource utilization, including:

[0094] If the resource block R j Load L j (t) is higher than the preset load difference of other resource blocks, then some services i are migrated to resource blocks R with lower load. k superior:

[0095] And L k (t) <L j (t).

[0096] Beneficial effects: This solution analyzes and models business requests, constructs physical resources into resource blocks, and reasonably schedules computing, storage, network and other resources to ensure that resource blocks match the needs of business requests, and performs dynamic scheduling to ensure resource block load balancing; and dynamically adjusts the number and capacity of resource blocks during use to adapt to instantaneous changes in business, and ensures that the underlying physical resources are fully utilized during the change of resource blocks.

[0097] The update and management process of resource blocks realizes the dynamic tracking and adjustment of resource block status. Through the resource consumption model, resource release model and remaining resource update formula, real-time monitoring of the resource status of each resource block is realized. At the same time, by introducing expansion and reduction strategies as well as migration and rebalancing strategies, it ensures that the system can efficiently schedule and manage resources when facing dynamic business requests.

[0098] This solution achieves efficient matching between business requests and resource blocks, ensuring dynamic scheduling of computing, storage and network resources; provides an elastic expansion mechanism based on resource blocks, dynamically adjusting the number and capacity of resource blocks according to changes in business requests; ensures efficient configuration and scheduling of resource blocks through full perception and virtualization of underlying physical resources; ensures that system resources are stable and reliable during dynamic scheduling, avoiding resource competition and load imbalance.

[0099] The present invention also provides a multimodal network resource scheduling system based on resource blocks. By analyzing and modeling business requests, it reasonably schedules computing, storage, network and other resources to ensure that resource blocks match the needs of business requests. It can dynamically adjust the number and capacity of resource blocks to adapt to instantaneous changes in business, and ensure that the underlying physical resources are fully utilized during the change of resource blocks.

[0100] The present invention provides the following basic solution: a multimodal network resource scheduling system based on resource blocks, which adopts the multimodal network resource scheduling method based on resource blocks. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] Figure 1 The present invention is a flowchart of an embodiment of a multi-modal network resource scheduling method based on resource blocks. DETAILED DESCRIPTION

[0102] The following is further described in detail through specific implementation methods:

[0103] The embodiment is basically as shown in the attached Figure 1 As shown: A multi-modal network resource scheduling method based on resource blocks, including the following contents:

[0104] S1. Demand modeling of business requests, establishing a multi-dimensional business demand model, and quantifying each resource requirement in the multi-dimensional business demand model to form quantitative indicators;

[0105] The resource requirements of the business in the business request are abstracted into a quantifiable and operational model for subsequent matching and scheduling with resource blocks. Business requests usually involve multiple resource requirements such as computing, storage, and network, and the resource requirements of each business may be different, so these support requirements need to be modeled in a refined manner and matched with the structure of the resource block.

[0106] The multi-dimensional business requirements in the multi-dimensional business requirements model include: computing resource requirements, storage resource requirements, network resource requirements, task execution time and scheduling priority;

[0107] Computing resource requirements, including the number of CPU cores, number and type of GPUs, and parallelism of computing tasks required by the business. Different businesses may have different performance requirements for computing resources, so it is also necessary to model the processing capabilities. For example, the requirements for computing performance can be expressed in terms of floating-point operations per second (FLOPS) or CPU frequency.

[0108] Storage resource requirements, including: the size of storage space required by the business, storage type (such as high-speed SSD, traditional HDD, etc.) and data access speed requirements; some businesses may require the division of temporary storage and long-term storage, as well as storage redundancy and disaster recovery requirements;

[0109] Network resource requirements, including: bandwidth, latency, jitter tolerance, etc. required by the service; this is especially important for data-intensive and latency-sensitive services; in addition, network resource modeling should take into account cross-node and cross-region resource scheduling to ensure quality of service (QoS).

[0110] Task execution time and scheduling priority, including: business execution time requirements, task scheduling urgency and priority; this can help the system prioritize high-priority tasks when resources are scarce.

[0111] In order to facilitate the system to make decisions during scheduling, the multi-dimensional business demand model should be quantifiable, that is, each resource requirement (computing, storage, network) is expressed as a quantitative indicator, which is expressed in the form of JSON string key-value pairs in this embodiment; in the multi-dimensional business demand model, the business request is modeled in detail through attributes such as business type, priority, required computing resources, storage resources and network resources to ensure that the business can match the resource block. Each part of the business demand directly corresponds to different resource block attributes in the system, such as:

[0112] {

[0113] "business_type":"video_streaming", / / Business type, such as video streaming

[0114] "business_priority":"high", / / Business priority, such as high priority

[0115] "business_modality":"IP", / / The modality of the business, here is IP modality

[0116] "cpu_flops_tflops":10, / / Required CPU floating point computing power (unit: TFLOPS)

[0117] "gpu_flops_tflops":40, / / Required GPU floating point computing power (unit: TFLOPS)

[0118] "memory_required_gb":16, / / Required memory size (unit: GB)

[0119] "storage_capacity_required_gb":500, / / Required storage capacity (unit: GB)

[0120] "storage_io_performance":"high", / / Storage I / O performance requirements: can be low, medium, high

[0121] "network_bandwidth_required_mbps":100 / / Required network bandwidth (unit: Mbps)}

[0122] S2: Modeling available resources, sensing physical resources, and abstracting physical resources, dividing the abstracted physical resources into resource blocks;

[0123] Specifically, it senses physical resources, including computing resources (CPU, GPU, etc.), storage resources (local storage, distributed storage, etc.), and network resources (bandwidth, latency, jitter, etc.). It senses the real-time status of physical resources on each node through various resource monitoring and management tools (such as performance monitoring agents, network probes, etc.).

[0124] Abstract physical resources. Through virtualization technology, physical resources are abstracted into logical resource pools to facilitate business scheduling and unified management. Logical resource pools provide basic resources for resource blocks, and resource blocks are allocation units of logical resource pools. They can be flexibly configured and scheduled according to business needs. Specifically, a JSON structure can be used to represent the resource information of each physical server. Given three physical servers (Server1, Server2, and Server3), the resource attributes of each server include CPU computing power, GPU computing power, memory size, storage capacity, and storage I / O performance. The attributes of the physical link represent the network bandwidth between servers.

[0125] The following is the resource information of these physical servers:

[0126]

[0127]

[0128] The abstracted physical resources are divided into resource blocks. Based on the perception and abstraction of physical resources, the physical resources are divided into resource blocks. That is, the subsets divided on demand in the logical resource pool are used as resource blocks. The resource block is a logical unit that aggregates and encapsulates multiple physical resources (computing, storage, and network). Each resource block can correspond to all resources on a physical node, or aggregate resources across multiple physical nodes to meet business needs. In addition, each resource block needs to have a set of clear attribute parameters to describe its resource characteristics, such as the computing power of the CPU / GPU, storage capacity, network bandwidth, latency, and reliability.

[0129] S3, business matching and resource scheduling, obtain business requests, adopt comprehensive allocation strategies, allocate resource blocks, and make dynamic adjustments during operation;

[0130] The goal of the comprehensive allocation strategy is to balance the load of resource blocks, and to achieve the best match between the service requirements and the resource block attributes in the service request;

[0131] When allocating service requests to resource blocks, it is necessary to comprehensively consider the matching degree between service requirements and resource block attributes as well as the load balancing of resource blocks. This process can be understood as a dual optimization problem, which is to maximize the matching degree between service requirements and resource block capabilities while avoiding resource block overload or insufficient utilization of some resource blocks. Therefore, a comprehensive allocation strategy with the goal of resource block load balancing and the best matching degree between service requirements and resource block attributes in service requests is set for allocation;

[0132] The matching degree between business requirements and resource block attributes includes:

[0133] Each business request has specific resource requirements, which can be divided into computing resource requirements, storage resource requirements, network resource requirements, etc. According to the type and intensity of business requirements, some attributes are defined for different business requests, such as:

[0134] Computation-intensive services: require a large amount of CPU / GPU computing resources;

[0135] Storage-intensive services: services that require a large amount of storage resources, such as large-scale databases or file processing services;

[0136] Bandwidth-intensive services: These services require a large amount of network resources, such as video streaming and cloud gaming.

[0137] In order to better allocate business requests to resource blocks, it is necessary to first establish a classification model for business requirements and quantize business attributes, which can be expressed as follows:

[0138] The business requirement attribute vector of business request i is D i =[d i,1 ,d i,2 ,…,d i,n ],in:

[0139] d i,1 Represents computing resource requirements (such as CPU or GPU);

[0140] d i,2 Represents storage resource requirements;

[0141] d i,3 Represents the network bandwidth requirement;

[0142] …

[0143] d i,n Represents other resource requirements (such as memory, I / O, etc.) that can be further refined according to the scenario.

[0144] The attribute vector of resource block j is R j =[r j,1 ,r j,2 ,…,r j,n ], indicating the supply capacity of various resources in the resource block, where:

[0145] r j,1 Represents the computing capacity of the resource block;

[0146] r j,2 Represents storage capacity;

[0147] r j,3 Represents network bandwidth;

[0148] …

[0149] rj,n Represents other resource attributes to be added based on actual conditions;

[0150] Set the matching function to measure the matching degree between the service request and the resource block, and use the vector inner product to measure the matching degree:

[0151]

[0152] Among them, w k is the weight coefficient of each resource dimension, and the weights of different resource types are adjusted according to the importance of business requirements; the goal is to allocate business requests to the resource blocks that best match their resource requirement attributes, that is, to maximize the matching degree M i,j .

[0153] The description format of the resource block in this embodiment corresponds to the business requirements:

[0154]

[0155]

[0156] The load balancing of resource blocks specifically includes:

[0157] In terms of resource block load balancing, the goal is to avoid overloading some resource blocks while underutilizing other resource blocks. Therefore, a load balancing mechanism needs to be designed to keep the load of resource blocks relatively balanced and ensure that the allocation of service requests on resource blocks can achieve the purpose of maximizing resource utilization.

[0158] Set the load function to the resource utilization of the resource block. If the resource utilization of each resource block j is L j,k , the overall load can be expressed as the weighted average of all resource utilizations in the resource block:

[0159]

[0160] in

[0161]

[0162] The goal of resource block load balancing is to keep the load of all resource blocks balanced by allocating service requests, that is, to minimize the load imbalance. This can be achieved by minimizing the maximum load L j With minimum load L j , or directly minimize the variance of the resource block:

[0163]

[0164] in, is the average load of all resource blocks.

[0165] In order to optimize the matching degree between business requirements and resource blocks and maintain the load balance of resource blocks, a comprehensive allocation strategy is set. The specific steps are as follows:

[0166] Preliminary allocation, with matching priority

[0167] According to the matching function M i,j , allocate the resource block j with the highest matching degree to the service request i, that is, prioritize the service request to the resource block that best matches its needs. The specific steps are:

[0168] a) For all service requests i and all resource blocks j, calculate the matching degree M i,j .

[0169] b) For each service request i, select the resource block j with the greatest matching degree, that is: j* = argmax j M i,j ;

[0170] c) After the initial allocation is completed, the remaining resource capacity of the resource block is updated.

[0171] Load balancing adjustment, with load balancing as priority

[0172] After the initial allocation, some resource blocks may be overloaded while other resource blocks may be underloaded. To ensure load balancing of the system, further adjustments need to be made to the allocation results. The specific steps are as follows:

[0173] a) Calculate the load L of each resource block j , and determine the resource block j with the highest current load max and the resource block j with the lowest load min ;

[0174] b) In j max Select several business requests and migrate them to j min , ensuring that the load difference after migration is reduced;

[0175] c) The adjusted migration process can be performed iteratively until the loads of all resource blocks are balanced within a preset allowable range;

[0176] The preset allowable range includes: setting the allowable load difference range and load threshold between resource blocks; when the load rate of a resource block reaches or exceeds 90%, regardless of whether the load difference with other resource blocks reaches the original difference (such as 30%), the new load will be allocated or transferred to the resource block with lower load. This strategy ensures that when the resource block load is close to saturation, tasks are dispersed in time to avoid bottlenecks in the system due to local overload. Specifically, basic load balancing conditions: Under normal circumstances, the system determines whether load migration or reallocation is required based on the load difference of each resource block. For example, if the allowable load difference is 30%, the system will perform load balancing operations when the load difference between resource blocks exceeds 30%.

[0177] High load threshold trigger condition: When the load of a resource block reaches or exceeds 90%, the system will trigger priority allocation or load migration even if the load difference with other resource blocks does not reach the allowed difference (for example, 30%); specifically,

[0178] ① When allocating new tasks, prioritize allocating tasks to resource blocks with a load below 90%, and avoid putting additional load into resource blocks that have already reached 90% load;

[0179] ② When performing load migration, even if the load difference requirement is not met, as long as the load of a resource block exceeds 90%, load transfer will be performed to relieve the pressure on the resource block.

[0180] Dynamic adjustment mechanism

[0181] During system operation, resource blocks and service requests change dynamically. The arrival and end of service requests will affect the system load and allocation strategy. Therefore, a dynamic adjustment mechanism needs to be designed to ensure that the system can adaptively adjust the service allocation strategy.

[0182] Dynamic adjustment mechanism, including the following:

[0183] a) Dynamic expansion and reduction of resource blocks: When the number of business requests exceeds the processing capacity of existing resource blocks, new resource blocks are dynamically added and the resource block capacity is expanded in real time according to business needs. At the same time, when the business load decreases, unnecessary resource blocks can be released to avoid resource waste.

[0184] b) Online monitoring and migration: Through real-time monitoring of resource blocks, if there is a resource block overload, load balancing adjustment is performed. That is, when an overloaded resource block is found, some business requests are migrated to the resource block with lower load to achieve load balancing.

[0185] The entire process of allocating services to resource blocks is formulated as follows:

[0186] ①Maximize matching degree:

[0187]

[0188] where x i,j is a binary variable, indicating whether service i is allocated to resource block j;

[0189] ②Load balancing:

[0190]

[0191] ③ Resource allocation control

[0192]

[0193] Ensure that the usage of computing, storage, and network resources of each resource block does not exceed its total resource capacity.

[0194] Through the above optimization process, the matching degree between business demand and resource block capacity and the load balancing of resource blocks are comprehensively considered in the allocation of business requests. The initial allocation is completed by giving priority to the matching degree, and the subsequent load balancing adjustment ensures the stability and efficiency of the system. Through the dynamic adjustment mechanism, the system can realize adaptive business allocation and scheduling in an environment where business demand and resource blocks change dynamically.

[0195] S4. Management and update of resource blocks. A dynamic change model is constructed to characterize the changes in the resource status of resource blocks when processing business. Based on the dynamic change model, the resource blocks are managed and updated. Because the remaining resources of resource blocks are dynamically adjusted as business processing changes, a mathematical model is needed to describe the resource consumption, release, and management process of resource blocks. This model will help us accurately understand the changes in the resource status of resource blocks when processing business, thereby achieving dynamic update and management.

[0196] Specifically,

[0197] Resource usage dynamic change model:

[0198] Resource Block Resource Model:

[0199] R j =(r j,1 ,r j,2 ,…,r j,n );

[0200] Business resource demand model:

[0201] D i =(d i,1 ,d i,2 ,…,d i,n )

[0202] When service i is allocated to resource block R j When the resource block is used, the resource consumption is expressed as follows:

[0203]

[0204] Among them, r′ j,k Indicates that during the execution of service i, resource block R j The remaining resources on the k-th resource.

[0205] Resource release after business completion

[0206] When service i is completed, resource block R j The resources occupied by the business will be released and restored to the previous state. This process can be expressed as:

[0207]

[0208] Dynamic update formula of resource blocks

[0209] In order to better describe the dynamic update process of resource blocks, time series is used to represent the state changes of resource blocks. Assume that at time t, resource block R j The resource usage is R j (t)=(r j,1 (t),r j,2 (t),…,r j,n (t));

[0210] When a new service request i arrives and is allocated to resource block R j When , the change of resource usage at time t+1 can be expressed as:

[0211]

[0212] When service request i arrives at time t i After completion, its resource requirements are released and the state of the resource block is restored to:

[0213]

[0214] Remaining resource management of resource blocks

[0215] In order to manage resource blocks more efficiently, a remaining resource vector is introduced for each resource block. Represents the remaining available resources of the resource block at time t:

[0216]

[0217] After service request i is allocated, the remaining resources are updated to:

[0218]

[0219] When service request i arrives at time t i After completion, its resource requirements are released and the remaining resources are restored to:

[0220]

[0221] S5, resource block management strategy, adopts management strategy to manage resource blocks, and makes adaptive adjustments according to dynamic changes in load conditions and business needs;

[0222] The resource block management strategy needs to be adaptively adjusted according to the system load and dynamic changes in business needs. The specific management strategies include:

[0223] Resource block expansion and reduction strategy

[0224] When the remaining resources of a resource block are insufficient to meet new business requests, the resource block is automatically expanded to add more physical resources (such as computing, storage or network resources); on the contrary, when the utilization rate of a resource block is low, the resource block is dynamically reduced to release unnecessary resources and reduce resource waste;

[0225] Scaling out and scaling down policies are based on preset thresholds, including:

[0226] If the load of a resource block is L j (t) is greater than or equal to the preset threshold, then resource expansion is triggered:

[0227]

[0228] If the load of a resource block is less than the preset threshold, the resource is reduced:

[0229] L j (t)<θ shrink ;

[0230] Resource block migration and balancing strategy

[0231] When the load of a resource block is too high, some service requests are migrated to other resource blocks to rebalance resources. The migrated service requests give priority to services that have a greater impact on the resource block load, thereby quickly reducing the load pressure.

[0232] Triggered by real-time monitoring of resource utilization, including:

[0233] If the resource block R j Load L j (t) is much higher than other resource blocks, then some services i are migrated to resource blocks R with lower load. k superior:

[0234] And L k (t) <L j (t);

[0235] Which is much higher than the load L j (t) is higher than the preset load difference of other resource blocks. Further, an allowable load difference range and a load threshold are set between resource blocks. When the load rate of a resource block reaches or exceeds the load threshold, regardless of whether the load difference with other resource blocks reaches the preset load difference, the new load will be allocated or transferred to the resource block with lower load on a priority basis. When the load difference between a resource block and other resource blocks is greater than the preset load difference, the new load will be allocated or transferred to the resource block with lower load on a priority basis.

[0236] The specific implementation process is as follows: select candidate resource blocks that meet business requirements from all available resource blocks, that is, the resource margin of the resource block must be greater than the business requirements;

[0237] Candidate resource block sorting: Sort candidate resource blocks according to the current load of the resource blocks and select the resource block with the lowest load; define the load degree L of the resource block j , if the utilization rate of each resource in resource block j is L j,k , the overall load can be expressed as the weighted average of all resource utilizations in the resource block:

[0238]

[0239] in

[0240]

[0241] Allocate resource blocks: select the resource block with the smallest load balancing degree, allocate the service to the resource block, and update the resource status of the resource block accordingly;

[0242] If the resource block is allocated to the service, update its remaining resources:

[0243]

[0244] When the business is completed, the occupied resource block resources are released;

[0245] Dynamic adjustment of resource blocks: Business requirements and physical resources may change dynamically; to adapt to such changes, resource blocks need to be able to expand and contract dynamically:

[0246] Expansion: If a resource block is insufficient, the system can add resources from other physical servers to the resource block through dynamic expansion;

[0247] Contraction: When the load on a resource block is low, the system can dynamically reclaim excess resources and allocate them to other business needs.

[0248] This embodiment also provides a resource block-based multimodal network resource scheduling system, which adopts the above-mentioned resource block-based multimodal network resource scheduling method.

[0249] The above is only an embodiment of the present invention. The common sense such as the known specific structure and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A multimodal network resource scheduling method based on resource blocks, characterized in that: It includes the following: S1. Establish a multi-dimensional business demand model and quantify each resource requirement in the multi-dimensional business demand model to form quantitative indicators; S2, perceive physical resources, abstract the physical resources, and divide the abstracted physical resources into resource blocks; S3, obtain business requests, adopt comprehensive allocation strategies, allocate resource blocks, and make dynamic adjustments during operation; The goal of the comprehensive allocation strategy is to balance the load of resource blocks, and to achieve the best match between the service requirements and the resource block attributes in the service request; S4. Construct a dynamic change model that characterizes the change of resource status of the resource block when processing services, and manage and update the resource block according to the dynamic change model; S5. Use management strategies to manage resource blocks and make adaptive adjustments based on dynamic changes in load conditions and business needs.

2. The multimodal network resource scheduling method based on resource blocks according to claim 1, characterized in that: The multi-dimensional business requirements in the multi-dimensional business requirements model include: computing resource requirements, storage resource requirements, network resource requirements, task execution time and scheduling priority; Computing resource requirements, including the number of CPU cores, number and type of GPUs, and parallelism of computing tasks required by the business; Storage resource requirements, including: storage space size, storage type, and data access speed requirements required by the business; Network resource requirements, including bandwidth, latency, and jitter tolerance required by the business; Task execution time and scheduling priority, including: business execution time requirements, task scheduling urgency and priority; The physical resources include computing resources, storage resources and network resources; Through resource monitoring and management tools, the real-time status of physical resources on each node can be perceived; Abstract physical resources: Through virtualization technology, physical resources are abstracted into logical resource pools; The abstracted physical resources are divided into resource blocks. Based on the perception and abstraction of physical resources, the physical resources are divided into resource blocks. A resource block is a logical unit that aggregates and encapsulates multiple physical resources. Each resource block corresponds to all resources on a physical node, or aggregates resources across multiple physical nodes. Each resource block has a set of attribute parameters to describe its resource characteristics.

3. The multimodal network resource scheduling method based on resource blocks according to claim 2 is characterized in that: The matching degree between the business requirements and the resource block attributes includes: Establish a classification model for business requirements and quantize business attributes: The business requirement attribute vector of business request i is D i =[d i,1 ,d i,2 ,…,d i,n ],in: d i,1 Represents computing resource requirements (such as CPU or GPU); d i,2 Represents storage resource requirements; d i,3 Represents the network bandwidth requirement; … d i,n Represents other resource requirements; The attribute vector of resource block j is R j =[r j,1 ,r j,2 ,…,r j,n ], indicating the supply capacity of various resources in the resource block, where: r j,1 Represents the computing capacity of the resource block; r j,2 Represents storage capacity; r j,3 Represents network bandwidth; … r j,n Represents other resource attributes; Set the matching function to measure the matching degree between the service request and the resource block, and use the vector inner product to measure the matching degree: Among them, w k is the weight coefficient of each resource dimension, and the weights of different resource types are adjusted according to the importance of business requirements; the goal is to allocate business requests to the resource blocks that best match their resource demand attributes, maximizing the matching degree M i,j .

4. The multimodal network resource scheduling method based on resource blocks according to claim 3 is characterized in that: The load balancing of the resource blocks includes: Set the load function to the resource utilization of the resource block. If the resource utilization of each resource block j is L j,k , the overall load can be expressed as the weighted average of all resource utilizations in the resource block: in The goal of resource block load balancing is to balance the load of all resource blocks by allocating service requests and to minimize the maximum load L. j With minimum load L j , or directly minimize the variance of the resource block: in, is the average load of all resource blocks.

5. The multimodal network resource scheduling method based on resource blocks according to claim 4 is characterized in that: The S3 includes: Initial allocation, based on the matching function M i,j , allocate the resource block j with the highest matching degree to service request i, the steps are: For all service requests i and all resource blocks j, calculate the matching degree M i,j . For each service request i, select the resource block j with the greatest matching degree, that is: j* = argmax j M i,j ; After the initial allocation is completed, the remaining resource capacity of the resource block is updated; Load balancing adjustment steps are: Calculate the load L of each resource block j , and determine the resource block j with the highest current load max and the resource block j with the lowest load min ; In j max Select several business requests and migrate them to j min ; The adjusted migration process can be performed iteratively until the loads of all resource blocks are balanced within the preset allowable range; Dynamic adjustment mechanism, the steps are: Dynamic expansion and reduction of resource blocks: When the number of business requests exceeds the processing capacity of existing resource blocks, new resource blocks are dynamically added and the resource block capacity is expanded in real time according to business needs. At the same time, when the business load decreases, the resource blocks are released. Online monitoring and migration: Through real-time monitoring of resource blocks, if there is a resource block overload, load balancing adjustment will be performed.

6. The multimodal network resource scheduling method based on resource blocks according to claim 5, characterized in that: The dynamic change model includes: Resource Block Resource Model: R j ＝(r j,1 ,r j,2 ,…,r j,n )； Business resource demand model: D i =(d i,1 ,d i,2 ,…,d i,n ) When service i is allocated to resource block R j When the resource block is on, the resource consumption is: Among them, r j ' ,k Indicates that during the execution of service i, resource block R j The remaining resources on the k-th resource.

7. The multimodal network resource scheduling method based on resource blocks according to claim 6, characterized in that: The management and update of resource blocks according to the dynamic change model includes: When service i is completed, resource block R j The resources occupied by the business will be released, thus returning to the previous state, as shown: At time t, resource block R j The resource usage is R j (t)=(r j,1 (t),r j,2 (t),…,r j,n (t)); When a new service request i arrives and is allocated to resource block R j When , the resource usage changes at time t+1 as follows: When service request i arrives at time t i After completion, its resource requirements are released and the state of the resource block is restored to: Each resource block introduces a residual resource vector Represents the remaining available resources of the resource block at time t: After service request i is allocated, the remaining resources are updated to: When service request i arrives at time t i After completion, its resource requirements are released and the remaining resources are restored to:

8. The resource block-based multimodal network resource scheduling method according to claim 7, characterized in that: The management strategy includes: expansion and reduction strategies of resource blocks; When the remaining resources of a resource block are insufficient to meet new service requests, the resource block is automatically expanded; conversely, when the utilization rate of a resource block is low, the resource block is dynamically reduced; Scaling out and scaling down policies are based on preset thresholds, including: If the load of the resource block L j (t) is greater than or equal to the preset threshold, then resource expansion is triggered: If the load of a resource block is less than the preset threshold, the resource is reduced: L j (t)<θ shrink 。 9. The multimodal network resource scheduling method based on resource blocks according to claim 7, characterized in that: The management strategy also includes: a migration and balancing strategy for resource blocks; When the load of a resource block is too high, some service requests are migrated to other resource blocks to rebalance resources. Triggered by real-time monitoring of resource utilization, including: If the resource block R j Load L j (t) is higher than the preset load difference of other resource blocks, then some services i are migrated to resource blocks R with lower load. k superior:

10. A multi-modal network resource scheduling system based on resource blocks, characterized in that: A multimodal network resource scheduling method based on resource blocks as described in any one of claims 1 to 9 is adopted.

Images