Method and system for dynamic scheduling of storage and network resources in a cloud computing environment

Abstract

This invention relates to the field of cloud computing technology, and more particularly to a method and system for dynamic scheduling of storage and network resources in a cloud computing environment. The method acquires resource requests, generates a resource topology model based on storage and network requirements and determines resource constraints, maps requests to available resources to generate candidate resource combinations, generates a resource allocation scheme based on the candidate resource combinations and scheduling strategies, executes scheduling, and collects usage status data to dynamically update constraints and scheduling strategies. This achieves collaborative optimization scheduling of storage and network resources, improving resource utilization efficiency and flexibility.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology, and in particular to a method and system for dynamic scheduling of storage and network resources in a cloud computing environment. Background Technology

[0002] In cloud computing environments, the scheduling of storage and network resources typically employs a separate or phased approach. In conventional practices, the resource management system manages the storage cluster and network topology separately. For user resource requests, the storage scheduling module first selects the target storage node based on metrics such as storage capacity and read / write performance. Then, the network scheduling module dynamically allocates network paths based on the location of the selected storage node to meet bandwidth and latency requirements. This sequential scheduling process is structurally clear and facilitates independent optimization of each metric, thus it is adopted by most cloud computing platforms.

[0003] However, separate scheduling overlooks the strong coupling between storage and network resources. The selection of storage nodes directly affects the availability and transmission efficiency of network paths, while changes in network link status can conversely constrain the load distribution of storage nodes. When storage scheduling prioritizes nodes with lower loads, these nodes may be located in network bottleneck areas, making it difficult for subsequent network scheduling to find low-latency paths, ultimately leading to decreased overall resource utilization or compromised service quality. Furthermore, existing scheduling strategies are mostly static or semi-static, lacking the ability to adapt to changes in runtime resource usage. Once the initial allocation is complete, the binding relationships are rarely adjusted based on real-time load fluctuations or network congestion, causing the resource scheduling scheme to quickly deviate from its optimal state after environmental changes, leading to localized overload or resource idleness.

[0004] Another common drawback is the oversimplification of resource constraint modeling. Traditional methods typically use the remaining capacity of a single node or the remaining bandwidth of a single link as constraints, neglecting the impact of inter-node cooperation and inter-link interference on the overall scheduling feasibility. For example, when multiple virtual links simultaneously contend for the same physical port, considering only the bandwidth limit of a single link may underestimate actual transmission conflicts, leading to performance bottlenecks in the implementation of the scheduling scheme. This limitation often results in resource allocation schemes lacking robustness and failing to meet the demands of high-concurrency or dynamically changing cloud computing services. Summary of the Invention

[0005] This invention provides a method and system for dynamic scheduling of storage and network resources in a cloud computing environment, which can solve the problems in the prior art.

[0006] A first aspect of this invention provides a method for dynamic scheduling of storage and network resources in a cloud computing environment, comprising:

[0007] Obtain resource requests in a cloud computing environment;

[0008] Based on the storage and network resource requirements in the resource request, a resource topology model containing storage nodes and network paths is generated. Resource constraints are then determined according to the load status and link transmission capacity of each node in the resource topology model, resulting in a set of resource constraints.

[0009] Based on the set of resource constraints, the resource request is associated with and mapped to available storage nodes and available network paths in the cloud computing environment to generate candidate resource combinations;

[0010] Based on the candidate resource combinations and the preset scheduling strategy, a resource allocation scheme containing the binding relationship between the target storage node and the target network path is generated;

[0011] Based on the resource allocation scheme, a resource scheduling operation is performed on the resource request, and resource usage status data is collected during the scheduling operation. The resource usage status data is then used to dynamically update the resource constraint relationship set and the preset scheduling strategy.

[0012] Based on the storage and network resource requirements in the resource request, a resource topology model containing storage nodes and network paths is generated. Resource constraints are then determined according to the load status and link transmission capacity of each node in the resource topology model, resulting in a set of resource constraints, including:

[0013] The storage resource requirements and network resource requirements in the resource request are parsed to determine the set of storage nodes in the cloud computing environment and the set of network paths connecting the set of storage nodes.

[0014] A resource topology model is generated by associating each storage node in the storage node set with each network path in the network path set.

[0015] Obtain the load status of each storage node and the link transmission capacity of each network path in the resource topology model;

[0016] Based on the topological relationships in the resource topology model, the first impact of changes in the load status of storage nodes on the link transmission capacity of the connected network paths, and the second impact of changes in the link transmission capacity of the network paths on the load status of the connected storage nodes are identified.

[0017] Based on the first influence relationship and the second influence relationship, the associated constraints are determined, and the associated constraints are used as resource constraints to form a set of resource constraints.

[0018] The storage nodes in the storage node set are topologically associated with each network path in the network path set to generate a resource topology model, including:

[0019] Obtain the load characteristics of each storage node in the storage node set and the transmission characteristics of each network path in the network path set;

[0020] Based on the connection information of the network path, determine the storage nodes connected to each network path and establish the connection mapping relationship between the storage nodes and the network path;

[0021] For each storage node in the connection mapping relationship, identify multiple network paths connecting the storage node, and analyze the superposition effect of the transmission characteristics of the multiple network paths on the load characteristics of the storage node;

[0022] For each storage node in the connection mapping relationship, analyze the distribution effect of the load characteristics of the storage node on the transmission characteristics of the multiple network paths connecting the storage node;

[0023] Based on the superposition effect and the allocation effect, the bidirectional coupling relationship between storage nodes and network paths is determined, and a topological association structure is constructed based on the connection mapping relationship and the bidirectional coupling relationship;

[0024] A resource topology model is generated based on the topology association structure, and the storage nodes and network paths in the resource topology model are connected by topology edges carrying bidirectional coupling relationships.

[0025] Based on the set of resource constraints, the resource request is associated and mapped with available storage nodes and available network paths in the cloud computing environment to generate candidate resource combinations, including:

[0026] Extract the constraints between storage nodes and network paths from the set of resource constraints, identify available storage nodes and available network paths in the cloud computing environment that satisfy the constraints based on the constraints, and obtain a set of resource candidates that satisfy the constraints.

[0027] Analyze the storage resource requirements and network resource requirements in the resource request;

[0028] For each available storage node in the resource candidate set, an available network path connected to the available storage node is determined according to the constraints, and an association mapping is performed between the available storage node and the available network path to determine the association mapping relationship.

[0029] Based on the association mapping relationship, obtain the available capacity of the available storage node and the available bandwidth of the available network path, and determine whether the combination of the available storage node and the available network path meets the storage resource requirement and the network resource requirement;

[0030] The combination of available storage nodes and available network paths that meet the storage resource requirements and the network resource requirements is selected as the candidate resource combination.

[0031] For each available storage node in the resource candidate set, an available network path connected to the available storage node is determined according to the constraints. An association mapping is then performed between the available storage node and the available network path to determine the association mapping relationship, including:

[0032] For each available storage node in the resource candidate set, parse the load constraint rules and connection constraint rules related to the available storage node in the constraint conditions;

[0033] Based on the connection constraint rules, select available network paths that are connected to the available storage nodes from the resource candidate set;

[0034] For each available network path selected, verify whether the load status of the available storage node and the transmission status of the available network path satisfy the load constraint rules according to the load constraint rules.

[0035] The available storage nodes that meet the load constraint rules are paired with available network paths to establish an association mapping between the available storage nodes and the available network paths;

[0036] The association mapping is labeled, and the label includes the constraint information of the load constraint rules and the connection constraint rules on the association between the available storage nodes and the available network paths;

[0037] The association mapping relationship is determined based on the annotation, and the association mapping relationship represents the association structure between the available storage node and the available network path under the conditions of satisfying the load constraint rules and the connection constraint rules.

[0038] Based on the candidate resource combinations and the preset scheduling strategy, a resource allocation scheme is generated that includes the binding relationship between target storage nodes and target network paths, including:

[0039] Based on the preset scheduling strategy, the scheduling objective function and scheduling constraints are determined. The scheduling objective function represents the optimization direction of the collaborative scheduling of storage nodes and network paths, and the scheduling constraints represent the restrictive rules that the collaborative scheduling of storage nodes and network paths must satisfy.

[0040] For each of the candidate resource combinations, obtain the load status of available storage nodes and the transmission status of available network paths in the combination;

[0041] Based on the load state and the transmission state, calculate the evaluation value of the combination under the scheduling objective function;

[0042] Filter out the combinations that satisfy the scheduling constraints to obtain a set of constrained combinations, and select the target combination with the largest evaluation value from the set of constrained combinations.

[0043] The available storage nodes in the target combination are determined as target storage nodes, and the available network paths in the target combination are determined as target network paths;

[0044] Establish a binding relationship between the target storage node and the target network path, and generate a resource allocation scheme based on the target storage node, the target network path and the binding relationship.

[0045] Based on the resource allocation scheme, a resource scheduling operation is performed on the resource request, and resource usage status data is collected during the scheduling operation. The resource usage status data is then used to dynamically update the resource constraint relationship set and the preset scheduling strategy, including:

[0046] Based on the target storage node and target network path in the resource allocation scheme, perform resource scheduling operations on the resource request;

[0047] During the execution of the resource scheduling operation, load change data of the target storage node and transmission change data of the target network path are collected, and resource usage status data is generated based on the load change data and the transmission change data.

[0048] Based on the resource usage status data, constraint relationship update information and scheduling strategy update information are determined. The constraint relationship update information is used to characterize the actual constraint relationship deviation between the target storage node and the target network path, and the scheduling strategy update information is used to characterize the effect deviation of the preset scheduling strategy in the actual execution process.

[0049] The resource constraint relationship set is updated according to the constraint relationship update information, and the scheduling objective function and scheduling constraints in the preset scheduling strategy are adjusted according to the scheduling strategy update information.

[0050] A second aspect of this invention provides a dynamic scheduling system for storage and network resources in a cloud computing environment, comprising:

[0051] The request retrieval unit is used to retrieve resource requests in the cloud computing environment;

[0052] The constraint construction unit is used to generate a resource topology model containing storage nodes and network paths based on the storage resource requirements and network resource requirements in the resource request, and to determine resource constraint relationships according to the load status and link transmission capacity of each node in the resource topology model, thereby obtaining a set of resource constraint relationships.

[0053] The combination mapping unit is used to associate and map the resource request with available storage nodes and available network paths in the cloud computing environment according to the set of resource constraint relationships, and generate candidate resource combinations;

[0054] The scheme binding unit is used to generate a resource allocation scheme that includes the binding relationship between the target storage node and the target network path based on the candidate resource combination and the preset scheduling strategy.

[0055] The scheduling update unit is used to perform resource scheduling operations on the resource request based on the resource allocation scheme, collect resource usage status data during the scheduling operation execution process, and dynamically update the resource constraint relationship set and the preset scheduling strategy using the resource usage status data.

[0056] A third aspect of the present invention provides an electronic device, comprising:

[0057] processor;

[0058] Memory used to store processor-executable instructions;

[0059] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0060] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0061] In a cloud computing environment, the construction of a resource topology model and a set of resource constraints enables this invention to accurately reflect the load status of storage nodes and link transmission capabilities. The mapping between resource requests and available resources based on these constraints ensures a high degree of match between candidate resource combinations and the real environment. This precise matching significantly reduces the probability of conflicts during resource allocation, avoids uncoordinated allocation of storage and network resources, and thus greatly improves the overall resource utilization of the cloud computing platform. Simultaneously, real-time consideration of load status and transmission capabilities ensures that the allocation results do not lead to localized overload, improving the stability and reliability of the system.

[0062] This invention combines candidate resource combinations with a preset scheduling strategy to generate a resource allocation scheme that binds target storage nodes to target network paths. This binding ensures end-to-end consistency of data storage and network transmission, eliminating performance bottlenecks that may arise from separate storage and network scheduling. In complex and ever-changing cloud computing environments, this scheme effectively reduces data transmission latency and improves application response speed. Furthermore, the explicit binding relationship makes subsequent resource scheduling operations more predictable, facilitating resource usage planning and capacity management, and providing users with a more stable service experience.

[0063] This invention uses real-time resource usage status data collected during resource scheduling operations to feed back to a set of resource constraints and a preset scheduling strategy. This allows the constraints and scheduling strategy to be dynamically updated based on actual operating conditions, automatically adapting to load fluctuations and node performance changes. Over long-term operation, the system continuously optimizes scheduling decisions, avoiding resource idleness or overload, and enhancing the platform's adaptability and robustness. Simultaneously, dynamic updates reduce the need for manual intervention, lower maintenance costs, and provide strong support for the automated management of large-scale cloud computing environments, enabling the invention to continuously evolve and optimize. Attached Figure Description

[0064] Figure 1 This is a flowchart illustrating the dynamic scheduling method for storage and network resources in a cloud computing environment according to an embodiment of the present invention.

[0065] Figure 2 This is a schematic diagram of the process for determining the set of resource constraint relationships according to an embodiment of the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0068] Figure 1 This is a flowchart illustrating the dynamic scheduling method for storage and network resources in a cloud computing environment, according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0069] Obtain resource requests in a cloud computing environment;

[0070] Based on the storage and network resource requirements in the resource request, a resource topology model containing storage nodes and network paths is generated. Resource constraints are then determined according to the load status and link transmission capacity of each node in the resource topology model, resulting in a set of resource constraints.

[0071] Based on the set of resource constraints, the resource request is associated with and mapped to available storage nodes and available network paths in the cloud computing environment to generate candidate resource combinations;

[0072] Based on the candidate resource combinations and the preset scheduling strategy, a resource allocation scheme containing the binding relationship between the target storage node and the target network path is generated;

[0073] Based on the resource allocation scheme, a resource scheduling operation is performed on the resource request, and resource usage status data is collected during the scheduling operation. The resource usage status data is then used to dynamically update the resource constraint relationship set and the preset scheduling strategy.

[0074] Figure 2 This is a schematic diagram illustrating the process of determining the set of resource constraint relationships according to an embodiment of the present invention. Figure 2 As shown, based on the storage resource requirements and network resource requirements in the resource request, a resource topology model containing storage nodes and network paths is generated. Resource constraints are then determined according to the load status and link transmission capacity of each node in the resource topology model, resulting in a set of resource constraints, including:

[0075] The storage resource requirements and network resource requirements in the resource request are parsed to determine the set of storage nodes in the cloud computing environment and the set of network paths connecting the set of storage nodes.

[0076] A resource topology model is generated by associating each storage node in the storage node set with each network path in the network path set.

[0077] Obtain the load status of each storage node and the link transmission capacity of each network path in the resource topology model;

[0078] Based on the topological relationships in the resource topology model, the first impact of changes in the load status of storage nodes on the link transmission capacity of the connected network paths, and the second impact of changes in the link transmission capacity of the network paths on the load status of the connected storage nodes are identified.

[0079] Based on the first influence relationship and the second influence relationship, the associated constraints are determined, and the associated constraints are used as resource constraints to form a set of resource constraints.

[0080] When parsing storage and network resource requirements in resource requests, two key types of information need to be extracted from the resource request message: First, storage-level requirements, including required storage capacity, read / write throughput thresholds, data persistence level, and access latency limits; second, network-level requirements, including bandwidth requirements, transmission latency requirements, packet loss tolerance range, and path redundancy requirements. By structurally parsing these two types of requirements, the set of storage nodes and network paths that meet the conditions can be located in the global view of the cloud computing environment. The storage node set consists of storage servers distributed across different physical racks or availability zones, each node having a unique identifier, capacity limit, and current used capacity record. The network path set consists of physical links and logical tunnels connecting the storage nodes, with each path recording attributes such as endpoint node identifier, physical hop count, current bandwidth utilization, and historical average latency.

[0081] The core step in generating a resource topology model is to establish topological associations between the storage nodes in the storage node set and the network paths in the network path set. This association is implemented by constructing a weighted directed graph structure as the resource topology model, using storage nodes as vertices and network paths connecting two storage nodes as edges. The weight of each edge is determined by the available bandwidth and transmission latency of the corresponding network path, comprehensively reflecting the actual transmission capacity of the link. When multiple redundant paths exist between the same pair of storage nodes, each path is represented as an independent edge in the topology graph, thus preserving the flexibility of path selection. After the resource topology model is established, all subsequent load status collection and constraint relationship derivation are performed on this graph structure, ensuring that the association between storage and network resources is modeled uniformly.

[0082] Obtaining the load status of each storage node and the link transmission capacity of each network path in the resource topology model relies on the real-time collection of data reported by monitoring probes deployed in the cloud computing environment. For storage nodes, the collected metrics include the current CPU utilization. Memory usage Disk I / O queue depth and remaining storage capacity percentage For network paths, the collected metrics include the current measured bandwidth utilization. Round-trip delay Instantaneous packet loss rate and link jitter value The aforementioned indicators are updated on a rolling basis according to a fixed collection cycle, and the weighted average value within the time window is used as the representative value of the current state, thereby smoothing out the interference of short-term bursts on the state assessment. The quantitative results of load status and link transmission capacity will serve as the input basis for subsequent constraint relationship derivation, ensuring that the constraints can truly reflect the actual availability of current resources.

[0083] Based on the topological relationships in the resource topology model, when identifying the primary impact of changes in storage node load status on the transmission capacity of connected network paths, it is necessary to analyze the mechanism by which storage nodes transmit pressure to the network layer under high load conditions, and when the disk I / O queue depth of a certain storage node... When the data traffic exceeds a preset threshold, the data traffic sent out by the node will suddenly increase, leading to a decrease in bandwidth utilization of the network paths directly connected to the node. This leads to an increase, which in turn causes transmission delay. Increase. The first quantitative description of the impact relationship is: the mapping function relationship between the change in the storage node load index and the change in the transmission capacity index of the network path links it is connected to. Specifically, when the node's overall load score... (Depend on , , , (Weighted synthesis) exceeds the threshold When this happens, a constraint tightening operation is triggered on the transmission capacity of adjacent paths, which proportionally reduces the upper limit of the allocable bandwidth of that path to prevent uncontrolled transmission of storage-side pressure to the network side.

[0084] When identifying the secondary impact of changes in network path link transmission capacity on the load status of connected storage nodes, the analysis direction is opposite to the primary impact. This is when the bandwidth utilization of a certain network path... Sustained high or instantaneous packet loss rate When the data transfer efficiency exceeds the tolerance range, it decreases, causing storage nodes that rely on this path for data read / write to experience prolonged I / O wait times, which in turn increases the disk I / O queue depth of the nodes. The increase is ultimately reflected in the overall load score of storage nodes. The passive increase. The second way to establish the influence relationship is to conduct statistical analysis on the time-series correlation between network path status indicators and storage node load indicators in historical monitoring data, extract indicator pairs with significant correlation, and record their correspondence in the form of conditional constraints. When the degradation of the path transmission capacity indicator exceeds the preset sensitivity parameter. At that time, the acceptable increase in load on the corresponding storage node will be reduced accordingly to maintain the stability of the overall resource status.

[0085] When determining the associated constraints based on the first and second influence relationships, the two types of influence relationships are transformed into operable constraint expressions. For the first influence relationship, the generated associated constraint is expressed as: If the comprehensive load score of the storage node... Exceeding the threshold Then the maximum available bandwidth of all network paths connected to that node needs to be multiplied by a reduction factor. ( The reduction factor is determined based on... Exceeding The magnitude is linearly determined. For the second influence relationship, the generated correlation constraint is expressed as: if the bandwidth utilization of the network path... Exceeding the upper limit threshold or packet loss rate Exceeding the upper limit threshold Then the maximum number of new I / O requests that the storage node connected to this path can handle needs to be reduced proportionally, by a reduction of... and The difference or and The larger value among the differences determines the resource constraint relationship. All the associated constraints derived from the first and second influence relationships, along with the capacity constraints of the storage nodes themselves and the bandwidth constraints of the network paths themselves, are aggregated to form a resource constraint relationship set. This set is stored in a structured key-value pair format, where the key is the combination of resource node or path identifiers involved, and the value is the corresponding constraint expression and the current constraint parameter value. This facilitates rapid retrieval and condition filtering during the candidate resource combination generation stage, thereby ensuring that the resource allocation scheme meets the actual operational constraints in both storage and network dimensions.

[0086] The storage nodes in the storage node set are topologically associated with each network path in the network path set to generate a resource topology model, including:

[0087] Obtain the load characteristics of each storage node in the storage node set and the transmission characteristics of each network path in the network path set;

[0088] Based on the connection information of the network path, determine the storage nodes connected to each network path and establish the connection mapping relationship between the storage nodes and the network path;

[0089] For each storage node in the connection mapping relationship, identify multiple network paths connecting the storage node, and analyze the superposition effect of the transmission characteristics of the multiple network paths on the load characteristics of the storage node;

[0090] For each storage node in the connection mapping relationship, analyze the distribution effect of the load characteristics of the storage node on the transmission characteristics of the multiple network paths connecting the storage node;

[0091] Based on the superposition effect and the allocation effect, the bidirectional coupling relationship between storage nodes and network paths is determined, and a topological association structure is constructed based on the connection mapping relationship and the bidirectional coupling relationship;

[0092] A resource topology model is generated based on the topology association structure, and the storage nodes and network paths in the resource topology model are connected by topology edges carrying bidirectional coupling relationships.

[0093] In constructing the resource topology model, it is necessary to first collect basic characteristic information of each element in the storage node set and network path set. For storage nodes, the collected load characteristics cover dimensions such as the node's current computing resource occupancy, memory usage, disk read / write throughput, and available storage capacity. These characteristics collectively characterize the overall operating status of the storage node at a given moment. For network paths, the collected transmission characteristics include metrics such as the path's measured available bandwidth, end-to-end transmission latency, packet loss rate, and link jitter. These metrics reflect the actual capability boundaries of the network path when carrying data transmission tasks. The above characteristic data is obtained through periodic probing and monitoring of each resource node in the cloud computing environment. The collection frequency is adaptively adjusted according to the dynamic changes in business load to ensure the timeliness and accuracy of the characteristic data.

[0094] After acquiring the aforementioned feature data, the endpoints of the storage nodes connected to each network path are determined based on the physical connection information or logical configuration information of the network paths. This establishes a connection mapping relationship between storage nodes and network paths, expressed in the form of a directed or undirected adjacency structure. Each storage node is represented as a vertex in the graph, and each network path is represented as an edge connecting two vertices, with path identification information attached to the edges. In cases where multiple parallel paths connect the same pair of storage nodes, each path is retained as an independent edge element in the connection mapping relationship to support subsequent multi-path transmission feature analysis. During the establishment of the connection mapping relationship, it is also necessary to handle cases where storage nodes are indirectly connected through network switching devices. In this case, the path needs to be expanded hop-by-hop to include the transmission characteristics of intermediate forwarding nodes within the calculation scope of the overall path transmission characteristics.

[0095] For each storage node in the connection mapping relationship, all network paths directly connected to it are identified, and the superposition effect of the transmission characteristics of these network paths on the load characteristics of the storage node is analyzed. The core meaning of the superposition effect is that when multiple network paths simultaneously transmit data to a storage node, the bandwidth usage, latency jitter, and other transmission characteristics of each path are superimposed at the storage node's entry point, leading to a deeper disk write queue and increased memory buffer usage, thereby causing changes in the overall load of the storage node. Specifically, suppose there are a total of [number missing] network paths connected to a storage node. The path, the first The effective data traffic transmitted to this storage node along this path at the current moment is The total inbound traffic that the storage node experiences on the network side is then... It can be represented as .when Exceeding the disk write throughput limit of the storage node In such cases, the superposition effect can cause storage nodes to enter a high-load state, requiring restrictions on the acceptable inflow to that storage node within resource constraints. The analysis results of the superposition effect are appended to the corresponding edges of the connection mapping relationship in the form of quantified coefficients, providing input for determining subsequent bidirectional coupling relationships.

[0096] For each storage node in the connection mapping relationship, it is also necessary to analyze the distribution effect of the storage node's load characteristics on the transmission characteristics of the multiple network paths connected to it. The distribution effect describes the impact of how the storage node distributes its output traffic among multiple available network paths when sending data externally on the transmission characteristics of each path. When the total output traffic of the storage node... exist The outgoing paths are arranged in a certain proportion (satisfy ,and When allocating, the first The actual traffic carried by this path is The ratio of this traffic to the available bandwidth of the path determines the magnitude of the bandwidth utilization change. If a storage node experiences a sudden surge in traffic due to excessive load, it can cause a sharp increase in the bandwidth utilization of certain network paths, leading to transmission quality degradation such as increased packet loss rate and latency. The analysis results of the allocation effect are also recorded in the corresponding edge structure of the connection mapping relationship in a quantitative description form, together with the superposition effect data, constituting the attribute information of the edge.

[0097] After analyzing the superposition and allocation effects, the results are combined to determine the bidirectional coupling relationship between storage nodes and network paths. This bidirectional coupling relationship expresses the mutual influence mechanism between changes in the state of storage nodes and changes in the state of network paths: on the one hand, changes in the transmission characteristics of network paths (such as bandwidth compression and increased latency) will affect the load characteristics of storage nodes through a superposition effect; on the other hand, changes in the load characteristics of storage nodes (such as a sudden increase in output traffic) will affect the transmission characteristics of network paths through an allocation effect. This bidirectional influence relationship is expressed in the topology structure through topological edges carrying coupling weights. The coupling weights are jointly determined by the superposition effect coefficient and the allocation effect coefficient, reflecting the strength and direction of the mutual influence between the two. For storage node and network path pairs with strong coupling relationships, the state changes of both need to be evaluated in a coordinated manner during resource scheduling to avoid performance degradation of the other due to adjusting one side alone.

[0098] Based on the aforementioned connection mapping and bidirectional coupling relationships, a topological association structure is constructed. This structure is expressed using a graph data model, where storage nodes and network paths are respectively represented as vertex types in the graph. Topological edges carry quantitative descriptions of the bidirectional coupling relationships, including superposition effect coefficients, allocation effect coefficients, and coupling direction identifiers. The topological association structure supports fast querying of the set of adjacent paths for any storage node, and also supports fast access to the endpoint storage nodes and their load status for any network path, thus providing an efficient graph traversal foundation for the subsequent derivation of resource constraints.

[0099] Based on the topological association structure, a resource topology model is ultimately generated. This model further integrates load characteristic snapshots of each storage node and transmission characteristic snapshots of each network path, forming a version-tagged topology representation over time. Each topological edge in the resource topology model carries not only connection information but also a complete description of the bidirectional coupling relationship between the storage node and the network path under that connection, including the current estimated coupling strength and the moving average of historical coupling strengths. This design enables the resource topology model to simultaneously support static topology queries and dynamic coupling state analysis, providing accurate topological basis for subsequent generation of candidate resource combinations based on resource constraint sets. It also supports local incremental updates to the topology model when resource usage status data is fed back, without requiring a full reconstruction of the entire topology.

[0100] Based on the set of resource constraints, the resource request is associated and mapped with available storage nodes and available network paths in the cloud computing environment to generate candidate resource combinations, including:

[0101] Extract the constraints between storage nodes and network paths from the set of resource constraints, identify available storage nodes and available network paths in the cloud computing environment that satisfy the constraints based on the constraints, and obtain a set of resource candidates that satisfy the constraints.

[0102] Analyze the storage resource requirements and network resource requirements in the resource request;

[0103] For each available storage node in the resource candidate set, an available network path connected to the available storage node is determined according to the constraints, and an association mapping is performed between the available storage node and the available network path to determine the association mapping relationship.

[0104] Based on the association mapping relationship, obtain the available capacity of the available storage node and the available bandwidth of the available network path, and determine whether the combination of the available storage node and the available network path meets the storage resource requirement and the network resource requirement;

[0105] The combination of available storage nodes and available network paths that meet the storage resource requirements and the network resource requirements is selected as the candidate resource combination.

[0106] When extracting constraints between storage nodes and network paths from the resource constraint set, each constraint record in the set needs to be parsed to identify its associated resource type, constraint direction, and constraint threshold. Constraints typically fall into two categories: one is capacity and load constraints for the storage node itself, describing the maximum storage service capacity the node can provide in its current state; the other is transmission constraints for the network path, describing the maximum data transfer scale the path can support under the current link quality. After extraction, all registered storage nodes and network paths in the cloud computing environment are traversed, and each is compared with the constraints. Nodes and paths that satisfy all constraints are included in the resource candidate set. Storage nodes or network paths that do not satisfy any constraint are directly excluded from the candidate range and do not proceed to the subsequent mapping stage. This filtering process effectively reduces the search space for subsequent processing, avoids introducing unavailable resources into candidate combinations, and thus reduces the probability of generating invalid combinations.

[0107] When parsing storage and network resource requirements in resource requests, the resource description fields in the request message are structured. Storage resource requirements are typically given in the form of data block size, persistent storage capacity, or read / write throughput, while network resource requirements are described in the form of required transmission bandwidth, maximum acceptable latency, or minimum reliability level. The parsed requirements are stored in standardized numerical form for direct use in subsequent availability assessment steps. If there are multiple concurrent subtasks in the resource request, the requirements for each subtask are parsed separately, and candidate resource combinations are generated independently for each subtask in the subsequent mapping stage. This on-demand parsing method can accurately correspond to the differentiated requirements of different tasks for resource types and quantities, avoiding resource allocation deviations caused by ambiguous requirement descriptions.

[0108] For each available storage node in the resource candidate set, when determining the available network paths connected to that storage node based on constraints, it is necessary to combine the adjacency relationships between nodes and paths recorded in the resource topology model. For the currently traversed storage node, firstly, a list of all network paths directly connected to it is obtained from the topology model. Then, network paths that have passed the constraint test and been included in the resource candidate set are selected from this list, resulting in a subset of associative paths for that storage node under the current constraints. If the subset of associative paths for a storage node is empty, that storage node will not participate in subsequent combination generation in the current round of mapping, but will be retained in the candidate set for re-evaluation after subsequent dynamic updates of constraints. For storage nodes with a non-empty subset of associative paths, a one-to-one association mapping relationship is established between that node and each path in its subset of associative paths, forming a mapping structure with the storage node as the anchor and available network paths as branches, recorded as an association mapping relationship table.

[0109] When obtaining the available capacity of available storage nodes and the available bandwidth of available network paths based on the association mapping relationship, real-time resource indicators for each node and path are read from the resource status database. The available capacity of an available storage node is defined as the difference between the node's currently allocated capacity and its total capacity, denoted as... The unit should be consistent with the storage resource requirement in the resource request for direct numerical comparison. The available bandwidth of an available network path is defined as the remaining transmission capacity of that path after deducting the occupied bandwidth in the current link state, denoted as... Also expressed in standardized units. After obtaining the above metrics, for each combination of storage nodes and network paths in the association mapping table, an availability judgment is performed: if Not less than the storage resource requirement obtained from the analysis, and If the network resource demand is not less than the amount obtained from the analysis, then the combination is considered to meet the current resource request requirements in terms of resource quantity.

[0110] In the availability assessment process, when multiple available network paths are associated with the same storage node, the available bandwidth of each path needs to be assessed independently, rather than simply summing the bandwidths of multiple paths and comparing them with the demand. This is because in actual scheduling, a single resource request is usually bound to a single network path. Assembling the bandwidth of multiple paths can lead to incorrect inclusion in candidate combinations even when the bandwidth of a single path is insufficient, causing resource allocation failures in subsequent scheduling phases. Therefore, the available bandwidth of each network path must be compared separately with the network resource demand; only paths that meet the conditions can form a valid candidate combination with their corresponding storage nodes.

[0111] After selecting combinations of available storage nodes and available network paths that meet both storage and network resource requirements as candidate resource combinations, all candidate resource combinations are deduplicated to avoid duplicate recording of the same combination due to path reuse or multiple node affiliations in the topology model. The deduplicated set of candidate resource combinations is output in the form of a structured list, with each record containing a storage node identifier, an associated network path identifier, and the available capacity of that node. and the available bandwidth of the path. This information is directly read and used in subsequent scheduling strategy stages. The number of candidate resource combinations directly affects the computational complexity of subsequent scheduling strategies. If the number of candidate combinations is too large, a pre-screening mechanism can be introduced at this stage to initially sort the candidate combinations based on their resource margins, prioritizing the retention of combinations with larger resource margins to improve the processing efficiency of subsequent scheduling stages.

[0112] Throughout the association mapping process, the extraction of constraints and the screening of resource candidate sets constitute the entry barrier of the mapping process, ensuring that all storage nodes and network paths entering the mapping stage have passed the basic constraint verification. The establishment of association mapping relationships is based on the topology model, ensuring the physical connectivity between nodes and paths. Availability judgment is based on real-time resource indicators, ensuring that the candidate resource combination can meet the actual needs of the current request in terms of resource quantity. These three stages proceed sequentially, jointly ensuring the effectiveness and availability of candidate resource combinations, providing a reliable input basis for the generation of subsequent resource allocation schemes.

[0113] For each available storage node in the resource candidate set, an available network path connected to the available storage node is determined according to the constraints. An association mapping is then performed between the available storage node and the available network path to determine the association mapping relationship, including:

[0114] For each available storage node in the resource candidate set, parse the load constraint rules and connection constraint rules related to the available storage node in the constraint conditions;

[0115] Based on the connection constraint rules, select available network paths that are connected to the available storage nodes from the resource candidate set;

[0116] For each available network path selected, verify whether the load status of the available storage node and the transmission status of the available network path satisfy the load constraint rules according to the load constraint rules.

[0117] The available storage nodes that meet the load constraint rules are paired with available network paths to establish an association mapping between the available storage nodes and the available network paths;

[0118] The association mapping is labeled, and the label includes the constraint information of the load constraint rules and the connection constraint rules on the association between the available storage nodes and the available network paths;

[0119] The association mapping relationship is determined based on the annotation, and the association mapping relationship represents the association structure between the available storage node and the available network path under the conditions of satisfying the load constraint rules and the connection constraint rules.

[0120] When performing association mapping on the resource candidate set, for each available storage node in the set, it is necessary to extract the constraint rules directly related to that node from the resource constraint relationship set and categorize and parse them according to rule type. Constraint rules are generally divided into two categories: load constraint rules and connectivity constraint rules. Load constraint rules describe the operational boundary conditions that a storage node must meet under the current load level and network path under the current transmission state, such as the node's overall load score not exceeding a specified threshold or the network path's bandwidth utilization not exceeding an upper limit. Connectivity constraint rules describe the topology-level connection legitimacy, i.e., whether a network path has a valid physical or logical connection to the storage node. Extracting and structuring these two types of rules from the constraint relationship set is a prerequisite for subsequent screening and verification steps.

[0121] For each available storage node, all available network paths in the resource candidate set are traversed according to connection constraint rules. A subset of paths connected to that storage node is identified. The determination of connection relationships relies on the pre-established node-path adjacency structure in the resource topology model. The adjacency table or adjacency matrix is ​​queried to determine whether the path endpoints match the access point corresponding to the target storage node. If one endpoint of a network path matches the network interface identifier of the storage node, the path is considered connected to that node and included in the candidate path subset. For multi-homed storage nodes (i.e., nodes connected to multiple network interfaces simultaneously), a connection relationship query is performed for each interface, and the query results are combined to ensure that no legitimate connection paths are missed.

[0122] After connection constraint filtering is completed, load constraint verification is performed on each selected available network path. The verification process consists of two dimensions: verification of the storage node's own load status and verification of the network path's transmission status. The storage node's load status verification checks whether its current comprehensive load score is within the range allowed for accepting new task assignments, based on the boundary conditions set in the load constraint rules. The network path's transmission status verification checks whether its current bandwidth utilization, transmission latency, packet loss rate, and other indicators all meet the corresponding upper limits in the load constraint rules. Only when both the storage node and the network path pass the verification in both dimensions is the combination of the node and the path considered to satisfy the load constraint rules and can proceed to the pairing stage. If a path fails to meet the constraints in any verification dimension, it is removed from the candidate path subset of the current node and does not participate in subsequent pairing.

[0123] Storage nodes and network paths that pass load constraint verification are paired one by one to establish an association mapping between nodes and paths. The basic form of pairing is an ordered pair. ,in Indicates the first Available storage nodes that meet the constraints. Indicates and The first pairing There are several available network paths that meet the constraints. When a storage node corresponds to multiple network paths that meet the constraints, an independent ordered pair of association mappings is established for each path, so that the same storage node can correspond to multiple network path entries in the association mapping structure, thereby supporting the generation of candidate resource combinations in subsequent multi-path load balancing scenarios.

[0124] After establishing the association mapping, annotation operations are performed on each mapping pair in an ordered manner. The annotation content includes two parts: load constraint annotation and connection constraint annotation. The load constraint annotation records the specific rule parameters used in the verification phase and their corresponding measured values, such as the current value and allowable upper limit of node load score, the current value and upper limit threshold of path bandwidth utilization, etc., making the constraint satisfaction of each association mapping pair traceable and auditable. The connection constraint annotation records the topological connection basis on which the mapping pair depends, including connection interface identifier, path endpoint information, and connection type (physical direct connection or logical tunnel). By embedding constraint information into the annotation fields, the association mapping structure not only describes the pairing relationship between nodes and paths, but also carries the constraint semantics on which the pairing relationship depends, providing sufficient contextual information for scheme selection and priority ranking in subsequent scheduling decisions.

[0125] After annotation, all annotated ordered pairs of association mappings are summarized to form an association mapping relationship set. Each record in the association mapping relationship set represents a valid storage node and network path association structure that satisfies load constraint rules and connection constraint rules. Logically, this set can be viewed as a bipartite graph, with vertices on one side representing available storage nodes that satisfy the constraints, and vertices on the other side representing available network paths that satisfy the constraints. Edges correspond to ordered pairs of association mappings, and the annotation information attached to the edges describes the constraints on which the edge is valid. When generating candidate resource combinations, subgraph structures that satisfy the combination constraints are selected from this bipartite graph according to a preset scheduling strategy, thus obtaining a resource allocation scheme that includes the binding relationship between the target storage node and the target network path.

[0126] In practical engineering deployments, the size of the resource candidate set can be very large, leading to a significant increase in the computational workload of association mapping. To address this, an indexing acceleration mechanism can be introduced during the connection constraint screening stage. An inverted index can be pre-built for the node-path adjacency structure in the resource topology model, reducing the time complexity of querying connection paths for any storage node to near constant level. During the load constraint verification stage, the real-time status data of each node and path can be cached in memory, avoiding repeated access to the underlying monitoring interface during batch verification. This improves the overall execution efficiency of association mapping and meets the real-time requirements of dynamic scheduling in a cloud computing environment.

[0127] Based on the candidate resource combinations and the preset scheduling strategy, a resource allocation scheme is generated that includes the binding relationship between target storage nodes and target network paths, including:

[0128] Based on the preset scheduling strategy, the scheduling objective function and scheduling constraints are determined. The scheduling objective function represents the optimization direction of the collaborative scheduling of storage nodes and network paths, and the scheduling constraints represent the restrictive rules that the collaborative scheduling of storage nodes and network paths must satisfy.

[0129] For each of the candidate resource combinations, obtain the load status of available storage nodes and the transmission status of available network paths in the combination;

[0130] Based on the load state and the transmission state, calculate the evaluation value of the combination under the scheduling objective function;

[0131] Filter out the combinations that satisfy the scheduling constraints to obtain a set of constrained combinations, and select the target combination with the largest evaluation value from the set of constrained combinations.

[0132] The available storage nodes in the target combination are determined as target storage nodes, and the available network paths in the target combination are determined as target network paths;

[0133] Establish a binding relationship between the target storage node and the target network path, and generate a resource allocation scheme based on the target storage node, the target network path and the binding relationship.

[0134] In generating resource allocation schemes, candidate resource combinations need to be combined with preset scheduling strategies. Through quantitative evaluation and constraint screening, the binding relationship between target storage nodes and target network paths is ultimately determined. The preset scheduling strategy contains two core elements: a scheduling objective function and scheduling constraints. The scheduling objective function describes the optimization direction of collaborative scheduling between storage nodes and network paths, that is, to achieve balanced overall resource utilization and optimal transmission quality while meeting resource request requirements. The scheduling constraints characterize the restrictive rules that must be met during collaborative scheduling, including hard constraints such as node capacity limits, path bandwidth limits, and latency thresholds. Only candidate combinations that simultaneously meet all constraints are eligible to enter the final selection stage.

[0135] The scheduling objective function is constructed by comprehensively considering performance metrics from both the storage and network sides. For the storage side, the current remaining capacity percentage of nodes, disk I / O response capability, and the combined idle capacity of CPU and memory are used as evaluation factors. For the network side, path bandwidth margin, end-to-end transmission latency, and packet loss stability are used as evaluation factors. These factors are then weighted and aggregated according to preset weights to form a comprehensive evaluation value for a single candidate combination. The calculation method is as follows:

[0136] ;

[0137] in, The aggregate weights of the storage-side evaluation factors. The aggregate weights of the network-side evaluation factors, and satisfying the following conditions: ; The overall performance score of the storage node. This is a comprehensive score for network path transmission quality. It is obtained by linearly superimposing the current remaining capacity percentage of the node, the disk I / O idle level, and the combined CPU and memory idle level according to their respective sub-weights; The result is obtained by linearly superimposing the normalized values ​​of available bandwidth margin, reciprocal normalized latency, and reciprocal normalized packet loss rate according to their respective sub-weights. The normalization process uses a max-min scaling method to map each indicator to... The interval ensures that indicators with different dimensions are comparable when aggregated.

[0138] The scheduling constraints consist of multiple sets of hard rules. Storage-side constraints require that the current available capacity of candidate storage nodes is not less than the storage demand declared in the resource request, and that the overall load score of the nodes not exceed a preset load limit threshold to prevent storage service quality degradation due to node overload. Network-side constraints require that the available bandwidth of candidate network paths is not less than the bandwidth demand declared in the resource request, the end-to-end transmission latency of the path does not exceed a preset latency limit, and the packet loss rate does not exceed a preset packet loss rate limit. These constraints are combined using a logical AND relationship; that is, a candidate combination must simultaneously satisfy all constraints to be included in the constrained combination set.

[0139] For each candidate resource combination, the load status of available storage nodes and the transmission status of available network paths within that combination are obtained sequentially. The load status information of storage nodes comes from real-time monitoring data of each node in the resource topology model, covering current CPU utilization, memory usage, disk I / O queue depth, and remaining storage capacity. The transmission status information of network paths comes from link probing and traffic acquisition results, covering current measured bandwidth utilization, round-trip latency, instantaneous packet loss rate, and link jitter. After obtaining the above status data, the scheduling objective function is used to calculate the load status of each combination according to its calculation rules. and This leads to the overall evaluation value of the combination. After all candidate combinations have completed their evaluation value calculations, they proceed to the constraint screening stage.

[0140] During the constraint screening phase, each candidate combination is checked to ensure it meets all scheduling constraints. For storage-side constraints, the current available storage capacity of the candidate storage nodes is compared with the declared resource request demand, and the node's overall load score is compared with the load upper limit threshold. For network-side constraints, the available bandwidth of the candidate network path is compared with the requested bandwidth demand, and the path latency and packet loss rate are compared with the corresponding upper limit threshold. Candidate combinations that do not meet any constraint are directly excluded. The remaining combinations that pass all constraint checks constitute the set of constrained combinations. If this set is empty, a constraint relaxation mechanism is triggered, gradually relaxing non-core constraints according to a preset relaxation priority, and the screening is re-executed until a non-empty set of constrained combinations is obtained or the maximum relaxation level is reached, at which point a resource shortage anomaly is reported.

[0141] Select the comprehensive evaluation value from the set of combinations that meet the constraints. The largest candidate combination is taken as the target combination. If multiple combinations exist... If the values ​​are the same, a secondary sorting rule is introduced, prioritizing the combination of storage nodes whose geographical locations are closest to the computing node that initiated the request, in order to reduce data access latency. If the secondary rule still cannot uniquely determine the combination, the first combination in ascending order of its number is selected to ensure the determinism of the optimal result. After the target combination is determined, the available storage nodes and available network paths within it are designated as target storage nodes and target network paths, respectively.

[0142] When establishing a binding relationship between a target storage node and a target network path, the following associated information must be recorded: the unique identifier of the target storage node, the routing identifier of the target network path, the binding activation timestamp, the binding validity period, and the resource request identifier that triggered the binding. The binding relationship is written to the resource scheduling database in the form of a structured record and simultaneously pushed to the resource topology model, so that subsequent sets of resource constraints can be aware of the existence of this binding and avoid duplicate allocation of already bound resources.

[0143] The resource allocation scheme consists of three parts: target storage node information, target network path information, and binding relationships. The target storage node information includes node identifier, node address, allocated capacity, and reserved I / O bandwidth; the target network path information includes path identifier, routing hop count, allocated bandwidth, and latency guarantee level; and the binding relationship information includes the complete content of the above binding records. After the resource allocation scheme is generated, it is output in a standardized scheme description format for subsequent resource scheduling operation modules to execute the actual resource reservation and configuration distribution actions. The scheme also includes a rollback contingency plan, recording the operation sequence required to restore the resource constraint relationship set to its pre-scheduling state in the event of scheduling failure, ensuring the atomicity and consistency of the scheduling process.

[0144] Based on the resource allocation scheme, a resource scheduling operation is performed on the resource request, and resource usage status data is collected during the scheduling operation. The resource usage status data is then used to dynamically update the resource constraint relationship set and the preset scheduling strategy, including:

[0145] Based on the target storage node and target network path in the resource allocation scheme, perform resource scheduling operations on the resource request;

[0146] During the execution of the resource scheduling operation, load change data of the target storage node and transmission change data of the target network path are collected, and resource usage status data is generated based on the load change data and the transmission change data.

[0147] Based on the resource usage status data, constraint relationship update information and scheduling strategy update information are determined. The constraint relationship update information is used to characterize the actual constraint relationship deviation between the target storage node and the target network path, and the scheduling strategy update information is used to characterize the effect deviation of the preset scheduling strategy in the actual execution process.

[0148] The resource constraint relationship set is updated according to the constraint relationship update information, and the scheduling objective function and scheduling constraints in the preset scheduling strategy are adjusted according to the scheduling strategy update information.

[0149] After the resource allocation plan is determined, mount commands are issued to the corresponding storage nodes according to the target storage nodes and target network paths specified in the plan, and a data transmission channel is established on the target network path to complete the actual scheduling and binding of resource requests. The execution of the scheduling operation is not a one-time action, but a continuous process. During this process, real-time monitoring of the target storage nodes and target network paths needs to be started simultaneously to ensure that the subsequent dynamic feedback mechanism can continue to function.

[0150] During resource scheduling operations, load change data for the target storage node is collected at fixed collection periods. This includes: time-series fluctuations in CPU utilization, trends in memory occupancy, dynamic changes in disk I / O queue depth, and real-time storage capacity consumption rates. Simultaneously, transmission change data for the target network path is collected, including: measured bandwidth utilization curves, round-trip latency jitter, fluctuations in instantaneous packet loss rate, and the statistical distribution of link jitter values. These two types of collected data are aligned and merged according to timestamps to form resource usage status data. This data is stored in time-series format, with one status record corresponding to each collection period. Each record contains a snapshot of the full operational metrics of the target storage node and the target network path at that specific moment.

[0151] Based on resource usage status data, a retrospective analysis is performed on the constraint relationship between the target storage node and the target network path during actual operation. When generating the resource allocation scheme, the resource constraint relationship set records the expected constraint boundaries between the target storage node and the target network path, such as the upper limit of available bandwidth corresponding to the storage node within a specific load range, and the maximum write rate of the storage node corresponding to the network path at a specific latency level. By comparing the measured indicators in the resource usage status data with the above expected constraint boundaries item by item, the actual deviation is calculated. If the measured indicators consistently exceed the expected constraint boundaries, the constraint relationship is determined to have an underestimated deviation; if the measured indicators consistently fall below the expected constraint boundaries, the constraint relationship is determined to have an overestimated deviation. The above deviation determination results are encapsulated into constraint relationship update information, which includes descriptive fields of three dimensions: deviation direction, deviation magnitude, and deviation duration, used to accurately characterize the actual constraint relationship deviation between the target storage node and the target network path.

[0152] The generation of scheduling policy update information and constraint relationship update information are executed in parallel. The preset scheduling policy comprises two core components: a scheduling objective function and scheduling constraints. The objective function describes the optimization goal of resource allocation, while the constraints describe the boundary limitations of resource allocation. During actual execution, statistical analysis of comprehensive evaluation indicators from resource usage status data is performed to calculate the deviation between the actual scheduling effect and the preset scheduling objective. Let the measured comprehensive load score of the target storage node after actual scheduling execution be... The target load score for this node in the preset scheduling strategy is [value]. Then the effect deviation amount satisfy Similarly, the deviation in network path transmission quality is calculated synchronously, assuming the actual measured comprehensive transmission quality score is... The expected transmission quality target value in the preset scheduling strategy is The deviation in transmission quality satisfy .Will and They are collectively encapsulated as scheduling policy update information, used to characterize the deviation of the preset scheduling policy in the actual execution process.

[0153] When updating the resource constraint set based on constraint update information, for constraints with underestimated deviations, the corresponding constraint boundaries are tightened, i.e., the allowed resource usage limit within that constraint is reduced. For constraints with overestimated deviations, the corresponding constraint boundaries are appropriately relaxed, i.e., the allowed resource usage limit within that constraint is increased. The adjustment magnitude of the constraint boundaries is proportional to the magnitude and duration of the deviation, avoiding unnecessary large-scale modifications to constraints triggered by short-term instantaneous fluctuations. The adjusted constraints are updated to the corresponding entries in the resource constraint set through an overwrite method, ensuring that subsequent resource requests can be associated and mapped based on the latest constraint relationships when generating candidate resource combinations.

[0154] When adjusting the scheduling objective function and scheduling constraints in the preset scheduling strategy based on the scheduling strategy update information, the process is divided into two levels. At the scheduling objective function level, if... and All values ​​are positive, indicating that both the actual load and actual transmission quality are higher than expected, and the aggregate weights of the storage-side evaluation factors in the scheduling objective function are positive. Aggregated weights with network-side evaluation factors The values ​​need to be adjusted upwards to allow for more rigorous screening of comprehensive evaluation values ​​in subsequent scheduling. A higher pool of candidate resources; if For negative values A positive value indicates that the actual load on the storage side is lower than expected, while the transmission quality on the network side is higher than expected, so only the value is increased. ,Keep The process remains unchanged, with a focus on optimizing the storage node selection logic in subsequent scheduling. At the scheduling constraint level, if the performance deviation continues to exceed the preset deviation tolerance threshold... This triggers a hard revision of the scheduling constraints, bringing the constraint thresholds of the corresponding indicators in the scheduling constraints closer to the measured average, making the scheduling constraints more closely reflect actual operating patterns. Deviation tolerance threshold. The settings are adaptively configured based on the statistical distribution of performance deviations in historical scheduling cycles. The initial value is given by the system configuration parameters, and it gradually converges to a stable value as state data accumulates.

[0155] The aforementioned dynamic update process is triggered once after each collection cycle, forming a closed-loop feedback mechanism of "collection, analysis, and update". As the scheduling operation continues to be executed, the parameters in the resource constraint relationship set and the preset scheduling strategy will continuously approach the real resource operation rules with the accumulation of actual operating data. This makes the scheduling decisions for subsequent resource requests more accurate, reduces the problem of unreasonable resource allocation caused by the deviation of the initial constraint setting, and improves the overall efficiency and stability of dynamic scheduling of storage and network resources in the cloud computing environment.

[0156] A second aspect of this invention provides a dynamic scheduling system for storage and network resources in a cloud computing environment, comprising:

[0157] The request retrieval unit is used to retrieve resource requests in the cloud computing environment;

[0158] The constraint construction unit is used to generate a resource topology model containing storage nodes and network paths based on the storage resource requirements and network resource requirements in the resource request, and to determine resource constraint relationships according to the load status and link transmission capacity of each node in the resource topology model, thereby obtaining a set of resource constraint relationships.

[0159] The combination mapping unit is used to associate and map the resource request with available storage nodes and available network paths in the cloud computing environment according to the set of resource constraint relationships, and generate candidate resource combinations;

[0160] The scheme binding unit is used to generate a resource allocation scheme that includes the binding relationship between the target storage node and the target network path based on the candidate resource combination and the preset scheduling strategy.

[0161] The scheduling update unit is used to perform resource scheduling operations on the resource request based on the resource allocation scheme, collect resource usage status data during the scheduling operation execution process, and dynamically update the resource constraint relationship set and the preset scheduling strategy using the resource usage status data.

[0162] A third aspect of the present invention provides an electronic device, comprising:

[0163] processor;

[0164] Memory used to store processor-executable instructions;

[0165] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0166] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0167] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for dynamic scheduling of storage and network resources in a cloud computing environment, characterized in that, include: Obtain resource requests in a cloud computing environment; Based on the storage and network resource requirements in the resource request, a resource topology model containing storage nodes and network paths is generated. Resource constraints are then determined according to the load status and link transmission capacity of each node in the resource topology model, resulting in a set of resource constraints. Based on the set of resource constraints, the resource request is associated with and mapped to available storage nodes and available network paths in the cloud computing environment to generate candidate resource combinations; Based on the candidate resource combinations and the preset scheduling strategy, a resource allocation scheme containing the binding relationship between the target storage node and the target network path is generated; Based on the resource allocation scheme, a resource scheduling operation is performed on the resource request, and resource usage status data is collected during the scheduling operation. The resource usage status data is then used to dynamically update the resource constraint relationship set and the preset scheduling strategy.

2. The method according to claim 1, characterized in that, Based on the storage and network resource requirements in the resource request, a resource topology model containing storage nodes and network paths is generated. Resource constraints are then determined according to the load status and link transmission capacity of each node in the resource topology model, resulting in a set of resource constraints, including: The storage resource requirements and network resource requirements in the resource request are parsed to determine the set of storage nodes in the cloud computing environment and the set of network paths connecting the set of storage nodes. A resource topology model is generated by associating each storage node in the storage node set with each network path in the network path set. Obtain the load status of each storage node and the link transmission capacity of each network path in the resource topology model; Based on the topological relationships in the resource topology model, the first impact of changes in the load status of storage nodes on the link transmission capacity of the connected network paths, and the second impact of changes in the link transmission capacity of the network paths on the load status of the connected storage nodes are identified. Based on the first influence relationship and the second influence relationship, the associated constraints are determined, and the associated constraints are used as resource constraints to form a set of resource constraints.

3. The method according to claim 2, characterized in that, The storage nodes in the storage node set are topologically associated with each network path in the network path set to generate a resource topology model, including: Obtain the load characteristics of each storage node in the storage node set and the transmission characteristics of each network path in the network path set; Based on the connection information of the network path, determine the storage nodes connected to each network path and establish the connection mapping relationship between the storage nodes and the network path; For each storage node in the connection mapping relationship, identify multiple network paths connecting the storage node, and analyze the superposition effect of the transmission characteristics of the multiple network paths on the load characteristics of the storage node; For each storage node in the connection mapping relationship, analyze the distribution effect of the load characteristics of the storage node on the transmission characteristics of the multiple network paths connecting the storage node; Based on the superposition effect and the allocation effect, the bidirectional coupling relationship between storage nodes and network paths is determined, and a topological association structure is constructed based on the connection mapping relationship and the bidirectional coupling relationship; A resource topology model is generated based on the topology association structure, and the storage nodes and network paths in the resource topology model are connected by topology edges carrying bidirectional coupling relationships.

4. The method according to claim 1, characterized in that, Based on the set of resource constraints, the resource request is associated and mapped with available storage nodes and available network paths in the cloud computing environment to generate candidate resource combinations, including: Extract the constraints between storage nodes and network paths from the set of resource constraints, identify available storage nodes and available network paths in the cloud computing environment that satisfy the constraints based on the constraints, and obtain a set of resource candidates that satisfy the constraints. Analyze the storage resource requirements and network resource requirements in the resource request; For each available storage node in the resource candidate set, an available network path connected to the available storage node is determined according to the constraints, and an association mapping is performed between the available storage node and the available network path to determine the association mapping relationship. Based on the association mapping relationship, obtain the available capacity of the available storage node and the available bandwidth of the available network path, and determine whether the combination of the available storage node and the available network path meets the storage resource requirement and the network resource requirement; The combination of available storage nodes and available network paths that meet the storage resource requirements and the network resource requirements is selected as the candidate resource combination.

5. The method according to claim 4, characterized in that, For each available storage node in the resource candidate set, an available network path connected to the available storage node is determined according to the constraints. An association mapping is then performed between the available storage node and the available network path to determine the association mapping relationship, including: For each available storage node in the resource candidate set, parse the load constraint rules and connection constraint rules related to the available storage node in the constraint conditions; Based on the connection constraint rules, select available network paths that are connected to the available storage nodes from the resource candidate set; For each available network path selected, verify whether the load status of the available storage node and the transmission status of the available network path satisfy the load constraint rules according to the load constraint rules. Pair available storage nodes that meet the load constraint rules with available network paths to establish an association mapping between the available storage nodes and the available network paths; The association mapping is labeled, and the label includes the constraint information of the load constraint rules and the connection constraint rules on the association between the available storage nodes and the available network paths; The association mapping relationship is determined based on the annotation, and the association mapping relationship represents the association structure between the available storage node and the available network path under the conditions of satisfying the load constraint rules and the connection constraint rules.

6. The method according to claim 1, characterized in that, Based on the candidate resource combinations and the preset scheduling strategy, a resource allocation scheme is generated that includes the binding relationship between target storage nodes and target network paths, including: Based on the preset scheduling strategy, the scheduling objective function and scheduling constraints are determined. The scheduling objective function represents the optimization direction of the collaborative scheduling of storage nodes and network paths, and the scheduling constraints represent the restrictive rules that the collaborative scheduling of storage nodes and network paths must satisfy. For each of the candidate resource combinations, obtain the load status of available storage nodes and the transmission status of available network paths in the combination; Based on the load state and the transmission state, calculate the evaluation value of the combination under the scheduling objective function; Filter out the combinations that satisfy the scheduling constraints to obtain a set of constrained combinations, and select the target combination with the largest evaluation value from the set of constrained combinations. The available storage nodes in the target combination are determined as target storage nodes, and the available network paths in the target combination are determined as target network paths; Establish a binding relationship between the target storage node and the target network path, and generate a resource allocation scheme based on the target storage node, the target network path and the binding relationship.

7. The method according to claim 1, characterized in that, Based on the resource allocation scheme, a resource scheduling operation is performed on the resource request, and resource usage status data is collected during the scheduling operation. The resource usage status data is then used to dynamically update the resource constraint relationship set and the preset scheduling strategy, including: Based on the target storage node and target network path in the resource allocation scheme, perform resource scheduling operations on the resource request; During the execution of the resource scheduling operation, load change data of the target storage node and transmission change data of the target network path are collected, and resource usage status data is generated based on the load change data and the transmission change data. Based on the resource usage status data, constraint relationship update information and scheduling strategy update information are determined. The constraint relationship update information is used to characterize the actual constraint relationship deviation between the target storage node and the target network path, and the scheduling strategy update information is used to characterize the effect deviation of the preset scheduling strategy in the actual execution process. The resource constraint relationship set is updated according to the constraint relationship update information, and the scheduling objective function and scheduling constraint conditions in the preset scheduling strategy are adjusted according to the scheduling strategy update information.

8. A dynamic scheduling system for storage and network resources in a cloud computing environment, used to implement the method as described in any one of claims 1-7, characterized in that, include: The request retrieval unit is used to retrieve resource requests in the cloud computing environment; The constraint construction unit is used to generate a resource topology model containing storage nodes and network paths based on the storage resource requirements and network resource requirements in the resource request, and to determine the resource constraint relationship according to the load status and link transmission capacity of each node in the resource topology model, thereby obtaining a set of resource constraint relationships. The combination mapping unit is used to associate and map the resource request with available storage nodes and available network paths in the cloud computing environment according to the set of resource constraint relationships, and generate candidate resource combinations; The scheme binding unit is used to generate a resource allocation scheme that includes the binding relationship between the target storage node and the target network path based on the candidate resource combination and the preset scheduling strategy. The scheduling update unit is used to perform resource scheduling operations on the resource request based on the resource allocation scheme, collect resource usage status data during the scheduling operation execution process, and dynamically update the resource constraint relationship set and the preset scheduling strategy using the resource usage status data.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images