Cross-domain network resource orchestration method and system, program product and electronic equipment

Abstract

The invention relates to the technical field of network resource management, and relates to a cross-domain network resource arrangement method and system, a computer program product and electronic equipment. The method is applied to a resource arrangement system, the system comprises a global coordinator, domain managers registered by network domains in the global coordinator and local agents deployed under each domain manager, and state summary information of each network domain is calculated through the global coordinator based on an attention mechanism to obtain attention weights of each network domain; determining a cross-domain resource allocation strategy through a global coordinator according to the state summary information and the attention weight, and allocating the cross-domain resource allocation strategy to a domain manager; for each domain manager, carrying out intra-domain decision-making by taking the sub-policy as a constraint, obtaining an intra-domain resource allocation policy, and issuing the intra-domain resource allocation policy to a local agent for execution; based on the attention weight, the global coordinator is utilized to allocate the global performance reward as a domain level reward for the domain manager, and the domain level reward is allocated by the domain manager as a local reward for the local agent.

Description

Technical Field

[0001] This disclosure relates to the field of network resource management technology, and more specifically, to a cross-domain network resource orchestration method, a cross-domain network resource orchestration system, a computer program product, and an electronic device. Background Technology

[0002] With the rapid development and widespread application of cloud computing, edge computing, and 5G (Fifth Generation Mobile Communication Technology), modern computing architectures are evolving towards large-scale, distributed, and heterogeneous architectures. Against this backdrop, cross-domain network resource orchestration has become a key technology supporting various distributed applications and services. Cross-domain refers to multiple heterogeneous, autonomous network environments that may belong to different management entities, such as multiple public clouds, private clouds, edge computing nodes, and 5G network slices. How to efficiently and intelligently coordinate and allocate computing, storage, and network resources in these dispersed domains to meet the diverse performance, cost, and reliability requirements of applications is a major challenge facing industry and academia.

[0003] Currently, cross-domain resource orchestration largely relies on rule-based heuristic algorithms or static optimization models. While these methods can achieve resource scheduling to some extent, they lack flexibility, adaptability, and intelligence. They struggle to cope with dynamically changing workloads, complex inter-domain dependencies, and multiple often conflicting optimization objectives, which negatively impacts the quality and effectiveness of cross-domain network resource orchestration.

[0004] It should be noted that the information in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide a cross-domain network resource orchestration method, a cross-domain network resource orchestration system, a computer program product, and an electronic device, thereby improving the quality and effectiveness of cross-domain network resource orchestration.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part by practice of this disclosure.

[0007] According to one aspect of this disclosure, a cross-domain network resource orchestration method is provided, applied to a resource orchestration system. The resource orchestration system includes a global coordinator, domain managers registered with the global coordinator for different network domains, and local agents deployed under each domain manager. The method includes: calculating the attention weight of each network domain based on the collected state summary information of each network domain by the global coordinator using an attention mechanism; determining a cross-domain resource allocation strategy by the global coordinator based on the state summary information and attention weight, and allocating the cross-domain resource allocation strategy to the domain managers according to the network domain; making intra-domain decisions for each domain manager with sub-policies as constraints to obtain an intra-domain resource allocation strategy and distributing it to the corresponding local agent for decision execution; and allocating global performance rewards to each domain manager based on attention weights by the global coordinator, and allocating the domain-level rewards to the local agents through the domain managers for decision optimization.

[0008] In one exemplary embodiment of this disclosure, the global coordinator calculates the attention weights of each network domain based on the collected state summary information of each network domain using an attention mechanism. This includes: the global coordinator generating a global query vector, which represents the target state of the current global resource allocation; determining a key vector based on the state summary information of each network domain, and calculating the similarity between the global query vector and each key vector to obtain the attention score of each network domain; and normalizing the attention scores to obtain the attention weights of each network domain.

[0009] In one exemplary embodiment of this disclosure, a cross-domain resource allocation strategy is determined by a global coordinator based on state summary information and attention weights, including: obtaining a meta-policy network, which is trained based on historical resource orchestration tasks and used to adjust the resource orchestration strategy; the global coordinator inputs the state summary information and attention weights into the meta-policy network to perform forward propagation in the meta-policy network and outputs the cross-domain resource allocation strategy.

[0010] In one exemplary embodiment of this disclosure, the cross-domain resource allocation strategy is assigned to the domain manager according to the network domain, including: dividing the cross-domain resource allocation strategy into action strategies for each network domain by a global coordinator based on state summary information and attention weights; and the global coordinator distributing the action strategies for each network domain to the corresponding domain manager.

[0011] In one exemplary embodiment of this disclosure, for each domain manager, intra-domain decision-making is performed with sub-policies as constraints to obtain intra-domain resource allocation policies and distribute them to the corresponding local agents. This includes: each domain manager, with the corresponding sub-policy as constraints, performs intra-domain decision-making based on a multi-agent cooperation algorithm and intra-domain state information to determine the task to be executed and the corresponding target agent; and the task to be executed is distributed to the corresponding target agent through the domain manager.

[0012] In one exemplary embodiment of this disclosure, the local agent performs decision execution, including: the local agent obtaining the current resource status of the corresponding computing node; and executing the received resource allocation strategy based on the current resource status.

[0013] In one exemplary embodiment of this disclosure, the method further includes: aggregating the local agent rewards within each network domain to obtain intra-domain rewards, which are used to reflect the performance of the network domain; and the global coordinator collecting the intra-domain rewards of the network domains reported by each domain manager and merging the intra-domain rewards according to attention weights to obtain a global performance reward.

[0014] In one exemplary embodiment of this disclosure, the method further includes: performing reinforcement learning on an intra-domain decision algorithm through a domain manager based on the received domain-level reward; and updating a local decision algorithm through a local agent based on the received local reward.

[0015] In one exemplary embodiment of this disclosure, the method further includes: collecting state summary information of each network domain, wherein the state summary information is obtained by the domain manager and / or local agent of the network domain aggregating or summarizing the original state information.

[0016] In one exemplary embodiment of this disclosure, the global coordinator, each of the domain managers, and the local agents employ different operating time steps; wherein the operating time step of the global coordinator is greater than the operating time step of each of the domain managers, and the operating time step of each domain manager is greater than the operating time step of the deployed local agents.

[0017] According to one aspect of this disclosure, a cross-domain network resource orchestration system is provided. The system includes a global coordinator, domain managers registered with the global coordinator for different network domains, and local agents deployed under each domain manager. The global coordinator is used to calculate the attention weight of each network domain based on the collected state summary information of each network domain using an attention mechanism. The attention weight reflects the current load level of the network domain. The global coordinator is also used to determine a cross-domain resource allocation strategy based on the state summary information and the attention weight, and to distribute the cross-domain resource allocation strategy to the domain managers according to the network domain. The domain managers are used to make intra-domain decisions based on sub-policies, obtain intra-domain resource allocation strategies, and distribute them to the corresponding local agents for decision execution. The global coordinator is also used to allocate global performance rewards as domain-level rewards to each domain manager based on the attention weight, and the domain managers are also used to allocate domain-level rewards as local rewards to the local agents for decision optimization.

[0018] According to one aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements any of the above methods.

[0019] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any of the above methods by executing the executable instructions.

[0020] The cross-domain network resource orchestration method in the exemplary embodiments of this disclosure calculates the attention weight of each network domain based on the collected state summary information of each network domain using a global coordinator based on an attention mechanism. The attention weight is used to reflect the current load level of the network domain. The global coordinator determines the cross-domain resource allocation strategy based on the state summary information and attention weight, and distributes the cross-domain resource allocation strategy to the domain managers according to the network domain. For each domain manager, intra-domain decision-making is performed with sub-policies as constraints to obtain the intra-domain resource allocation strategy, which is then distributed to the corresponding local agent for decision execution. The global coordinator allocates the global performance reward to the domain-level reward of each domain manager based on the attention weight, and the domain managers allocate the domain-level reward to the local agent for decision optimization.

[0021] On the one hand, by decomposing the complex global optimization problem into multiple sub-problems through a hierarchical decision-making mechanism, the decision-making dimensionality is significantly reduced, solving the action space explosion problem and facilitating real-time intelligent orchestration of large-scale cross-domain resources, thereby improving decision-making efficiency. On the other hand, a three-layer distributed decision-making architecture of global coordination, intra-domain decision-making, and local execution is adopted. The global coordinator is only responsible for formulating macro-level cross-domain allocation strategies, while the specific intra-domain resource decision-making authority is delegated to each domain manager, reducing the computational burden and decision-making complexity of the global coordinator. Simultaneously, domains do not need to frequently exchange detailed internal state information; they only need to report state summary information to the global coordinator, protecting the privacy and data security of each domain. Furthermore, through an attention-weighted reward allocation mechanism, the local optimization goals of each domain and each local agent are aligned with the system's global optimization goal, thus ensuring the global effectiveness of decision optimization.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0023] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation.

[0024] Figure 1 An architecture diagram of a resource orchestration system according to an exemplary embodiment of the present disclosure is shown.

[0025] Figure 2 A flowchart of a cross-domain network resource orchestration method according to an exemplary embodiment of the present disclosure is shown.

[0026] Figure 3 An interaction diagram is shown, illustrating an example of global collaborative resource orchestration among entities using different operational time steps, according to an exemplary embodiment of this disclosure.

[0027] Figure 4 A schematic diagram of the composition of a cross-domain network resource orchestration system according to an exemplary embodiment of the present disclosure is shown.

[0028] Figure 5 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown.

[0029] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0030] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar structures, and therefore their detailed description will be omitted.

[0031] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details described, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known structures, methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0032] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, or in one or more software-hardened modules, or in different network and / or processor devices and / or microcontroller devices.

[0033] Currently, cross-domain resource orchestration largely relies on rule-based heuristic algorithms or static optimization models. While these methods can achieve resource scheduling to some extent, they lack flexibility, adaptability, and intelligence. They struggle to handle dynamically changing workloads, complex inter-domain dependencies, and multiple often conflicting optimization objectives (such as resource utilization, service quality, and cost). Furthermore, the decision-making logic of these methods is usually predefined manually, making it impossible to learn from historical data and continuously optimize, resulting in poor performance in unforeseen scenarios. Moreover, in reinforcement learning-based schemes, there are issues with convergence difficulties and decision delays. Privacy constraints in cross-domain environments require that state information from each domain cannot be shared without reservation, raising privacy or data security concerns. Furthermore, potential objective conflicts exist among multiple autonomous domains; local optimization in one domain may compromise global performance, and a single reward function is insufficient to coordinate such complex multi-objective, multi-agent optimization problems.

[0034] Therefore, due to one or more of the aforementioned problems, it is difficult to achieve efficient, intelligent, privacy-secure, and globally optimized cross-domain network resource orchestration. Based on this, exemplary embodiments of this disclosure provide a cross-domain network resource orchestration method, such as... Figure 1 The diagram shown is an architecture diagram of a resource orchestration system.

[0035] like Figure 1 The resource orchestration system includes a global coordinator, domain managers registered with the global coordinator for different network domains, and local agents deployed under each domain manager.

[0036] The global coordinator is the highest logical layer in the entire resource orchestration system, used for information aggregation, global decision-making, and policy distribution. A network domain is an autonomous network unit with independent management authority and a specific geographical area; it can be understood as the basic unit of resource management and allocation, such as cloud domains, edge domains, and endpoint domains (IoT domains). The domain manager registers with the global coordinator, representing its entire network domain. It acts as a bridge connecting the global coordinator and resources within the domain, and can perform domain-wide information abstraction, policy reception and domain transformation, and domain-wide decision-making. Local agents are specific resource nodes deployed within the network domain, such as servers, switches, base stations, and storage devices, responsible for the lowest-level operations, such as status reporting and policy execution.

[0037] As an example, a global coordinator instance can be created and domain managers representing the cloud, edge, and device network domains can be registered for it. Under each domain manager, a corresponding number of local agents can be deployed. For example, the cloud domain contains five compute node agents, the edge domain contains five edge node agents, and the IoT domain contains three gateway device agents.

[0038] It should be noted that, Figure 1 The number of domain managers and their deployed local agents in the system interactions shown are merely exemplary. The exemplary embodiments of this disclosure may be used to determine the number of deployed entities based on the actual application scenario.

[0039] The following is combined Figure 1 The cross-domain network resource orchestration method of exemplary embodiments of this disclosure will be described, such as... Figure 2 As shown, the cross-domain network resource orchestration method includes steps S210 to S240: Step S210: The global coordinator calculates the state summary information of each network domain based on the attention mechanism to obtain the attention weight of each network domain. The attention weight is used to reflect the current load level of the network domain.

[0040] Step S220: The global coordinator determines the cross-domain resource allocation strategy based on the state summary information and attention weights, and distributes the cross-domain resource allocation strategy to the domain manager according to the network domain.

[0041] Step S230: For each domain manager, make intra-domain decisions based on sub-policies to obtain intra-domain resource allocation policies and distribute them to the corresponding local agents for decision execution.

[0042] Step S240: Using the global coordinator based on attention weights, global performance rewards are allocated to domain-level rewards for each domain manager, and the domain-level rewards are allocated to local agents through the domain managers for decision optimization.

[0043] The cross-domain network resource orchestration method in the exemplary embodiments of this disclosure, on the one hand, decomposes the complex global optimization problem into multiple sub-problems through a hierarchical decision-making mechanism, significantly reducing the decision-making dimensionality, solving the action space explosion problem, and facilitating real-time intelligent orchestration of large-scale cross-domain resources, thereby improving decision-making efficiency. On the other hand, it adopts a three-layer distributed decision-making architecture of global coordination, intra-domain decision-making, and local execution. The global coordinator is only responsible for formulating macro-level cross-domain allocation strategies, while the specific intra-domain resource decision-making authority is delegated to each domain manager, reducing the computational burden and decision-making complexity of the global coordinator. Simultaneously, each domain does not need to frequently exchange detailed internal state information; it only needs to report state summary information to the global coordinator, protecting the privacy and data security of each domain. Furthermore, through an attention-weight-based reward allocation mechanism, the local optimization objectives of each domain and each local agent are aligned with the system's global optimization objective, thus ensuring the effectiveness of decision optimization.

[0044] Steps S210 to S240 will be described in more detail below.

[0045] In step S210, the global coordinator calculates the state summary information of each network domain based on the attention mechanism to obtain the attention weight of each network domain. The attention weight is used to reflect the current load level of the network domain.

[0046] In the exemplary embodiments of this disclosure, the state summary information is not the original state of all devices within the network domain, but rather a high-level feature representation oriented towards global decision-making, formed by the domain manager and / or local agent after aggregating, filtering, and abstracting detailed, multi-dimensional resource data. In other words, the state summary information is obtained by the domain manager and / or local agent of the network domain through aggregation or summarization of the original state information, thereby avoiding the exposure of domain privacy or overly complex details to the global coordinator, and also reducing communication and computational overhead to some extent.

[0047] The attention mechanism is used to implement cross-domain communication. The global coordinator calculates the importance weights (attention weights) of each domain's state based on this mechanism, allowing for more focused attention on critical network domains when integrating information, thus achieving more efficient and intelligent information aggregation and decision-making. The attention weight reflects the current load level of the network domain; a higher attention weight indicates a higher current load level, and vice versa.

[0048] In an exemplary embodiment, the attention weights for each network domain are calculated by a global coordinator based on an attention mechanism, which calculates the state summary information of each network domain. First, the global coordinator generates a global query vector, which represents the target state of the current global resource allocation. Then, key vectors are determined based on the state summary information of each network domain, and the similarity between the global query vector and each key vector is calculated to obtain the attention score for each network domain. Finally, the attention scores are normalized to obtain the attention weight for each network domain.

[0049] The global coordinator is a dynamic, goal-oriented vector generated by the global coordinator. It is not a fixed rule but encapsulates the core requirements and target state of the current global resource allocation task. It can globally optimize objectives, such as minimizing total latency or maximizing throughput. The key vector is a vector mapped from the state summary information of each network domain through a transformation function (such as a neural network or embedding layer). This transforms the state summary into a representation in the same semantic space as the global query vector, allowing for meaningful similarity comparisons. The attention score reflects the degree of correlation between the domain state of the network domain and the global goal. For example, if a network domain is heavily loaded (e.g., high resource utilization, long task queue), its state vector (value vector) will have a higher similarity to the global query vector (e.g., focusing on high load), thus obtaining a higher attention weight. Normalization can be achieved by using the softmax function to normalize the attention score into attention weights.

[0050] Optionally, the target state of global resource allocation can be mapped through a pre-trained state representation model to obtain a global query vector. The pre-trained state representation model can be a network structure such as CNN (Convolutional Neural Networks), RNN (Recurrent Neural Network), or multilayer perceptron, or of course, a large language model. There are no specific restrictions on this.

[0051] In an exemplary embodiment of this disclosure, the scheduling strategy is associated with the target state, which improves task execution efficiency and the rationality of resource utilization, and avoids misallocation of resources.

[0052] In step S220, the global coordinator determines the cross-domain resource allocation strategy based on the state summary information and attention weights, and then assigns the cross-domain resource allocation strategy to the domain manager according to the network domain.

[0053] In an exemplary embodiment of this disclosure, the cross-domain resource allocation strategy is a macro-level, goal-oriented coordination instruction used to characterize how resources flow and are redistributed between different network domains at the global level. When the global coordinator determines the cross-domain resource allocation strategy based on state summary information and attention weights, the state summary information provides the objective resource capacity and capabilities of each domain, serving as the resource basis for decision-making. The decision-making process involves optimally matching global task requirements (implied in attention weights) with global resource supply (state summary information) to determine resource allocation between different network domains, such as migrating tasks from heavily loaded domain networks to lightly loaded network domains.

[0054] In one exemplary embodiment, a cross-domain resource allocation strategy is determined by a global coordinator based on state summary information and attention weights, including: First, a meta-policy network is obtained. This network is trained based on historical resource orchestration tasks and is used to adjust resource orchestration strategies. Then, the global coordinator inputs state summary information and attention weights into the meta-policy network for forward propagation, outputting cross-domain resource allocation strategies.

[0055] Among these, meta-learning algorithms enable the global coordinator to quickly adapt to different cross-domain task requirements. The global coordinator integrates meta-learning capabilities, allowing it to extract experience from various cross-domain tasks coordinated in historical resource encoding tasks, thus quickly adapting to new and unknown task environments. The meta-policy network, also known as a trained neural network, models the complex, multi-objective resource allocation decision problem as a data-driven neural network learning and execution process; it is a learnable and adaptively adjustable neural network. Historical resource orchestration tasks are complete cross-domain resource scheduling events that have occurred in the past. Each task contains a data chain from state to decision to result, which may include, but is not limited to, the global state at the time (such as state summary information of each network domain, calculated attention weights, etc.), the cross-domain resource allocation strategy formulated at the time, and the global performance reward obtained after executing the strategy (such as task completion time, resource utilization, cost, etc.). A neural network can be trained based on historical resource orchestration tasks to obtain a meta-policy network. The forward propagation refers to the process of feeding the current state summary information and attention weights as input into the pre-trained meta-policy network, and after calculation by each layer of the network, finally obtaining the specific cross-domain resource allocation policy vector at the output layer.

[0056] As an example, the global coordinator could be based on the Meta-SAC algorithm (a maximum entropy-based reinforcement learning algorithm), which includes a meta-learner (meta-policy network) that learns rapid adjustment policies from historical resource orchestration tasks. When a new cross-domain task arises, the meta-policy network leverages prior experience (i.e., policy adjustment methods learned from handling similar tasks in the past) to quickly generate a resource allocation policy adapted to the new task. Specifically, the meta-policy network performs forward propagation based on state summary information and attention weights, outputting a cross-domain resource allocation policy. This policy could include allocating resources between different domains (e.g., migrating a task from a heavily loaded domain to a lightly loaded domain).

[0057] The exemplary embodiments of this disclosure, by learning the relationship between states, attention and optimal strategies through neural networks, can efficiently and accurately obtain allocation strategies that conform to global resources and performance when facing load resource orchestration.

[0058] In one exemplary embodiment, assigning cross-domain resource allocation policies to the domain manager according to network domains includes: First, the global coordinator divides the cross-domain resource allocation strategy into action policies for each network domain based on state summary information and attention weights. Then, the global coordinator distributes the action policies for each network domain to the corresponding domain managers.

[0059] Each network domain's action policy is a specific instantiation of the cross-domain resource allocation policy within a single network domain. It is a set of instructions issued by the global coordinator to a specific domain manager, instructing the network domain on the necessary actions for this global collaborative task. Dividing the cross-domain resource allocation policy into action policies for each network domain can be understood as decomposing the cross-domain resource allocation policy into domain-level action policies. These action policies can be issued to each domain manager in the form of domain actions. For example, for the edge domain, the action policy might be to migrate 30% of the computing tasks; for the cloud domain, it might be to accept these migrated tasks and reserve 20% of the resources; for the IoT domain, it might be to maintain the status quo. Optionally, a weighted allocation method based on attention weights can be used. Here, the total resource requirement in the cross-domain resource allocation policy is R, and the attention weight of each network domain is α. i The state summary information for each network domain is c i Therefore, for each network domain, its target allocation amount Ai can be determined as: Ai=R α i Formula 1 Then, the state summary information can be compared with the corresponding target allocation. If the target allocation is less than or equal to the resource indicated by the corresponding state summary information, the allocation for that network domain is determined to be Ai. Conversely, if the target allocation is greater than the resource indicated by the corresponding state summary information, the allocation for that network domain is determined to be the resource indicated by the state summary information. In this approach, if there is a resource gap due to insufficient capacity, allocation can be made in network domains that have not yet reached their capacity limits based on attention weights. Therefore, under the guidance of attention weights, near-optimal resource orchestration can be achieved.

[0060] Alternatively, a reinforcement learning policy network can be used to divide the cross-domain resource allocation strategy into action policies for each network domain through a global coordinator based on state summary information and attention weights. Specifically, a policy partitioning network can be pre-trained, encoding the current state summary, attention weights, and cross-domain resource allocation strategy into feature vectors, and inputting them into the policy partitioning network to directly generate action policies for each network domain. This policy partitioning network can still be trained based on historical resource arrangement of characters. Based on this, the trained policy partitioning network can learn the optimal partitioning pattern from historical data, automatically handle various complex situations, and has high generalization ability.

[0061] In step S230, for each domain manager, intra-domain decisions are made with sub-policies as constraints to obtain intra-domain resource allocation policies, which are then sent to the corresponding local agents for decision execution.

[0062] In an exemplary embodiment of this disclosure, the sub-policy includes domain-level instructions that each domain manager can execute, encompassing the role and actions that the domain needs to assume in the global policy, such as resource adjustment amounts and task migration amounts. After receiving the instructions, the domain manager uses them as constraints for intra-domain decisions and further refines them into actions for local agents based on the intra-domain state.

[0063] Intra-domain decision-making is the resource reallocation process performed by the domain manager within its managed network domain. The intra-domain resource allocation policy is the final output of this decision, including a set of executable instructions for local agents within the domain. For example, if the sub-policy is to allocate 500 CPU cores, the intra-domain resource allocation policy might be a combination of atomic operations such as creating two 4-core virtual machines on Server-01, migrating three 2-core virtual machines on Server-02 to free up resources, and then creating one 6-core virtual machine on Server-05.

[0064] The local agent is the ultimate executor of domain policies. It receives specific instructions from the domain manager and directly manipulates the resources it manages by calling the local API (Application Programming Interface) or configuration command line.

[0065] In one exemplary embodiment, for each domain manager, intra-domain decisions are made with sub-policies as constraints to obtain intra-domain resource allocation policies, which are then distributed to the corresponding local agents, including: Each domain manager, constrained by its corresponding sub-policy, makes intra-domain decisions based on a multi-agent cooperation algorithm and intra-domain state information to determine the tasks to be executed and their corresponding target agents. Then, the domain manager distributes the tasks to be executed to the corresponding target agents.

[0066] In this approach, each local agent can be considered an agent, and the entire network domain constitutes a multi-agent environment. In this multi-agent cooperative algorithm, the agents (local agents) learn to cooperate with each other while learning to execute tasks. For example, the global coordinator can employ the MASAC algorithm (an extension of SAC (Soft Actor-Critic) multi-agent algorithm), which, based on state summary information and attention weights, divides cross-domain resource allocation strategies into action policies for each network domain. For instance, migrating three 2-core virtual machines on Server-02 to free up resources.

[0067] The exemplary embodiments of this disclosure can solve the problem of resource allocation among nodes in a domain based on a multi-agent cooperative algorithm, while maintaining the autonomy of each domain and preventing the global coordinator from getting bogged down in overly detailed decision-making.

[0068] In one exemplary embodiment, the local agent performs decision execution, including: the local agent first obtains the current resource status of the corresponding computing node, and then executes the received resource allocation strategy based on the current resource status.

[0069] As mentioned above, a local agent is a lightweight software entity deployed on a specific computing node, which is a concrete physical or logical unit capable of independently providing computing power. Current resource status refers to the hardware and software operating data of a computing node when it receives execution instructions, including but not limited to real-time CPU utilization, memory usage, disk I / O rate, network bandwidth usage, GPU utilization, total CPU cores, total memory capacity, list of running processes / virtual machines / containers, and current load.

[0070] Specifically, based on the current resource status, the local agent executes specific resource allocation actions according to the received resource allocation policy, such as allocating additional computing resources, adjusting bandwidth limits, or performing task migration. All local decisions, while considering resource allocation policies, must prioritize ensuring the quality and stability of local services. This can be understood as a local agent being an execution unit with local decision-making capabilities, performing local feasibility checks and adaptive adjustments to ensure the success rate of operations and the stability of nodes.

[0071] Optionally, the local agent can parse the resource allocation policy, extract resource requirements, and compare the resource requirements with the current resource status for verification. If the status meets the requirements, the operation is executed; if the status does not meet the requirements, execution is rejected, and the error reason is immediately reported to the domain manager.

[0072] Optionally, after receiving the policy, the local agent can first logically reserve the amount of resources required by the policy in the local resource manager. Once the reservation is successful, the status of the resources is updated to reserved and will not be occupied by other tasks. The specific resource allocation operation will then be executed at the agreed time or after receiving a trigger signal.

[0073] The exemplary embodiments of this disclosure improve the accuracy of policy execution by having a local agent execute the received resource allocation policy based on the current resource status, thereby avoiding system-level failures.

[0074] In step S240, the global coordinator uses attention weights to allocate global performance rewards to domain-level rewards for each domain manager, and the domain-level rewards are then allocated to local agents by the domain managers for decision optimization.

[0075] In the exemplary embodiments of this disclosure, the global performance reward is an overall performance evaluation of the entire cross-domain resource orchestration system after completing a task once or over a period of time, reflecting the overall system performance. The domain-level reward is a decomposition of the global performance reward across different network domains, i.e., a reward allocated to each domain manager, used to evaluate its contribution to the overall collaborative task. The local reward is a reward further allocated by the domain manager to each local agent within its domain, used to evaluate the contribution of each compute node to task execution within the domain.

[0076] In one exemplary embodiment, obtaining the global performance reward may include: First, the local agent rewards within each network domain are aggregated to obtain the intra-domain reward, which reflects the performance of the network domain. Then, the global coordinator collects the intra-domain rewards reported by each domain manager and merges the intra-domain rewards according to attention weights to obtain the global performance reward.

[0077] Local agent rewards are the incentives for individual computing nodes executing strategies. These rewards are derived from the node's own monitoring data and do not directly depend on the success or failure of the global task. Examples include the node's CPU, memory, and GPU utilization, average task processing latency, I / O throughput, node energy efficiency, and error rate. High or low values ​​in these metrics will result in low rewards. The correlation between each metric and its reward percentage can be pre-defined to determine the appropriate local agent reward.

[0078] Intra-domain rewards are the value obtained by aggregating the rewards of all local agents under the jurisdiction of the domain manager. This can be done through weighted averaging, summation, or taking the minimum value, without restriction. Merging intra-domain rewards based on attention weights can be achieved by weighted summation based on attention weights, then dividing by the total weight to obtain the global performance reward. The global coordinator then distributes the global performance reward back to each domain manager based on the attention weight of each network domain. For example, a domain with a higher attention weight receives a larger reward. This ensures a fair reflection of each domain's contribution to the global performance reward.

[0079] In the exemplary embodiments disclosed herein, global performance is no longer a direct measurement of a single dimension, but rather a fusion of the local performance of countless underlying entities. This is beneficial for improving the accuracy and globality of global performance rewards, and the overall reward obtained from global optimization is reasonably back-allocated to each domain or even local agent. This ensures that decision-making entities at each level can maintain consistency with the global optimization goal while pursuing their own local rewards, thus avoiding inconsistencies between goals at different levels.

[0080] In one exemplary embodiment, a reward-based optimization decision-making implementation method is also provided, specifically: Based on the received domain-level rewards, the domain manager performs reinforcement learning on the domain-specific decision-making algorithm, and based on the received local rewards, the local agent updates the local decision-making algorithm.

[0081] In this system, the domain manager receives domain-level rewards and uses them to update its multi-agent cooperative algorithm model. Through reinforcement learning updates, the domain manager can optimize its intra-domain resource allocation strategy, thereby making better decisions in the future. Similarly, the domain manager further distributes the domain-level rewards to local agents within the domain, who then use these rewards to update their local policies. Thus, from global to local levels, the entire system continuously optimizes through reward allocation and model updates, constantly improving the quality of resource orchestration. The finer-grained decision-making algorithms of the domain manager and local agents involve adjusting the corresponding model network parameters, which are not specifically limited in this regard.

[0082] In one exemplary embodiment, the global coordinator, each domain manager, and the local agent employ different operating time steps. Specifically, the global coordinator's operating time step is larger than that of each domain manager, and the domain managers' operating time steps are larger than those of the deployed local agents.

[0083] As an example, in a hierarchical framework, decision-making entities at different levels learn and make decisions at different frequencies. The global coordinator (minute-level) focuses on long-term trends, the domain manager (second-level) focuses on medium-term coordination, and the local agent (millisecond-level) focuses on instantaneous responses. Figure 3 As shown, Figure 3 An interaction diagram is shown illustrating the global collaborative resource orchestration achieved between entities through different operating time steps. This design significantly improves the system's decision-making efficiency and avoids unnecessary computational overhead. Of course, the exemplary embodiments of this disclosure can adjust the operating time steps of each decision-making entity according to actual needs.

[0084] It should be noted that, Figure 3 The details of the resource arrangement involved have been described in the exemplary embodiments above, and will not be repeated here.

[0085] The cross-domain network resource orchestration method in the exemplary embodiments of this disclosure, on the one hand, decomposes the complex global optimization problem into multiple sub-problems through a hierarchical decision-making mechanism, significantly reducing the decision-making dimensionality, solving the action space explosion problem, and facilitating real-time intelligent orchestration of large-scale cross-domain resources, thereby improving decision-making efficiency. On the other hand, it adopts a three-layer distributed decision-making architecture of global coordination, intra-domain decision-making, and local execution. The global coordinator is only responsible for formulating macro-level cross-domain allocation strategies, while the specific intra-domain resource decision-making authority is delegated to each domain manager, reducing the computational burden and decision-making complexity of the global coordinator. Simultaneously, each domain does not need to frequently exchange detailed internal state information; it only needs to report state summary information to the global coordinator, protecting the privacy and data security of each domain. Furthermore, through an attention-weight-based reward allocation mechanism, the local optimization goals of each domain and each local agent are kept consistent with the system's global optimization goal, thus ensuring the effectiveness of long-term decision optimization. Therefore, the exemplary embodiments of this disclosure can achieve efficient utilization of cross-domain resources, guaranteed quality of service, privacy and security protection, and adaptive large-scale intelligent orchestration of network resources.

[0086] In an exemplary embodiment of this disclosure, a cross-domain network resource orchestration system is also provided. (See reference...) Figure 4 As shown, the cross-domain network resource orchestration system 400 may include a global coordinator 410, domain managers 420 registered with the global coordinator 410 for different network domains, and local agents 440 deployed under each domain manager 420. Specifically: The global coordinator 410 is used to calculate the attention weight of each network domain based on the collected state summary information of each network domain using an attention mechanism. The attention weight is used to reflect the current load level of the network domain. The global coordinator 410 is also used to determine the cross-domain resource allocation strategy based on the state summary information and attention weight, and to distribute the cross-domain resource allocation strategy to the domain managers according to the network domain. The domain manager 420 is used to make intra-domain decisions with sub-policies as constraints, to obtain the intra-domain resource allocation strategy, and to send it to the corresponding local agent 440 for decision execution. The global coordinator 410 is also used to allocate the global performance reward to the domain-level reward of each domain manager 420 based on the attention weight. The domain manager 420 is also used to allocate the domain-level reward to the local agent 440 for decision optimization.

[0087] Since the details of each functional part of the cross-domain network resource orchestration system of the exemplary embodiments of this disclosure have been described in the exemplary embodiments of the cross-domain network resource orchestration method described above, they will not be repeated here.

[0088] Exemplary embodiments of this disclosure also provide a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the cross-domain network resource orchestration method described above.

[0089] In one implementation, the computer program product can be a tangible product containing a computer program, such as a computer-readable storage medium storing the computer program. The readable storage medium can be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory, hard disk drive (HDD), solid-state drive (SSD), etc. For example, the computer program product can be implemented as a non-volatile storage medium storing a computer program, such as read-only memory, NAND flash memory, etc.

[0090] In one implementation, the computer program product can be an intangible product containing a computer program. For example, the computer program product can be implemented as a virtual digital product, such as an executable file, installation package, or other digital file storing the computer program.

[0091] Computer program code can be written in one or more programming languages. The program code can execute entirely on the user's computing device, or partially on the user's computing device, or as a standalone software package, or partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or it can be connected to an external computing device (e.g., through an internet connection provided by a mobile network operator).

[0092] Computer programs can be carried or transmitted via signals such as electricity, magnetism, light, electromagnetic fields, and infrared radiation. Electronic devices can convert the signals carrying computer programs into digital signals, thereby running the computer programs. When a computer program runs on an electronic device, its code is used to cause the electronic device to execute (more specifically, the processor of the electronic device to execute) the method steps of various exemplary embodiments of this disclosure, such as the steps of the cross-domain network resource orchestration method described above.

[0093] Furthermore, in exemplary embodiments of this disclosure, an electronic device capable of implementing the above-described methods is also provided. Those skilled in the art will understand that various aspects of this disclosure can be implemented as systems, methods, or program products. Therefore, various aspects of this disclosure can be specifically implemented as: entirely hardware embodiments, entirely software embodiments (including firmware, microcode, etc.), or embodiments combining hardware and software aspects, collectively referred to herein as "circuit," "module," or "system."

[0094] The following reference Figure 5 To describe an electronic device 500 according to such an embodiment of the present disclosure. Figure 5 The electronic device 500 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0095] like Figure 5 As shown, the electronic device 500 is presented in the form of a general-purpose computing device. The components of the electronic device 500 may include, but are not limited to: at least one processing unit 510, at least one storage unit 520, a bus 530 connecting different system components (including storage unit 520 and processing unit 510), and a display unit 550.

[0096] The storage unit stores program code that can be executed by the processing unit 510, causing the processing unit 510 to perform the steps described in the "Exemplary Methods" section above, according to various exemplary embodiments of this disclosure.

[0097] Storage unit 520 may include readable media in the form of volatile storage units, such as random access memory (RAM) 521 and / or cache memory 522, and may further include read-only memory (ROM) 523.

[0098] Storage unit 520 may also include a program / utility 525 having a set (at least one) program module 525, such program module 525 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0099] Bus 530 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0100] Electronic device 500 can also communicate with one or more external devices 600 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 500, and / or with any device that enables electronic device 500 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 550. Furthermore, electronic device 500 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 560. As shown, network adapter 560 communicates with other modules of electronic device 500 via bus 530. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0101] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0102] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0103] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A method for orchestrating cross-domain network resources, characterized in that, The method, applied to a resource orchestration system including a global coordinator, domain managers registered with the global coordinator for different network domains, and local agents deployed under each domain manager, includes: The global coordinator calculates the state summary information of each network domain based on the attention mechanism to obtain the attention weight of each network domain. The attention weight is used to reflect the current load level of the network domain. The global coordinator determines the cross-domain resource allocation strategy based on the state summary information and attention weights, and allocates the cross-domain resource allocation strategy to the domain manager according to the network domain. For each domain manager, intra-domain decisions are made with sub-policies as constraints to obtain intra-domain resource allocation policies, which are then sent to the corresponding local agents for decision execution. The global coordinator, based on the attention weight, allocates global performance rewards to each domain manager as domain-level rewards, and the domain managers then allocate these domain-level rewards to local agents for decision optimization.

2. The method according to claim 1, characterized in that, The step of calculating the attention weights of each network domain based on the collected state summary information of each network domain through the global coordinator using an attention mechanism includes: The global coordinator generates a global query vector, which is used to characterize the target state of the current global resource allocation. The key vector is determined based on the state summary information of each network domain, and the similarity between the global query vector and each key vector is calculated to obtain the attention score of each network domain. The attention scores are normalized to obtain the attention weights for each network domain.

3. The method according to claim 1, characterized in that, The step of determining the cross-domain resource allocation strategy through the global coordinator based on the state summary information and attention weights includes: Obtain a meta-policy network, which is trained based on historical resource orchestration tasks and is used to adjust resource orchestration strategies; The global coordinator inputs the state summary information and attention weights into the meta-policy network for forward propagation, and outputs the cross-domain resource allocation policy.

4. The method according to claim 1, characterized in that, The step of allocating the cross-domain resource allocation policy to the domain manager according to the network domain includes: The global coordinator divides the cross-domain resource allocation strategy into action strategies for each network domain based on the state summary information and the attention weight. The global coordinator distributes the action policy for each network domain to the corresponding domain manager.

5. The method according to claim 1, characterized in that, For each domain manager, the process involves making intra-domain decisions based on sub-policies to obtain intra-domain resource allocation policies, which are then distributed to the corresponding local agents. This includes: Each domain manager, constrained by its corresponding sub-policy, makes intra-domain decisions based on a multi-agent cooperation algorithm and intra-domain state information to determine the task to be executed and the corresponding target agent. The domain manager distributes the task to be executed to the corresponding target agent.

6. The method according to claim 5, characterized in that, The local agent performs decision-making and execution, including: The local agent obtains the current resource status of the corresponding computing node; Based on the current resource status, execute the received resource allocation strategy.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: The local agent rewards within each network domain are aggregated to obtain the domain-specific rewards, which are used to reflect the performance of the network domain. The global coordinator collects the intra-domain rewards of the network domains reported by each domain manager, and merges the intra-domain rewards according to the attention weight to obtain the global performance reward.

8. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Based on the received domain-level reward, the domain manager performs reinforcement learning on the domain-specific decision-making algorithm. Based on the received local reward, the local decision-making algorithm is updated through the local agent.

9. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Collect state summary information for each of the network domains. The state summary information is obtained by aggregating or summarizing the original state information by the domain manager and / or local agent of the network domain.

10. The method according to any one of claims 1 to 6, characterized in that, The global coordinator, each of the domain managers, and the local agents use different operating time steps; The global coordinator's operating time step is greater than that of each domain manager, and the domain manager's operating time step is greater than that of the deployed local agents.

11. A cross-domain network resource orchestration system, characterized in that, The system includes a global coordinator, domain managers registered with the global coordinator for different network domains, and local agents deployed under each domain manager, wherein: The global coordinator is used to calculate the state summary information of each network domain based on the attention mechanism to obtain the attention weight of each network domain. The attention weight is used to reflect the current load level of the network domain. The global coordinator is also used to determine a cross-domain resource allocation strategy based on the state summary information and attention weights, and to allocate the cross-domain resource allocation strategy to the domain manager according to the network domain. The domain manager is used to make decisions within the domain based on sub-policies, obtain the resource allocation policy within the domain, and send it to the corresponding local agent for decision execution. The global coordinator is further configured to allocate global performance rewards as domain-level rewards to each of the domain managers based on the attention weights, and the domain managers are further configured to allocate the domain-level rewards as local rewards to local agents for decision optimization.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 10.

13. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 10 by executing the executable instructions.

Images