Network resource allocation method, electronic device, and storage medium

Abstract

The application discloses a network resource allocation method, an electronic device and a storage medium. The method comprises the following steps: taking current incentive parameters distributed by a global decision model in a cloud platform as a weighted item of a reward function of a domain-level decision model, so as to control the decision model in a single domain to achieve the optimal local network resource while taking into account the optimal global network performance of the cross-domain network; quantifying the global benefit index after each candidate operation is performed according to the global decision model, so as to determine a global resource allocation strategy; and finally adjusting the network resource allocation state of each data center by a network controller deployed in each data center, wherein the domain-level decision model is deployed in multiple data centers in different regions. The method solves the technical problem that the cross-domain network resources in the cloud environment cannot be reasonably allocated in the related art, and achieves the technical effect of intelligently and cooperatively allocating the local network resources in the domain and the global network resources.

Description

Technical Field

[0001] This application relates to the field of computers, and more particularly to a method for allocating network resources, an electronic device, and a storage medium. Background Technology

[0002] With the rapid development of cloud computing, big data, and artificial intelligence technologies, IT infrastructure is evolving from traditional physical servers to large-scale, virtualized, and distributed private and hybrid cloud architectures. The resulting deployment of business applications exhibits significant characteristics of being cross-regional, cross-data center, or cross-virtual network. This distributed nature poses unprecedented challenges to the cross-domain networks carrying business traffic.

[0003] In related technologies, traditional traffic engineering-based cross-domain path calculation methods are commonly used to calculate the optimal static path for cross-domain traffic and then configure these paths on network devices; alternatively, passive responses based on local thresholds (such as triggering a fixed response action to switch some traffic to backup links when the utilization rate of outbound links within a single domain exceeds a threshold) are used to solve local network problems. However, these methods only focus on solving local network resource problems and do not consider the global optimal goal of cross-domain networks. Therefore, this results in the technical problem of being unable to reasonably allocate cross-domain network resources in a cloud environment. Summary of the Invention

[0004] This application provides a network resource allocation method, electronic device, and storage medium to at least solve the technical problem in related technologies that cannot reasonably allocate cross-domain network resources in a cloud environment. According to one aspect of an embodiment of this application, a network resource allocation method is provided, comprising: acquiring network status information of multiple data centers distributed in different regions; when the network status information indicates that the current network resources of a target data center are abnormal during the processing of cross-domain services, determining a candidate operation list for optimizing the reward of a domain-level decision model, wherein the domain-level decision model is used to determine the allocation strategy of local network resources in each data center; based on a pre-deployed global decision model, determining a global benefit index after executing each candidate operation in the candidate operation list, wherein the current incentive parameter issued by the global decision model serves as a weighting term of the reward function of the domain-level decision model, and the current incentive parameter is used to adjust the local resource allocation strategy of the domain-level decision model to ensure that it provides a positive contribution to the optimal allocation of global resources; and based on the global benefit index, determining a global resource allocation strategy and adjusting the network resource allocation status of each data center through a network controller deployed in each data center.

[0005] According to another aspect of the embodiments of this application, a network resource allocation apparatus is also provided, comprising: a first acquisition unit, configured to acquire network status information of multiple data centers distributed in different regions; a first processing unit, configured to determine a candidate operation list for optimizing the reward of a domain-level decision model when the network status information indicates that the current network resources of a target data center are abnormal during the processing of cross-domain services, wherein the domain-level decision model is used to determine the allocation strategy of local network resources of each data center; a second processing unit, configured to determine a global benefit index after executing each candidate operation in the candidate operation list based on a pre-deployed global decision model, wherein the current incentive parameter issued by the global decision model is used as a weighting term of the reward function of the domain-level decision model, and the current incentive parameter is used to adjust the local resource allocation strategy of the domain-level decision model to ensure that it provides a positive contribution to the optimal allocation of global resources; and a third processing unit, configured to determine a global resource allocation strategy based on the global benefit index, and adjust the network resource allocation status of each data center through a network controller deployed in each data center.

[0006] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the steps of any of the above-described network resource allocation methods through the computer program.

[0007] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the steps of any of the above-described network resource allocation methods when running.

[0008] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the network resource allocation methods described above.

[0009] Using the embodiments provided in this application, in the process of executing cross-domain services using cross-domain networks of multiple data centers in a cloud environment, if the current network resources of the target data center are abnormal, the domain-level decision model, guided by the current incentive parameters issued by the global decision model, first outputs a list of candidate operations that are beneficial to its own reward generation and can also make a positive contribution to the optimal allocation of global resources. Then, the global decision model calculates the global benefit after executing each candidate operation and generates a global resource allocation strategy based on the global benefit index. In other words, through the collaborative interaction between the domain-level decision model that controls the network resources of each data center and the global decision model that coordinates the global network resources, a network resource allocation strategy that can solve local network anomalies and improve global network performance is generated. This solves the technical problem in related technologies that cannot reasonably allocate cross-domain network resources in a cloud environment, realizes multi-objective optimization of local network resources within the domain and global network resources, and fundamentally improves the intelligence level of cross-domain networks in a cloud environment. Attached Figure Description

[0010] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram illustrating an application scenario of a network resource allocation method according to an embodiment of this application.

[0012] Figure 2 This is a flowchart of an optional network resource allocation method according to an embodiment of this application.

[0013] Figure 3 This is a schematic diagram of the overall architecture of an optional network resource allocation method according to an embodiment of this application.

[0014] Figure 4 This is a specific example of an optional network resource allocation method according to an embodiment of this application.

[0015] Figure 5 This is a structural block diagram of an optional network resource allocation device according to an embodiment of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0017] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0018] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] According to one aspect of the embodiments of this application, a network resource allocation method is provided. Optionally, in this embodiment, the above-described network resource allocation method may be applied to, but is not limited to, [examples of applications]. Figure 1 In the hardware scenario shown, the server device may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The server device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the server equipment described above. For example, the server equipment may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0020] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the network resource allocation method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to server devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0021] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the server device. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0022] The network resource allocation method of this application embodiment can be executed by a server device, or by a server device in combination with at least one of a terminal device (which can also be understood as input / output device 108). The terminal device executing the network resource allocation method of this application embodiment can also be executed by a client installed on it.

[0023] Before explaining the technical solution of this application, let's first briefly introduce the problems existing in the current cloud environment.

[0024] In today's cloud environment, enterprises typically build multiple geographically dispersed or logically isolated data centers, forming multiple independent network management domains. As businesses evolve towards cloud-native and microservice architectures, individual business requests or data flows often need to traverse multiple domains to complete. For example, a front-end application might be deployed in data center 1 of region 1, its dependent database service in data center 2 of region 2, and its backup service in data center 3 of region 3. The scheduling and management of this cross-domain traffic faces the following core challenges.

[0025] (1) The contradiction between locality and globality in resource allocation: Each domain's SDN controller usually only has a complete network view and control over its own domain, and its decision-making is essentially local and self-interested. When multiple domain controllers independently adjust routing or bandwidth, it may lead to congestion of cross-domain boundary links or the formation of suboptimal end-to-end paths, that is, local optimum is not equal to global optimum.

[0026] (2) Complex constraints with multiple dimensions and objectives: Cross-domain resource allocation must not only consider network performance indicators such as bandwidth, latency, and packet loss rate, but also meet the SLA requirements of different tenants and different services. For example, transaction services require ultra-low latency, video conferencing requires stable bandwidth, while big data analysis requires high throughput. In addition, it is also necessary to take into account multiple, and even conflicting, constraints such as link cost, equipment energy consumption, and security isolation strategies.

[0027] (3) The dynamic and uncertain nature of network status: Traffic patterns in the cloud environment are highly dynamic and sudden. The tidal effect of business load, the dynamic migration of virtual machines or containers, and the sudden failure of network devices can all lead to rapid changes in network status. Traditional adjustment methods based on static configuration or threshold rules are slow to respond and difficult to adapt to this high degree of uncertainty.

[0028] The technical solution of this application is intended to solve the above problems by introducing an intelligent learning method with forward-looking decision-making capabilities, which transforms the complex and dynamic cross-domain resource allocation problem into a modelable, solvable, and optimizable multi-agent collaborative decision-making problem.

[0029] Based on the above analysis, it is clear that the technical solutions in this application embodiment can be applied, but are not limited to, to network resource management scenarios in various complex cloud environments, aiming to improve the global optimization capability and intelligence level of network resources. Specific examples of several common application scenarios are given below.

[0030] (1) Private cloud with unified management of multiple data centers: Enterprises own multiple self-built data centers and manage computing, storage and network resources in a unified manner through a cloud platform. The technical solution of this application can coordinate and schedule SDN network resources between data centers to provide high-performance network connections across regions for upper-layer virtual machines, containers and other devices.

[0031] (2) Hybrid Cloud and Multi-Cloud Interconnection: Enterprises use private clouds and multiple public clouds simultaneously, or use multiple public cloud services simultaneously. The technical solution of this application can be deployed in a cloud management platform to coordinate dedicated line or VPN connection resources between private clouds and public clouds, or between different public cloud VPCs, to optimize the access experience of hybrid cloud services.

[0032] (3) Multi-tenant / multi-VPC network isolation and interconnection within the cloud: In a single region of a large cloud platform, there are hundreds or thousands of isolated tenant networks. When these VPCs need to interconnect on demand, the technical solution of this application can intelligently schedule the backbone network resources that carry these interconnection traffic, ensuring the quality of interconnection and achieving fair allocation.

[0033] In these scenarios, the technical solution of this application needs to address the following typical constraints.

[0034] (1) Multi-tenancy and Service Level Agreement (SLA): It is necessary to ensure resource isolation for different tenants and meet the SLA indicators agreed in their contracts, such as ensuring that the cross-domain latency of tenant A's critical business is no more than 50 milliseconds, or ensuring that tenant B's video conferencing bandwidth is no less than 100Mbps.

[0035] (2) Security and compliance policies: The generated network policies (such as routing paths) must comply with the preset security group rules, access control lists and data compliance requirements (for example, some data may not be allowed to flow out of a specific geographical area).

[0036] (3) Physical and virtual link capacity: Any bandwidth allocation or traffic scheduling decision must not exceed the capacity limit of the underlying physical or virtual link.

[0037] (4) Cost and energy consumption: When selecting a path or allocating resources, it is necessary to consider the usage cost of different links (such as public network links charged by traffic) and the energy consumption of network equipment, and tend to choose the more cost-effective or greener solutions.

[0038] (5) Business priority: It is necessary to identify and prioritize the quality of network services for critical businesses (such as online transactions and core database synchronization), and ensure their performance even when the network is congested.

[0039] The following example uses a server executing the network resource allocation method in this embodiment. Figure 2 This is a flowchart illustrating an optional network resource allocation method according to an embodiment of this application, such as... Figure 2 As shown, the process of this method may include steps S202 to S208.

[0040] Step S202: Obtain network status information of multiple data centers distributed in different regions.

[0041] Step S204: When the network status information indicates that the current network resources of the target data center are abnormal during the processing of cross-domain services, a list of candidate operations for optimizing the reward of the domain-level decision model itself is determined, wherein the domain-level decision model is used to determine the allocation strategy of local network resources of each data center.

[0042] Step S206: Based on the pre-deployed global decision model, determine the global benefit index after executing each candidate operation in the candidate operation list. The current incentive parameter issued by the global decision model is used as a weighting term of the reward function of the domain-level decision model. The current incentive parameter is used to adjust the local resource allocation strategy of the domain-level decision model to ensure that it makes a positive contribution to the optimal allocation of global resources.

[0043] The global benefit metric can be used, but is not limited to, to quantify the improvement in overall network performance across domains after executing candidate operations. Cross-domain refers to, but is not limited to, SDN network domains deployed in multiple data centers in different regions; that is, the allocation or optimization of network resources between local networks in two or more data centers can be understood as the allocation or optimization of resources across domains.

[0044] Step S208: Based on the global revenue metric, determine the global resource allocation strategy and adjust the network resource allocation status of the network controllers deployed in each data center.

[0045] To facilitate understanding, the technical terms involved in the embodiments of this application will be briefly described first.

[0046] SDN (Software-Defined Networking): An architecture that separates the network control plane from the data plane, enabling flexible control and centralized management of network traffic.

[0047] SLA (Service-Level Agreement): A formal commitment between a service provider and a customer regarding service quality, performance, liability, and other related matters.

[0048] VPC (Virtual Private Cloud): A logically isolated network space provided to a single tenant in a public cloud environment.

[0049] API (Application Programming Interface): A set of rules and protocols that allow different software components to interact and exchange data.

[0050] QoS (Quality of Service): Used to measure the degree to which network services meet user expectations, involving metrics such as bandwidth, latency, jitter, and packet loss rate.

[0051] OVS (Open vSwitch): A multi-layer virtual switch widely used in virtualization environments, supporting programmable protocols such as OpenFlow.

[0052] MADDPG (Multi-Agent Deep Deterministic Policy Gradient): A reinforcement learning algorithm suitable for multi-agent environments, employing a centralized training and distributed execution model.

[0053] CTDE (Centralized Training with Decentralized Execution): A multi-agent learning framework that utilizes global information during training but relies only on local observations during execution.

[0054] TE (Traffic Engineering): The process of analyzing, predicting, and managing data flows in a network to optimize resource utilization and network performance.

[0055] The core of this application's technical solution lies in providing a method, system, and device for cross-domain SDN resource collaborative allocation and performance optimization driven by multi-agent game-theoretic reinforcement learning. This aims to solve the challenges of software-defined network resource allocation across regions, data centers, and virtual networks in existing private and multi-cloud environments. It primarily achieves global collaborative optimization and dynamic adaptive adjustment of cross-domain network resources while satisfying multiple constraints such as multi-tenant service level agreements (SLAs), cost, and energy consumption. This improves end-to-end service performance, resource utilization, and operational automation.

[0056] To achieve the above objectives, each independent SDN network domain (e.g., a data center network, a VPC, or an availability zone network) is abstracted as an intelligent agent (which can also be understood as a decision-making model) with autonomous decision-making capabilities, i.e., a domain-level intelligent agent or a domain-level decision-making model. Simultaneously, a global coordinating intelligent agent (which can also be understood as a global decision-making model) is established to guide and coordinate the behavior of all domain-level intelligent agents from a global perspective. By constructing a game theory model among multiple intelligent agents and using reinforcement learning to train and solve this model, each domain, while pursuing its own interests (such as reducing congestion within its domain and meeting its tenant SLAs), can align with global optimization goals (such as ensuring cross-domain end-to-end latency for critical services, maximizing overall network throughput, and reducing overall energy consumption), ultimately forming a dynamic, closed-loop, and adaptive cross-domain network resource management and performance assurance system.

[0057] To facilitate understanding, let's first combine... Figure 3 The overall architecture diagram shown provides a brief overview of the network resource allocation method described above. Among other things, Figure 3The system architecture shown serves as the core network intelligent component of the cloud platform, and is tightly integrated with other functional modules of the cloud platform (such as computing and storage management) as well as the underlying SDN infrastructure.

[0058] like Figure 3 As shown, the overall architecture of the technical solution in this application includes a cloud management / virtualization integration interface layer, a performance monitoring and verification feedback layer, a multi-agent game reinforcement learning decision layer, a cross-domain SDN control and resource abstraction layer, and an SDN network infrastructure layer.

[0059] Among them, the cross-domain SDN control and resource abstraction layer is the bridge connecting the upper-layer intelligent decision-making and the lower-layer network infrastructure. It is responsible for shielding the heterogeneity of the lower layer and providing a unified and standardized resource view.

[0060] The intra-domain SDN controller cannot replace the existing intra-domain SDN controller (e.g., a controller implemented using a general framework), but rather acts as an execution agent. Each domain controller is responsible for specific network management tasks within its domain, including but not limited to performing intra-domain topology discovery using technologies such as link-layer discovery protocols; generating and distributing flow table rules to network devices (such as switches supporting the OpenFlow protocol) according to upper-layer instructions; and continuously monitoring intra-domain link status, port traffic, and device health through simple network management protocols, flow data export, and other methods.

[0061] The cross-domain resource aggregation and abstraction module is a key foundational component of the technical solution in this application. It communicates with the SDN controllers of each domain through the southbound interface and collects the following information periodically or when triggered by events (such as topology changes).

[0062] (1) Topology information: switches, routers, ports and their connections within each domain; connection points (i.e. border gateways or switches) across domain boundaries and their counterpart information.

[0063] (2) Resource information: static bandwidth capacity, current load (utilization), queue configuration and depth of each link.

[0064] (3) Traffic information: the traffic matrix of key business or tenant, that is, the amount of traffic from the source to the target.

[0065] (4) Performance information: Real-time performance data such as latency and packet loss rate provided by the monitoring module.

[0066] After collecting the aforementioned raw data, this module processes and abstracts it to construct a globally unified cross-domain virtual network resource graph (which can also be understood as a cross-domain network resource graph). Nodes in this resource graph can represent switches, hosts, or aggregation points of an entire domain; edges represent physical or logical links; and edge weights can be multi-dimensional vectors containing various attributes such as bandwidth, latency, cost, and SLA fulfillment. This standardized resource graph forms the basis for upper-layer multi-agent environmental perception and decision-making.

[0067] The decision layer of multi-agent game reinforcement learning is mainly responsible for intelligent strategies for cross-domain resource collaborative allocation, and mainly includes domain-level agents and global coordination agents.

[0068] In this system, a domain-level agent is instantiated for each SDN domain. Each agent is an independent computational process or service that receives local state information centered on its own domain from the resource abstraction layer, as well as some cross-domain information (such as incentive signals and boundary link congestion warnings) distributed by the global coordinating agent. Its core responsibility is to output candidate actions for its own domain based on this input information and its own reinforcement learning model. These actions may include, but are not limited to, suggesting video traffic from domain A to domain B, switching from egress 1 to egress 2, requesting 2Gbps bandwidth to reserve for cross-domain backup traffic of high-priority tenants within the domain, and adjusting the weights of specific queues on the core switches within the domain. The goal of the domain-level agent is to maximize its own reward, which is designed to include both the performance metrics of its own domain and its contribution to the global objective.

[0069] The Global Coordinator Agent is the "brain" of the entire system. It does not directly control any network devices, but rather coordinates the behavior of all domain-level agents. It receives candidate actions and state summaries reported by all domain-level agents and possesses a complete cross-domain network resource graph. Its core responsibilities include executing game theory solutions, calculating payoffs and incentive parameters, and generating cooperative schemes. The following will describe each step with specific examples.

[0070] In addition, the multi-agent game reinforcement learning decision layer includes a policy storage and version management module. This module is a persistent storage system used to record and manage the policy models of all agents (including domain-level and global). It saves different versions of each policy model, performance evaluation data during training, and records of the effects after deployment in a real environment. This enables the system to support multiple tests of the policy, rapid rollback when the effect is unsatisfactory, and auditing and analysis of the policy evolution process.

[0071] The performance monitoring and verification feedback layer constitutes a closed-loop feedback for intelligent decision-making, ensuring the effectiveness and security of the strategy, and specifically includes the following:

[0072] (1) Online monitoring module: This module is responsible for end-to-end, real-time measurement of network performance. It uses a combination of various technical means.

[0073] 1) Active Probe: Periodically inject synthetic probe packets into the network (e.g., using techniques similar to In-band Network Telemetry, INT, carrying path and timestamp information in the packet header) to accurately measure end-to-end latency, jitter, and path information between any two network nodes.

[0074] 2) Passive data collection: Analyze the streaming data exported from network devices and statistically analyze metrics such as throughput, packet loss rate, and TCP retransmission of real business traffic.

[0075] 3) SLA monitoring: For specific SLA policies defined in the cloud management platform, continuously track whether their key performance indicators meet the standards and record default events.

[0076] All collected data is formatted as time-series data with timestamps and stored in a dedicated time-series database for querying and analysis by other modules.

[0077] (2) Performance evaluation and verification module: This module provides security for the deployment of the strategy.

[0078] 1) Offline / Pre-evaluation: After the global coordinating agent generates a new collaborative policy, the policy is not immediately deployed to the network. Instead, it is first pre-rendered by the module in a digital twin or simulation environment. A digital twin is a virtual copy that closely resembles the real network topology, configuration, and traffic patterns. By executing the new policy in the twin environment, its potential impact can be predicted, such as whether it will cause new congestion points or lead to SLA breaches for certain services. Only policies that pass the pre-evaluation can proceed to the release phase.

[0079] 2) Online evaluation: After the strategy is deployed, this module will continuously compare the actual performance data collected by the online monitoring module with the expected performance target of the strategy to evaluate the actual effect of the strategy.

[0080] (3) Anomaly Alarm and Policy Review Module: This module will trigger an alarm when online monitoring data exceeds the preset normal threshold (for example, the utilization rate of a cross-domain link exceeds 90% for 5 consecutive minutes), or when the performance evaluation module finds that the actual performance is seriously inconsistent with the expectations. The alarm information will notify the operation and maintenance personnel and automatically trigger the policy re-review process of the decision-making layer, prompting the intelligent agent system to respond quickly to the current network conditions.

[0081] The cloud management / virtualization integration interface layer is responsible for seamlessly connecting the intelligent network capabilities in this application embodiment to the upper-layer cloud platform and the underlying infrastructure.

[0082] (1) Northbound Interface: In the form of RESTful API or gRPC, a set of advanced network policy orchestration capabilities are exposed to the cloud platform. Cloud platform administrators or tenants can define their business requirements through graphical interfaces or declarative configurations without having to worry about the complex implementation of the underlying network. For example, APIs can be provided to "create a cross-domain network service with low latency guarantees from region 1 to region 2", or "apply a financial-grade QoS template for my online payment business line".

[0083] (2) Southbound Interface: Responsible for translating the abstract policies output by the global coordinating agent into specific configuration instructions that can be understood by the underlying SDN controller and virtual switches (such as Open vSwitch, OVS). For example, an abstract policy for adjusting the path will be translated into a series of specific OpenFlow flow table rules and distributed to the switches along the path; a bandwidth allocation policy will be translated into rate limiting and shaping configurations for specific ports or queues.

[0084] (3) Collaboration with compute / storage orchestration interfaces: This application is not limited to network resource optimization. By collaborating with the cloud platform's compute scheduling module (responsible for the creation and migration of virtual machines / containers), deeper compute-network collaboration can be achieved. For example, when high network latency is detected for a critical application, the system can not only try to find a better network path, but also collaborate with the compute scheduling module to migrate the application's microservice instances to a data center closer to its dependent services, thereby fundamentally solving the latency problem. This joint decision-making capability is an important extension of the technical solution of this application.

[0085] In the specific process of network resource allocation and optimization, the domain-level intelligent agents (intelligent decision-making models or domain-level decision-making models) and the global coordination intelligent agents (global decision-making models) deployed in each data center collect network status information in real time by communicating with the SDN controller inside the data center. For example, the domain-level intelligent agent in data center 1 will obtain detailed data such as switch status, link utilization, and queue depth within its managed domain.

[0086] When an anomaly is detected in network resources within the SDN network domain corresponding to a data center, such as link congestion, the domain-level agent will generate a series of possible candidate operations based on its own reward function and the current network state. These operations aim to optimize the allocation of network resources in the domain, while also considering their contribution to the global optimization objective. For example, in the embodiment of data center 1, the domain-level decision model may generate the following candidate operations.

[0087] Action A1: Switch some critical business traffic from the congested outbound link to another backup link.

[0088] Action A2: Adjust the queue scheduling strategy of critical egress links to prioritize high-priority traffic.

[0089] After receiving candidate operations submitted by all domain-level agents, the global decision-making model uses its built-in game theory model (such as the Stackelberg game model) to evaluate the impact of each operation on global resource allocation and network performance. It calculates the global payoff metric for each candidate operation as the core standard for measuring whether the operation is beneficial to the overall cross-domain network performance.

[0090] Based on the global benefit metrics of all candidate operations, the global decision model will select one or more optimal operations to generate a global resource allocation strategy. These strategies include, but are not limited to, reallocation of link bandwidth, adjustment of routing paths, and modification of Quality of Service (QoS) parameters. After determining the global resource allocation scheme, the global resource allocation strategy will be translated into specific configuration instructions that can be understood by each domain-level SDN controller via the southbound interface. The configuration instructions will be sent to the SDN controllers of the relevant data centers, whereby the controllers will execute these strategies, adjust the state of network devices, and complete the resource reallocation.

[0091] Based on the above analysis, the core of this application's technical solution lies in its use of a multi-agent game-theoretic reinforcement learning framework to automate and intelligently optimize cross-domain SDN resource allocation and performance in private and multi-cloud environments. It not only optimizes local resources within a single data center but, more importantly, ensures globally optimal resource allocation across multiple data centers through the participation of a globally coordinating agent. This also considers multiple constraints such as service level agreements, cost, and energy consumption, thereby improving the overall performance and operational efficiency of the cross-domain network.

[0092] In an exemplary embodiment, when the network status information indicates that the current network resources of the target data center are abnormal during the processing of cross-domain services, the above-mentioned method of determining a list of candidate operations for optimizing the domain-level decision model's own reward includes: obtaining the target network status information of the target data center, wherein the target network status information includes network resource distribution and link status data; and obtaining a list of candidate operations that can handle the abnormality and improve its own cumulative reward by inputting the target network status information into the target domain-level decision model pre-deployed in the target data center.

[0093] In multi-agent systems, each domain-level agent needs to observe not only the local network status within its own domain but also the network status of the target data center it interacts with, in order to make more accurate and globally optimized decisions. This information acquisition process involves multiple layers of communication and data processing.

[0094] (1) Real-time communication and data synchronization: Domain-level agents communicate with the target SDN controller in real time through a standardized southbound interface to obtain the latest network resource distribution (such as the availability and status of links, nodes, queues, etc.) and link status data (such as utilization, latency, packet loss rate, etc.) of the target data center.

[0095] (2) Information aggregation and abstraction: With the assistance of the resource aggregation and abstraction module, the domain-level agent processes and abstracts the acquired raw network state data to form network resource description information including but not limited to the topology of the target domain, the performance indicators of key links, and the current traffic load.

[0096] (3) Fusion of global and local information: The domain-level agent fuses the target network state information with its own observed local state information, and the fused information provides rich context for the output of the target domain-level decision model.

[0097] The target domain-level decision model predicts and outputs a series of candidate operations based on the input target network state information. These operations aim to resolve detected anomalies while increasing the cumulative reward of the domain-level agent.

[0098] The above solution enables precise perception and intelligent decision-making of the target data center, which can not only resolve local network anomalies, but also promote global network optimization, improve the dynamic adjustment capability of network resources and the SLA guarantee level of cross-domain services.

[0099] In an exemplary embodiment, the above-mentioned method of inputting target network state information into a target domain-level decision model pre-deployed in a target data center to obtain a candidate operation list that can handle anomalies and improve its own cumulative reward includes: inputting target network state information into a reinforcement learning model embedded in the target domain-level decision model, and combining the target incentive parameters in the reward function corresponding to the reinforcement learning model to output a candidate operation list that takes into account both improving the cumulative reward of the reinforcement learning model itself and satisfying the cross-domain network global resource optimization objective.

[0100] The input of target network state information is crucial when an agent makes decisions. This input ensures that the target domain-level agent can make decisions based on the latest network resource distribution and link state data of the target data center. This target network state information includes, but is not limited to, the target domain's topology, link load, queue status, traffic matrix, and SLA compliance. By inputting this information into a reinforcement learning model, the target domain-level agent gains a comprehensive understanding of the target network domain's health status, laying the foundation for generating forward-looking and global decisions.

[0101] In the process of reinforcement learning models generating operation suggestions (which can also be understood as candidate operations in a candidate operation list), the reward function plays a decisive role. It not only includes local reward metrics related to the target domain-level agent itself (such as link utilization, SLA satisfaction, cost, etc.), but also integrates the target incentive parameters issued by the global coordinating agent (i.e., the global decision-making model). This allows the target domain-level agent to pursue the goal of improving its own cumulative reward while also taking into account the goal of optimizing global network resources.

[0102] The target incentive parameters can be, but are not limited to, parameters generated by the global coordinating agent based on pre-set constraints (such as SLA guarantees for high-priority services), guiding directions (such as load balancing and cost savings), or penalty items (such as penalties for using high-cost links). Through the target incentive parameters, the global agent can macroscopically regulate the decisions of domain-level agents to ensure that their behavior conforms to the global optimization strategy.

[0103] By employing the above methods, not only can the domain-level intelligent agent make optimization decisions based on the target network state information, but also the design of the reward function and target incentive parameters achieves the unification of local optimization and global objectives, effectively improving the efficiency of collaborative allocation of network resources and the level of performance assurance in the cloud environment.

[0104] In an exemplary embodiment, before determining the candidate operation list for optimizing the reward of the domain-level decision model itself, the method further includes: determining the cross-domain global network state based on network state information; obtaining the prediction incentive parameters for the predicted operations output by each domain-level decision model in the current decision cycle by inputting the cross-domain global network state into the global decision model; obtaining the domain-level decision set in the current decision cycle by inputting the local network state information of multiple data centers into the domain-level decision model, wherein the domain-level decision set includes the predicted domain-level operations output by the domain-level decision model; and determining the current incentive parameters based on the domain-level decision set, the prediction incentive parameters, and the historical decision results in the previous decision cycle.

[0105] Determining the global network state is the foundation for the global coordinating agent to make decisions. It needs to obtain network state information within its domain from various domain-level agents, including but not limited to topology, link utilization, queue status, traffic patterns, and SLA satisfaction.

[0106] The global decision-making model, based on the cross-domain global network state, predicts and outputs the predicted incentive parameters that each domain-level decision-making model might obtain after taking certain actions within the current decision-making cycle. These parameters aim to guide domain-level agents to make decisions that benefit global optimization, rather than simply pursuing local interests. The global decision-making model generates a set of incentive parameters by evaluating the impact of various possible actions on global objectives (such as SLA satisfaction, cost, and energy consumption). These parameters are then fed back to each domain-level decision-making model to adjust its resource allocation strategy.

[0107] Upon receiving the predicted incentive parameters from the global coordinating agent, the domain-level decision-making models in each data center begin making decisions based on their own local network state information. This process involves the model's assessment of the environmental state, prediction of possible actions, and response to the predicted incentive parameters. The domain-level decision-making model will consider factors such as the utilization of local network resources, business needs, and SLA constraints to generate a set of predicted actions that reflects the agent's policy inclination within the current decision-making cycle.

[0108] After receiving the prediction operations from the domain-level agent, the global coordinating agent determines the current incentive parameters based on the set of prediction operations, the prediction incentive parameters, and historical decision results. This process includes evaluating the benefits of the prediction operations, analyzing historical decision results, and considering policy stability and SLA satisfaction to ensure that the generated incentive parameters guide the agent system toward the global optimization goal while avoiding potential policy conflicts and performance degradation.

[0109] The above process demonstrates the dynamic adjustment capability and global optimization potential of multi-agent game reinforcement learning systems. Through continuous closed-loop feedback and intelligent decision-making, it solves the complex challenges in cross-domain network resource allocation and improves the efficiency of network resource utilization and business performance in the cloud environment.

[0110] In an exemplary embodiment, determining the current incentive parameters based on the domain-level decision set, predicted incentive parameters, and historical decision results from the previous decision cycle includes: summarizing the domain-level decision set to obtain global decisions, wherein global decisions include global operations executed within the cross-domain network; using a global objective function to predict the global performance increment gain of the cross-domain network after executing the global decisions, wherein the global objective function is obtained by weighted summation of service compliance satisfaction, resource utilization, and network resource consumption costs of key services; determining the marginal contribution of each domain-level decision model based on historical decision results and global performance increment gains, wherein the marginal contribution is used to measure the incremental impact of the domain-level operation corresponding to the domain-level decision on the global network performance; and adjusting the predicted incentive parameters based on the marginal contribution to obtain the current incentive parameters, wherein the greater the contribution of the domain-level decision output by a domain-level decision model to the improvement of global performance, the greater the current incentive obtained.

[0111] In this embodiment, a game theory model is mainly used to allocate global payoffs, thereby determining the current incentive parameters that are distributed from the global decision model to the domain-level decision model. The specific game process is described below.

[0112] (1) Leader Action: At the beginning of each decision cycle, the global coordinating agent (Leader) takes the first action. Based on the global state S_global, it predicts the possible reactions of each follower, and then formulates and publishes a set of "rules of the game" that are optimal for all followers, namely the global action A_global. This action is mainly reflected in the pricing or incentive parameters for the use of cross-domain resources (such as bandwidth) in each domain. For example, for a congested link, it will increase its usage price.

[0113] (2) Follower response: After each domain-level agent (Follower) perceives the rules (incentive parameters) announced by the leader, it treats them as part of its own environment. Then, each domain-level agent makes independent and simultaneous decisions to maximize its own reward function (which already includes the price / incentive set by the leader).

[0114] (3) Benefit Calculation and Allocation: After all domain-level agents report their decisions, the global coordinating agent calculates the global performance improvement based on these decisions and allocates a portion of the global benefit as an incentive parameter R_global_incentive to each domain according to the principle of marginal contribution. The greater the contribution of a domain's decision to the global performance improvement (e.g., actively avoiding congested links to make way for higher-priority services), the more incentive it receives. This mechanism encourages domain-level agents to make altruistic and self-interested decisions.

[0115] In this embodiment, a core process for generating global decisions and adjusting incentive parameters in a multi-agent system is constructed. The global coordinating agent first aggregates the decision sets of all domain-level agents to generate a global decision scheme; then, it uses a global objective function to predict the incremental performance gain of the cross-domain network after executing the scheme; by analyzing historical decision results and current global performance gains, it determines the marginal contribution of each agent; finally, it adjusts the predicted incentive parameters based on the marginal contributions to generate the current incentive parameters, which are used to guide the agent system to make the next round of collaborative decisions, forming a closed loop of dynamically adjusted and adaptively optimized network resource management.

[0116] The above methods promote the collaborative optimization of multi-agent systems in cross-domain network resource allocation, realize the leap from local intelligence to global intelligence, and provide intelligent solutions for network management in private cloud, hybrid cloud and multi-cloud environments.

[0117] In another optional embodiment, the above-described resource collaborative allocation workflow includes, but is not limited to, a training phase and an online inference phase. The processing procedures for each of the two phases are described below.

[0118] In an offline environment, the domain-level decision model and the global decision model are trained through the following steps.

[0119] S11, Data Acquisition and Preparation.

[0120] The cloud platform continuously collects historical topology, traffic, and performance data from various domains of the production network through its monitoring system. This data is then cleaned, labeled, and stored in a dedicated training set.

[0121] S12, Simulation Environment Construction and Update.

[0122] Based on training samples in the training set, the system automatically or semi-automatically constructs and calibrates a high-fidelity cross-domain network digital twin environment. This environment can simulate the behavior of a real network and serves as the foundation for the agent to learn through trial and error.

[0123] S13, iterative training of the model.

[0124] S13-1, The training controller samples a scenario (e.g., traffic patterns and network status during a historical time period) from the training set.

[0125] S13-2, Reset the scene to its initial state in the digital twin environment.

[0126] S13-3, the multi-agent system begins to interact with the environment.

[0127] At each time step, each agent observes the state, performs an action, and the environment returns a reward and a new state. These experience tuples (state, action, reward, new state) are stored in the experience replay pool.

[0128] S13-4, the training controller periodically draws a batch of experiences randomly from the experience replay pool to update the Actor and Critic network parameters of all agents.

[0129] S13-5, Repeat steps S13-3 and S13-4 until the policy models of each agent converge or reach the preset number of training rounds.

[0130] S14, Strategy Evaluation and Screening.

[0131] After training, the generated policy model is evaluated under standard testing scenarios. Evaluation metrics include SLA satisfaction rate, average latency, throughput, and fairness index. Only policy versions that perform excellently and stably across all metrics are marked as "production-ready" and stored in the policy storage and version management module.

[0132] For the online inference stage, it can be run in real time in a production environment, but is not limited to, using a pre-trained model to make decisions. The specific steps are as follows.

[0133] S21, Real-time Status Awareness.

[0134] S21-1, the online monitoring module continuously collects real-time performance data from the entire network.

[0135] S21-2, the resource aggregation and abstraction module obtains the latest topology and configuration information from each domain controller.

[0136] S21-3, this information is integrated into the real-time domain-level state S_local and the global state S_global.

[0137] S22, Distributed Decision Making and Collaborative Computing.

[0138] In S22-1, each domain-level agent loads the "production-ready" Actor model, receives the latest S_local and the global stimulus from the previous round, instantly calculates the candidate action A_local for its domain, and reports it to the global coordinating agent.

[0139] In S22-2, after receiving candidate actions from all domains, the global coordinating agent loads its Leader model, combines it with the global state S_global, performs game theory to solve the problem, and calculates the final collaborative resource allocation scheme and the incentive parameters for the next round.

[0140] S23, Strategy issuance and execution.

[0141] S23-1, the generated collaborative solution is first sent to the performance evaluation and verification module for rapid pre-testing.

[0142] S23-2 If the pre-rehearsal passes, the southbound interface module will translate the abstract collaboration scheme into specific configuration instructions.

[0143] S23-3 These instructions are distributed to the SDN controllers of each domain, and the controllers ultimately modify the configurations of flow tables, queues, etc. on the network devices.

[0144] S23-4 At the same time, the network orchestration view of the cloud management platform will also be updated to reflect the new network status and paths.

[0145] The entire online inference process is designed to be executed periodically (e.g., every 5 minutes) or event-driven (e.g., when a congestion alert is detected) to ensure timely decision-making.

[0146] In an exemplary embodiment, the above-mentioned determination of cross-domain global network status based on network status information includes: determining the topology, traffic matrix, and bandwidth capacity of network devices in multiple data centers based on network status information; abstracting the topology, traffic matrix, and bandwidth capacity of network links into a cross-domain network resource graph, wherein the cross-domain network resource graph is used to characterize the cross-domain global network status, a node in the cross-domain network resource graph represents a physical network device or virtual network device in a data center, and an edge in the cross-domain network resource graph represents a physical link or logical link for transmitting network resources.

[0147] As can be seen from the description in the above embodiments, cross-domain global network state can be implemented, but is not limited to, using a cross-domain SDN control and resource abstraction layer.

[0148] After acquiring the raw data of network status information at each domain level, the cross-domain SDN control and resource abstraction layer processes and abstracts the raw data to construct a globally unified cross-domain virtual network resource graph. Nodes in this resource graph can, but are not limited to, represent switches, hosts, or aggregation points of an entire domain; edges represent physical or logical links; and edge weights can be multi-dimensional vectors containing various attributes such as bandwidth, latency, cost, and SLA fulfillment. The specific steps for constructing the cross-domain network resource graph are given below.

[0149] (1) Network status information collection: The resource aggregation and abstraction module communicates with the SDN controller of each data center through the southbound interface to obtain the original network status information such as network device topology, traffic matrix and link bandwidth capacity.

[0150] (2) Information processing and abstraction: The collected information is processed and abstracted to form a cross-domain network resource graph. The nodes in the graph represent physical or virtual network devices of the data center, the edges represent links, and the weight of the edges includes various attributes such as bandwidth, latency, and cost.

[0151] (3) Construction of cross-domain network resource graph: The resource graph integrates the local network status of each data center and cross-domain boundary information to form a globally unified view, providing a high-dimensional and structured environment model for the decision-making of the global coordinating agent.

[0152] Through the above process, a mechanism for perceiving and expressing the global network state in a multi-agent system was established, providing a solid data foundation for the intelligent collaborative allocation and performance optimization of cross-domain SDN resources. The globally coordinating agent can efficiently solve games and generate strategies based on a clear and globally unified resource graph, achieving intelligent management of complex network environments.

[0153] In an exemplary embodiment, the above-described determination of the global benefit metric after executing each candidate operation in the candidate operation list based on a pre-deployed global decision model includes: sequentially obtaining each candidate operation from the candidate operation list as the current candidate operation; evaluating the current impact parameters on global network performance after executing the current candidate operation based on the target game model embedded within the global decision model, wherein the current impact parameters include the overall network status of the cross-domain network, the predicted traffic pattern, and the service compliance status satisfaction within the future decision cycle; and determining the current global benefit metric corresponding to the current candidate operation based on the current impact parameters.

[0154] Based on the description of the multi-agent game reinforcement learning decision layer in the above embodiments, the specific process of generating a global payoff index and generating the final global resource collaborative optimization scheme through the game process is as follows.

[0155] (1) Game Theory Solution: The candidate actions submitted by each domain are regarded as strategies in the game. Cooperative game theory or Stackelberg game models are used to evaluate the comprehensive impact of different strategy combinations on the global objective. For example, if two domains want to compete for the bandwidth of the same cross-domain link, the global coordinating agent will arbitrate based on principles such as global SLA, fairness, and cost.

[0156] (2) Calculate the benefits and incentives: Based on the game results, calculate the "contribution" of each domain agent in this round of decision-making, and convert it into reward signals or incentive parameters (such as virtual price, resource allocation weight, etc.) and feed them back to the domain agents to guide them to make choices that are more in line with the overall interests in the next round of decision-making.

[0157] (3) Generate a collaborative scheme: At the game equilibrium point, synthesize a globally consistent and conflict-free cross-domain collaborative resource allocation scheme. The technical solution of this application clarifies the end-to-end path of key cross-domain traffic, the bandwidth allocation of each link, QoS parameters, etc.

[0158] By using the above methods, the expected impact of each candidate operation on the global network performance is effectively evaluated and quantified, ensuring that the decision not only solves the current problem, but also takes into account the future evolution trend of the network state, thus achieving true global optimization and forward-looking decision-making.

[0159] In an exemplary embodiment, the above-mentioned determination of the global resource allocation strategy based on the global benefit index includes: selecting the target candidate operation corresponding to the target global benefit index with the highest value from the candidate operation list, and generating a global resource allocation strategy based on the target candidate operation, wherein a candidate operation in the candidate operation list includes at least one of network resource transmission link adjustment, bandwidth adjustment, and queue priority adjustment.

[0160] After evaluating the current impact parameters of each operation in the candidate operation list on the global network performance, the global coordinating agent calculates a global benefit metric for each operation. This metric integrates the operation's impact on multiple dimensions of objectives, including resource allocation, service performance, cost, and energy consumption. The global agent then selects the operation with the highest target global benefit metric from these operations as the target candidate operation, because this operation is evaluated as contributing the most to the global optimization objective.

[0161] After identifying the target candidate operation, the global coordinating agent generates a specific global resource allocation strategy based on that operation. This strategy details how to adjust cross-domain network resources, including but not limited to link usage, bandwidth allocation, and queue priority settings, to execute the selected operation and achieve the expected optimization effect. The global resource allocation strategy must ensure that it does not conflict with other existing strategies and meets multi-dimensional constraints, such as SLA requirements, cost control, and fairness principles.

[0162] The aforementioned global resource allocation strategy ensures an efficient decision-making mechanism for multi-agent systems in cross-domain SDN resource collaborative allocation. By selecting the operation with the highest objective global benefit metric, the system can achieve optimal improvement in global network performance. Simultaneously, by generating a global resource allocation strategy, the system ensures the enforceability of decisions and their positive impact on network status.

[0163] The above mechanism is the core of the technical solution of this application to achieve intelligent, dynamic, and adaptive cross-domain network resource management. It can significantly improve the utilization efficiency and service quality of network resources and provide an intelligent solution for network operation and maintenance in private cloud, hybrid cloud and multi-cloud environments.

[0164] In another exemplary embodiment, the above-mentioned determination of the global resource allocation strategy based on the global benefit index further includes: selecting multiple candidate operations from the candidate operation list that result in the optimal incremental improvement of the overall network performance, and generating a global resource allocation strategy based on the combined strategy composed of the multiple candidate operations.

[0165] After evaluating the current impact parameters of each operation in the candidate operation list on the global network performance, the global coordinating agent further analyzes the impact of different operation combinations on the overall network performance. This process involves multi-objective optimization of the candidate operations, with the global agent searching for the optimal combination of multiple candidate operations that maximizes the incremental improvement in overall network performance. In this way, the system can explore the synergistic effects between operations, achieving more efficient utilization of network resources.

[0166] For example, suppose operation 1 is to switch the video service traffic path from link 1 to link 2 from data center 1 in region 1 to data center 2 in region 2; operation 2 is to increase the bandwidth reservation for high-priority services in data center 1 in region 1 to 3Gbps; operation 3 is to adjust the priority of the core switch queue in the SDN network domain of data center 3 in region 3, for example, to prioritize the processing of critical service traffic sent from data center 1 in region 1.

[0167] The global coordinating agent evaluated the immediate impact of these operations on network performance and further analyzed the long-term impact of different operation combinations on overall network performance. Calculations showed that the combination of operations 1-3 improved the SLA satisfaction of critical services between Region 1 and Region 2 data centers. Simultaneously, operation 3 pre-adjusted network link resources between Region 3 and Region 2 data centers to cope with upcoming traffic peaks, thus avoiding potential congestion. Therefore, the global coordinating agent selected the combination strategy of operations 1-3 as the optimal candidate operation combination.

[0168] After determining the optimal combination of candidate operations, the global coordinating agent generates a global resource allocation strategy based on this combination strategy. The global resource allocation strategy specifies in detail how to execute these combined operations in cross-domain networks, including link usage, bandwidth allocation, queue priority settings, etc., to achieve incremental improvements in overall network performance.

[0169] This embodiment further improves the decision-making mechanism for cross-domain SDN resource collaborative allocation and performance optimization. By selecting multiple candidate operation combinations that maximize the incremental improvement in overall network performance, deep collaborative utilization of network resources can be achieved, improving the efficiency of resource allocation and the stability of network performance.

[0170] In an exemplary embodiment, the above-described method of adjusting the network resource allocation status by deploying a network controller in each data center includes: mapping a global resource allocation policy to network configuration instructions that the network controller can recognize, wherein the network configuration instructions include updating the traffic data table of the switch, adjusting the queue priority, adjusting the routing policy of the router, and adjusting the configuration of quality of service parameters; sending the network configuration instructions to the network controller of each data center; sending network resource adjustment instructions to the downstream network devices of each data center through the network controller, and adjusting the network resource allocation status based on the network resource adjustment instructions.

[0171] After generating global resource allocation policies, these policies are translated into specific configuration instructions that the underlying SDN network controller can understand and execute. These instructions include, but are not limited to, updating traffic data tables on switches, adjusting queue priorities, adjusting routing policies on routers, and configuring quality of service parameters. Through this mapping process, global resource allocation policies are transformed into specific control actions on the underlying network devices, ensuring that decisions directly impact the network and achieve optimized resource allocation.

[0172] The mapped network configuration commands will be sent to the SDN network controller in each data center, which will then perform the specific network resource adjustments. This process involves the transmission and parsing of commands to ensure that the commands reach the target controller accurately and are executed correctly.

[0173] After receiving network configuration instructions, the network controllers within the SDN network domain of each data center send network resource adjustment instructions to downstream network devices, such as switches and routers, to execute global resource allocation policies. This adjustment of network resource allocation status ensures that the network operates according to intelligent policies, achieving dynamic, closed-loop control of network resources.

[0174] The above methods ensure that the global resource allocation strategy generated by the intelligent agent system can be accurately converted, distributed, and executed, realizing dynamic and closed-loop control of network resources and improving the utilization efficiency and service quality of network resources.

[0175] In an exemplary embodiment, after adjusting the network resource allocation status based on the network resource adjustment instruction, the method further includes: periodically sending preset probe packets to the cross-domain network to obtain network latency, network jitter, and path information between any two network nodes; and generating monitoring results of overall network performance based on the network latency, network jitter, and path information.

[0176] After the global coordinating agent generates a new cooperative strategy, it is usually necessary to perform online performance monitoring on each domain-level network in order to measure network performance end-to-end and in real time.

[0177] Online performance monitoring methods include, but are not limited to, proactive probing. For example, the system periodically sends pre-set probe packets to the network. These probe packets are designed to measure key performance indicators such as latency and jitter in the network. The sending of pre-set probe packets can be done at fixed time intervals or triggered by specific events, such as changes in network state or after an agent's decision is executed.

[0178] When a pre-set probe packet propagates through the network and eventually reaches its destination node, the system collects the timestamp and path information carried within the packet to calculate the network latency, jitter, and complete end-to-end path between any two network nodes. This information is crucial for evaluating the effectiveness of network resource allocation strategies and for timely detection of potential problems in the network.

[0179] Based on the collected latency, jitter, and path information, overall network performance monitoring results will be generated. These results reflect the actual operating status of the network, including but not limited to whether network latency is within the threshold specified in the SLA, whether network jitter affects user experience, and whether there are abnormal hop counts or congestion points in the network path. The monitoring results are a crucial basis for the intelligent agent system's decision-making, ensuring that decisions are based on a real-time and accurate understanding of the current network performance.

[0180] In an exemplary embodiment, after adjusting the network resource allocation status based on the network resource adjustment instruction, the above method further includes: collecting flow data from each network device in the cross-domain network in real time, and generating monitoring results based on the flow data, wherein the flow data includes the throughput, packet loss rate, and protocol metrics of real service traffic.

[0181] The online monitoring module periodically or event-triggeredly collects flow data from various network devices across the cross-domain network. This flow data includes key metrics of network devices processing real-world traffic, such as throughput, packet loss rate, and protocol metrics. This data reflects the actual operating status of the network and the characteristics of its traffic.

[0182] The online monitoring module generates detailed monitoring results based on the collected flow data, including but not limited to throughput, packet loss rate, and protocol metrics of real business traffic. These results not only provide real-time operational status of network devices but also reflect the quality and characteristics of business traffic, serving as a key basis for evaluating the decision-making effectiveness of the intelligent agent system and identifying network performance problems.

[0183] This embodiment describes how to collect real-time flow data from various network devices in a cross-domain network and generate monitoring results reflecting the actual service traffic conditions based on this data. This ensures that the intelligent agent system's decisions are based on the actual network load, improving the accuracy of decisions and the speed of response to network performance issues.

[0184] In an exemplary embodiment, before issuing network configuration instructions to the network controller of each data center, the method further includes: creating a verification environment similar to the network topology and traffic patterns of the cross-domain network; executing a global resource allocation strategy in the verification environment to obtain simulation evaluation results; and issuing network configuration instructions if the simulation evaluation results indicate that the global resource allocation strategy has a positive optimization effect on the overall network performance.

[0185] To ensure a stable production environment, the implementation of the new strategy followed a strict process.

[0186] (1) Pre-verification: As above, a risk assessment must be conducted in the digital twin before any strategy change.

[0187] (2) Phased release (canary release): For major policy updates that have passed pre-verification, the system will not take effect on the entire network at once, but will adopt a canary release strategy.

[0188] For example, by tenant / business canary release, the new strategy is first applied to a small number of non-critical tenants or businesses to observe its performance; by domain canary release, the new strategy is first piloted in one or a few domains; during the canary release, the online monitoring module closely monitors the performance metrics of the affected objects.

[0189] (3) Promotion or Rollback: If the monitoring results show that the new strategy brings the expected performance improvement and has no negative impact, the system will automatically expand its application scope until the entire network is covered; if performance degradation or SLA breach is detected, the system will immediately trigger the automatic rollback mechanism to restore the network configuration of the affected part to the previous known stable strategy version. At the same time, the system will save all relevant data of this failed release (including strategy content, performance data before and after, event logs, etc.) for subsequent root cause analysis and model optimization.

[0190] The above embodiments ensure that the global resource allocation strategy generated by the intelligent agent system can be fully evaluated in a simulated verification environment, thereby improving the accuracy and security of decision-making.

[0191] In addition to the performance verification and pre-release evaluation mentioned above, this application also provides exception handling, robustness, and security control mechanisms. This is because an intelligent control system for a production environment must possess strong robustness and security assurance mechanisms.

[0192] (1) Anomaly detection and fault domain isolation.

[0193] 1) Multi-dimensional anomaly detection: The system has built-in anomaly detection capabilities based on rules and machine learning.

[0194] 2) Rule Engine: Predefines a series of thresholds and rules, such as "link utilization > 95%", "continuous packet loss rate > 1%", "topology change frequency too high", etc.

[0195] 3) Model detection: Train an unsupervised learning model (such as a variational autoencoder) to learn the patterns of normal network states. When the reconstruction error of a new network state in the model is too large, it is judged as abnormal.

[0196] 4) Fault Domain Isolation: When a serious fault is detected in a network domain (such as controller disconnection or a large number of devices going offline) or its agents become uncontrollable (such as continuously outputting unfavorable policies), the global coordinating agent can temporarily isolate the domain and its agents from game decision-making. Cross-domain traffic to this domain will be switched to a preset static and conservative routing policy, and agents in other domains will also treat this domain as unavailable when making decisions, thereby preventing the spread of the fault.

[0197] (2) Game constraint mechanism to prevent malicious agents.

[0198] Although game theory models aim to guide self-interested agents to make globally optimal decisions, mechanisms are still needed to prevent extremely selfish or malicious agents from disrupting the system.

[0199] Policy Consistency and Compliance Verification: A verification module is located at the global coordinating agent. Before executing the game-theoretic solution, it checks whether all candidate actions submitted by domain-level agents comply with basic security and compliance policies. For example, does an action attempt to establish a path across a security isolation zone, or does it violate a tenant's bandwidth limit? Any non-compliant action will be rejected outright.

[0200] Punishment and Reputation Mechanism: The global coordinating agent maintains the "reputation score" for each domain-level agent. If an agent frequently submits rejected and violates policies, or if the policies it executes frequently lead to negative effects in practice, its reputation score will be reduced. When allocating rewards, agents with lower reputation scores will receive correspondingly smaller incentive weights, and in extreme cases, their action space may be restricted (e.g., they may be prohibited from adjusting the routing of high-priority services).

[0201] (3) Strategy downgrade and manual takeover process.

[0202] In extremely complex or unknown scenarios, AI decision-making may not always be optimal or safest. Therefore, a multi-layered safeguard mechanism has been designed.

[0203] Policy Degradation: The system continuously monitors the health of the multi-agent model itself, such as the stability of decisions and the trend of reward value changes. If the model output begins to oscillate violently or the reward continues to decline, it indicates that the model may have encountered "unknown territory" outside its training data. At this time, the system will automatically degrade to a set of pre-configured, simpler baseline policies. This set of baseline policies may not be optimal, but it ensures the basic connectivity and stability of the network. For example, it could be a set of static paths calculated based on traditional traffic engineering algorithms, or a fixed cross-domain bandwidth quota scheme.

[0204] Manual intervention: The cloud management platform provides network operations and maintenance personnel with the highest-level intervention interface. At any time, operations and maintenance personnel can perform the following actions.

[0205] 1) Pause Intelligent Decision Making: Pause the entire multi-agent system with one click, freezing the network configuration in the current state.

[0206] 2) Manual Configuration: For any one or more critical service flows, manually specify their routing path, bandwidth, and QoS parameters. Manual configuration has higher priority than agent decision-making.

[0207] 3) Force rollback: Choose a strategy to force a full or partial network to revert to any stable version in history.

[0208] This "agent-led, expert-assisted, and readily available takeover" model ensures that while enjoying the convenience brought by intelligence, the ultimate control of the system always remains in the hands of the operation and maintenance personnel.

[0209] To ensure feasibility and long-term evolution in existing cloud platform environments, this application also proposes a deployment and evolution path for the intelligent agent system.

[0210] (1) Integration method with existing cloud operating system / cloud management platform.

[0211] Plug-in deployment: The core decision-making layer (multi-agent system) and verification layer of this application's technical solution are encapsulated as a set of independent microservices or plugins. These plugins can be deployed in the cloud platform's management cluster and interact with existing cloud platform modules via standard APIs. This approach minimizes invasiveness to the cloud platform and requires no large-scale modifications to its underlying architecture.

[0212] Compatibility with existing SDN controllers and networks: Through a standardized southbound interface, the technical solution of this application is compatible with mainstream SDN controllers (such as controllers based on frameworks like OpenDaylight) and network virtualization solutions in the industry. This means that users can smoothly introduce the intelligent decision-making capabilities in the embodiments of this application without replacing their existing SDN infrastructure.

[0213] (2) Evolution steps and gray-scale deployment strategy.

[0214] A complex intelligent system cannot be built overnight; the technical solution of this application supports a phased evolution path.

[0215] 1) Initial Phase (Monitoring and Observation Mode): In the initial deployment phase, the multi-agent system can operate only in "shadow mode." That is, the system performs complete online decision-making calculations, but the results are not distributed to the real network; they are only compared and recorded with the strategies running on the existing network. The goal of this phase is to verify the accuracy of the model and the rationality of the decisions, and to accumulate data in specific scenarios.

[0216] 2) Expansion Phase (Partial Business and Decision Support): After verifying the model's reliability, the system will be applied to some non-critical business operations or tenants. Simultaneously, the system can serve as a "decision support tool" for operations personnel, providing optimization suggestions for manual confirmation and execution.

[0217] 3) Mature stage (closed-loop autonomy): As the model is continuously iterated and optimized in the real environment, it gradually expands the scope of business it takes over and the decision-making authority, and finally realizes the automated closed loop of "perception-decision-execution-verification" in core business and the entire network, forming a long-term, sustainable network intelligent control capability.

[0218] (3) Measurement and acceptance index system.

[0219] To quantify the value brought by this invention, a three-dimensional indicator system is defined in the embodiments of this application.

[0220] 1) Network performance metrics.

[0221] End-to-end latency / jitter: Has the average latency / jitter of critical cross-domain services been reduced?

[0222] Network throughput: Whether the total throughput of the global network or critical links has improved.

[0223] Link utilization: Has the average utilization of backbone links improved, and is the load more balanced?

[0224] Congestion event rate: Whether the number of congestion alerts occurring in the network has decreased.

[0225] 2) Business and commercial metrics.

[0226] SLA default rate: Whether the number of SLA default events requiring protection has decreased significantly.

[0227] Multi-tenant fairness: In the event of resource competition, whether the allocation of resources among different tenants is more in line with the preset fair strategy.

[0228] Resource cost: Under the premise of meeting performance requirements, has the use of high-cost links been reduced, and has the overall network operating cost been reduced?

[0229] 3) Operation and maintenance efficiency indicators.

[0230] Strategy change automation rate: The proportion of strategy adjustments that are automatically executed by the system out of the total number of adjustments.

[0231] Mean Time To Recovery (MTBF): The average time from when a network performance problem occurs to when it is automatically resolved.

[0232] Number of manual interventions: Has the number of network events requiring manual intervention by operations and maintenance personnel decreased?

[0233] Model-related overhead: The computational and storage resources required for the training and inference processes.

[0234] By continuously measuring and tracking these indicators, the actual application effect and return on investment of the technical solution in this application in the private cloud management platform can be clearly evaluated.

[0235] In summary, the technical solution of this application transforms the complex cross-domain SDN resource allocation problem into an automatically solvable collaborative decision-making problem through an innovative multi-agent game-theoretic reinforcement learning architecture. This achieves self-optimization of network performance under multiple constraints, providing an efficient, intelligent, and reliable network brain for private cloud and multi-cloud environments.

[0236] In an exemplary embodiment, the method further includes: training an initial domain-level decision model deployed in multiple data centers based on a domain-level reward function to obtain a trained domain-level decision model; and jointly training the domain-level decision model and the initial global decision model to obtain a trained domain-level decision model and a global decision model.

[0237] In this embodiment, the system trains an initial domain-level decision model based on the domain-level reward function of each data center. The goal of the training is to enable the model to predict and execute actions that maximize the aforementioned reward function, i.e., to achieve optimal resource allocation within a single data center.

[0238] After training the domain-level decision-making model within a single data center, the system will proceed to a joint training phase, where the domain-level decision-making model and the initial global decision-making model learn collaboratively. The goal of the global decision-making model is to achieve optimal allocation of cross-domain resources while ensuring that the learning behavior of the domain-level agents converges to the global optimization objective. Through joint training, the system can explore behavioral collaborations between different domain-level agents, optimize global resource allocation strategies, and improve the decision-making capabilities of domain-level agents to adapt to more complex multi-data center network environments.

[0239] This embodiment describes how to locally train an initial domain-level decision model based on a domain-level reward function, and then jointly train it with an initial global decision model to optimize the global resource allocation strategy. This mechanism ensures that the multi-agent system can autonomously learn and collaboratively formulate the optimal resource allocation strategy in a multi-datacenter network environment, achieving a leap from single-domain optimization to global optimization.

[0240] In an exemplary embodiment, the above-mentioned training of an initial domain-level decision model deployed in multiple data centers based on a domain-level reward function includes: sequentially obtaining each data center from the multiple data centers as the current data center; training the current initial decision model deployed in the current data center based on the current domain-level reward function corresponding to the current data center and the current local network state obtained from the current data center, wherein the current domain-level reward function is obtained by weighting the domain performance reward, service compliance status satisfaction, cost penalty and incentive parameters, and the cost penalty includes the negative reward generated by using a link with high network resource consumption to perform cross-domain data synchronization or the negative reward generated by the current initial decision model outputting a high-energy-consuming decision.

[0241] To enable multi-agent systems to learn effective cooperative strategies, embodiments of this application employ a complex yet efficient training method. The training framework adopts a centralized training and distributed execution model.

[0242] Centralized training involves updating the policies of all agents (including global and domain-level agents) in a centralized training controller during the offline training phase. This controller has access to the states, actions, and rewards of all agents, allowing for better coordination and guidance of their learning process. Specifically, the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm can be employed. This algorithm trains an Actor network (policy network for decision-making) and a Critic network (value network for evaluating the quality of decisions) for each agent. During training, each agent's policy network can access information from all agents, enabling a more accurate evaluation of the value of joint actions and guiding the updates of the policy network.

[0243] Distributed execution includes an online inference phase (i.e., real-world deployment) where each domain-level agent makes decisions solely based on its own policy network and local observations (S_local), without needing to communicate with other agents or know the global state. This effectively reduces the complexity and communication overhead of online decision-making and ensures real-time decision-making.

[0244] In addition, a multi-stage training process is used to train domain-level and global decision-making models. Specifically, the single-domain decision-making models are trained individually in isolated environments, enabling each domain-level model to learn how to optimize resource allocation within its own domain.

[0245] The reward function is the core of guiding an agent's learning; it defines what constitutes "good" behavior. The domain-level reward function R_local can be, but is not limited to, the one shown in Equation (1).

[0246] R_local=w1×R_perf+w2×R_sla-w3×R_cost+w4×R_global_incentive (1)

[0247] As can be seen, the reward for each domain-level agent is composed of multiple weighted components. The aforementioned R_perf is the domain-level performance reward, such as a negative correlation function with average link utilization (encouraging load balancing) or a negative correlation function with average queue length (encouraging low congestion).

[0248] R_sla is the SLA satisfaction reward; for example, a large positive value is awarded for meeting the SLA of the services carried by this domain, while a large negative value is awarded for breach. R_cost is the cost penalty; for example, using high-cost links or decisions that lead to high energy consumption will result in a negative reward. R_global_incentive is an incentive parameter from the global coordinating agent, which can also be understood as a global incentive signal. This is key to aligning local and global goals. For example, if the global agent wants traffic to take path A, it will give a positive incentive value to the domain agent that chooses path A.

[0249] The w1, w2, w3, and w4 mentioned above are all weighting coefficients used to quantify and adjust the relative importance and priority of each optimization objective in the intelligent decision-making process. Specifically, w1 and w2 represent the positive incentive weights for the agent's intra-domain network performance (such as reducing congestion) and SLA satisfaction (such as ensuring latency), respectively; w3 serves as a negative penalty sensitivity coefficient for cost and energy consumption, used to constrain high-cost resource usage behavior; and w4 is the global cooperative response weight, used to guide the domain-level agent to align with the global optimal objective by responding to global incentive signals while pursuing local gains.

[0250] Through the above methods, the multi-agent game reinforcement learning system can continuously improve the intelligent decision-making ability of domain-level agents by personalized training based on the local network state of each data center and the domain-level reward function, thereby achieving local optimization and global coordination of resource allocation strategies.

[0251] In an exemplary embodiment, the joint training of the domain-level decision model and the initial global decision model includes: generating a cross-domain global network state based on network state information obtained from multiple data centers; and iteratively training the domain-level decision model and the initial global decision model based on the cross-domain global network state to obtain the reinforcement learning experience output by each decision model. The reinforcement learning experience includes the network environment state perceived by each decision model in the current training round, the current operation performed by each decision model in the current network state, the immediate reward given to each decision model after performing the current operation, and the next state to which the cross-domain global network state transitions after each decision model performs the current operation.

[0252] All pre-trained agents are placed in a complete game environment containing a global coordinating agent for joint training. In this stage, the agents begin to learn how to respond to global stimuli and engage in games with other agents.

[0253] Robustness and Generalization Training: In the later stages of training, various disturbances and anomalies are injected into the simulation environment, such as simulated link failures, traffic bursts, and SLA requirement changes. This is equivalent to adversarial training, forcing the agent to learn coping strategies under non-ideal conditions, thereby improving the model's robustness and generalization ability.

[0254] Convergence and stability guarantees: (1) Policy clipping: Similar to the idea in the PPO (Proximal Policy Optimization, reinforcement learning algorithm), the magnitude of each policy update is limited to prevent drastic policy changes due to a bad learning experience, thereby ensuring the smoothness of the learning process. (2) Entropy regularization: A term related to policy entropy is added to the reward function to encourage the agent to explore more diverse actions and avoid premature convergence to a local optimum. (3) Target network: When updating the Critic network, a slowly updated "target network" is used to calculate the target Q value, which can reduce the problem of overestimation of the value function in the bootstrap process and increase training stability.

[0255] The target Q-value can be, but is not limited to, the action value in reinforcement learning algorithms. It is used to quantify or evaluate the total cumulative reward that an agent is expected to obtain in the future after performing a specific resource allocation action (such as bandwidth adjustment) in the current network state. Its physical meaning represents the degree of superiority or inferiority of different execution decisions in terms of ensuring SLA and optimizing network performance in the long run.

[0256] The specific operation for calculating the target Q-value is to add the immediate reward obtained at the current moment to the target network's prediction of the state at the next moment (target Q-value = immediate reward + decay coefficient × prediction value at the next moment). The Critic network updates its parameters by continuously reducing the difference between its output value and the target Q-value, thereby achieving accurate evaluation and optimization of cross-domain resource allocation strategies.

[0257] In other words, after each round of decision-making, the intelligent agent system collects reinforcement learning experience, specifically the current network state, decision actions, immediate rewards, and the global network state after state transitions, which constitute reinforcement learning experience and provide samples for subsequent training.

[0258] Model Iterative Optimization: The training controller periodically draws experience samples from the experience replay pool to update the decision-making models of all agents, continuously improving their decision-making ability and policy effectiveness.

[0259] The above methods ensure that the multi-agent game reinforcement learning system can continuously improve the intelligent decision-making capabilities of domain-level agents and global agents through iterative training based on cross-domain global network states, thereby achieving global optimization of resource allocation strategies.

[0260] In an exemplary embodiment, the method further includes: encapsulating the reinforcement learning experience output by each decision model according to a preset data structure to obtain an encapsulated array set; saving the array set to an experience replay pool and extracting a portion of the array from the experience replay pool according to a preset time period; and updating the network parameters of the policy network and value evaluation network inside each decision model based on a batch of experiences indicated by the portion of the array.

[0261] After each decision cycle, the system encapsulates the reinforcement learning experience output by each decision model (including domain-level agents and globally coordinating agents) based on a pre-defined data structure. This experience includes the current network environment state, the actions performed by each agent, the immediate rewards received, and information on the transition of the environment state after the actions were performed. The encapsulated experience is organized into an array for easy storage and analysis later.

[0262] The encapsulated set of experience arrays is saved to an experience replay pool, a database used to store historical decision-making experiences. The system randomly selects a portion of the array from the experience replay pool at preset time intervals (e.g., hourly or daily) for subsequent updates to the decision model. This random selection process ensures that the model can learn from different historical scenarios, improving the diversity and generalization ability of the learning process.

[0263] After extracting experience, the system uses this experience and employs a multi-agent deep deterministic policy gradient algorithm to update the policy network and value evaluation network within all decision models. During training, the system calculates the value of each action based on the current state, the action performed, the immediate reward received, and the next state, and adjusts the policy network accordingly to output better actions in the future. Simultaneously, the value evaluation network is updated to more accurately predict the reward after an action is performed, providing guidance for optimizing the policy network.

[0264] The above embodiments describe the entire process of how to encapsulate the decision-making experience of an agent, save it to an experience replay pool, and use this experience to update the decision-making model. This ensures that the agent system can continuously learn from historical data, improve its decision-making ability and strategy effectiveness, and achieve continuous optimization and upgrading of resource allocation strategies.

[0265] As can be seen from the descriptions of the above embodiments, the core of the technical solution of this application lies in formalizing the cross-domain resource allocation problem as a multi-agent game and solving it through reinforcement learning. Before implementing the solution, it is first necessary to define the types and responsibilities of the agents.

[0266] In this embodiment, two types of intelligent agents are included: domain-level agents and globally coordinating agents. The domain-level agent, as a participant in the game, aims to maximize the cumulative reward of its own domain. It is a rational, self-interested decision-maker, but its behavior is guided by the rules and incentives set by the globally coordinating agent. It is responsible for the fine-tuning of network resources within its domain and acts as the executor of local optimization.

[0267] The global coordinating agent acts as the organizer and arbitrator of the game, aiming to maximize the global utility function (which can also be understood as the global objective function) of the entire cross-domain system. Examples include the average SLA satisfaction rate of all key business operations and the overall network resource utilization rate. It guides domain-level agents by designing game mechanisms (such as pricing and incentive allocation), ensuring that their self-interested behaviors converge into outcomes beneficial to the whole system. This hybrid architecture of "centralized coordination + distributed decision-making" balances the global perspective of decision-making with the scalability of execution.

[0268] Furthermore, the state is the basis for the agent's decision-making and is based on the observation of the environment. This application's embodiments design a hierarchical state space.

[0269] Domain-level state: The state information observed by each domain-level agent, mainly focusing on the domain itself and its boundaries. It is a high-dimensional vector that may contain the following:

[0270] This domain topology summary includes aggregated information such as the number of nodes within the domain, the number of links, the hop count of the critical path, and available bandwidth.

[0271] (2) Link and queue status: Real-time bandwidth utilization of all (or critical) links within the domain, average length of input / output queues, packet loss count, etc. These are usually the average or maximum values ​​within the most recent time window (e.g., 1 minute).

[0272] (3) Traffic matrix: The traffic distribution of important tenants or services within this domain, such as the amount of traffic from entry point A to exit point B, the amount of traffic from server C to exit point D, etc.

[0273] (4) SLA status: The current satisfaction status of SLA indicators related to this domain in cross-domain services carried by this domain, such as the latency contribution of this domain and packet loss events within this domain.

[0274] (5) Incentive signals from the global system: Incentive parameters issued by the global coordinating agent after the previous round of decision-making, such as the "virtual cost" or "reward points" for using a certain outbound link.

[0275] (6) Global State: The state information observed by the global coordinating agent, covering all domains. It consists of state summaries reported by each domain-level agent and global information provided by the resource abstraction layer, specifically including: 1) Cross-domain link state: load, latency, packet loss rate, etc. of all cross-domain boundary links; 2) End-to-end service path: the current complete path of all (or critical) cross-domain service flows, and the performance of the path; 3) Global SLA constraints: all services that require end-to-end assurance and their complete SLA requirements; 4) Candidate action set for each domain: the list of candidate actions submitted by all domain-level agents in the current decision cycle; 5) Historical reward allocation: the incentive allocation for each domain over a period of time, used to achieve long-term fairness.

[0276] Actions are the ways in which an intelligent agent interacts with its environment, that is, the decisions that the agent can make. The action space of this invention is also hierarchical.

[0277] (1) Domain-level actions: Decisions output by domain-level agents are typically discrete or parameterized. For example, they include the following.

[0278] 1) Path selection: For a certain type of traffic (such as video stream) destined for a specific domain, select one from N available outbound paths. For example, the action can be one of {path1, path2, ..., pathN}.

[0279] 2) Traffic Splitting: Distributing a certain type of traffic across multiple paths according to a specific ratio. For example, an action could be (p1, p2, ..., p...). N ), where p i It represents the proportion of traffic allocated to path i, and the sum is 1.

[0280] 3) Queue and Priority Adjustment: Increase or decrease the scheduling priority of a queue, or adjust its guaranteed bandwidth or maximum rate limit. For example, the action could be {increase priority, decrease priority, set bandwidth to X Mbps}.

[0281] 4) Cross-domain exit rate limiting: Actively limit the total flow rate from this domain to a neighboring domain to avoid congestion at the other end.

[0282] (2) Global Actions): The decisions made by the global coordinating agent are mainly about resource allocation and the setting of game rules.

[0283] 1) Cross-domain bandwidth quota allocation: determines the maximum amount of bandwidth each domain can use on a shared cross-domain link in the next decision cycle.

[0284] 2) Cross-domain primary and backup path specification: For an important cross-domain business, explicitly specify a primary path and one or more backup paths.

[0285] 3) Incentive parameter settings: Adjust the parameters used to calculate domain-level agent rewards. For example, increase the reward weight for "reducing latency" behavior, or impose a higher virtual price penalty for using scarce link resources.

[0286] To more clearly understand the above network resource allocation methods, the following will combine... Figure 4 The specific examples shown further illustrate this.

[0287] Scenario Setting: Assume a large enterprise has built a private cloud platform that manages three data centers (Data Center 1, Data Center 2, and Data Center 3) located in Region 1, Region 2, and Region 3 respectively. Each data center is an independent SDN domain (SDN network domain) with its own SDN controller. Assume a critical business requires cross-region data synchronization, specifically the continuous transmission of large amounts of data between the data centers in Region 1 and Region 2, with an SLA requiring an end-to-end bandwidth of at least 1Gbps and a latency of no more than 30ms. Simultaneously, there is a costly backup link between the SDN network domains of Region 1 and Region 3, typically used for low-priority services. The implementation steps are as follows.

[0288] Step 1: System initialization and agent deployment.

[0289] (1) Deploy a global coordination agent on the core management node of the cloud management platform (which can be a single node or a distributed node).

[0290] (2) Deploy a domain-level agent next to the SDN controller in each of the three data centers, namely domain-level agent 1, domain-level agent 2, and domain-level agent 3.

[0291] (3) The cross-domain resource aggregation and abstraction module is started. It establishes a connection with the SDN controllers of the three domains through the API, starts to collect static information such as topology and link capacity, and builds the initial global virtual network resource graph.

[0292] (4) The online monitoring module starts working, injecting active probe packets between data centers and collecting flow data from the SDN controller to measure the bandwidth utilization, latency, packet loss rate, etc. of each cross-domain link (such as the main link from region 1 to region 2, the backup link from region 1 to region 3, and the link from region 3 to region 2) in real time.

[0293] Step 2: Online decision-making and collaboration based on game theory models (an example of a decision cycle).

[0294] Suppose that at a certain time t, the online monitoring module discovers that the main link from region 1 to region 2 becomes congested due to a sudden surge in video conferencing traffic, with its real-time bandwidth utilization reaching 95% and latency rising to 40ms, resulting in a breach of the SLA for the "cross-regional data synchronization" service.

[0295] (1) Status awareness and reporting.

[0296] Domain-level agent 1 perceives its local network status: outbound links (to region 2) are congested, queue lengths are increasing, and the SLA for the data synchronization service it carries is in breach.

[0297] Domain-level agent 2 perceives its local network state: the inbound link (from region 1) is congested.

[0298] Domain-level agent 3 perceives the status of its local network: the load on the links it manages is normal.

[0299] Each domain-level agent reports its own state summary to the global coordinating agent.

[0300] (2) Domain-level agents generate candidate actions.

[0301] After receiving the state input, the reinforcement learning model (Actor network) of Domain-level Agent 1 outputs a list of candidate actions to address the congestion and SLA default issues. A specific example is shown below.

[0302] Action A1 (aggressive): Switch 50% of the data synchronization traffic from the main path "Region 1 → Region 2" to the backup path "Region 1 → Region 3 → Region 2".

[0303] Action A2 (Conservative): Request that 1Gbps bandwidth be forcibly guaranteed for data synchronization services on the main path of "Region 1 → Region 2" (may affect other services).

[0304] Domain-level agents 2 and 3 may output "no action" candidate actions because they have no urgent local problems.

[0305] (3) The global coordinating agent performs game theory to solve the problem.

[0306] The global coordinating agent receives candidate actions A1 and A2 from domain-level agent 1. For each action, it performs a deduction within its Stackelberg game model.

[0307] Analysis of Action A1: If A1 is executed, the high-cost link from Region 1 to Region 3 will be activated, leading to increased operating costs. However, the latency of the data synchronization service will change from 40ms (congestion) to latency (Region 1 to Region 3) + latency (Region 3 to Region 2), assuming a total latency of 25ms, thus satisfying the SLA. Simultaneously, congestion on the main link from Region 1 to Region 2 will be alleviated.

[0308] Analysis of Action A2: If A2 is executed, forcibly guaranteeing bandwidth may lead to a decrease in the service quality of other services on the main link (such as video conferencing), triggering a new SLA default, thereby reducing the overall utility.

[0309] The leader role in the game theory model, namely the global coordinating agent, determines that although action A1 introduces costs, it has a higher overall benefit in terms of global SLA satisfaction and network health.

[0310] (4) Generate collaborative schemes and allocate incentives.

[0311] The global coordinating agent makes a decision, namely, adopting action A1 from the Beijing agent. It generates a collaborative resource allocation scheme, which includes: offloading 50% of the "data synchronization service" (identified by its IP 5-tuple) on the boundary switch of the region 1 data center to the port heading to region 3.

[0312] Simultaneously, calculate the incentive or incentive parameters: Since the action proposed by domain-level agent 1 effectively solves the global problem, the global coordinating agent sends it a higher R_global_incentive (global incentive reward), which will positively reinforce the tendency of domain-level agent 1 to make similar "global resource optimization" decisions in the future.

[0313] Step 3: Policy verification and secure distribution.

[0314] (1) Digital Twin Pre-Drill: The generated collaborative solution is first sent to the performance evaluation and verification module. In a digital twin environment that is synchronized with the real network status in real time, the solution is virtually executed. The pre-drill results show that after the traffic switch, the SLA of the "data synchronization" service is met and no new congestion is caused on the Area 3→Area 2 link, so the pre-drill is passed.

[0315] (2) Southbound Interface Translation and Distribution: The southbound interface module translates the abstract collaboration scheme into specific instructions. It sends an interface call instruction to the SDN controller of data center 1, instructing it to add a new OpenFlow flow table rule on the border switch: match the data packets of the "data synchronization" service, and set the egress action to the physical port leading to data center 3 with a 50% probability.

[0316] SDN Controller 1 sends this flow table rule to the specified OVS or physical switch.

[0317] Step 4: Closed-loop feedback and model iteration.

[0318] (1) Performance monitoring: After the strategy is executed, the online monitoring module continuously tracks the end-to-end performance of the "data synchronization" service. Data shows that the latency is stable at around 25ms, the bandwidth meets the 1Gbps requirement, and the SLA returns to normal.

[0319] (2) Experience storage: The entire decision-making process, including the initial state, the actions of each agent, the global decision, the final reward and the effect, is stored as a complete experience tuple (S,A,R,S') in the experience replay pool for offline training.

[0320] (3) Offline training and model update: In the background, the centralized training controller continuously extracts experience data from the experience replay pool and uses the MADDPG algorithm to update the policy network (Actor) and value network (Critic) of all agents (including global and domain level).

[0321] Through this successful experience, the domain-level agent 1 learned to prioritize using alternative paths when the main path is congested; the global agent optimized its trade-off strategy between different path costs and SLA benefits. The trained new model, after evaluation, can be pushed to the policy repository for online updates at appropriate times, thereby enabling continuous learning and evolution of the system.

[0322] Through the above implementation methods, the technical solution of this application constructs a closed-loop control system that integrates real-time perception, intelligent game, secure execution and continuous learning, which can automatically complete the complex cross-domain fault diagnosis and performance optimization work that originally required network experts to complete for hours or even days in minutes or even seconds.

[0323] Compared with existing technologies, the technical solution of this application introduces a multi-agent game reinforcement learning framework, which brings at least the following beneficial effects to cross-domain SDN resource management and performance optimization.

[0324] (1) Global Cooperative Optimization and Superior Performance: The advantage of the technical solution in this application lies in achieving true global optimization. By establishing a global coordinating agent and constructing a game theory model, it can transcend the local perspective of a single network domain and make decisions from the perspective of the entire cross-domain system. When resource competition or conflict occurs, it can arbitrate and coordinate based on multi-dimensional objectives such as global SLA, cost, and fairness, guiding the "self-interested" behaviors of each domain-level agent to converge into the most favorable result for the whole. This effectively solves the fundamental problem in existing technologies that "local optimization is not equal to global optimization," reduces end-to-end latency of critical services, increases total network throughput, and maximizes SLA satisfaction.

[0325] (2) Dynamic Adaptation and Proactive Prediction: The reinforcement learning mechanism employed empowers the system with the ability to learn from historical data and continuous interaction with the environment. It can autonomously discover complex, nonlinear spatiotemporal patterns in network traffic, rather than relying on manually set, rigid thresholds and rules. This gives the system a forward-looking predictive capability, allowing it to anticipate risks and proactively adjust before congestion occurs, achieving a shift from "passive response" to "proactive protection." For dynamic changes common in cloud environments such as traffic bursts and service migrations, this invention can achieve rapid and smooth adaptive adjustments, maintaining continuous stability of network performance.

[0326] (3) Highly Automated Operations and Maintenance and Improved Efficiency: The experience and wisdom of network experts are solidified and elevated into a strategy model of intelligent agents. It can automatically complete the entire closed loop from performance monitoring, problem diagnosis, policy generation, risk assessment to instruction issuance, greatly liberating network operations and maintenance manpower. Complex cross-domain performance problems that traditionally took hours or even days to solve can now be automatically identified and repaired by the system within minutes. This significantly reduces the mean time to recovery, reduces network failures caused by human error, and greatly improves the efficiency of the operations and maintenance team, allowing them to focus more on business innovation and architecture evolution.

[0327] (4) Good scalability and robustness: Adopting an architecture of "centralized coordination + distributed decision-making" and "centralized training, distributed execution", the system has good scalability. As the network scale expands, only domain-level agents need to be added accordingly, effectively distributing the complexity of decision-making and avoiding the single-point bottleneck of a purely centralized architecture. At the same time, the system is designed with a comprehensive multi-level security mechanism, including anomaly detection, fault domain isolation, policy degradation, and manual takeover. Even in extreme cases where some agents or network domains fail, the system can ensure the basic stability of core services and prevent the chain reaction of failures, demonstrating strong robustness.

[0328] (5) Improved resource utilization and cost savings: Intelligent global load balancing enables fuller utilization of all available paths in the network, including non-shortest paths that might be idle under traditional routing protocols. This effectively improves the utilization of cross-domain backbone links and avoids unnecessary network expansion investments. Furthermore, the decision-making model incorporates considerations of link costs, and the system tends to prioritize the use of low-cost links while meeting performance requirements, thereby directly reducing the enterprise's network operating expenses.

[0329] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0330] According to another aspect of the embodiments of this application, a network resource allocation device is also provided, the structural schematic diagram of which is shown below. Figure 5 As shown, the system includes the following modules: a first acquisition unit 502, used to acquire network status information of multiple data centers distributed in different regions; a first processing unit 504, used to determine a list of candidate operations for optimizing the reward of the domain-level decision model when the network status information indicates that the current network resources of the target data center are abnormal during the processing of cross-domain services, wherein the domain-level decision model is used to determine the allocation strategy of local network resources of each data center; a second processing unit 506, used to determine the global benefit index after executing each candidate operation in the candidate operation list based on a pre-deployed global decision model, wherein the current incentive parameter issued by the global decision model is used as a weighting term of the reward function of the domain-level decision model, and the current incentive parameter is used to adjust the local resource allocation strategy of the domain-level decision model to ensure that it provides a positive contribution to the optimal allocation of global resources; and a third processing unit 508, used to determine the global resource allocation strategy based on the global benefit index and adjust the network resource allocation status of each data center through the network controller deployed in each data center.

[0331] The specific execution steps involved in the various calculation processes and dynamic optimization of storage space in the above modules can be referred to the description in the above embodiments, and will not be repeated here.

[0332] Obviously, the network resource allocation device described above can be used to implement the network resource allocation method provided in the above embodiments, and details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0333] It should be noted that the first acquisition unit 502 in this embodiment can be used to execute the above step S202, the first processing unit 504 in this embodiment can be used to execute the above step S204, the second processing unit 506 in this embodiment can be used to execute the above step S206, and the third processing unit 508 in this embodiment can be used to execute the above step S208.

[0334] In an exemplary embodiment, the first processing unit 504 includes: a first acquisition module, configured to acquire target network status information of the target data center, wherein the target network status information includes network resource distribution and link status data; and a first processing module, configured to obtain a list of candidate operations that can handle anomalies and improve their cumulative rewards by inputting the target network status information into a target domain-level decision model pre-deployed in the target data center.

[0335] In an exemplary embodiment, the first processing module includes: a first processing submodule, configured to input the target network state information into a reinforcement learning model embedded within the target domain-level decision model, and, in conjunction with the target incentive parameters in the reward function corresponding to the reinforcement learning model, output a list of candidate operations that balances improving the cumulative reward of the reinforcement learning model itself and satisfying the cross-domain network global resource optimization objective.

[0336] In an exemplary embodiment, the apparatus further includes: a fourth processing unit, configured to determine a cross-domain global network state based on network state information; a fifth processing unit, configured to obtain prediction excitation parameters for the prediction operations output by each domain-level decision model in the current decision cycle by inputting the cross-domain global network state into a global decision model; a sixth processing unit, configured to obtain a domain-level decision set in the current decision cycle by inputting the local network state information of multiple data centers into a domain-level decision model, wherein the domain-level decision set includes the prediction domain-level operations output by the domain-level decision model; and a seventh processing unit, configured to determine the current excitation parameters based on the domain-level decision set, the prediction excitation parameters, and the historical decision results in the previous decision cycle.

[0337] In an exemplary embodiment, the seventh processing unit includes: a summarization module for summarizing the set of domain-level decisions to obtain global decisions, wherein the global decisions include global operations performed within a cross-domain network; a first prediction module for predicting the global performance increment gains of the cross-domain network after executing global decisions using a global objective function, wherein the global objective function is obtained by weighted summation of service compliance status satisfaction, resource utilization, and network resource consumption costs of key services; a second processing module for determining the marginal contribution of each domain-level decision model based on historical decision results and global performance increment gains, wherein the marginal contribution is used to measure the incremental impact of the domain-level operation corresponding to the domain-level decision on the global network performance; and a first adjustment module for adjusting the prediction incentive parameters based on the marginal contribution to obtain current incentive parameters, wherein the greater the contribution of the domain-level decision output by a domain-level decision model to the improvement of global performance, the more current incentives are obtained.

[0338] In an exemplary embodiment, the fourth processing unit includes: a third processing module, configured to determine the topology, traffic matrix, and bandwidth capacity of network devices in multiple data centers based on network state information; and a fourth processing module, configured to abstract the topology, traffic matrix, and bandwidth capacity of network links into a cross-domain network resource graph, wherein the cross-domain network resource graph is used to characterize the cross-domain global network state, a node in the cross-domain network resource graph represents a physical network device or virtual network device in a data center, and an edge in the cross-domain network resource graph represents a physical link or logical link for transmitting network resources.

[0339] In an exemplary embodiment, the second processing unit 506 includes: a second acquisition module, configured to sequentially acquire each candidate operation from the candidate operation list as the current candidate operation; a first evaluation module, configured to evaluate the current impact parameters on the global network performance after executing the current candidate operation based on the target game model embedded in the global decision model, wherein the current impact parameters include the overall network state of the cross-domain network, the predicted traffic pattern, and the service compliance status satisfaction within the future decision period; and a fifth processing module, configured to determine the current global benefit index corresponding to the current candidate operation based on the current impact parameters.

[0340] In an exemplary embodiment, the third processing unit 508 includes: a first selection module, configured to select the target candidate operation corresponding to the target global benefit index with the highest value from the candidate operation list, and generate a global resource allocation strategy based on the target candidate operation, wherein a candidate operation in the candidate operation list includes at least one of network resource transmission link adjustment, bandwidth adjustment, and queue priority adjustment.

[0341] In an exemplary embodiment, the third processing unit 508 further includes a second selection module, configured to select from the candidate operation list a plurality of candidate operations that optimize the incremental improvement of the overall network performance, and generate a global resource allocation strategy based on a combination strategy composed of the plurality of candidate operations.

[0342] In an exemplary embodiment, the third processing unit 508 further includes: a mapping module, configured to map the global resource allocation policy to network configuration instructions that the network controller can recognize, wherein the network configuration instructions include updates to the traffic data table of the switch, adjustments to queue priorities, adjustments to the routing policy of the router, and adjustments to the configuration of quality of service parameters; a first issuing module, configured to issue network configuration instructions to the network controller of each data center; and a second issuing module, configured to issue network resource adjustment instructions to the downstream network devices of each data center through the network controller, and adjust the network resource allocation status based on the network resource adjustment instructions.

[0343] In an exemplary embodiment, the above apparatus further includes: a second acquisition unit, configured to, after adjusting the network resource allocation status based on a network resource adjustment instruction, periodically send preset probe packets to the cross-domain network to acquire network latency, network jitter, and path information between any two network nodes; and generate a monitoring result of the overall network performance based on the network latency, network jitter, and path information.

[0344] In an exemplary embodiment, the above apparatus further includes: a data acquisition unit, configured to acquire streaming data in real time from each network device in the cross-domain network after adjusting the network resource allocation status based on a network resource adjustment instruction, and generate monitoring results based on the streaming data, wherein the streaming data includes the throughput, packet loss rate, and protocol metrics of real business traffic.

[0345] In one exemplary embodiment, the apparatus further includes: a first creation unit, configured to create a verification environment similar to the network topology and traffic patterns of the cross-domain network before issuing network configuration instructions to the network controller of each data center; an execution unit, configured to execute a global resource allocation strategy in the verification environment to obtain simulation evaluation results; and a publishing unit, configured to publish network configuration instructions when the simulation evaluation results indicate that the global resource allocation strategy has a positive optimization effect on the overall network performance.

[0346] In an exemplary embodiment, the apparatus further includes: a first training unit, configured to train an initial domain-level decision model deployed in multiple data centers based on a domain-level reward function to obtain a trained domain-level decision model; and a second training unit, configured to jointly train the domain-level decision model and the initial global decision model to obtain a trained domain-level decision model and a global decision model.

[0347] In an exemplary embodiment, the first training unit includes: a third acquisition module, configured to sequentially acquire each data center from multiple data centers as the current data center; and a first training module, configured to train the current initial decision model deployed in the current data center based on the current domain-level reward function corresponding to the current data center and the current local network state acquired from the current data center, wherein the current domain-level reward function is obtained by weighting the domain performance reward, service compliance status satisfaction, cost penalty, and incentive parameters, and the cost penalty includes the negative reward generated by using a link with high network resource consumption to perform cross-domain data synchronization or the negative reward generated by the current initial decision model outputting a high-energy-consuming decision.

[0348] In an exemplary embodiment, the second training unit includes: a sixth processing module, configured to generate a cross-domain global network state based on network state information obtained from multiple data centers; and a second training module, configured to iteratively train a domain-level decision model and an initial global decision model based on the cross-domain global network state to obtain reinforcement learning experience output by each decision model, wherein the reinforcement learning experience includes the network environment state perceived by each decision model in the current training round, the current operation performed by each decision model in the current network state, the immediate reward given to each decision model after performing the current operation, and the next state to which the cross-domain global network state transitions after each decision model performs the current operation.

[0349] In an exemplary embodiment, the above apparatus further includes: an encapsulation unit, configured to encapsulate the reinforcement learning experience output by each decision model according to a preset data structure to obtain an encapsulated array set; a storage unit, configured to store the array set in an experience replay pool and extract a portion of the array from the experience replay pool according to a preset time period; and an update unit, configured to update the network parameters of the policy network and value evaluation network within each decision model based on a batch of experiences indicated by the portion of the array.

[0350] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0351] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above-described network resource allocation method embodiments.

[0352] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, which is configured to execute the steps in any of the above-described network resource allocation method embodiments when it is run.

[0353] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0354] According to another aspect of the embodiments of this application, a computer program product is also provided, the computer program product including a computer program, which, when executed by a processor, implements the steps in any of the above-described network resource allocation method embodiments.

[0355] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described network resource allocation method embodiments.

[0356] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0357] The above provides a detailed description of a network resource allocation method provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for allocating network resources, characterized in that: include: Obtain network status information from multiple data centers distributed across different regions; When the network status information indicates that the current network resources of the target data center are abnormal during the processing of cross-domain services, a list of candidate operations for optimizing the reward of the domain-level decision model is determined, wherein the domain-level decision model is used to determine the allocation strategy of local network resources of each data center. Based on a pre-deployed global decision-making model, a global benefit index is determined after executing each candidate operation in the candidate operation list. The current incentive parameter issued by the global decision-making model is used as a weighted term of the reward function of the domain-level decision-making model. The current incentive parameter is used to adjust the local resource allocation strategy of the domain-level decision-making model to ensure that it provides a positive contribution to the optimal allocation of global resources. The positive contribution represents the contribution of the domain-level decision output by the domain-level decision-making model to the improvement of global network performance. Based on the global benefit metric, a global resource allocation strategy is determined, and the network resource allocation status is adjusted by the network controller deployed in each data center. The global resource allocation strategy is used to achieve incremental improvement in global network performance. The step of determining the candidate operation list for optimizing the domain-level decision model's own reward includes: obtaining the target network status information of the target data center, wherein the target network status information includes network resource distribution and link status data; inputting the target network status information into a reinforcement learning model embedded within the target domain-level decision model, and combining it with the target incentive parameters in the reward function corresponding to the reinforcement learning model, outputting the candidate operation list that balances improving the cumulative reward of the reinforcement learning model itself and satisfying the cross-domain network global resource optimization objective.

2. The method according to claim 1, characterized in that, Before determining the list of candidate operations for optimizing the reward of the domain-level decision model itself, the method further includes: Based on the network status information, determine the cross-domain global network status; By inputting the cross-domain global network state into the global decision model, the prediction incentive parameters for the prediction operations output by each domain-level decision model in the current decision cycle are obtained. By inputting the local network status information of the multiple data centers themselves into the domain-level decision model, a domain-level decision set for the current decision cycle is obtained, wherein the domain-level decision set includes the predicted domain-level operations output by the domain-level decision model; The current incentive parameters are determined based on the domain-level decision set, the predicted incentive parameters, and the historical decision results from the previous decision cycle.

3. The method according to claim 2, characterized in that, The determination of the current incentive parameters based on the domain-level decision set, the predicted incentive parameters, and the historical decision results from the previous decision cycle includes: The domain-level decision set is aggregated to obtain a global decision, wherein the global decision includes global operations performed within a cross-domain network; Using a global objective function, the global performance increment gain of the cross-domain network after executing the global decision is predicted, wherein the global objective function is obtained by weighted summation of service compliance status satisfaction, resource utilization, and network resource consumption cost of key services; Based on the historical decision results and the global performance increment gains, the marginal contribution of each domain-level decision model is determined, wherein the marginal contribution is used to measure the incremental impact of the domain-level operation corresponding to the domain-level decision on the global network performance. Based on the marginal contribution, the predicted incentive parameters are adjusted to obtain the current incentive parameters. The greater the contribution of the domain-level decision output by a domain-level decision model to the improvement of global performance, the more current incentives are obtained.

4. The method according to claim 2, characterized in that, Determining the cross-domain global network status based on the network status information includes: Based on the network status information, the topology, traffic matrix, and bandwidth capacity of the network devices in the multiple data centers are determined. The topology, the traffic matrix, and the bandwidth capacity of the network links are abstracted into a cross-domain network resource graph. The cross-domain network resource graph is used to characterize the cross-domain global network state. A node in the cross-domain network resource graph represents a physical network device or virtual network device in the data center, and an edge in the cross-domain network resource graph represents a physical link or logical link for transmitting network resources.

5. The method according to claim 1, characterized in that, The global benefit metric, determined by the pre-deployed global decision-making model after executing each candidate operation in the candidate operation list, includes: Each candidate operation is sequentially retrieved from the candidate operation list as the current candidate operation; Based on the objective game model embedded in the global decision-making model, the current impact parameters on the global network performance after executing the current candidate operation are evaluated, wherein the current impact parameters include the overall network status of the cross-domain network, the predicted traffic pattern, and the service compliance status satisfaction within the future decision-making cycle. Based on the current impact parameters, determine the current global benefit index corresponding to the current candidate operation.

6. The method according to claim 1, characterized in that, The step of determining the global resource allocation strategy based on the global revenue metric includes: Select the target candidate operation corresponding to the target global benefit indicator with the highest value from the candidate operation list, and generate the global resource allocation strategy based on the target candidate operation. The candidate operation in the candidate operation list includes at least one of network resource transmission link adjustment, bandwidth adjustment and queue priority adjustment.

7. The method according to claim 1, characterized in that, The step of determining the global resource allocation strategy based on the global revenue metric also includes: Select multiple candidate operations from the candidate operation list that result in the optimal incremental improvement of overall network performance, and generate the global resource allocation strategy based on the combined strategy of the multiple candidate operations.

8. The method according to claim 1, characterized in that, The adjustment of network resource allocation status by the network controller deployed in each data center includes: The global resource allocation policy is mapped to network configuration instructions that the network controller can recognize. The network configuration instructions include updating the traffic data table of the switch, adjusting the queue priority, adjusting the routing policy of the router, and adjusting the configuration of quality of service parameters. The network configuration command is sent to the network controller of each data center; The network controller issues network resource adjustment instructions to downstream network devices in each data center and adjusts the network resource allocation status based on the network resource adjustment instructions.

9. The method according to claim 8, characterized in that, After adjusting the network resource allocation status based on the network resource adjustment instruction, the method further includes: By periodically sending preset probe packets to the cross-domain network, network latency, network jitter, and path information between any two network nodes are obtained; and based on the network latency, network jitter, and path information, overall network performance monitoring results are generated.

10. The method according to claim 8, characterized in that, After adjusting the network resource allocation status based on the network resource adjustment instruction, the method further includes: Real-time flow data is collected from various network devices in a cross-domain network, and monitoring results are generated based on the flow data. The flow data includes the throughput, packet loss rate, and protocol metrics of real business traffic.

11. The method according to claim 8, characterized in that, Before issuing the network configuration command to the network controller of each data center, the method further includes: Create a verification environment that closely resembles the network topology and traffic patterns of a cross-domain network; The global resource allocation strategy is executed in the verification environment to obtain simulation evaluation results; If the simulation evaluation results indicate that the global resource allocation strategy has a positive optimization effect on the overall network performance, the network configuration command is issued.

12. The method according to any one of claims 1 to 11, characterized in that, The method further includes: Based on the domain-level reward function, the initial domain-level decision model deployed in the multiple data centers is trained to obtain the trained domain-level decision model. The domain-level decision model and the initial global decision model are jointly trained to obtain the trained domain-level decision model and the global decision model.

13. The method according to claim 12, characterized in that, The training of the initial domain-level decision model deployed in the multiple data centers, based on the domain-level reward function, includes: Each of the plurality of data centers is selected sequentially as the current data center; Based on the current domain-level reward function corresponding to the current data center and the current local network state obtained from the current data center, the current initial decision model deployed in the current data center is trained. The current domain-level reward function is obtained by weighting the domain performance reward, service compliance status satisfaction, cost penalty and incentive parameters. The cost penalty includes the negative reward generated by using a link with high network resource consumption to perform cross-domain data synchronization or the negative reward generated by the current initial decision model outputting a high-energy-consuming decision.

14. The method according to claim 12, characterized in that, The joint training of the domain-level decision model and the initial global decision model includes: Based on the network status information obtained from the multiple data centers, a cross-domain global network status is generated; Based on the cross-domain global network state, the domain-level decision model and the initial global decision model are iteratively trained to obtain the reinforcement learning experience output by each decision model. The reinforcement learning experience includes the network environment state perceived by each decision model in the current round of training, the current operation performed by each decision model in the current network state, the immediate reward given to each decision model after performing the current operation, and the next state to which the cross-domain global network state transitions after each decision model performs the current operation.

15. The method according to claim 14, characterized in that, The method further includes: According to the preset data structure, the reinforcement learning experience output by each decision model is encapsulated to obtain an encapsulated array set; The array set is saved to the experience replay pool, and a portion of the array is extracted from the experience replay pool according to a preset time period. Based on a set of experiences indicated by the partial array, the network parameters of the policy network and value assessment network within each decision model are updated.

16. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the network resource allocation method as described in any one of claims 1 to 15 when executing the computer program.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the network resource allocation method as described in any one of claims 1 to 15.

18. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the network resource allocation method as described in any one of claims 1 to 15.

Images