Cross-domain heterogeneous node resource cooperative scheduling method and related device

Abstract

The application provides a cross-domain heterogeneous node resource cooperative scheduling method and related equipment, the method comprises collecting the heterogeneous resource indicators of each node, performing normalization processing and calculating the corresponding comprehensive performance score; according to the comprehensive performance score, the nodes are divided into core nodes and general nodes; a complete multi-agent near-end strategy optimization model is deployed in the core nodes, and a lightweight multi-agent near-end strategy optimization model is deployed in the general nodes; the model parameters of all nodes in the network domain are aggregated in the domain center node to obtain domain-shared model parameters; the domain-shared model parameters of all network domains are integrated in the global center node to obtain global-shared model parameters; the global-shared model parameters are distributed to each node; according to the global-shared model parameters, the model parameters of each node are updated, and a resource scheduling task is executed. The cross-domain heterogeneous node resource cooperative scheduling method and related equipment provided by the application have high data interaction efficiency and good cooperative adaptation.

Description

Technical Field

[0001] This application relates to the field of optical communication network optimization technology, and in particular to a cross-domain heterogeneous node resource collaborative scheduling method and related equipment. Background Technology

[0002] With the rapid development of optical communication technology, optical networks, as the core infrastructure of modern communication, have been widely used in backbone networks, metropolitan area networks, and access networks. The efficiency and reliability of optical network resource scheduling directly affect the performance of the entire communication system. In current cross-domain scenarios, nodes belong to different network domains, and the status data formats and transmission protocols of nodes in each domain differ. How to solve the problems of incompatible data interaction formats and high interaction latency between domains when implementing cross-domain resource collaborative scheduling has become a core issue that current resource scheduling technologies need to address. Summary of the Invention

[0003] In view of this, the purpose of this application is to propose a cross-domain heterogeneous node resource collaborative scheduling method and related equipment to solve or partially solve the above-mentioned technical problems.

[0004] A first aspect of this application provides a cross-domain heterogeneous node resource collaborative scheduling method applied to a network system, the network system comprising multiple network domains, each network domain comprising multiple nodes, the method comprising: collecting heterogeneous resource indicators of each node, performing normalization processing and calculating the corresponding comprehensive performance score; dividing the nodes into core nodes and general nodes according to the comprehensive performance score; deploying a complete multi-agent proximal policy optimization model on the core nodes and a lightweight multi-agent proximal policy optimization model on the general nodes; designating the node with the highest comprehensive performance score within the network domain as the domain center node, each domain center node interacting with other nodes within the network domain through a standardized data interaction interface, the domain center node aggregating model parameters of all nodes within the network domain to obtain domain-shared model parameters; designating the node with the highest comprehensive performance score within the network system as the global center node, the global center node integrating the domain-shared model parameters of all network domains to obtain global shared model parameters; distributing the global shared model parameters to each node; updating the model parameters corresponding to each node according to the global shared model parameters, and executing resource scheduling tasks.

[0005] Furthermore, nodes The heterogeneous resource metrics mentioned above include computing power. Storage capacity and cross-domain data processing capabilities ,node The overall performance score , , and The weighting coefficients are used; the core node is... The node, the general node is The node, This is the performance scoring threshold.

[0006] Furthermore, the policy network of the complete multi-agent proximal policy optimization model includes 5 hidden layers, each containing 256 neurons; the value network of the complete multi-agent proximal policy optimization model includes 3 hidden layers, each containing 128 neurons; the policy network of the lightweight multi-agent proximal policy optimization model includes 3 hidden layers, each containing 128 neurons; the value network of the lightweight multi-agent proximal policy optimization model includes 1 hidden layer, each containing 64 neurons.

[0007] Furthermore, the shared model parameters within the domain , This is the sum of the number of core nodes and the number of general nodes within the network domain. For nodes Model parameters, For nodes The weight, The globally shared model parameters , The total number of network domains. For network domain The weight, , For network domain The number of all nodes in the system.

[0008] Furthermore, the cross-domain heterogeneous node resource collaborative scheduling method also includes: determining a global reward function based on model actions and the environmental state of core nodes; calculating a global value function based on the global reward function; and optimizing model parameters based on the global value function until the model converges.

[0009] Furthermore, the global reward function , , and These are the weighting coefficients. For throughput, This represents the total network transmission latency. The packet loss rate for the current task. For the joint action of all models, This represents the current global environment state. The environmental status of the core node , The set of all nodes within the network domain. A set of intra-domain and inter-domain links. For link Maximum bandwidth, For link The remaining allocable bandwidth; the global value function , For strategy The generated state and actions The distribution takes the expectation. As a discount factor, For time steps, For instant rewards, These are the parameters of the value network.

[0010] A second aspect of this application provides a cross-domain heterogeneous node resource collaborative scheduling device applied to a network system. The network system includes multiple network domains, each network domain including multiple nodes. The cross-domain heterogeneous node resource collaborative scheduling device includes: a node performance grading unit configured to collect heterogeneous resource indicators of each node, perform normalization processing, and calculate a corresponding comprehensive performance score; classifying the nodes into core nodes and general nodes based on the comprehensive performance score; a differentiated multi-agent deployment unit configured to deploy a complete multi-agent near-end policy optimization model on the core nodes and a lightweight multi-agent near-end policy optimization model on the general nodes; and a hierarchical aggregation and coordination unit configured to coordinate the resource allocation of nodes within the network domain. The node with the highest overall performance score is designated as the domain center node. Each domain center node interacts with other nodes within the network domain through a standardized data interaction interface. The domain center node aggregates model parameters for all nodes within the network domain to obtain domain-shared model parameters. The node with the highest overall performance score within the network system is designated as the global center node. The global center node integrates the domain-shared model parameters for all network domains to obtain global shared model parameters. A model parameter distribution unit is configured to distribute the global shared model parameters to each node. A scheduling execution unit is configured to update the model parameters corresponding to each node based on the global shared model parameters and execute resource scheduling tasks.

[0011] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the cross-domain heterogeneous node resource collaborative scheduling method described in the first aspect above.

[0012] A fourth aspect of this application provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the cross-domain heterogeneous node resource collaborative scheduling method described in the first aspect above.

[0013] A fifth aspect of this application provides a computer program product, including computer program instructions that, when executed on a computer, cause the computer to perform the cross-domain heterogeneous node resource collaborative scheduling method as described in the first aspect above.

[0014] As described above, this application provides a cross-domain heterogeneous node resource collaborative scheduling method and related equipment. The method includes collecting heterogeneous resource indicators of each node, performing normalization processing, and calculating the corresponding comprehensive performance score; dividing nodes into core nodes and general nodes based on the comprehensive performance score; deploying a complete multi-agent near-end policy optimization model on the core nodes and a lightweight multi-agent near-end policy optimization model on the general nodes; designating the node with the highest comprehensive performance score within the network domain as the domain center node, and having each domain center node interact with other nodes within the network domain through a standardized data interaction interface, aggregating model parameters of all nodes within the network domain at the domain center node to obtain domain-shared model parameters; designating the node with the highest comprehensive performance score within the network system as the global center node, integrating the domain-shared model parameters of all network domains at the global center node to obtain global shared model parameters; distributing the global shared model parameters to each node; updating the model parameters corresponding to each node based on the global shared model parameters, and executing resource scheduling tasks. By normalizing and comprehensively scoring the computing and storage capabilities of each node, network nodes are divided into core nodes and general nodes, enabling different nodes to undertake corresponding tasks according to their own capabilities. This approach fully utilizes the resources of each node, avoiding performance bottlenecks. By employing a unified intra-domain format, inter-domain transmission, and a three-level distribution system (global central node - domain central node - intra-domain nodes), it resolves the issue of incompatible cross-domain data formats. Furthermore, by setting up a domain central node within each network domain, local model parameters are uploaded to the domain central node for parameter aggregation. Then, the domain central nodes of multiple domains upload their respective shared intra-domain model parameters to the global central node for integration, forming globally shared model parameters. This multi-layered aggregation mechanism not only reduces the communication pressure caused by single-point aggregation but also enables rapid local optimization within domains and global collaboration between domains, thereby effectively improving the overall system's collaborative scheduling capabilities. This cross-domain heterogeneous node resource collaborative scheduling method and related equipment are simple and convenient. By introducing a multi-layered aggregation mechanism and combining it with the dynamic decision-making capabilities of multi-agent reinforcement learning, it provides a resource scheduling scheme with optimized communication overhead for heterogeneous optical networks, thereby improving the overall network utilization efficiency and service quality. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating a cross-domain heterogeneous node resource collaborative scheduling method in an embodiment of this application.

[0017] Figure 2 This is a diagram of the network system and resource scheduling architecture in the embodiments of this application.

[0018] Figure 3 This is a schematic diagram illustrating the logical relationship of a cross-domain heterogeneous node resource collaborative scheduling method in an embodiment of this application.

[0019] Figure 4 This is a schematic diagram of the structure of a cross-domain heterogeneous node resource collaborative scheduling device in an embodiment of this application.

[0020] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0022] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0023] Several technologies have proposed an agent-based collaborative perception and resource scheduling method, constructing a hierarchical collaborative architecture by integrating edge computing, federated learning, and deep reinforcement learning. The bottom-level agents are responsible for data preprocessing, the middle-level agents perform data fusion and decision-making, and the top-level agents handle global resource scheduling. The system employs a deep Q-network to dynamically adjust resource allocation and communication strategies, and optimizes communication energy consumption through an adaptive communication mechanism. Furthermore, global scheduling is optimized using Lagrange duality decomposition and genetic algorithms, achieving efficient collaboration and resource balancing. However, while using Lagrange duality decomposition combined with genetic algorithms for global resource scheduling, this method may face high computational overhead in large-scale systems, especially when dealing with dynamically changing environments, potentially failing to adjust and respond in real time.

[0024] Other technologies propose a cross-domain intelligent routing method for SDN based on multi-agent deep reinforcement learning. This method divides a large-scale SDN network into multiple subdomains and assigns an agent to each subdomain for concurrent learning, reducing the overhead of model training time. However, it does not differentiate the performance of edge nodes, and all nodes execute uniform collaborative logic. This can lead to the ineffective utilization of resources by edge nodes with different performance levels, resulting in insufficient network performance optimization.

[0025] Some technologies propose a cross-domain resource allocation and offloading method based on edge computing. This method divides the network into multiple clusters and assigns edge servers to each cluster for resource scheduling and task offloading. It rationally allocates computing resources and offloads tasks based on the computing power of field devices and edge servers, as well as task requirements, thereby optimizing task completion time and improving resource utilization. However, although this method considers the differences in computing power among different devices and servers, it still does not fully handle the dynamic changes in the network environment. Furthermore, although branch-and-bound methods and heuristic simulated annealing algorithms are introduced to optimize task offloading decisions, the computational load of the algorithm remains significant when dealing with large-scale networks and complex tasks.

[0026] In summary, centralized scheduling requires the collection of massive amounts of raw data across domains. Data format adaptation and long-distance transmission lead to high interaction latency, and single-point controllers are prone to becoming performance bottlenecks when processing cross-domain data, failing to adapt to the dynamic changes in cross-domain networks. Furthermore, the unoptimized data interaction mode of distributed methods increases latency and, by not considering node performance differences, easily creates performance bottlenecks, resulting in poor cross-domain collaborative scheduling performance. Therefore, addressing the problems of low data interaction efficiency, high communication overhead, and poor collaborative adaptability in cross-domain scenarios is particularly crucial.

[0027] To address the aforementioned issues, this application provides a cross-domain heterogeneous edge node resource collaborative scheduling device based on multi-agent systems. It aims to provide a resource scheduling scheme with optimized communication overhead for heterogeneous optical networks by introducing a multi-layer aggregation mechanism and combining the dynamic decision-making capabilities of multi-agent reinforcement learning. This will improve the overall utilization efficiency and service quality of the network, resulting in high cross-domain data interaction efficiency, low communication overhead, and good collaborative adaptability.

[0028] The following describes specific embodiments in conjunction with... Figures 1 to 5 The technical solution of this application will be described in detail below.

[0029] Some embodiments of this application provide a cross-domain heterogeneous node resource collaborative scheduling method, such as... Figure 2 As shown, it is applied to a network system, which includes multiple network domains, and each network domain includes multiple nodes.

[0030] A network system consists of multiple network domains, which can be logical domains, physical domains, management domains, geographical region domains, or business domains. For example... Figure 2 As shown, different colored areas represent different network domains. Each network domain includes multiple nodes, primarily edge nodes. Nodes 1-10 in the diagram represent nodes. Each network domain is managed by a domain center node. In cross-domain scenarios, each domain center node identifies the cross-domain topology through standardized data interaction interfaces (such as SDN cross-domain data protocols), collects parameters such as the computing power and storage capacity of nodes within each domain, and calculates cross-domain data processing capabilities, forming a two-level resource view of "intra-domain" and "inter-domain" (e.g., ...). Figure 2 This provides a foundation for optimizing cross-domain data interaction paths and classifying nodes. Cross-domain data processing capability is a comprehensive indicator used for node performance classification, which is obtained by weighting sub-indicators such as cross-domain link bandwidth adaptation accuracy and cross-domain data format conversion efficiency.

[0031] like Figure 1 As shown, the cross-domain heterogeneous node resource collaborative scheduling method includes the following steps:

[0032] S1. Collect heterogeneous resource indicators of each node, perform normalization processing and calculate the corresponding comprehensive performance score; divide the nodes into core nodes and general nodes according to the comprehensive performance score.

[0033] node The heterogeneous resource metrics mentioned above include computing power. Storage capacity and cross-domain data processing capabilities ,node The overall performance score , , and The weighting coefficients are used; the core node is... The node, the general node is The node, This is the performance scoring threshold.

[0034] computing power This refers to the processing power of a node, such as the computing performance of a CPU / GPU, which can be measured by processing speed (e.g., FLOPS) or the number of tasks completed per unit of time. Storage capacity. The memory size and storage capacity of a node determine the scale of tasks it can support. Cross-domain data processing capability. It can be weighted and synthesized from two sub-indicators: cross-domain link bandwidth adaptation accuracy and cross-domain data format conversion efficiency, as shown in the following formula: , in, and These are the weighting coefficients, and their sum is 1. For nodes within the statistical period (default 1 second, configurable) The average effective bandwidth of stable transmission through cross-domain links can be obtained by the node's local monitoring module through periodic status collection. This represents the real-time available bandwidth of the cross-domain link within the same statistical period. Nodes within the statistical period The total amount of cross-domain data that has successfully completed format conversion and been correctly transmitted. Successful format conversion and correct transmission can be determined using conventional verification methods, such as normal parsing of the target domain, data integrity verification, or a reception confirmation mechanism. Nodes within the same statistical period The total amount of cross-domain data received for conversion can be counted by the node's local data receiving queue, cross-domain data processing module, or gateway interface within a statistical period. That is, how much cross-domain format conversion data the node receives within a statistical period. Set the fixed statistical period length (default is 1s, configurable).

[0035] In order to make different performance indicators have the same dimensions, computational power Storage capacity and cross-domain data processing capabilities The indicators are normalized as follows: , in, It is a certain indicator. and These are the minimum and maximum values ​​of the indicator, respectively. The index is the normalized version.

[0036] To quantify these three performance metrics, the following comprehensive performance scoring formula is proposed: , Among them, here , , These are nodes The normalized computing power, storage capacity, and data processing capacity; , , The weighting coefficients (summing to 1) can be configured according to the business type and scheduling objectives, and are used to characterize the relative importance of computing power, storage capacity, and cross-domain data processing capability in the overall performance score of a node.

[0037] Based on comprehensive performance score The value of is used to divide network nodes into core nodes and general nodes, as follows: , in, It is the performance score threshold of the node. Its value can be preset according to the actual network size, node performance distribution and scheduling requirements. For example, the mean threshold method can be used to set the average value of the comprehensive performance score of all nodes in the domain as the threshold; or the quantile threshold method can be used to set the top 20% or top 30% of nodes as core nodes.

[0038] S2. Deploy a complete multi-agent proximal policy optimization model on the core node, and deploy a lightweight multi-agent proximal policy optimization model on the general node.

[0039] This step involves differentiated model deployment. Based on the node grading results, differentiated MAPPO (Multi-Agent Proximal Policy Optimization) models are deployed to adapt to the resource capabilities of heterogeneous nodes. Resource capabilities refer to the comprehensive support capabilities that nodes possess when performing multi-agent model training and resource scheduling tasks, specifically including the computing power, storage power, and cross-domain data processing capabilities involved in the aforementioned node performance grading.

[0040] The core node is deployed with a complete MAPPO model. As the aggregation center within the domain and the main agent for cross-domain collaboration, the core node needs to have a global perspective to coordinate resources within the domain and participate in cross-domain decision-making. Specifically, the policy network of the complete multi-agent proximal policy optimization model includes 5 hidden layers, each with 256 neurons, and the value network includes 3 hidden layers, each with 128 neurons.

[0041] General nodes possess certain computational and communication capabilities, but their resources are limited, making it impossible for them to fully undertake the complete model training task. Therefore, a lightweight MAPPO model is deployed for general nodes to simplify computation. Specifically, the policy network of the lightweight multi-agent proximal policy optimization model includes three hidden layers, each containing 128 neurons, which only output the probability of local task execution actions; the value network includes one hidden layer, each containing 64 neurons, used to evaluate the value of the local state.

[0042] To eliminate the risk of parameter inconsistencies between the core node's complete model and the lightweight model of general nodes, the dimensions of their input and output layers can be unified. This mandates that the number of neurons in the input and output layers of both core and general node models be exactly the same, avoiding dimensional differences at key interaction interfaces. After unifying the dimensions, only the dimensional differences in the intermediate hidden layers need to be addressed. Before subsequent aggregation of the domain center nodes, a fully connected layer mapping is used to stretch the low-dimensional parameters of the hidden layers of general nodes to the same dimensions as the core nodes in real time, significantly reducing the adaptation difficulty.

[0043] S3. The node with the highest comprehensive performance score within the network domain is designated as the domain center node. Each domain center node interacts with other nodes within the network domain through a standardized data interaction interface. The domain center node aggregates model parameters for all nodes within the network domain to obtain domain-shared model parameters. The node with the highest comprehensive performance score within the network system is designated as the global center node. The global center node integrates the domain-shared model parameters for all network domains to obtain globally shared model parameters.

[0044] Through two-level collaboration of intra-domain and inter-domain aggregation, global optimization of cross-domain resources is achieved, avoiding the performance bottleneck of centralized scheduling.

[0045] For intra-domain aggregation, multiple nodes train the model locally without a central server acting as an intermediary. The node with the highest score within the domain is selected as the domain center node, which performs local aggregation based on the model parameters received from other nodes. Within each domain, the domain center node aggregates the model parameters (policy network parameters) uploaded by core nodes and general nodes. ), forming shared model parameters within the domain The system aggregates and integrates the local learning results of nodes within the domain, reducing inconsistencies between nodes and improving the overall optimization effect within the domain. Specifically, model parameters are shared within the domain. , This is the sum of the number of core nodes and the number of general nodes within the network domain. For nodes Model parameters, For nodes The weight, .

[0046] Inter-domain aggregation integrates intra-domain shared model parameters from multiple domains at a global central node, forming globally shared model parameters. Inter-domain aggregation solves cross-domain optimization problems and improves the overall performance of the entire network. Specifically, it involves globally sharing model parameters. , The total number of network domains. For network domain The weight, , For network domain The number of all nodes in the system.

[0047] S4. Distribute the globally shared model parameters to each node.

[0048] The aggregated globally optimized model parameters are distributed back to each node for the next round of local training. This process ensures that nodes continuously obtain the latest global model parameters from across the network and make local decisions and updates based on the optimization results.

[0049] Globally shared model parameters The data is distributed to all domain center nodes. After receiving the globally shared model parameters, the domain center nodes update the domain-wide shared model parameters. Then the domain center node will update the... Distribute to all core nodes and general nodes within the domain.

[0050] S5. Update the model parameters corresponding to each node according to the global shared model parameters, and execute the resource scheduling task.

[0051] Each node updates its model parameters and then inputs the current local environment status into the updated model's policy network. Combining the current link's remaining bandwidth, node load, task requirements, and other status information, the policy network infers specific scheduling actions. The scheduling actions of multiple nodes are aggregated to form a global resource allocation scheme for the current scheduling period. Each node then executes resource scheduling tasks according to the global resource allocation scheme.

[0052] In some embodiments, the cross-domain heterogeneous node resource collaborative scheduling method further includes: determining a global reward function based on model actions and the environmental state of core nodes; calculating a global value function based on the global reward function; and optimizing model parameters based on the global value function until the model converges.

[0053] Define a global reward function Used to measure the joint actions of all agents. Overall performance in global resource scheduling tasks.

[0054] Specifically, the global reward function , , and These are the weighting coefficients. For throughput, This represents the total network transmission latency. The packet loss rate for the current task. For the joint action of all models, This represents the current global environment state. The environmental status of the core node , The set of all nodes within the network domain. A set of intra-domain and inter-domain links. For link Maximum bandwidth, For link The remaining allocable bandwidth. Weighting coefficient. , and The sum of these three parameters is 1, which can be preset based on service type, network optimization goals, or historical experimental results. Throughput, total latency, and packet loss rate are feedback indicators obtained from simulation environment or actual network operation monitoring results after the agent executes joint scheduling actions. Specifically, the agent outputs joint actions based on the current environmental state. After the system performs this action in a simulation environment or a real network, it collects the corresponding statistical indicators within the statistical period, which are used to construct the reward function to guide subsequent model training.

[0055] This global reward function is the core objective of the multi-agent collaborative training in this application, forming a complete closed loop with the subsequent content: its output will be used to calculate the global value function. The system assists in optimizing the policy network parameters of each core node; the trained local model parameters are then aggregated within and between domains; the aggregated global optimized model parameters are distributed to all nodes to ultimately guide cross-domain resource allocation.

[0056] The model's value network is primarily used to evaluate policies during the training phase. A centralized value network is employed to uniformly calculate the global value function. This value network can be deployed on a global central node, a domain central node, or the training server. The global value function is calculated by the centralized value network. The global value function is mainly used to evaluate the impact of scheduling actions selected by core nodes on the overall resource scheduling effect. During training, core nodes output scheduling actions based on their current state. The system calculates the global reward based on the throughput, latency, packet loss rate, and other results after the action is executed, and uses the global value function to determine whether the action is beneficial to the overall scheduling effect. If an action can improve throughput, reduce latency, and decrease packet loss rate, the probability of selecting this type of action is increased in subsequent training; conversely, the probability of selecting this type of action is decreased. In this way, the global value function can assist in optimizing the policy network parameters of core nodes.

[0057] Specifically, the global value function , For strategy The generated state and actions The distribution takes the expectation. The discount factor measures the importance of future rewards to current decisions. Its range is greater than 0 and less than 1. The closer the discount factor is to 1, the more emphasis is placed on long-term gains; the smaller the discount factor is, the more emphasis is placed on current immediate gains. Its specific value can be preset according to the task duration, business real-time requirements, or experimental results. Let t be the time step, representing the t-th time step after the current state. In task scheduling, one time step may be one scheduling cycle. At the t-th time step, the state is... Select Action And receive instant rewards. . These are the parameters of the value network, including the weight parameters and bias parameters in the value network, which are obtained through iterative updates during the model training process.

[0058] The global value function formula is expressed as a whole under the current strategy. Starting from the current state, the agent will receive an immediate reward at every future time step. The system then applies a discount factor to the future rewards and accumulates the discounts. Finally, it takes the expectation of all possible states and action trajectories to obtain the long-term cumulative reward estimate for the current state.

[0059] Example 1 A cross-domain heterogeneous node resource collaborative scheduling method, such as Figure 3 As shown, it includes: Step 1: In the initial stage, the nodes in the network are first classified and grouped according to their performance characteristics, and the performance indicators of the nodes are collected.

[0060] Step 2: For each node, train a local model based on its type and task requirements. Nodes utilize the MAPPO model for local task optimization. Deploy a complete MAPPO model for core nodes, including: a policy network with 5 hidden layers (256 neurons per layer) outputting resource scheduling action probabilities, and a value network with 3 hidden layers (128 neurons per layer) calculating state values. Deploy a lightweight MAPPO model for general nodes, including: a policy network with 3 hidden layers (128 neurons) outputting only local task execution actions; and a value network with 1 hidden layer (64 neurons per layer).

[0061] Step 3: After local training is completed, each node uploads the locally learned parameters to its corresponding domain center node.

[0062] Step 4: The domain center node receives the learning parameters from each node and performs local aggregation processing to obtain the shared model parameters within the domain, providing data support for subsequent cross-domain collaboration and global optimization.

[0063] Step 5: After receiving the intra-domain shared model parameters from each domain, the system will perform cross-domain data aggregation processing at the global central node to obtain the global shared model parameters. Taking into account the scheduling requirements and resource status of all domains, a global optimization strategy will be formulated.

[0064] Step 6: The global central node evaluates the aggregated globally shared model parameters and checks whether the global reward value has converged. If it has not converged, it returns to the adjustment phase. Convergence here means that the reward value of the global reward function tends to stabilize during training; that is, the global reward value remains stable over several consecutive training iterations, indicating that the current strategy has achieved a relatively stable optimization effect. This step corresponds to the global reward function and model training optimization process described earlier, and is used to determine whether to proceed to the subsequent scheduling and execution phase.

[0065] Step 7: If the predetermined optimization objectives are met, the system enters the resource scheduling execution phase and begins executing the final resource scheduling task. Each node executes the task according to the optimized global resource allocation scheme.

[0066] The input to the global model is the domain-wide aggregated model parameters uploaded by all domain central nodes. ; It is obtained by aggregating data after each domain has been independently trained based on its local environment state, and then input into the global model. It is an abstract representation of the global state. The output of the global model is the globally shared optimized model parameters. The model parameters will be distributed to all domain center nodes through the model parameter distribution unit to update the local model of each domain.

[0067] Based on this, a global resource allocation scheme for cross-domain tasks is derived from the globally optimized model parameters, which clarifies the resource quotas for each domain, cross-domain task routing, and load balancing strategies. Finally, the scheduling and execution unit guides each node to complete the specific task execution.

[0068] Step 8: After the scheduling is completed, the system enters the final stage. All resource scheduling tasks have been completed, and the resource utilization of all nodes and domains has reached the expected optimization goal.

[0069] This embodiment forms a closed-loop process of status acquisition → hierarchical deployment → aggregated decision-making → cross-domain optimization → scheduling execution.

[0070] In some embodiments of this application, a cross-domain heterogeneous node resource collaborative scheduling device is provided, applied to a network system, wherein the network system includes multiple network domains, and each network domain includes multiple nodes, such as... Figure 4 As shown, the cross-domain heterogeneous node resource collaborative scheduling device includes: a node performance grading unit 51, configured to collect heterogeneous resource indicators of each node, perform normalization processing, and calculate the corresponding comprehensive performance score; and divide the nodes into core nodes and general nodes according to the comprehensive performance score; a differentiated multi-agent deployment unit 52, configured to deploy a complete multi-agent near-end policy optimization model on the core nodes and a lightweight multi-agent near-end policy optimization model on the general nodes; and a hierarchical aggregation and coordination unit 53, configured to take the node with the highest comprehensive performance score in the network domain as the domain center node, and each domain center node... The nodes interact with other nodes within the network domain through a standardized data interaction interface. The domain center node aggregates model parameters of all nodes within the network domain to obtain domain-shared model parameters. The node with the highest comprehensive performance score in the network system is designated as the global center node. The global center node integrates the domain-shared model parameters of all network domains to obtain global shared model parameters. The model parameter distribution unit 54 is configured to distribute the global shared model parameters to each node. The scheduling execution unit 55 is configured to update the model parameters corresponding to each node according to the global shared model parameters and execute resource scheduling tasks.

[0071] The apparatus described above is used to implement the cross-domain heterogeneous node resource collaborative scheduling method corresponding to any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0072] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the cross-domain heterogeneous node resource collaborative scheduling method described in any of the above embodiments.

[0073] Figure 5 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0074] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0075] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0076] The input / output interface 1030 is used to connect input / output modules to realize information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input devices can include keyboards, mice, touchscreens, microphones, various sensors, etc., and the output devices can include displays, speakers, vibrators, indicator lights, etc.

[0077] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (e.g., USB, Ethernet cable, etc.) or wireless means (e.g., mobile network, WIFI, Bluetooth, etc.).

[0078] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0079] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0080] The electronic devices described above are used to implement the cross-domain heterogeneous node resource collaborative scheduling method corresponding to any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0081] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to execute the cross-domain heterogeneous node resource collaborative scheduling method as described in any of the above embodiments.

[0082] The non-transitory computer-readable medium of this embodiment includes both permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0083] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the cross-domain heterogeneous node resource collaborative scheduling method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0084] Based on the same concept, corresponding to the methods of any of the above embodiments, this application also provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the cross-domain heterogeneous node resource collaborative scheduling method as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0085] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application is limited to these examples; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in detail for the sake of brevity.

[0086] Furthermore, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the apparatus may be shown in block diagram form. This is to prevent the embodiments of this application from being difficult to understand, and it also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully within the understanding of those skilled in the art). In setting forth specific details to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application may be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0087] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description.

[0088] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A cross-domain heterogeneous node resource collaborative scheduling method, characterized in that, Applied to a network system comprising multiple network domains, each network domain comprising multiple nodes, the method includes: Collect heterogeneous resource indicators of each node, perform normalization processing, and calculate the corresponding comprehensive performance score; divide the nodes into core nodes and general nodes according to the comprehensive performance score; A complete multi-agent proximal policy optimization model is deployed on the core node, and a lightweight multi-agent proximal policy optimization model is deployed on the general nodes. The node with the highest overall performance score within the network domain is designated as the domain center node. Each domain center node interacts with other nodes within the network domain through a standardized data interaction interface. The domain center node aggregates model parameters for all nodes within the network domain to obtain domain-shared model parameters. The node with the highest overall performance score within the network system is designated as the global center node. The global center node integrates the domain-shared model parameters for all network domains to obtain globally shared model parameters. The globally shared model parameters are distributed to each node; Update the model parameters corresponding to each node according to the globally shared model parameters, and execute the resource scheduling task.

2. The cross-domain heterogeneous node resource collaborative scheduling method according to claim 1, characterized in that, node The heterogeneous resource metrics mentioned above include computing power. Storage capacity and cross-domain data processing capabilities ,node The overall performance score , , and The weighting coefficients are used; the core node is... The nodes, the general nodes are The node, This is the performance scoring threshold.

3. The cross-domain heterogeneous node resource collaborative scheduling method according to claim 2, characterized in that, The policy network of the complete multi-agent proximal policy optimization model includes 5 hidden layers, each of which contains 256 neurons; the value network of the complete multi-agent proximal policy optimization model includes 3 hidden layers, each of which contains 128 neurons. The policy network of the lightweight multi-agent proximal policy optimization model includes three hidden layers, each containing 128 neurons, and the value network of the lightweight multi-agent proximal policy optimization model includes one hidden layer, each containing 64 neurons.

4. The cross-domain heterogeneous node resource collaborative scheduling method according to claim 3, characterized in that, The domain-shared model parameters , This is the sum of the number of core nodes and the number of general nodes within the network domain. For nodes Model parameters, For nodes The weight, ; The globally shared model parameters , The total number of network domains. For network domain The weight, , For network domain The number of all nodes in the system.

5. The cross-domain heterogeneous node resource collaborative scheduling method according to claim 4, characterized in that, Also includes: The global reward function is determined based on the model actions and the environmental state of the core nodes, and the global value function is calculated based on the global reward function. Optimize the model parameters based on the global value function until the model converges.

6. The cross-domain heterogeneous node resource collaborative scheduling method according to claim 5, characterized in that, The global reward function , , and These are the weighting coefficients. For throughput, This represents the total network transmission latency. The packet loss rate for the current task. For the joint action of all models, This represents the current global environment state. The environmental status of the core node , The set of all nodes within the network domain. A set of intra-domain and inter-domain links. For link Maximum bandwidth, For link The remaining allocable bandwidth; The global value function , For strategy The generated state and actions The distribution takes the expectation. As a discount factor, For time steps, For instant rewards, These are the parameters of the value network.

7. A cross-domain heterogeneous node resource collaborative scheduling device, characterized in that, Applied to a network system, the network system comprising multiple network domains, each network domain comprising multiple nodes, the cross-domain heterogeneous node resource collaborative scheduling device comprises: The node performance grading unit is configured to collect heterogeneous resource indicators of each node, perform normalization processing, and calculate the corresponding comprehensive performance score; based on the comprehensive performance score, the nodes are divided into core nodes and general nodes. The differentiated multi-agent deployment unit is configured to deploy a complete multi-agent proximal policy optimization model on the core node and a lightweight multi-agent proximal policy optimization model on the general node. The hierarchical aggregation and collaboration unit is configured to designate the node with the highest comprehensive performance score within the network domain as the domain center node. Each domain center node interacts with other nodes within the network domain through a standardized data interaction interface. The domain center node aggregates model parameters of all nodes within the network domain to obtain intra-domain shared model parameters. The node with the highest comprehensive performance score within the network system is designated as the global center node. The global center node integrates the intra-domain shared model parameters of all network domains to obtain globally shared model parameters. The model parameter distribution unit is configured to distribute the globally shared model parameters to each node; The scheduling execution unit is configured to update the model parameters corresponding to each node according to the globally shared model parameters and execute resource scheduling tasks.

8. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the cross-domain heterogeneous node resource collaborative scheduling method as described in any one of claims 1-6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the cross-domain heterogeneous node resource collaborative scheduling method as described in any one of claims 1-6.

10. A computer program product comprising computer program instructions, characterized in that, When the computer program instructions are executed on the computer, the computer performs the cross-domain heterogeneous node resource collaborative scheduling method as described in any one of claims 1-6.

Images