A large-scale optical network resource allocation method based on iterative learning

Abstract

The application provides a large-scale optical network resource allocation method based on iterative learning, comprising: initializing an optical network topology and a state of spectrum resources of each link; initializing a node feature code of each node and a link feature code of each link; dynamically updating v i , e j , and a current service request according to a current network state, and simultaneously generating an input state S I of a corresponding Agent; the input state S I is sent into the Agent as a basis for decision-making; a feasible solution conforming to an optical path constraint is gradually constructed; the decision-making is reinforced; a reward value R of an action is calculated; the updated reward value R is used for further optimization of the decision-making of the Agent; an EGAT model is trained and optimized; the EGAT model is evaluated and simulated; and a service allocation strategy is gradually constructed in combination with iterative learning, which can effectively cope with large-scale network topology changes and significantly improve the global performance and efficiency of scheduling.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of communication, in particular to a large-scale optical network resource allocation method based on iterative learning. BACKGROUND

[0002] With the gradual expansion of optical networks from backbone networks to metropolitan area networks, the rapid growth of optical network node scale has become the main development trend of the next generation of optical networks; the introduction of digital twin technology also provides a solid foundation for the deployment of large-scale optical networks.

[0003] However, the growth of network size leads to an exponential increase in computational complexity, bringing great challenges to fast and reliable network configuration; especially the demand for models that can effectively generalize in a diverse and dynamic environment further exacerbates these challenges; therefore, it is crucial to enhance the cross-topology generalization capability of service provision in large-scale optical networks.

[0004] In the prior art, there are many technical problems in the resource scheduling of large-scale optical networks, for example:

[0005] (1) Large network size: Large-scale optical networks involve hundreds to thousands of nodes and links, with complex and frequently changing topological structures, making the solution of the scheduling problem extremely complex due to the vast network state space.

[0006] As the network size continues to expand, traditional scheduling algorithms are difficult to meet the requirements of computational complexity and real-time performance;

[0007] (2) Dynamic topology and multi-objective optimization: The topology and traffic in large-scale optical networks change frequently, which requires the scheduling algorithm to have efficient real-time adaptability.

[0008] At the same time, multiple objectives such as bandwidth demand and network congestion need to be considered during the scheduling process, and how to balance these objectives and ensure efficient resource allocation in complex environments is a major challenge;

[0009] (3) Balance between global information and local decision-making: In large-scale networks, how to reasonably utilize global network information for optimization without increasing excessive computational overhead is still a problem to be solved.

[0010] Local decision-making, although computationally efficient, may lead to waste of global resources, and how to find an effective balance between the two is the key to algorithm design.

[0011] In order to overcome the above difficulties, existing technical personnel have designed many optical network resource allocation methods, although they can effectively operate in certain specific scenarios, but when faced with large-scale optical networks, they still have many shortcomings, such as:

[0012] (1) Dependence on predefined paths and fixed topology: Existing methods often rely on a predefined set of paths or static topology structures.

[0013] In large-scale optical networks, as the number of nodes increases dramatically, the difficulty of optical network operation and maintenance also rises rapidly. Due to faults or adjustments, nodes or links are dynamically added or deleted, leading to continuous changes in optical network topology. The above methods are difficult to quickly adapt to these changes, resulting in low resource scheduling efficiency, especially when dealing with dynamic demands, often unable to quickly find the optimal path.

[0014] (2) High computational complexity: Existing resource scheduling methods, such as shortest path-based, heuristic search, etc., often need to enumerate and evaluate all possible paths of the network.

[0015] When the network scale expands, the computational complexity of this method grows exponentially; for large-scale optical networks, such methods are difficult to give effective scheduling schemes within a reasonable time, especially in scenarios with high real-time requirements.

[0016] (3) Lack of cross-topology generalization ability: Many traditional scheduling algorithms are designed for specific network topologies or structures, lacking good cross-topology generalization ability.

[0017] This means that when the network topology changes, traditional methods need to be adjusted or retrained, which increases the computational and storage overhead and may lead to unstable performance. SUMMARY

[0018] To solve the above technical problems, the present application provides a large-scale optical network resource allocation method based on iterative learning, which gradually constructs service allocation strategies through iterative learning and uses reinforcement learning for training. This method not only can dynamically adjust the scheduling strategy according to the real-time network state, autonomously learn the optimal end-to-end resource allocation scheme, but also can effectively cope with large-scale network topology changes, significantly improve the global performance and efficiency of scheduling; in addition, the present application has strong cross-topology generalization ability, can stably operate in optical networks of different scales, and achieve higher resource utilization and network adaptability.

[0019] In a first aspect, a large-scale optical network resource allocation method based on iterative learning includes:

[0020] Step 1, initialize the optical network topology, background traffic distribution, and current traffic request;

[0021] As an example, the network topology includes: the number of nodes, the number of links, the topology connection relationship, the number of available wavelengths or frequency slots on the link; the background traffic distribution refers to: filling a certain number of random background traffic in the optical network, and the number of background traffic is determined by the network load rate; the current service request includes: source node, destination node, bandwidth demand.

[0022] Step two, according to the current optical network state, calculate the node feature code of all nodes and the link feature code of all links{v i},{e j}and mask vector M, which together constitute the input state S of the decision Agent I ;

[0023] As an example, the calculation of the node feature code of all nodes and the link feature code of all links{v i},{e j}and mask vector M is as follows: according to the topology structure relationship, the background traffic distribution and the current service request state, dynamically update the node feature code v i of each node and the link feature code e j of each link; according to the topology, a mask vector N is the current number of topology nodes; the Agent refers to: the agent of reinforcement learning.

[0024] As an example, the node feature code v i of each node refers to: each node has a K-dimensional node feature code N is the current number of topology nodes, wherein: each dimension corresponds to a state feature of the node, and each feature needs to reflect the key information required by the Agent decision.

[0025] The node feature code includes: the number of services passing through the current node, the total number of time slots occupied by the services passing through the current node, the spectral connectivity, the spectral connectivity block, the shortest path length from the current service endpoint, and the node label.

[0026] As an example, the link feature code e j of each link refers to: each link has an L-dimensional link feature code E is the number of edges in the entire topology; wherein: each dimension corresponds to a state feature of the link, which is customized according to the network state and service demand.

[0027] The link feature code includes: the total number of available wavelengths or frequency slots, the starting position of the first large block of available spectrum, the size of the largest continuous available spectrum block, the link fragmentation entropy, and whether there is a spectrum block that meets the conditions.

[0028] As an example, the dynamic update refers to updating the v i , e j Whenever a new node joins the partial solution, the network state adjusts and updates the spectrum resource usage of each link according to the current service request; by calculating the common available wavelength or frequency slot of the links on the selected path, the v i , e j Ensure that the input state S I of the Agent reflects the latest network resource situation;

[0029] The partial solution refers to a sequence of nodes selected by the decision Agent, but not yet forming a complete route from the source node to the destination node.

[0030] Step three, according to the input state S I , the Agent filters invalid adjacent nodes through the action mask mechanism, and selects one of the legal adjacent nodes as the action selection result, i.e. the next hop route, and calculates the reward function value;

[0031] According to the input state S I of the Agent, the Agent selects one of the adjacent nodes of the current node as the action selection result and adds it to the partial solution; through incremental iteration of one-hop selection, a feasible solution that meets the optical path constraints is gradually constructed; at the same time, according to the link wavelength or frequency slot usage between the next hop node and the current node and the existing common wavelength or frequency slot usage, the common available wavelength or frequency slot is calculated again to determine whether the common available wavelength or frequency slot meets the frequency slot requirement.

[0032] As an example, the model in the Agent adopts an edge feature enhanced graph neural network structure, uses a multi-layer EGAT architecture to extract the complex coupling relationship between link feature encoding and node feature encoding, and captures the relationship and change law between the input state S I , the action selection result and the reward function value.

[0033] As an example, in the multi-layer edge feature enhanced graph attention network EGAT, the link feature encoding e j and the node feature encoding v i are respectively input into the neural network as link features and node features for updating; through the multi-layer edge feature enhanced graph attention network EGAT, the node features and link features are iteratively updated, and the information of adjacent nodes and links is combined to optimize path selection.

[0034] As an example, the reward function value R is calculated according to the following formula:

[0035]

[0036] wherein: R is a reward value, which is the return in the current state; R terminal is a terminal reward, which is a fixed reward value returned when the task is successfully completed or an exception occurs; D is a network diameter, which represents the maximum length of the shortest path between any two nodes in the network; P is a penalty value, which is calculated according to the partial spectrum usage; S is a step number, which represents the number of steps that the current task has been performed; C is a condition value, which calculates the relative progress of the current node to the target node, a positive value indicating that the target is being approached, and a negative value indicating that the target is being moved away;

[0037] The calculation formula of the condition value C is designed as follows:

[0038] C = μ · (γ · len(P last,dst ) - len(P current,dst ))

[0039] wherein: μ is a proportional factor or a scaling factor, which is used to adjust the influence degree of the potential function; it controls the contribution degree of the path length difference to the reward;

[0040] γ is a constant, which is used to adjust the weight of the path length difference, and is used in reinforcement learning to control the discount factor of long-term return; in this formula, it is used to weight the calculation of the shortest path length from the last node to the target node; d curr,dst : the shortest path length from the current node to the target node.

[0041] Step four, repeat steps two and three until the termination condition of the current service is met, and determine whether the current service can be successfully routed according to the termination condition, if successful, select the frequency slot according to the First Fit strategy;

[0042] As an example, the termination condition refers to: reaching the end point, having no next hop node to choose; there is no public available wavelength or frequency slot that can meet the bandwidth requirement of the service.

[0043] As an example, the First Fit strategy refers to: arranging the wavelengths or frequency slots in a certain order, and each time a wavelength or frequency slot is allocated, all wavelengths or frequency slots are traversed in order, and the first available wavelength or frequency slot block that meets the bandwidth requirement of the service is selected;

[0044] Step five, record the input state S I , action selection result, reward function value generated in each iteration in the above process for Agent training;

[0045] As an example, through multiple rounds of training, the agent continuously adjusts the decision; after each training, the difference between the predicted action and the actual action is calculated using the cross-entropy loss function, and the weights of the model are updated through the back propagation algorithm; during the training process, a learning rate scheduler is used to optimize the learning rate, and the convergence speed of the model is improved.

[0046] In a second aspect, the present application shows an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the method of any one of the above aspects.

[0047] In a third aspect, the present application shows a non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of an electronic device, the electronic device can execute the method of any one of the above aspects.

[0048] In a fourth aspect, the present application shows a computer program product, when the instructions in the computer program product are executed by the processor of an electronic device, the electronic device can execute the method of any one of the above aspects.

[0049] The beneficial effects of the present application are:

[0050] The present application overcomes the problem that the existing optical network resource scheduling method cannot consider topology generalization and global optimization in a large-scale heterogeneous network, and dynamically generates service routing and spectrum allocation strategies that adapt to different network topologies through deep reinforcement learning.

[0051] The present application first proposes a method of gradually constructing the solution space based on iterative learning to autonomously learn the optimal scheduling strategy of the elastic optical network, and trains through reinforcement learning.

[0052] The present application innovatively proposes a separation type node feature encoding and link feature encoding method, further extracts the above features by using an edge-enhanced graph attention network (Edge-Enhanced Graph Attention Network), enhances the model's perception of optical path constraints in the optical network, and thus more accurately considers the topology structure and spectrum resource constraints of the optical network during resource scheduling. Compared with the traditional scheduling method based on path predefinition, the present method can effectively avoid path dependence, improve the accuracy and flexibility of scheduling.

[0053] The present application can more effectively handle the model retraining problem caused by topology changes in a large-scale optical network, and enhance the actual deployment capability of the algorithm; compared with the traditional heuristic algorithm, the present application can better adapt to a large-scale dynamic network environment, reduce the computational complexity and optimize the scheduling effect. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 This is a flowchart illustrating a large-scale optical network resource allocation method based on iterative learning according to the present invention.

[0055] Figure 2 This is a schematic diagram of an electronic device, which is one of the methods for allocating large-scale optical network resources based on iterative learning according to the present invention.

[0056] Figure 3 This is a schematic diagram of another electronic device as part of the large-scale optical network resource allocation method based on iterative learning according to the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Figures 1 to 3 As shown,

[0058] Firstly, a method for allocating large-scale optical network resources based on iterative learning includes:

[0059] Step 1: Initialize the optical network topology, background service distribution, and current service requests;

[0060] As an example, the network topology includes: the number of nodes, the number of links, the topological connection relationships, and the number of available wavelengths or frequency slots on the links;

[0061] As an example, the background service distribution refers to filling the optical network with a certain number of random background services, the number of which can be determined by the network load rate.

[0062] As an example, the current business request includes: source node, destination node, and bandwidth requirement.

[0063] Step 2: Based on the current optical network status, calculate the node feature code for all nodes and the link feature code for all links {v i},{e j}, and the mask vector M, together constitute the input state S of the decision agent. I ;

[0064] Based on the topological structure, background service distribution, and current service request status, dynamically update the node feature code v of each node. i and the link feature code e for each link j Based on the topology, a mask vector is calculated. N is the current number of nodes in the topology;

[0065] As an example, the Agent refers to the agent of reinforcement learning.

[0066] As an example, the node feature encoding v i Each node has a K-dimensional node feature encoding N is the current number of nodes in the topology, wherein each dimension corresponds to a feature of the node.

[0067] As an example, the node feature encoding includes but is not limited to the following information: the number of services passing through the current node, the total number of time slots occupied by the services passing through the current node, the spectrum connectivity, the spectrum connectivity block, the shortest path length from the current service endpoint, and the node label.

[0068] As an example, the link feature encoding e j Each link has an L-dimensional link feature encoding E is the number of edges in the entire topology, wherein each dimension corresponds to a state feature of the link, which is customized according to the network state and service request.

[0069] As an example, the link feature encoding includes but is not limited to the following information: the total number of available wavelengths or frequency slots, the starting position of the first large block of available spectrum, the size of the largest continuous available spectrum block, the link fragmentation entropy, and whether there is a spectrum block that meets the conditions.

[0070] As an example, the dynamic update refers to updating the v i , e j in real time according to the dynamic changes of the Agent node selection, whenever a new node joins the partial solution, the network state is adjusted, and the spectrum resource usage of each link is updated according to the current service request; by calculating the common available wavelength or frequency slot of the links on the selected path, updating the v i , e j , to ensure that the input state S I of the Agent reflects the latest network resource situation.

[0071] As an example, the partial solution refers to a sequence of nodes composed of nodes selected by the decision Agent according to the current service request, but not yet forming a complete route from the source node to the destination node.

[0072] The specific scheme of the dynamic update operation is:

[0073] The source node and the destination node of the current service request are represented by node labels, wherein the node labels in the node feature codes of the source node and the destination node are set to 1, and the node labels of other nodes are set to 0.

[0074] For the selected nodes in the partial solution, the node labels in the node feature codes of each node are set to 1, and the node labels of the remaining nodes are set to 0.

[0075] According to the usage of spectrum resources on each link in the optical network, the node feature codes and the link feature codes are updated, which includes: obtaining the common available wavelength or frequency slot according to the partial path obtained from the selected partial solution, and then taking the intersection of the usage of the common available wavelength or frequency slot and the usage of the available wavelength or frequency slot on any link in the entire network, that is, only retaining the wavelength or frequency slot that is available in the common available wavelength or frequency slot and is available on the current link; the intersection result is used as the remaining available wavelength or frequency slot on each link, which is used as the basis for calculating the link feature codes and the node feature codes.

[0076] According to the topology, a mask vector M is calculated N is the number of nodes in the current topology; in the mask vector M, the positions of the nodes adjacent to the current node are set to 1, and the positions of the nodes not adjacent to the current node are set to 0; all nodes located in the partial solution are set to 0.

[0077] Step three, according to the input state S I , the agent filters invalid adjacent nodes through the action mask mechanism, and selects one of the legal adjacent nodes as the action selection result, that is, the next hop route, and calculates the reward function value;

[0078] According to the input state S I of the agent, the agent selects one of the adjacent nodes of the current node as the action selection result and adds it to the partial solution; through incremental iteration of one-hop selection, a feasible solution that meets the optical path constraints is gradually constructed; at the same time, according to the link wavelength or frequency slot usage between the next hop node and the current node and the existing common wavelength or frequency slot usage, the common available wavelength or frequency slot is calculated again to determine whether the common available wavelength or frequency slot meets the bandwidth requirement.

[0079] As an example, the model in the agent adopts an edge feature enhanced graph neural network structure, uses a multi-layer EGAT architecture to extract the complex coupling relationship between the link feature codes and the node feature codes, and captures the relationship and change law between the input state S I , the action selection result, and the reward function value.

[0080] As an example, the EGAT is: Edge-Enhanced Graph Attention Network.

[0081] As an example, in the EGAT model, the link feature encoding e j and the node feature encoding v i are respectively sent into the neural network for updating; through the EGAT model, the node feature and the link feature are iteratively updated, and the information of the adjacent nodes and links is combined to optimize the path selection.

[0082] As an example, the EGAT model adopts a two-layer structure, the link feature encoding e j of the link and the node feature encoding v i of the node are respectively normalized, and then a fully connected layer is used for feature pre-extraction, and then input into the EGAT model for further feature interaction.

[0083] As an example, the reward function value R in step three is designed as follows:

[0084]

[0085] Wherein: R is the reward value, which is the return in the current state; R terminal is the terminal reward, which is a fixed reward value returned when the task is successfully completed or an exception occurs; D is the network diameter, which represents the maximum length of the shortest path between any two nodes in the network; P is the penalty value, which is calculated according to the partial spectrum usage (such as the number of zero blocks); S is the step number, which represents the number of steps the current task has taken; C is the condition value, which calculates the relative progress of the current node to the target node, a positive value indicates that it is approaching the target, and a negative value indicates that it is far away from the target;

[0086] As an example, the condition value C is designed as follows:

[0087] C = μ·(γ·len(P last.dst )-len(P current.dst ))

[0088] Wherein, μ: is a proportional factor or a scaling factor, which is used to adjust the influence degree of the potential function; it controls the contribution degree of the path length difference to the reward;

[0089] As an example, in actual application, the μ can be adjusted according to the task requirements.

[0090] γ: is a constant used to adjust the weight of path length difference, used in reinforcement learning to control the discount factor of long-term return; in this formula, it is used to weight the calculation of the shortest path length from the last node to the target node; P current.dst : the shortest path length from the current node to the target node;

[0091] As an example, the termination condition refers to reaching the destination, having no available next-hop route, or the public available wavelength or frequency slot not meeting the service bandwidth requirement.

[0092] Step four, repeat steps two and three until the termination condition of the current service is met, determine whether the current service can be successfully routed according to the termination condition, and select the wavelength or frequency slot according to the First Fit strategy; the wavelength and the frequency slot represent the allocable resources in the wavelength switching optical network and the elastic optical network respectively. The present application is not limited to a specific optical network type, and both types of optical networks can be configured in this way.

[0093] As an example, the termination condition includes: there is no available next-hop node; there is no public available wavelength or frequency slot that can meet the service bandwidth requirement;

[0094] As an example, the First Fit strategy refers to arranging the wavelengths or frequency slots in a certain order (from large to small), and each time a wavelength or frequency slot is allocated, all wavelengths or frequency slots are traversed in order, and the first available wavelength or frequency slot block that meets the service bandwidth requirement is selected;

[0095] Step five, record the input state S I , action selection result, reward function value generated in each iteration in the above process for agent training;

[0096] Through multiple rounds of training, the agent continuously adjusts the decision; after each training, the difference between the predicted action and the actual action is calculated using the cross-entropy loss function, and the weights of the model are updated through the backpropagation algorithm; during the training process, a learning rate scheduler is used to optimize the learning rate and improve the convergence speed of the model.

[0097] As an example, the agent trained on a 24-node topology can directly infer results on a larger optical network topology and guarantee the blocking rate performance without retraining or parameter fine-tuning, such as a 150-node optical network topology. The practical application effect of the trained agent model in a large-scale optical network ensures that it can effectively solve the spectrum resource allocation problem in a large-scale optical network.

[0098] In a second aspect, the present application shows an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the method according to any one of the preceding aspects.

[0099] In a third aspect, the present application shows a non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method according to any one of the preceding aspects.

[0100] In a fourth aspect, the present application shows a computer program product, when instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to perform the method according to any one of the preceding aspects.

[0101] In order to better illustrate the design principles of the present application, the following is illustrated by specific embodiments:

[0102] Embodiment 1;

[0103] For step two, each node generates a 6-dimensional node feature code, including the number of services passing through the current node, the total number of wavelengths or frequency slots occupied by the services passing through the current node, the spectral connectivity, the spectral connectivity block, the shortest path length to the current service endpoint, and the node label. The node label is a Boolean variable, indicating whether the node is a starting point or endpoint or has been in part of the solution. The spectral connectivity and the spectral connectivity block are calculated as follows:

[0104]

[0105] SAB n : average available spectrum block number of node n;

[0106] SC n : average spectral connectivity of node n;

[0107] count: total number of considered neighbor pairs;

[0108] c block (i, j): number of consecutive zero blocks between node n and neighbor i and j;

[0109] c simi (i, j): spectral similarity between node n and neighbor i and j;

[0110] For each link, a 6-dimensional state feature code is generated, recording the total number of available wavelengths or frequency slots, the starting position of the first large block of available wavelengths or frequency slots, the size of the largest continuous available wavelength or frequency slot, the number of available wavelengths or frequency slots meeting the requirements, the link fragmentation entropy, and whether there is a wavelength or frequency block meeting the conditions, the content includes:

[0111] ①Total Zeros (T z ): Total number of available wavelengths or frequency slots on the link.

[0112] ②First Large Zero Block Index (I fl ): Starting index position of the first available wavelength or frequency slot that meets the specified size on the link.

[0113] ③Max Consecutive Zeros (M c ): Maximum length of consecutive available wavelengths or frequency slots on the link.

[0114] ④Count of Large Zero Blocks (N b ): Number of available wavelengths or frequency slots that meet the specified size requirement on the link.

[0115] ⑤Entropy Metric (H): Calculate the fragmentation entropy of the link based on the distribution of wavelength or frequency slot occupation and idle state, which represents the degree of wavelength or frequency slot fragmentation of the link.

[0116] ⑥Has Large Zero Block (F b ): Whether there is at least one available wavelength or frequency slot that meets the specified condition on the link, which is used to quickly judge the availability of wavelength or frequency resource of the link.

[0117] It should be noted that for the method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, certain steps can be performed in other order or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification are all optional embodiments, and the actions involved are not necessarily required by the present application.

[0118] Optionally, the present application also provides an electronic device, which comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program is executed by the processor to implement various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, details are not described here.

[0119] The embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to realize each process of the method embodiment and achieve the same technical effects. To avoid repetition, details are not described herein. The computer readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0120] Figure 2 is a block diagram of an electronic device 800 shown in the present application. For example, the electronic device 800 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0121] Referring to Figure 2 The electronic device 800 can include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0122] The processing component 802 generally controls the overall operation of the electronic device 800 such as operations associated with displaying, making phone calls, data communications, camera operations, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to complete all or part of steps of the above methods. In addition, the processing component 802 can include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0123] The memory 804 is configured to store various types of data to support operations of the device 800; examples of these data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, images, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0124] The power component 806 provides power to the various components of the electronic device 800. The power component 806 can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0125] The multimedia component 808 includes a screen providing an output interface between the electronic device 800 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes the touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0126] The audio component 810 is configured to output and / or input an audio signal. For example, the audio component 810 includes a microphone (MIC) configured to receive an external audio signal when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting an audio signal.

[0127] The I / O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0128] The sensor component 814 includes one or more sensors for providing status assessments for various aspects of the electronic device 800. For example, the sensor component 814 can detect an open / closed position of the device 800, relative positioning of components, such as a display and a keypad of the electronic device 800, a change in position of the electronic device 800 or a component of the electronic device 800, presence or absence of user contact with the electronic device 800, orientation or acceleration / deceleration / g-force and temperature of the electronic device 800. The sensor component 814 can include an optical sensor for detecting ambient light, a proximity sensor for detecting nearby objects without any physical touch, a CMOS or CCD image sensor for use in imaging applications, or an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor in some embodiments.

[0129] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, a cellular network (e.g., 2G, 3G, 4G, or 5G), or a combination thereof. In an example embodiment, the communication component 816 receives broadcast signals or broadcast operation information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 816 can further include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) techniques, infrared data association (IrDA) techniques, ultra-wideband (UWB) techniques, Bluetooth (BT) techniques, and other techniques.

[0130] In an example embodiment, the electronic device 800 can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements for performing the above-described methods.

[0131] In an example embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 804 including instructions, is also provided, which can be executed by the processor 820 of the electronic device 800 to implement the above-described methods. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0132] Figure 3is a block diagram of an electronic device 1900 shown in the present application. For example, the electronic device 1900 can be provided as a server.

[0133] Referring to Figure 3 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932, for storing instructions, such as an application program, executable by the processing component 1922. The application program stored in the memory 1932 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-mentioned method.

[0134] The electronic device 1900 can also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

[0135] It should be noted that in this paper, the term "including", "containing" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0136] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-described specific embodiments, which are only illustrative, not limiting. Those skilled in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, which are all within the protection of the present application.

[0137] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0138] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0139] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0140] The functions can be realized in the form of software function units and sold or used as independent products when the functions are realized in the form of software function units and sold or used as independent products. Based on such understanding, the technical solutions of the present application can be embodied in the form of software product in essence or in the form of part of the technical solutions that make contributions to the prior art. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, and various program code storage media.

[0141] The above description is only the preferred embodiments of the present application, and it should be understood that the above embodiment description is only used to help understand the method of the present application and its core idea, and does not limit the protection scope of the present application. Any modification, equivalent replacement, etc. within the idea and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for large scale optical network resource allocation based on iterative learning, characterized in that, The application relates to a method for routing a current service request in an optical network. Step one, initializing an optical network topology, background service distribution and a current service request; Step two, compute the node feature code of all nodes and the link feature code of all links {v i},{e j} and the mask vector M according to the current optical network state, which together constitute the input state S I of the decision Agent. Step two is to calculate the node feature encoding of all nodes and the link feature encoding of all links{v i},{e j} and the mask vector M is as follows: according to the topological structure relationship, the background service distribution and the current service request state, dynamically update the node feature encoding v i of each node and the link feature encoding e j of each link; according to the topological situation, a mask vector M is calculated{v N is the current number of topological nodes; the agent refers to the agent of reinforcement learning; The node feature encoding v of each node i Each node has a K-dimensional node feature encoding N is the number of nodes in the current topology, wherein each dimension corresponds to a feature of a node The node feature code comprises the number of services passing through the current node, the total number of time slots occupied by the services passing through the current node, spectral connectivity, spectral connectivity blocks, the shortest path length from the current service terminal point and a node label; The link characteristic code e of each link j It means that each link has a L-dimensional link characteristic code E is the number of edges in the entire topology; wherein: each dimension corresponds to a state feature of a link, which is customized according to network state and service issuing demand; The link feature code comprises the total number of available wavelengths or frequency slots, the starting position of the first large block of available spectrum, the size of the largest continuous available spectrum block, link fragmentation entropy and whether there is a spectrum block meeting the condition; Step three, according to the input state S I , the agent filters invalid adjacent nodes through the action mask mechanism, selects one of the legal adjacent nodes as the action selection result, that is, the next hop route, and calculates the reward function value; Step four, repeating steps two and three until a termination condition of the current service is met, judging whether the current service can be successfully routed according to the termination condition, and selecting a wavelength or a frequency slot according to a first fit strategy if the current service can be successfully routed; Step five, the input state S generated in each iteration in the above process I , action selection results, reward function value are recorded for Agent training.

2. The method of claim 1, wherein the method is based on iterative learning. The network topology comprises the number of nodes, the number of links, a topology connection relationship and the number of available wavelengths or frequency slots on the links; the background service distribution refers to filling a certain number of random background services in the optical network, and the number of background services is determined by a network load rate; and the current service request comprises a source node, a destination node and bandwidth demand.

3. The method of claim 1, wherein the method is based on iterative learning. The dynamic updating refers to updating the v i , e j Whenever a new node joins the partial solution, the network state adjusts and updates the spectrum resource usage of each link according to the current service request; by calculating the common available wavelength or frequency slot of the links on the selected path, the v i , e j , ensures that the input state S I of the Agent reflects the latest network resource situation; The partial solution refers to a node sequence formed by the nodes selected by the decision agent according to the current service request, but the node sequence has not formed a complete route from the source node to the destination node.

4. The method of claim 1, wherein the method is based on iterative learning. The model in the Agent adopts an edge feature enhanced graph neural network structure, adopts a multi-layer edge feature enhanced graph attention network EGAT architecture to extract the complex coupling relationship between the link feature encoding and the node feature encoding, and capture the relationship and change rule between the input state S I , the action selection result and the reward function value.

5. The method of claim 4, wherein, In the multi-layer edge feature enhanced graph attention network EGAT, the link feature encoding e j and the node feature encoding v i are respectively input into the neural network for updating; Through the EGAT (Edge Feature Enhanced Graph Attention Network) of the multi-layer edge feature, the node features and the link features are iteratively updated, and the information of adjacent nodes and links is combined to optimize path selection.

6. The method of claim 1, wherein the method is based on iterative learning. The calculation formula of the reward function value R in step three is designed as follows: wherein: R is the reward value, the return in the current state; R terminal is the termination reward, a fixed reward value returned when the task is successfully completed or an exception occurs; D is the network diameter, representing the maximum length of the shortest path between any two nodes in the network; P is the penalty value, a penalty value calculated according to the partial spectrum usage; S is the step number, representing the number of steps the current task has taken; C is the condition value, calculating the relative progress of the current node to the target node, a positive value indicating that the target is approaching, and a negative value indicating that the target is far away. The calculation formula of the condition value C is designed as follows: C = μ · (γ · len(P last,dst ) - len(P current,dst )) Wherein, mu is a proportional factor or a scaling factor, which is used for adjusting the influence degree of the potential function; it controls the contribution degree of the path length difference to the reward; Y: is a constant used to adjust the weight of the path length difference, a discount factor used in reinforcement learning to control the long-term return; in this formula, it is used to weight the calculation of the shortest path length from the previous node to the target node; P current.dst : the shortest path length from the current node to the target node.

7. The method of claim 1, wherein the method is based on iterative learning. The termination condition in step four refers to reaching the terminal point, having no next hop node to be selected, and having no public available wavelength or frequency slot meeting the bandwidth demand of the service.

8. The method of claim 1, wherein the method is based on iterative learning. Step five further comprises that the agent continuously adjusts the decision through multiple rounds of training; after each training, the difference between the predicted action and the actual action is calculated by using a cross-entropy loss function, and the weights of the model are updated through a back propagation algorithm; in the training process, a learning rate scheduler is used to optimize the learning rate, and the convergence speed of the model is improved.

Images