Optical communication path design method and optical communication path design learning method

Abstract

This optical communication path design method includes: acquiring frequency slot information indicating the use state of a frequency slot in each link of an optical network; performing at least one of first weighting processing of each frequency slot in the frequency slot information for increasing the weight of data corresponding to a link included in a candidate for a route used by a desired optical communication path in an optical network among data indicating unused frequency slots, and second weighting processing of each link in the frequency slot information for increasing the weight of data corresponding to a frequency slot that can be allocated to a candidate for a route used by a desired optical communication path in the optical network among data indicating unused frequency slots; generating weighted frequency slot information; performing reinforcement learning using a feature amount based on the weighted frequency slot information as input data; generating a learning model; and determining the frequency slot and the route used by an optical communication path using the learning model.

Description

Optical communication path design method and optical communication path design learning method

[0001] The present invention relates to an optical communication path design method and an optical communication path design learning method.

[0002] Wavelength multiplexing technology enables high-capacity optical fiber communications by multiplexing multiple optical signals with different frequencies. In a wavelength multiplexing optical network constructed using wavelength multiplexing technology, it is necessary to appropriately determine the route to be taken and the optical frequency (frequency slot) to be used according to the communication traffic demand forecast (hereinafter referred to as "optical path demand"). The technology for determining such routes and optical frequencies is called optical path accommodation design technology. An optical path is a communication path generated by predetermining the forwarding destination for each wavelength at one or more optical cross-connects.

[0003] One method for realizing optical path accommodation design technology is to use reinforcement learning. In this method, the quality of optical path design results obtained in response to a certain optical path demand is repeatedly fed back to a reinforcement learning algorithm. This makes it possible to design optical paths with desired characteristics, such as minimizing the rejection rate.

[0004] Generally, neural networks are used in reinforcement learning to determine behavior based on the obtained environmental state. When applying a neural network to optical path design, frequency slot table information is input to the neural network as the state space. The frequency slot table information here refers to information that represents the optical path allocation status on the frequency axis for each link in the optical network. Then, information indicating the route and the optical frequency to be allocated is output from the neural network as the behavior space.

[0005] Masayuki Shimoda and Takafumi Tanaka, “Mask RSA: End-To-End Reinforcement Learning-based Routing and Spectrum Assignment in Elastic Optical Networks,” European Conference on Optical Communication (ECOC), Sep. 2021.

[0006] However, neural networks cannot distinguish between links that may be used and those that will not be used. Therefore, in the conventional optical path accommodation design technology described above, it is necessary to input information representing the usage status of the optical frequencies (frequency slots) of all links in the optical network into the neural network. This results in a huge number of input and output parameters for the neural network, which can make it difficult to perform machine learning efficiently. In particular, as the scale of optical networks increases, conventional optical path accommodation design technology faces the challenge of making little progress in machine learning, making it difficult to design optical paths with desired characteristics, such as minimizing the blocking rate.

[0007] In view of the above circumstances, the present invention aims to provide an optical communication path design method and an optical communication path design learning method that can improve the learning efficiency in machine learning required to design optical paths with desired characteristics and improve the accuracy of optical path accommodation design.

[0008] One aspect of the present invention is an optical communication path design method comprising: an acquisition step of acquiring frequency slot information indicating the usage status of frequency slots in each link of an optical network; a weighting step of performing at least one of a first weighting process for the frequency slot information, for each frequency slot, increasing a weight of data indicating unused frequency slots that corresponds to the links included in candidate routes used by a desired optical communication path in the optical network, among data indicating unused frequency slots, with respect to the frequency slot information; and a second weighting process for the frequency slot information, for each link, increasing a weight of data indicating unused frequency slots that corresponds to frequency slots that can be assigned to candidate routes used by the desired optical communication path in the optical network, with respect to the frequency slot information, to generate weighted frequency slot information; a learning step of performing reinforcement learning using features based on the weighted frequency slot information as input data to generate a learning model; and a determination step of determining the route and the frequency slot to be used by the optical communication path using the learning model.

[0009] Another aspect of the present invention is an optical communication path design learning method, comprising: an acquisition step of acquiring frequency slot information indicating the usage status of frequency slots in each link of an optical network, and information indicating a reward based on a design result of the optical communication path; a weighting step of executing at least one of a first weighting process for each frequency slot of the frequency slot information, increasing a weight of data indicating unused frequency slots that correspond to links included in candidate routes used by the desired optical communication path in the optical network, among data indicating unused frequency slots, with respect to the frequency slot information; and a second weighting process for each link of the frequency slot information, increasing a weight of data indicating unused frequency slots that correspond to frequency slots that can be assigned to candidate routes used by the desired optical communication path in the optical network, to generate weighted frequency slot information; and a learning step of inputting features based on the weighted frequency slot information and the reward as training data into a neural network, performing reinforcement learning, and generating a learning model.

[0010] The present invention makes it possible to improve the learning efficiency in machine learning required to design optical paths with desired characteristics, and to improve the accuracy of optical path accommodation design.

[0011] FIG. 1 is a diagram for explaining an overview of a general method for determining an optical path route and frequency slot using reinforcement learning. FIG. 2 is a block diagram showing the functional configuration of an optical path design learning device 1 in a first embodiment of the present invention. FIG. 3 is a block diagram showing the functional configuration of an optical path design device 6 in a first embodiment of the present invention. FIG. 4 is a diagram for explaining the flow of an optical path design method in a first embodiment of the present invention. FIG. 5 is a diagram for explaining the flow of an optical path design method in a third embodiment of the present invention. FIG. 6 is a diagram for explaining the flow of an optical path design method in a fourth embodiment of the present invention. FIG. 7 is a diagram for explaining the flow of an optical path design method in a fifth embodiment of the present invention. FIG. 8 is a diagram for explaining the flow of an optical path design method in a sixth embodiment of the present invention. FIG. 9 is a flowchart showing the operation of the optical path design learning device in each embodiment of the present invention. FIG. 10 is a flowchart showing the operation of the optical path design device in each embodiment of the present invention.

[0012] Hereinafter, an optical communication path design method and an optical communication path design learning method according to an embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions in each embodiment will be assigned the same reference numerals, and repeated explanations of those functions may be omitted.

[0013] First Embodiment An optical communication path design method and an optical communication path design learning method according to a first embodiment of the present invention will be described below.

[0014] First, we will explain a general method for using reinforcement learning to determine the route and frequency slot (optical frequency) of an optical path. Reinforcement learning is a machine learning technique that learns by repeating interactions between the environment and an agent. Figure 1 is a diagram for explaining an overview of a general method for determining the route and frequency slot of an optical path using reinforcement learning.

[0015] As shown in Fig. 1, the environment includes an optical network and optical path demands. Hereinafter, data indicating the state of the optical network and the optical path demands will be referred to as state S ｔ State S ｔis input to the agent. The agent includes a neural network that outputs an action according to the input of the feature vector. A state S representing the optical network and the optical path demand is ｔ The feature vector generated based on the data indicating the state S is input to the neural network. ｔ The behavior is determined according to the input of the feature vector generated based on the above. The behavior includes information indicating the path and frequency slot (optical frequency). Hereinafter, the data (vector) indicating the behavior is referred to as behavior A. ｔ The neural network is expressed as follows: ｔ Output.

[0016] In optical path design, a frequency slot table and vectors representing the start and end nodes of optical path demands are used as data representing the state of the optical network. Generally, a frequency slot table is data in which used frequency slots (optical frequencies) and unused frequency slots (optical frequencies) are expressed as binary values ​​of "0" and "1" (or "1" and "0"), respectively. Then, based on the vectors output from the neural network, the route and frequency slot (optical frequency) for the optical path demand are determined.

[0017] Action A, including the path and frequency slot determined in the agent ｔ is reflected in the environment. ｔ The optical network after the completion of the optical path design and the newly generated optical path demand are reflected in the state data S ｔ＋１ and is again input to the agent.

[0018] At this time, data indicating the reward, which indicates the quality of the lightpath design result, is also passed from the environment to the agent. Hereinafter, the data indicating the reward will be referred to as the reward R ｔ＋１ Then, based on the action taken in a certain state and the reward obtained as a result of that action, machine learning is performed by the agent, and the weights of the neural network parameter values ​​are updated.

[0019] The data indicating the reward can be designed arbitrarily. For example, by designing it so that a value of 1 is assigned when the lightpath allocation is successful and a value of 0 is assigned when the lightpath allocation is not successful, machine learning can be performed to increase the success rate of lightpath allocation.

[0020] [Configuration of the Optical Path Design Learning Device] An example of the functional configuration of the optical path design learning device 1 that executes processing related to learning an optical path design, which is the optical communication path design learning method according to the first embodiment, will be described below. The optical path design learning device 1 is a device that generates a large number of optical path demands for a virtual optical network in a simulation environment and executes neural network learning based on the optical path design results.

[0021] In this embodiment, the optical path design learning device 1 and the optical path design device 6 that executes the processing related to the optical path design described later are separate devices, but these devices may also be integrated into one device.

[0022] Fig. 2 is a block diagram showing the functional configuration of the optical path design learning device 1 in the first embodiment of the present invention. As shown in Fig. 2, the optical path design learning device 1 is configured to include an optical path design environment unit 2 and an optical path design agent unit 3. The optical path design environment unit 2 is a functional unit that executes processing in the "environment" shown in Fig. 1 described above, and the optical path design agent unit 3 is a functional unit that executes processing in the "agent" shown in Fig. 1 described above.

[0023] The optical path design environment unit 2 includes an optical path demand generation unit 21, an optical network status management unit 22, and a remuneration calculation unit 23. The optical path demand generation unit 21 simulates the generation of optical path demand. The optical network status management unit 22 holds and manages information such as topology information of the optical network and information indicating the allocation status of optical paths to optical fibers. The remuneration calculation unit 23 calculates remuneration based on the optical path design results.

[0024] The optical path design agent unit 3 includes an input data processing unit 31, a neural network 32, an output data processing unit 33, a state transition information storage unit 34, and a neural network update unit 35. The input data processing unit 31 includes a route calculation unit 311 and an input vector generation unit 312.

[0025] The input data processing unit 31 receives information (state S) output from the optical path design environment unit 2 indicating the optical path demand and the state of the optical network. ｔ ) is acquired. The input data processing unit 31 generates an input vector to be input to the neural network 32 based on the acquired optical path demand and information indicating the state of the optical network. The method of generating the input vector will be explained in detail later. The input data processing unit 31 inputs the generated input vector to the neural network 32. The input data processing unit 31 also acquires information indicating the state of the optical network (state S ｔ ) to the state transition information storage unit 34.

[0026] The neural network 32 generates an output vector in response to the input vector input from the input data processing unit 31. The neural network 32 outputs the generated output vector to the output data processing unit 33.

[0027] The output data processing unit 33 acquires the output vector output from the neural network 32. The output data processing unit 33 converts the acquired output vector into data indicating an action (action A ｔ The output data processing unit 33 outputs data indicating the behavior to the state transition information holding unit 34 and the optical path design environment unit 2.

[0028] The state transition information storage unit 34 stores data indicating the state of the optical network output from the input data processing unit 31 (state S ｔ The state transition information storage unit 34 also acquires data indicating the action output from the output data processing unit 33 (action A ｔ ) is acquired. The state transition information storage unit 34 also acquires data indicating the reward output from the optical path design environment unit 2 (reward R ｔThe state transition information storage unit 34 acquires data indicating the state of the optical network (state S ｔ ), data showing the behavior (behavior A ｔ ), and data indicating the reward (reward R ｔ The state transition information storage unit 34 stores data indicating a combination of the state of the optical network (state S ｔ ), data showing the behavior (behavior A ｔ ), and data indicating the reward (reward R ｔ ) combination to the neural network update unit 35.

[0029] The neural network update unit 35 updates the state of the optical network using the data (state S ｔ ), data showing the behavior (behavior A ｔ ), and data indicating the reward (reward R ｔ The neural network update unit 35 inputs the acquired data combinations to the neural network 32 as training data, and updates the weighting of the parameter values ​​of the neural network 32.

[0030] The following describes in more detail the flow of the machine learning process for optical path design by the optical path design learning device 1 of the first embodiment. First, the input data processing unit 31 generates vector data (input vectors) to be input to the neural network 32 based on the optical path demand generated by the optical path demand generation unit 21, the frequency slot table output from the optical network status management unit 22, and the topology information.

[0031] The route calculation unit 311 of the input data processing unit 31 searches for a route connecting a start node and an end node. The input vector generation unit 312 generates a via link vector based on the route searched by the route calculation unit 311. The input data processing unit 31 uses the generated via link vector to weight the data included in the frequency slot table.

[0032] The input data processing unit 31 inputs an input vector generated based on the weighted frequency slot table to the neural network 32. The output data processing unit 33 determines the route and frequency slot (optical frequency) of the optical path from the output vector output from the neural network 32.

[0033] The optical network status management unit 22 in the optical path design environment unit 2 updates the status of the optical network based on the optical path design results, and the remuneration calculation unit 23 calculates the remuneration.

[0034] The state transition information storage unit 34 of the optical path design agent unit 3 stores, as state transition information, a combination of information indicating the state of the optical network, information indicating actions performed on the state of the optical network, and information indicating rewards obtained as a result of the actions.

[0035] This optical path design process is repeatedly executed, and when the number of stored state transition information reaches a certain number, the neural network update unit 35 executes machine learning and updates the weighting of the neural network parameter values. By repeating this operation, the optical path design learning device 1 can obtain, as a learning result, a learning model that can design optical paths that can obtain more rewards.

[0036] [Configuration of Optical Path Design Device] An example of the functional configuration of the optical path design device 6 that executes processing related to optical path design, which is the optical communication path design method in the first embodiment, will be described below. The optical path design device 6 is a device that executes optical path design for actual optical path demands using a learning model generated by machine learning performed by the optical path design learning device 1 shown in FIG.

[0037] In this embodiment, the optical path design device 6 and the optical path design learning device 1 that executes the processing related to learning the optical path design described above are separate devices, but these devices may also be integrated devices.

[0038] 3 is a block diagram showing the functional configuration of the optical path design device 6 in the first embodiment of the present invention. As shown in Fig. 3, the optical path design device 6 includes an optical path demand acquisition unit 61, an optical network status management unit 62, an input data processing unit 63, a neural network 64, an output data processing unit 65, and an optical network control unit 66. The input data processing unit 63 includes a route calculation unit 631 and an input vector generation unit 632.

[0039] The optical path demand acquisition unit 61 acquires information indicating actual optical path demand that has occurred. The optical network status management unit 62 holds and manages topology information of the actual optical network, information indicating the allocation status of optical paths to optical fibers, etc. The topology of the actual optical network and the number of frequency slots of the optical fibers managed by the optical network status management unit 62 are the same as those of the simulated optical network managed by the optical network status management unit 22 of the optical path design learning device 1 shown in Figure 2 above. The information handled by the optical network status management unit 62 reflects the status of the actual optical network. The optical network control unit 66 controls the optical paths of the actual optical network.

[0040] The input data processing unit 63 receives the optical path demand (state S ｔ ), and information indicating the state of the optical network output from the optical network state management unit 62 (state S ｔ ) is acquired. The input data processing unit 63 generates an input vector to be input to the neural network 64 based on the acquired optical path demand and information indicating the state of the optical network. The method for generating the input vector will be explained in detail later. The input data processing unit 63 inputs the generated input vector to the neural network 64.

[0041] The neural network 64 generates an output vector in response to the input vector input from the input data processing unit 63. The neural network 64 outputs the generated output vector to the output data processing unit 65.

[0042] The output data processing unit 65 acquires the output vector output from the neural network 64. The output data processing unit 65 converts the acquired output vector into data indicating an action (action A ｔ The output data processing unit 65 outputs the data indicating the behavior to the optical network control unit 66.

[0043] The optical network control unit 66 receives the data indicating the action (action A) output from the output data processing unit 65. ｔ The optical network control unit 66 allocates optical paths to the actual optical network based on information indicating the route and frequency slot (optical frequency) included in the acquired data indicating the behavior.

[0044] The flow of processing for optical path design by the optical path design device 6 of the first embodiment will be explained in more detail below. First, the input data processing unit 63 generates vector data (input vector) to be input to the neural network 64 based on the optical path demand acquired by the optical path demand acquisition unit 61, the frequency slot table output from the optical network status management unit 62, and the topology information.

[0045] The route calculation unit 631 of the input data processing unit 63 searches for a route connecting a start node and an end node. The input vector generation unit 632 generates a via link vector based on the route searched by the route calculation unit 631. The input data processing unit 63 uses the generated via link vector to weight the data included in the frequency slot table.

[0046] An input data processing unit 63 inputs an input vector generated based on the weighted frequency slot table to a neural network 64. An output data processing unit 65 determines the route and frequency slot of the optical path from the output vector output from the neural network 64. An optical network control unit 66 assigns the optical path to an actual optical network based on the route and frequency slot (optical frequency) obtained as the optical path design result.

[0047] [Optical Path Design Method] The optical path design method according to the first embodiment will be described in more detail below, with specific examples. Note that the processing related to the optical path design method described below is executed by the input data processing unit 31 of the optical path design learning device 1 shown in Fig. 2 and the input data processing unit 63 of the optical path design device 6 shown in Fig. 3. Note that, although the following description will focus on the processing executed by the input data processing unit 31 of the optical path design learning device 1, the processing executed by the input data processing unit 63 of the optical path design device 6 is similar.

[0048] 4 is a diagram for explaining the flow of a method for designing optical paths in the first embodiment of the present invention. For example, as shown in FIG. 4, an optical network is assumed to be composed of six nodes, a through f, and seven links, 0 through 6. Furthermore, for example, a link is capable of wavelength division multiplexing communication accommodating up to 10 wavelengths, and if the bandwidth of the optical path is variable, one path can occupy multiple frequency slots, which are units of the frequency band occupied by the optical path.

[0049] In reinforcement learning for selecting an action based on the current state of the optical network, etc., a neural network is often used. A frequency slot table is used as information indicating the state of the optical network, which is an input parameter to such a neural network. The frequency slot table is data indicating the usage status of frequency slots (optical frequencies) for each link. As shown in Figure 4, for example, in the frequency slot table, a value of 0 is set to frequency slots that are in use by other optical paths, and a value of 1 is set to frequency slots that are available.

[0050] Here, assume that a demand for an optical path between node a and node d arises, and the route calculation unit 311 of the input data processing unit 31 selects links 0, 1, and 2 as the route of the optical path. Based on the determined route, the input vector generation unit 312 of the input data processing unit 31 generates a via link vector v that indicates the usage status of the via links.

[0051] 4, the input vector generation unit 312 generates a via link vector v by setting the values ​​of links 0, 1, and 2 to 1 and the values ​​of the other links to 0. The input vector generation unit 312 generates a via link vector v′ by applying the softplus function expressed by the following equation (1) to the generated via link vector v.

[0052] f(x)=ln(1+e ｘ ) ... (1)

[0053] Here, x represents a scalar value. As shown in Fig. 4, the via link vector v' generated by the SoftPlus function has the values ​​of links 0, 1, and 2 set to 1.3, and the values ​​of the other links set to 0.7.

[0054] The input vector generation unit 312 multiplies each value of the generated passed link vector v' as a scalar value in the row direction of the frequency slot table. With this configuration, the optical path design learning device 1 in the first embodiment can obtain a frequency slot table in which passed links are weighted, for input to the neural network. The input vector generation unit 312 inputs the generated frequency slot table to the neural network.

[0055] As a result, according to the optical path design method of the first embodiment, it is possible to improve the learning efficiency in machine learning required to design an optical path with desired characteristics, and to improve the accuracy of optical path accommodation design.

[0056] The input vector generation unit 312 may input the values ​​of the frequency slot table in which the passed links are weighted to the neural network in any manner. For example, the input vector generation unit 312 may convert the frequency slot table in which the passed links are weighted to a one-dimensional vector before inputting it to the neural network. Alternatively, the input vector generation unit 312 may input the values ​​of the frequency slot table in which the passed links are weighted to the convolutional neural network.

[0057] The softplus function used to generate the via link vector v' is intended to prevent information loss by multiplying non-via links in the frequency slot table by 0. Therefore, instead of the softplus function, another function having an equivalent function may be used.

[0058] Second Embodiment An optical communication path design method and an optical communication path design learning method according to a second embodiment of the present invention will now be described. Block diagrams showing the functional configurations of the optical path design learning device and the optical path design device according to the second embodiment are similar to the block diagrams showing the functional configurations of the optical path design learning device 1 and the optical path design device 6 according to the first embodiment described above.

[0059] The optical path design learning device and optical path design device in the second embodiment and the optical path design learning device 1 and optical path design device 6 in the first embodiment described above have different configurations of their input data processing units. Hereinafter, the input data processing unit of the optical path design learning device in the second embodiment will be referred to as the "input data processing unit 31a," and the input data processing unit of the optical path design device in the second embodiment will be referred to as the "input data processing unit 63a."

[0060] [Optical Path Design Method] The optical path design method according to the second embodiment will be described in more detail below, with specific examples. Note that the processing related to the optical path design method described below is executed by the input data processing unit 31a of the optical path design learning device and the input data processing unit 63a of the optical path design device 1. Note that although the following description will focus on the processing executed by the input data processing unit 31a of the optical path design learning device, the processing executed by the input data processing unit 63a of the optical path design device 6 is similar.

[0061] 5 is a diagram for explaining the flow of a method for designing optical paths in a second embodiment of the present invention. For example, as shown in FIG. 5, an optical network is assumed to be composed of six nodes, a through f, and seven links, 0 through 6. Furthermore, for example, a link is capable of wavelength division multiplexing communication accommodating up to 10 wavelengths, and if the bandwidth of the optical path is variable, one path can occupy multiple frequency slots, which are units of the frequency band occupied by the optical path.

[0062] As shown in FIG. 5, for example, in the frequency slot table, a value of 0 is set to a frequency slot that is in use by another optical path, and a value of 1 is set to a frequency slot that is available.

[0063] Now, let us assume that a demand for an optical path between node a and node c arises, and the route calculation unit of the input data processing unit 31a selects route candidate 1 and route candidate 2 as candidates for the optical path. As shown in Fig. 5, route candidate 1 is a route that passes through links 0 and 1, and route candidate 2 is a route that passes through links 5 and 6. These links 0, 1, 5, and 6 are links that may be used.

[0064] The input vector generation unit of the input data processing unit 31a (hereinafter referred to as the "input vector generation unit 312a") generates a via link vector v indicating the usage status of the via links based on the two selected route candidates.

[0065] 5, the input vector generation unit 312a generates a via link vector v by setting the values ​​of links 0, 1, 5, and 6 to 1 and the values ​​of the other links to 0. The input vector generation unit 312a generates a via link vector v' by applying the softplus function expressed by the above-mentioned equation (1) to the generated via link vector v.

[0066] As shown in FIG. 5, the via link vector v' generated by the SoftPlus function has the values ​​of links 0, 1, 5, and 6 set to 1.3, and the values ​​of the other links set to 0.7.

[0067] The input vector generation unit 312a multiplies each value of the generated passed link vector v' by a row direction of the frequency slot table as a scalar value. With this configuration, the optical path design learning device in the second embodiment can obtain a frequency slot table in which passed links are weighted to be input to the neural network while maintaining the correspondence between the route candidates and the frequency slot table. The input vector generation unit 312a inputs the generated frequency slot table to the neural network.

[0068] As a result, according to the optical path design method of the second embodiment, it is possible to improve the learning efficiency in machine learning required to design an optical path with desired characteristics, and to improve the accuracy of optical path accommodation design.

[0069] <Third embodiment> An optical communication path design method and an optical communication path design learning method according to a third embodiment of the present invention will be described below. Block diagrams showing the functional configurations of the optical path design learning device and the optical path design device according to the third embodiment are similar to the block diagrams showing the functional configurations of the optical path design learning device 1 and the optical path design device 6 according to the first embodiment described above.

[0070] The optical path design learning device and optical path design device in the third embodiment and the optical path design learning device 1 and optical path design device 6 in the first embodiment described above have different configurations of their input data processing units. Hereinafter, the input data processing unit of the optical path design learning device in the third embodiment will be referred to as the "input data processing unit 31b," and the input data processing unit of the optical path design device in the second embodiment will be referred to as the "input data processing unit 63b."

[0071] [Optical Path Design Method] The optical path design method according to the third embodiment will be described in more detail below, with specific examples. Note that the processing related to the optical path design method described below is executed by the input data processing unit 31b of the optical path design learning device and the input data processing unit 63b of the optical path design device. Note that, although the following description will focus on the processing executed by the input data processing unit 31b of the optical path design learning device, the processing executed by the input data processing unit 63b of the optical path design device 6 is similar.

[0072] 6 is a diagram for explaining the flow of a method for designing optical paths in a third embodiment of the present invention. For example, as shown in FIG. 6, an optical network is assumed to be composed of six nodes, a through f, and seven links, 0 through 6. Furthermore, for example, a link is capable of wavelength division multiplexing communication accommodating up to 10 wavelengths, and if the bandwidth of the optical path is variable, one path can occupy multiple frequency slots, which are units of the frequency band occupied by the optical path.

[0073] As shown in FIG. 6, for example, in the frequency slot table, a value of 0 is set to a frequency slot that is in use by another optical path, and a value of 1 is set to a frequency slot that is available.

[0074] Now, let us assume that a demand for an optical path between node a and node d arises, and the route calculation unit of the input data processing unit 31b selects route candidate 1 and route candidate 2 as candidates for the optical path. As shown in Fig. 6, route candidate 1 is a route that passes through links 0, 1, and 2, and route candidate 2 is a route that passes through links 5, 6, and 2. These links 0, 1, 2, 5, and 6 are links that may be used.

[0075] The input vector generation unit of the input data processing unit 31b (hereinafter referred to as the "input vector generation unit 312b") generates a via link vector indicating the usage status of the links via each of the two selected route candidates. Hereinafter, the via link vector indicating the usage status of the links via route candidate 1 will be referred to as the "via link vector v １” and the via link vector indicating the usage status of the links via the route candidate 2 is represented as “via link vector v ２ " is expressed as ".

[0076] As shown in FIG. 6, the input vector generation unit 312b generates a via link vector v １ 6, the input vector generation unit 312b generates a via link vector v ２ Generate.

[0077] The input vector generation unit 312b generates the passed link vector v １ By applying the softplus function expressed by the above equation (1) to １ As shown in FIG. 6, the via link vector v １ ' has the values ​​of links 0, 1, and 2 set to 1.3, and the values ​​of the other links set to 0.7.

[0078] The input vector generation unit 312b generates the passed link vector v ２ By applying the softplus function expressed by the above equation (1) to ２ As shown in FIG. 6, the via link vector v ２ In the example, the values ​​of links 5, 6, and 2 are set to 1.3, and the values ​​of the other links are set to 0.7.

[0079] The input vector generation unit 312b generates the passed link vector v １ Similarly, the input vector generation unit 312b multiplies the generated passed link vector v ２ Each value of ' is treated as a scalar value and multiplied in the row direction of the frequency slot table.

[0080] As a result, two weighted frequency slot tables are generated. The input vector generator 312b inputs the two generated frequency slot tables to the neural network.

[0081] As a result, according to the optical path design method of the third embodiment, it is possible to improve the learning efficiency in machine learning required to design an optical path with desired characteristics, and to improve the accuracy of optical path accommodation design.

[0082] Furthermore, by having such a configuration, the optical path design learning device in the third embodiment can obtain a frequency slot table in which the via links are weighted for input to the neural network while maintaining the correspondence between the route candidates and the frequency slot table.

[0083] <Fourth embodiment> An optical communication path design method and an optical communication path design learning method according to a fourth embodiment of the present invention will be described below. Block diagrams showing the functional configurations of the optical path design learning device and the optical path design device according to the fourth embodiment are similar to the block diagrams showing the functional configurations of the optical path design learning device 1 and the optical path design device 6 according to the first embodiment described above.

[0084] The optical path design learning device and optical path design device in the fourth embodiment and the optical path design learning device 1 and optical path design device 6 in the first embodiment described above have different configurations of their input data processing units. Hereinafter, the input data processing unit of the optical path design learning device in the fourth embodiment will be referred to as the "input data processing unit 31c," and the input data processing unit of the optical path design device in the second embodiment will be referred to as the "input data processing unit 63c."

[0085] [Optical Path Design Method] The optical path design method according to the fourth embodiment will be described in more detail below, with specific examples. Note that the processing related to the optical path design method described below is executed by the input data processing unit 31c of the optical path design learning device and the input data processing unit 63c of the optical path design device. Note that, although the following description will focus on the processing executed by the input data processing unit 31c of the optical path design learning device, the same applies to the processing executed by the input data processing unit 63c of the optical path design device 6.

[0086] 7 is a diagram for explaining the flow of a method for designing optical paths in a fourth embodiment of the present invention. For example, as shown in FIG. 7, an optical network is assumed to be composed of six nodes, a through f, and seven links, 0 through 6. Furthermore, for example, a link is capable of wavelength division multiplexing communication accommodating up to 10 wavelengths, and if the bandwidth of the optical path is variable, one path can occupy multiple frequency slots, which are units of the frequency band occupied by the optical path.

[0087] As shown in FIG. 7, for example, in the frequency slot table, a value of 0 is set to a frequency slot that is in use by another optical path, and a value of 1 is set to a frequency slot that is available.

[0088] Now, let us assume that a demand for an optical path between node a and node d arises, and the route calculation unit of the input data processing unit 31c selects route candidate 1 and route candidate 2 as candidates for the optical path. As shown in Fig. 7, route candidate 1 is a route that passes through links 0, 1, and 2, and route candidate 2 is a route that passes through links 5, 6, and 2. These links 0, 1, 2, 5, and 6 are links that may be used.

[0089] The input vector generation unit of the input data processing unit 31c (hereinafter referred to as "input vector generation unit 312c") generates a via link vector indicating the usage status of the links via each of the two selected route candidates. Hereinafter, the via link vector indicating the usage status of the links via route candidate 1 will be referred to as the "via link vector v １” and the via link vector indicating the usage status of the links via the route candidate 2 is represented as “via link vector v ２ " is expressed as ".

[0090] As shown in FIG. 7, the input vector generation unit 312c generates a via link vector v １ 7, the input vector generation unit 312c generates a via link vector v ２ Generate.

[0091] The input vector generation unit 312c generates the via link vector v １ and the via link vector v ２ The link vector v is added to the link vector v. １ and the link vector v ２ The via link vector generated by adding and is called the "via link vector v." As shown in Figure 7, in the via link vector v, the values ​​of links 0, 1, 5, and 6 are set to 1, the value of link 3 is set to 2, and the values ​​of the other links are set to 0.

[0092] The input vector generation unit 312c generates a via link vector v′ by applying the softplus function expressed by the above-mentioned equation (1) to the generated via link vector v. As shown in FIG. 7, the via link vector v′ generated by the softplus function １ In the example, the values ​​of links 0, 1, 5, and 6 are set to 1.3, the value of link 3 is set to 2.1, and the values ​​of the other links are set to 0.7.

[0093] The input vector generation unit 312c multiplies each value of the generated via link vector v' as a scalar value in the row direction of the frequency slot table. This generates a weighted frequency slot table. The input vector generation unit 312c inputs the generated frequency slot table to the neural network.

[0094] As a result, according to the optical path design method of the fourth embodiment, it is possible to improve the learning efficiency in machine learning required to design an optical path with desired characteristics, and to improve the accuracy of optical path accommodation design.

[0095] Furthermore, by having such a configuration, the optical path design learning device in the fourth embodiment can obtain a frequency slot table in which passed links are weighted for input to the neural network while maintaining the correspondence between route candidates and frequency slot tables. Furthermore, the optical path design learning device in the fourth embodiment can further reduce the data size of input parameters input to the neural network compared to the optical path design learning device in the third embodiment.

[0096] Fifth Embodiment An optical communication path design method and an optical communication path design learning method according to a fifth embodiment of the present invention will now be described. Block diagrams showing the functional configurations of the optical path design learning device and the optical path design device according to the fifth embodiment are similar to the block diagrams showing the functional configurations of the optical path design learning device 1 and the optical path design device 6 according to the first embodiment described above.

[0097] The optical path design learning device and optical path design device in the fifth embodiment and the optical path design learning device 1 and optical path design device 6 in the first embodiment described above have different configurations of their input data processing units. Hereinafter, the input data processing unit of the optical path design learning device in the fifth embodiment will be referred to as the "input data processing unit 31d," and the input data processing unit of the optical path design device in the second embodiment will be referred to as the "input data processing unit 63d."

[0098] [Optical Path Design Method] The optical path design method according to the fifth embodiment will be described in more detail below, with specific examples. Note that the processing related to the optical path design method described below is executed by the input data processing unit 31 of the optical path design learning device 1 shown in Fig. 2 and the input data processing unit 63 of the optical path design device 6 shown in Fig. 3. Note that, although the following description will focus on the processing executed by the input data processing unit 31d of the optical path design learning device, the same applies to the processing executed by the input data processing unit 63d of the optical path design device 6.

[0099] 8 is a diagram for explaining the flow of a method for designing optical paths in a fifth embodiment of the present invention. For example, as shown in Fig. 8, an optical network is assumed to be composed of six nodes, a through f, and seven links, 0 through 6. Furthermore, for example, a link is capable of wavelength division multiplexing communication accommodating up to 10 wavelengths, and if the bandwidth of the optical path is variable, one path can occupy multiple frequency slots, which are units of the frequency band occupied by the optical path.

[0100] Assume now that a demand for an optical path arises between node a and node d, and the route calculation unit of input data processing unit 31d selects links 0, 1, and 2 as the route of the optical path. The input vector generation unit of input data processing unit 31d (hereinafter referred to as input vector generation unit 312d) generates a vector indicating frequency slots in which all of links 0, 1, and 2 can be used.

[0101] 8, for example, the frequency slots in which the values ​​of links 0, 1, and 2 are all 1 are 0, 1, 4, 5, and 6. Therefore, the input vector generation unit 312d generates a vector in which the values ​​of frequency slots 0, 1, 4, 5, and 6 are set to 1, and the values ​​of the other frequency slots are set to 0.

[0102] Next, the input vector generation unit 312d identifies the number of frequency slots required to accommodate the optical path demand, and calculates a vector indicating the leading positions of frequency slots to which that number can be assigned. Hereinafter, this vector will be referred to as the "allocation candidate slot vector v ｓ"

[0103] For example, it is assumed that the number of frequency slots required to accommodate the optical path demand is two. In this case, as shown in FIG. 8, the leading position of two consecutive frequency slots that can be allocated is either frequency slot 0 (when frequency slots 0 and 1 are allocated), 4 (when frequency slots 4 and 5 are allocated), or 5 (when frequency slots 5 and 6 are allocated). Therefore, the input vector generation unit 312d generates an allocation candidate slot vector v in which the values ​​of frequency slots 0, 4, and 5 are set to 1 and the values ​​of the other frequency slots are set to 0. ｓ Generate.

[0104] The input vector generation unit 312d generates the allocation candidate slot vector v ｓ By applying the softplus function expressed by the above equation (1) to the allocation candidate slot vector v ｓ As shown in FIG. 8, the allocation candidate slot vector v ｓ ' has the values ​​of links 0, 4, and 5 set to 1.3, and the values ​​of the other links set to 0.7.

[0105] The input vector generation unit 312d generates the allocation candidate slot vector v ｓ Each value of ' is treated as a scalar value and multiplied in the column direction of the frequency slot table. With this configuration, the optical path design learning device in the fifth embodiment can obtain a frequency slot table in which available frequency slots are weighted for input to the neural network. The input vector generation unit 312d inputs the generated frequency slot table to the neural network.

[0106] As a result, according to the optical path design method of the fifth embodiment, it is possible to improve the learning efficiency in machine learning required to design an optical path with desired characteristics, and to improve the accuracy of optical path accommodation design.

[0107] The input vector generation unit 312d may input the values ​​of the frequency slot table in which the available frequency slots are weighted to the neural network in any manner. For example, the input vector generation unit 312d may convert the frequency slot table in which the available frequency slots are weighted to a one-dimensional vector before inputting it to the neural network. Alternatively, the input vector generation unit 312d may input the values ​​of the frequency slot table in which the available frequency slots are weighted to the convolutional neural network.

[0108] In addition, the allocation candidate slot vector v ｓ The softplus function used to generate ' is intended to prevent information loss by multiplying by 0 links in the frequency slot table that are not passed through. Therefore, instead of the softplus function, other functions having equivalent functionality may be used.

[0109] <Sixth embodiment> An optical communication path design method and an optical communication path design learning method according to a sixth embodiment of the present invention will be described below. Block diagrams showing the functional configurations of the optical path design learning device and the optical path design device according to the sixth embodiment are similar to the block diagrams showing the functional configurations of the optical path design learning device 1 and the optical path design device 6 according to the first embodiment described above.

[0110] The optical path design learning device and optical path design device in the sixth embodiment and the optical path design learning device 1 and optical path design device 6 in the first embodiment described above have different configurations of their input data processing units. Hereinafter, the input data processing unit of the optical path design learning device in the sixth embodiment will be referred to as the "input data processing unit 31e," and the input data processing unit of the optical path design device in the second embodiment will be referred to as the "input data processing unit 63e."

[0111] [Optical Path Design Method] The optical path design method according to the sixth embodiment will be described in more detail below, with specific examples. Note that the processing related to the optical path design method described below is executed by the input data processing unit 31 of the optical path design learning device 1 shown in Fig. 2 and the input data processing unit 63 of the optical path design device 6 shown in Fig. 3. Note that, although the following description will focus on the processing executed by the input data processing unit 31e of the optical path design learning device, the same applies to the processing executed by the input data processing unit 63e of the optical path design device 6.

[0112] The optical path design method in the sixth embodiment is a method that combines the optical path design method in the fifth embodiment shown in FIG. 8 described above and the optical path design method in the first embodiment shown in FIG. 4 described above.

[0113] 9 is a diagram for explaining the flow of a method for designing an optical path in the sixth embodiment of the present invention. For example, as shown in FIG. 9, first, the input data processing unit 31e calculates the generated allocation candidate slot vector v ｓ Next, the input data processing unit 31e multiplies each of the generated passed link vectors v′ as scalar values ​​in the row direction of the frequency slot table.

[0114] In addition, the allocation candidate slot vector v ｓ The order may be reversed, that is, the value of the via link vector v' is multiplied first, and then the value of the via link vector v' is multiplied (i.e., the order may be reversed).

[0115] With this configuration, the optical path design learning device in the sixth embodiment can obtain a frequency slot table that is weighted in both the link direction and the frequency slot direction. The input vector generation unit 312e inputs the generated frequency slot table to the neural network.

[0116] As a result, according to the optical path design method of the sixth embodiment, it is possible to improve the learning efficiency in machine learning required to design an optical path with desired characteristics, and to improve the accuracy of optical path accommodation design.

[0117] (Operation of the Optical Path Design Learning Apparatus) An example of the operation of the optical path design learning apparatus in each of the above-described embodiments will now be described. Fig. 10 is a flowchart showing the operation of the optical path design learning apparatus in each of the embodiments of the present invention.

[0118] First, the optical path design learning device acquires frequency slot information indicating the usage status of frequency slots in each link of the optical network and information indicating rewards (step S01). The frequency slot information is, for example, the frequency slot table in each of the above-mentioned embodiments, and the reward is, for example, the reward R ｔ is.

[0119] Next, the optical path design learning device executes at least one of the first weighting process and the second weighting process (step S12).

[0120] The first weighting process is a process for increasing the weight of data indicating unused frequency slots for each frequency slot in the frequency slot information, the data corresponding to links included in candidate routes used by desired optical communication paths in the optical network. The first weighting process is, for example, a weighting process performed by the optical path design learning device in the first to fourth embodiments shown in FIGS.

[0121] The second weighting process is a process for increasing the weight of data indicating unused frequency slots in the frequency slot information for each link, the data corresponding to frequency slots that can be assigned to candidate routes used by desired optical communication paths in the optical network. The second weighting process is, for example, the weighting process performed by the optical path design learning device in the fifth embodiment shown in FIG. 8 above.

[0122] The configuration for performing both the first weighting process and the second weighting process corresponds to the configuration for the weighting process by the optical path design learning device in the sixth embodiment shown in, for example, the above-mentioned Fig. 9. The optical path design device generates weighted frequency slot information.

[0123] Next, the optical path design learning device inputs the feature quantities based on the weighted frequency slot information and the reward as training data into the neural network, performs reinforcement learning, and generates a learning model (step S03). The feature quantities are, for example, the input vectors in each of the above-mentioned embodiments. This completes the operation of the optical path design learning device shown in the flowchart of FIG. 11.

[0124] (Operation of the Optical Path Design Apparatus) An example of the operation of the optical path design apparatus in each of the above-described embodiments will now be described. Fig. 11 is a flowchart showing the operation of the optical path design apparatus in each of the embodiments of the present invention.

[0125] First, the optical path design device acquires frequency slot information indicating the usage status of frequency slots in each link of the optical network (step S11).

[0126] Next, the optical path design apparatus executes at least one of the first weighting process and the second weighting process (step S12).

[0127] The first weighting process is a process for increasing the weight of data indicating unused frequency slots for each frequency slot in the frequency slot information, the data corresponding to links included in candidate routes used by the desired optical communication path in the optical network.

[0128] The second weighting process is a process of increasing the weight of data indicating unused frequency slots in the frequency slot information for each link, the data corresponding to frequency slots that can be assigned to candidate routes used by desired optical communication paths in the optical network. The optical path design device generates weighted frequency slot information.

[0129] Next, the optical path design device determines the route and frequency slot (optical frequency) to be used by the optical communication path using the learning model acquired from the optical path design learning device (step S13). This completes the operation of the optical path design device shown in the flowchart of FIG.

[0130] The optical communication path design device (optical path design device 6 in the embodiment) and the optical communication path design learning device (optical path design learning device 1 in the embodiment) of the present invention can also be realized by a computer and a program, and the program can be recorded on a storage medium or provided via a network.

[0131] Part or all of the configuration of the optical path design learning device 1 and the optical path design device 6 in the above-described embodiments may be implemented by a computer. In this case, a program for implementing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into a computer system and executed to implement the function. Note that the term "computer system" as used herein includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Furthermore, the term "computer-readable recording medium" may also include devices that dynamically store programs for a short period of time, such as communication lines when transmitting programs via networks such as the Internet or communication lines such as telephone lines, or devices that store programs for a certain period of time, such as volatile memory within a computer system that serves as a server or client in such cases. The program may be for implementing part of the above-described functions, or may be capable of implementing the above-described functions in combination with a program already recorded in the computer system, or may be implemented using a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0132] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope that do not deviate from the gist of the present invention.

[0133] 1...optical path design learning device, 2...optical path design environment unit, 3...optical path design agent unit, 4...frequency slot, 6...optical path design device, 21...optical path demand generation unit, 22...optical network state management unit, 23...reward calculation unit, 31, 31a to 31e...input data processing unit, 32...neural network, 33...output data processing unit, 34...state transition information holding unit, 35...neural network update unit, 61...optical path demand acquisition unit, 62...optical network state management unit, 63, 63a to 63e...input data processing unit, 64...neural network, 65...output data processing unit, 66...optical network control unit, 311...route calculation unit, 312, 312a to 312e...input vector generation unit, 631...route calculation unit, 632...input vector generation unit

Claims

1. An optical communication path design method comprising: an acquisition step of acquiring frequency slot information indicating the usage status of frequency slots in each link of an optical network; a weighting step of executing at least one of a first weighting process for the frequency slot information, for each frequency slot, increasing a weight of data indicating unused frequency slots, the data corresponding to the links included in candidate routes used by a desired optical communication path in the optical network, among data indicating unused frequency slots, for each frequency slot of the frequency slot information, and a second weighting process for increasing a weight of data corresponding to frequency slots that can be assigned to candidate routes used by the desired optical communication path in the optical network, among data indicating unused frequency slots, for each link of the frequency slot information, to generate weighted frequency slot information; a learning step of performing reinforcement learning using features based on the weighted frequency slot information as input data, to generate a learning model; and a determination step of determining the route and frequency slot to be used by the optical communication path using the learning model.

2. The optical communication path design method according to claim 1, wherein, in the weighting step, when there are multiple candidates for the route used by the desired optical communication path and the first weighting process is performed, multiple pieces of frequency slot information are generated, each of which has been subjected to the first weighting process based on the links included in each of the multiple candidate routes, or one piece of frequency slot information is generated, each of which has been subjected to the first weighting process based on the links included in all of the candidate routes.

3. The optical communication path design method according to claim 1 or 2, wherein in the weighting step, at least one of the first weighting process and the second weighting process is performed using a softplus function.

4. An optical communication path design learning method comprising: an acquisition step of acquiring frequency slot information indicating the usage status of frequency slots in each link of an optical network, and information indicating a reward based on a design result of an optical communication path; a weighting step of executing at least one of a first weighting process for the frequency slot information, for each frequency slot, increasing a weight of data indicating an unused frequency slot, the data corresponding to the link included in a candidate route used by the desired optical communication path in the optical network, among data indicating an unused frequency slot, for each frequency slot of the frequency slot information, and a second weighting process for increasing a weight of data indicating an unused frequency slot, the data corresponding to the frequency slot that can be assigned to a candidate route used by the desired optical communication path in the optical network, among data indicating an unused frequency slot, for each link of the frequency slot information, to generate weighted frequency slot information; and a learning step of inputting features based on the weighted frequency slot information and the reward into a neural network as teacher data, performing reinforcement learning, and generating a learning model.

Images