Integrated control device, integrated control system, integrated control method and program

Abstract

An integrated control device in the present disclosure performs an integrated control on a plurality of control engines for constructing a plurality of different virtual networks on a physical network. The integrated control device comprises: an information distribution unit which receives, from the plurality of control engines, control requests pertaining to the plurality of virtual networks that are under the jurisdiction of the control engines by acquiring NFV environmental information that indicates a condition of the physical network and distributing the acquired information to the plurality of control engines; a comprehensive evaluation estimation unit which aggregates the total number of control commands in the control requests for the current condition of the physical network, which are requested by the plurality of control engines, on the basis of the control requests received from the plurality of control engines; a control approval destination learning unit which uses reinforcement learning and updates a behavior value function for control approval determination so that the total number of the control commands, which is aggregated by the comprehensive evaluation estimation unit, decreases in comparison to the previous aggregation; and a control application determination unit which transmits, to the physical network, a prescribed control request received from a prescribed control engine among the plurality of control engines when an evaluation for the control approval determination changes.

Description

[Technical Field]

[0001] The present disclosure relates to a technology for integrated control of multiple control engines for constructing multiple virtual networks for different purposes on a physical network (network infrastructure = NFV environment). [Background technology]

[0002] Each service on the network is operated by a control algorithm designed or provided by a service provider or network operator (communications carrier).

[0003] In the future, for example, as shown in Fig. 6, it is conceivable that, for a data transmission path between user terminal 330 and destination server 310z, a virtual network (control algorithm aimed at minimizing path length) operated by one service provider and a virtual network (control algorithm aimed at balancing facility load) operated by another service provider will be independently constructed on a single physical network operated by a network operator. In this case, in the control algorithm aimed at minimizing path length, server 310a constructs a virtual network without considering the facility load status of each node (transfer device 320, servers 310x, 310y), etc. On the other hand, in the control algorithm aimed at balancing facility load, server 310b constructs a virtual network to level the facility load status of each node, etc., without considering the required quality.

[0004] However, when multiple virtual networks for different purposes are constructed on a single physical network, conflicts in network control may occur, as shown in Fig. 7. For example, as shown in Fig. 7, consider a situation in which the data transfer capacity of each of nodes (transfer devices) 320a to 320e is "10," and server 310b, which employs a control algorithm aimed at equipment load balancing, first distributes the data capacity of "10" into data capacities of "3," "3," and "4," and transfers the data to user terminal 330b. In this case, if server 310a, which employs a control algorithm aimed at minimizing path length, subsequently transfers the data capacity of "10" as is without distributing it, none of nodes 320a to 320e will have a remaining data transfer capacity of "10," resulting in a problem of insufficient resources when attempting to construct a virtual network for transferring data to user terminal 330a.

[0005] In contrast, the above problem can be solved by coordinating network control as shown in Fig. 8. For example, as shown in Fig. 8, even if the same physical network as Fig. 7 is used, if server 310a performs a data transfer of a data capacity of "10" using nodes 320a and 320d, and then server 310b constructs a virtual network using nodes 320b, 320c, and 320e that were not used by server 310a, the data capacity of "10" can be divided into data capacities of "5" and "5" for data transfer, and each objective can be achieved.

[0006] Therefore, in order to realize the coordination of network control, the technology shown in Non-Patent Document 1 has been proposed. Here, conventional virtual network control will be explained using Fig. 9 and Fig. 10. Fig. 9 is a diagram showing the overall configuration of a conventional network. Fig. 10 is a diagram showing an example of the functional configuration of a conventional integrated control engine 200.

[0007] As shown in FIG. 9 , the integrated control system 10 is connected to a physical network 300 as a network infrastructure. The physical network 300 is a network environment including a general-purpose server 310, a node (transfer device) 320, and a user terminal 330. The integrated control system 100 is one or more computers having an integrated control engine 200. The integrated control system 10 controls a virtual network on the physical network 300 using the integrated control engine. The integrated control system 10 acquires network information indicating the network state (status) from the physical network 300 via the information collection and storage device 30, and transmits the control results and setting commands calculated by the integrated control engine 200 to each node 320 of the physical network via the orchestrator 50, thereby achieving coordination of network control.

[0008] Specifically, as shown in FIG. 10 , the integrated control engine 200 allocates virtual resources and virtual functions to the optimal general-purpose server 310 and calculates the optimal route based on NFV environment information (network configuration information, user demand, network observation information (traffic information, server usage information, etc.)). The integrated control engine 200 is realized by an information sharing engine 400 for sharing information between various modularized control engines e100 (e101 to e105, etc.), assuming cooperation with the various control engines. The information sharing engine 400 includes a learning instruction agent 410, each control agent 430 (431, 432, 433, 434, 435, etc.), a comprehensive evaluation calculation unit 450, and an input / output conversion unit 460.

[0009] Of these, the learning instruction agent 410 learns the selection of each control agent based on the overall evaluation. Each control agent 430 learns to change the solution of each control engine e100 based on the overall evaluation. The overall evaluation calculation unit 450 calculates a reward, which is the overall evaluation, from the evaluation of each control engine e100. The input / output conversion unit 460 converts the format of input and output of each control engine e100. The input / output conversion unit 460 formulates each combination of each control engine e100.

[0010] In this way, in Non-Patent Document 1, the information sharing engine 400 collects the various pieces of information described above from each control engine e100, recognizes how each solution is desired to be controlled, and then modifies the solution while evaluating the objective function of each control algorithm of each control engine. Next, the information sharing engine 400 sends this modified solution to each control engine e100, which then determines a new solution. The information sharing engine 400 repeats the process of collecting the various pieces of information described above from each control engine e100 again, thereby determining a final solution. However, Non-Patent Document 1 assumes that each control algorithm held by each control engine e100 discloses a control policy (e.g., improving quality, improving resource efficiency) and constraints (e.g., service provision conditions, link capacity, server capacity). [Prior art documents] [Non-patent literature]

[0011] [Non-Patent Document 1] Akito Suzuki, Ryoichi Kawahara, Masahiro Kobayashi, Yousuke Takahashi, Shigeaki Harada, and Keisuke Ishibashi, "Extendable NFV-Integrated Control Method Using Reinforcement Learning," IEICE Trans. Communications, vol. E103-B, no. 8, pp. 826-841, Aug. 2020. Summary of the Invention [Problem to be solved by the invention]

[0012] However, for example, when a third party creates a control engine (control algorithm), the control policy, constraints, etc. of the control engine are not necessarily disclosed to the network operator. Therefore, the technology disclosed in Non-Patent Document 1 poses a problem in that it may not be possible to calculate the evaluation of each control engine or to learn changes to the solution of each control engine.

[0013] The present invention has been made in consideration of the above-mentioned circumstances, and aims to efficiently carry out integrated control of all control engines up to a control completion state (an optimal state in which each control policy is satisfied, or a state in which further improvement as a whole is not possible) even without disclosing the control policies, constraints, etc. of the control engines. [Means for solving the problem]

[0014] In order to achieve the above object, the invention of claim 1 is an integrated control device that integrates and controls a plurality of control engines for constructing a plurality of different virtual networks on a physical network, comprising: an information distribution unit that acquires NFV (Network Functions Virtualization) environment information that indicates the status of the physical network and distributes it to the plurality of control engines, and thereby receives control requests related to the plurality of virtual networks from the plurality of control engines; a comprehensive evaluation estimation unit that counts the total number of control instructions in the control requests from the plurality of control engines with respect to the current status of the physical network, based on the control requests received from the plurality of control engines; and a control permission destination learning unit that updates an action value function for determining control permission by utilizing reinforcement learning so that the total number of control instructions counted by the comprehensive evaluation estimation unit is smaller than the total number of control instructions counted previously; The total number of control instructions tallied by the comprehensive evaluation estimation unit is the difference between the previous and current If there is a change, one of the plurality of control engines , determining a predetermined control engine based on the action value function, and a control application determination unit that transfers a predetermined control request received from a predetermined control engine to the physical network side. [Effects of the Invention]

[0015] As described above, according to the present invention, it is possible to efficiently perform integrated control of all control engines up to the control completion state, even without disclosing the control policies, constraints, etc. of the control engines. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is an overall configuration diagram of a communication system according to a first embodiment. [Figure 2] FIG. 2 is an electrical hardware configuration diagram of the integrated control system. [Figure 3] 4 is a flowchart showing an integrated control process according to the first embodiment. [Figure 4] FIG. 10 is a diagram illustrating the overall configuration of a communication system according to a second embodiment. [Figure 5] 10 is a flowchart showing an integrated control process according to the second embodiment. [Figure 6] FIG. 1 is a conceptual diagram illustrating a case where multiple virtual networks for different purposes are constructed on a physical network. [Figure 7] FIG. 1 is a conceptual diagram illustrating a case where a conflict in network control occurs when multiple virtual networks for different purposes are constructed on a physical network. [Figure 8] FIG. 1 is a conceptual diagram of a case where network control is coordinated when multiple virtual networks for different purposes are constructed on a physical network. [Figure 9] FIG. 1 is a diagram illustrating an overall configuration of a conventional communication system. [Figure 10] FIG. 1 is a functional configuration diagram of a conventional integrated control engine 200. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0018] First embodiment First, a first embodiment of the present invention will be described.

[0019] [System configuration of the embodiment] The outline of the configuration of the communication system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing the overall configuration of the communication system according to the first embodiment.

[0020] As shown in FIG. 1, the communication system of this embodiment is constructed by an integrated control system 10, an information collection and storage device 30, an orchestrator 50, and a physical network 300 as a network infrastructure.

[0021] The physical network 300 is managed and operated by a network operator, and is constructed from a plurality of general-purpose servers 310, a plurality of nodes (transfer devices) 320, and a plurality of user terminals 330.

[0022] Among these, the general-purpose server 310 is constructed by a single computer or multiple computers, and realizes virtual resources (VRs) and virtual network functions (VNFs) set by the integrated control system 10 via the orchestrator 50. Examples of VRs include virtual machines (VMs) that combine central processing units (CPUs), hard disk drives (HDDs), graphics processing units (GPUs), field programmable gate arrays (FPGAs), etc. Examples of VNFs include customer premises equipment (CPEs), firewalls (FWs), load balancers (LBs), deep packet inspection (DPIs), proxies, intrusion detection systems (IDSs), and network address translations (NATs). The general-purpose server 310 acquires server usage information and transmits the server usage information to the integrated control system 10 via the information collection and storage device 30. In addition, the general-purpose server 310 processes traffic according to the VR and VNF set by the integrated control system 10 via the orchestrator 50, and generates traffic.

[0023] The node 320 is a forwarding device such as a switch or a server, and functions as a forwarding device that forwards traffic. The node 320 acquires traffic information and transmits the traffic information to the integrated control system 10 via the information collection and storage device 30. The node 320 also processes the traffic according to the route calculated by the integrated control system 10.

[0024] The user terminal 330 is constructed by a single computer or multiple computers, and transmits traffic demand generated by the user's communication behavior (for example, an operation to start using a service, etc.) to other user terminals 330, the general-purpose server 310, etc. The user terminal 330 also transmits the user demand to the integrated control system 10 via the information collection and storage device 30.

[0025] A virtual network is a logical network that combines VR, VNF, and user terminals according to the services provided. Various services are realized by setting VR and VNF on general-purpose servers and forwarding traffic.

[0026] The server usage information is the utilization rate of the physical resources (e.g., CPU, memory, etc.) of the general-purpose server 310. The traffic information is the amount of data of any link included in the physical network 300. The user demand is the demand information of VR and VNF (the amount of VR and VNF required for each user terminal 330), the service constraints (the VR and VNF that need to be passed through for each user terminal 330), etc.

[0027] The information collection and storage device 30 is managed and operated by a network operator and is constructed with one or more computers. The information collection and storage device 30 centrally monitors the load status of the physical network 300 and various resources. To this end, the information collection and storage device 30 acquires and stores network observation information (traffic information, server usage information, etc.) and user demand from the physical network 300, and transfers this information to the integrated control system.

[0028] The orchestrator 50 is managed and operated by a network operator and is configured with one or more computers. The orchestrator 50 centrally receives and applies control instructions to the physical network 300. For example, the orchestrator 50 sends control signals to network devices (routers, switches, virtual servers, etc.) to control and modify the virtual network in real time using software. Therefore, the orchestrator 50 obtains control results and configuration commands from the integrated control system 10 according to solutions (control solutions) indicating optimal placement and optimal paths calculated by the integrated control system 10, and transmits these to each general-purpose server 310 and each node 320 to control a Network Functions Virtualization (NFV) environment capable of providing various types of network services. The orchestrator 50 controls VRs and VNFs using functions called "resource orchestration" and "service orchestration." Resource orchestration is a function that deploys VRs and VNFs on a physical network. Service orchestration is a function that calculates end-to-end paths connecting VRs and VNFs.

[0029] The integrated control system 10 is constructed with one or more computers and is managed by a network operator. The integrated control system 10 has multiple control engines e1, e2, and e3, which may be control engines created by the network operator itself or by third parties such as service providers. Hereinafter, the control engines e1, e2, and e3 will be collectively referred to as "control engine e." Each control engine e calculates the functional allocation of an individual, independent service.

[0030] Each control engine e is executed by its own control algorithm. Although three control engines are shown in FIG. 1 due to space limitations, the number may be two, four, or more. The integrated control system 10 has an integrated control algorithm A10. The control engine e may be constructed on a server (individual control device) separate from the integrated control algorithm.

[0031] The integrated control algorithm A10 is an algorithm that issues a configuration command to the NFV orchestrator to realize resource orchestration and service orchestration. The integrated control algorithm A10 selects a single predetermined control engine from among a plurality of control engines, and applies a control request (control result and configuration command) from the selected single control engine to the physical network 300 via the orchestrator 50. Note that each control engine e in this embodiment is premised on being able to issue control requests for all of the paths, VRs, and VNFs in order to independently construct a virtual network, which differs from the conventional control engine e100 (see FIG. 10 ) that can only issue control requests for any of the paths, VRs, and VNFs.

[0032] In this way, the integrated control system 10 controls the NFV environment by using the integrated control algorithm A10 to calculate the optimal placement of VRs and VNFs on the physical network 300 and the optimal end-to-end route connecting these VRs and VNFs. At this time, the integrated control system 10 calculates the optimal placement of VRs and VNFs and the optimal route based on network observation information (traffic information, server usage information, etc.) and user demand received from the physical network 300, and network configuration information.

[0033] The network configuration information includes network topology information of the physical network 300 and resource constraints (for example, the amount of each physical resource included in the physical network 300 (such as the server capacity of each general-purpose server 310 and the link capacity of each node 320)). The functions realized by the integrated control algorithm A10 will be described later. The integrated control algorithm A10 may be built in a server (integrated control device) separate from each control engine e.

[0034] [Hardware configuration] Next, the electrical hardware configuration of the integrated control system 10 will be described with reference to Fig. 2. Fig. 2 is a diagram showing the electrical hardware configuration of the integrated control system.

[0035] As shown in FIG. 2, the integrated control system 10 is a computer that includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an SSD (Solid State Drive) 104, an external device connection I / F (Interface) 105, a network I / F 106, a media I / F 109, and a bus line 110.

[0036] Of these, the CPU 101 controls the overall operation of the integrated control system 10. The ROM 102 stores programs such as an IPL (Initial Program Loader) used to drive the CPU 101. The RAM 103 is used as a work area for the CPU 101.

[0037] The SSD 104 reads or writes various data under the control of the CPU 101. Note that instead of the SDD 104, an HDD (Hard Disk Drive) may be used.

[0038] The external device connection I / F 105 is an interface for connecting various external devices, such as a display, a speaker, a keyboard, a mouse, a USB (Universal Serial Bus) memory, and a printer.

[0039] The network I / F 106 is an interface for performing data communication via a communication network such as the physical network 300 .

[0040] The media I / F 109 controls reading and writing (storing) of data from and to a recording medium 109m such as a flash memory, etc. The recording medium 109m includes a DVD (Digital Versatile Disc) and a Blu-ray Disc (registered trademark).

[0041] The bus line 110 is an address bus, a data bus, etc. for electrically connecting the components such as the CPU 101 shown in FIG.

[0042] The information collection and storage device 30, orchestrator 50, general-purpose server 310, node 320, and user terminal 330 have the same configuration as the integrated control system 10, and therefore a description of the hardware configuration will be omitted.

[0043] [Functional configuration of the integrated control system] Next, the functional configuration of the integrated control system 10 according to this embodiment will be described with reference to Fig. 1. The integrated control system 10 estimates an evaluation of the current network state held by each control engine e from the control request output by each control engine e, without using information related to the control policy and constraint conditions of each control engine e.

[0044] Specifically, the number of control instructions to the orchestrator received from the corresponding control engine e is regarded as the amount of deviation from the control policy or optimal state that the control engine e should satisfy.

[0045] The integrated control system 10 repeatedly performs processing based on the function realized by the integrated control algorithm A10 until the integrated control converges.

[0046] This integrated control algorithm A10 can realize the functions of an information distribution unit 11, a comprehensive evaluation estimation unit 12, a control permission destination learning unit 13, and a control application determination unit 14.

[0047] Of these, the information distribution unit 11 acquires NFV environment information (network observation information, user demand, etc.) indicating the network status of the physical network from the information collection and storage device 30 and distributes it to multiple control engines e, thereby accepting control requests from multiple control engines e regarding the virtual networks under the jurisdiction of each control engine.

[0048] The comprehensive evaluation estimation unit 12 counts the total number of control instructions issued by the multiple control engines e for the current network situation based on the control requests received from the multiple control engines e. The control instructions may be issued in units of a virtual network, or in units of individual elements such as a change in the placement of VRs and VNFs constituting a virtual network or a change in traffic route, but a common unit is used for each control engine. For example, if control engine e1 issues a control request including control instructions for two virtual networks, control engine e2 issues a control request including control instructions for three virtual networks, and control engine e3 issues a control request including control instructions for four virtual networks, the total number (cumulative number) of control instructions is 9 (= 2 + 3 + 4).

[0049] The control permission destination learning unit 13 updates the action value function for determining control permission using reinforcement learning so that the total number of control instructions tallied by the overall evaluation estimation unit 12 is reduced compared to the previous talliation. Only control requests from a predetermined control engine among the multiple control engines e can be transferred from the integrated control algorithm A10 to the physical network via the orchestrator 50. Therefore, the control permission destination learning unit 13 converges the integrated control of S11 to S20 described below by determining a predetermined control engine e (control request) from the multiple control engines e so that the total number of control instructions from each control engine e is reduced compared to the previous time.

[0050] When the evaluation for determining control permission changes, the control application determination unit 14 transfers the predetermined control request received from a predetermined control engine e among the multiple control engines e to the orchestrator 50, thereby transferring the control request to the physical network side. This makes it possible to apply the predetermined control request to the physical network 300. By processing in this manner, in the next process, for example, if the control engine e1 makes a control request including one control instruction, the control engine e2 makes a control request including two control instructions, and the control engine e3 makes a control request including two control instructions, the total number (cumulative number) of control instructions becomes 6 (= 1 + 2 + 3), which is less than the previous total number "9," and therefore the integration process gradually converges.

[0051] [Processing or operation of the integrated control system] Next, the processing or operation of the integrated control system 10 according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the integrated control processing according to the first embodiment.

[0052] S11: The information collection and storage device 30 collects NFV environment information (network observation information, user demand, etc.).

[0053] S12: The information distribution unit 11 distributes the NFV environment information acquired from the information collection and storage device 30 to each control engine e.

[0054] Here, the processing of each control engine e will be explained.

[0055] S101: The control engine e1 receives NFV environment information from the information distribution unit 11.

[0056] S102: The control engine e1 performs calculations for undisclosed control content.

[0057] S103: The control engine e1 transmits a control request to the information distribution unit 11.

[0058] The processes in steps S101 to S103 are also performed by the control engines e2 and e3.

[0059] S13: The information distribution unit 11 receives a control request from each control engine e.

[0060] S14: The comprehensive evaluation estimation unit 12 counts the number of control instructions in each control request.

[0061] S15: The comprehensive evaluation estimation unit 12 calculates the difference between the number of control instructions at the time of the previous control application determination and the number of control instructions at this time for each control engine e.

[0062] S16: The control permission destination learning unit 13 updates the evaluation of the previous control application permission.

[0063] S17: The control application determination unit 14 determines whether the evaluation has changed between the previous evaluation and the current evaluation.

[0064] S18: If the control application judgment unit 14 determines that the evaluation has changed in step S17 (YES), the control application judgment unit 14 determines one control request from the control requests (evaluation results and setting commands) received from each control engine e to which application permission will be granted based on the evaluation, and transfers this one control request to the orchestrator 50.

[0065] S19: The orchestrator 50 applies the one control request received from the control application determining unit 14 to the physical network 300.

[0066] S20: The orchestrator 50 confirms the completion of application. After that, the process returns to step S11. These steps S11 to S20 and S101 to S103 are repeated until the evaluation does not change (S17: NO).

[0067] On the other hand, in step S17, if the control application determination unit 14 does not determine that the evaluation has changed, that is, if it determines that the evaluation has not changed (NO), the control application determination unit 14 rejects the control requests from any of the control engines e and temporarily terminates the control. Thereafter, the process returns to step S11. Then, the information collection and storage device 30 collects NFV environment information (network observation information and user demand) (S11), and when the evaluation begins to change (S17; YES), the control application determination unit 14 determines one control request to be granted application permission based on the evaluation from among the control requests (evaluation results and setting commands) received from each control engine e, and transfers this one control request to the orchestrator 50.

[0068] This concludes the description of the integrated control process according to the first embodiment.

[0069] [Specific example of processing] Next, two specific examples of the integrated control process according to the first embodiment will be described.

[0070] <Example 1-1> First, a specific example 1-1 of the integrated control process according to the first embodiment will be described.

[0071] Time is discretized by a certain time interval Δt and expressed as t (=(t-Δt, t)). Δt is assumed to be longer than the response time of each control engine and the network control application time by the orchestrator 50. The number of control engines is also assumed to be N. As a process at time t, the integrated control algorithm A10 generates a control request M for each control engine i at time t-1. i (x t-1 ) is assumed to be held.

[0072] S111: The information distribution unit 11 receives the network status s from the information collection and storage device 30. t and for each control engine i∈{1,…,N}, the network state s t Notify the network status tis generated based on NFV environment information (traffic volume, resource usage, etc.) collected during the time period (t-Δt, t).

[0073] S112: Each control engine i∈{1,...,N} calculates the function allocation according to the internal logic and sends a control request M to the integrated control algorithm A10. i (x t ) to notify.

[0074] S113: The comprehensive evaluation estimation unit 12 receives the control request M from all the control engines. i (x t ) and receives the state evaluation y given by (Eq. 1) i,t-1 Calculate.

[0075]

number

[0076] S114: The control permission destination learning unit 13 calculates the sum r of the state evaluations. t and update the action value function Q(s',i) as shown in (Equation 2).

[0077]

number

[0078] S115: The control application determination unit 14 determines the network state s' t Based on this, the control engine a t ∈{1,…,N} and request control.

[0079]

number

[0080]

number

[0081] Note that an evaluation of how much the number of control instructions for the entire system has decreased as a result of the previous decision (at time t-1) (S111 to S113) is reflected in the Q function (S114), and then the application permission destination at the current time (time t) is decided (S115). Therefore, compared to specific example 1-2 described later, the amount of information when deciding the control application destination is larger by one time. However, if there is a control engine with a slow response in calculating the solution, it takes time to update the Q function in the previous stage, and it takes time to decide the application permission destination.

[0082] This concludes the description of the specific example 1-1 of the integrated control process according to the first embodiment.

[0083] <Example 1-2> Next, a specific example 1-2 of the integrated control process according to the first embodiment will be described.

[0084] The same conditions as in the above specific example 1-1 are used, that is, time is discretized by a certain time interval Δt and expressed as t (= (t - Δt, t)). Δt is assumed to be longer than the response time of each control engine and the network control application time by the orchestrator 50. In addition, the number of control engines is assumed to be N. As a process at time t, the integrated control algorithm A10 generates a control request M for each control engine i at time t-1. i (x t-1 ) is assumed to be held.

[0085] S121: The information distribution unit 11 receives the network status s from the information collection and storage device 30. t Get the network status t is the NFV collected during the time period (t-Δt,t). It is generated based on environmental information (traffic volume, resource usage, etc.).

[0086] S122: The control application determination unit 14 determines the network state s't Based on this, the control engine a t Determine ∈{1,…,N}.

[0087] S123: The information distribution unit 11 notifies each control engine i∈{1,...,N} of the network state s t Notify.

[0088] S124: Each control engine i∈{1,...,N} calculates the function allocation according to its internal logic and issues a control request M to the integrated control algorithm A10. i (x t ) to notify.

[0089] S125: The control application determination unit 14 determines whether the control engine a t Control request from

[0090]

number

[0091] S126: The comprehensive evaluation estimation unit 12 receives the control request M from all the control engines. i (x t ) and the state evaluation y given by (Eq. 1) above. i,t-1 Here, |M(x)| is the number of control instructions to the orchestrator 50 when the control request is M(x).

[0092] S127: The control permission destination learning unit 13 calculates the sum r of the state evaluations. t and update the action value function Q(s',i) as shown above (Equation 2).

[0093] Note that the control application destination is determined (S121 to S125) based on the Q function up to the previous time (time t-1), and then the Q function is updated (S126, S127) for the previous determination, so the response to the control request is faster than in the above-mentioned specific example 1-1. However, if the response of the selected control algorithm is slow, the result will be the same as in specific example 1-1. Also, since the evaluation of the previous determination (updating the Q function) is performed after the current determination, the ability to follow changes in the situation will be reduced.

[0094] [Effects of the first embodiment] As described above, the integrated control system 10 of this embodiment realizes an evaluation estimation of the current network state held by each control engine from the control request output by each control engine, without using information about the control policy or constraint conditions of the control engine. Then, based on the result, the integrated control system 10 forwards the control request received from each control engine to an orchestrator that issues control instructions to the network infrastructure (physical network 300). This advantageously enables the integrated control system 10 to efficiently perform integrated control for all control engines up to a control completion state (an optimal state in which each control policy is satisfied, or a state in which further improvement as a whole is impossible) without being disclosed the algorithms of the control engines.

[0095] Second embodiment Next, a second embodiment of the present invention will be described. The second embodiment is a modified example of the first embodiment, and an integrated control system 20 according to the second embodiment is configured by adding mediators m1, m2, and m3 to the integrated control system 10 according to the first embodiment. Although three mediators m1, m2, and m3 are shown here, the number may be two, or four or more. The mediators m1, m2, and m3 are collectively referred to as "mediator m." A mediator m is provided for each control engine e.

[0096] [System configuration of the embodiment] The configuration of the communication system according to the second embodiment will be outlined below with reference to Fig. 4. Fig. 4 is a diagram showing the overall configuration of the communication system according to the second embodiment.

[0097] 14, the communication system of this embodiment is constructed by an integrated control system 20, an information collection and storage device 30, an orchestrator 50, and a physical network 300 as a network infrastructure. Note that the same devices, configurations, or functions as those of the first embodiment are denoted by the same symbols and descriptions thereof will be omitted.

[0098] The integrated control system 20 is constructed by one or more computers and is managed by a network operator. The integrated control system 20 has a plurality of control engines e, such as a control engine created by the network operator itself and a control engine created by a third-party service provider. The integrated control system 20 also has an integrated control algorithm A20 and a plurality of mediators m.

[0099] The integrated control algorithm A20 basically functions in the same way as the integrated control algorithm A10 according to the first embodiment. The differences between the integrated control algorithm A20 and the integrated control algorithm A10 will be described later.

[0100] The mediator m processes the NFV environment information received from the information collection and storage device 30 by adding an adjustment amount according to the control permission frequency for each control engine e, and distributes the processed NFV environment information to each control engine e. That is, the mediator m processes the multiple pieces of NFV environment information output by the information distribution unit 21 into a predetermined single piece of NFV environment information, and distributes the processed NFV environment information to each control engine e. For example, the mediator m obtains the processed NFV environment information by adding a uniformly random value to the multiple pieces of NFV environment information and taking the average using the FPL (Follow the Perturbed Leader) algorithm described below.

[0101] [Functional configuration of the integrated control system] In this embodiment, the integrated control algorithm A20 determines the control application permission destination based on the learned action value function. Therefore, if the frequency of control application permission differs between control engines e, the status of the service handled by the corresponding control engine e may deteriorate during periods when control permission is not obtained. Therefore, in this embodiment, an online decision-making method is utilized to improve the time-averaged performance of each control engine e. Specifically, the integrated control system 20 uses a mediator e for each control engine between the integrated control algorithm A20 and each control engine e, and adds an adjustment amount corresponding to the frequency of control permission to the information distributed to the control engine e, thereby allowing the control engine e to calculate a solution that is not strongly dependent on temporary network conditions.

[0102] The integrated control system 20 repeatedly performs processing using the functions realized by the integrated control algorithm A20 until the integrated control converges.

[0103] This integrated control algorithm A20 can realize the functions of the information distribution unit 21, the overall evaluation estimation unit 22, the control permission destination learning unit 23, and the control application determination unit 24. The information distribution unit 21, the overall evaluation estimation unit 22, the control permission destination learning unit 23, and the control application determination unit 24 basically have the same functions as the information distribution unit 11, the overall evaluation estimation unit 12, the control permission destination learning unit 13, and the control application determination unit 14, respectively. However, the information distribution unit 21 can distribute processed NFV environment information to each control engine e via each mediator m. In addition, the overall evaluation estimation unit 22 receives a control request (control instruction) from each control engine e via each mediator m.

[0104] [Processing or operation of the integrated control system] Next, the processing or operation of the integrated control system 20 according to the second embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the integrated control processing according to the second embodiment.

[0105] S31: The information collection and storage device 30 collects NFV environment information (network observation information, user demand, etc.) that indicates the network state of the physical network.

[0106] S32: The information distribution unit 21 transmits the NFV environment information acquired from the information collection and storage device 30 to each mediator m. Furthermore, each mediator m processes the NFV environment information and distributes it to each corresponding control engine e.

[0107] Here, the processing of each control engine e will be explained.

[0108] S201: The control engine e1 receives the processed NFV environment information from the mediator e1.

[0109] S202: The control engine e1 performs calculations for undisclosed control content.

[0110] S203: The control engine e1 transmits a control request to the information distribution unit 11.

[0111] The processes in S201 to S203 are also performed by the control engines e2 and e3.

[0112] S33: Each mediator m accepts a control request from each control engine e and passes it to the comprehensive evaluation estimation unit 22.

[0113] S34: The comprehensive evaluation estimation unit 22 counts the number of control instructions in each control request.

[0114] S35: The comprehensive evaluation estimation unit 22 calculates the difference between the number of control instructions at the time of the previous control application determination and the number of control instructions at this time for each control engine e.

[0115] S36: The control permission destination learning unit 23 updates the evaluation of the previous control application permission.

[0116] S37: The control application determination unit 24 determines whether the evaluation has changed between the previous evaluation and the current evaluation.

[0117] S38: If the control application judgment unit 24 determines that the evaluation has changed in step S37 (YES), the control application judgment unit 24 determines one control request from the control requests (evaluation results and setting commands) received from each control engine e to which application permission will be granted based on the evaluation, and transfers this one control request to the orchestrator 50.

[0118] S39: The orchestrator 50 applies the one control request received from the control application determining unit 24 to the physical network 300.

[0119] S40: The orchestrator 50 confirms the completion of application. After that, the process returns to step S31. These steps S31 to S40 and S201 to S203 are repeated until the evaluation no longer changes (S37: NO).

[0120] On the other hand, in step S37, if the control application determination unit 24 does not determine that the evaluation has changed, that is, if it determines that the evaluation has not changed (NO), the control application determination unit 24 rejects the control requests from any of the control engines e and temporarily terminates control. Thereafter, the process returns to step S31. Then, the information collection and storage device 30 collects NFV environment information (network observation information and user demand) (S31), and when the evaluation begins to change (S37; YES), the control application determination unit 24 determines one control request to be granted application permission based on the evaluation from among the control requests (evaluation results and setting commands) received from each control engine e, and transfers this one control request to the orchestrator 50.

[0121] This concludes the description of the integrated control process according to the second embodiment.

[0122] [Specific example of processing] Next, two specific examples of the integrated control process according to the second embodiment will be described.

[0123] <Example 2-1> First, a specific example 2-1 of the integrated control process according to the second embodiment will be described.

[0124] The same conditions as in the above specific example 1-1 are used, that is, time is discretized by a certain time interval Δt and expressed as t (= (t - Δt, t)). Δt is assumed to be longer than the response time of each control engine and the network control application time by the orchestrator 50. In addition, the number of control engines is assumed to be N. As a process at time t, the integrated control algorithm A20 calculates the control request M of each control engine i at time t-1. i (x t-1 ) is assumed to be held.

[0125] S211: The information distribution unit 11 receives the network status s from the information collection and storage device 30. t and assigns the network state s t Notify the network status t is generated based on NFV environment information (traffic volume, resource usage, etc.) collected during the time period (t-Δt, t).

[0126] S212: Each mediator i∈{1,…,N} receives the processed network state x given by (Eq. 4). i,t Each control algorithm M i Notify.

[0127]

number

[0128] S213: Each control engine i∈{1,...,N} calculates the function allocation according to its internal logic and sends a control request M to the integrated control algorithm A20 via each mediator. i (x i,t ) to notify.

[0129] S214: The comprehensive evaluation estimation unit 12 receives the control request M from all the control engines. i (xi,t ) and the state evaluation y given by (Eq. 5) i,t-1 Calculate.

[0130]

number

[0131] S215: The control permission destination learning unit 13 calculates the sum r of the state evaluations. t and update the action value function Q(s',i) as shown above (Equation 2).

[0132] S216: The control application determination unit 14 determines the network state s' t Based on this, the control engine a t ∈{1,…,N} and request control.

[0133]

number

[0134] In this specific example 2-1, the FPL algorithm described below is used to process the network information, but it may be replaced with a prediction algorithm that utilizes, for example, regression analysis.

[0135] This concludes the description of the specific example 2-1 of the integrated control process according to the second embodiment.

[0136] <Example 2-2> Next, a specific example 2-2 of the integrated control process according to the second embodiment will be described.

[0137] The same conditions as in the above specific example 1-1 are used, that is, time is discretized by a certain time interval Δt and expressed as t (= (t - Δt, t)). Δt is assumed to be longer than the response time of each control engine and the network control application time by the orchestrator 50. In addition, the number of control engines is assumed to be N. As a process at time t, the integrated control algorithm A20 calculates the control request M of each control engine i at time t-1. i (x t-1 ) is assumed to be held.

[0138] S221: The information distribution unit 11 receives the network status s from the information collection and storage device 30. t Get the network status t is generated based on NFV environment information (traffic volume, resource usage, etc.) collected during the time period (t-Δt, t).

[0139] S222: The control application determination unit 14 determines the network state s' t Based on this, the control engine a t Determine ∈{1,…,N}.

[0140] S223: The information distribution unit 11 notifies each mediator i∈{1,...,N} of the network state s t Notify.

[0141] S224: Each mediator i∈{1,…,N} receives the processed network state x given by (Equation 4) above. i,t Each control algorithm M i Notify.

[0142] S225: Each control engine i∈{1,...,N} calculates the function allocation according to its internal logic and issues a control request M to the integrated control algorithm A20. i (x i,t ) to notify.

[0143] S226: The control application determination unit 14 determines whether the control engine a t Control request from

[0144]

number

[0145] S227: The comprehensive evaluation estimation unit 12 receives the control request M from all the control engines. i (x i,t ) and the state evaluation y given by (Eq. 5) above. i,t-1 Calculate.

[0146] S228: The control permission destination learning unit 13 calculates the sum r of the state evaluations. t and update the action value function Q(s',i) as shown above (Equation 2).

[0147] In this specific example 2-2, as in specific example 2-1, the FPL algorithm described below is used to process the network information, but this may be replaced with a prediction algorithm that utilizes, for example, regression analysis.

[0148] This concludes the description of the specific example 2-2 of the integrated control process according to the second embodiment.

[0149] <FPLアルゴリズム> Next, as an example of processing of network information performed by the mediator m, a case where the FPL algorithm is used will be described.

[0150] First, let us assume that the online decision-making problem with linear sum costs in n-dimensional space is in the following state.

[0151] The decision at time t is d t Let's say:d t ∈D⊂R n decision making t After (network) state s t is observed: t ∈S⊂R n The cost is defined as the dot product d·s of the decision and the (network) state. Let M(s) be the function that gives the minimum cost solution when s is given.

[0152]

number

[0153]

number

[0154]

number

[0155] Furthermore, as a theorem, the expected cumulative cost by the FPL algorithm is given below with δ as a parameter.

[0156]

number

[0157]

number

[0158]

number

[0159]

number

[0160]

number

[0161] [Effects of the second embodiment] As described above, the integrated control system 10 of this embodiment has the same effects as the first embodiment.

[0162] Furthermore, because the orchestrator 50 only accepts control requests from one specific control engine e, it takes a long time for the integrated control to converge, or the frequency with which integrated control can be performed within a given time period decreases. To prevent this, in the second embodiment, the mediator m stores, for example, ten pieces of acquired network information, and processes it into a single piece of network information using an FPL algorithm or the like before sending it to the control engine e. This solves the problem.

[0163] 〔supplement〕 As described above, the present invention is not limited to the above-described embodiment, and various modifications and applications are possible, for example, as shown below.

[0164] (1) The integrated control systems 10, 20 and the integrated control devices A10, A20 can be realized by a computer and a program, but this program can also be recorded on a (non-temporary) recording medium or provided via a communication network such as the Internet.

[0165] (2) The CPU 101 may be multiple, not just single. [Explanation of symbols]

[0166] 10 Integrated Control System A10 Integrated control algorithm (integrated control device) 11 Information Distribution Department 12 Comprehensive Evaluation and Estimation Section 13 Control permission destination learning unit 14 Control application decision unit e, e1~e3 control engine A20 Integrated control algorithm (integrated control device) 21 Information Distribution Department 22 Comprehensive Evaluation and Estimation Department 23 Control permission destination learning unit 24 Control application decision unit m, m1~m3 mediator 30 Information collection and storage device 50 Orchestrator 300 Physical Network (Network Infrastructure, NFV Environment) 310 General-purpose Server 320 nodes (transport devices) 330 User Terminal

Claims

1. An integrated control device that integrates and controls a plurality of control engines for constructing a plurality of different virtual networks on a physical network, an information distribution unit that receives control requests for the plurality of virtual networks from the plurality of control engines by acquiring NFV (Network Functions Virtualization) environment information indicating the status of the physical network and distributing the information to the plurality of control engines; a comprehensive evaluation estimation unit that counts the total number of control instructions in the control requests from the plurality of control engines with respect to a current state of the physical network based on the control requests received from the plurality of control engines; a control permission destination learning unit that updates an action value function for determining control permission by using reinforcement learning so that the total number of the control instructions tallied by the comprehensive evaluation estimation unit becomes smaller than that at the time of the previous talliation; a control application determination unit that, when the total number of control instructions tallied by the comprehensive evaluation estimation unit has changed between the previous time and the current time, determines a predetermined control engine from among the plurality of control engines based on the action value function, and transfers a predetermined control request received from the predetermined control engine to the physical network side; An integrated control device having:

2. The integrated control device according to claim 1; the plurality of control engines; Integrated control system with

3. 3. The integrated control system according to claim 2, An integrated control system having a mediator that processes the multiple NFV environmental information output by the information distribution unit into a predetermined single NFV environmental information and distributes the processed NFV environmental information to the multiple control engines.

4. The integrated control system according to claim 3 , wherein the mediator uses an FPL algorithm to add uniformly random values ​​to the plurality of pieces of NFV environmental information and take an average to obtain the processed NFV environmental information.

5. An integrated control system that integrates and controls a plurality of control engines for constructing a plurality of different virtual networks on a physical network, an information distribution unit that receives control requests for the plurality of virtual networks from the plurality of control engines by acquiring NFV environment information indicating the status of the physical network and distributing the information to the plurality of control engines; a comprehensive evaluation estimation unit that counts the total number of control instructions in the control requests from the plurality of control engines with respect to a current state of the physical network based on the control requests received from the plurality of control engines; a control permission destination learning unit that updates an action value function for determining control permission by using reinforcement learning so that the total number of the control instructions tallied by the comprehensive evaluation estimation unit becomes smaller than that at the time of the previous talliation; a control application determination unit that, when the total number of control instructions tallied by the comprehensive evaluation estimation unit has changed between the previous time and the current time, determines a predetermined control engine from among the plurality of control engines based on the action value function, and transfers a predetermined control request received from the predetermined control engine to the physical network side; and an integrated control algorithm that realizes the plurality of control engines; Integrated control system with

6. 6. The integrated control system according to claim 5, An integrated control system having a mediator that processes the multiple NFV environmental information output by the information distribution unit into a predetermined single NFV environmental information and distributes the processed NFV environmental information to the multiple control engines.

7. 1. An integrated control method executed by an integrated control device that integrates and controls a plurality of control engines for constructing a plurality of different virtual networks on a physical network, comprising: The integrated control device an information distribution process for receiving control requests for the plurality of virtual networks from the plurality of control engines by acquiring NFV environment information indicating the status of the physical network and distributing the information to the plurality of control engines; a comprehensive evaluation estimation process for tallying up the total number of control instructions in the control requests from the plurality of control engines with respect to the current state of the physical network, based on the control requests received from the plurality of control engines; a control permission destination learning process that updates an action value function for determining control permission using reinforcement learning so that the total number of control instructions tallied by the comprehensive evaluation estimation process is smaller than that at the time of the previous talliation; a control application determination process for determining a predetermined control engine from among the plurality of control engines based on the action value function when the total number of control instructions tallied by the comprehensive evaluation estimation process has changed between the previous time and the current time, and forwarding a predetermined control request received from the predetermined control engine to the physical network side; An integrated control method for performing the above.

8. A program causing a computer to execute the method according to claim 7.

Images