Determining the machine learning model to be used for a given prediction objective related to a communication system

Abstract

The present invention enables appropriate determination of a machine learning model suited to a communication system used to predict a performance indicator value of the communication system. A monitoring function unit (72) monitors at least one type of performance indicator value relating to the communication system. An AI unit (70) identifies an additional performance indicator value type for each of a plurality of machine learning models used for a given predictive purpose relating to the communication system, the additional performance indicator value type being a type of performance indicator value required to be added to an object to be monitored in order to use this machine learning model. The AI unit (70) determines at least one of the plurality of machine learning models on the basis of the additional performance indicator value type identified for each machine learning model.

Description

[Technical Field]

[0001] The present invention relates to determining a machine learning model to be used for a given predictive purpose in a communication system. [Background technology]

[0002] There are techniques for predicting performance index values ​​of communication systems. As an example of such a technique, Patent Document 1 describes a technique for estimating throughput based on the number of terminals present in a mesh i, the number of terminals present in the mesh i and currently communicating.

[0003] Moreover, in recent years, predictions using machine learning have become increasingly common. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-92125 Summary of the Invention [Problem to be solved by the invention]

[0005] For example, it is conceivable to use a trained machine learning model to predict the performance index value of a communication system based on the actual performance index value of the communication system.

[0006] Here, even if the prediction purpose is the same, various patterns can be assumed for the types of actual values ​​of performance index values ​​that are input to the machine learning model, which correspond to explanatory variables for prediction.

[0007] On the other hand, the types of performance indicators whose actual values ​​are monitored vary depending on the situation, so even if a machine learning model is used for the same prediction purpose, some may be suitable for the communication system in question and some may not.

[0008] The present invention has been made in consideration of the above-mentioned situation, and one of its purposes is to enable accurate determination of a machine learning model suitable for a communication system, which is used to predict the performance index value of the communication system. [Means for solving the problem]

[0009] In order to solve the above problem, the model determination system of the present disclosure includes a monitoring means for monitoring at least one type of performance indicator value related to a communication system, an additional performance indicator value type identification means for identifying, for each of a plurality of machine learning models used for a given prediction purpose related to the communication system, an additional performance indicator value type that is a type of performance indicator value that needs to be added to the monitoring targets in order to use the machine learning model, and a model determination means for determining at least one of the plurality of machine learning models based on the additional performance indicator value type identified for each of the machine learning models.

[0010] In addition, a model determination method according to the present disclosure includes monitoring at least one type of performance indicator value related to a communication system; identifying, for each of a plurality of machine learning models used for a given prediction purpose related to the communication system, an additional performance indicator value type that is a type of performance indicator value that needs to be added to the monitoring targets in order to use the machine learning model; and determining at least one of the plurality of machine learning models based on the additional performance indicator value type identified for each of the machine learning models. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating an example of a communication system according to an embodiment of the present invention. [Figure 2] 1 is a diagram illustrating an example of a communication system according to an embodiment of the present invention. [Figure 3] FIG. 1 is a diagram illustrating an example of a network service according to an embodiment of the present invention. [Figure 4]FIG. 1 is a diagram illustrating an example of associations between elements established in a communication system according to an embodiment of the present invention. [Figure 5] FIG. 2 is a functional block diagram showing an example of functions implemented in a platform system according to an embodiment of the present invention. [Figure 6] FIG. 2 illustrates an example of a data structure of physical inventory data. [Figure 7] FIG. 2 is a diagram illustrating an example of a data bus unit according to an embodiment of the present invention. [Figure 8] FIG. 10 is a diagram illustrating an example of acquiring performance index value data through a performance determination process. [Figure 9] FIG. 10 is a diagram illustrating an example of acquiring performance index value data through a performance determination process and an estimation process. [Figure 10] FIG. 10 is a diagram illustrating an example of model management data. [Figure 11] FIG. 10 is a diagram showing an example of a recommendation screen. [Figure 12] FIG. 2 is a flowchart showing an example of a flow of processing performed in a platform system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0013] 1 and 2 are diagrams illustrating an example of a communication system 1 according to an embodiment of the present invention. Fig. 1 is a diagram focusing on the locations of a group of data centers included in the communication system 1. Fig. 2 is a diagram focusing on various computer systems implemented in the group of data centers included in the communication system 1.

[0014] As shown in FIG. 1, the data centers included in the communication system 1 are classified into a central data center 10, regional data centers 12, and edge data centers .

[0015] For example, several central data centers 10 are distributed and placed within the area covered by the communication system 1 (for example, within Japan).

[0016] For example, several tens of regional data centers 12 are distributed and placed within the area covered by the communication system 1. For example, if the area covered by the communication system 1 is the entire country of Japan, one or two regional data centers 12 may be placed in each prefecture.

[0017] For example, several thousand edge data centers 14 are distributed within the area covered by the communication system 1. Each edge data center 14 is capable of communicating with communication equipment 18 equipped with an antenna 16. As shown in FIG. 1, one edge data center 14 may be capable of communicating with several communication equipment 18. The communication equipment 18 may include a computer such as a server computer. The communication equipment 18 according to this embodiment performs wireless communication with a UE (User Equipment) 20 via the antenna 16. The communication equipment 18 equipped with the antenna 16 is provided with, for example, a radio unit (RU) (described later).

[0018] In the central data center 10, the regional data center 12, and the edge data center 14 according to this embodiment, multiple servers are arranged.

[0019] In this embodiment, for example, the central data center 10, the regional data centers 12, and the edge data centers 14 are capable of communicating with each other. Furthermore, the central data centers 10, the regional data centers 12, and the edge data centers 14 are also capable of communicating with each other.

[0020] 2, the communication system 1 according to this embodiment includes a platform system 30, multiple radio access networks (RANs) 32, multiple core network systems 34, and multiple UEs 20. The core network systems 34, the RANs 32, and the UEs 20 cooperate with each other to realize a mobile communication network.

[0021] The RAN 32 is a computer system equipped with an antenna 16, which corresponds to an eNodeB (eNB) in a fourth-generation mobile communication system (hereinafter referred to as 4G) or a gNB (NR base station) in a fifth-generation mobile communication system (hereinafter referred to as 5G). The RAN 32 according to this embodiment is mainly implemented by a group of servers and communication equipment 18 arranged in an edge data center 14. Note that part of the RAN 32 (for example, a distributed unit (DU), a central unit (CU), a virtual distributed unit (vDU), and a virtual central unit (vCU)) may be implemented in the central data center 10 or the regional data center 12, rather than in the edge data center 14.

[0022] The core network system 34 is a system equivalent to an EPC (Evolved Packet Core) in 4G or a 5G Core (5GC) in 5G. The core network system 34 according to this embodiment is implemented mainly by a group of servers arranged in the central data center 10 and the regional data centers 12.

[0023] The platform system 30 according to this embodiment is configured on, for example, a cloud platform, and includes a processor 30a, a storage unit 30b, and a communication unit 30c, as shown in FIG. 2. The processor 30a is a program-controlled device such as a microprocessor that operates according to a program installed in the platform system 30. The storage unit 30b is, for example, a storage element such as a ROM or RAM, a solid-state drive (SSD), or a hard disk drive (HDD). The storage unit 30b stores programs executed by the processor 30a. The communication unit 30c is, for example, a communication interface such as a network interface controller (NIC) or a wireless local area network (LAN) module. Note that software-defined networking (SDN) may be implemented in the communication unit 30c. The communication unit 30c exchanges data with the RAN 32 and the core network system 34.

[0024] In this embodiment, the platform system 30 is implemented by a group of servers located in the central data center 10. Note that the platform system 30 may also be implemented by a group of servers located in the regional data centers 12.

[0025] In this embodiment, for example, in response to a purchase request for a network service (NS) from a purchaser, the requested network service is established in the RAN 32 and the core network system 34. Then, the established network service is provided to the purchaser.

[0026] For example, a purchaser such as an MVNO (Mobile Virtual Network Operator) is provided with network services such as voice communication services and data communication services. The voice communication services and data communication services provided by this embodiment are ultimately provided to customers (end users) of the purchaser (MVNO in the above example) who use the UE 20 shown in FIGS. 1 and 2. The end users can perform voice communication and data communication with other users via the RAN 32 and the core network system 34. The UE 20 of the end user can also access a data network such as the Internet via the RAN 32 and the core network system 34.

[0027] In addition, in this embodiment, an IoT (Internet of Things) service may be provided to an end user who uses a robot arm, a connected car, etc. In this case, for example, the end user who uses the robot arm, the connected car, etc. may become a purchaser of the network service according to this embodiment.

[0028] In this embodiment, a container-based virtualized application execution environment such as Docker (registered trademark) is installed on servers located in the central data center 10, the regional data centers 12, and the edge data center 14, allowing containers to be deployed and run on these servers. A cluster consisting of one or more containers generated by such virtualization technology may be built on these servers. For example, a Kubernetes cluster managed by a container management tool such as Kubernetes (registered trademark) may be built. Then, a processor on the built cluster may execute a container-based application.

[0029] In this embodiment, the network service provided to the purchaser is composed of one or more functional units (for example, network functions (NFs)). In this embodiment, the functional units are implemented as NFs realized by virtualization technology. NFs realized by virtualization technology are called VNFs (Virtualized Network Functions). It does not matter what virtualization technology is used to virtualize them. For example, in this description, a CNF (Containerized Network Function) realized by container-type virtualization technology is also included in the VNF. In this embodiment, the network service will be described as being implemented by one or more CNFs. Furthermore, the functional units in this embodiment may correspond to network nodes.

[0030] Fig. 3 is a diagram illustrating an example of an operating network service. The network service illustrated in Fig. 3 includes, as software elements, NFs such as a plurality of RUs 40, a plurality of DUs 42, a plurality of CUs 44 (CU-CP (Central Unit - Control Plane) 44a and CU-UP (Central Unit - User Plane) 44b), a plurality of AMFs (Access and Mobility Management Functions) 46, a plurality of SMFs (Session Management Functions) 48, and a plurality of UPFs (User Plane Functions) 50.

[0031] In the example of Figure 3, RU 40, DU 42, CU-CP 44a, AMF 46, and SMF 48 correspond to elements of the control plane (C-Plane), and RU 40, DU 42, CU-UP 44b, and UPF 50 correspond to elements of the user plane (U-Plane).

[0032] The network service may include other types of NFs as software elements. The network service is implemented on computer resources (hardware elements) such as multiple servers.

[0033] In this embodiment, for example, a communication service in a certain area is provided by the network service shown in FIG.

[0034] In this embodiment, it is assumed that multiple RUs 40, multiple DUs 42, multiple CU-UPs 44b, and multiple UPFs 50 shown in FIG. 3 belong to one end-to-end network slice.

[0035] Fig. 4 is a diagram schematically illustrating an example of associations between elements established in the communication system 1 in this embodiment. The symbols M and N shown in Fig. 4 represent any integers equal to or greater than 1, and indicate the relationship between the numbers of elements connected by a link. When both ends of a link are a combination of M and N, the elements connected by the link have a many-to-many relationship, and when both ends of a link are a combination of 1 and N or a combination of 1 and M, the elements connected by the link have a one-to-many relationship.

[0036] As shown in Figure 4, network services (NS), network functions (NF), CNFCs (Containerized Network Function Components), pods, and containers have a hierarchical structure.

[0037] An NS corresponds to, for example, a network service configured from multiple NFs. Here, an NS may correspond to an element of granularity such as a 5G RAN (gNB), an EPC, a 5G RAN (eNB), or the like.

[0038] In 5G, NFs correspond to elements with granularity such as RU, DU, CU-CP, CU-UP, AMF, SMF, and UPF. In 4G, NFs correspond to elements with granularity such as MME (Mobility Management Entity), HSS (Home Subscriber Server), S-GW (Serving Gateway), vDU, and vCU. In this embodiment, for example, one NS includes one or more NFs. In other words, one or more NFs are subordinate to one NS.

[0039] A CNFC corresponds to an element of granularity such as DU mgmt or DU Processing. A CNFC may be a microservice deployed on a server as one or more containers. For example, a CNFC may be a microservice that provides some of the functions of DU, CU-CP, CU-UP, etc. Also, a CNFC may be a microservice that provides some of the functions of UPF, AMF, SMF, etc. In this embodiment, for example, one NF includes one or more CNFCs. In other words, one or more CNFCs are subordinate to one NF.

[0040] A pod refers to the smallest unit for managing a Docker container in Kubernetes, for example. In this embodiment, for example, one CNFC includes one or more pods. In other words, one or more pods are under the control of one CNFC.

[0041] In this embodiment, for example, one pod includes one or more containers. That is, one or more containers are subordinate to one pod.

[0042] Also, as shown in Figure 4, network slices (NSIs) and network slice subnet instances (NSSIs) have a hierarchical structure.

[0043] The NSI can also be considered an end-to-end virtual circuit spanning multiple domains (e.g., from the RAN 32 to the core network system 34). The NSI may be a slice for high-speed, high-capacity communication (e.g., for enhanced Mobile Broadband (eMBB)), a slice for high-reliability and low-latency communication (e.g., for Ultra-Reliable and Low Latency Communications (URLLC)), or a slice for connecting a large number of terminals (e.g., for massive Machine Type Communication (mMTC)). The NSSI can also be considered a virtual circuit of a single domain obtained by dividing the NSI. The NSSI may be a slice of the RAN domain, a slice of a transport domain such as the Mobile Back Haul (MBH) domain, or a slice of the core network domain.

[0044] In this embodiment, for example, one NSI includes one or more NSSIs. That is, one or more NSSIs are subordinate to one NSI. Note that in this embodiment, multiple NSIs may share the same NSSI.

[0045] Furthermore, as shown in FIG. 4, NSSI and NS generally have a many-to-many relationship.

[0046] Furthermore, in this embodiment, for example, one NF can belong to one or more network slices. Specifically, for example, one NF can be configured with NSSAI (Network Slice Selection Assistance Information) including one or more S-NSSAI (Sub Network Slice Selection Assist Information). Here, S-NSSAI is information associated with a network slice. Note that an NF does not necessarily have to belong to a network slice.

[0047] Fig. 5 is a functional block diagram showing an example of functions implemented in the platform system 30 according to this embodiment. Note that the platform system 30 according to this embodiment does not need to implement all of the functions shown in Fig. 5, and functions other than the functions shown in Fig. 5 may also be implemented.

[0048] As shown in FIG. 5 , the platform system 30 according to this embodiment functionally includes, for example, an operations support system (OSS) unit 60, an orchestration (E2EO: End-to-End-Orchestration) unit 62, a service catalog storage unit 64, a big data platform unit 66, a data bus unit 68, an AI (Artificial Intelligence) unit 70, a monitoring function unit 72, an SDN controller 74, a configuration management unit 76, a container management unit 78, and a repository unit 80. The OSS unit 60 includes an inventory database 82, a ticket management unit 84, a fault management unit 86, and a performance management unit 88. The E2EO unit 62 includes a policy manager unit 90, a slice manager unit 92, and a lifecycle management unit 94. These elements are implemented primarily using a processor 30 a, a storage unit 30 b, and a communication unit 30 c.

[0049] The functions shown in Fig. 5 may be implemented by having a processor 30a execute a program that is installed in a platform system 30, which is one or more computers, and that includes instructions corresponding to the functions. This program may be supplied to the platform system 30 via a computer-readable information storage medium, such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or via the Internet. The functions shown in Fig. 5 may also be implemented using circuit blocks, memory, or other LSIs. Those skilled in the art will understand that the functions shown in Fig. 5 can be realized in various forms, such as hardware alone, software alone, or a combination thereof.

[0050] The container management unit 78 manages the lifecycle of a container, which includes, for example, processes related to the construction of a container, such as the deployment and configuration of the container.

[0051] Here, the platform system 30 according to this embodiment may include a plurality of container management units 78. A container management tool such as Kubernetes and a package manager such as Helm may be installed in each of the plurality of container management units 78. Each of the plurality of container management units 78 may execute container construction, such as container deployment, for a server group (e.g., a Kubernetes cluster) associated with the corresponding container management unit 78.

[0052] It should be noted that the container management unit 78 does not need to be included in the platform system 30. The container management unit 78 may be provided, for example, in a server managed by the container management unit 78 (i.e., the RAN 32 or the core network system 34), or may be provided in another server that is annexed to the server managed by the container management unit 78.

[0053] In this embodiment, the repository unit 80 stores, for example, container images of containers included in a group of functional units (for example, a group of NFs) that realize a network service.

[0054] The inventory database 82 is a database that stores inventory information, which includes, for example, information about servers that are installed in the RAN 32 and the core network system 34 and that are managed by the platform system 30.

[0055] In this embodiment, inventory data is stored in the inventory database 82. The inventory data indicates the configuration of the elements included in the communication system 1 and the current status of the associations between the elements. The inventory data also indicates the status of resources managed by the platform system 30 (for example, the usage status of the resources). The inventory data may be physical inventory data or logical inventory data. The physical inventory data and logical inventory data will be described later.

[0056] Fig. 6 is a diagram showing an example of the data structure of physical inventory data. The physical inventory data shown in Fig. 6 is associated with one server. The physical inventory data shown in Fig. 6 includes, for example, a server ID, location data, building data, floor data, rack data, specification data, network data, an operating container ID list, a cluster ID, and the like.

[0057] The server ID included in the physical inventory data is, for example, an identifier of the server associated with the physical inventory data.

[0058] The location data included in the physical inventory data is, for example, data indicating the location (for example, the address of the location) of the server associated with the physical inventory data.

[0059] The building data included in the physical inventory data is, for example, data indicating the building (for example, the building name) in which the server associated with the physical inventory data is located.

[0060] The floor number data included in the physical inventory data is, for example, data indicating the floor number on which the server associated with the physical inventory data is located.

[0061] The rack data included in the physical inventory data is, for example, an identifier of the rack in which the server associated with the physical inventory data is located.

[0062] The specification data included in the physical inventory data is, for example, data indicating the specifications of the server associated with the physical inventory data, and the specification data indicates, for example, the number of cores, memory capacity, hard disk capacity, etc.

[0063] The network data included in the physical inventory data is, for example, data indicating information about the network of the server associated with the physical inventory data, and the network data indicates, for example, the NICs that the server has, the number of ports that the NICs have, the port IDs of the ports, etc.

[0064] The operating container ID list included in the physical inventory data is, for example, data that indicates information about one or more containers operating on a server associated with the physical inventory data, and the operating container ID list indicates, for example, a list of identifiers (container IDs) of instances of the containers.

[0065] The cluster ID included in the physical inventory data is, for example, an identifier of a cluster (for example, a Kubernetes cluster) to which a server associated with the physical inventory data belongs.

[0066] The logical inventory data includes topology data indicating the current state of associations between multiple elements included in the communication system 1, such as those shown in Fig. 4. For example, the logical inventory data includes topology data including an identifier of a certain NS and identifiers of one or more NFs under the NS. Also, for example, the logical inventory data includes topology data including an identifier of a certain network slice and identifiers of one or more NFs belonging to the network slice.

[0067] The inventory data may also include data indicating the current status of the geographical relationships and topological relationships between the elements included in the communication system 1. As described above, the inventory data includes location data indicating the locations where the elements included in the communication system 1 are operating, i.e., the current locations of the elements included in the communication system 1. From this, it can be said that the inventory data indicates the current status of the geographical relationships between the elements (for example, the geographical proximity between the elements).

[0068] The logical inventory data may also include NSI data indicating information about the network slice. The NSI data indicates attributes such as an identifier of an instance of the network slice and a type of the network slice. The logical inventory data may also include NSSI data indicating information about the network slice subnet. The NSSI data indicates attributes such as an identifier of an instance of the network slice subnet and a type of the network slice subnet.

[0069] The logical inventory data may also include NS data indicating information about an NS. The NS data indicates attributes such as an NS instance identifier and an NS type. The logical inventory data may also include NF data indicating information about an NF. The NF data indicates attributes such as an NF instance identifier and an NF type. The logical inventory data may also include CNFC data indicating information about a CNFC. The CNFC data indicates attributes such as an instance identifier and a CNFC type. The logical inventory data may also include pod data indicating information about a pod included in the CNFC. The pod data indicates attributes such as a pod instance identifier and a pod type. The logical inventory data may also include container data indicating information about a container included in the pod. The container data indicates attributes such as a container ID of a container instance and a container type.

[0070] The container ID of the container data included in the logical inventory data and the container ID included in the operating container ID list included in the physical inventory data associate a container instance with the server on which the container instance is running.

[0071] Furthermore, the logical inventory data may include data indicating various attributes such as a host name and an IP address. For example, container data may include data indicating an IP address of a container corresponding to the container data. For example, NF data may include data indicating an IP address and a host name of the NF indicated by the NF data.

[0072] The logical inventory data may also include data indicating an NSSAI, including one or more S-NSSAIs, configured in each NF.

[0073] Furthermore, the inventory database 82 is able to grasp the resource status as needed in cooperation with the container management unit 78. The inventory database 82 then updates the inventory data stored therein as needed based on the latest resource status.

[0074] In addition, in response to actions being performed, such as constructing a new element included in the communication system 1, changing the configuration of an element included in the communication system 1, scaling an element included in the communication system 1, or replacing an element included in the communication system 1, the inventory database 82 updates the inventory data stored in the inventory database 82.

[0075] The service catalog storage unit 64 stores service catalog data. The service catalog data may include, for example, service template data indicating logic used by the life cycle management unit 94. This service template data includes information necessary for building a network service. For example, the service template data includes information defining NSs, NFs, and CNFCs, and information indicating the correspondence between NSs, NFs, and CNFCs. Furthermore, for example, the service template data includes a workflow script for building a network service.

[0076] An example of service template data is an NSD (NS Descriptor). The NSD is associated with a network service and indicates the types of multiple functional units (e.g., multiple CNFs) included in the network service. The NSD may also indicate the number of each type of functional unit, such as CNF, included in the network service. The NSD may also indicate the file names of CNFDs (described later) related to the CNFs included in the network service.

[0077] An example of service template data is a CNF Descriptor (CNFD). The CNFD may indicate the computer resources (e.g., CPU, memory, hard disk, etc.) required by the CNF. For example, the CNFD may indicate the computer resources (CPU, memory, hard disk, etc.) required by each of multiple containers included in the CNF.

[0078] The service catalog data may also include information about thresholds (for example, anomaly detection thresholds) that are used by the policy manager 90 to compare with the calculated performance index values. The performance index values ​​will be described later.

[0079] The service catalog data may also include, for example, slice template data, which includes information necessary to perform instantiation of a network slice, including, for example, logic utilized by the slice manager unit 92.

[0080] The slice template data includes information on the "Generic Network Slice Template" defined by the GSMA (GSM Association) ("GSM" is a registered trademark). Specifically, the slice template data includes network slice template data (NST), network slice subnet template data (NSST), and network service template data. The slice template data also includes information indicating the hierarchical structure of these elements, as shown in FIG. 4.

[0081] In this embodiment, for example, the life cycle management unit 94 constructs a new network service in response to a purchase request for an NS from a purchaser.

[0082] For example, in response to a purchase request, the lifecycle management unit 94 may execute a workflow script associated with the network service to be purchased. By executing this workflow script, the lifecycle management unit 94 may instruct the container management unit 78 to deploy a container included in the new network service to be purchased. The container management unit 78 may then obtain a container image of the container from the repository unit 80 and deploy the container corresponding to the container image to a server.

[0083] In addition, in this embodiment, the life cycle management unit 94 executes, for example, scaling and replacement of elements included in the communication system 1. Here, the life cycle management unit 94 may output a container deployment instruction or deletion instruction to the container management unit 78. Then, the container management unit 78 may execute processing such as container deployment or container deletion in accordance with the instruction. In this embodiment, the life cycle management unit 94 is capable of executing scaling and replacement that cannot be handled by a tool such as Kubernetes in the container management unit 78.

[0084] Furthermore, the life cycle management unit 94 may output an instruction to create a communication path to the SDN controller 74. For example, the life cycle management unit 94 presents two IP addresses at both ends of the communication path to be created to the SDN controller 74, and the SDN controller 74 creates a communication path connecting these two IP addresses. The created communication path may be managed in association with these two IP addresses.

[0085] Furthermore, the life cycle management unit 94 may output to the SDN controller 74 an instruction to create a communication path between the two IP addresses that is associated with the two IP addresses.

[0086] In this embodiment, for example, the slice manager unit 92 performs instantiation of a network slice. In this embodiment, for example, the slice manager unit 92 performs instantiation of a network slice by executing logic indicated by a slice template stored in the service catalog storage unit 64.

[0087] The slice manager unit 92 is configured to include the functions of the NSMF (Network Slice Management Function) and the NSSMF (Network Slice Sub-network Management Function), for example, as described in the specification "TS28 533" of the 3GPP (registered trademark) (Third Generation Partnership Project). The NSMF is a function that generates and manages network slices and provides management services for NSIs. The NSSMF is a function that generates and manages network slice subnets that constitute part of the network slice and provides management services for NSSIs.

[0088] Here, the slice manager unit 92 may output a configuration management instruction related to instantiation of the network slice to the configuration management unit 76. Then, the configuration management unit 76 may perform configuration management such as setting in accordance with the configuration management instruction.

[0089] Furthermore, the slice manager unit 92 may present two IP addresses to the SDN controller 74 and output an instruction to create a communication path between these two IP addresses.

[0090] In this embodiment, the configuration management unit 76 executes configuration management such as setting of element groups such as NFs in accordance with configuration management instructions received from the life cycle management unit 94 and the slice manager unit 92, for example.

[0091] In this embodiment, the SDN controller 74 creates a communication path between two IP addresses associated with a communication path creation instruction received from, for example, the life cycle management unit 94 or the slice manager unit 92. The SDN controller 74 may create a communication path between two IP addresses using a known path calculation method such as Flex Algo.

[0092] For example, the SDN controller 74 may use a segment routing technology (e.g., SRv6 (segment routing IPv6)) to construct NSIs and NSSIs for aggregation routers, servers, and the like present along the communication paths. The SDN controller 74 may also generate NSIs and NSSIs across multiple target NFs by issuing commands to multiple target NFs to set up a common Virtual Local Area Network (VLAN) and commands to assign the bandwidth and priority indicated in the setting information to the VLAN.

[0093] In addition, the SDN controller 74 may perform operations such as changing the maximum bandwidth available for communication between two IP addresses without constructing a network slice.

[0094] The platform system 30 according to this embodiment may include multiple SDN controllers 74. Each of the multiple SDN controllers 74 may execute processing such as creating a communication path for a group of network devices such as an AG associated with the SDN controller 74.

[0095] In this embodiment, for example, the monitoring function unit 72 monitors the group of elements included in the communication system 1 in accordance with a given management policy. Here, the monitoring function unit 72 may monitor the group of elements in accordance with a monitoring policy specified by a purchaser when purchasing a network service, for example.

[0096] In this embodiment, the monitoring function unit 72 performs monitoring at various levels, such as the slice level, the NS level, the NF level, the CNFC level, and the hardware level of a server or the like.

[0097] For example, the monitoring function unit 72 may set a module that outputs metric data in hardware such as a server or in a software element included in the communication system 1 so that monitoring can be performed at the various levels described above. Here, for example, an NF may output metric data indicating metrics that are measurable (identifiable) in the NF to the monitoring function unit 72. Also, a server may output metric data indicating metrics related to hardware that is measurable (identifiable) in the server to the monitoring function unit 72.

[0098] Furthermore, for example, the monitoring function unit 72 may deploy a sidecar container on the server that aggregates metric data indicating metrics output from multiple containers for each CNFC (microservice). This sidecar container may include an agent called an exporter. The monitoring function unit 72 may repeatedly execute, at a given monitoring interval, a process of acquiring metric data aggregated for each microservice from the sidecar container, using the mechanisms of a monitoring tool such as Prometheus that can monitor container management tools such as Kubernetes.

[0099] The monitoring function unit 72 may monitor performance indicator values ​​for performance indicators described in, for example, “TS 28.552, Management and orchestration; 5G performance measurements” or “TS 28.554, Management and orchestration; 5G end to end Key Performance Indicators (KPI).” Then, the monitoring function unit 72 may obtain metric data indicating the monitored performance indicator values.

[0100] In this embodiment, the monitoring function unit 72 performs a process (enrichment) to aggregate metric data, for example, in a predetermined aggregation unit, thereby generating performance index value data indicating the performance index values ​​of the elements included in the communication system 1 in that aggregation unit.

[0101] For example, for one gNB, performance index value data for the gNB is generated by aggregating metric data indicating metrics of elements (e.g., network nodes such as DU42 and CU44) under the control of the gNB. In this way, performance index value data indicating communication performance in the area covered by the gNB is generated. Here, for example, performance index value data indicating multiple types of communication performance such as traffic volume (throughput) and latency may be generated for each gNB. Note that the communication performance indicated by the performance index value data is not limited to traffic volume and latency.

[0102] Then, the monitoring function unit 72 outputs the performance index value data generated by the above-mentioned enrichment to the data bus unit 68.

[0103] In this embodiment, for example, the data bus unit 68 receives performance index value data output from the monitoring function unit 72. Then, based on the received one or more pieces of performance index value data, the data bus unit 68 generates a performance index value file including the one or more pieces of performance index value data. Then, the data bus unit 68 outputs the generated performance index value file to the big data platform unit 66.

[0104] In addition, elements such as network slices, NSs, NFs, CNFCs, and hardware such as servers included in the communication system 1 notify the monitoring function unit 72 of various alerts (for example, notification of an alert triggered by the occurrence of a failure).

[0105] Then, for example, when the monitoring function unit 72 receives the above-mentioned alert notification, it outputs alert message data indicating the notification to the data bus unit 68. Then, the data bus unit 68 generates an alert file in which alert message data indicating one or more notifications are compiled into a single file, and outputs the alert file to the big data platform unit 66.

[0106] In this embodiment, the big data platform unit 66 accumulates, for example, performance index value files and alert files output from the data bus unit 68.

[0107] In this embodiment, for example, a plurality of trained machine learning models are stored in advance in the AI ​​unit 70. The AI ​​unit 70 uses the various machine learning models stored in the AI ​​unit 70 to perform estimation processing such as future prediction processing of the usage status and service quality of the communication system 1. The AI ​​unit 70 may generate estimation result data indicating the results of the estimation processing.

[0108] The AI ​​unit 70 may perform estimation processing based on the files stored in the big data platform unit 66 and the above-mentioned machine learning model. This estimation processing is suitable for infrequently predicting long-term trends.

[0109] The AI ​​unit 70 is also capable of acquiring performance index value data stored in the data bus unit 68. The AI ​​unit 70 may perform estimation processing based on the performance index value data stored in the data bus unit 68 and the above-described machine learning model. This estimation processing is suitable for performing short-term predictions frequently.

[0110] In this embodiment, for example, the performance management unit 88 calculates a performance index value (e.g., KPI) based on a plurality of metric data and the metrics indicated by the metric data. The performance management unit 88 may also calculate a performance index value that is an overall evaluation of a plurality of types of metrics (e.g., a performance index value related to an end-to-end network slice) that cannot be calculated from a single metric data. The performance management unit 88 may also generate overall performance index value data that indicates the performance index value that is the overall evaluation.

[0111] The performance management unit 88 may acquire the above-mentioned performance index value file from the big data platform unit 66. The performance management unit 88 may also acquire estimation result data from the AI ​​unit 70. Then, performance index values ​​such as KPIs may be calculated based on at least one of the performance index value file and the estimation result data. The performance management unit 88 may also directly acquire metric data from the monitoring function unit 72. Then, performance index values ​​such as KPIs may be calculated based on the metric data.

[0112] In this embodiment, the fault management unit 86 detects the occurrence of a fault in the communication system 1 based on, for example, at least one of the above-mentioned metric data, the above-mentioned alert notification, the above-mentioned estimation result data, and the above-mentioned overall performance index value data. The fault management unit 86 may detect the occurrence of a fault that cannot be detected from a single piece of metric data or a single alert notification, for example, based on a predetermined logic. The fault management unit 86 may generate detected fault data that indicates the detected fault.

[0113] The fault management unit 86 may obtain metric data and alert notifications directly from the monitoring function unit 72. The fault management unit 86 may also obtain performance index value files and alert files from the big data platform unit 66. The fault management unit 86 may also obtain alert message data from the data bus unit 68.

[0114] In this embodiment, the policy manager unit 90 executes a predetermined judgment process based on, for example, at least one of the above-mentioned metric data, the above-mentioned performance index value data, the above-mentioned alert message data, the above-mentioned performance index value file, the above-mentioned alert file, the above-mentioned estimation result data, the above-mentioned overall performance index value data, and the above-mentioned detected fault data.

[0115] The policy manager unit 90 may then execute an action according to the result of the determination process. For example, the policy manager unit 90 may output a command to construct a network slice to the slice manager unit 92. The policy manager unit 90 may also output a command to scale or replace an element to the life cycle management unit 94 according to the result of the determination process.

[0116] The policy manager unit 90 according to this embodiment is capable of acquiring performance index value data stored in the data bus unit 68. The policy manager unit 90 may then execute a predetermined determination process based on the performance index value data acquired from the data bus unit 68. The policy manager unit 90 may also execute a predetermined determination process based on alert message data stored in the data bus unit 68.

[0117] In this embodiment, for example, the ticket management unit 84 generates a ticket indicating the content to be notified to the administrator of the communication system 1. The ticket management unit 84 may generate a ticket indicating the content of the occurred fault data. The ticket management unit 84 may also generate a ticket indicating the value of performance index value data or metric data. The ticket management unit 84 may also generate a ticket indicating the determination result by the policy manager unit 90.

[0118] Then, the ticket management unit 84 notifies the administrator of the communication system 1 of the generated ticket. For example, the ticket management unit 84 may send an email with the generated ticket attached to the email address of the administrator of the communication system 1.

[0119] The generation of the performance index value file, the determination process based on the performance index value data stored in the data bus unit 68, and the estimation process based on the performance index value data stored in the data bus unit 68 will be further explained below.

[0120] 7 is a diagram schematically illustrating an example of the data bus unit 68 according to this embodiment. As shown in Fig. 7, the data bus unit 68 according to this embodiment includes, for example, a plurality of queues 100 that hold performance index value data in a first-in, first-out list structure.

[0121] Each queue 100 belongs to either the first queue group 102a or the second queue group 102b.

[0122] Furthermore, in this embodiment, for example, a plurality of aggregation processes 104 are running in the monitoring function unit 72. In each aggregation process 104, an element to be aggregated by that aggregation process 104 is set in advance. For example, in each aggregation process 104, a gNB to be aggregated by that aggregation process 104 is set in advance. Then, each aggregation process 104 acquires metric data from NFs (e.g., RU 40, DU 42, and CU-UP 44b) under the gNB that is the aggregation target of that aggregation process 104. Then, based on the acquired metric data, that aggregation process 104 executes enrichment processing to generate performance index value data indicating the communication performance of that gNB.

[0123] In this embodiment, for example, the counting process 104 and the queue 100 are associated in advance. For convenience, Fig. 7 shows that the counting process 104 and the queue 100 are associated in a one-to-one relationship, but the counting process 104 and the queue 100 may be associated in a many-to-many relationship.

[0124] Hereinafter, the counting process 104 associated with the queue 100 included in the first queue group 102a will be referred to as the first group counting process 104a, and the counting process 104 associated with the queue 100 included in the second queue group 102b will be referred to as the second group counting process 104b.

[0125] Then, each first group aggregation process 104a generates performance index value data by aggregating the metric data associated with that first group aggregation process 104a from the previous aggregation to the present time at a predetermined time interval (for example, every minute).

[0126] The first group counting process 104a acquires metric data from one or more NFs associated with the first group counting process 104a, for example, at one-minute intervals. Then, the first group counting process 104a counts the metric data for the same counting period to generate performance index value data for the counting period.

[0127] Then, every time the first group counting process 104a generates performance index value data, it enqueues the performance index value data in one or more queues 100 associated with the first group counting process 104a.

[0128] Then, each second group aggregation process 104b generates performance index value data by aggregating the metric data associated with that second group aggregation process 104b from the previous aggregation to the present time at a predetermined time interval (for example, every 15 minutes).

[0129] The second group counting process 104b acquires metric data from one or more NFs associated with the second group counting process 104b, for example, at 15-minute intervals. The second group counting process 104b then counts the metric data for the same counting period to generate performance index value data for the counting period.

[0130] Then, every time the second group counting process 104b generates performance index value data, it enqueues the performance index value data in one or more queues 100 associated with the second group counting process 104b.

[0131] In this embodiment, the maximum number of performance index value data that can be stored in the queues 100 included in the first queue group 102a is predetermined. Here, for example, it is assumed that a maximum of 240 performance index value data can be stored in the queues 100. In other words, the maximum number is set to "240."

[0132] In this embodiment, the maximum number of performance index value data that can be stored in the queues 100 included in the second queue group 102b is predetermined. Here, for example, it is assumed that a maximum of four performance index value data can be stored in the queues 100. In other words, the maximum number is set to "4."

[0133] In this embodiment, for example, a plurality of determination processes 106 (see FIGS. 8 and 9) are running in the policy manager unit 90. Some of these determination processes 106 execute determination processing based on the performance index value data stored in the data bus unit 68, and the rest execute determination processing based on the files stored in the big data platform unit 66.

[0134] Among the determination processes 106 according to this embodiment, there is one that acquires performance index value data that indicates the actual value of the performance index value related to the communication system 1. For example, there is a determination process 106 that acquires performance index value data in response to the performance index value data being enqueued in a queue 100 included in the first queue group 102a.

[0135] In this embodiment, for the queues 100 included in the first queue group 102a, any performance index value data included in the queues 100 can be accessed (obtained) without dequeuing.

[0136] The determination process 106 then determines the state of the communication system 1 based on the acquired performance index value data. Here, for example, the state of an element included in the communication system 1 and associated with the determination process 106 may be determined. For example, the state of an element that is the target of aggregation in the first group aggregation process 104a that generated the performance index value data acquired by the determination process 106 may be determined. Hereinafter, such a determination process 106 will be referred to as a performance determination process 106a.

[0137] In this embodiment, for example, the performance determination process 106a and the queue 100 are associated in advance. For convenience, although the performance determination process 106a and the queue 100 are shown associated in a one-to-one relationship in Figures 8 and 9, the performance determination process 106a and the queue 100 may be associated in a many-to-many relationship.

[0138] Here, for example, in response to performance index value data being enqueued in a queue 100 included in the first queue group 102a, the data bus unit 68 may output a notification indicating that the performance index value data has been enqueued to one or more performance determination processes 106a associated with the queue 100.

[0139] Then, the performance determination process 106a that has received the notification may acquire the latest performance index value data stored in the queue 100 in response to the reception of the notification.

[0140] Furthermore, some of the determination processes 106 according to this embodiment acquire estimation result data indicating the estimation result by an estimation process 108 (see FIG. 9 ) associated with the determination process 106. The determination process 106 then judges the state of the communication system 1 based on the acquired estimation result data. Here, for example, the state of an element included in the communication system 1 and associated with the determination process 106 may be judged. For example, the state of an element that is the target of aggregation in the first group aggregation process 104a that generated the performance index value data acquired by the estimation process 108 may be judged. Hereinafter, such a determination process 106 will be referred to as a prediction determination process 106b.

[0141] In this embodiment, for example, a plurality of estimation processes 108 (see FIG. 9 ) are running in the AI ​​unit 70. Some of these estimation processes 108 execute estimation processing based on performance index value data stored in the data bus unit 68, and the rest execute estimation processing based on files stored in the big data platform unit 66.

[0142] In the present embodiment, for example, the estimation process 108 is associated in advance with the queue 100. For convenience, Fig. 8 shows that the estimation process 108 and the queue 100 are associated in a one-to-one relationship, but the estimation process 108 and the queue 100 may be associated in a many-to-many relationship.

[0143] In this embodiment, for example, each estimation process 108 acquires performance index value data stored in the queue 100 included in the first queue group 102a corresponding to that estimation process 108. Then, that estimation process executes an estimation process that is predetermined for that estimation process 108 based on the performance index value data.

[0144] Here, for example, in response to performance index value data being enqueued in a queue 100 included in the first queue group 102a, the estimation process 108 acquires performance index value data for the nearest constant number or nearest period that includes at least the most recent performance index value data from the performance index value data stored in the queue 100.

[0145] Here, for example, in response to performance index value data being enqueued in a queue 100 included in the first queue group 102a, the data bus unit 68 may output a notification indicating that the performance index value data has been enqueued to one or more estimation processes 108 associated with the queue 100.

[0146] Then, the estimation process 108 that has received the notification may, in response to receiving the notification, acquire performance index value data for the nearest constant number or nearest period that includes at least the latest performance index value data from the performance index value data stored in the queue 100.

[0147] 9 acquires 60 pieces of estimated performance index value data, including the most recent performance index value data. These pieces of performance index value data correspond to the most recent 60 minutes of performance index value data, including the most recent performance index value data. Then, the estimation process 108 executes estimation processing based on the performance index value data.

[0148] For example, suppose that a first group aggregation process 104a associated with a specific gNB generates performance index value data for the gNB by aggregating metric data related to elements included in the gNB (e.g., elements subordinate to the gNB). Then, suppose that an estimation process 108 that acquires the performance index value data generated by the first group aggregation process 104a acquires 60 pieces of performance index value data including the latest performance index value data stored in the queue 100 in response to the performance index value data being enqueued in the queue 100.

[0149] In this case, the estimation process 108 predicts the level of network load of the gNB from the present time to 20 minutes from now based on these 60 performance index value data using a trained machine learning model pre-stored in the AI ​​unit 70. Here, for example, traffic volume (throughput), latency, etc. may be predicted as the level of network load of the gNB.

[0150] The machine learning model may be, for example, an existing predictive model. Alternatively, the machine learning model may be, for example, a trained machine learning model that has been previously subjected to supervised learning using a plurality of training data elements. Each of the plurality of training data elements may include, for example, training input data indicating, for each different given point in time, the traffic volume at the gNB for the 60 minutes up to that point in time, and supervised data indicating the level of network load (e.g., traffic volume or latency) at the gNB from that point in time to 20 minutes ahead of that point in time.

[0151] It should be noted that the estimation process 108 does not need to acquire a portion of the performance index value data stored in the queue 100 as described above, but may acquire all of the performance index value data stored in the queue 100 .

[0152] Then, the estimation process 108 outputs estimation result data indicating the execution result (estimation result) of the estimation process to the prediction determination process 106b associated with the estimation process 108. Then, the prediction determination process 106b acquires the estimation result data. Then, the prediction determination process 106b judges the state of the communication system 1 based on the acquired estimation result data.

[0153] As described above, the queue 100 according to this embodiment is associated with the aggregation process 104, the performance determination process 106a, the prediction determination process 106b, and the estimation process 108.

[0154] In addition, in this embodiment, the data bus unit 68 generates a performance index value file containing at least a portion of the performance index value data stored in the queue 100, for example, less frequently than the frequency at which the AI ​​unit 70 acquires the performance index value data.

[0155] For example, the data bus unit 68 may generate, at predetermined time intervals, a performance index file containing performance index data stored in the queue 100 after the previous performance index file was generated.

[0156] Here, the time interval may or may not match the time corresponding to the maximum number of performance index value data that can be stored in the queues 100 included in the first queue group 102a (60 minutes in the above example).

[0157] Furthermore, for example, when all of the performance index value data included in the generated performance index value file has been dequeued, the data bus unit 68 may generate a file containing all of the performance index value data stored in the queue 100. In other words, when all of the performance index value data stored in the queue 100 has been replaced, a file containing all of the performance index value data stored in the queue 100 may be generated.

[0158] Furthermore, in this embodiment, when 60 pieces of performance index value data are stored in a queue 100 included in the first queue group 102a, if new performance index value data is enqueued, the oldest performance index value data stored in that queue 100 is dequeued. In other words, the oldest performance index value data stored in that queue 100 is deleted from that queue 100.

[0159] In this embodiment, when four pieces of performance index value data are stored in the queues 100 included in the second queue group 102b, the data bus unit 68 generates a performance index value file that combines these four pieces of performance index value data into a single file. The data bus unit 68 then outputs the generated performance index value file to the big data platform unit 66.

[0160] Then, the data bus unit 68 dequeues all of the performance index value data stored in the queue 100. In other words, all of the performance index value data stored in the queue 100 is deleted from the queue 100.

[0161] As described above, the queues 100 included in the first queue group 102a and the queues 100 included in the second queue group 102b perform different processes in response to the generation of a performance index file. In the queues 100 included in the second queue group 102b, all performance index data stored in the queues 100 are deleted from the queues 100 in response to the generation of a performance index file. On the other hand, in the queues 100 included in the first queue group 102a, dequeuing is not performed in response to the generation of a performance index file.

[0162] In this embodiment, for example, a purchaser of a network service can select an option related to monitoring settings when purchasing the network service. In the following description, it is assumed that the purchaser of the network service can select one of a low-level option, a medium-level option, and a high-level option.

[0163] Here, for example, if the low-level option is selected, when a network service is constructed, not only the elements included in the network service but also the queue 100 associated with the elements and the aggregation process 104 associated with the elements are generated, as shown in Fig. 7. In this case, performance index value files related to the elements included in the network service are accumulated in the big data platform unit 66.

[0164] Furthermore, for example, when the medium-level option is selected, when a network service is constructed, not only the elements included in the network service but also the queues 100 associated with the elements and the aggregation processes 104 associated with the elements are generated, as in the low-level option. Furthermore, as shown in Fig. 8, a performance determination process 106a associated with the queue 100 is also generated.

[0165] At this time, the policy manager unit 90 may refer to the inventory data and check the attributes of the elements associated with the performance determination process 106a to be generated.The policy manager unit 90 may then generate the performance determination process 106a in which a workflow corresponding to the checked attributes is set.The performance determination process 106a may then perform the determination process by executing the workflow set in the performance determination process 106a.

[0166] For example, the performance determination process 106a may determine whether or not a scale-out is necessary based on the acquired performance index value data.

[0167] In this embodiment, for example, when it is determined that scale-out is necessary, the platform system 30 may execute scale-out of the elements included in the communication system 1. For example, the policy manager unit 90, the life cycle management unit 94, the container management unit 78, and the configuration management unit 76 may execute scale-out in cooperation with each other. For example, when it is determined that scale-out is necessary based on performance index value data related to a specific gNB, scale-out of the DU 42 and the CU-UP 44b included in the gNB may be executed.

[0168] For example, the performance determination process 106a may determine whether the acquired performance index value data satisfies a predetermined first scale-out condition. Here, it may be determined whether the performance index value indicated by the performance index value data exceeds a threshold th1. This performance index value may be a value indicating the level of network load, such as traffic volume (throughput) or latency. Then, in response to the determination that the first scale-out condition is satisfied (for example, the performance index value is determined to exceed the threshold th1), scale-out of the elements included in the communication system 1 may be executed.

[0169] Also, for example, if a high-level option is selected, when the network service is constructed, not only the elements included in the network service are generated, but also the queue 100 associated with the element, the aggregation process 104 associated with the element, and the performance determination process 106a associated with the queue 100, as with the low-level and medium-level options.

[0170] 9, the AI ​​unit 70 generates an estimation process 108 associated with the performance determination process 106a, and the policy manager unit 90 generates a prediction determination process 106b associated with the performance determination process 106a. Here, for example, the estimation process 108 and the prediction determination process 106b may be activated. At this time, instantiation of a trained machine learning model may also be performed. Then, the estimation process 108 may perform estimation using the machine learning model instantiated in this manner.

[0171] Then, the prediction determination process 106b may execute a predetermined determination process based on the estimation result data output by the estimation process 108 associated with the prediction determination process 106b. For example, the prediction determination process 106b may determine whether or not a scale-out is necessary based on the network load prediction result.

[0172] 9 , in response to performance index value data being enqueued in a queue 100 included in the first queue group 102a, the performance determination process 106a may acquire the enqueued performance index value data, and the estimation process 108 may acquire performance index value data for an immediately neighboring constant number of periods or an immediately neighboring period that includes at least the enqueued performance index value data from the performance index value data stored in the queue 100. In this way, in response to performance index value data being enqueued in the queue 100, the enqueued performance index value data may be acquired by both the performance determination process 106a and the estimation process 108.

[0173] Then, the performance determination process 106a may determine whether or not a scale-out is necessary based on the acquired performance index value data.

[0174] Furthermore, the estimation process 108 may generate estimation result data indicating the network load prediction result based on the acquired performance index value data. Then, the estimation process 108 may output the generated estimation result data to the prediction determination process 106b. Then, the prediction determination process 106b may acquire the estimation result data.

[0175] Then, the prediction determination process 106b may determine whether or not a scale-out is necessary based on the obtained estimation result data.

[0176] It is not necessary for the AI ​​unit 70 to generate the estimation process 108 and the policy manager unit 90 to generate the prediction determination process 106b. For example, the performance determination process 106a may generate the estimation process 108 and the policy manager unit 90.

[0177] In this embodiment, for example, the platform system 30 executes scale-out of the elements included in the communication system 1 in response to determining that scale-out is necessary.

[0178] For example, the prediction determination process 106b may determine whether a predicted value of the network load indicated by the estimation result data satisfies a predetermined second scale-out condition. For example, it may determine whether the predicted value exceeds a threshold th2. Here, for example, it may determine whether any of a plurality of predicted values ​​from the present time up to 20 minutes from the present time exceeds the threshold th2. This predicted value may be a value indicating the level of the network load, such as traffic volume (throughput), latency, etc. Then, in response to the determination that the second scale-out condition is satisfied, scaling out of the elements included in the communication system 1 may be executed. Note that the second scale-out condition may be the same as or different from the first scale-out condition described above.

[0179] In addition, in this embodiment, when a purchaser of a network service selects a medium-level option, the purchaser may be able to specify elements to be included in the communication system 1. Then, a performance determination process 106a for the specified elements may be generated.

[0180] In addition, in this embodiment, when a purchaser of a network service selects a high-level option, the purchaser may be able to specify the elements to be included in the communication system 1. Then, a performance determination process 106a, an estimation process 108, and a prediction determination process 106b may be generated for the specified elements.

[0181] Also, when a high-level option is selected, the performance determination process 106a may not be generated. Also, options related to monitoring settings may be changed according to the purchaser's request.

[0182] Also, in this embodiment, for example, the AI ​​unit 70 determines at least one (for example, at least one suitable for the communication system 1) from among a plurality of machine learning models used for a given prediction purpose related to the communication system 1.

[0183] The process of determining the machine learning model will be further explained below.

[0184] In the following description, it is assumed that a purchaser of a network service selects the medium-level option when purchasing the network service, and that the performance determination process 106a has performed a determination process (in other words, a process of monitoring at least one type of performance index value related to the communication system 1) for some elements included in the network service.

[0185] It is assumed that the inventory database 82 stores performance determination target data indicating the type of element being monitored by the performance determination process 106a and the type of performance index value being monitored for that element.

[0186] Furthermore, in the communication system 1 according to this embodiment, the AI ​​unit 70 stores multiple machine learning models to be used for each of a plurality of given prediction purposes, and these machine learning models are in a state where they can be instantiated.

[0187] The inventory database 82 or the AI ​​unit 70 stores model management data, such as the data shown in FIG. 10, for managing these machine learning models.

[0188] 10 shows model management data associated with one prediction objective. In this embodiment, for example, the inventory database 82 or the AI ​​unit 70 stores model management data associated with each of a plurality of prediction objectives.

[0189] As shown in FIG. 10, the model management data includes objective management data and AI management data.

[0190] The goal management data included in the model management data is data associated with the above-mentioned prediction goal. The goal management data includes, for example, a goal ID and goal data, as shown in FIG. 10. The goal ID included in the goal management data is, for example, an identifier of the prediction goal associated with the goal management data. The goal data included in the goal management data is, for example, data indicating the prediction goal associated with the goal management data. In the goal management data shown in FIG. 10, the prediction goal indicated by the goal data is expressed as "a1".

[0191] The machine learning model according to this embodiment may output a predicted value of at least one type of performance index value. The type of predicted performance index value may be indicated in the target data. For example, the type of performance index value that is the predicted value output by the machine learning model may be indicated in the target data. Specifically, for example, the value of the target data may be "throughput," "latency," "number of registrations," "number of completed connections," "number of active users," etc.

[0192] The objective data may also indicate a predicted objective for a specific type of element included in the communication system 1, such as "UPF throughput" (e.g., the type of element and the type of performance index value predicted for that type of element).

[0193] The target data may also indicate the type of value calculated based on multiple types of performance index values. For example, a calculation formula for calculating an overall performance evaluation value based on throughput and latency may be set in the value of the target data.

[0194] The AI ​​management data included in the model management data is data for managing machine learning models used for prediction purposes that are associated with the model management data. The model management data includes multiple AI data that are each associated with a different machine learning model. The AI ​​data includes an AIID and one or more input performance index value data.

[0195] For example, if three machine learning models with a prediction objective of "a1" are prepared, the model management data will contain three pieces of AI data, as shown in Figure 10. Note that the number of machine learning models used for one prediction objective is not limited to three.

[0196] The AIID included in the AI ​​data is the identifier of the machine learning model used for the prediction purpose associated with the model management data. In the example of Figure 10, the AIIDs of the three machine learning models with the prediction purpose "a1" are "001," "002," and "003," respectively.

[0197] The input performance index value data included in the AI ​​data is data indicating the types of performance index values ​​to be input to the machine learning model associated with the AI ​​data. In this embodiment, for example, the AI ​​data associated with the machine learning model includes input performance index value data in the same number as the number of performance index values ​​to be input to the machine learning model.

[0198] The example in Figure 10 shows that the type of performance index value input to the machine learning model with AIID "001" is "b11." It also shows that the types of performance index values ​​input to the machine learning model with AIID "002" are "b21" and "b22." It also shows that the types of performance index values ​​input to the machine learning model with AIID "003" are "b31," "b32," and "b33."

[0199] In this way, the number of types of performance index values ​​input to a machine learning model may differ depending on the machine learning model. Also, in the example of Fig. 10, the number of types of performance index values ​​input to a machine learning model is one to three, but the number of types of performance index values ​​input to a machine learning model may be four or more.

[0200] Furthermore, the type of performance index value input to one machine learning model may be included in the type of performance index value input to another machine learning model. Furthermore, some of the types of performance index values ​​input to one machine learning model may overlap with some of the types of performance index values ​​input to another machine learning model. For example, "b11" and "b21" may be the same type of performance index value.

[0201] Furthermore, in this embodiment, the type of performance index value indicated by the input performance index value data and the type of performance index value associated with the predicted objective indicated by the objective data may be the same or different.

[0202] For example, "a1" and "b11" may be the same type of performance index value. For example, a case where a performance index value after a certain point in time is predicted based on the output when the actual value of throughput at that point in time is input into a machine learning model corresponds to a case where the type of performance index value indicated by the input performance index value data and the type of performance index value indicated by the target data are the same.

[0203] Specific examples of the types of performance index values ​​that can be input include "throughput," "latency," "number of registrations," "number of completed connections," and "number of active users."

[0204] Here, the input performance index value data may indicate the type of element and the type of performance index value for that type of element. For example, if the performance index value "throughput" for the element "UPF" is input to a machine learning model, the AI ​​data associated with the machine learning model may include input performance index value data whose value is "UPF throughput."

[0205] Then, in this embodiment, for example, the AI ​​unit 70 identifies, for each of a plurality of machine learning models used for a given prediction purpose related to the communication system 1, an additional performance index value type, which is the type of performance index value that needs to be added to the monitoring targets in order to use the machine learning model.

[0206] Here, the AI ​​unit 70 may identify, for each of the multiple machine learning models, a type of performance index value that is not included in the monitoring targets among the types of performance index values ​​that are input to the machine learning model, as an additional performance index value type.

[0207] For example, for each of multiple AI data included in model management data that includes goal management data indicating a given prediction goal, it may be determined whether the type of performance index value indicated by the input performance index value data included in the AI ​​data is indicated in the above-mentioned performance judgment target data.

[0208] Then, among the types of performance index values ​​indicated by the input performance index value data, types of performance index values ​​that are not indicated in the above-mentioned performance judgment target data may be determined as additional performance index value types for the machine learning model associated with the AI ​​data.

[0209] Then, in this embodiment, for example, the AI ​​unit 70 determines at least one of the multiple machine learning models based on the type of additional performance index value identified for each machine learning model.

[0210] Here, the AI ​​unit 70 may determine at least one of the multiple machine learning models based on the number of types of additional performance indicator values. For example, a machine learning model associated with AI data having the fewest specified types of additional performance indicator values ​​may be determined. Alternatively, a machine learning model associated with AI data having fewer specified types of additional performance indicator values ​​than a predetermined number may be determined.

[0211] Furthermore, the AI ​​unit 70 may determine at least one of the multiple machine learning models based on the ratio of the number of additional performance indicator value types to the number of types of performance indicator values ​​input to the machine learning model. For example, a machine learning model may be determined that is associated with AI data for which the ratio of the number of identified additional performance indicator value types to the total number of input performance indicator value data is the smallest. Alternatively, a machine learning model may be determined that is associated with AI data for which the ratio of the number of identified additional performance indicator value types to the total number of input performance indicator value data is smaller than a predetermined ratio.

[0212] Then, the AI ​​unit 70 may add performance indicator values ​​of the additional performance indicator value type that need to be added to use the machine learning model determined in this way to the targets to be monitored by the performance determination process 106a. For example, a performance determination process 106a may be generated that is associated with the performance indicator values ​​of the additional performance indicator value type for the trained machine learning model determined in this way.

[0213] Then, an estimation process 108 and a prediction determination process 106b associated with the performance determination process 106a may be generated. Here, for example, the estimation process 108 and the prediction determination process 106b may be activated. Then, the trained machine learning model determined in this manner may be instantiated. Then, the estimation process 108 may predict a performance index value of the communication system 1 using the machine learning model determined in this manner.

[0214] Alternatively, the machine learning model determined in this manner may be recommended to a user such as a purchaser. For example, a recommendation screen shown in FIG. 11 may be displayed on a terminal used by a purchaser of a network service. Then, in response to clicking a purchase button 110 arranged on the recommendation screen, a performance determination process 106a, an estimation process 108, and a prediction determination process 106b may be generated, and the machine learning model may be instantiated. Then, the estimation process 108 may predict a performance index value of the communication system 1 using the machine learning model determined in this manner.

[0215] In addition, in this embodiment, for each of a plurality of prediction purposes, at least one (for example, at least one suitable for the communication system 1) may be determined from a plurality of machine learning models used for the prediction purpose.

[0216] Here, we will explain an example of the processing flow for determining a machine learning model suitable for a network service purchased by a specific purchaser, which is part of the communication system 1, performed in the platform system 30 of this embodiment, with reference to the flow diagram shown in Figure 11.

[0217] In this processing example, it is assumed that the prediction objective of the machine learning model to be determined is given, and that model management data associated with the prediction objective is stored in advance in the inventory database 82 or the AI ​​unit 70.

[0218] First, the AI ​​unit 70 identifies the type of element monitored by the performance determination process 106a in the network service and the type of performance index value monitored for the element (S101). Here, for example, performance determination target data related to the network service may be identified.

[0219] Then, the AI ​​unit 70 identifies a plurality of AI data included in the model management data for the machine learning model for the given prediction purpose (S102).

[0220] Then, for each of the multiple AI data identified in the process shown in S102, the AI ​​unit 70 identifies the number of additional performance index value types for the machine learning model associated with the AI ​​data based on the type of element and type of performance index value identified in the process shown in S101 and the AI ​​data (S103).

[0221] Then, the AI ​​unit 70 determines at least one of the multiple machine learning models as a machine learning model suitable for the network service based on the number of additional performance index value types identified for each of the multiple machine learning models in the process shown in S103 (S104).Then, the process shown in this process example ends.

[0222] As described above, the ratio of the number of types of added performance index values ​​to the number of types of performance index values ​​input to the machine learning model may be specified in the process shown in S103. Then, in the process shown in S104, a machine learning model suitable for the network service may be determined based on the ratio specified in this way.

[0223] Even if the prediction purpose is the same, various patterns can be assumed for the types of actual values ​​of performance indicators that are input into a machine learning model, which correspond to explanatory variables for prediction.

[0224] On the other hand, the types of performance index values ​​for which actual values ​​are monitored vary depending on the situation, so even if a machine learning model is used for the same prediction purpose, some may be suitable for the communication system 1 and some may not.

[0225] In this embodiment, as described above, a machine learning model suitable for the communication system 1 is determined based on the type of additional performance index value. For example, a machine learning model that imposes a small burden on additional monitoring is determined. In this way, according to this embodiment, it is possible to accurately determine a machine learning model suitable for the communication system 1, which is used to predict the performance index value of the communication system 1.

[0226] The present invention is not limited to the above-described embodiment.

[0227] For example, in the present embodiment, scale-out may be performed on elements of the core network system 34 instead of elements of the RAN 32 such as gNB. For example, scale-out may be performed on the AMF 46, the SMF 48, or the UPF 50. In this case, performance index value data related to the elements of the core network system 34 may be used to determine whether to perform scale-out. Alternatively, performance index value data related to the elements of the RAN 32 and the elements of the core network system 34 may be used to make this determination.

[0228] Similarly, transport scale-out may be performed.

[0229] In addition, in this embodiment, a purchaser of a network service may be able to refer to the contents of a performance index value file for the elements included in the network service, which is stored in the big data platform unit 66, for example, via a dashboard screen.

[0230] Furthermore, the above-described process of determining a machine learning model and processes related to the determination process may be executed by a functional module other than the AI ​​unit 70.

[0231] Furthermore, the functional units according to this embodiment are not limited to those shown in FIG.

[0232] Furthermore, the functional unit according to this embodiment does not need to be a 5G NF. For example, the functional unit according to this embodiment may be a 4G network node such as an eNodeB, a vDU, a vCU, a Packet Data Network Gateway (P-GW), a Serving Gateway (S-GW), a Mobility Management Entity (MME), or a Home Subscriber Server (HSS).

[0233] Furthermore, the functional units according to the present embodiment may be realized using hypervisor-type or host-type virtualization technology instead of container-type virtualization technology. Furthermore, the functional units according to the present embodiment do not need to be implemented by software, but may be implemented by hardware such as electronic circuits. Furthermore, the functional units according to the present embodiment may be implemented by a combination of electronic circuits and software.

[0234] The technology described in this disclosure can also be expressed as follows. [1] monitoring means for monitoring at least one type of performance indicator associated with the communications system; an additional performance index value type specifying means for specifying, for each of a plurality of machine learning models used for a given prediction purpose related to the communication system, an additional performance index value type that is a type of performance index value that needs to be added to the monitoring targets in order to use the machine learning model; a model determination means for determining at least one of the plurality of machine learning models based on the type of additional performance index value identified for each of the machine learning models; A model determination system comprising: [2] the additional performance index value type identification means identifies, for each of the plurality of machine learning models, a type of performance index value that is not included in the targets of monitoring, from among types of performance index values ​​that are input to the machine learning model; and The model determination system according to [1]. [3] the model determination means determines at least one of the plurality of machine learning models based on the number of types of additional performance index values. The model determination system according to [1] or [2], [4] the model determination means determines at least one of the plurality of machine learning models based on a ratio of the number of types of additional performance index values ​​to the number of types of performance index values ​​input to the machine learning model; The model determination system according to [1] or [2], [5] a monitoring target adding means for adding a performance index value of the additional performance index value type that needs to be added in order to use the determined machine learning model to a monitoring target by the monitoring means; A model determination system according to any one of [1] to [4]. [6] and a prediction means for predicting a performance index value of the communication system using the determined machine learning model. A model determination system according to any one of [1] to [5]. [7] monitoring at least one type of performance indicator associated with the communications system; Identifying, for each of a plurality of machine learning models used for a given prediction purpose related to the communication system, an additional performance index value type that is a type of performance index value that needs to be added to the monitoring targets in order to use the machine learning model; determining at least one of the plurality of machine learning models based on the additional performance metric value type identified for each of the machine learning models; A model determination method comprising:

Claims

1. A monitoring process for monitoring at least one type of performance index value related to a communication system; an additional performance index value type identification process for identifying, for each of a plurality of machine learning models used for a given prediction purpose related to the communication system, an additional performance index value type that is a type of performance index value that needs to be added to the monitoring targets in order to use the machine learning model; a model determination process for determining at least one of the plurality of machine learning models based on the additional performance index value type identified for each of the machine learning models; A model decision system that executes.

2. In the additional performance index value type identification process, for each of the plurality of machine learning models, a type of performance index value that is not included in the targets of monitoring is identified as the additional performance index value type from among types of performance index values ​​that are input to the machine learning model. The model determination system of claim 1 .

3. In the model determination process, at least one of the plurality of machine learning models is determined based on the number of types of additional performance index values. The model determination system of claim 1 .

4. In the model determination process, at least one of the plurality of machine learning models is determined based on a ratio of the number of types of the additional performance index values ​​to the number of types of performance index values ​​input to the machine learning model. The model determination system of claim 1 .

5. further performing a monitoring target addition process of adding a performance index value of the additional performance index value type that needs to be added in order to use the determined machine learning model to a monitoring target in the monitoring process; The model determination system of claim 1 .

6. and further performing a prediction process for predicting a performance index value of the communication system using the determined machine learning model. The model determination system of claim 1 .

7. monitoring at least one type of performance indicator associated with a communications system; Identifying, for each of a plurality of machine learning models used for a given prediction purpose related to the communication system, an additional performance index value type that is a type of performance index value that needs to be added to the monitoring targets in order to use the machine learning model; determining at least one of the plurality of machine learning models based on the additional performance indicator value type identified for each of the machine learning models; A method for determining a model, performed by one or more computers, comprising:

Images