Display control of monitoring screen on which performance index value of element included in communication system is indicated

Abstract

The present invention makes it possible to suppress waste of a resource to be used for displaying a monitoring screen on which a performance index value of an element included in a communication system is indicated. A management assistance unit (84) repeatedly updates a monitoring screen on which a performance index value, at at least one time point, of an element included in a communication system has been indicated. A policy manager unit (90), a life cycle management unit (94), a container management unit (78), and a configuration management unit (76) each execute an action with respect to an element upon receiving an execution instruction of a prescribed action with respect to the element. The management assistance unit (84) executes the start of display of a prediction value of the performance index value and / or the shortening of an interval of updating the monitoring screen when a reception status of the execution instruction satisfies a prescribed condition.

Description

Display control of a monitoring screen showing performance index values ​​of elements included in a communication system

[0001] The present invention relates to display control of a monitoring screen showing performance index values ​​of elements included in a communication system.

[0002] Patent Document 1 describes that a traffic analysis module monitors traffic on a customer's network and analyzes the type and flow of traffic.

[0003] International Publication No. 2018 / 181826

[0004] In the technology described in Patent Literature 1, for example, the amount of information displayed on the monitoring screen can be increased by executing a process of predicting performance index values ​​of elements included in a communication system and displaying a monitoring screen showing the predicted values, or a process of frequently updating a monitoring screen showing the latest performance index values ​​of the elements. However, executing such processes uniformly for all elements included in a communication system is a waste of resources.

[0005] The present invention has been made in consideration of the above-mentioned situation, and one of its purposes is to make it possible to reduce the waste of resources used to display a monitoring screen showing the performance index values ​​of elements included in a communication system.

[0006] In order to solve the above problems, the display control system of the present disclosure includes a monitoring screen update means that repeatedly updates a monitoring screen showing performance index values ​​of elements included in a communication system at at least one point in time, an action execution means that executes a given action on the element in response to receiving an instruction to execute the action, and a monitoring change execution means that, when the reception status of the execution instruction satisfies a given condition, starts displaying a predicted value of the performance index value or shortens the update interval of the monitoring screen.

[0007] In addition, the display control method according to the present disclosure includes repeatedly updating a monitoring screen showing performance index values ​​of elements included in a communication system at at least one point in time, executing a given action on the element in response to receiving an instruction to execute the action, and, when the reception status of the execution instruction satisfies a given condition, performing at least one of starting to display a predicted value of the performance index value or shortening the update interval of the monitoring screen.

[0008] 1 is a diagram illustrating an example of a communication system according to an embodiment of the present invention. FIG. 1 is a diagram illustrating an example of a communication system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a network service according to an embodiment of the present invention. FIG. 3 is a diagram illustrating an example of an association between elements established in a communication system according to an embodiment of the present invention. FIG. 4 is a functional block diagram illustrating an example of a function implemented in a platform system according to an embodiment of the present invention. FIG. 5 is a diagram illustrating an example of a data structure of physical inventory data. FIG. 6 is a diagram illustrating an example of a data bus unit according to an embodiment of the present invention. FIG. 7 is a diagram illustrating an example of acquisition of a performance index value file by a file determination process. FIG. 8 is a diagram illustrating an example of acquisition of performance index value data by a current status determination process, and an acquisition of a performance index value file by a file determination process. FIG. 9 is a diagram illustrating an example of acquisition of performance index value data by a current status determination process and an estimation process, and an acquisition of a performance index value file by a file determination process. FIG. 10 is a diagram illustrating an example of a monitoring screen. FIG. 11 is a diagram illustrating an example of a monitoring screen. FIG. 12 is a diagram illustrating an example of a recommendation screen. FIG. 13 is a diagram illustrating an example of model management data. FIG. 14 is a diagram illustrating an example of a learning process and a test process. FIG. 15 is a diagram illustrating an example of a training data element set. FIG. 16 is a diagram illustrating an example of training data elements. FIG. 17 is a diagram illustrating an example of a test data element set. FIG. 18 is a diagram illustrating an example of test data elements. FIG. 2 is a flowchart showing an example of a flow of processing performed in a platform system according to an embodiment of the present invention.

[0009] [Overall Configuration] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0010] 1 and 2 are diagrams illustrating an example of a communication system 1 according to an embodiment of the present invention. Fig. 1 is a diagram focusing on the locations of a group of data centers included in the communication system 1. Fig. 2 is a diagram focusing on various computer systems implemented in the group of data centers included in the communication system 1.

[0011] As shown in FIG. 1 , the data centers included in the communication system 1 are classified into a central data center 10 , regional data centers 12 , and edge data centers 14 .

[0012] For example, several central data centers 10 are distributed and located within the area covered by the communication system 1 (for example, within Japan).

[0013] For example, several tens of regional data centers 12 are distributed and placed within the area covered by the communication system 1. For example, if the area covered by the communication system 1 is the entire country of Japan, one or two regional data centers 12 may be placed in each prefecture.

[0014] For example, several thousand edge data centers 14 are distributed within the area covered by the communication system 1. Each edge data center 14 is capable of communicating with communication equipment 18 equipped with an antenna 16. As shown in FIG. 1 , one edge data center 14 may be capable of communicating with several pieces of communication equipment 18. The communication equipment 18 may include a computer such as a server computer. The communication equipment 18 according to this embodiment performs wireless communication with a UE (User Equipment) 20 via the antenna 16. The communication equipment 18 equipped with the antenna 16 is provided with, for example, a radio unit (RU) (described later).

[0015] In this embodiment, the central data center 10, the regional data center 12, and the edge data center 14 each have a plurality of servers arranged therein.

[0016] In this embodiment, for example, the central data center 10, the regional data centers 12, and the edge data centers 14 are capable of communicating with each other. Furthermore, the central data centers 10, the regional data centers 12, and the edge data centers 14 are also capable of communicating with each other.

[0017] 2, the communication system 1 according to this embodiment includes a platform system 30, multiple radio access networks (RANs) 32, multiple core network systems 34, purchaser terminals 36, and multiple UEs 20. The core network systems 34, the RANs 32, and the UEs 20 cooperate with each other to realize a mobile communication network.

[0018] The RAN 32 is a computer system equipped with an antenna 16, which corresponds to an eNodeB (eNB) in a fourth-generation mobile communication system (hereinafter referred to as 4G) or a gNB (NR base station) in a fifth-generation mobile communication system (hereinafter referred to as 5G). The RAN 32 according to this embodiment is mainly implemented by a group of servers and communication equipment 18 arranged in an edge data center 14. Note that part of the RAN 32 (for example, a distributed unit (DU), a central unit (CU), a virtual distributed unit (vDU), and a virtual central unit (vCU)) may be implemented in the central data center 10 or the regional data center 12, rather than in the edge data center 14.

[0019] The core network system 34 is a system equivalent to an EPC (Evolved Packet Core) in 4G or a 5G Core (5GC) in 5G. The core network system 34 according to this embodiment is implemented mainly by a group of servers arranged in the central data center 10 and the regional data centers 12.

[0020] The platform system 30 according to this embodiment is configured, for example, on a cloud platform and includes a processor 30a, a storage unit 30b, and a communication unit 30c, as shown in FIG. 2 . The processor 30a is a program-controlled device such as a microprocessor that operates according to a program installed in the platform system 30. The storage unit 30b is, for example, a storage element such as a ROM or RAM, a solid-state drive (SSD), or a hard disk drive (HDD). The storage unit 30b stores programs executed by the processor 30a. The communication unit 30c is, for example, a communication interface such as a network interface controller (NIC) or a wireless local area network (LAN) module. Note that software-defined networking (SDN) may be implemented in the communication unit 30c. The communication unit 30c exchanges data with the RAN 32 and the core network system 34.

[0021] In this embodiment, the platform system 30 is implemented by a group of servers located in the central data center 10. Note that the platform system 30 may also be implemented by a group of servers located in the regional data centers 12.

[0022] In this embodiment, for example, in response to a purchase request for a network service (NS) by a purchaser, the requested network service is constructed in the RAN 32 or the core network system 34. Then, the constructed network service is provided to the purchaser.

[0023] For example, a purchaser such as an MVNO (Mobile Virtual Network Operator) is provided with network services such as voice communication services and data communication services. The voice communication services and data communication services provided by this embodiment are ultimately provided to customers (end users) of the purchaser (MVNO in the above example) who use the UE 20 shown in Figures 1 and 2. The end users can perform voice communication and data communication with other users via the RAN 32 and the core network system 34. The UE 20 of the end user can also access a data network such as the Internet via the RAN 32 and the core network system 34.

[0024] In addition, in this embodiment, an IoT (Internet of Things) service may be provided to an end user who uses a robot arm, a connected car, etc. In this case, for example, the end user who uses the robot arm, the connected car, etc. may become a purchaser of the network service according to this embodiment.

[0025] The purchaser terminal 36 according to this embodiment is, for example, a general computer such as a smartphone, tablet, or personal computer used by the purchaser. The purchaser terminal 36 is used by, for example, a user such as an administrator of the network service purchased by the purchaser. The purchaser terminal 36 is capable of communicating with the platform system 30 via a computer network such as the Internet.

[0026] In addition, in this embodiment, for example, a purchaser terminal 36 used by a user such as an administrator of a network service purchased by a purchaser can access information about the network service purchased by that purchaser, but cannot access information about network services purchased by other purchasers.

[0027] In this embodiment, a container-type virtualized application execution environment such as Docker (registered trademark) is installed on servers located in the central data center 10, the regional data centers 12, and the edge data center 14, allowing containers to be deployed and run on these servers. A cluster consisting of one or more containers generated by such virtualization technology may be constructed on these servers. For example, a Kubernetes cluster managed by a container management tool such as Kubernetes (registered trademark) may be constructed. Then, a processor on the constructed cluster may execute a container-type application.

[0028] In this embodiment, the network service provided to the purchaser is composed of one or more functional units (for example, network functions (NFs)). In this embodiment, the functional units are implemented as NFs realized by virtualization technology. NFs realized by virtualization technology are called VNFs (Virtualized Network Functions). It does not matter what virtualization technology is used for virtualization. For example, in this description, a CNF (Containerized Network Function) realized by container-type virtualization technology is also included in the VNF. In this embodiment, the network service is described as being implemented by one or more CNFs. Furthermore, the functional units according to this embodiment may correspond to network nodes.

[0029] Fig. 3 is a diagram illustrating an example of a network service in operation. The network service illustrated in Fig. 3 includes, as software elements, NFs such as a plurality of RUs 40, a plurality of DUs 42, a plurality of CUs 44 (CU-CPs (Central Unit - Control Plane) 44a and CU-UPs (Central Unit - User Plane) 44b), a plurality of AMFs (Access and Mobility Management Functions) 46, a plurality of SMFs (Session Management Functions) 48, and a plurality of UPFs (User Plane Functions) 50.

[0030] In the example of Figure 3, RU 40, DU 42, CU-CP 44a, AMF 46, and SMF 48 correspond to elements of the control plane (C-Plane), and RU 40, DU 42, CU-UP 44b, and UPF 50 correspond to elements of the user plane (U-Plane).

[0031] The network service may include other types of NF as software elements. The network service is implemented on computer resources (hardware elements) such as multiple servers.

[0032] In this embodiment, for example, a communication service in a certain area is provided by the network service shown in FIG.

[0033] In this embodiment, it is assumed that the multiple RUs 40, multiple DUs 42, multiple CU-UPs 44b, and multiple UPFs 50 shown in Figure 3 belong to one end-to-end network slice.

[0034] 4 is a diagram schematically illustrating an example of associations between elements established in the communication system 1 in this embodiment. The symbols M and N shown in FIG. 4 represent any integers equal to or greater than 1, and indicate the relationship between the numbers of elements connected by a link. When both ends of a link are a combination of M and N, the elements connected by the link have a many-to-many relationship, and when both ends of a link are a combination of 1 and N or a combination of 1 and M, the elements connected by the link have a one-to-many relationship.

[0035] As shown in FIG. 4, the network service (NS), network function (NF), CNFC (Containerized Network Function Component), pod, and container have a hierarchical structure.

[0036] The NS corresponds to, for example, a network service configured from a plurality of NFs. Here, the NS may correspond to an element of granularity such as 5GC, EPC, 5G RAN (gNB), or 4G RAN (eNB). In addition, in this embodiment, the NS may have a nested structure.

[0037] In 5G, NFs correspond to elements with granularity such as RU, DU, CU-CP, CU-UP, AMF, SMF, and UPF. In 4G, NFs correspond to elements with granularity such as MME (Mobility Management Entity), HSS (Home Subscriber Server), S-GW (Serving Gateway), vDU, and vCU. In this embodiment, for example, one NS includes one or more NFs. In other words, one or more NFs are under the control of one NS.

[0038] CNFC corresponds to a granularity element such as DU mgmt or DU Processing, for example. CNFC may be a microservice deployed on a server as one or more containers. For example, a certain CNFC may be a microservice that provides some of the functions of DU, CU-CP, CU-UP, etc. Also, a certain CNFC may be a microservice that provides some of the functions of UPF, AMF, SMF, etc. In this embodiment, for example, one NF includes one or more CNFCs. In other words, one or more CNFCs are under the control of one NF.

[0039] A pod is the smallest unit for managing a Docker container in Kubernetes. In this embodiment, for example, one CNFC includes one or more pods. In other words, one CNFC has one or more pods under its control.

[0040] In this embodiment, for example, one pod includes one or more containers, that is, one or more containers are subordinate to one pod.

[0041] Also, as shown in Figure 4, network slices (NSIs) and network slice subnet instances (NSSIs) have a hierarchical structure.

[0042] The NSI can also be considered an end-to-end virtual circuit spanning multiple domains (e.g., from the RAN 32 to the core network system 34). The NSI may be a slice for high-speed, large-capacity communication (e.g., for enhanced Mobile Broadband (eMBB)), a slice for high-reliability and low-latency communication (e.g., for Ultra-Reliable and Low Latency Communications (URLLC)), or a slice for connecting a large number of terminals (e.g., for massive Machine Type Communication (mMTC)). The NSSI can also be considered a virtual circuit of a single domain obtained by dividing the NSI. The NSSI may be a slice of the RAN domain, a slice of a transport domain such as a Mobile Back Haul (MBH) domain, or a slice of the core network domain.

[0043] In this embodiment, for example, one NSI includes one or more NSSIs. That is, one or more NSSIs are subordinate to one NSI. Note that in this embodiment, multiple NSIs may share the same NSSI.

[0044] Furthermore, as shown in FIG. 4, NSSIs and NSs generally have a many-to-many relationship.

[0045] Furthermore, in this embodiment, for example, one NF can belong to one or more network slices. Specifically, for example, one NF can be configured with NSSAI (Network Slice Selection Assistance Information) including one or more S-NSSAI (Sub Network Slice Selection Assist Information). Here, S-NSSAI is information associated with a network slice. Note that an NF does not necessarily have to belong to a network slice.

[0046] Fig. 5 is a functional block diagram showing an example of functions implemented in the platform system 30 according to this embodiment. Note that the platform system 30 according to this embodiment does not need to implement all of the functions shown in Fig. 5, and functions other than the functions shown in Fig. 5 may also be implemented.

[0047] As shown in FIG. 5 , the platform system 30 according to this embodiment functionally includes, for example, an operations support system (OSS) unit 60, an orchestration (E2EO: End-to-End-Orchestration) unit 62, a service catalog storage unit 64, a big data platform unit 66, a data bus unit 68, an AI (Artificial Intelligence) unit 70, a monitoring function unit 72, an SDN controller 74, a configuration management unit 76, a container management unit 78, and a repository unit 80. The OSS unit 60 includes an inventory database 82, a management support unit 84, a fault management unit 86, and a performance management unit 88. The E2EO unit 62 includes a policy manager unit 90, a slice manager unit 92, and a lifecycle management unit 94. These elements are implemented primarily using a processor 30 a, a storage unit 30 b, and a communication unit 30 c.

[0048] The functions shown in Figure 5 may be implemented by installing a program including instructions corresponding to the functions in a platform system 30, which is one or more computers, on the platform system 30 and having the processor 30a execute the program. This program may be supplied to the platform system 30 via a computer-readable information storage medium, such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or via the Internet. The functions shown in Figure 5 may also be implemented using circuit blocks, memory, or other LSIs. Those skilled in the art will understand that the functions shown in Figure 5 can be realized in various forms, such as hardware alone, software alone, or a combination thereof.

[0049] The container management unit 78 manages the life cycle of a container, including processes related to the construction of the container, such as the deployment and configuration of the container.

[0050] Here, the platform system 30 according to the present embodiment may include a plurality of container management units 78. A container management tool such as Kubernetes and a package manager such as Helm may be installed in each of the plurality of container management units 78. Each of the plurality of container management units 78 may execute container construction, such as container deployment, for a server group (e.g., a Kubernetes cluster) associated with the corresponding container management unit 78.

[0051] The container management unit 78 does not need to be included in the platform system 30. The container management unit 78 may be provided, for example, in a server managed by the container management unit 78 (i.e., the RAN 32 or the core network system 34), or may be provided in another server that is annexed to the server managed by the container management unit 78.

[0052] In this embodiment, the repository unit 80 stores, for example, container images of containers included in a group of functional units (for example, a group of NFs) that realize a network service.

[0053] The inventory database 82 is a database that stores inventory information, which includes, for example, information about servers that are placed in the RAN 32 and the core network system 34 and that are managed by the platform system 30.

[0054] In this embodiment, inventory data is stored in the inventory database 82. The inventory data indicates the configuration of the elements included in the communication system 1 and the current status of the associations between the elements. The inventory data also indicates the status of resources managed by the platform system 30 (e.g., resource usage status). The inventory data may be physical inventory data or logical inventory data. Physical inventory data and logical inventory data will be described later.

[0055] Fig. 6 is a diagram showing an example of the data structure of physical inventory data. The physical inventory data shown in Fig. 6 is associated with one server. The physical inventory data shown in Fig. 6 includes, for example, a server ID, location data, building data, floor data, rack data, specification data, network data, an operating container ID list, a cluster ID, and the like.

[0056] The server ID included in the physical inventory data is, for example, an identifier of the server associated with the physical inventory data.

[0057] The location data included in the physical inventory data is, for example, data indicating the location (for example, the address of the location) of the server associated with the physical inventory data.

[0058] The building data included in the physical inventory data is, for example, data indicating the building (for example, the building name) in which the server associated with the physical inventory data is located.

[0059] The floor number data included in the physical inventory data is, for example, data indicating the floor number on which the server associated with the physical inventory data is located.

[0060] The rack data included in the physical inventory data is, for example, an identifier of the rack in which the server associated with the physical inventory data is located.

[0061] The specification data included in the physical inventory data is, for example, data indicating the specifications of the server associated with the physical inventory data, and the specification data indicates, for example, the number of cores, memory capacity, hard disk capacity, etc.

[0062] The network data included in the physical inventory data is, for example, data indicating information about the network of the server associated with the physical inventory data, and the network data indicates, for example, the NIC equipped in the server, the number of ports equipped in the NIC, the port ID of the port, etc.

[0063] The operating container ID list included in the physical inventory data is, for example, data that indicates information about one or more containers operating on a server associated with the physical inventory data, and the operating container ID list indicates, for example, a list of identifiers (container IDs) of instances of the containers.

[0064] The cluster ID included in the physical inventory data is, for example, an identifier of the cluster (for example, a Kubernetes cluster) to which the server associated with the physical inventory data belongs.

[0065] The logical inventory data includes topology data indicating the current state of associations between multiple elements included in the communication system 1, such as those shown in Figure 4. For example, the logical inventory data includes topology data including an identifier of a certain NS and identifiers of one or more NFs under the NS. Also, for example, the logical inventory data includes topology data including an identifier of a certain network slice and identifiers of one or more NFs belonging to the network slice.

[0066] The inventory data may also include data indicating the current status of geographical relationships and topological relationships between elements included in the communication system 1. As described above, the inventory data includes location data indicating the locations where the elements included in the communication system 1 are operating, i.e., the current locations of the elements included in the communication system 1. From this, it can be said that the inventory data indicates the current status of the geographical relationships between elements (e.g., the geographical proximity between elements).

[0067] The logical inventory data may also include NSI data indicating information about the network slice. The NSI data indicates attributes such as an identifier of an instance of the network slice and a type of the network slice. The logical inventory data may also include NSSI data indicating information about the network slice subnet. The NSSI data indicates attributes such as an identifier of an instance of the network slice subnet and a type of the network slice subnet.

[0068] The logical inventory data may also include NS data indicating information about NS. The NS data indicates attributes such as an NS instance identifier and an NS type, for example. The logical inventory data may also include NF data indicating information about NF. The NF data indicates attributes such as an NF instance identifier and an NF type, for example. The logical inventory data may also include CNFC data indicating information about CNFC. The CNFC data indicates attributes such as an instance identifier and a CNFC type, for example. The logical inventory data may also include pod data indicating information about pods included in the CNFC. The pod data indicates attributes such as a pod instance identifier and a pod type, for example. The logical inventory data may also include container data indicating information about containers included in the pod. The container data indicates attributes such as a container ID of a container instance and a container type, for example.

[0069] The container ID of the container data included in the logical inventory data and the container ID included in the operating container ID list included in the physical inventory data associate a container instance with the server on which the container instance is running.

[0070] Furthermore, the logical inventory data may include data indicating various attributes such as a host name and an IP address. For example, the container data may include data indicating an IP address of a container corresponding to the container data. For example, the NF data may include data indicating an IP address and a host name of the NF indicated by the NF data.

[0071] The logical inventory data may also include data indicating an NSSAI, including one or more S-NSSAIs, that is set in each NF.

[0072] The inventory database 82 is also able to grasp the resource status as needed in cooperation with the container management unit 78. The inventory database 82 then updates the inventory data stored therein as needed based on the latest resource status.

[0073] In addition, in response to actions being performed, such as constructing a new element included in the communication system 1, changing the configuration of an element included in the communication system 1, scaling an element included in the communication system 1, or replacing an element included in the communication system 1, the inventory database 82 updates the inventory data stored in the inventory database 82.

[0074] The service catalog storage unit 64 stores service catalog data. The service catalog data may include, for example, service template data indicating logic used by the life cycle management unit 94. This service template data includes information necessary for building a network service. For example, the service template data includes information defining NS, NF, and CNFC, and information indicating the correspondence between NS, NF, and CNFC. Furthermore, for example, the service template data includes a workflow script for building a network service.

[0075] An example of service template data is an NSD (NS Descriptor). The NSD is associated with a network service and indicates the types of multiple functional units (e.g., multiple CNFs) included in the network service. The NSD may also indicate the number of each type of functional unit, such as a CNF, included in the network service. The NSD may also indicate the file name of a CNFD (described later) related to the CNF included in the network service.

[0076] An example of service template data is a CNF Descriptor (CNFD). The CNFD may indicate computer resources (e.g., a CPU, memory, hard disk, etc.) required by the CNF. For example, the CNFD may indicate, for each of multiple containers included in the CNF, the computer resources (e.g., a CPU, memory, hard disk, etc.) required by the container.

[0077] The service catalog data may also include information about thresholds (for example, anomaly detection thresholds) that are used by the policy manager 90 to compare with the calculated performance index values. The performance index values ​​will be described later.

[0078] The service catalog data may also include, for example, slice template data, which includes information necessary to perform instantiation of a network slice, including, for example, logic utilized by the slice manager unit 92.

[0079] The slice template data includes information on the "Generic Network Slice Template" defined by the GSM Association (GSMA) ("GSM" is a registered trademark). Specifically, the slice template data includes network slice template data (NST), network slice subnet template data (NSST), and network service template data. The slice template data also includes information indicating the hierarchical structure of these elements, as shown in FIG. 4.

[0080] In this embodiment, for example, the life cycle management unit 94 constructs a new network service in response to a purchase request for an NS from a purchaser.

[0081] For example, in response to a purchase request, the lifecycle management unit 94 may execute a workflow script associated with the network service to be purchased. By executing this workflow script, the lifecycle management unit 94 may instruct the container management unit 78 to deploy a container included in the new network service to be purchased. The container management unit 78 may then obtain a container image of the container from the repository unit 80 and deploy the container corresponding to the container image to a server.

[0082] In addition, in this embodiment, the life cycle management unit 94 executes, for example, scaling and replacement of elements included in the communication system 1. Here, the life cycle management unit 94 may output a container deployment instruction or deletion instruction to the container management unit 78. Then, the container management unit 78 may execute processing such as container deployment or container deletion in accordance with the instruction. In this embodiment, the life cycle management unit 94 is capable of executing scaling and replacement that cannot be handled by a tool such as Kubernetes in the container management unit 78.

[0083] Furthermore, the life cycle management unit 94 may output an instruction to create a communication path to the SDN controller 74. For example, the life cycle management unit 94 presents two IP addresses at both ends of the communication path to be created to the SDN controller 74, and the SDN controller 74 creates a communication path connecting these two IP addresses. The created communication path may be managed in association with these two IP addresses.

[0084] Furthermore, the life cycle management unit 94 may output to the SDN controller 74 an instruction to create a communication path between the two IP addresses that is associated with the two IP addresses.

[0085] In this embodiment, for example, the slice manager unit 92 performs instantiation of a network slice. In this embodiment, for example, the slice manager unit 92 performs instantiation of a network slice by executing logic indicated by a slice template stored in the service catalog storage unit 64.

[0086] The slice manager unit 92 is configured to include the functions of the NSMF (Network Slice Management Function) and the NSSMF (Network Slice Sub-network Management Function), for example, as described in the specification "TS28 533" of the 3GPP (registered trademark) (Third Generation Partnership Project). The NSMF is a function that generates and manages network slices and provides NSI management services. The NSSMF is a function that generates and manages network slice subnets that constitute part of the network slice and provides NSSI management services.

[0087] Here, the slice manager unit 92 may output a configuration management instruction related to the instantiation of the network slice to the configuration management unit 76. Then, the configuration management unit 76 may perform configuration management such as setting in accordance with the configuration management instruction.

[0088] The slice manager unit 92 may also present two IP addresses to the SDN controller 74 and output an instruction to create a communication path between these two IP addresses.

[0089] In this embodiment, the configuration management unit 76 performs configuration management such as setting of element groups such as NFs in accordance with configuration management instructions received from the life cycle management unit 94 and the slice manager unit 92, for example.

[0090] In this embodiment, the SDN controller 74 creates a communication path between two IP addresses associated with a communication path creation instruction received from, for example, the life cycle management unit 94 or the slice manager unit 92. The SDN controller 74 may create a communication path between two IP addresses using a known path calculation method such as Flex Algo.

[0091] Here, for example, the SDN controller 74 may use a segment routing technology (for example, SRv6 (Segment Routing IPv6)) to construct NSIs and NSSIs for aggregation routers, servers, and the like present along the communication paths. Furthermore, the SDN controller 74 may generate NSIs and NSSIs across multiple NFs to be configured by issuing commands to configure a common VLAN (Virtual Local Area Network) for multiple NFs to be configured, and commands to assign the bandwidth and priority indicated in the configuration information to the VLAN.

[0092] In addition, the SDN controller 74 may perform operations such as changing the maximum bandwidth available for communication between two IP addresses without constructing a network slice.

[0093] The platform system 30 according to this embodiment may include multiple SDN controllers 74. Each of the multiple SDN controllers 74 may execute processing such as creating a communication path for a group of network devices such as an AG associated with the SDN controller 74.

[0094] In this embodiment, for example, the monitoring function unit 72 monitors the group of elements included in the communication system 1 in accordance with a given management policy. Here, the monitoring function unit 72 may monitor the group of elements in accordance with a monitoring policy specified by a purchaser when purchasing a network service, for example.

[0095] In this embodiment, the monitoring function unit 72 performs monitoring at various levels, such as the slice level, the NS level, the NF level, the CNFC level, and the hardware level of a server or the like.

[0096] For example, in order to perform monitoring at the various levels described above, the monitoring function unit 72 may set a module that outputs metric data in hardware such as a server or in a software element included in the communication system 1. Here, for example, an NF may output metric data indicating metrics that are measurable (identifiable) in the NF to the monitoring function unit 72. Also, a server may output metric data indicating metrics related to hardware that is measurable (identifiable) in the server to the monitoring function unit 72.

[0097] Furthermore, for example, the monitoring function unit 72 may deploy a sidecar container on the server that aggregates metric data indicating metrics output from multiple containers on a CNFC (microservice) basis. This sidecar container may include an agent called an exporter. The monitoring function unit 72 may repeatedly execute, at a given monitoring interval, a process of acquiring metric data aggregated on a microservice basis from the sidecar container using a mechanism of a monitoring tool such as Prometheus, which can monitor container management tools such as Kubernetes.

[0098] The monitoring function unit 72 may monitor performance indicator values ​​for performance indicators described in, for example, “TS 28.552, Management and orchestration; 5G performance measurements” or “TS 28.554, Management and orchestration; 5G end to end Key Performance Indicators (KPI).” Then, the monitoring function unit 72 may acquire metric data indicating the monitored performance indicator values.

[0099] In this embodiment, the monitoring function unit 72 performs a process (enrichment) of aggregating metric data, for example, in a predetermined aggregation unit, thereby generating performance index value data indicating the performance index values ​​of the elements included in the communication system 1 in that aggregation unit.

[0100] For example, for one gNB, performance index value data for the gNB is generated by aggregating metric data indicating the metrics of elements (e.g., network nodes such as DU42 and CU44) under the control of the gNB. In this way, performance index value data indicating communication performance in the area covered by the gNB is generated. Here, for example, performance index value data indicating multiple types of communication performance such as throughput and latency may be generated for each gNB. Note that the communication performance indicated by the performance index value data is not limited to throughput and latency.

[0101] Then, the monitoring function unit 72 outputs the performance index value data generated by the above-mentioned enrichment to the data bus unit 68.

[0102] In this embodiment, for example, the data bus unit 68 receives performance index value data output from the monitoring function unit 72. Then, based on the received one or more pieces of performance index value data, the data bus unit 68 generates a performance index value file including the one or more pieces of performance index value data. Then, the data bus unit 68 outputs the generated performance index value file to the big data platform unit 66.

[0103] In addition, elements such as network slices, NS, NF, CNFC, etc. included in the communication system 1, and hardware such as servers, notify the monitoring function unit 72 of various alerts (for example, notification of an alert triggered by the occurrence of a failure).

[0104] Then, for example, when the monitoring function unit 72 receives the above-mentioned alert notification, it outputs alert message data indicating the notification to the data bus unit 68. Then, the data bus unit 68 generates an alert file in which alert message data indicating one or more notifications are compiled into a single file, and outputs the alert file to the big data platform unit 66.

[0105] In this embodiment, the big data platform unit 66 accumulates, for example, performance index value files and alert files output from the data bus unit 68 .

[0106] In this embodiment, for example, a plurality of trained machine learning models are stored in advance in the AI ​​unit 70. The AI ​​unit 70 uses the various machine learning models stored in the AI ​​unit 70 to perform estimation processing such as future prediction processing of the usage status and service quality of the communication system 1. The AI ​​unit 70 may generate estimation result data indicating the results of the estimation processing.

[0107] The AI ​​unit 70 may perform estimation processing based on the files stored in the big data platform unit 66 and the above-mentioned machine learning model. This estimation processing is suitable for low-frequency prediction of long-term trends.

[0108] The AI ​​unit 70 is also capable of acquiring performance index value data stored in the data bus unit 68. The AI ​​unit 70 may perform estimation processing based on the performance index value data stored in the data bus unit 68 and the above-described machine learning model. This estimation processing is suitable for performing short-term predictions frequently.

[0109] In this embodiment, for example, the performance management unit 88 calculates a performance index value (e.g., KPI) based on metrics indicated by multiple metric data. The performance management unit 88 may calculate a performance index value that is an overall evaluation of multiple types of metrics (e.g., a performance index value related to an end-to-end network slice) that cannot be calculated from a single metric data. The performance management unit 88 may generate overall performance index value data that indicates the performance index value that is the overall evaluation.

[0110] The performance management unit 88 may acquire the above-mentioned performance index value file from the big data platform unit 66. The performance management unit 88 may also acquire estimation result data from the AI ​​unit 70. Then, performance index values ​​such as KPIs may be calculated based on at least one of the performance index value file and the estimation result data. The performance management unit 88 may also directly acquire metric data from the monitoring function unit 72. Then, performance index values ​​such as KPIs may be calculated based on the metric data.

[0111] In this embodiment, the fault management unit 86 detects the occurrence of a fault in the communication system 1 based on, for example, at least one of the above-mentioned metric data, the above-mentioned alert notification, the above-mentioned estimation result data, and the above-mentioned overall performance index value data. The fault management unit 86 may detect the occurrence of a fault that cannot be detected from a single piece of metric data or a single alert notification, for example, based on a predetermined logic. The fault management unit 86 may generate detected fault data that indicates the detected fault.

[0112] The fault management unit 86 may obtain metric data and alert notifications directly from the monitoring function unit 72. The fault management unit 86 may also obtain performance index value files and alert files from the big data platform unit 66. The fault management unit 86 may also obtain alert message data from the data bus unit 68.

[0113] In this embodiment, the policy manager unit 90 executes a predetermined judgment process based on, for example, at least one of the above-mentioned metric data, the above-mentioned performance index value data, the above-mentioned alert message data, the above-mentioned performance index value file, the above-mentioned alert file, the above-mentioned estimation result data, the above-mentioned overall performance index value data, and the above-mentioned detected fault data.

[0114] The policy manager unit 90 may then execute an action according to the result of the determination process. For example, the policy manager unit 90 may output an instruction to construct a network slice to the slice manager unit 92. The policy manager unit 90 may also output an instruction to scale or replace an element to the life cycle management unit 94 according to the result of the determination process.

[0115] The policy manager unit 90 according to this embodiment is capable of acquiring performance index value data stored in the data bus unit 68. The policy manager unit 90 may then execute a predetermined determination process based on the performance index value data acquired from the data bus unit 68. The policy manager unit 90 may also execute a predetermined determination process based on alert message data stored in the data bus unit 68.

[0116] In this embodiment, the management support unit 84 provides functions to support users such as administrators of the communication system 1, such as the administrator of the entire communication system 1 or the administrator of a network service purchased by a purchaser that is part of the communication system 1.

[0117] Here, the management support unit 84 may generate a ticket indicating the content to be notified to the administrator of the communication system 1. The management support unit 84 may generate a ticket indicating the content of the occurred fault data. The management support unit 84 may also generate a ticket indicating the values ​​of performance index value data and metric data. The management support unit 84 may also generate a ticket indicating the determination result by the policy manager unit 90.

[0118] Then, the management support unit 84 notifies the administrator of the communication system 1 of the generated ticket. For example, the management support unit 84 may send an email with the generated ticket attached to the email address of the administrator of the communication system 1.

[0119] The management support unit 84 may also generate a monitoring screen (dashboard screen) described below. The management support unit 84 may then transmit the generated monitoring screen to the purchaser terminal 36. The purchaser terminal 36 may then display the transmitted monitoring screen on a display or the like.

[0120] Furthermore, the management support unit 84 accepts operations on the purchaser terminal 36 by users such as administrators of network services. For example, in response to a user's operation on the purchaser terminal 36, the purchaser terminal 36 may transmit an operation signal representing the operation to the platform system 30. Then, the management support unit 84 may accept the operation signal.

[0121] [Executing processing based on performance index value data or performance index value file] Below, we will further explain the generation of a performance index value file, the judgment processing based on the performance index value data stored in the data bus unit 68, and the estimation processing based on the performance index value data stored in the data bus unit 68.

[0122] 7 is a diagram schematically illustrating an example of the data bus unit 68 according to this embodiment. As shown in Fig. 7, the data bus unit 68 according to this embodiment includes, for example, a plurality of queues 100 that hold performance index value data in a first-in, first-out list structure.

[0123] Each queue 100 belongs to either the first queue group 102a or the second queue group 102b.

[0124] In this embodiment, for example, a plurality of counting processes 104 are running in the monitoring function unit 72. Each counting process 104 is set in advance with respect to elements to be counted by that counting process 104. The elements may be elements included in the RAN 32 or elements included in the core network system 34.

[0125] For example, each aggregation process 104 may be preset with a gNB that is the target of aggregation in that aggregation process 104. Then, each aggregation process 104 may acquire metric data from NFs (e.g., RU 40, DU 42, and CU-UP 44b) under the gNB that is the target of aggregation in that aggregation process 104. Then, the aggregation process 104 may perform an enrichment process that generates performance index value data indicating the communication performance of the gNB based on the acquired metric data.

[0126] Furthermore, for example, each aggregation process 104 may be set in advance to an NS that is the aggregation target of the aggregation process 104. For example, each aggregation process 104 may be set in advance to a UPF service including one or more UPFs 50 or an AMF service including one or more AMFs 46 that is the aggregation target of the aggregation process 104. Each aggregation process 104 may acquire metric data from an NF under the NS that is the aggregation target of the aggregation process 104. Then, the aggregation process 104 may execute an enrichment process that generates performance index value data indicating the communication performance of the NS based on the acquired metric data.

[0127] In this embodiment, for example, the counting process 104 is associated in advance with the queue 100. For convenience, Fig. 7 shows that the counting process 104 and the queue 100 are associated in a one-to-one relationship, but the counting process 104 and the queue 100 may be associated in a many-to-many relationship.

[0128] Hereinafter, the counting process 104 associated with the queue 100 included in the first queue group 102a will be referred to as the first group counting process 104a, and the counting process 104 associated with the queue 100 included in the second queue group 102b will be referred to as the second group counting process 104b.

[0129] Then, each first group aggregation process 104a generates performance index value data by aggregating the metric data associated with that first group aggregation process 104a from the previous aggregation to the present time at a predetermined time interval (for example, every minute).

[0130] The first group counting process 104a acquires metric data from one or more NFs associated with the first group counting process 104a, for example, at one-minute intervals. The first group counting process 104a then counts the metric data for the same counting period to generate performance index value data for the counting period.

[0131] Then, every time the first group counting process 104a generates performance index value data, it enqueues the performance index value data in one or more queues 100 associated with the first group counting process 104a.

[0132] Then, each second group aggregation process 104b generates performance index value data by aggregating the metric data associated with that second group aggregation process 104b from the previous aggregation to the present time at a predetermined time interval (for example, every 15 minutes).

[0133] The second group counting process 104b acquires metric data from one or more NFs associated with the second group counting process 104b, for example, at 15-minute intervals. The second group counting process 104b then counts the metric data for the same counting period to generate performance index value data for the counting period.

[0134] Then, every time the second group counting process 104b generates performance index value data, it enqueues the performance index value data in one or more queues 100 associated with the second group counting process 104b.

[0135] In this embodiment, the maximum number of performance index value data that can be stored in the queues 100 included in the first queue group 102a is predetermined. Here, for example, it is assumed that a maximum of 60 performance index value data can be stored in the queues 100. In other words, the maximum number is set to "60."

[0136] In this embodiment, the maximum number of performance index value data that can be stored in the queues 100 included in the second queue group 102b is predetermined. Here, for example, it is assumed that a maximum of four performance index value data can be stored in the queues 100. In other words, the maximum number is set to "4."

[0137] In this embodiment, for example, a plurality of determination processes 106 (see FIGS. 8, 9, and 10) are running in the policy manager unit 90. Some of these determination processes 106 execute determination processing based on the performance index value data stored in the data bus unit 68, and the rest execute determination processing based on the files stored in the big data platform unit 66.

[0138] Among the determination processes 106 according to this embodiment, there is one that acquires a performance index file containing one or more pieces of performance index data. The determination process 106 then determines the state of the communication system 1 based on the acquired performance index file. Here, for example, the state of an element included in the communication system 1 and associated with the determination process 106 may be determined. For example, the state of an element that is the target of aggregation in the first group aggregation process 104a that generated the performance index data included in the performance index file acquired by the determination process 106 may be determined. Hereinafter, such a determination process 106 will be referred to as a file determination process 106a.

[0139] In this embodiment, for example, the file determination process 106a is associated in advance with the queue 100. For convenience, although the file determination process 106a and the queue 100 are shown as being associated in a one-to-one relationship in Figures 8, 9, and 10, the file determination process 106a and the queue 100 may be associated in a many-to-many relationship.

[0140] Here, for example, in response to a performance index value file generated based on performance index value data contained in a queue 100 included in the first queue group 102a being output to the big data platform unit 66, the data bus unit 68 may output a notification indicating that the performance index value file has been output to one or more file determination processes 106a associated with the queue 100.

[0141] Then, the file determination process 106a that has received the notification may acquire the performance index value file that has been output to the big data platform unit 66 in response to the reception of the notification.

[0142] Furthermore, some of the determination processes 106 according to this embodiment acquire performance index value data indicating the actual values ​​of the performance index values ​​related to the communication system 1. For example, there is a determination process 106 that acquires performance index value data in response to performance index value data being enqueued in a queue 100 included in the first queue group 102a.

[0143] In this embodiment, for the queues 100 included in the first queue group 102a, any performance index value data included in the queues 100 can be accessed (obtained) without dequeuing.

[0144] The determination process 106 then determines the state of the communication system 1 based on the acquired performance index value data. Here, for example, the state of an element included in the communication system 1 and associated with the determination process 106 may be determined. For example, the state of an element that is the target of aggregation in the first group aggregation process 104a that generated the performance index value data acquired by the determination process 106 may be determined. Hereinafter, such a determination process 106 will be referred to as a current status determination process 106b.

[0145] In this embodiment, for example, the current status determination process 106b is associated in advance with the queue 100. For convenience, although Figures 9 and 10 show that the current status determination process 106b and the queue 100 are associated in a one-to-one relationship, the current status determination process 106b and the queue 100 may be associated in a many-to-many relationship.

[0146] Here, for example, in response to performance index value data being enqueued into a queue 100 included in the first queue group 102a, the data bus unit 68 may output a notification indicating that the performance index value data has been enqueued to one or more current status determination processes 106b associated with the queue 100.

[0147] Then, the current status determination process 106b that has received the notification may acquire the latest performance index value data stored in the queue 100 in response to the reception of the notification.

[0148] Furthermore, some of the determination processes 106 according to this embodiment acquire estimation result data indicating the estimation results obtained by an estimation process 108 (see FIG. 10 ) associated with the determination process 106. The determination process 106 then determines the state of the communication system 1 based on the acquired estimation result data. Here, for example, the state of an element included in the communication system 1 and associated with the determination process 106 may be determined. For example, the state of an element that is the target of aggregation in the first group aggregation process 104a that generated the performance index value data acquired by the estimation process 108 may be determined. Hereinafter, such a determination process 106 will be referred to as a prediction determination process 106c.

[0149] In this embodiment, for example, a plurality of estimation processes 108 (see FIG. 10 ) are running in the AI ​​unit 70. Some of these estimation processes 108 execute estimation processing based on performance index value data stored in the data bus unit 68, and the remaining execute estimation processing based on files stored in the big data platform unit 66.

[0150] In the present embodiment, for example, the estimation process 108 is associated in advance with the queue 100. For convenience, Fig. 10 shows that the estimation process 108 and the queue 100 are associated in a one-to-one relationship, but the estimation process 108 and the queue 100 may be associated in a many-to-many relationship.

[0151] In this embodiment, for example, each estimation process 108 acquires performance index value data stored in the queue 100 included in the first queue group 102a corresponding to that estimation process 108. Then, that estimation process executes an estimation process that is predetermined for that estimation process 108 based on the performance index value data.

[0152] Here, the estimation process 108, for example, in response to performance index value data being enqueued in a queue 100 included in the first queue group 102a, acquires performance index value data for the nearest constant number or nearest period that includes at least the most recent performance index value data among the performance index value data stored in the queue 100.

[0153] Here, for example, in response to performance index value data being enqueued in a queue 100 included in the first queue group 102a, the data bus unit 68 may output a notification indicating that the performance index value data has been enqueued to one or more estimation processes 108 associated with the queue 100.

[0154] Then, the estimation process 108 that receives the notification may, in response to receiving the notification, acquire performance index value data for the nearest constant number or nearest period that includes at least the latest performance index value data from the performance index value data stored in the queue 100.

[0155] For example, the estimation process 108 shown in Fig. 10 acquires 60 pieces of performance index value data, including the most recent performance index value data. These pieces of performance index value data correspond to the most recent 60 minutes of performance index value data, including the most recent performance index value data. The estimation process 108 then executes estimation processing based on the performance index value data.

[0156] For example, suppose that a first group counting process 104a associated with a specific gNB generates performance index value data for the gNB by counting metric data related to elements included in the gNB (e.g., elements subordinate to the gNB). Then, suppose that an estimation process 108 that acquires the performance index value data generated by the first group counting process 104a acquires 60 pieces of performance index value data including the latest performance index value data stored in the queue 100 in response to the performance index value data being enqueued in the queue 100.

[0157] In this case, the estimation process 108 predicts communication performance such as the level of network load of the gNB from the present time to 20 minutes ahead based on these 60 performance index value data using a trained machine learning model pre-stored in the AI ​​unit 70. Here, for example, the level of network load of the gNB may be predicted as throughput, latency, or the like.

[0158] The machine learning model may be, for example, an existing predictive model. Alternatively, the machine learning model may be, for example, a trained machine learning model that has been previously subjected to supervised learning using multiple training data elements. Each of the multiple training data elements may include, for example, training input data indicating the throughput of the gNB for 60 minutes up to a given time point, and supervised data indicating the level of network load (e.g., throughput or latency) at the gNB from that time point to 20 minutes ahead.

[0159] Alternatively, for example, a first group counting process 104a associated with a specific NS (e.g., a UPF service or an AMF service) may generate performance index value data related to the element. Then, the estimation process 108, which acquires the performance index value data generated by the first group counting process 104a, acquires 60 pieces of performance index value data including the latest performance index value data stored in the queue 100 in response to the performance index value data being enqueued in the queue 100.

[0160] In this case, the estimation process 108 predicts communication performance, such as the level of network load of the element from the present time to 20 minutes ahead, based on these 60 performance index value data using a trained machine learning model pre-stored in the AI ​​unit 70. Here, for example, the level of network load of the element may be predicted as throughput, latency, or the like.

[0161] The machine learning model may be, for example, an existing predictive model. Alternatively, the machine learning model may be, for example, a trained machine learning model that has been previously subjected to supervised learning using multiple training data elements. Each of the multiple training data elements may include, for example, training input data indicating the throughput of the NS for the 60 minutes up to each given time point, and supervised data indicating the level of network load (e.g., throughput or latency) from each given time point to the next 20 minutes.

[0162] It should be noted that the estimation process 108 does not need to acquire all of the performance index value data stored in the queue 100 as described above, and may acquire only a portion of the performance index value data stored in the queue 100 .

[0163] The estimation process 108 then outputs estimation result data indicating the execution result of the estimation process (estimation result) to the prediction determination process 106c associated with the estimation process 108. The prediction determination process 106c then acquires the estimation result data. The prediction determination process 106c then judges the state of the communication system 1 based on the acquired estimation result data.

[0164] As described above, the queue 100 according to this embodiment is associated with the aggregation process 104, the file determination process 106a, the current status determination process 106b, the prediction determination process 106c, and the estimation process 108.

[0165] In addition, in this embodiment, the data bus unit 68 generates a performance index value file containing at least a portion of the performance index value data stored in the queue 100, for example, less frequently than the frequency at which the AI ​​unit 70 acquires the performance index value data.

[0166] For example, the data bus unit 68 may generate, at predetermined time intervals, a performance index file containing performance index data stored in the queue 100 after the previous performance index file was generated.

[0167] Here, the time interval may or may not match the time corresponding to the maximum number of performance index value data that can be stored in the queues 100 included in the first queue group 102a.

[0168] Furthermore, for example, when all of the performance index value data included in the generated performance index value file has been dequeued, the data bus unit 68 may generate a file containing all of the performance index value data stored in the queue 100. In other words, when all of the performance index value data stored in the queue 100 has been replaced, a file containing all of the performance index value data stored in the queue 100 may be generated.

[0169] Furthermore, in this embodiment, when 60 pieces of performance index value data are stored in a queue 100 included in the first queue group 102a, if new performance index value data is enqueued, the oldest performance index value data stored in that queue 100 is dequeued. In other words, the oldest performance index value data stored in that queue 100 is deleted from that queue 100.

[0170] In this embodiment, when four pieces of performance index value data are stored in the queues 100 included in the second queue group 102b, the data bus unit 68 generates a performance index value file that combines these four pieces of performance index value data into a single file.The data bus unit 68 then outputs the generated performance index value file to the big data platform unit 66.

[0171] Then, the data bus unit 68 dequeues all of the performance index value data stored in the queue 100. In other words, all of the performance index value data stored in the queue 100 is deleted from the queue 100.

[0172] In this way, the queues 100 included in the first queue group 102a and the queues 100 included in the second queue group 102b perform different processes in response to the generation of a performance index value file. In the queues 100 included in the second queue group 102b, all performance index value data stored in the queues 100 are deleted from the queues 100 in response to the generation of a performance index value file. On the other hand, in the queues 100 included in the first queue group 102a, dequeuing is not performed in response to the generation of a performance index value file.

[0173] In the following description, it is assumed that performance index files are generated at 60-minute intervals, i.e., one performance index file contains performance index data for the most recent 60 minutes.

[0174] [Selection of an Option Related to Monitoring Settings] In the present embodiment, for example, a purchaser of a network service can select an option related to monitoring settings when purchasing the network service. In the following description, it is assumed that the purchaser of the network service can select one of a low-level option, a medium-level option, or a high-level option.

[0175] For example, if the low-level option is selected, when a network service is constructed, not only the elements included in the network service but also the queue 100 associated with the elements and the aggregation process 104 associated with the elements are generated, as shown in Fig. 8. In this case, performance index value files related to the elements included in the network service are accumulated in the big data platform unit 66.

[0176] A file determination process 106a associated with the queue 100 is also generated. At this time, the policy manager unit 90 may refer to inventory data to confirm the attributes of the element associated with the generated file determination process 106a. The policy manager unit 90 may then generate the file determination process 106a in which a workflow corresponding to the confirmed attributes is set. The file determination process 106a may then execute the workflow set in the file determination process 106a to perform the determination process.

[0177] For example, the file determination process 106a may determine whether or not an action needs to be executed (for example, whether or not a scale-out needs to be executed) based on the acquired performance index value file.

[0178] In this embodiment, for example, as described above, when it is determined that scale-out is necessary, the platform system 30 may execute scale-out of the elements determined based on the performance index value file. For example, the policy manager unit 90, the lifecycle management unit 94, the container management unit 78, and the configuration management unit 76 may execute scale-out in cooperation with each other.

[0179] Furthermore, for example, when the medium-level option is selected, when a network service is constructed, not only the elements included in the network service are generated, but also, as with the low-level option, the queue 100 associated with the elements, the aggregation process 104 associated with the elements, and the file determination process 106a associated with the queue 100. Furthermore, as shown in Fig. 9, a current status determination process 106b associated with the queue 100 is also generated.

[0180] At this time, the policy manager unit 90 may refer to the inventory data and confirm the attributes of the elements associated with the current status determination process 106b to be generated.The policy manager unit 90 may then generate the current status determination process 106b in which a workflow corresponding to the confirmed attributes is set.The current status determination process 106b may then perform the determination processing by executing the workflow set in the current status determination process 106b.

[0181] For example, the current status determination process 106b may determine whether or not an action needs to be executed (for example, whether or not a scale-out needs to be executed) based on the acquired performance index value data.

[0182] In this embodiment, for example, as described above, the platform system 30 may perform scale-out of elements determined based on the performance index value data in response to determining that scale-out is necessary.

[0183] Also, for example, if a high-level option is selected, when a network service is constructed, not only the elements included in the network service are generated, but also the queue 100 associated with the element, the aggregation process 104 associated with the element, the file determination process 106a associated with the queue 100, and the current status determination process 106b associated with the queue 100, as with the low-level and medium-level options.

[0184] 10 , the AI ​​unit 70 generates an estimation process 108 associated with the current status determination process 106b, and the policy manager unit 90 generates a prediction determination process 106c associated with the current status determination process 106b. For example, the estimation process 108 and the prediction determination process 106c may be activated. At this time, instantiation of a trained machine learning model may also be performed. The estimation process 108 may then perform estimation using the machine learning model instantiated in this manner.

[0185] The prediction determination process 106c may then execute a predetermined determination process based on the estimation result data output by the estimation process 108 associated with the prediction determination process 106c. For example, the prediction determination process 106c may determine whether or not to execute an action (e.g., whether or not to execute a scale-out) based on the network load prediction result.

[0186] 9 , in response to performance index value data being enqueued in a queue 100 included in the first queue group 102a, the current status determination process 106b may acquire the enqueued performance index value data, and the estimation process 108 may acquire performance index value data for an immediately neighboring constant number of periods or an immediately neighboring period that includes at least the enqueued performance index value data, from the performance index value data stored in the queue 100. In this way, in response to performance index value data being enqueued in the queue 100, the enqueued performance index value data may be acquired by both the current status determination process 106b and the estimation process 108.

[0187] The current status determination process 106b may then determine whether or not an action needs to be executed (for example, whether or not a scale-out needs to be executed) based on the acquired performance index value data.

[0188] Furthermore, the estimation process 108 may generate estimation result data indicating a prediction result of communication performance such as a network load based on the acquired performance index value data. Then, the estimation process 108 may output the generated estimation result data to the prediction determination process 106c. Then, the prediction determination process 106c may acquire the estimation result data.

[0189] Then, the prediction determination process 106c may determine whether or not an action needs to be executed (for example, whether or not a scale-out needs to be executed) based on the acquired estimation result data.

[0190] It is not necessary for the AI ​​unit 70 to generate the estimation process 108 and for the policy manager unit 90 to generate the prediction determination process 106c. For example, the current situation determination process 106b may generate the estimation process 108 and the policy manager unit 90.

[0191] Then, in this embodiment, for example, as described above, the platform system 30 may perform scale-out of elements determined based on the performance index value data or the estimation result data in response to determining that scale-out is necessary.

[0192] Hereinafter, the operation of the network service described above that is performed when the low-level option is selected will be referred to as operation with the low-level option. Also, the operation of the network service described above that is performed when the medium-level option is selected will be referred to as operation with the medium-level option. Also, the operation of the network service described above that is performed when the high-level option is selected will be referred to as operation with the high-level option.

[0193] In this embodiment, the purchaser of the network service may be able to select an option related to monitoring settings for each of a plurality of elements included in the network service.

[0194] For example, in this embodiment, when a purchaser of a network service selects a medium-level option, the purchaser may be able to specify the elements included in the communication system 1. Then, a file determination process 106a and a current status determination process 106b may be generated for the specified elements. In this way, only some of the elements included in the network service may be operated with the medium-level option. Then, the remaining elements may be operated with the low-level option.

[0195] Furthermore, in this embodiment, when a purchaser of a network service selects a high-level option, the purchaser may be able to specify the elements to be included in the communication system 1. Then, a file determination process 106a, a current status determination process 106b, an estimation process 108, and a prediction determination process 106c may be generated for the specified elements. In this way, only some of the elements included in the network service may be operated using the high-level option. Then, the remaining elements may be operated using the low-level option or the medium-level option.

[0196] Furthermore, when a medium-level option is selected, the file determination process 106a may not be generated. Furthermore, when a high-level option is selected, the file determination process 106a and the current status determination process 106b may not be generated. Furthermore, options related to monitoring settings may be changed according to the purchaser's request.

[0197] [Monitoring Screen Display Control] In the present embodiment, the management support unit 84 may provide the user with performance information indicating the performance index values ​​of the elements included in the communication system 1. Here, the management support unit 84 may provide the user with performance information of multiple elements included in the communication system 1.

[0198] For example, as described above, the management support unit 84 may generate a monitoring screen (dashboard screen) as shown in Fig. 11 , which is displayed on the display of the purchaser terminal 36 or the like. The monitoring screen shown in Fig. 11 has a plurality of performance information images 110 (110a, 110b, 110c, and 110d) arranged thereon. Here, the performance information images 110 are associated with combinations of elements included in the communication system 1 and types of performance information for the elements.

[0199] For example, performance information image 110a shows performance information indicating a transition in a performance index value of type a1 for a UPF service with an identifier of 31. Furthermore, performance information image 110b shows performance information indicating a transition in a performance index value of type a2 for an AMF service with an identifier of #32. Furthermore, performance information image 110c shows performance information indicating a transition in a performance index value of type a1 for a UPF service with an identifier of 51. Furthermore, performance information image 110d shows performance information indicating a transition in a performance index value of type a2 for an AMF service with an identifier of 52. In this way, performance information image 110 shows performance index values ​​of at least one point in time for elements included in communication system 1.

[0200] 11 , the monitoring screen may display performance information indicating the same type of performance index value for a plurality of different elements. Furthermore, the monitoring screen may display multiple types of performance information for a single element. That is, multiple performance information images 110 indicating different types of performance information for a single element may be arranged on the monitoring screen.

[0201] In this embodiment, for example, a user such as an administrator of a network service purchased by a purchaser can perform an operation to specify, from among a plurality of elements included in the communication system 1, an element whose performance information is to be displayed on the monitoring screen and the type of performance index value indicated by the performance information. Here, for example, the user can perform an operation to specify one or more elements from among a plurality of elements included in the network service purchased by the purchaser.

[0202] Then, the management support unit 84 receives, from the purchaser terminal 36, a designation of an element from among the multiple elements included in the communication system 1. For example, the management support unit 84 receives an operation signal indicating the designated element and the type of performance index value, which is transmitted from the purchaser terminal 36 in response to the above-mentioned designation operation.

[0203] Then, the management support unit 84 displays a monitoring screen showing the performance information of the specified element on a display or the like of the purchaser terminal 36. For example, the management support unit 84 generates a monitoring screen showing the performance information of the specified element and transmits the monitoring screen to the purchaser terminal 36. Then, the purchaser terminal 36 displays the monitoring screen on a display or the like.

[0204] Furthermore, in this embodiment, for example, the management support unit 84 repeatedly updates the displayed monitoring screen. For example, the management support unit 84 may update the displayed monitoring screen to one that shows the latest performance information at a predetermined time interval. Furthermore, for example, the displayed monitoring screen may be updated to one that shows the latest performance information every time a new performance index value file corresponding to the performance information shown on the monitoring screen is output to the big data platform unit 66.

[0205] In this embodiment, for example, a plurality of action execution buttons are arranged on the monitoring screen as shown in Fig. 11. The action execution buttons are associated with performance information images 110.

[0206] The action execution buttons are buttons for executing a given action on an element associated with the action execution button. In Fig. 11, as an example of the action execution buttons, scale-out buttons 112 (112, 112b, 112c, and 112d) for executing a scale-out are arranged.

[0207] 11 , a scale-out button 112 associated with a performance information image 110 is arranged below the performance information image 110. For example, a scale-out button 112a, a scale-out button 112b, a scale-out button 112c, and a scale-out button 112d are arranged below the performance information image 110a, the performance information image 110b, the performance information image 110c, and the performance information image 110d, respectively.

[0208] In this embodiment, for example, the management support unit 84 receives an instruction to execute a given action on an element whose performance information is displayed on the monitoring screen. Then, in response to the reception of the instruction to execute the given action on the element, the platform system 30 executes the action on the element.

[0209] For example, in response to a user such as an administrator performing a predetermined operation (e.g., a click operation) on the scale-out button 112, the purchaser terminal 36 transmits to the platform system 30 a scale-out instruction associated with the identifier of the NS associated with the clicked scale-out button 112. The management support unit 84 then accepts the scale-out instruction. The platform system 30 then executes the scale-out of the NFs included in the NS.

[0210] In this embodiment, for example, when the reception status of an instruction to execute a given action on an element satisfies given conditions, the management support unit 84 performs at least one of starting to display the predicted value of the performance index value of the element or shortening the update interval of the monitoring screen.

[0211] For example, suppose the given condition is "receiving an instruction to execute the action a predetermined number of times (e.g., three times) within a recent predetermined period (e.g., three days)." In this case, when an instruction to execute the action for a certain element is received a predetermined number of times within the recent predetermined time period, at least one of starting to display a predicted value of the performance index value of the element or shortening the update interval of the monitoring screen may be performed.

[0212] For example, suppose that while the UPF service is being operated with a low-level option, the execution status of a predetermined operation on the scale-out button 112a satisfies a given condition. In this case, the operation of the UPF service may be changed to operation with a medium-level option. Here, the policy manager unit 90 may generate a current status determination process 106b associated with the UPF service. Then, the current status determination process 106b may start a process of determining whether or not the UPF service needs to be scaled out.

[0213] Then, the current status determination process 106b may determine whether or not it is necessary to scale out the UPF 50 included in the UPF service based on the performance index value data in response to the performance index value data being enqueued in the queue 100 associated with the UPF service. Then, in response to the determination that scale-out is necessary, the scale-out of the UPF 50 included in the UPF service may be executed.

[0214] In this case, the management support unit 84 may shorten the update interval of the monitoring screen (for example, shorten the update interval of the performance information image 110a). For example, the update interval of the monitoring screen may be shortened from 60 minutes to 1 minute. Alternatively, the update interval of the performance information image 110a arranged on the monitoring screen may be shortened from 60 minutes to 1 minute. Note that when the update interval of the monitoring screen is shortened from 60 minutes to 1 minute, the performance information image 110a may be updated every time the monitoring screen is updated, and the remaining performance information images 110 may be updated every 60 times the monitoring screen is updated.

[0215] Furthermore, for example, when the UPF service is operated with the low-level option or the medium-level option, the execution status of a predetermined operation on the scale-out button 112a satisfies a given condition. In this case, the operation of the UPF service may be changed to the high-level option. Here, when the UPF service is operated with the low-level option, the policy manager unit 90 may generate a current status determination process 106b, a prediction determination process 106c, and an estimation process 108 associated with the UPF service. When the UPF service is operated with the medium-level option, the policy manager unit 90 may generate a prediction determination process 106c and an estimation process 108 associated with the UPF service. Then, the estimation process 108 may output estimation result data, and the prediction determination process 106c may start a process of determining whether or not to scale out the UPF 50 included in the UPF service based on the estimation result data. Then, if it is determined that scale-out is necessary, the UPF 50 included in the UPF service may be scaled out.

[0216] In this way, when the reception status of an instruction to execute a given action on an element satisfies a given condition, the management support unit 84 may start displaying a predicted value of the performance index value. Here, for example, when the reception status of an instruction to execute a given action on an element satisfies a given condition, the estimation process 108 that predicts the performance index value of the element may start predicting the performance index value of the element. Then, the management support unit 84 may start displaying the predicted value based on the prediction.

[0217] For example, as shown in FIG. 12, the management support unit 84 may start generating a monitoring screen on which a performance information image 110a showing a predicted value of the performance index value related to the UPF service type a1 is arranged.

[0218] The amount of information displayed on the monitoring screen can be increased by performing a process of predicting the performance index values ​​of elements included in communication system 1 and displaying a monitoring screen showing the predicted values, and a process of frequently updating the monitoring screen showing the latest performance index values ​​of the elements.

[0219] However, executing such processing uniformly for all elements included in the communication system 1 would be a waste of resources.

[0220] As described above, in this embodiment, when the status of a user's instruction to execute a given action on an element satisfies a given condition, at least one of the following is executed: start of display of predicted values ​​of performance index values; or shortening of the interval between updates of the monitoring screen. Therefore, the amount of information displayed on the monitoring screen increases only for some elements that satisfy the given condition. In this way, this embodiment can reduce waste of resources used to display a monitoring screen showing performance index values ​​of elements included in the communication system 1.

[0221] In this embodiment, when the acceptance status of an instruction to execute a given action on an element satisfies given conditions, the policy manager unit 90 may decide whether to start predicting the performance index value of the element based on the performance index value at least once when the execution instruction is accepted.

[0222] In this way, the performance index value at the time when the user instructs the execution of the action is reflected in whether or not to start prediction.

[0223] For example, as described above, it is assumed that the given condition is "accepting an instruction to execute the action a predetermined number of times (for example, three times) within the most recent predetermined period (for example, three days)."

[0224] Assume that an instruction to execute the action on a certain element has been received a predetermined number of times within the most recent predetermined time period. In this case, the policy manager unit 90 may identify a representative value (e.g., average, maximum, minimum, etc.) of the performance index value of the element at each timing when the instruction to execute the action was received. If the representative value is less than a predetermined threshold, the policy manager unit 90 may start predicting the performance index value of the element.

[0225] For example, suppose that a specific operation on the scale-out button 112a was performed a specific number of times during the most recent specific period of time while the system was operating with a low-level option. In this case, a representative value of the performance index value related to the UPF service type p1 at the timing of the specific number of times the operation was performed may be identified.

[0226] If the identified representative value is less than a predetermined threshold, prediction of the performance index value for the UPF service type p1 may be started. Then, the management support unit 84 may start displaying the predicted value of the performance index value.

[0227] On the other hand, if the performance index value for the type p1 of the UPF service at the time the operation is performed is equal to or less than a predetermined threshold, the update interval of the performance information image 110a may be shortened.

[0228] Furthermore, the management support unit 84 may shorten the update interval of the monitoring screen when the acceptance status of execution instructions satisfies a first condition, and start displaying the predicted value of the performance index value when the acceptance status of action execution instructions after the update interval of the monitoring screen has been shortened satisfies a second condition. In this way, the shortening of the update interval of the monitoring screen and the display of the predicted value of the performance index value are carried out in stages.

[0229] Here, the first condition and the second condition may be the same condition or may be different conditions.

[0230] For example, suppose that the first condition and the second condition are both "accepting an instruction to execute the action a predetermined number of times (e.g., three times) within a recent predetermined period (e.g., three days)."

[0231] In this situation, if the acceptance status of the execution instruction for the UPF service with identifier #31 satisfies the condition, the operation of the UPF service may be changed to operation with the medium level option, and the update interval of the performance information image 110a may be shortened from 60 minutes to 1 minute.

[0232] Then, when the acceptance status of the execution instruction for the UPF service satisfies the condition, the operation of the UPF service may be changed to the high-level option operation. Then, prediction of the performance index value for the type p1 of the UPF service may be started. Then, display of the predicted value of the performance index value may be started.

[0233] The management support unit 84 may also notify a user, such as an administrator, of an approval request for starting execution of a determination process for determining whether or not to execute a given action on a monitored element based on the performance index value of the monitored element. Then, the policy manager unit 90 may start execution of the determination process in response to approval of the approval request from the user.

[0234] For example, the management support unit 84 may transmit a recommendation screen shown in Fig. 13 to the purchaser terminal 36. Then, the purchaser terminal 36 may display the recommendation screen on a display or the like. Then, in response to clicking a purchase button 120 arranged on the recommendation screen, the policy manager unit 90 may start operation of the UPF service at the medium level option.

[0235] 14 to the purchaser terminal 36. The purchaser terminal 36 may then display the recommendation screen on a display or the like. When the purchase button 122 on the recommendation screen is clicked, the policy manager unit 90 may start operation of the UPF service with a high-level option.

[0236] Furthermore, as described above, after execution of the determination process based on the current performance index value of the element is started in response to approval of the approval request, a user such as an administrator may be notified of the approval request to start predicting the performance index value of the element. Then, in response to approval of the approval request from the user, the policy manager unit 90 may start predicting the performance index value of the element.

[0237] For example, when a predetermined time has elapsed (e.g., three months have elapsed) since operation with the medium-level option began, the management support unit 84 may transmit the recommendation screen shown in FIG. 14 to the purchaser terminal 36. The purchaser terminal 36 may then display the recommendation screen on a display or the like. Then, in response to clicking the purchase button 122 on the recommendation screen, the policy manager unit 90 may start operation with the high-level option for the UPF service.

[0238] The above-mentioned action in the present invention is not limited to scale-out, but may be, for example, replacement or scale-in.

[0239] Furthermore, although the above explanation has been given of a situation in which operation is changed from low-level options to medium-level or high-level options, the present invention is also applicable to a situation in which operation is changed from medium-level options to high-level options.

[0240] [Determination of Machine Learning Model] In the present embodiment, the AI ​​unit 70 may determine a machine learning model that outputs the above-described predicted value from among a plurality of trained machine learning models.

[0241] In the following description, it is assumed that a machine learning model that outputs a predicted value of a performance index value of type a1 for a UPF service with identifier #31 is determined.

[0242] Here, as described above, it is assumed that the management support unit 84 displays a monitoring screen showing a plurality of types of performance index values.

[0243] In this case, the AI ​​unit 70 may determine a machine learning model that will output a predicted value from among the multiple trained machine learning models based on the type of performance index value shown on the monitoring screen and the type of performance index value included in the input data input to each of the multiple trained machine learning models.

[0244] An example of the process of determining a machine learning model in such a determination will be described below.

[0245] Furthermore, in the communication system 1 according to this embodiment, the AI ​​unit 70 stores multiple machine learning models to be used for each of a plurality of given prediction purposes, and these machine learning models are in a state where they can be instantiated.

[0246] The inventory database 82 or the AI ​​unit 70 stores model management data, such as the data shown in FIG. 15, for managing these machine learning models.

[0247] 15 shows model management data associated with one prediction objective. In this embodiment, for example, the inventory database 82 or the AI ​​unit 70 stores model management data associated with each of a plurality of prediction objectives.

[0248] As shown in FIG. 15, the model management data includes objective management data and AI management data.

[0249] The goal management data included in the model management data is data associated with a prediction goal. The goal management data includes, for example, a goal ID and goal data, as shown in FIG. 15 . The goal ID included in the goal management data is, for example, an identifier of the prediction goal associated with the goal management data. The goal data included in the goal management data is, for example, data indicating the prediction goal associated with the goal management data. In the goal management data shown in FIG. 15 , the prediction goal indicated by the goal data is expressed as "a1".

[0250] [Correction based on Rule 91, 16.03.2023] The machine learning model according to this embodiment may output a predicted value of at least one type of performance index value. The type of predicted performance index value may be indicated in the target data. For example, the type of performance index value that is the predicted value output by the machine learning model may be indicated in the target data. Specifically, for example, the value of the target data may be "throughput," "latency," "number of registrations," "number of completed connections," "number of active users," etc.

[0251] The objective data may also indicate a prediction objective for a specific type of element included in the communication system 1, such as "UPF throughput" (e.g., the type of element and the type of performance index value predicted for that type of element).

[0252] The target data may also indicate the type of value calculated based on multiple types of performance index values. For example, a calculation formula for calculating an overall performance evaluation value based on throughput and latency may be set in the target data value.

[0253] The AI ​​management data included in the model management data is data for managing machine learning models used for prediction purposes that are associated with the model management data. The model management data includes multiple pieces of AI data that are each associated with a different machine learning model. The AI ​​data includes an AI ID and one or more pieces of input performance index value data.

[0254] For example, if three machine learning models with a prediction objective of "a1" are prepared, the model management data will include three pieces of AI data, as shown in Fig. 15. Note that the number of machine learning models used for one prediction objective is not limited to three.

[0255] The AIID included in the AI ​​data is an identifier of the machine learning model used for the prediction purpose associated with the model management data. In the example of Figure 15, the AIIDs of three machine learning models with the prediction purpose "a1" are "001," "002," and "003," respectively.

[0256] The input performance index value data included in the AI ​​data is data indicating the types of performance index values ​​to be input to the machine learning model associated with the AI ​​data. In this embodiment, for example, the AI ​​data associated with the machine learning model includes input performance index value data in the same number as the number of performance index values ​​to be input to the machine learning model.

[0257] The example in Figure 15 shows that the type of performance index value input to the machine learning model with AIID "001" is "b11". It also shows that the types of performance index values ​​input to the machine learning model with AIID "002" are "b21" and "b22". It also shows that the types of performance index values ​​input to the machine learning model with AIID "003" are "b31", "b32", and "b33".

[0258] In this way, the number of types of performance index values ​​input to a machine learning model may differ depending on the machine learning model. Also, in the example of Fig. 15, the number of types of performance index values ​​input to a machine learning model is one to three, but the number of types of performance index values ​​input to a machine learning model may be four or more.

[0259] Furthermore, the type of performance index value input to one machine learning model may be included in the type of performance index value input to another machine learning model. Furthermore, some of the types of performance index values ​​input to one machine learning model may overlap with some of the types of performance index values ​​input to another machine learning model. For example, "b11" and "b21" may be the same type of performance index value.

[0260] Furthermore, in this embodiment, the type of performance index value indicated by the input performance index value data and the type of performance index value associated with the predicted objective indicated by the objective data may be the same or different.

[0261] For example, "a1" and "b11" may be the same type of performance index value. For example, a case where a performance index value after a certain point in time is predicted based on the output when the actual throughput value at that point in time is input into a machine learning model corresponds to a case where the type of performance index value indicated by the input performance index value data is the same as the type of performance index value indicated by the target data.

[0262] [Correction based on Rule 91 16.03.2023] Examples of the types of performance indicator values ​​that can be entered include "throughput," "latency," "number of registrations," "number of completed connections," and "number of active users."

[0263] Here, the input performance index value data may indicate the type of element and the type of performance index value for that type of element. For example, if a performance index value "throughput" for an element "UPF" is input to a machine learning model, the AI ​​data associated with the machine learning model may include input performance index value data whose value is "UPF throughput."

[0264] In this embodiment, for example, for each of a plurality of AI data included in model management data that includes objective management data indicating a given prediction objective, it may be determined whether the type of performance index value indicated by the input performance index value data included in the AI ​​data is included in the types of performance index values ​​shown on the monitoring screen.

[0265] Here, for example, for each of a plurality of AI data included in model management data including objective management data in which the predicted objective indicated by the objective data is "a1", as shown in Figure 15, it may be determined whether the type of performance index value indicated by the input performance index value data included in the AI ​​data is included in the types of performance index values ​​shown on the monitoring screen.

[0266] Then, among the types of performance index values ​​indicated by the input performance index value data, a type of performance index value that is not shown on the monitoring screen may be determined as an additional performance index value type for the machine learning model associated with the AI ​​data.

[0267] Then, in this embodiment, for example, the AI ​​unit 70 determines at least one of the multiple machine learning models based on the type of additional performance index value identified for each machine learning model.

[0268] Here, the AI ​​unit 70 may determine at least one of the multiple machine learning models based on the number of types of additional performance indicator values. For example, a machine learning model associated with AI data having the fewest specified types of additional performance indicator values ​​may be determined. Alternatively, a machine learning model associated with AI data having fewer specified types of additional performance indicator values ​​than a predetermined number may be determined.

[0269] Furthermore, the AI ​​unit 70 may determine at least one of the multiple machine learning models based on the ratio of the number of additional performance indicator value types to the number of types of performance indicator values ​​input to the machine learning model. For example, a machine learning model may be determined that is associated with AI data for which the ratio of the number of identified additional performance indicator value types to the total number of input performance indicator value data is smallest. Alternatively, a machine learning model may be determined that is associated with AI data for which the ratio of the number of identified additional performance indicator value types to the total number of input performance indicator value data is smaller than a predetermined ratio.

[0270] Then, the AI ​​unit 70 may add performance index values ​​of the additional performance index value type that need to be added to use the machine learning model determined in this way to the targets to be monitored by the current status determination process 106 b. For example, a current status determination process 106 b associated with the performance index values ​​of the additional performance index value type for the trained machine learning model determined in this way may be generated.

[0271] In addition, in this embodiment, the AI ​​unit 70 may determine a machine learning model that will output a predicted value from among the multiple trained machine learning models based on the prediction accuracy of the predicted value evaluated for each of the multiple trained machine learning models.

[0272] An example of the process of determining a machine learning model in such a determination will be described below.

[0273] In this case, as shown in FIG. 16, the AI ​​unit 70 instantiates an untrained machine learning model 130 and generates a learning process 132 and a test process 134 associated with the machine learning model 130.

[0274] For example, assume that three untrained machine learning models 130 with AIIDs "001," "002," and "003" are instantiated. Hereinafter, the machine learning models 130 with AIIDs "001," "002," and "003" will be referred to as machine learning model 130a, machine learning model 130b, and machine learning model 130c, respectively.

[0275] Then, a learning process 132a associated with the machine learning model 130a and a test process 134a associated with the machine learning model 130a are generated. Also, a learning process 132b associated with the machine learning model 130b and a test process 134b associated with the machine learning model 130b are generated. Also, a learning process 132c associated with the machine learning model 130c and a test process 134c associated with the machine learning model 130c are generated.

[0276] In this embodiment, as described above, the big data platform unit 66 accumulates performance index value files relating to elements included in a network service purchased by a purchaser of the network service.

[0277] In this embodiment, for example, the AI ​​unit 70 acquires data indicating a time series of actual values ​​of multiple types of performance index values ​​related to the communication system 1.

[0278] A part of the data acquired in this manner corresponds to test data indicating a time series of actual values ​​of multiple types of performance index values ​​related to the communication system 1. The rest corresponds to training data indicating a time series of actual values ​​of multiple types of performance index values ​​related to the communication system 1.

[0279] In the following description, it is assumed that the multiple types include at least "a1", "b11", "b21", "b22", "b31", "b32", and "b33" shown in FIG.

[0280] Here, the training data is different from the test data. For example, data showing performance index values ​​up to a certain point in time may be used as the training data, and data showing performance index values ​​after that point in time may be used as the test data. Alternatively, data showing performance index values ​​up to a certain point in time may be used as the test data, and data showing performance index values ​​after that point in time may be used as the training data.

[0281] Here, for example, the AI ​​unit 70 acquires at least one performance index value file stored in the big data platform unit 66 and related to elements included in the network service purchased by the above-mentioned purchaser.

[0282] Then, the learning process 132 generates a training data element set shown in Fig. 17 based on training data, which is part of the data included in the acquired performance index value file. As shown in Fig. 17, the training data element set includes multiple training data elements, and each training data element includes learning input data and teacher data.

[0283] Here, for example, the learning process 132a may generate training data elements including learning input data that includes performance index value data whose performance index value type is "b11" and that is included in the performance index value file, and teacher data that includes performance index value data whose performance index value type is "a1".

[0284] The learning process 132b may then generate training data elements including learning input data including performance index value data whose performance index value type is "b21" and performance index value data whose performance index value type is "b22" contained in the performance index value file, and teacher data including performance index value data whose performance index value type is "a1".

[0285] The learning process 132c may then generate training data elements including learning input data including performance index value data whose performance index value type is "b31", performance index value data whose performance index value type is "b32", and performance index value data whose performance index value type is "b33", which are contained in the performance index value file, and teacher data including performance index value data whose performance index value type is "a1".

[0286] 18 is a diagram schematically illustrating an example of training data elements generated by the learning process 132c. Here, for example, training input data is generated, which includes performance index value data D1 indicating a performance index value of type "b31," performance index value data D2 indicating a performance index value of type "b32," and performance index value data D3 indicating a performance index value of type "b33" for a period of length T1 (e.g., 60 minutes) up to a certain reference time point. Then, teacher data D4 is generated, which includes performance index value data indicating a performance index value of type "a1" for a period of length T2 from the reference time point (e.g., a period from the reference time point to 20 minutes after the reference time point). Finally, training data elements are generated, which include the training input data and teacher data generated in this manner.

[0287] A training data element set including a plurality of training data elements generated as described above for various reference time points is then generated by the learning process 132c.

[0288] Similarly, training data element sets are generated by the learning processes 132a and 132b.

[0289] Here, when the machine learning model 130 accepts input of performance index value data at multiple points in time, the learning input data includes performance index value data at multiple points in time, as shown in Fig. 18. On the other hand, when the machine learning model 130 accepts input of performance index value data at a single point in time, the learning input data includes performance index value data at a single point in time.

[0290] Furthermore, when the machine learning model 130 outputs predicted values ​​at multiple points in time, the training data includes performance index value data at multiple points in time, as shown in Fig. 18. On the other hand, when the machine learning model 130 outputs predicted values ​​at a single point in time, the training data includes performance index value data at a single point in time.

[0291] Then, the learning process 132 uses the training data element set generated as described above to perform learning of the machine learning model 130 associated with the learning process 132, thereby generating a trained machine learning model 130.

[0292] 18, the learning process 132c may calculate the value of a given evaluation function (error function) based on the output D5 obtained when learning input data included in the training data elements is input to the machine learning model 130c and the teacher data D4 included in the training data elements. The learning process 132c may then update the parameters of the machine learning model 130c based on the calculated value of the evaluation function. The machine learning model 130c may then be trained by updating the parameters of the machine learning model 130c based on each of the multiple training data elements included in the training data element set generated by the learning process 132c, thereby generating a trained machine learning model 130c.

[0293] Similarly, trained machine learning model 130a may be generated by training machine learning model 130a using the training data element set generated by training process 132a, and trained machine learning model 130b may be generated by training machine learning model 130b using the training data element set generated by training process 132b.

[0294] Here, as described above, the machine learning model may output a predicted value calculated based on a plurality of types of performance index values.

[0295] In this case, the training data may include performance index value data indicating the plurality of types of performance index values. An overall performance evaluation value may be calculated based on the plurality of types of performance index values ​​according to a given calculation formula. A value of a given evaluation function (error function) may be calculated based on the calculated overall performance evaluation value and a predicted value of the overall performance evaluation value output from the machine learning model.

[0296] Alternatively, training data elements may be generated that include teacher data for which an overall performance evaluation value is set, the overall performance evaluation value being calculated according to a given formula based on multiple types of performance index values.The value of a given evaluation function (error function) may then be calculated based on the overall performance evaluation value indicated by the teacher data and a predicted value of the overall performance evaluation value output from the machine learning model.

[0297] Furthermore, as described above, a machine learning model may output a predicted value of a performance index value of the same type as the input performance index value.

[0298] In this case, training data elements may be generated that include learning input data indicating a certain type of performance index value for a period of length T1 up to a reference time point, and teacher data indicating that type of performance index value for a period of length T2 from the reference time point. Then, the value of a given evaluation function (error function) may be calculated based on the performance evaluation value indicated by the teacher data and the predicted value output from the machine learning model.

[0299] Furthermore, the test process 134 generates a test data element set shown in Fig. 19 based on the test data included in the performance index value file acquired as described above. As shown in Fig. 19, the test data element set includes a plurality of test data elements, and each test data element includes test input data and comparison data.

[0300] Here, for example, the test process 134a may generate test data elements including test input data that includes performance index value data whose performance index value type is "b11" and that is included in the performance index value file, and comparison data that includes performance index value data whose performance index value type is "a1".

[0301] Then, the test process 134b may generate test data elements including performance index value data whose performance index value type is "b21" and performance index value data whose performance index value type is "b22" contained in the performance index value file, and comparison data which includes performance index value data whose performance index value type is "a1".

[0302] Then, the test process 134c may generate test data elements including test input data including performance index value data whose performance index value type is "b31", performance index value data whose performance index value type is "b32", and performance index value data whose performance index value type is "b33", which are included in the performance index value file, and comparison data including performance index value data whose performance index value type is "a1".

[0303] 20 is a diagram schematically illustrating an example of a test data element generated by the test process 134c. Here, for example, test input data is generated that includes performance index value data D6 indicating a performance index value of type "b31," performance index value data D7 indicating a performance index value of type "b32," and performance index value data D8 indicating a performance index value of type "b33" for a period of length T1 (e.g., 60 minutes) up to a certain reference time point. Then, comparison data D9 is generated that includes performance index value data indicating a performance index value of type "a1" for a period of length T2 from the reference time point (e.g., a period from the reference time point to 20 minutes after the reference time point). Finally, a test data element is generated that includes the test input data and comparison data generated in this manner.

[0304] Then, a test data element set including a plurality of test data elements generated as described above for various reference points in time is generated by the test process 134c.

[0305] Similarly, test processes 134a and 134b generate test data element sets.

[0306] In this way, test data elements are generated that are formatted similarly to the format of the training data.

[0307] In this embodiment, for example, the type of performance index value indicated by the performance index value data included in the training input data corresponding to the machine learning model is the same as the type of performance index value indicated by the performance index value data included in the test input data corresponding to the machine learning model.Furthermore, the type of performance index value indicated by the performance index value data included in the training data corresponding to the machine learning model is the same as the type of performance index value indicated by the performance index value data included in the comparison data corresponding to the machine learning model.

[0308] In this embodiment, for example, the number of performance index value data included in the training input data corresponding to a machine learning model is the same as the number of performance index value data included in the test input data corresponding to the machine learning model. Also, the number of performance index value data included in the teacher data corresponding to a machine learning model is the same as the number of performance index value data included in the comparison data corresponding to the machine learning model.

[0309] In this embodiment, as described above, the training data and the test data are different data, and the training data is not used as the test data.

[0310] In the present embodiment, for example, the AI ​​unit 70 inputs input data corresponding to each of a plurality of trained machine learning models 130 used for a given prediction purpose related to the communication system 1. Here, the input data is part of test data and is data indicating actual performance index values ​​of at least one type at at least one point in time. Furthermore, the input data input to each of the plurality of trained machine learning models 130 is different from one another. In the present embodiment, for example, the AI ​​unit 70 acquires, as an output from the machine learning model 130, a predicted value at a prediction point in time that is later than any of the at least one point in time.

[0311] As described above, the type of actual value indicated by the input data and the type of predicted value output from the machine learning model 130 may be the same or different.

[0312] For example, the test process 134 inputs test input data, which is included in a test data element and indicates a performance index value at at least one time point, into the trained machine learning model 130. The test process 134 then obtains the output when the test input data is input into the machine learning model 130. This output indicates a predicted value at a prediction time point that is later than any of the at least one time point described above. For example, as shown in FIG. 20 , the test process 134c obtains the output D10 when the test data element is input into the machine learning model 130c. The predicted value indicated by this output D10 is the predicted value of a performance index value of type "a1".

[0313] Then, in this embodiment, for example, the AI ​​unit 70 evaluates the accuracy of the prediction for the prediction purpose by each of the multiple trained machine learning models based on the obtained predicted value and a portion of the test data that indicates the actual value at the time of prediction for at least one type corresponding to the predicted value.

[0314] For example, the test process 134 evaluates the accuracy of the prediction made by the machine learning model 130 for the above-mentioned prediction purpose based on the comparison data contained in the test data element and the output when the test input data contained in the test data element is input into the machine learning model 130.

[0315] Here, for example, the test process 134 may calculate the value of a given evaluation function (error function) based on the comparison data included in the test data element and the output when the test input data included in the test data element is input to the machine learning model 130. Then, the test process 134 may calculate a representative value (sum, average, etc.) of the evaluation functions calculated for multiple test data elements as an evaluation value of the accuracy of prediction for the above-mentioned prediction purpose by the machine learning model.

[0316] For example, the test process 134c calculates the value of a given evaluation function based on the comparison data D9 included in the test data element and the output D10. Then, the test process 134c evaluates the accuracy of the prediction for the prediction objective "a1" by the machine learning model 130c based on the value of the evaluation function calculated for each of the multiple test data elements included in the test data element set.

[0317] Then, the AI ​​unit 70 determines at least one of the plurality of trained machine learning models 130 based on the evaluation results of the prediction accuracy for each of the plurality of machine learning models 130. Here, for example, a machine learning model suitable for the communication system 1 is determined.

[0318] Here, the AI ​​unit 70 may determine, for example, the machine learning model 130 having the smallest representative value of the evaluation function as the machine learning model suitable for the network service. Alternatively, the AI ​​unit 70 may determine, for example, one or more machine learning models 130 having a representative value of the evaluation function smaller than a predetermined value as the machine learning model suitable for the network service.

[0319] Here, as described above, the machine learning model may output a predicted value calculated based on a plurality of types of performance index values.

[0320] In this case, the comparison data may include performance index value data indicating the plurality of types of performance index values. An overall performance evaluation value may be calculated based on the plurality of types of performance index values ​​according to a given calculation formula. A value of a given evaluation function (error function) may be calculated based on the calculated overall performance evaluation value and a predicted value of the overall performance evaluation value output from the machine learning model.

[0321] Alternatively, test data elements may be generated that include comparison data for which an overall performance evaluation value is set, the overall performance evaluation value being calculated according to a given formula based on multiple types of performance index values. Then, a value of a given evaluation function (error function) may be calculated based on the overall performance evaluation value indicated by the comparison data and a predicted value of the overall performance evaluation value output from the machine learning model.

[0322] Furthermore, as described above, a machine learning model may output a predicted value of a performance index value of the same type as the input performance index value.

[0323] In this case, test data elements may be generated that include test input data indicating a certain type of performance index value for a period of length T1 up to a reference time point and comparison data indicating the same type of performance index value for a period of length T2 from the reference time point. Then, a value of a given evaluation function (error function) may be calculated based on the performance evaluation value indicated by the comparison data and the predicted value output from the machine learning model.

[0324] The AI ​​unit 70 may then add the type of performance index value that needs to be added to use the machine learning model determined in this manner to the monitoring targets of the current status determination process 106b. For example, a current status determination process 106b associated with the type of performance index value that needs to be added to use the machine learning model determined in this manner may be generated. The generated current status determination process 106b may then perform a determination process (in other words, a process of monitoring at least one type of performance index value related to the communication system 1).

[0325] In this manner, the machine learning model that outputs the above-mentioned predicted value is determined.

[0326] Then, an estimation process 108 and a prediction determination process 106c associated with the current status determination process 106b may be generated. Here, for example, the estimation process 108 and the prediction determination process 106c may be activated. Then, the trained machine learning model determined in this manner may be instantiated. Then, the estimation process 108 may predict a performance index value of the communication system 1 using the machine learning model determined in this manner.

[0327] In addition, in this embodiment, for each of a plurality of prediction purposes, at least one (e.g., at least one suitable for the communication system 1) may be determined from a plurality of machine learning models used for that prediction purpose.

[0328] In this way, it is possible to accurately determine a machine learning model suitable for the communication system 1, which is used to predict the performance index value of the communication system 1.

[0329] [Processing Flow] Here, an example of the flow of processing performed in the platform system 30 according to this embodiment will be described with reference to the flow diagram shown in Fig. 21. The following processing is executed for each element whose performance information is displayed on the monitoring screen. In the following explanation, attention is focused on one of these elements, and an example of the flow of processing executed for that element will be described.

[0330] First, the management support unit 84 monitors whether the acceptance status of an instruction to execute a given action for the element satisfies a given condition (S101).

[0331] When the element satisfies a given condition, the policy manager unit 90 determines whether or not to start displaying the predicted value of the performance index (S102).

[0332] Then, the policy manager unit 90 determines whether or not to shorten the update interval of the monitoring screen (S103).

[0333] The policy manager unit 90 then executes processing according to the determination results of the processing shown in S102 and S103 (S104), and returns to the processing shown in S101. If it is determined in the processing shown in S102 that the display of the predicted value of the performance index value should be started, the processing shown in S104 starts displaying the predicted value of the performance index value of the element. Also, if it is determined in the processing shown in S103 that the update interval of the monitoring screen should be shortened, the processing shown in S104 shortens the update interval of the performance information image 110 corresponding to the element.

[0334] [Supplementary Note] The present invention is not limited to the above-described embodiment.

[0335] For example, in this embodiment, scale-out of elements of types other than the elements described above may be performed. For example, scale-out of the SMF 48 may be performed. In this case, performance index value data related to the elements of the RAN 32 and the elements of the core network system 34 may be used to determine whether to perform scale-out.

[0336] Similarly, transport scale-out may be performed.

[0337] Furthermore, the machine learning model determination process described above and the processes related to this determination process may be executed by a functional module other than the AI ​​unit 70.

[0338] Furthermore, the functional units according to this embodiment are not limited to those shown in FIG.

[0339] Furthermore, the functional unit according to this embodiment does not need to be a NF in 5G. For example, the functional unit according to this embodiment may be a network node in 4G, such as an eNodeB, a vDU, a vCU, a Packet Data Network Gateway (P-GW), a Serving Gateway (S-GW), a Mobility Management Entity (MME), or a Home Subscriber Server (HSS).

[0340] Furthermore, the functional units according to the present embodiment may be realized using hypervisor-type or host-type virtualization technology instead of container-type virtualization technology. Furthermore, the functional units according to the present embodiment do not need to be implemented by software, but may be implemented by hardware such as electronic circuits. Furthermore, the functional units according to the present embodiment may be implemented by a combination of electronic circuits and software.

[0341] The technology described in the present disclosure can also be expressed as follows: [1] A display control system comprising: a monitoring screen update means for repeatedly updating a monitoring screen showing performance index values ​​of elements included in a communication system at at least one point in time; an action execution means for executing a given action on the element in response to receipt of an instruction to execute the action; and a monitoring change execution means for at least one of starting to display a predicted value of the performance index value or shortening the update interval of the monitoring screen when the receipt status of the execution instruction satisfies a given condition. [2] The display control system described in [1], further comprising: a prediction means for predicting the performance index value of the element, wherein when the receipt status of the execution instruction satisfies a given condition, the prediction means starts predicting the performance index value of the element, and the monitoring change execution means starts displaying the predicted value based on the prediction. [3] The display control system according to [3], further comprising: a start determination means for determining whether to start predicting the performance index value of the element based on the performance index value at least once when the execution instruction is accepted, when the acceptance status of the execution instruction satisfies a given condition; wherein when it is determined to start the prediction, the prediction means starts predicting the performance index value of the element, and the monitoring change execution means starts displaying the predicted value based on the prediction. [4] The display control system according to any one of [1] to [3], characterized in that the monitoring change execution means shortens the update interval of the monitoring screen when the acceptance status of the execution instruction satisfies a first condition; and starts displaying the predicted value of the performance index value when the acceptance status of the action execution instruction after the update interval of the monitoring screen is shortened satisfies a second condition. [5] The display control system according to any one of [1] to [4], further comprising: a machine learning model determination means for determining a machine learning model that outputs the predicted value from among a plurality of trained machine learning models.[6] The display control system of [5], wherein the machine learning model determination means determines a machine learning model that outputs the predicted value from among the plurality of trained machine learning models based on the type of performance index value displayed on the monitoring screen and the type of performance index value included in input data input to each of the plurality of trained machine learning models. [7] The display control system of [5], wherein the machine learning model determination means determines a machine learning model that outputs the predicted value from among the plurality of trained machine learning models based on the prediction accuracy of the predicted value evaluated for each of the plurality of trained machine learning models. [8] A display control method comprising: repeatedly updating a monitoring screen showing performance index values ​​of elements included in a communication system at at least one time point; executing a given action on the element in response to reception of an instruction to execute the action; and, when a reception status of the execution instruction satisfies a given condition, performing at least one of starting to display the predicted value of the performance index value or shortening the update interval of the monitoring screen.

Claims

Monitoring screen update means for repeatedly updating a monitoring screen showing performance index values at at least one point in time of elements included in a communication system; Action execution means for executing the action on the element in response to receiving an execution instruction for a given action on the element; Monitoring change execution means for executing at least one of starting display of a predicted value of the performance index value or shortening an update interval of the monitoring screen when a reception status of the execution instruction satisfies a given condition; A display control system including the above. The display control system according to claim 1, further comprising prediction means for predicting the performance index value of the element. When the reception status of the execution instruction satisfies a given condition, the prediction means starts predicting the performance index value of the element, and the monitoring change execution means starts displaying a predicted value based on the prediction. The display control system according to claim 1. The display control system according to claim 2, further comprising start determination means for determining whether to start prediction in the prediction process based on the performance index value at at least one timing when the execution instruction is received when the reception status of the execution instruction satisfies a given condition. When it is determined to start prediction in the prediction process, the prediction means starts predicting the performance index value of the element, and the monitoring change execution means starts displaying a predicted value based on the prediction. The display control system according to claim 2.

4. The monitoring change execution means: Shortens the update interval of the monitoring screen when the reception status of the execution instruction satisfies a first condition; Starts displaying a predicted value of the performance index value when the reception status of the execution instruction after the update interval of the monitoring screen is shortened satisfies a second condition. The display control system according to claim 1. The display control system according to claim 1, further comprising machine learning model determination means for determining a machine learning model that outputs the predicted value from among a plurality of learned machine learning models. The display control system according to claim 1.

6. The machine learning model determination means determines, from among the plurality of learned machine learning models, a machine learning model that outputs the predicted value based on the type of performance index value shown on the monitoring screen and the type of performance index value included in the input data input to the machine learning model for each of the plurality of learned machine learning models. The display control system according to claim 5.

7. The machine learning model determination means determines, from among the plurality of learned machine learning models, a machine learning model that outputs the predicted value based on the prediction accuracy of the predicted value to be evaluated for each of the plurality of learned machine learning models. The display control system according to claim 5.

8. Repeatedly updating a monitoring screen showing a performance index value at at least one point in time of an element included in the communication system; Executing the action on the element in response to receiving an execution instruction for a given action on the element; When the reception status of the execution instruction satisfies a given condition, performing at least one of starting display of a predicted value of the performance index value or shortening an update interval of the monitoring screen; A display control method including the above.