Management device and method for identifying network abnormality

Abstract

According to one embodiment, a method performed by a management device may comprise the operations of: acquiring connection state information for a plurality of network functions (NFs) connected to the management device; inputting at least a part of the connection state information into a statistics-based first model included in a prediction model for anomaly detection for the plurality of NFs; inputting the remaining part of the connection state information into an artificial intelligence-based second model included in the prediction model; identifying, on the basis of output data of the prediction model, at least one NF from which an anomaly has been detected among the plurality of NFs; and adjusting a call connection to the at least one NF.

Description

Management device and method for identifying network anomalies

[0001] The present disclosure relates to a communication system. More specifically, the present disclosure relates to a management device and method for identifying network anomalies.

[0002] To support the communication system, base stations can be connected to a core network. The 5G core network is based on a service-based architecture (SBA) centered around network function (NF) services. NFs can be organized into groups. If a problem occurs in a group of NFs, the entire service will be affected.

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above-described matters constitute prior art related to the present disclosure.

[0004] According to one embodiment, a management device may include at least one transceiver, a memory storing one or more instructions and including a storage medium, and at least one processor including a processing circuit. The one or more instructions, when individually or collectively executed by the at least one processor, may cause the management device to obtain connection state information for a plurality of network functions (NFs) connected to the management device, input at least a portion of the connection state information into a statistical-based first model included in a predictive model for anomaly detection for the plurality of NFs, input the remaining portion of the connection state information into an artificial intelligence-based second model included in the predictive model, identify at least one NF among the plurality of NFs in which an anomaly has been detected based on output data of the predictive model, and adjust a call connection for the at least one NF.

[0005] According to one embodiment, a method performed by a management device may include an operation of obtaining connection status information for a plurality of network functions (NFs) connected to the management device, an operation of inputting at least a portion of the connection status information into a statistical first model included in a prediction model for anomaly detection for the plurality of NFs, an operation of inputting the remaining portion of the connection status information into an artificial intelligence-based second model included in the prediction model, an operation of identifying at least one NF among the plurality of NFs in which an anomaly is detected based on output data of the prediction model, and an operation of adjusting a call connection for the at least one NF.

[0006] According to one embodiment, a non-transitory computer-readable storage medium may store one or more programs. The one or more programs may include instructions that, when executed by at least one processor of a management device, cause the management device to obtain connection status information for a plurality of network functions (NFs) connected to the management device, input at least some of the connection status information into a statistical-based first model included in a predictive model for anomaly detection for the plurality of NFs, input the remaining some of the connection status information into an artificial intelligence-based second model included in the predictive model, identify at least one NF among the plurality of NFs in which an anomaly has been detected based on output data of the predictive model, and adjust a call connection for the at least one NF.

[0007] Figure 1a illustrates an example of a communication system.

[0008] Figure 1b illustrates an example of a core network.

[0009] Figure 2a shows an example of NF (network function) selection.

[0010] Figure 2b shows an example of a group for NFs.

[0011] Figure 3 shows an example of components of a management device for NF selection.

[0012] Figure 4a shows an example of signaling of a management device and an NF for obtaining connection status information.

[0013] Figure 4b illustrates an example of data collected from a management device.

[0014] Figure 5a illustrates an example of a technique for identifying anomalies in NF.

[0015] Figure 5b illustrates an example of a technique for identifying anomalies in NF.

[0016] Figure 5c illustrates an example of a technique for identifying anomalies in NF.

[0017] Figure 5d illustrates an example of a technique for identifying anomalies in NF.

[0018] Figure 5e illustrates an example of a technique for identifying anomalies in NF.

[0019] Figure 6 illustrates an example of an ensemble model constructed based on one or more models.

[0020] Figure 7a illustrates an example of the operation of the management device when an abnormality occurs in NF.

[0021] Figure 7b illustrates an example of the operation of the management device when an abnormality occurs in NF.

[0022] Figure 8 shows an example of a quality indicator for anomaly detection of PCF (policy control function).

[0023] Figure 9 shows the change in the number of occurrences of abnormalities according to abnormality detection of the management device.

[0024] Figure 10 illustrates an example of the operation of the management device.

[0025] The terms used in this disclosure are used only to describe specific embodiments and may not be intended to limit the scope of other embodiments. The singular expression may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by those of ordinary skill in the art described in this disclosure. Terms defined in general dictionaries among the terms used in this disclosure may be interpreted as having the same or similar meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined in this disclosure. In some cases, even if a term is defined in this disclosure, it cannot be interpreted to exclude embodiments of the present disclosure.

[0026] The various embodiments of the present disclosure described below illustrate a hardware-based approach as an example. However, since the various embodiments of the present disclosure include techniques utilizing both hardware and software, the various embodiments of the present disclosure do not exclude a software-based approach.

[0027] In the following description, terms referring to signals (e.g., signal, information, message, signaling, data), terms referring to data types (e.g., list, set, subset), terms for operational states (e.g., step, operation, procedure), terms referring to data (e.g., packet, user stream, information, bit, symbol, codeword), terms referring to resources (e.g., symbol, slot, subframe, radio frame, subcarrier, resource element (RE), resource block (RB), bandwidth part (BWP), occasion), terms referring to channels, terms referring to network entities, terms referring to components of devices, etc. are examples for convenience of description. Therefore, the present disclosure is not limited to the terms described below, and other terms having equivalent technical meanings may be used.

[0028] In the following description, terms referring to signals (e.g., signal, information, message, signaling), terms referring to resources (e.g., symbol, slot, subframe, radio frame, subcarrier, resource element (RE), resource block (RB), bandwidth part (BWP), occasion), terms for operational states (e.g., step, operation, procedure), terms referring to data (e.g., packet, user stream, information, bit, symbol, codeword), terms referring to channels, terms referring to network entities, terms referring to components of devices, etc. are examples for convenience of description. Therefore, the present disclosure is not limited to the terms described below, and other terms having equivalent technical meanings may be used.

[0029] In the present disclosure, expressions such as “more than” or “less than” may be used to determine whether a specific condition is satisfied or fulfilled, but this is merely a description for expressing an example and does not exclude descriptions such as “more than” or “less than.” A condition described as “more than” may be replaced with “more than,” a condition described as “less than” may be replaced with “less than,” and a condition described as “more than and less than” may be replaced with “more than and less than.” In addition, hereinafter, “A” to “B” mean at least one of elements from A (including A) to B (including B). hereinafter, “C” and / or “D” mean at least one of “C” or “D,” that is, including {“C,” “D,” “C” and “D”}.

[0030] Although the present disclosure describes various embodiments using terms used in some communication standards (e.g., 3rd Generation Partnership Project (3GPP), European Telecommunications Standards Institute (ETSI), extensible radio access network (xRAN), open-radio access network (O-RAN), etc.), these are merely examples for explanation. The various embodiments of the present disclosure can be easily modified and applied to other communication systems.

[0031] Figure 1a illustrates an example of a communication system.

[0032] Referring to FIG. 1A, a communication system (100) may include a terminal (110). The terminal (110) is a device used by a user and communicates with a base station (120) via a wireless channel. The link from the base station (120) to the terminal (110) is referred to as a downlink (DL), and the link from the terminal (110) to the base station (120) is referred to as an uplink (UL). In addition, although not shown in FIG. 1A, the terminal (110) and another terminal may communicate with each other via a wireless channel. In this case, the link between the terminal (110) and another terminal (device-to-device link, D2D) is referred to as a sidelink, and the sidelink may be used interchangeably with the PC5 interface. In some other embodiments, the terminal (110) may be operated without the intervention of a user. According to one embodiment, the terminal (110) is a device that performs machine type communication (MTC) and may not be carried by a user. Furthermore, according to one embodiment, the terminal (110) may be a narrowband (NB)-internet of things (IoT) device. The terminal (110) may be referred to as a 'user equipment (UE)', a 'customer premises equipment (CPE)', a 'mobile station', a 'subscriber station', a 'remote terminal', a 'wireless terminal', an electronic device', a 'user device', or other terms having equivalent technical meanings thereto. Hereinafter, in describing the mobility of the terminal of the present disclosure, the terminal (110) is referred to as a UE, but it is to be understood that other terms may be used depending on the communication environment or embodiment.

[0033] The base station (120) is a network infrastructure that provides wireless access to the terminal (110). The base station (120) has coverage defined based on the distance over which a signal can be transmitted. In terms of providing an access network (AN) other than a base station, the base station (120) may be referred to as a 'RAN node', a 'network node', or an 'access point (AP)', or in terms of a supported radio access technology (RAT), it may be referred to as an 'eNodeB (eNB)', a '5G node (5th generation node)', a 'next generation nodeB (gNB)', a 'wireless point', a 'transmission / reception point (TRP)', or other terms having an equivalent technical meaning thereto.

[0034] Although a single network entity is illustrated in FIG. 1A, embodiments of the present disclosure are not limited thereto. For example, the base station (120) may be implemented in a distributed deployment according to a central unit (CU) configured to perform functions of upper layers (e.g., PDCP, RRC) of the access network and a distributed unit (DU) configured to perform functions of lower layers (e.g., RLC, MAC, PHY). For example, to reduce installation costs and increase the available cell coverage, the base station (120) may be implemented with geographically distributed DUs and RUs.

[0035] The core network (133) may be configured to connect the base station (120) to a data network. The core network (133) may include various network entities for managing mobility, session management, policy management, and / or data network connections, and each network entity may represent a node defining a specific network function. For example, the core network (133) may be referred to as an evolved packet core (EPC) (or evolved packet system (EPS)) as a set of network entities for an LTE access network. For example, the core network (133) may be referred to as a 5GC (5 th generation core)(or 5GS(5 th As an example of a core network (133), 5GC is described in detail through Fig. 1b.

[0036] Figure 1b illustrates an example of a core network.

[0037] Referring to FIG. 1B, the UE exemplifies the terminal (110) of FIG. 1A, and the RAN Node exemplifies the base station (120) of FIG. 1B. Network entities of the core network (133) may include various network functions (NFs). The terminal (110) and the base station (120) may communicate with the NFs of the core network. For example, the core network (133) may include an access and mobility management function (AMF) (130), a session management function (SMF) (140), a user plane function (UPF) (150), a policy and charging function (PCF) (170), and a unified data management (UDM) (180).

[0038] AMF (130) provides a function for access and mobility management per UE (e.g., terminal (110)), and one UE can basically be connected to one AMF.Specifically, the AMF (130) provides signaling between CN (e.g., core network (133)) nodes for mobility between 3GPP access networks, termination of a radio access network (RAN) control plane (CP) interface (i.e., N2 interface), termination of NAS signaling (N1), non-access stratum (NAS) signaling security (NAS ciphering and integrity protection), access stratum (AS) security control, registration management (registration area management), connection management, idle mode UE reachability (including control and performance of paging retransmission), mobility management control (subscription and policy), intra-system mobility and inter-system mobility support, support of network slicing, SMF selection, lawful intercept (for AMF events and interfaces to the LI system), and communication between the UE and the SMF (e.g., It can support functions such as providing transmission of session management (SM) messages between SMFs (140), transparent proxy for SM message routing, access authentication, access authorization including roaming permission check, providing transmission of short message service (SMS) messages between UEs and short message service function (SMSF), security anchor function (SAF) and / or security context management (SCM). Some or all of the functions of AMF (130) can be supported within a single instance of one AMF.Depending on the example, the AMF (130) may select an NF from among multiple NFs. For example, the AMF (130) may perform SMF selection or PCF (policy charging function) selection.

[0039] SMF (140) can provide session management functions. If a UE (e.g., terminal (110)) has multiple sessions, each session can be managed by a different SMF. Specifically, the SMF (140) may support functions such as session management (e.g., session establishment, modification, and termination, including tunnel maintenance between the UPF (150) and the AN node (e.g., base station (120)), UE IP address allocation and management (optionally including authentication), selection and control of UP functions, traffic steering setup to route traffic from the UPF to the appropriate destination, termination of the interface to policy control functions, enforcement of the control portion of policies and quality of service (QoS), lawful intercept (for SM events and interfaces to the LI system), termination of the SM portion of NAS messages, downlink data notification, initiator of AN specific SM information (delivered to the AN node via N2 via AMF), determination of the SSC mode of the session (e.g., SSC mode 2, SSC mode 3), roaming functions, etc. According to embodiments, UPF selection may be performed by the SMF (140). Some or all of the functions of the SMF (140) may be supported within a single instance of one SMF. According to embodiments, the SMF (140) may select an NF from among multiple NFs. For example, the SMF (140) may perform UPF selection or PCF selection.

[0040] UPF (150) can transmit a downlink PDU received from DN (155) to terminal (110) via base station (120), or transmit an uplink PDU received from terminal (110) via base station (120) to DN (155). Specifically, the UPF (150) may support functions such as an anchor point for intra / inter RAT mobility, an external PDU session point for interconnection to a data network, a user plane portion of packet routing and forwarding, packet inspection and policy rule enforcement, an uplink classifier to support lawful intercept, traffic usage reporting, routing of traffic flows to the data network, a branching point to support multi-homed PDU sessions, QoS handling for the user plane (e.g., packet filtering, gating, uplink / downlink rate enforcement), uplink traffic validation (service data flow (SDF) mapping between SDFs and QoS flows), transport level packet marking in uplink and downlink, downlink packet buffering and downlink data notification triggering. Some or all of the functions of UPF (150) may be supported within a single instance of a UPF.

[0041] DN (155) represents an Internet network for connecting to an external communication network. For example, DN (155) may be an operator service, Internet access, or a third party (3 rdparty) service, etc. DN (155) transmits a downlink protocol data unit (PDU) to UPF (150) or receives a PDU transmitted from a terminal (110) from UPF (150).

[0042] PCF (170) can receive information about packet flow from an application server and provide a function to determine policies such as mobility management and session management. Specifically, PCF (170) supports functions such as supporting a unified policy framework for controlling network operations, providing policy rules so that CP function(s) (e.g., AMF (130), SMF (140), etc.) can enforce the policy rules, and implementing a front end for accessing related subscription information for policy determination within a user data repository (UDR).

[0043] UDM (180) stores user subscription data, policy data, etc. UDM (180) may include two parts: an application front end (FE) and a user data store (UDR).

[0044] The core network (133) may include various NFs in addition to the network entities / network functions described above. For example, the core network (133) may include a network slice selection function (NSSF) (191), a network exposure function (NEF) (192), a network repository function (NRF) (193), a network slice-specific authentication and authorization (NSSAAF) (194), an authentication server function (AUSF) (195), an application function (AF) (196), a service communication proxy (SCP) (197), and a network slice admission control function (NSACF) (198).

[0045] NSSF (191) may support the function of selecting a set of network slice instances that provide services to the terminal (110). NSSF (191) may determine allowed NSSAI (network slice selection assistance information) and, if needed, determine a mapping to a subscribed S (single)-NSSAI. NSSF (191) may determine a configured NSSAI and, if needed, determine a mapping to a subscribed S-NSSAI. NSSF (191) may determine a set of AMFs used to serve the UE, or, based on the configuration, by querying the NRF for a list of AMFs. NSSF (191) may provide support for network slice restrictions and network slice instance restrictions based on NWDAF analysis.

[0046] The NEF (192) may provide a means to securely expose services and capabilities provided by 3GPP NFs, for example, for third parties, internal exposure / re-exposure, application functions, and edge computing. The NEF (192) receives information from other NFs or based on capabilities exposed to other NFs. The NEF (192) may store the received information as structured data using a standardized interface to a data storage network function. The stored information may be re-exposed by the NEF (192) to other NFs and AFs (e.g., AF (196)) and used for other purposes, such as analysis.

[0047] NRF (193) can support service discovery functionality. NRF (193) can receive NF discovery requests from NF instances and provide information about discovered NF instances to the NF instances. In addition, NRF (193) maintains available NF instances and the services they support. NF discovery and selection can be performed independently by a specific NF or with reference to NRF (193).

[0048] NSSAAF(194) can support authentication and authorization functions per network slice.

[0049] AUSF (195) stores data for authentication of UE (e.g. terminal (110)).

[0050] The AF (196) can interact with the 3GPP core network to provide services (e.g., support functions such as application influence on traffic routing, network capability exposure access, and interaction with the policy framework for policy control).

[0051] The SCP (197) can perform functions such as indirect communication, delegated discovery, message forwarding and routing to target NF / NF services, message forwarding and routing to next hop SCPs, communication security (e.g., authorizing NF service consumers to access NF service producer APIs), load balancing, monitoring, and overload control. The SCP (197) can be deployed in a distributed manner. For example, there can be two or more SCPs in a communication path between NF services. Messages can be routed through SCPs. For example, to enable the message routing (i.e., next SCP hop discovery), the SCP (197) can register a profile with the NRF (e.g., the NRF (193)). For another example, the SCP (197) can use a local configuration. Some or all of the functions of the SCP (197) can be supported within a single instance of an AMF.

[0052] NSACF (198) can monitor and control the number of registered UEs per network slice for a network slice to which network slice admission control is applied.

[0053] In the 3GPP system, the conceptual links connecting NFs within a 5G system are defined as reference points or interfaces. The following illustrates reference points included in the 5G system architecture depicted in Figure 1.

[0054] - N1: Reference point or interface between terminal (110) and AMF (130)

[0055] - N2: Reference point or interface between base station (120) and AMF (130)

[0056] - N3: Reference point or interface between base station (120) and UPF (150)

[0057] - N4: Reference point or interface between SMF (140) and UPF (150)

[0058] - N5: Reference point or interface between PCF (170) and AF (196)

[0059] - N6: Reference point or interface between UPF (150) and DN (155)

[0060] - N7: Reference point or interface between SMF (140) and PCF (170)

[0061] - N8: Reference point or interface between UDM (180) and AMF (130)

[0062] - N9: Reference point or interface between two core UPFs (e.g. UPF(150))

[0063] - N10: Reference point or interface between UDM (180) and SMF (140)

[0064] - N11: Reference point or interface between AMF (130) and SMF (140)

[0065] - N12: Reference point or interface between AMF (130) and AUSF (195)

[0066] - N13: Reference point or interface between UDM (180) and AUSF (195)

[0067] - N14: Reference point or interface between two AMFs (e.g. AMF(130))

[0068] - N15: For non-roaming scenarios, a reference point between the PCF (170) and the AMF (130), and for roaming scenarios, a reference point or interface between the PCF (170) and the AMF (130) within the visited network.

[0069] Figure 2a shows an example of NF (network function) selection.

[0070] Referring to FIG. 2A, NF selection refers to identifying an NF among multiple NFs. NF selection for a service in a core network (e.g., core network (133), EPC, 5GC)) can be used for load balancing and fault management. The types of the multiple NFs are one of the NFs exemplified through FIG. 1B (e.g., AMF (130), SMF (140), UPF (150), PCF (170), NEF (192), etc.). The management device (201) can perform NF selection according to embodiments of the present disclosure. For example, the management device (201) can select an NF among multiple NFs (e.g., NF (230-1), NF (230-2), ..., NF (230-N)). The management device (201) can activate a call connection using the NF or use services provided by the NF (hereinafter, NF services). The NF can expose one or more NF services. An NF service represents one type of capability exposed by an NF (e.g., an NF service producer) to another NF (e.g., an NF service consumer) through a service-based interface (SBI). Hereinafter, in the present disclosure, the management device (201) is understood as an NF consumer in terms of being an entity that performs NF selection, and the NF selected by the management device (201) can be understood as an NF producer.

[0071] The management device (201) may be a different NF than the NF being selected or a network entity associated with a specific NF. In one embodiment, the management device (201) may be an entity operating as the SMF (140) or a separate device connected to the SMF (140). For example, the NF (230) may be a UPF (150). The management device (201) may select a specific UPF from among a plurality of UPFs as or for the SMF (140). For example, the NF (230) may be a PCF (170). The management device (201) may select a specific PCF from among a plurality of PCFs as or for the SMF (140). For example, the NF (230) may be a CHF. The management device (201) can select a specific CHF from among a plurality of CHFs as or for the SMF (140).

[0072] In one embodiment, the management device (201) may be an entity operating as the AMF (130) or a separate device connected to the AMF (130). For example, the NF (230) may be an SMF (140). The management device (201) may select a specific SMF from among a plurality of SMFs as or for the AMF (130). For example, the NF (230) may be a PCF (170). The management device (201) may select a specific PCF from among a plurality of PCFs as or for the AMF (130). For example, the NF (230) may be a CHF (charging function). The management device (201) may select a specific CHF from among a plurality of CHFs as or for the AMF (130).

[0073] In one embodiment, the management device (201) may be an entity that operates as the PCF (170) or a separate device connected to the PCF (170). For example, the NF (230) may be a CHF. The management device (201) may select a specific CHF from among a plurality of CHFs as or for the PCF (170).

[0074] In one embodiment, the management device (201) may be a separate device connected to the base station (120). For example, the NF (230) may be an AMF (130). The management device (201) may select a specific AMF from among a plurality of AMFs for the base station (120).

[0075] According to one embodiment, the management device (201) may select a group based on a data network name (DNN), single network slice selection assistance information (S-NSSAI), and / or location information. The management device (201) may distribute NFs within the selected group in a round-robin manner at a fixed ratio according to set capabilities.

[0076] Figure 2b shows an example of a group for NFs.

[0077] Referring to FIG. 2B, NFs can form groups. For example, multiple PCFs can form a group (250). Multiple UDMs can form a group (260). Multiple CHFs can form a group (270). For example, each NF within each group (e.g., multiple PCFs, multiple UDMs, or multiple CHFs) can serve as a backup NF for another NF. For example, if a connection with a specific NF is lost, an NF consumer can continue to receive service through another NF within the group.

[0078] As technology advances, the types of NFs within a core network (e.g., core network (133)) are increasing in number and becoming smaller, making network management through NF selection more complex. Previously, backup NFs were selected only in extreme cases where connectivity with a specific NF was lost. Therefore, it was difficult to exclude NFs with degraded quality from selection while maintaining connectivity or to take preemptive action before communication became unavailable. Manual control by an operator is required to exclude problematic NFs or adjust the selection ratio. However, monitoring and manually controlling multiple NFs is realistically difficult. Therefore, technologies for automatic fault control by utilizing network data collected from each NF are required. To address the above-described issues, the present disclosure will describe technical features for monitoring all NFs and identifying at least one NF in which an error has occurred using a fault detection model constructed based on one or more models.

[0079] According to one embodiment, the management device (201) can monitor a group of PCFs (250). The management device (201) can monitor a group of UDMs (260). The management device (201) can monitor a group of CHFs (270). For example, the management device (201) can detect and control anomalies according to network conditions by using network data collected from each NF.

[0080] For example, the management device (201) can identify that an abnormality has occurred in PCF #4 within the group (250) and that an abnormality has occurred in UDM #2 within the group (260). The management device (201) can control a plurality of PCFs within the group (250) to perform the operations that were performed in PCF #4 in at least one of the other PCFs within the group (250). The management device (201) can control a plurality of UDMs within the group (260) to perform the operations that were performed in UDM #2 in at least one of the other UDMs within the group (260).

[0081] In this disclosure, technical features for a management device (201) to monitor all NFs, identify anomalies in NFs through AI / ML, and perform actions according to the anomalies in NFs will be described.

[0082] Fig. 3 shows examples of components of a management device for NF selection. The management device (201) of Figs. 2a and 2b may be communication equipment operating as another NF connected to the NF (230) that is the selection target, a separate device connected to the other NF, and / or a separate device connected to a base station (e.g., base station (120)). Terms such as “... unit” and “... device” used hereinafter mean a unit that processes at least one function or operation, and this may be implemented by hardware, software, or a combination of hardware and software.

[0083] Referring to FIG. 3, the management device (201) may include a transceiver (310), a memory (320), and a processor (330).

[0084] The transceiver (310) provides an interface for communicating with other devices within the network. That is, the transceiver (310) converts a bit string transmitted from the management device (201) to another electronic device into a physical signal, and converts a physical signal received from another electronic device into a bit string. That is, the transceiver (310) can transmit or receive a signal. Accordingly, the transceiver (310) may be referred to as a modem, a communication unit, a transmit unit, a receive unit, or a transmit / receive unit. In this case, the transceiver (310) enables the management device (201) to communicate with other electronic devices or systems via a backhaul connection (e.g., a wired backhaul or a wireless backhaul) or via a network. The transceiver (310) may include one or more transceivers.

[0085] The transceiver (310) may perform functions for transmitting and receiving signals in a wired communication environment. The transceiver (310) may include a wired interface for controlling direct connection between devices via a transmission medium (e.g., copper wire, optical fiber). For example, the transceiver (310) may support an N2 interface. The transceiver (310) may transmit electrical signals to a node of a base station (e.g., gNB, gNB-CU, gNB-CU-CP) via copper wire, or may perform conversion between electrical signals and optical signals. The management device (210) may be connected to one or more base stations (e.g., base station (120)) via the transceiver (310). In addition, the transceiver (310) may also perform functions for transmitting and receiving signals in a wireless communication environment. For example, the transceiver (310) may support an N1 interface. The management device (201) can support NAS signaling through the transceiver (310). The management device (201) can transmit NAS messages to the terminal (110) through the transceiver (310).

[0086] The memory (320) stores data such as basic programs, application programs, and setting information for the operation of the management device (201). The memory (320) may be composed of volatile memory, non-volatile memory, or a combination of volatile memory and non-volatile memory. In addition, the memory (320) provides stored data upon request from the processor (330). The memory (320) may be referred to as a storage unit.

[0087] The processor (330) controls the overall operations of the management device (201). For example, the processor (330) transmits and receives signals via the transceiver (310). Additionally, the processor (330) writes and reads data to and from the memory (320). The processor (330) may be referred to as a control unit. For this purpose, the processor (330) may be composed of multiple processors or may include at least one sub-processor. According to various embodiments, the processor (330) may control the management device (201) to perform operations according to various embodiments described in the present disclosure.

[0088] In the present disclosure, NF is a logical element for supporting a specific network service and can be implemented as software as well as separate hardware. For example, when transmitting or receiving a message with NF, the management device (201) may call and execute a command corresponding to the message transmission, rather than directly transmitting the message to a physical entity. The management device (201) may be implemented as software including one or more commands. For example, the management device (201) may call and execute at least one command among one or more commands stored from a storage medium. This enables the device to operate to perform at least one function according to the at least one command called. The one or more commands may include code generated by a compiler or code executable by an interpreter. For network virtualization, the operations of the management device (201) according to embodiments may be implemented in the form of a storage medium (e.g., memory).

[0089] Fig. 4a illustrates an example of signaling of a management device and an NF for obtaining connection status information. NF (230) exemplifies one NF among NF (230-1), NF (230-2), ..., NF (230-N) of Fig. 2a. Hereinafter, the operations of NF (230) can be understood as operations of a device configured to perform NF (230).

[0090] Referring to FIG. 4A, in operation 401, the management device (201) may transmit a request message to the NF (230). The request message may be used to request the NF (230) to provide an NF service. The request message may be used to confirm the connection status of the NF (230). The connection status may indicate a communication status related to a call connection using the NF (230).

[0091] In operation 403, the management device (201) may receive a response message from the NF (230). The NF (230) may generate the response message in response to a request from the management device (201), which is a service consumer. The NF (230) may transmit the response message to the management device (201) within a specified time (e.g., the length of a timer) from the time at which the request message is transmitted. Feedback for the request message may be divided into three types. The feedback may be acceptance of the request message, rejection of the request message, or no response. If the management device (201) does not receive any response to the request message from the NF (230) within the specified time, the management device (201) may determine that the NF (230) is in a no-response state.

[0092] Although signaling through direct communication between the management device (201) and the NF (230) is illustrated in FIG. 4A, embodiments of the present disclosure are not limited thereto. For example, the management device (201) and the NF (230) may perform indirect communication using an SCP (e.g., SCP (197)). In the indirect communication, an NF service consumer may communicate with an NF service producer through the SCP (197). The NF service consumer may directly perform discovery of a target NF service producer, or delegate discovery of the target NF service producer to the SCP (197). For example, the management device (201) may transmit a request message to the NF (230) through the SCP (197). The management device (201) may transmit a response message (e.g., an acceptance message, a rejection message) from the NF (230) to the NF (230) through the SCP (197).

[0093] Figure 4b illustrates an example of data collected from a management device.

[0094] Referring to FIG. 4b, the management device (201) can identify various types of quality attribute data to evaluate the quality of NF (230). For example, the management device (201) can obtain connection failure data (410), timeout data (420), round trip time (RTT) data (430), and signaling data (440).

[0095] According to one embodiment, the management device (201) may obtain connection failure data (410). For example, if a rejection message is received in response to a request message (e.g., the request message of FIG. 4A), the management device (201) may determine that the request message has failed. The management device (201) may obtain the connection failure data (410) based on the frequency of reception of rejection messages. For example, the management device (201) may periodically transmit a request message to the NF (230) for a specified period of time. The management device (201) may count the number of rejection messages received from the NF (230). The management device (201) may record the number of rejection messages or record the ratio of the number of rejection messages to the number of request messages. As another example, the management device (201) may transmit the request messages to the NF (230) a specified number of times. The management device (201) can count the number of rejection messages received from the NF (230). The management device (201) can record the number of rejection messages or the ratio of the number of rejection messages to the specified number of times. The higher the number of rejection messages, the more likely the connection failure data (410) may indicate that the connection status of the NF (230) (e.g., a call connection using the NF (230)) is unstable.

[0096] In one embodiment, the management device (201) may apply weights to derived data (e.g., number of rejections, ratio of rejection messages to request messages) when determining the degree of connection status. For example, a rejection message may include cause information to inform the management device (201) of the cause of the rejection. The management device (201) may collect cause information of the received rejection message. The management device (201) may determine a weight based on the cause information of the rejection message and apply the determined weight to the connection failure data (410). For example, the management device (201) may determine a weight value corresponding to the type of the cause information. The management device (201) may determine a weight value within a specified range (e.g., 0 to 5) depending on the type of the cause information. For example, if the type of the cause information is related to call processing (e.g., overload), the weight value to be applied may be greater than the weight value to be applied if the type of the cause information is not related to call processing (e.g., identification error, authentication error). For example, if the type of the cause information is related to the communication quality of the NF (230), the weight value to be applied may be greater than the weight value to be applied if the type of the cause information is not related to the communication quality of the NF (230).

[0097] According to one embodiment, the management device (201) may obtain timeout data (420). For example, if a response message is not received for a specified time (e.g., the length of a timer) in response to a request message, the management device (201) may determine that the NF (230) is unresponsive. The management device (201) may obtain the timeout data (420) based on the frequency with which unresponsiveness occurs, i.e., the frequency with which timeouts occur. For example, the management device (201) may periodically transmit a request message to the NF (230) for a specified time. The management device (201) may count the number of instances in which a response does not arrive from the NF (230) within a specified time period. The management device (201) may record the number of instances or the ratio of unresponsiveness to the number of request messages. For another example, the management device (201) may transmit request messages to the NF (230) a specified number of times. The management device (201) may count the number of rejection messages received from the NF (230). The management device (201) may count the number of instances in which a response does not arrive from the NF (230). The management device (201) may record the number of instances or record the ratio of no response to the number of request messages. A higher number of no response times may indicate that the timeout data indicates that the connection state of the NF (230) (e.g., a call connection using the NF (230)) is unstable.

[0098] In one embodiment, the management device (201) may apply weights to derived data (e.g., number of non-responses, non-response rate) when determining the severity of a connection condition. Even in the case of a timeout, weights may be applied, similar to the cause information of the rejection message described above. Since a timeout may indicate a more critical connection condition than the weight, the management device (201) may apply a higher weight to the timeout data (420) than to the connection failure data (410).

[0099] According to one embodiment, the management device (201) can obtain round trip time (RTT) data (430). For example, the management device (201) can measure the time taken from the time of transmitting a request message to the time of receiving a response message corresponding to the request message. For example, the response message may be an acceptance message. Measurement of the RTT may be performed only if the request is accepted. For another example, the response message may include an acceptance message and a rejection message. Measurement of the RTT may be performed even if the request is not accepted. The management device (201) can obtain the RTT data whenever it is linked with the NF. For example, the management device (201) can periodically transmit a request message to the NF (230) for a specified period of time. The management device (201) can measure the RTT based on the response message received from the NF (230) and record data including the measured RTT. For example, the management device (201) can transmit request messages to the NF (230) based on a specified number of times. The management device (201) can measure the RTT based on a response message received from the NF (230) and record data including the measured RTT.

[0100] According to one embodiment, the management device (201) may obtain signaling data (440). For example, the management device (201) may identify the number of one or more messages exchanged with the NF (230). For example, the management device (201) may identify the number of one or more messages transmitted and received between the management device (201) and the NF (230) during a specified time period. For example, the management device (201) may distinguish one or more messages exchanged with the NF (230) according to a service. For example, the management device (201) may distinguish one or more messages exchanged with the NF (230) according to a message type.

[0101] Hereinafter, technical features for identifying an abnormality of NF (230) by using at least one of connection failure data (410), timeout data (420), RTT data (430), and / or signaling data (440) by the management device (201) will be described.

[0102] Figure 5a illustrates an example of a technique for identifying anomalies in NF.

[0103] Figure 5b illustrates an example of a technique for identifying anomalies in NF.

[0104] Referring to FIGS. 5A and 5B , the management device (201) can identify an anomaly in the NF (230) based on statistical data regarding the NF (230). For example, the management device (201) can transform data regarding the NF (230) (e.g., connection failure data (410), timeout data (420), RTT data (430), and / or signaling data (440)) based on an interquartile range (IQR), a Z-Score, or a multivariate Gaussian distribution. For example, the management device (201) can set a threshold value based on the data.

[0105] According to one embodiment, the management device (201) can identify an anomaly in the NF (230) based on at least one of a univariate search method for analyzing one type of data (or factor) one by one and a multivariate search method for analyzing various types of data at once. An example of the operation of the management device (201) for identifying an anomaly in the NF (230) according to the univariate search method will be described in FIG. 5A. An example of the operation of the management device (201) for identifying an anomaly in the NF (230) according to the multivariate search method will be described in FIG. 5B.

[0106] Referring to FIG. 5A, the management device (201) can identify an anomaly in the NF (230) based on each of a plurality of types of data. For example, the management device (201) can identify an anomaly in connection failure data (410). The management device (201) can identify an anomaly in timeout data (420). The management device (201) can identify an anomaly in RTT data (430). The management device (201) can identify an anomaly in signaling data (440). The management device (201) can identify that an anomaly in the NF (230) has occurred based on the occurrence of an anomaly in one or more types of data.

[0107] According to one embodiment, the management device (201) can identify an IQR (501) for a data set according to one data type. The management device (201) can identify the IQR (501) as a range corresponding to 50% of the data.

[0108] The management device (201) can sort the values ​​included in the data set according to a specified criterion (e.g., ascending order, descending order). The management device (201) can divide the values ​​included in the data set into four equal parts and identify a lower quartile (Q1), a middle quartile (Q2), and an upper quartile (Q3) for the values ​​included in the data.

[0109] For example, the management device (201) can identify the IQR (501) based on the lower quartile (Q1) and the upper quartile (Q3). The IQR (501) can be identified based on the difference between the upper quartile (Q3) and the lower quartile (Q1).

[0110] For example, the management device (201) can identify a lower bound and an upper bound. The lower bound can be identified based on mathematical expression 1. The upper bound can be identified based on mathematical expression 2.

[0111]

[0112]

[0113] Referring to Equations 1 and 2, LB represents the lower bound. Q1 represents the lower quartile. Q3 represents the upper quartile. IRQ represents the IQR (501). α is a value for setting the bound. For example, α may be set to 1.5. Depending on the embodiment, α may be changed depending on the data type. FIG. 5A illustrates an example in which α is set to 1.5, but is not limited thereto.

[0114] According to one embodiment, the management device (201) can identify a normal range (502) based on mathematical expressions 1 and 2. The management device (201) can obtain data (or data values) regarding the NF (230) (e.g., connection failure data (410), timeout data (420), RTT data (430), and / or signaling data (440)). The management device (201) can identify that an abnormality has occurred in the NF (230) based on identifying that the data (or data values) regarding the NF (230) (e.g., connection failure data (410), timeout data (420), RTT data (430), and / or signaling data (440)) is outside the normal range (502).

[0115] According to one embodiment, the management device (201) can identify a Z-Score to identify the extent to which the acquired data deviates from the average. The Z-Score can be identified according to mathematical equation 3.

[0116]

[0117] Referring to Equation 3, Z represents the Z-score. X represents the data value. μ represents the mean of the data set. σ represents the standard deviation of the data set.

[0118] For example, the management device (201) can identify an anomaly in the NF (230) based on identifying that the Z-Score is outside the threshold value.

[0119] Referring to FIG. 5B, the management device (201) can identify anomalies for multiple types of data. For example, the management device (201) can identify probabilities for a first type of data set and a second type of data set. A graph (511) represents probabilities for the first type of data set and the second type of data set. A graph (512) corresponding to the x-axis represents the probability for the first type of data set. A graph (513) corresponding to the y-axis represents the probability for the second type of data set.

[0120] The graph (511) can be identified based on mathematical expression 4.

[0121]

[0122] Referring to mathematical formula 4, is expressed as a graph (511). μ is the mean vector. is the covariance matrix. can be configured as in mathematical formula 5.

[0123]

[0124] Referring to mathematical formula 5, is the variance for the data set about the x-axis. is the variance of the data set along the y-axis. is the covariance.

[0125] For example, the management device (201) Based on identifying that the threshold value is exceeded, an anomaly of NF (230) can be identified.

[0126] Figure 5c illustrates an example of a technique for identifying anomalies in NF.

[0127] Figure 5d illustrates an example of a technique for identifying anomalies in NF.

[0128] Referring to FIGS. 5C and 5D , the management device (201) can identify an anomaly in the NF (230) based on machine learning. For example, the management device (201) can identify an anomaly in the NF (230) by performing clustering on a data set (e.g., connection failure data (410), timeout data (420), RTT data (430), and / or signaling data (440)) related to the NF (230). For example, the management device (201) can identify an anomaly in the NF (230) by using a local outlier factor (LOF).

[0129] A technique for identifying anomalies in NF (230) based on clustering will be described later in Fig. 5c. A technique for identifying anomalies in NF (230) based on LOF (local outlier factor) will be described later in Fig. 5d.

[0130] Referring to FIG. 5c, the management device (201) can perform clustering on a data set (e.g., connection failure data (410), timeout data (420), RTT data (430), and / or signaling data (440)) regarding NF (230). The management device (201) can form at least one cluster based on a clustering algorithm (e.g., K-means, DBSCAN (density-based spatial clustering of applications with noise)). At least one variable (e.g., x-axis or y-axis of FIG. 5c) for forming a cluster can be changed depending on the type of data.

[0131] For example, the management device (201) may configure a first cluster (521) and a second cluster (522). The first cluster (521) may include first data (531). The second cluster (522) may include second data (532). The management device (201) may identify the center of the first cluster (521). The management device (201) may identify data exceeding a reference distance from the center of the first cluster (521) as abnormal data (533). The management device (201) may identify the center of the second cluster (522). The management device (201) may identify data exceeding a reference distance from the center of the second cluster (522) as abnormal data (533).

[0132] For example, the management device (201) can identify that an abnormality has occurred in the NF (230) based on the identification of abnormal data (533).

[0133] Referring to FIG. 5d, the management device (201) can perform scaling on a data set (e.g., connection failure data (410), timeout data (420), RTT data (430), and / or signaling data (440)) regarding NF (230) by performing preprocessing. The management device (201) can identify an LOF score for each data (or data point) based on an LOF algorithm. The management device (201) can identify data having an LOF score exceeding a reference value as an outlier. The management device (201) can change the sensitivity for identifying outliers based on changing the reference value.

[0134] For example, the management device (201) can identify an LOF score for each data (or data point) based on the LOF algorithm. At least one variable (e.g., the x-axis or the y-axis in FIG. 5D) for identifying the LOF score can be changed depending on the type of data. The management device (201) can identify that the LOF scores for the data (or data points) included in the region (541) are below a reference value. The management device (201) can identify that the LOF scores for the data (or data points) included in the region (542) are below a reference value. The management device (201) can identify that the LOF score for the data (543) exceeds the reference value. Based on identifying that the LOF score for the data (543) exceeds the reference value, the management device (201) can identify that an abnormality has occurred in the function of the NF (230) with respect to the data (543).

[0135] Figure 5e illustrates an example of a technique for identifying anomalies in NF.

[0136] Referring to FIG. 5e, the management device (201) can identify anomalies in data (e.g., connection failure data (410), timeout data (420), RTT data (430), and / or signaling data (440)) regarding NF (230) based on deep learning. For example, the management device (201) can identify anomalies in data regarding NF (230) using an auto encoder (550).

[0137] According to one embodiment, the management device (201) can train the autoencoder (550) using a data set regarding the NF (230). The autoencoder (550) can be trained so that the output data is configured identically (or similarly) to the input data. For example, the data set for training can be arranged in units of reference time intervals, including data regarding the NF (230) (e.g., connection failure data (410), timeout data (420), RTT data (430), and / or signaling data (440)). The data arranged in time order can be used as input for a learning model.

[0138] For example, an auto encoder (550) may include an encoder (551) and a decoder (553). Input data (x) may be input to the auto encoder (550). The input data (x) may be input to the encoder (551) of the auto encoder (550). A latent vector (z) may be obtained based on an output of the encoder (551). The dimension of the latent vector (z) may be smaller than the dimension of the input data (x). The latent vector (z) may be input to the decoder (553) of the auto encoder (550). Output data (x') may be obtained based on an output of the decoder (553). The dimension of the output data (x') may be the same as the dimension of the input data (x).

[0139] According to one embodiment, the management device (201) can identify that an abnormality has occurred in the input data (x) based on identifying that a value representing a difference between the input data (x) and the output data (x') is greater than or equal to a threshold value.

[0140] For example, the management device (201) can input data regarding NF (230) into the autoencoder (550). The data regarding NF (230) can be arranged and scaled in time series based on data preprocessing. The management device (201) can identify that the output data of the autoencoder (550) is different from the input data regarding NF (230). The management device (201) can identify that an abnormality has occurred in the data regarding NF (230) based on identifying that a value (or reconstruction error) representing a difference between the output data of the autoencoder (550) and the input data regarding NF (230) is greater than or equal to a threshold value.

[0141] Figure 6 illustrates an example of an ensemble model constructed based on one or more models.

[0142] Referring to FIG. 6, the management device (201) can identify anomalies in NF (230) using an ensemble model (600) configured based on one or more models. For example, the management device (201) can use the ensemble model (600) to increase reliability by reducing false positives.

[0143] According to one embodiment, the ensemble model (600) may include at least one of a first model (601), a second model (602), a third model (603), a fourth model (604), and / or a fifth model (605). For example, the first model (601) may be constructed based on a technique for identifying anomalies according to the univariate search method described in FIG. 5A. The second model (602) may be constructed based on a technique for identifying anomalies according to the multivariate search method described in FIG. 5B. The third model (603) may be constructed based on a technique for identifying anomalies according to clustering described in FIG. 5C. The fourth model (604) may be constructed based on a technique for identifying anomalies according to LOF described in FIG. 5D. The fifth model (605) may be constructed based on a technique for identifying anomalies according to an autoencoder described in FIG. 5E. The ensemble model (600) illustrated in FIG. 6 is exemplary, and may include other models in addition to the models illustrated in FIG. 6.

[0144] According to one embodiment, the smaller the values ​​according to the connection failure data (410), the timeout data (420), the RTT data (430), and / or the signaling data (440), the closer the NF (230) may be to an ideal state. Accordingly, a minimum value may be set for one or more models included in the ensemble model (600) (e.g., the first model (601), the second model (602), the third model (603), the fourth model (604), and / or the fifth model (605)). For example, the management device (201) may identify as abnormal only data whose Z-Score exceeds the first reference value (e.g., '3') and exceeds the upper bound according to the IQR. For example, the management device (201) may not identify as abnormal data whose Z-Score exceeds the first reference value (e.g., '3') and exceeds the upper bound according to the IQR.

[0145] According to one embodiment, the management device (201) can determine threshold values ​​for each of one or more models included in the ensemble model (600). For example, the management device (201) can set a plurality of threshold values ​​for the ensemble model (600) (or one of the one or more models included in the ensemble model (600). The management device (201) can set a first threshold value, a second threshold value, and a third threshold value. The first threshold value, the second threshold value, and the third threshold value can be set to identify the abnormality severity of the NF (230). For example, based on an output value of the ensemble model (600) (or one of the one or more models included in the ensemble model (600)) that is greater than or equal to a first threshold value and less than a second threshold value, the management device (201) can identify that a minor anomaly has occurred in the NF (230). Based on an output value of the ensemble model (600) (or one of the one or more models included in the ensemble model (600)) that is greater than or equal to a second threshold value and less than a third threshold value, the management device (201) can identify that a major anomaly has occurred in the NF (230). Based on the output value of the ensemble model (600) (or one of the models included in the ensemble model (600)) that is greater than or equal to the third threshold value, the management device (201) can identify that a critical anomaly has occurred in the NF (230). The management device (201) can resolve the anomaly that has occurred in the NF (230) by performing an operation according to the degree of the anomaly described above.

[0146] According to one embodiment, the first model (601) and the second model (602) may operate based on statistical data. The third model (603) and the fourth model (604) may operate based on machine learning. The fifth model (605) may operate based on deep learning. The resource for using (or driving, learning) the model may be the smallest for the model that operates based on statistical data. The resource for using (or driving, learning) the model may be the largest for the model that operates based on deep learning. The management device (201) may configure an ensemble model (600) with at least one of the first model (601) to the fifth model (605) depending on the size of the resources available to the management device (201).

[0147] According to one embodiment, the management device (201) may use some of one or more models of the ensemble model (600) depending on the type of data. For example, abnormalities due to connection failure and / or timeout may rarely occur when the NF (230) operates normally. In addition, abnormalities due to connection failure and / or timeout may occur intermittently. Therefore, the management device (201) may train the first model (601) to the fourth model (604) based on the connection failure data (410) and / or timeout data (420) in order to reduce the resources required for training. The management device (201) may not use the connection failure data (410) and / or timeout data (420) for training the fifth model (605) configured based on deep learning. The management device (201) can identify whether there is an abnormality in the NF (230) through the first model (601) to the fourth model (604) using the connection failure data (410) and / or timeout data (420).

[0148] The RTT data (430) and / or signaling data (440) can be identified even when the NF (230) is operating normally. In addition, a pattern (e.g., a pattern over time) can be identified for each of the RTT data (430) and / or signaling data (440). Accordingly, the management device (201) can convert the RTT data (430) and / or signaling data (440) into time series data and train the model (605) using the time series data. The management device (201) can identify whether the NF (230) is abnormal through the fifth model (605) using the RTT data (430) and / or signaling data (440).

[0149] As described above, the management device (201) can reduce resources for training the ensemble model (600) and improve the accuracy of the ensemble model (600) by configuring the ensemble model (600) based on the data type.

[0150] According to one embodiment, the management device (201) can identify whether the NF (230) is abnormal by using the ensemble model (600). For example, the management device (201) can identify whether the NF (230) is abnormal by using one or more models to minimize false alarms.

[0151] For example, the management device (201) can identify results on whether or not an abnormality occurs in the NF (230) based on one or more models included in the ensemble model (600). For example, if more than half of the models among one or more models included in the ensemble model (600) identify that an abnormality has occurred in the NF (230), the management device (201) can determine that an abnormality has occurred in the NF (230). For example, if more than half of the models among one or more models included in the ensemble model (600) identify that an abnormality has not occurred in the NF (230), the management device (201) can determine that an abnormality has not occurred in the NF (230).

[0152] According to one embodiment, the management device (201) can identify recall as an evaluation criterion for each model. The management device (201) can increase the recall and increase the sensitivity for detecting anomalies in the NF (230) by changing the minimum number of votes up.

[0153] In one embodiment, the management device (201) can identify evaluation criteria for each model. For example, the evaluation criteria for each model can include F1 Score, recall, and / or precision. The management device (201) can refrain from training and detection for a specific period of time for a model whose evaluation criteria are below a threshold. The management device (201) can utilize the model after a specific period of time or when the data type changes, as the performance of the model may change depending on the training data. In another embodiment, the management device (201) can permanently exclude from the ensemble model (600) a model whose use is restricted more than a specified number of times (or more than a specified period of time).

[0154] Figure 7a illustrates an example of the operation of the management device when an abnormality occurs in NF.

[0155] Figure 7b illustrates an example of the operation of the management device when an abnormality occurs in NF.

[0156] Referring to FIGS. 7A and 7B, an abnormality may occur in at least one NF among a plurality of NFs managed by the management device (201). For example, the management device (201) may be associated with the SMF (140). NFs (701, 702, 703) related to the management device (201) may be associated with the PCF (170). For example, the management device (201) may be associated with the SMF (140). NFs (701, 702, 703) related to the management device (201) may be associated with the UDM (180). For example, the management device (201) may be associated with the SMF (140). NFs (701, 702, 703) related to the management device (201) may be associated with the CHF. For example, the management device (201) may be associated with an AMF (130). NFs (701, 702, 703) of the management device (201) may be associated with an SMF (140). For example, the management device (201) may be associated with an AMF (130). NFs (701, 702, 703) of the management device (201) may be associated with a PCF (170). For example, the management device (201) may be associated with a base station (120) (or a RAN node). NFs (701, 702, 703) of the management device (201) may be associated with a PCF (170).

[0157] For example, the management device (201) may reduce the distribution of new calls to at least one NF in which an error has occurred. The management device (201) may allocate new calls that were scheduled to be allocated to at least one NF in which an error has occurred to the remaining NFs. The specific operation of the management device (201) for reducing the distribution of new calls to at least one NF in which an error has occurred will be described in FIG. 7A.

[0158] For example, the management device (201) may restrict (or reject) some of the new calls being processed in at least one NF where an abnormality has occurred. The specific operation of the management device (201) for restricting some of the new calls being processed in at least one NF where an abnormality has occurred will be described in FIG. 7b.

[0159] Referring to FIG. 7A, the management device (201) can manage NF (701), NF (702), and NF (703). NF (701), NF (702), and NF (703) can form one NF group. The management device (201) can perform an NF selection procedure. For example, the management device (201) can select an NF to be linked among the NFs included in the NF group based on the NF selection procedure. The management device (201) can identify an anomaly for NF (701) in the NF selection procedure. The management device (201) can reduce new call distribution to NF (701).

[0160] For example, the management device (201) can distribute new calls to NFs according to a specified ratio. The management device (201) can distribute new calls to NF (701) according to a first ratio. The management device (201) can distribute new calls to NF (702) according to a second ratio. The management device (201) can distribute new calls to NF (703) according to a third ratio. The specified ratios can be set based on the capacity of each of the NFs. The management device (201) can change the specified ratios based on identifying that an abnormality has occurred in the NF (701). For example, the management device (201) can decrease the first ratio for NF (701). The management device (201) can increase the second ratio for NF (702) and the third ratio for NF (703). The management device (201) can reduce the number of new calls allocated to the NF (701) according to the reduced ratio. The management device (201) can allocate the reduced number of new calls to another NF within the group.

[0161] For example, the management device (201) can distribute new calls to the NF (701) and the NF (702). The management device (201) can distribute new calls to the NF (701) and the NF (702) at the same rate. The management device (201) can identify that a major anomaly has occurred in the NF (701). The management device (201) can reduce the first rate for allocating new calls to the NF (701). The management device (201) can reduce the first rate for the NF (701) by 30%. The management device (201) can distribute new calls to the NF (701) and the NF (702) at a rate of "0.7:1". Therefore, the NF (702) that is operating normally can receive more new calls than the NF (701) that is experiencing the anomaly.

[0162] Referring to FIG. 7B, the management device (201) can manage NF (701), NF (702), and NF (703). The management device (201) can identify that a critical anomaly has occurred in NF (701). The management device (201) can reject some of the new calls assigned to NF (701) according to a specified percentage (e.g., 50%). For example, the specified percentage can be changed depending on the severity of the anomaly.

[0163] For example, a UE (710) may request a new call. The new call may be assigned to NF (701). The management device (201) may reject the new call requested by the UE (710). The UE (710) may request a new call again. However, if the UE (710) repeatedly requests a new call, the number of signaling times may increase. Therefore, the management device (201) may transmit a backoff timer to the UE (710). The UE (710) may not perform a new call request during the time period according to the received backoff timer. The management device (201) may configure the backoff timer through a random offset so as not to set the same backoff timer for all UEs whose new calls are rejected. For example, the management device (201) may reject the new calls requested by the first UE and the second UE. The management device (201) can transmit a first backoff timer (e.g., A+RO1) configured based on a fixed value (A) and a random offset value (RO1) to the first UE. The management device (201) can transmit a second backoff timer (e.g., A+RO2) configured based on a fixed value (A) and a random offset value (RO2) to the second UE. The first UE can request a new call again after the first time interval according to the first backoff timer has elapsed. The second UE can request a new call again after the second time interval according to the second backoff timer has elapsed.

[0164] Figure 8 illustrates an example of quality indicators for anomaly detection of a policy control function (PCF). The PCF selection may be performed by a management device (201) (e.g., SMF (140) or AMF (130)).

[0165] Referring to Fig. 8, a graph (800) represents a quality index for each PCF. The management device (201) may be an NF operating as an AMF (130) or an SMF (140). The management device (201) may be connected to two PCFs. For example, the management device (201) may be connected to a first PCF (871) and a second PCF (872).

[0166] For example, the management device (201) can obtain a first quality indicator (881a) for the first PCF (871) (e.g., probability of a rejection message for a request message), a second quality indicator (882a) for the first PCF (871) (e.g., probability of a timeout for a request message), and a third quality indicator (883a) for the first PCF (871) (e.g., average RTT). The management device (201) can obtain a first quality indicator (881b) for the second PCF (872) (e.g., probability of a rejection message for a request message), a second quality indicator (882b) for the second PCF (872) (e.g., probability of a timeout for a request message), and a third quality indicator (883b) for the second PCF (872) (e.g., average RTT). The first quality indicator (881a, 881b) can be obtained based on connection failure data (410). The second quality indicator (882a, 882b) can be obtained based on timeout data (420). The third quality indicator (883a, 883b) can be obtained based on RTT data (430).

[0167] The horizontal axis of the graph (800) represents the quality indicator for each PCF, and the vertical axis of the graph (800) represents the relative ratio of the quality indicator for each threshold (e.g., the first threshold (850), the second threshold (840), and the third threshold (830)). The first threshold (850) can be used as a criterion for minor anomalies. The second threshold (840) can be used as a criterion for major anomalies. The third threshold (830) can be used as a criterion for major anomalies.

[0168] The management device (201) may determine that the first quality indicator (881a) exceeds the second threshold (840) and is less than or equal to the third threshold (830). For example, the management device (201) may determine that the abnormality severity associated with the first quality indicator (881a) is at the second level. The management device (201) may determine that the second quality indicator (882b) exceeds the first threshold (850) and is less than or equal to the second threshold (840). For example, the management device (201) may determine that the abnormality severity associated with the second quality indicator (882b) is at the first level. According to one embodiment, when multiple quality indicators exceeding the thresholds are detected, the management device (201) may determine an action method for the corresponding NF based on a high level.

[0169] According to one embodiment, the management device (201) can perform actions according to the second level corresponding to the first quality indicator (881a). For example, the management device (201) can change the distribution ratio for new calls according to the method described in FIG. 7A. The management device (201) can lower the items of the first PCF (871) by 30%. For example, the management device (201) can restrict (or reject) some of the new calls according to a specified ratio according to the method described in FIG. 7B. The management device (201) can determine the rejection ratio of the first PCF (871) to 30% for a new call connection request using the first PCF (371).

[0170] Although Figure 8 illustrates an example of identifying anomaly severity based on each individual quality indicator, the present invention is not limited thereto. If various types of data are used to train an anomaly detection model (e.g., an ensemble model (600)), a single anomaly severity level may be obtained.

[0171] Figure 9 shows the change in the number of occurrences of abnormalities according to abnormality detection of the management device.

[0172] Referring to FIG. 9, the graph (910) represents the number of times an abnormality occurs over time when the management device (201) is not operating. The graph (920) represents the number of times an abnormality occurs over time when the management device (201) detects an abnormality and takes action.

[0173] For example, the number of abnormal occurrences of NFs regarding the management device (201) may increase at point (900).

[0174] According to graph (910), when the management device (201) is not operating, the number of abnormal occurrences may increase from point (900) to point (902). At point (902), actions may be taken in response to the abnormal occurrence of NFs through manual control by the operator. From point (902), the number of abnormal occurrences may decrease.

[0175] According to graph (920), the management device (201) can perform anomaly detection and action. At point (901), the management device (201) can identify that anomalies have occurred in the NFs. The management device (201) can perform actions in response to the occurrence of anomalies in the NFs at point (901) earlier than point (902). Therefore, the number of occurrences of anomalies can be reduced from point (901).

[0176] Referring to graph (910) and graph (920), by identifying and taking action in advance for abnormal occurrences of NFs by the management device (201), service quality can be improved and operating costs for network management can be reduced.

[0177] Figure 10 illustrates an example of the operation of the management device. In the following embodiments, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0178] Referring to FIG. 10, in operation 1010, the management device (201) (or the processor (330) of the management device (201)) may obtain connection status information for a plurality of NFs. For example, the management device (201) may be associated with an SMF. The plurality of NFs may include PCFs. For example, the management device (201) may be associated with an SMF. The plurality of NFs may include CHFs. For example, the management device (201) may be associated with an SMF. The plurality of NFs may include UDMs. For example, the management device (201) may be associated with an AMF. The plurality of NFs may include SMFs. For example, the management device (201) may be associated with an AMF. The plurality of NFs may include PCFs.

[0179] According to one embodiment, the connection status information for the plurality of NFs may include at least one of connection failure data, timeout data, RTT data, or signaling data.

[0180] For example, the management device (201) can transmit a request message to each of the plurality of NFs. The management device (201) can obtain a response message from each of the plurality of NFs. For example, connection failure data can be obtained based on the frequency of reception of rejection messages received in response to transmission of request messages to the corresponding NF. For example, timeout data can be obtained based on the frequency of occurrence of timeouts for request messages. For example, RTT data can be obtained based on the round trip time (RTT) between the corresponding NF and the management device. Signaling data can be obtained based on the number of messages exchanged between the corresponding NF and the management device (201).

[0181] In operation 1010, the management device (201) may input at least a portion of the connection status information into a statistically based first model (e.g., the first model (601), the second model (602) of FIG. 6) included in a predictive model (e.g., the ensemble model (600) of FIG. 6) for anomaly detection for a plurality of NFs. For example, at least a portion of the connection status information may include at least one of connection failure data or timeout data.

[0182] According to one embodiment, the management device (201) may train a statistically based first model based on at least a portion of the connection status information. For example, abnormalities due to connection failures or timeouts may occur intermittently. Accordingly, the management device (201) may train a statistically based first model based on at least one of the connection failure data or timeout data.

[0183] In operation 1030, the management device (201) may input the remaining portion of the connection status information into an artificial intelligence-based second model (e.g., the fifth model (605) of FIG. 6) included in the prediction model. For example, the remaining portion of the connection status information may include at least one of RTT data or signaling data.

[0184] According to one embodiment, the management device (201) can train an AI-based second model based on the remaining portion of the connection status information. For example, the RTT data or signaling data may be structured according to a time series. Accordingly, the management device (201) can train an AI-based second model based on at least one of the RTT data or signaling data.

[0185] In operation 1040, the management device (201) can identify at least one NF in which an abnormality is detected among a plurality of NFs based on output data of the prediction model. For example, the management device (201) can obtain output data of the prediction model. The management device (201) can obtain output data of the prediction model based on first output data of a statistical-based first model and second output data of an artificial intelligence-based second model. For example, the output data of the prediction model can indicate at least one NF in which an abnormality is detected among a plurality of NFs.

[0186] In operation 1050, the management device (201) can coordinate a call connection to at least one NF.

[0187] For example, the management device (201) may adjust call connections for at least one NF based on the operations described in FIG. 7A. The management device (201) may decrease call connections assigned to at least one NF. The management device (201) may increase call connections assigned to the remaining NFs among the plurality of NFs.

[0188] For example, the management device (201) may coordinate a call connection to at least one NF based on the operations described in FIG. 7B. The management device (201) may reject some of the call connections requested from at least one terminal to at least one NF. The management device (201) may determine a waiting time for each of at least one terminal. The management device (201) may transmit information about the determined waiting time to each of at least one terminal. Each of at least one terminal may request a call connection after the determined waiting time has elapsed.

[0189] According to one embodiment, a management device may include at least one transceiver, a memory storing one or more instructions and including a storage medium, and at least one processor including a processing circuit. The one or more instructions, when individually or collectively executed by the at least one processor, may cause the management device to obtain connection state information for a plurality of network functions (NFs) connected to the management device, input at least a portion of the connection state information into a statistical-based first model included in a predictive model for anomaly detection for the plurality of NFs, input the remaining portion of the connection state information into an artificial intelligence-based second model included in the predictive model, identify at least one NF among the plurality of NFs in which an anomaly has been detected based on output data of the predictive model, and adjust a call connection for the at least one NF.

[0190] For example, at least a portion of the connection status information may include at least one of connection failure data or timeout data. The remaining portion of the connection status information may include at least one of round trip time (RTT) data or signaling data. The connection failure data may be obtained based on a reception frequency of rejection messages received in response to transmission of request messages to the corresponding NF. The timeout data may be obtained based on a frequency of occurrence of timeouts for the request messages. The RTT data may be obtained based on a round trip time (RTT) between the corresponding NF and the management device. The signaling data may be obtained based on a number of messages exchanged between the corresponding NF and the management device.

[0191] For example, the one or more instructions, when individually or collectively executed by the at least one processor, may cause the management device to train the statistical-based first model based on at least a portion of the connection state information, and to train the artificial intelligence-based second model based on the remaining portion of the connection state information.

[0192] For example, the output data of the prediction model can be obtained based on the first output data of the statistics-based first model and the second output data of the artificial intelligence-based second model.

[0193] For example, the one or more instructions, when individually or collectively executed by the at least one processor, may cause the management device to decrease call connections assigned to the at least one NF and increase call connections assigned to remaining NFs among the plurality of NFs.

[0194] For example, the one or more instructions, when individually or collectively executed by the at least one processor, may cause the management device to reject some of the call connections requested from at least one terminal to the at least one NF, determine a waiting time for each of the at least one terminal, and transmit information about the determined waiting time to each of the at least one terminal.

[0195] For example, the management device may be associated with a session management function (SMF). The plurality of NFs may include policy control functions (PCFs).

[0196] According to one embodiment, a method performed by a management device may include an operation of obtaining connection status information for a plurality of network functions (NFs) connected to the management device, an operation of inputting at least a portion of the connection status information into a statistical first model included in a prediction model for anomaly detection for the plurality of NFs, an operation of inputting the remaining portion of the connection status information into an artificial intelligence-based second model included in the prediction model, an operation of identifying at least one NF among the plurality of NFs in which an anomaly is detected based on output data of the prediction model, and an operation of adjusting a call connection for the at least one NF.

[0197] For example, at least a portion of the connection status information may include at least one of connection failure data or timeout data. The remaining portion of the connection status information may include at least one of round trip time (RTT) data or signaling data. The connection failure data may be obtained based on a reception frequency of rejection messages received in response to transmission of request messages to the corresponding NF. The timeout data may be obtained based on a frequency of occurrence of timeouts for the request messages. The RTT data may be obtained based on a round trip time (RTT) between the corresponding NF and the management device. The signaling data may be obtained based on a number of messages exchanged between the corresponding NF and the management device.

[0198] For example, the method may include an operation of training the statistical-based first model based on at least a portion of the connection state information, and an operation of training the artificial intelligence-based second model based on the remaining portion of the connection state information.

[0199] For example, the output data of the prediction model can be obtained based on the first output data of the statistics-based first model and the second output data of the artificial intelligence-based second model.

[0200] For example, the method may include an operation of decreasing call connections assigned to at least one NF, and an operation of increasing call connections assigned to remaining NFs among the plurality of NFs.

[0201] For example, the method may include the actions of rejecting some of the call connections requested from at least one terminal to the at least one NF, determining a waiting time for each of the at least one terminal, and transmitting information about the determined waiting time to each of the at least one terminal.

[0202] For example, the management device may be associated with a session management function (SMF). The plurality of NFs may include policy control functions (PCFs).

[0203] According to one embodiment, a non-transitory computer-readable storage medium may store one or more programs. The one or more programs may include instructions that, when executed by at least one processor of a management device, cause the management device to obtain connection status information for a plurality of network functions (NFs) connected to the management device, input at least some of the connection status information into a statistical-based first model included in a predictive model for anomaly detection for the plurality of NFs, input the remaining some of the connection status information into an artificial intelligence-based second model included in the predictive model, identify at least one NF among the plurality of NFs in which an anomaly has been detected based on output data of the predictive model, and adjust a call connection for the at least one NF.

[0204] For example, at least a portion of the connection status information may include at least one of connection failure data or timeout data. The remaining portion of the connection status information may include at least one of round trip time (RTT) data or signaling data. The connection failure data may be obtained based on a reception frequency of rejection messages received in response to transmission of request messages to the corresponding NF. The timeout data may be obtained based on a frequency of occurrence of timeouts for the request messages. The RTT data may be obtained based on a round trip time (RTT) between the corresponding NF and the management device. The signaling data may be obtained based on a number of messages exchanged between the corresponding NF and the management device.

[0205] For example, the one or more programs may include instructions that, when executed by the at least one processor, cause the management device to train the statistical-based first model based on at least a portion of the connection state information and to train the artificial intelligence-based second model based on the remaining portion of the connection state information.

[0206] For example, the output data of the prediction model can be obtained based on the first output data of the statistics-based first model and the second output data of the artificial intelligence-based second model.

[0207] For example, the one or more programs may include instructions that, when executed by the at least one processor, cause the management device to decrease call connections assigned to the at least one NF and increase call connections assigned to remaining NFs among the plurality of NFs.

[0208] For example, the one or more programs may include instructions that, when executed by the at least one processor, cause the management device to reject some of the call connections requested from at least one terminal to the at least one NF, determine a waiting time for each of the at least one terminal, and transmit information about the determined waiting time to each of the at least one terminal.

[0209] According to the above-described embodiments, the management device can improve service quality by avoiding NF interworking with degraded service quality. The management device can evaluate the quality indicators of NFs in real time without operator intervention, and if an anomaly is detected in at least some of the NFs, it can restrict new calls based on the severity of the anomaly as determined by the quality indicators, and recover from the anomaly.

[0210] According to the embodiments described above, the management device can measure the NF and RTT and identify the number of call connection request attempts for the NF. The management device can identify connection delays or overload. If overload occurs, the management device can restrict new call connections. The management device can send a backoff timer along with a rejection to the UE. The management device can prevent the overload situation from worsening by restricting the UE from performing services for a certain period of time. The management device can use an AI model to predict overload situations in advance and respond to the predicted overload situations in advance.

[0211] According to the above-described embodiments, since the rejection of a connection request, timeout, and / or RTT are quality attributes related to the connection delay or overload of the corresponding NF, the management device can detect and predict the expected connection delay and / or overload. By automatically detecting NFs predicted to have a fault and restricting connections to said NFs before an actual problem occurs in the connected NF, the fault situation can be easily resolved without manual operator intervention. In addition, as private networks increase and the core network structure becomes more sophisticated, automated network management can achieve improved accuracy and reduced operating costs.

[0212] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs from the description below.

[0213] For one or more embodiments, at least one of the components described in one or more of the preceding drawings may be configured to perform one or more operations, techniques, processes, and / or methods as described herein. For example, a processor (e.g., a baseband processor) described herein with respect to one or more of the preceding drawings may be configured to operate according to one or more examples described herein. For another example, circuitry associated with a user equipment (UE), a base station, a network element, and the like, as described above with respect to one or more of the preceding drawings, may be configured to operate according to one or more examples described herein.

[0214] Any of the embodiments described above may be combined with any other embodiment (or combination of embodiments) unless explicitly stated otherwise. The foregoing description of one or more implementations provides examples and descriptions, but is not intended to be exhaustive or limit the scope of the embodiments to the precise forms disclosed. Modifications and variations are possible in light of the above teachings or may be learned from practicing various embodiments.

[0215] The methods according to the embodiments described in the claims or specification of the present disclosure may be implemented in the form of hardware, software, or a combination of hardware and software.

[0216] When implemented in software, a computer-readable storage medium storing one or more programs (software modules) may be provided. The one or more programs stored in the computer-readable storage medium are configured to be executed by one or more processors in an electronic device. The one or more programs include instructions that cause the electronic device to execute methods according to embodiments described in the claims or specification of the present disclosure. The one or more programs may be provided as a computer program product. The computer program product may be traded between a seller and a buyer as a commodity. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0217] These programs (software modules, software) may be stored in random access memory, non-volatile memory including flash memory, read only memory (ROM), electrically erasable programmable read only memory (EEPROM), magnetic disc storage devices, compact disc-ROM (CD-ROM), digital versatile discs (DVDs) or other forms of optical storage devices, magnetic cassettes, or may be stored in memories formed by a combination of some or all of these. In addition, each configuration memory may include multiple copies.

[0218] Additionally, the program may be stored on an attachable storage device that is accessible via a communication network, such as the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a storage area network (SAN), or a combination thereof. Such a storage device may be connected to a device implementing an embodiment of the present disclosure via an external port. Additionally, a separate storage device on the communication network may be connected to a device implementing an embodiment of the present disclosure.

[0219] In the specific embodiments of the present disclosure described above, components included in the disclosure are expressed singularly or plurally, depending on the specific embodiment presented. However, the singular or plural expressions are selected to suit the presented situation for convenience of explanation, and the present disclosure is not limited to singular or plural components. Components expressed in plural may be composed of singular elements, or components expressed in singular may be composed of plural elements.

[0220] According to embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., modules or programs) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0221] Meanwhile, although the detailed description of the present disclosure has described specific embodiments, it is obvious that various modifications are possible within the scope of the present disclosure.

Claims

1. In the management device, At least one transmitter / receiver; A memory storing one or more instructions and including a storage medium; and At least one processor comprising a processing circuit, The one or more instructions, when individually or collectively executed by the at least one processor, Obtain connection status information for multiple NFs (network functions) connected to the above management device, Inputting at least some of the above connection status information into a statistical first model included in a prediction model for anomaly detection for the plurality of NFs, The remaining part of the above connection status information is input into the second artificial intelligence-based model included in the above prediction model, Based on the output data of the above prediction model, at least one NF among the plurality of NFs in which an abnormality is detected is identified, Causing the management device to adjust a call connection to at least one NF; Management device.

2. In the first paragraph, at least some of the connection status information, Contains at least one of connection failure data or timeout data, The remaining part of the above connection status information, Contains at least one of RTT (round trip time) data or signaling data, The above connection failure data is, Obtained based on the frequency of reception of rejection messages received in response to transmission of request messages to the corresponding NF, The above timeout data is, Obtained based on the frequency of occurrence of timeouts for the above request messages, The above RTT data is, It is obtained based on the RTT (round trip time) between the above NF and the management device, The above signaling data is, Obtained based on the number of messages exchanged between the above NF and the above management device, Management device.

3. In the second paragraph, when the one or more instructions are individually or collectively executed by the at least one processor, Training the statistical-based first model based on at least some of the above connection status information, Causing the management device to train the artificial intelligence-based second model based on the remaining part of the connection status information. Management device.

4. In the first paragraph, the output data of the prediction model is Obtained based on the first output data of the first model based on statistics and the second output data of the second model based on artificial intelligence, Management device.

5. In the first paragraph, when the one or more instructions are individually or collectively executed by the at least one processor, Reduce the number of call connections assigned to at least one NF, Causing the management device to increase the number of call connections allocated to the remaining NFs among the plurality of NFs; Management device.

6. In the first paragraph, when the one or more instructions are individually or collectively executed by the at least one processor, Reject some of the call connections requested from at least one terminal to said at least one NF, Determine a waiting time for each of the above at least one terminal, Causing the management device to transmit information about the determined waiting time to each of the at least one terminal; Management device.

7. In claim 1, the management device, Associated with SMF (session management function), The above multiple NFs are, Including PCF (policy control functions), Management device.

8. In a method performed by a management device, An operation of obtaining connection status information for a plurality of network functions (NFs) connected to the above management device; An action of inputting at least some of the above connection status information into a statistical-based first model included in a prediction model for anomaly detection for the plurality of NFs; An action of inputting the remaining part of the above connection status information into an artificial intelligence-based second model included in the above prediction model; An operation of identifying at least one NF among the plurality of NFs in which an abnormality is detected based on the output data of the above prediction model; and comprising an operation of adjusting a call connection to at least one NF; method.

9. In paragraph 8, at least some of the connection status information, Contains at least one of connection failure data or timeout data, The remaining part of the above connection status information, Contains at least one of RTT (round trip time) data or signaling data, The above connection failure data is, Obtained based on the frequency of reception of rejection messages received in response to transmission of request messages to the corresponding NF, The above timeout data is, Obtained based on the frequency of occurrence of timeouts for the above request messages, The above RTT data is, It is obtained based on the RTT (round trip time) between the above NF and the management device, The above signaling data is, Obtained based on the number of messages exchanged between the above NF and the above management device, method.

10. In the 9th paragraph, the method, An operation of training the statistically based first model based on at least a portion of the connection status information; and An operation of training the second artificial intelligence-based model based on the remaining part of the connection status information, method.

11. In the 8th paragraph, the output data of the prediction model is Obtained based on the first output data of the first model based on statistics and the second output data of the second model based on artificial intelligence, method.

12. In the 8th paragraph, the method, An operation of reducing the number of call connections assigned to at least one NF; and An operation for increasing the number of call connections allocated to the remaining NFs among the plurality of NFs, method.

13. In the 8th paragraph, the method, An action of rejecting some of the call connections requested from at least one terminal to said at least one NF; An operation for determining a waiting time for each of the at least one terminal; and Including an operation of transmitting information about a determined waiting time to each of the at least one terminal, method.

14. In the 8th paragraph, the management device, Associated with SMF (session management function), The above multiple NFs are, Including PCF (policy control functions), method.

15. In a non-transitory computer-readable storage medium storing one or more programs, the one or more programs, when executed by at least one processor of a management device, Obtain connection status information for multiple NFs (network functions) connected to the above management device, Inputting at least some of the above connection status information into a statistical first model included in a prediction model for anomaly detection for the plurality of NFs, The remaining part of the above connection status information is input into the second artificial intelligence-based model included in the above prediction model, Based on the output data of the above prediction model, at least one NF among the plurality of NFs in which an abnormality is detected is identified, Including instructions that cause the management device to adjust a call connection to at least one NF. Non-transitory computer-readable storage medium.

Images