Network pseudo-failure generation system, network pseudo-failure generation method and program

Abstract

To provide a network pseudo-fault generation system that can train a highly accurate AI / ML learning model regarding a network failure.SOLUTION: A network pseudo-fault generation system includes a failure generation unit that generates a pseudo failure of a node included in a network, an information database that stores information regarding the pseudo failure of the node, an AI / ML training unit that trains an AI / ML learning model on the basis of information stored in the information database, an AI / ML evaluation unit that evaluates the detection accuracy of the learning model, and a failure determination unit that determines a mode of the pseudo failure caused by the failure generation unit on the basis of the evaluation result of the AI / ML evaluation unit.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a network pseudo-failure generation system, a network pseudo-failure generation method, and a program. [Background technology]

[0002] In order to maintain high-quality communication in the network services provided by telecommunications carriers, network operations utilizing AI (Artificial Intelligence) / ML (Machine Learning) are expected to become more widespread. In particular, by utilizing AI / ML technology to detect network failures that occur daily on networks and identify the causes of those failures, high-quality communication quality can be maintained.

[0003] Among the networks, the latest mobile network, the 5G Mobile Core (5GC), adopts a service-based architecture and is a cloud-native architecture. In cloud-native architectures, a technique called chaos engineering is used to intentionally create simulated failures in commercial networks and verify whether the system is operating as intended.

[0004] Patent Document 1 describes a system for generating pseudo-faults in a computer system, which is necessary to realize chaos engineering (see Patent Document 1). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-090586 Summary of the Invention [Problem to be solved by the invention]

[0006] However, Patent Document 1 does not specify a method for using pseudo-faults to train learning models for AI / ML.

[0007] In addition, while there is a demand for the use of AI / ML technology to detect and identify the causes of network failures that occur on networks, failures in commercial networks are rare, so it is not possible to obtain sufficient statistical data for each piece of equipment in the network when a network failure occurs.As a result, it is not possible to train a highly accurate AI / ML learning model to detect and identify the causes of network failures. The statistical data includes, for example, the utilization rate of a CPU (Central Processing Unit) or the utilization rate of a memory.

[0008] The present disclosure has been made in consideration of the above circumstances, and aims to provide a network pseudo-fault generation system, a network pseudo-fault generation method, and a program that can train a highly accurate AI / ML learning model regarding network faults. [Means for solving the problem]

[0009] As an example configuration, the network includes a fault generation unit that generates pseudo-failures in nodes included in the network, an information database that stores information about the pseudo-failures in the nodes, an AI / ML training unit that trains an AI / ML learning model based on the information stored in the information database, an AI / ML evaluation unit that evaluates the detection accuracy of the learning model, and a fault determination unit that determines the nature of the pseudo-failures generated by the fault generation unit based on the evaluation results of the AI / ML evaluation unit. the node; Equipped with The node determines whether to generate the pseudo-failure as a real-time determination based on the type of the pseudo-failure to be generated. This is a network pseudo-fault generation system.

[0010] In one configuration example, the failure generating unit generates a pseudo-failure in a node included in the network, An information database stores information about the pseudo-fault of the node, an AI / ML training unit trains an AI / ML learning model based on the information stored in the information database, an AI / ML evaluation unit evaluates the detection accuracy of the learning model, and a fault determination unit determines the mode of the pseudo-fault to be generated by the fault generation unit based on the evaluation result of the AI / ML evaluation unit. The node determines whether to generate the pseudo-failure as a real-time determination based on the mode of the pseudo-failure to be generated. This is a method for generating a pseudo network failure.

[0011] As one configuration example, a program for realizing in a computer a failure generation function that generates a pseudo-failure in a node included in a network, and a failure judgment function that judges the nature of the pseudo-failure generated by the failure generation function based on an evaluation result of an AI / ML evaluation unit, wherein an information database stores information about the pseudo-failure of the node, an AI / ML training unit trains an AI / ML learning model based on the information stored in the information database, and the AI / ML evaluation unit evaluates the detection accuracy of the learning model. death , The node determines whether to generate the pseudo-failure as a real-time determination based on the type of the pseudo-failure to be generated, and the program further realizes in the computer a function of requesting the node to execute the pseudo-failure and a function of receiving a report of the execution result of the pseudo-failure from the node. It is a program. [Effects of the Invention]

[0012] The network pseudo-failure generation system, network pseudo-failure generation method, and program according to the present disclosure enable training of a highly accurate AI / ML learning model regarding network failures. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating a schematic configuration of a communication system including a network pseudo-failure occurrence system according to an embodiment. [Figure 2] 1 is a diagram illustrating a configuration of functional blocks of a communication system including a network pseudo-failure occurrence system according to an embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of a pseudo-failure target node list according to the embodiment. [Figure 4]FIG. 10 is a diagram illustrating an example of a failure information table according to the embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of the flow of a pseudo-failure process in the communication system according to the embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of a procedure for selecting a network node to be a pseudo-failure target according to the embodiment. [Figure 7] FIG. 10 is a diagram illustrating an example of a procedure for a pseudo-fault execution process according to the embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a procedure of a learning-related process according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In this embodiment, the term AI / ML is used for explanation, but AI / ML may also be referred to as machine learning or the like.

[0015] [Communication Systems] FIG. 1 is a diagram showing a schematic configuration of a communication system 1 including a network pseudo-fault occurrence system A1 according to an embodiment. The communication system 1 includes a network management node (NW management node) 11, a pseudo-fault management node 12, an AI / ML management node 13, an information database 14, an AMF (Access & Mobility Management Function) 21, an SMF (Session Management Function) 22, an NF (Network Function) 23, a UPF (User Plane Function) 24, UEs (User Equipment) 31 to 33, a RAN (Radio Access Network) 41 including a base station 51, and a data network 61.

[0016] The example in Figure 1 shows a schematic diagram of a procedure P1 in which the pseudo-fault management node 12 intentionally generates pseudo-faults Q1 to Q4 on the network, a procedure P2 in which the information database 14 collects data on the pseudo-faults, and a procedure P3 in which the AI / ML management node 13 trains a highly accurate learning model.

[0017] In this embodiment, the network pseudo-fault occurrence system A1 includes a NW management node 11, a pseudo-fault management node 12, an AI / ML management node 13, and an information database .

[0018] FIG. 2 is a diagram showing a configuration of functional blocks of a communication system 1 including a network pseudo-fault occurrence system A1 according to the embodiment. FIG. 2 shows a configuration of functional blocks in the communication system 1 shown in FIG. 2 shows a schematic representation of part of the configuration shown in FIG.

[0019] FIG. 2 shows a NW management node 11, a pseudo-fault management node 12, an AI / ML management node 13, an information database 14, and a managed network 211 including NW nodes 231 to 233. The NW management node 11 includes a node management unit 111 and an AI / ML management unit 112. The pseudo-fault management node 12 includes a fault determination unit 121 and a fault occurrence unit 122 . The AI / ML management node 13 includes an AI / ML training unit 131 and an AI / ML evaluation unit 132. The information database 14 stores statistical information 141 and fault information 142.

[0020] In the example of FIG. 2, components other than the NW management node 11, pseudo-fault management node 12, AI / ML management node 13, and information database 14 shown in FIG. 1 are shown as a managed network 211. Furthermore, the NW nodes 231 to 233 may be, for example, the UEs 31 to 33, the base station 51, or each instance shown in FIG.

[0021] Each node and each function will be described with reference to the example of FIG. The managed network 211 is a network that is constructed and operated by a certain telecommunications carrier for the purpose of providing communication services, and is a target network to which AI / ML-based network operation is applied. Here, specific examples of network operation using AI / ML include detecting network failures and identifying their causes. The type of network is not particularly limited, as long as it is possible to generate pseudo-failures in the NW nodes 231 to 233 that are the components, and statistical information can be acquired from the NW nodes 231 to 233 that are the components. Specifically, the network may be, for example, a mobile network consisting of base stations or mobile core equipment such as 5G, or an IP (Internet Protocol) network consisting of routers such as FTTH (Fiber To The Home).

[0022] The NW nodes 231 to 233 are elements that make up the network 211 to be managed. In this embodiment, one or more NW nodes 231 to 233 are physically and logically connected to form a managed network 211. The NW nodes 231 to 233 are not particularly limited and may be, for example, a server, a router, or a terminal.

[0023] The NW management node 11 is a node that manages the NW nodes 231 to 233 of the network 211 to be managed.

[0024] The node management unit 111 is a component of the NW management node 11, and is a node that manages information on the NW nodes 231 to 233 that exist within the network 211 to be managed. Information on NW nodes 231 to 233 may include, for example, the name of the node (node ​​name), the identification information of the node (node ​​ID), the IP address of the node, the name or version of the OS (Operating System) used, the name or version of the software used, setting configuration information, the current operating status, etc.

[0025] The AI / ML management unit 112 is a component of the NW management node 11, and manages the training status of the AI / ML learning model for each of the NW nodes 231 to 233 for network operation using AI / ML. The training state may include the state of the trained learning model, the version of the learning model, the detection accuracy of the learning model, and the like.

[0026] The pseudo-fault management node 12 is a node that manages the occurrence of pseudo-faults in the managed network 211 . In this embodiment, the pseudo failure refers to a network failure that is intentionally generated on the managed network 211 .

[0027] The fault determination unit 121 is a component of the pseudo fault management node 12, and determines detailed information of the pseudo fault to be generated. The detailed information of the pseudo-failure may include the network node that causes the pseudo-failure, the content of the pseudo-failure, and the time when the failure occurred. The pseudo-faults include, for example, a sudden increase in CPU usage or a blocked network port.

[0028] The failure generating unit 122 is a component of the pseudo failure management node 12, and causes the NW nodes 231 to 233 to execute a pseudo failure based on the detailed information of the pseudo failure determined by the failure determining unit 121.

[0029] The AI / ML management node 13 trains a learning model for AI / ML-based network operation based on statistical information of the NW nodes 231 to 233, and evaluates the accuracy of the learning model.

[0030] The AI / ML training unit 131 is a component of the AI / ML management node 13, and trains a learning model for network operation using AI / ML from various information stored in the information database 14.

[0031] The AI / ML evaluation unit 132 is a component of the AI / ML management node 13, and evaluates the accuracy of the learning model for network operation using AI / ML trained by the AI / ML training unit 131.

[0032] The information database 14 is a database that manages various types of information related to the managed network 211 . The information database 14 collects and stores (records) such information.

[0033] The statistical information 141 is statistical information relating to nodes transmitted from the NW nodes 231 to 233 of the management target network 211. The statistical information 141 may include information on the resource usage status of the NW nodes 231 to 233 and information on the services provided by the NW nodes 231 to 233. Here, the information about the resource usage status of the NW nodes 231 to 233 includes, for example, CPU usage rates, and the information about the services provided by the NW nodes 231 to 233 includes the number of received requests per second.

[0034] The fault information 142 is information relating to a network fault that has occurred in the managed network 211 . This information may include the name of the NW node where the network failure occurred (NW node name), the type of network failure, the start time of the network failure, and the end time of the network failure. In this embodiment, the network failure includes, for example, both a network failure caused by the failure causing unit 122 of the pseudo failure management node 12 and a naturally occurring network failure.

[0035] [Pseudo-failure target node list] FIG. 3 is a diagram showing an example of the pseudo-failure target node list 2011 according to the embodiment. The pseudo-failure target node list 2011 describes a list of nodes (network node names) that will ultimately be pseudo-failure targets. In the example of FIG. 3, the pseudo-failure target node list 2011 describes "NW node 1," "NW node 3," "NW node 4," and "NW node 5."

[0036] [Pseudo fault information table] FIG. 4 is a diagram showing an example of the failure information table 2021 according to the embodiment. The fault information table 2021 stores the network node name, the fault content of the simulated fault, the time of the fault occurrence, and the result in association with each other. In the example of Figure 4, the NW node name "NW node 1", the fault content "CPU spike", the fault occurrence time "2025 / 05 / 01 22:30:00", and the result "Yes" are associated with each other.

[0037] Other examples of the failure include "node restart," "process crash," and "network port blockage." The "result" can be "◯" which indicates that the pseudo-failure was generated, or "×" which indicates that the pseudo-failure was not generated.

[0038] Here, the accuracy of the AI / ML learning model in this embodiment will be described. In this embodiment, the accuracy of detection (detection accuracy) using an AI / ML learning model is evaluated, and the evaluation result (evaluation accuracy) is obtained.

[0039] Such precision may be determined in any unit, for example, in terms of nodes, time, items, etc. For example, the accuracy may be determined for each node (single node). For example, the accuracy may be calculated for each time period, such as "morning," "afternoon," or "evening," each of which has a predetermined start time and end time. For example, the accuracy may be determined for each type of failure (failure item), such as "CPU spike" or "node restart."

[0040] The accuracy may be calculated for two or more of the following units: node, time, item, etc. As an example, the accuracy may be calculated for a certain node, a certain time period, and a certain fault type (fault item). Furthermore, the accuracy may be determined in a unit where, for example, the node is determined but other elements (time, item, etc.) can be arbitrary. Furthermore, the accuracy may be determined in a unit where, for example, time is fixed but other elements (nodes, items, etc.) can be determined in any unit. Furthermore, the accuracy may be determined in units where, for example, the items are defined but other elements (nodes, time, etc.) can be arbitrary.

[0041] The fault information table 2021 may store, for example, the evaluation result of the detection accuracy (evaluation accuracy) for each NW node name. Such accuracy may be expressed, for example, using a percentage (%) or by other indicators.

[0042] [Pseudo-failure processing] FIG. 5 is a diagram showing an example of the flow of pseudo-failure processing in the communication system 1 according to the embodiment. Referring to FIG. 5, an example of a procedure for generating a pseudo-failure in the managed network 211 will be described. FIG. 5 shows a NW management node 11, a pseudo-fault management node 12, an information database 14, and a NW node 231. In the example of FIG. 5, the target in which a pseudo-failure is to be generated is the NW node 231, but the same applies to other targets.

[0043] (Process T1) The pseudo-fault management node 12 starts the process.

[0044] (Step 1: Process T31) The pseudo-failure management node 12 transmits a request to the NW management node 11 to acquire a list of pseudo-failure target nodes.

[0045] (Step 2: Process T11) When the NW management node 11 receives the acquisition request, it checks the list of pseudo-failure targets. Here, the NW management node 11 selects NW nodes to be simulated failure targets and generates a list of simulated failure target nodes. The pseudo-failure target node list 2011 shown in FIG. 3 is a specific example of the final pseudo-failure target node list.

[0046] (Step 3: Process T32) The NW management node 11 provides the generated pseudo-failure target node list to the pseudo-failure management node 12 .

[0047] (Step 4: Process T2) The pseudo-failure management node 12 determines detailed information about the pseudo-failure to be generated this time based on the pseudo-failure target node list received from the NW management node 11 . Here, the information other than the "result" in the fault information table 2021 shown in FIG. 4 is an example of detailed information about the pseudo fault. The detailed information about the pseudo-failure includes, for example, the network node that causes the pseudo-failure, the content of the pseudo-failure (for example, a sudden increase in CPU usage or a blocked network port), and the time when the failure occurred.

[0048] Regarding the determination of the type of failure, there are no particular limitations in this embodiment. For example, a method of randomly determining the type of failure from among the pseudo-failures that the target network node can currently execute, or a method of determining the most effective pseudo-failure based on the accuracy results of the currently trained learning model and executing that pseudo-failure may be used. Furthermore, although the present embodiment does not particularly limit the determination of the time when a pseudo-failure occurs, for example, a method of randomly determining the time for each NW node, a method of taking into account the user usage status of the NW node and determining the time when the communication impact on the user will be minimal as the time when the failure occurs, or a method of determining the most effective time when the failure occurs based on the accuracy results of the currently trained learning model may be used.

[0049] (Step 5: Process T33) The pseudo-fault management node 12 requests the NW node 231 that will cause a pseudo-fault to execute a pseudo-fault (generate a pseudo-fault) in accordance with the determined detailed information of the pseudo-fault. Here, the request content includes, for example, the type of failure to be caused and the time of the failure occurrence. As shown in FIG. 4, when a plurality of NW nodes are targeted, the pseudo failure management node 12 makes requests to each NW node in parallel.

[0050] (Step 6: Process T21) The NW node 231 performs processing to execute a pseudo-failure specified by the pseudo-failure management node 12 . Depending on the situation, the NW node 231 may not execute the pseudo-failure.

[0051] (Step 7: Process T34) The NW node 231 records its own statistical information in the information database 14. Here, in this procedure, for example, a method of recording all statistical information that has been recorded up to that point in the information database 14 after the occurrence of a failure, or a method of periodically recording one's own statistical information in the information database 14 may be used.

[0052] (Step 8: Process T35) The NW node 231 reports the execution result of the pseudo-fault to the pseudo-fault management node 12 . Here, the report contents include, for example, the execution result, and if a pseudo-failure was executed in step 6, the actual time when the pseudo-failure occurred. In this embodiment, the time of occurrence of a failure in the failure information table 2021 shown in FIG. 4 is a scheduled time.

[0053] (Step 9: Process T36) Step 9 is performed only when the NW node 231 is executing a pseudo-failure. In step 9, the pseudo-failure management node 12 records the pseudo-failure information in the information database 14. The pseudo-failure information includes, for example, the name of the NW node where the network failure occurred, the type of network failure, the start time of the network failure, and the end time of the network failure.

[0054] As described above, the pseudo-fault management node 12 can manage the occurrence of pseudo-faults in the managed network 211 in order to train a learning model for network operation using AI / ML. Here, the series of processes shown in Fig. 5 are repeatedly executed for each NW node, for example. As a specific example, the series of processes shown in Fig. 5 may be executed periodically at any timing, such as once per week or once per day.

[0055] [Selection of network node to simulate failure in step 2] 6 is a diagram illustrating an example of a procedure for selecting a network node to be a pseudo-failure target according to the embodiment. This procedure is an example of the procedure of process T11 shown in FIG. An example of a detailed selection algorithm is shown with reference to FIG.

[0056] (Step S1) The node management unit 111 of the NW management node 11 lists all registered NW nodes and selects NW nodes one by one from this list. Then, the process proceeds to step S2.

[0057] Here, in this embodiment, the processing of step S1 (selection of a NW node) is performed using a list including all registered NW nodes. However, as another example, the processing of step S1 (selection of a NW node) may be performed using a list including only a predetermined portion of NW nodes, rather than all of these NW nodes.

[0058] (Step S2) The node management unit 111 of the NW management node 11 checks (determines) whether the selected NW node is a target for pseudo-failure occurrence based on the current accuracy of the learning model.

[0059] Specifically, the node management unit 111 checks (determines) whether the evaluation accuracy of the trained AI / ML learning model for the target NW node is insufficient compared to a standard threshold (required accuracy). As a result, if the node management unit 111 determines that the current accuracy is insufficient (lower than the required accuracy) (step S2: YES), it determines that it is necessary to generate a pseudo-failure, and proceeds to the processing of step S3. On the other hand, if the node management unit 111 determines that the current accuracy is sufficient (higher than the required accuracy) (step S2: NO), it determines that it is not necessary to generate a pseudo-failure, and proceeds to the processing of step S4.

[0060] Here, the accuracy (evaluation accuracy) is, for example, accuracy for each predetermined unit such as node, time, or item. Furthermore, the threshold value of the standard regarding the evaluation accuracy is not particularly limited, and for example, an accuracy value such as 90 percent may be used.

[0061] (Step S3) The node management unit 111 of the NW management node 11 adds the corresponding NW node to the pseudo-failure target node list. Then, the process proceeds to step S4.

[0062] (Step S4) The node management unit 111 of the NW management node 11 determines whether or not to end the processing of this flow. As a result, if the node management unit 111 determines that the processing of this flow should be ended (step S4: YES), the node management unit 111 ends the processing of this flow. On the other hand, if the node management unit 111 determines not to end the processing of this flow (step S4: NO), the process proceeds to step S1 to select the next NW node. Here, if all NW nodes have not been selected, the node management unit 111 determines not to end the processing of this flow.

[0063] In this way, the NW management node 11 checks the series of operations for all NW nodes and determines a target list of NW nodes in which a pseudo-failure will occur.

[0064] [Dummy failure execution process in step 6] FIG. 7 is a diagram illustrating an example of a procedure for the pseudo-fault execution process according to the embodiment. An example of a detailed algorithm is shown with reference to FIG.

[0065] (Step S11) When the NW node 231 receives the pseudo-failure execution request, the NW node 231 proceeds to the process of step S12.

[0066] (Step S12) The NW node 231 waits until the specified pseudo-failure occurrence time. Then, the process proceeds to step S13.

[0067] (Step S13) The NW node 231 checks (determines) whether or not the pseudo-failure can be executed in real time (that is, for example, a predetermined time before the scheduled pseudo-failure occurrence time). As a result of this determination, if the NW node 231 determines that it is possible to execute the pseudo-failure (step S13: YES), the NW node 231 proceeds to the processing of step S14. On the other hand, if the NW node 231 determines that the pseudo-failure cannot be executed (step S13: NO), the NW node 231 ends the process of this flow. In other words, if the NW node 231 is in a state where the pseudo-failure cannot be generated, the NW node 231 ends the process of this flow without generating the pseudo-failure.

[0068] Here, the method for determining whether or not the NW node 231 can execute a pseudo-failure is not particularly limited in this embodiment, but for example, a method for determining that a pseudo-failure cannot be executed when an operator of a telecommunications carrier is performing some kind of work on the relevant NW node 231, a method for determining that a pseudo-failure cannot be executed when an emergency call is (for example, temporarily) being processed on the relevant NW node 231, or a method for determining that a pseudo-failure cannot be executed when there is a high possibility that the SLA (Service Level Agreement) that compensates users will be violated if a pseudo-failure is generated may be used.

[0069] Here, as a method for determining whether an emergency call is being processed on a network, for example, a method may be used in which an emergency call is determined to be being processed on a network if the number of sessions for emergency calls on the network is greater than or equal to a predetermined number (for example, 1). Furthermore, as a method for determining whether or not there is a high possibility of violating the SLA, for example, a method may be used in which it is determined that there is a high possibility of violating the SLA if a specified access (e.g., a telephone call) is rejected a specified number of times or more within a specified period of time (e.g., five or more times per hour).

[0070] Furthermore, the determination of whether or not a pseudo-failure can be executed, such as a determination regarding an emergency call or an SLA, may be made solely by the relevant NW node 231, or may be made by any device other than the relevant NW node 231 (e.g., another node), and the determination result may be notified to the relevant NW node 231, which may then determine whether or not a pseudo-failure can be executed based on the contents of the notification.

[0071] 4, multiple pseudo-failures are scheduled, and there are two or more pseudo-failures with different failure occurrence times (scheduled). In this case, for example, a determination may be made as to whether or not the pseudo-failure can be executed for each pseudo-failure with different failure occurrence times (scheduled). Furthermore, for example, even in the case of two or more pseudo failures that have the same failure occurrence time (scheduled), a determination may be made for each pseudo failure as to whether or not the pseudo failure can be executed.

[0072] (Step S14) The NW node 231 generates a pseudo-failure and ends the process of this flow. That is, if the NW node 231 is in a state where a pseudo-failure can occur, it generates a pseudo-failure with specified content (for example, a CPU spike) and ends the process of this flow.

[0073] In this embodiment, in the processing of step S13, if the NW node 231 determines that it is not possible to execute the pseudo-failure in real time, the processing of this flow is terminated. However, as another example, in such a case, the processing of this flow may be terminated by postponing the corresponding failure occurrence time (planned) in the failure information table 2021 shown in FIG. 4 by a predetermined time.

[0074] [Learning-related processing] FIG. 8 is a diagram illustrating an example of a procedure of a learning-related process according to the embodiment. Referring to FIG. 8, an example of a process for training a learning model for network operation by AI / ML of a certain NW node in the AI / ML management node 13 is shown.

[0075] (Process T111) The AI / ML management node 13 starts the processing.

[0076] (Step 1: Process T131) In order to train a learning model for network operation using AI / ML, the AI / ML management node 13 sends a request to the information database 14 to obtain statistical information and fault information regarding the NW nodes that are the targets of training the learning model. Here, the period for obtaining the information is not particularly limited in this embodiment, but for example, a method of requesting information for all periods, or a method of requesting information for only a specific period may be used.

[0077] (Step 2: Process T132) The information database 14 acquires information (statistical information and fault information) specified by the AI / ML management node 13, and transmits the information to the AI / ML management node 13 for provision.

[0078] (Step 3: Process T112) The AI / ML management node 13 uses the acquired information to train a learning model for network operation using AI / ML.

[0079] (Step 4: Process T113) The AI / ML management node 13 evaluates the accuracy of learning models for network operation using AI / ML. It should be noted that the accuracy (detection accuracy) of the learning model may change each time training is performed based on a fault (including a pseudo fault). Furthermore, the accuracy (detection accuracy) of the learning model may change when, for example, the configuration of the network changes. The configuration of a network changes, for example, when nodes included in the network are added or removed.

[0080] In this embodiment, there are no particular limitations on the training method or evaluation method. In this embodiment, for example, a method may be used in which database data acquired by the information database 14 is divided into a training dataset and an evaluation dataset at a certain ratio, a learning model for network operation using AI / ML is trained using the training dataset, and the learning model is used to evaluate the accuracy of fault detection and fault cause identification using the evaluation dataset.

[0081] (Step 5: Process T133) The AI / ML management node 13 transmits to the NW management node 11 the training status of the learning model for the current target NW node. Here, the training status includes, for example, information about the network node, the trained learning model, the version of the learning model, and the detection accuracy of the learning model.

[0082] As a result of the above, a learning model for highly accurate AI / ML network operation is trained based on statistical information from the simulated failure, and the training status is transmitted to the NW management node 11. Here, the series of processes shown in Fig. 8 are repeatedly executed for each NW node, for example. As a specific example, the series of processes shown in Fig. 8 may be executed periodically at any timing, such as once every hour or once every day.

[0083] In the example of Figure 8, a case is shown in which the training of the learning model and the evaluation of its accuracy are performed without distinguishing between the results of natural failures (i.e., the results of failures that are not pseudo-failures) and the results of pseudo-failures. However, for example, a mode in which the results of natural failures and the results of pseudo-failures are distinguished in one or both of the training of the learning model and the evaluation of its accuracy may be used. For example, a configuration may be used in which the results of a simulated disorder are used to train a learning model, and the results of a natural disorder are used to evaluate the accuracy of the learning model.

[0084] In addition, the example of Figure 8 shows a case where training of the AI / ML learning model (process T112) and accuracy evaluation of the AI / ML learning model (process T113) are performed as a series of processes, but these processes may be performed independently at any timing. That is, as an example separate from Figure 8, training of the AI / ML learning model may be performed at any timing. Similarly, as an example separate from Figure 8, accuracy evaluation of the AI / ML learning model may be performed at any timing.

[0085] As described above, the pseudo network failure occurrence system A1 in the communication system 1 according to this embodiment generates a pseudo failure on the network, thereby making it possible to acquire statistical information for a learning model for highly accurate AI / ML-based network operation, thereby enabling training of the learning model. The pseudo network failure occurrence system A1 according to this embodiment makes it possible to train a highly accurate AI / ML learning model for network failures. The network pseudo-failure occurrence system A1 according to this embodiment can appropriately control the occurrence of pseudo-failures and train a highly accurate learning model.

[0086] In the network pseudo-failure occurrence system A1 according to this embodiment, information on the occurrence of such pseudo-failures is stored in a database. In the network pseudo failure occurrence system A1 according to this embodiment, the training status is transmitted to the NW management node 11 when training a learning model for network operation using AI / ML.

[0087] In the network pseudo-fault generation system A1 according to this embodiment, pseudo-faults are generated while checking the accuracy of the learning model for network operation using AI / ML, thereby preventing unnecessary pseudo-faults and over-learning of the learning model. In the network pseudo-failure occurrence system A1 according to this embodiment, the NW management node 11 checks the accuracy of a learning model for AI / ML-based network operation and determines the target for pseudo-failure occurrence.

[0088] In the network pseudo-fault occurrence system A1 of this embodiment, by checking whether a pseudo-fault can currently (in real time) occur in the NW node before the pseudo-fault occurs (for example, immediately before the pseudo-fault occurs), it is possible to prevent excessive impact on communication services.

[0089] In the network pseudo-fault occurrence system A1 according to this embodiment, a node that generates a pseudo-fault is introduced on the network in order to train a highly accurate AI / ML learning model for detecting network faults and identifying their causes. Furthermore, in the network pseudo-fault occurrence system A1 according to this embodiment, a pseudo-fault is generated within the network, making it possible to train a highly accurate AI / ML learning model for detecting network faults and identifying their causes. Furthermore, in order to prevent overlearning, the network pseudo fault occurrence system A1 according to this embodiment stops generating pseudo faults for nodes for which training of a highly accurate model has been completed. Furthermore, the network pseudo-failure occurrence system A1 according to this embodiment checks whether a certain node currently (in real time) allows a pseudo-failure before causing a pseudo-failure.

[0090] In this way, in the network pseudo-failure occurrence system A1 according to this embodiment, the learning model is trained by AI / ML using pseudo-failures.

[0091] Here, the configurations of the communication system 1 and the network pseudo-failure occurrence system A1 shown in FIGS. 1 and 2 are merely examples, and the present invention is not limited to these examples. For example, in this embodiment, the NW management node 11, pseudo-fault management node 12, AI / ML management node 13, and information database 14 are each configured as separate devices, but as another example, two or more of these (or all of them) may be configured as an integrated unit.

[0092] In addition, in this embodiment, a case has been shown in which the network pseudo-fault occurrence system A1 exists in a network separate from the managed network 211, but as another configuration example, a configuration in which part or all of the network pseudo-fault occurrence system A1 exists in the managed network 211 may be used.

[0093] Furthermore, the network in which a pseudo-fault is generated by the network pseudo-fault generation system A1 of this embodiment is not necessarily limited to a 5G network, nor is it necessarily limited to a mobile network, but may be applied to various other networks.

[0094] As an example configuration, the network pseudo-fault generation system A1 includes a fault generation unit 122 that generates pseudo-faults in nodes (in the example of Figure 2, NW nodes 231 to 233) included in a network (in the example of Figure 2, managed network 211), an information database 14 that stores information related to pseudo-faults in nodes, an AI / ML training unit 131 that trains an AI / ML learning model based on the information stored in the information database 14, an AI / ML evaluation unit 132 that evaluates the detection accuracy of the learning model, and a fault determination unit 121 that determines the nature of the pseudo-fault to be generated by the fault generation unit 122 based on the evaluation result of the AI / ML evaluation unit 132.

[0095] As an example configuration, in the network pseudo-fault occurrence system A1, the node management unit 111 determines that a pseudo-fault needs to occur if it determines, based on the evaluation results of the AI / ML evaluation unit 132, that the detection accuracy is insufficient compared to a threshold (for example, below the threshold or below the threshold), and determines that a pseudo-fault does not need to occur if it determines that the detection accuracy is not insufficient compared to the threshold (for example, above the threshold or above the threshold).

[0096] As an example configuration, in the network pseudo-fault occurrence system A1, the detection accuracy is determined for each of one or more units of node, time period, and fault item.

[0097] In one configuration example, the network pseudo-fault occurrence system A1 further includes a node, which determines in real time whether to cause a pseudo-fault based on the type of pseudo-fault to be caused. In one configuration, the node decides not to simulate a failure if an emergency call is being processed. In one configuration example, if a node determines that generating a pseudo-failure would violate the SLA it is compensating for, it decides not to generate a pseudo-failure. Here, the network pseudo-failure occurrence system A1 may be considered to include, for example, some or all of the components of the managed network 211.

[0098] For example, it is possible to provide a method (for example, a network pseudo-failure generation method) for executing part or all of the processes performed in the communication system 1 and the network pseudo-failure generation system A1.

[0099] For example, it is possible to provide a program that causes a computer to implement the functions of executing part or all of the processes performed in the communication system 1 and the network pseudo-failure occurrence system A1.

[0100] [Regarding the above embodiment] As a result, it will be possible to improve the overall service quality of wireless networks, which will contribute to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "Build resilient infrastructure, promote sustainable industrialization and foster innovation."

[0101] A program for implementing the functions of any of the components of any of the above-described devices may be recorded on a computer-readable recording medium and loaded into a computer system for execution. The term "computer system" as used herein includes hardware such as an operating system or peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and compact discs (CDs) or read-only memories (ROMs), as well as storage devices such as hard disks built into computer systems. The term "computer-readable recording medium" also includes devices that retain a program for a certain period of time, such as volatile memory within a computer system that acts as a server or client when a program is transmitted over a network such as the Internet or a communication line such as a telephone line. Such volatile memory may be, for example, random access memory (RAM). The recording medium may also be, for example, a non-transitory recording medium.

[0102] The above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network such as the Internet or a communication line such as a telephone line. The above program may also be one that realizes part of the above-mentioned functions. Furthermore, the above program may be a so-called differential file that can realize the above-mentioned functions in combination with a program already recorded in a computer system. A differential file may also be called a differential program.

[0103] Furthermore, the functions of any of the components in any of the above-described devices may be implemented by a processor. For example, each process in the embodiments may be implemented by a processor operating based on information such as a program and a computer-readable recording medium storing information such as the program. Here, the functions of each unit of the processor may be implemented by, for example, individual hardware, or may be implemented by integrated hardware. For example, the processor may include hardware, and the hardware may include at least one of a circuit for processing digital signals and a circuit for processing analog signals. For example, the processor may be configured using one or more circuit devices mounted on a circuit board, or one or both of one or more circuit elements. An integrated circuit (IC) or the like may be used as the circuit device, and a resistor or a capacitor may be used as the circuit element.

[0104] Here, the processor may be, for example, a CPU. However, the processor is not limited to a CPU, and various types of processors such as a GPU (Graphics Processing Unit) or a DSP (Digital Signal Processor) may be used. The processor may also be, for example, a hardware circuit such as an ASIC (Application Specific Integrated Circuit). The processor may also be, for example, composed of multiple CPUs, or may be, for example, composed of a hardware circuit such as a multiple ASIC. The processor may also be, for example, composed of a combination of multiple CPUs and a hardware circuit such as a multiple ASIC. The processor may also include, for example, one or more of an amplifier circuit or a filter circuit that processes analog signals.

[0105] The embodiments of this disclosure have been described in detail above with reference to the drawings, but the specific configuration is not limited to this embodiment, and includes designs within the scope that do not deviate from the gist of this disclosure. [Explanation of symbols]

[0106] 1...communication system, 11...network management node, 12...pseudo-fault management node, 13...AI / ML management node, 14...information database, 21...AMF, 22...SMF, 23...NP, 24...UPF, 31-33...UE, 41...RAN, 51...base station, 61...data network, 111...node management unit, 112...AI / ML management unit, 121...fault determination unit, 122...fault occurrence unit, 131...AI / ML training unit, 132...AI / ML evaluation unit, 141...statistical information, 142...fault information, 211...managed network, 231-233...network nodes, 2011...pseudo-fault target node list, 2021...fault information table, A1...network pseudo-fault occurrence system

Claims

1. a failure generating unit that generates a pseudo-failure in a node included in the network; an information database that stores information about the pseudo-failure of the node; an AI / ML training unit that trains an AI / ML learning model based on the information stored in the information database; an AI / ML evaluation unit that evaluates the detection accuracy of the learning model; a fault determination unit that determines the type of the pseudo fault to be generated by the fault generation unit based on the evaluation result of the AI / ML evaluation unit; the node; The node determines whether to generate the pseudo-failure as a real-time determination based on the mode of the pseudo-failure to be generated. Network pseudo-fault generation system.

2. The node determines not to generate the pseudo failure when an emergency call is being processed.

2. The network pseudo-fault generation system according to claim 1.

3. the node determines not to cause the pseudo-failure if it determines that causing the pseudo-failure would violate the SLA that it is compensating for; 2. The network pseudo-fault generation system according to claim 1.

4. a node management unit that determines, based on the evaluation result of the AI / ML evaluation unit, that the occurrence of the pseudo failure is necessary when it is determined that the detection accuracy is insufficient compared to a threshold, and that the occurrence of the pseudo failure is unnecessary when it is determined that the detection accuracy is not insufficient compared to a threshold; 4. The network pseudo-fault generation system according to claim 1.

5. The detection accuracy is determined for each of one or more units of the node, the time period, and the fault item.

4. The network pseudo-fault generation system according to claim 1.

6. The failure generation unit generates a pseudo-failure in a node included in the network, an information database stores information about the pseudo-failure of the node; an AI / ML training unit that trains an AI / ML learning model based on the information stored in the information database; an AI / ML evaluation unit that evaluates the detection accuracy of the learning model; a fault determination unit determines the type of the pseudo fault to be generated by the fault generation unit based on the evaluation result of the AI / ML evaluation unit; The node determines whether to generate the pseudo-failure as a real-time determination based on the mode of the pseudo-failure to be generated. A method for generating pseudo-faults in a network.

7. On the computer, a failure generation function that generates pseudo-failures in nodes included in the network; a fault determination function that determines the nature of the pseudo fault to be generated by the fault generation function based on the evaluation result of the AI / ML evaluation unit; A program for realizing the above, an information database stores information about the pseudo-failure of the node; an AI / ML training unit that trains an AI / ML learning model based on the information stored in the information database; The AI / ML evaluation unit evaluates the detection accuracy of the learning model, The node determines whether to generate the pseudo-failure as a real-time determination based on the mode of the pseudo-failure to be generated; The program further comprises: The computer, a function of requesting the node to execute the pseudo-failure simulation; a function of receiving a report of the execution result of the pseudo-failure from the node; To achieve this, program.

Images