Measure execution device, measure execution method, and program

Abstract

The purpose of the present disclosure is to take a measure against a newly occurred event by referring to similar events occurred in the past.　The present disclosure is a measure execution device that executes a measure against an event that has occurred in a communication device in a prescribed network, the measure execution device comprising: a management unit that manages, in association with each other, abstracted event information including a content obtained by abstracting an event that occurred in the past, and measure information indicating a measure for the abstracted content; and a measure execution unit that executes a measure against a newly occurred event by using prescribed measure information corresponding to prescribed abstracted event information indicating a content obtained by abstracting the newly occurred event in the management unit.

Description

Measure execution device, measure execution method, and program

[0001] The present disclosure relates to a measure execution device, a measure execution method, and a program.

[0002] A technique has been proposed for generating a fault isolation procedure that minimizes the expected value of the total work cost estimated to be required to identify a faulty device in a network (see Patent Document 1).

[0003] JP 2013-141122 A

[0004] However, although the conventional technology can generate a procedure for identifying the location of a fault, it cannot perform a process, i.e., a measure, for recovering from the fault.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to take measures to recover from the failure.

[0006] In order to achieve the above-mentioned object, the present disclosure provides a measure execution device that executes measures against an event that occurs in a communication device in a specified network, the measure execution device having: a management unit that associates and manages abstracted event information including abstracted content of events that occurred in the past with measure information indicating measures against the abstracted content; and a measure execution unit that executes measures against the newly occurred event by using specified measure information that corresponds to the specified abstracted event information indicating the abstracted content of a newly occurred event in the management unit.

[0007] As described above, the present disclosure provides the advantage of being able to take measures to recover from a failure by referring to similar events that have occurred in the past.

[0008] 1 is an overall configuration diagram of a communication system according to an embodiment. FIG. 1 is an electrical hardware configuration diagram of a measure execution device according to an embodiment. FIG. 2 is a diagram mainly illustrating the functional configuration of the measure execution device according to an embodiment. FIG. 3 is a diagram illustrating the contents of a plurality of alarm information. FIG. 4 is a diagram illustrating a measure-related information table. FIG. 5 is a diagram illustrating an example of fault information and abstracted fault information generated from the fault information. FIG. 6 is a diagram illustrating the network configuration of a predetermined management network. FIG. 7 is a diagram illustrating an overview of each piece of information for pre-check, measure, and post-check. FIG. 8 is a diagram illustrating information in one field of the measure-related information table. FIG. 9 is a diagram illustrating information in one field of the measure-related information table. FIG. 10 is a diagram illustrating information in one field of the measure-related information table. FIG. 11 is a diagram illustrating information in one field of the measure-related information table. FIG. 12 is a diagram illustrating information in one field of the measure-related information table. FIG. 13 is a diagram illustrating an example of an intrinsic function. FIG. 14 is a diagram illustrating an example of an intrinsic variable. FIG. 15 is a diagram illustrating an example of an intrinsic variable. FIG. 16 is a sequence diagram illustrating processing by a fault identification unit to acquire fault information including a fault location and a fault type. FIG. 17 is a flowchart illustrating processing performed by a registration determination unit. FIG. 18 is a flowchart illustrating processing performed by a measure execution unit. FIG. 19 is a flowchart illustrating processing performed by a registration processing unit.

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited to the embodiments shown below, and various modifications are possible within the scope of the technical concept of the present invention. Since the drawings are intended to conceptually explain the present invention, dimensions, ratios, or numbers may be exaggerated or simplified as necessary to facilitate understanding.

[0010] [Overview of the embodiment] In the current trend of DX (digital transformation), pioneering ICT (Information and Communication Technology) technologies are being demonstrated. It is expected that large-scale solution projects with limited area or time limits, such as smart cities and world expos, will increase in the future. Networks are essential infrastructure for such large-scale solution projects, and it is anticipated that there will be a need for their maintenance and operation. Research is currently underway into the operation and management of MNs (Managed Networks) for such large-scale solutions. This embodiment discloses the operation and management of MNs.

[0011] Furthermore, in network operations at the MN, recovery measures for a fault involve significant risks, such as the impact of failure, and there is psychological resistance to devices taking such measures.

[0012] Meanwhile, research attempting to automate network operation measures has proposed using seq2seq to automatically generate commands from logs such as syslog (Reference 1). <Reference 1> H. Ikeuchi et al., "Recovery command generation towards automatic recovery in ICT systems by Seq2Seq learning," IEEE / IFIP NOMS, Apr. 2022. However, seq2seq is a machine learning method, and appropriate measures cannot be taken until it has been trained using a large amount of training data. Machine learning and general AI (artificial intelligence), including seq2seq, basically acquire generalized knowledge through machine learning in advance, and then input data into it once operation begins to produce output.

[0013] In contrast to this, case-based reasoning (CBR) is known as a methodology that does not generalize in advance, but instead repeatedly infers from past cases and outputs inferences based on input data (Reference 2). <Reference 2> M. Richiter et al., Case-Based Reasoning: A Textbook, Nov. 2013. CBR is a reasoning method that, when a new problem needs to be solved, searches a database (DB) that organizes and stores cases consisting of known (past) problems (e.g., problems) and their solutions (measures) in a common format, extracts similar cases, and modifies those solutions to obtain a solution to the new problem. In this case, the obtained solution is stored in the DB as a new case along with the problem.

[0014] Considering that machine learning based on large amounts of data is not suitable for automating actions, CBR is a promising methodology. Furthermore, because the output solution (action) can be based on past cases (i.e., track records) registered by humans, it is believed to contribute to improving psychological safety. Therefore, in this embodiment, processing using case-based reasoning will be described.

[0015] First, the overall configuration of a communication system according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing the overall configuration of a communication system according to an embodiment.

[0016] As shown in FIG. 1, a communication system 10 of this embodiment is constructed by a communication device 20, a monitoring and collection server 30, a measure execution device 50, an RCA (Root Cause Analysis) server 70, and an input device 80.

[0017] The communication device 20, the monitoring and collection server 30, the measure execution device 50, and the RCA server 70 are installed in a special zone X such as a smart city or an expo. The communication device 20, the monitoring and collection server 30, the measure execution device 50, and the RCA server 70 can communicate with each other via a local area network (LAN) 90 included in an MN established within the special zone X. The communication device 20, the monitoring and collection server 30, the measure execution device 50, and the RCA server 70 can also communicate with an input device 80 via the LAN 90 and a wide area network (WAN) 100 outside the special zone X. The RCA server 70 and the measure execution device 50 may also be installed outside the special zone X.

[0018] The input device 80 is installed in a network operation center (NOC), which is an organization of a telecommunications carrier, etc. The input device 80 is operated by an operator y of the NOC.

[0019] In Fig. 1, one input device 80 is connected to each LAN 90 of multiple special zones X, and can remotely control the action execution device 50 in each special zone X. The LAN 90 and WAN 100 are examples of communication networks. The connection form of part of the communication network may be either wireless or wired.

[0020] <Communication Device> The communication device 20 is a target for measures to be taken when a failure occurs within the LAN 90, and is, for example, a server, a modem, a router, a switch, a hub, or a communication terminal (such as a PC). For the sake of convenience, one communication device 20 is shown in Fig. 1, but this communication device 20 refers to a single or multiple communication devices.

[0021] <Monitoring and Collection Server> The monitoring and collection server 30 has a function of monitoring the communication devices 20 in the LAN 90, collecting context information, and transmitting alarm information (see FIG. 4) to the RCA server 70 or the action execution device 50 when a failure occurs in the communication device 20. The monitoring and collection server 30 includes an SNMP (Simple Network Management Protocol) manager and a log management server.

[0022] Note that "context information" is an example of information (situation information) that indicates the situation when a failure occurs and can be used to characterize the situation, and includes, for example, traffic volume, memory usage, OS (operating system) version, time when the failure occurred (or was detected), etc. Furthermore, after the service starts operating, new types of context information (for example, bit rate of video distribution) may be additionally generated or deleted when a new service is provided.

[0023] Furthermore, "alarm information" is information that notifies that a failure has occurred, and includes, for example, information on the time when the failure occurred or was detected, the source of the alarm, and the alarm content, as shown in Fig. 4. Note that the alarm information may also include a detection log, metrics, etc. Fig. 4 is a diagram showing the content of multiple pieces of alarm information.

[0024] <Measure Execution Device> The measure execution device 50 plays a central role in this embodiment, and is a device that executes measures in response to a failure (event) that occurs in a communication device 20 in a predetermined network such as the LAN 90 .

[0025] In this embodiment, within the LAN 90, the action execution device 50 basically performs autonomous (spontaneous) maintenance on the communication device 20, so that if any failure occurs in the communication device 20, the action execution device 50 executes recovery measures for the failure on the communication device 20. Furthermore, if the action execution device 50 is unable to execute the measures or if the communication device 20 does not recover even after execution of the measures, the input device 80 of the NOC can transmit information related to the measures to be taken on the communication device 20 to the action execution device 50, thereby causing the action execution device 50 to remotely execute the measures. In this way, in this embodiment, hybrid operation management is performed using autonomous measures within the LAN 90 and remote measures.

[0026] However, research into automating fault response measures has not progressed in the field of network maintenance in general, including future MNs. One reason for this is that faults in MNs occur infrequently, making it difficult to collect a large number of cases of faults and their response measures for machine learning. Furthermore, while the situations that characterize faults and appropriate response measures are extremely diverse, the fact that actual cases are only observed cases in certain specific situations makes learning even more difficult.

[0027] Therefore, in this embodiment, while following the concept of hybrid operation management described above, the measure execution device 50 on the MN side automatically executes measures not only for failures identical to those that occurred in the past, but also for similar failures that satisfy certain conditions. Specifically, for a new failure, an operator y on the NOC side manually and remotely instructs the MN side to execute measures, and the measure execution device 50 on the MN side records the measures taken for the new failure (measure-related information). Then, when a similar (identical or similar) failure occurs again, the measure execution device 50 refers to the situation at that time (context information), extracts the measures taken for the similar failure in the past, and automatically executes a measure command to execute the measure.

[0028] <RCA Server> The RCA server 70 analyzes alarm information (see FIG. 4 ) and the like acquired from the monitoring and collection server 30 through root cause analysis to derive information including fault location information and fault type information, and transmits this information to the action execution device 50 as fault information. The type and format of the information included in the fault information differ depending on the specifications of the RCA server 70. "Fault location information" indicates the communication device 20 where the fault occurred or the specific part of this communication device 20 where the fault occurred, such as L3SW#3 or L3SW#4. "Fault type information" is derived from the alarm content in the alarm information and indicates the type of fault, such as a buffer overflow error or link down.

[0029] <Input device> In the event that the measure execution device 50 is unable to execute a measure or executes a measure but the communication device 20 does not recover, the input device 80 transmits information regarding the measure to be taken on the communication device 20 to the measure execution device 50, thereby causing the measure execution device 50 to remotely execute the measure, in order to realize hybrid operational management.

[0030] [Hardware Configuration] Next, the electrical hardware configuration of the measure execution device 50 will be described with reference to Fig. 2. Fig. 2 is a diagram showing the electrical hardware configuration of the measure execution device according to the embodiment.

[0031] As shown in Figure 2, the action execution device 50 includes a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a processor 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, etc., which are all interconnected by a bus 1010 such as a data bus.

[0032] The program that realizes the processing on the computer is provided by a recording medium 1001, such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via the communication network 100. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

[0033] When an instruction to start a program is received, the memory device 1003 reads the program from the auxiliary storage device 1002 and stores it. The processor 1004 realizes functions related to the device in accordance with the program stored in the memory device 1003. The processor 1004 may include not only a CPU (Central Processing Unit) but also a GPU (Graphics Processing Unit).

[0034] The interface device 1005 is used as an interface for connecting to a communication network, etc. The display device 1006 displays a GUI (Graphical User Interface) etc. according to a program. The input device 1007 is composed of a keyboard, mouse, buttons, a touch panel, etc., and is used to input various operation instructions. The output device 1008 outputs the calculation results to the outside.

[0035] In addition, when the communication device 20 is a server or the like, the monitoring and collection server 30, the RCA server 70, and the input device 80 have the same hardware configuration as the action execution device 50, so the description thereof will be omitted.

[0036] [Functional Configuration of the Measure Execution Device] Next, the functional configuration of the measure execution device 50 will be described with reference to Fig. 3. Fig. 3 is a diagram showing the functional configuration of the measure execution device according to the embodiment.

[0037] 3, the measure execution device 50 has a fault identification unit 51, a registration determination unit 52, a measure execution unit 53, and a registration processing unit 54. Each of these units has a function realized by an instruction from the processor 1004 in FIG. 2 based on a program. The measure execution device 50 also has a storage unit 60 constructed by an auxiliary storage device 1002 or a memory device 1003, and has a measure-related information management DB 61 stored in the storage unit 60.

[0038] <Measure-related information management DB> Next, the measure-related information management DB 61 will be described with reference to Fig. 5 to Fig. 11. Fig. 5 is a diagram showing a measure-related information table. The measure-related information management DB 61 is configured by the measure-related information table shown in Fig. 5. Note that in Fig. 5, examples of Python scripts for pre-confirmation, measures, and post-confirmation are pseudo-codes shown for the sake of convenience.

[0039] As shown in Figure 5, the action-related information table manages case records (case records) for one failure, and each case record is managed by associating a record ID with information on the failure, abstracted failure, pre-check, action, post-check, number of successes / failures, and case context.

[0040] (Fault Information) "Fault Information" corresponds to "Problem" in case-based reasoning, and includes key-values ​​for fault location information and fault type information. Fault information is information sent from the RCA server 70 to the action execution device 50. Note that in Fig. 5, for the sake of simplicity, the fault location and fault type are expressed in one sentence within one field called fault information, but strictly speaking, it is composed of two key-values.

[0041] (Abstracted fault information) "Abstracted fault information" includes information indicating the abstracted content of the fault location information in the fault information, and fault type information. In Fig. 5, the records with record IDs "001" and "002" indicate the same abstracted fault information. The abstracted fault information is generated by the registration determination unit 52 based on the fault information and a predetermined "group," and is a search key used by the registration determination unit 52 when searching the action-related information management DB 61. Fig. 6 is a diagram showing an example of fault information and abstracted fault information generated from the fault information.

[0042] Furthermore, a "group" refers to a classification of communication devices 20 (having the same role or properties) that perform the same type of processing on the network at the same or different locations. A group is identified by a combination of the topology hierarchy in the LAN 90 and the type of device (router, switch, hub, etc.). In other words, in an object-oriented approach, each communication device 20 corresponds to an instance, and the group corresponds to a class.

[0043] The concept of a group will now be explained using FIG. 7. FIG. 7 is a diagram showing the network configuration of a specified management network. In FIG. 7, the communication devices (L3SW#01, L3SW#02) at the same aggregation point b0 are L3 switches, etc., and are a group (CoreSW) with the same role or property, called L3SW, located in a core network called the aggregation point. Furthermore, the communication devices (L3SW#1, L3SW#2, L3SW#3, L3SW#4, L3SW#5, L3SW#6) at different points b1 to b3 are L3 switches, etc., and are a group (AccessL3SW) with the same role or property, called L3SW, located at the entrance from the aggregation point to each point. The communication devices (L2SW#1, L2SW#2, L2SW#3, L2SW#4, L2SW#5, L2SW#6) at different points b1 to b3 are L2 switches, etc., and are a group (AccessL2SW) with the same role or property, that is, located between the AccessL3SW and a terminal. Furthermore, the communication devices (terminal #1, terminal #2, terminal #3, terminal #4, terminal #5, terminal #6) at different points b1 to b3 are PCs or the like and belong to the same group (UE).

[0044] In this way, by grouping communication devices 20 that perform the same type of processing (that play the same role) into a group (corresponding to a class in object-oriented programming), it is possible to reuse the content of measures taken to deal with similar failures that have occurred in the past for communication devices 20 that belong to the same group. Note that the components of a group can be anything as long as they are of the same type, and are not limited to devices, but can also be interfaces (ports), for example.

[0045] (Pre-check information, action information, and post-check information) Figure 8 is a diagram showing an overview of each piece of information for pre-check, action, and post-check. Furthermore, the contents of fields (A) to (F) that could not be shown in Figure 5 due to space limitations are shown in Figures 9A to 9F, respectively. In Figures 9A to 9F, the underlined parts indicate cases where built-in functions are used, and the bolded parts indicate cases where built-in variables are used.

[0046] (Built-in Functions) A "built-in function" is a Python function that predefines the method of obtaining information (current context information and past case context information) required for performing pre-checks, measures, and post-checks, and the method of determining whether a condition is met by the registration determination unit 52. Specifically, the built-in function implements functions such as accessing the monitoring and collection server 30, the measure-related information management DB 61, various repositories, and each device in the LAN 90 using SSH (Secure Shell) or API (Application Programming Interface), and obtaining context information such as traffic volume from these access destinations. Figure 10 is a diagram showing an example of a built-in function. Note that the numbers in parentheses represent arguments.

[0047] ((Built-in variables)) "Built-in variables" are predefined and implemented Python variables that, like built-in functions, indicate the information (current context information and past case context information) required for pre-checks, measures, and post-checks. Specifically, there are the following types of built-in variables:

[0048] (1) Variables for accessing information held by the fault identification unit 51 These are the contents of the fault information and indicate the information acquired by the fault identification unit 51. For example, the name of the faulty device, the port to be corrected, ALARM_TIME (time when the fault occurred), etc.

[0049] (2) Variables for accessing the instance and context information at the time the record was recorded in the registration determination unit 52 and the action execution unit 53. For example, CASE_TIME (the time when the case was registered in the record) and CASE_MACHINE ((the device subject to action in the case registered in the record)).

[0050] (3) A variable for dynamically accessing context information related to a new (current) failure in the registration determination unit 52. For example, it is device information (OS version, IP address, etc.) obtained by accessing the action-related information management DB 61, CURRENT_TIME (the time when action is attempted for the new failure), or CURRENT_MACHINE (the device on which action is to be taken for the new failure).

[0051] 11A to 11C are diagrams showing examples of built-in variables.

[0052] By defining the pre-check, action, and post-check codes using the built-in functions or built-in variables described above, the implementation of the actual values ​​and acquisition methods can be hidden from each pre-check, action, and post-check script. This is also expected to improve the readability and reusability of each pre-check, action, and post-check script. Python scripts containing built-in functions or built-in variables are executed by the registration determination unit 52 and the action execution unit 53.

[0053] Furthermore, built-in functions and the advantages of using them will be explained in detail.

[0054] First, it can improve the readability of the contents (scripts) of pre-checks, measures, and post-checks. For example, if an IP address is written in the pre-check, measures, and post-check scripts, it becomes difficult to read, and it becomes difficult for a person to register another record in the future by referring to the script.

[0055] Second, the pre-check allows for flexible description of the conditions for determining whether a condition is met. When the script's built-in functions or built-in variables are read by the registration / check unit 52, context values ​​(e.g., the current time, the record registration time (a type of case context), and current and past network traffic volume) are dynamically accessed (such code is manually written and predefined). In this embodiment, the context has a much broader scope than "computer parameter values," and built-in functions or built-in variables can be used to obtain any information related to the "situation" (context information) in which a failure occurred. Therefore, the action execution device 50 can obtain not only the context information (e.g., network traffic volume) stored in the monitoring / collection server 30 but also the context information stored in the communication device 20 where the failure occurred.

[0056] Next, the "pre-check information" is information indicating a command sequence that conditionally determines whether the content of the action information described in the record containing this information is appropriate for a new problem, and that automatically verifies whether the action can be taken by checking the situation before the action is executed. Specifically, the "pre-check information" is a Python script that is executed before the action content registered in the "action" field is executed. The "pre-check information" is used to determine whether the context information in the secondary search (see S33) performed by the registration determination unit 52 matches the condition. For example, the "pre-check information" is used to determine whether the value of the context information related to the new problem matches the value of the case context information. In this embodiment, the pre-check information is written using Python assert statements. The assert statement returns an exception if the condition expression is false. Assert statements are generally used for debugging in software development and are often disabled at the time of release. However, in this embodiment, the condition match determination can be expressed simply in a single statement without using a unique description format. Therefore, all the pre-check information is written using assert statements using the built-in functions or built-in variables described above. Generally, assertions (assert statements) are used in unit testing, but in this example, they are applied to the purposes of "pre-check" and "post-check." However, other description methods such as if statements can be used instead of assert statements.

[0057] The conditional expression in the assert statement can use the built-in functions or built-in variables defined in "Common Functions" above.

[0058] "Action information" is information indicating a command sequence for an action to automatically recover from the "fault" of the record containing this information. The action content indicated by this action information corresponds to a "Solution" in case-based reasoning. Specifically, the "action information" is an action command and a Python script for inputting (injecting) the command into the fault location (communication device 20). When indicating the fault location to which the action command is to be input, the above-mentioned built-in function or built-in variable can be used instead of describing the actual address. In this way, the actual address can be hidden from the action command sequence.

[0059] The "post-confirmation information" is information indicating a command string that confirms whether the above-mentioned "measure" was successful. Specifically, the "post-confirmation information" indicates a Python script that confirms whether recovery was achieved as intended after the measure execution unit 53 executes the recovery measure. The post-confirmation information is also written entirely in assert statements using the above-mentioned built-in functions or built-in variables. A ping is sent within the assert statement. However, other description methods, such as if statements, may be used instead of assert statements.

[0060] In addition, in the measure-related information table shown in FIG. 5, the pre-check information, measure information, and post-check information may be combined into one, and a "script" column indicating the definition of an embedded function or embedded variable may be added.

[0061] (Success / Failure Count Information) "Success / Failure Count Information" is information that indicates the results of the post-check (the success / failure results when the measure is executed). Specifically, "Success / Failure Count Information" indicates the number of successes and failures of the post-check measure. If all assert statements in one piece of post-check information are successful, the number of successes is counted up by one, and if at least one assert statement in one piece of post-check information fails, the number of failures is counted up by one.

[0062] (Case Context Information) "Case context information" is context information related to the case when the case was recorded as a case record, such as the circumstances under which the failure occurred. Specifically, when built-in variables indicating case context information (e.g., the date and time the case record was registered) are used in the Python scripts for pre-checks, actions, and post-checks, the actual values ​​are recorded in this case context information. That is, as shown in Figures 5 and 9A to 9F, case context information is referenced by built-in variables (e.g., CASE_MACHINE) in pre-checks and actions. For example, case context information includes the date and time the failure occurred, the date and time the case record was registered, and additional information about the location of the failure (e.g., the OS version of the communication device 20). Case context is necessary because, for example, during pre-checks, when confirming whether a failure actually occurred at the time of the "date and time the failure occurred," it is necessary to reference the information on the "date and time the failure occurred."

[0063] In the above, it was stated that the Python script itself is managed for pre-checks, measures, and post-checks, but in the measure-related information management DB61, only the path where the Python script exists may be managed for pre-checks, measures, and post-checks.

[0064] <Fault Identification Unit> The fault identification unit 51 acquires alarm information from the monitoring and collection server 30 and determines whether a fault has occurred in the communication device 20 in the LAN 90 in the special zone X. At this time, the fault identification unit 51 may request and acquire fault information on a new (current) fault from the RCA server 70. Furthermore, the fault identification unit 51 identifies the location and type of the fault from the fault information acquired from the RCA server 70, and outputs the fault information to the registration determination unit 52 either as is or in a data format that can be processed by the registration determination unit 52. This identified event is referred to as a "new fault."

[0065] <Registration Determination Unit> The registration determination unit 52 generates abstracted fault information from the fault information by classifying the fault information for new (current) faults acquired from the fault identification unit 51 into groups that perform predetermined similar processing (play the same role). The registration determination unit 52 also attempts to extract corresponding case records by searching the action-related information management DB 61 using the generated abstracted fault information as a search key (primary search).

[0066] Furthermore, if the registration determination unit 52 extracts multiple or a single case record in the primary search, it executes the pre-check contents for each case record by executing the string of assert statements indicated in the pre-check information registered in each case record. During execution, when a built-in function used in the assert statement is called or a built-in variable is referenced, the unit obtains the case context value for the case in accordance with the implementation defined in the built-in function or built-in variable. The unit also requests context information related to the new fault from the monitoring and collection server 30 to dynamically obtain the value of the current context information (i.e., the new fault). If the pre-checks for a given case record using all assert statements in its pre-check information are successful, the unit determines that the pre-check for that record is successful. On the other hand, if the pre-check for at least one assert statement in the pre-check information for that record fails, the unit 52 determines that the pre-check for that record has failed. Then, if all assert statements are successful (normal termination), that is, if there is one or more case records for which the preliminary check was successful, the registration determination unit 52 extracts a single specified case record (measure-related information) from those case records in accordance with one of the specified rules shown below (secondary search).

[0067] α: Case record with the most assert statements β: Case record with the latest registration date and time γ: Case record with the most number of successes Furthermore, when the registration determination unit 52 extracts a single specified case record, it calls the action execution unit 53 and outputs specified action-related information indicating the contents of the specified case record and fault information to the action execution unit 53. On the other hand, when the registration determination unit 52 cannot extract any case records in the primary search or secondary search, it calls the registration processing unit 54 and outputs fault information to the registration processing unit 54.

[0068] As described above, in the secondary search, not only is matching of context conditions performed but also advance confirmation can be performed at the same time, making the process efficient.

[0069] <Measure Execution Unit> The measure execution unit 53 acquires a single predetermined case record (measure-related information) and fault information from the registration determination unit 52, and then executes the measure information (command sequence, i.e., Python script) included in the measure-related information. When a built-in function is called or a built-in variable is referenced in the action information, the measure execution unit 53 executes the implementation content defined in the built-in function or built-in variable to dynamically acquire the new fault, i.e., the value of the current context information. Furthermore, if the built-in function or built-in variable is implemented to reference the fault location of the new fault, the measure execution unit 53 acquires the new fault location (i.e., the communication device 20) by executing the measure content according to the command sequence corresponding to the new fault location (i.e., the communication device 20), and takes recovery measures for the new fault location (i.e., the communication device 20) according to the executed measure content. Furthermore, the action execution unit 53 executes the post-check information (a sequence of assert statements) included in the action-related information. When a built-in function is called or a built-in variable is referenced in the post-check information, the action execution unit 53 executes the defined implementation content to dynamically acquire the new fault, i.e., the value of the current context information. Furthermore, if the implementation references the fault location of the new fault in the built-in function or built-in variable, the action execution unit 53 acquires the new fault location (i.e., the communication device 20) by doing so, thereby performing the post-check content using the command sequence corresponding to the new fault location. The action execution unit 53 then registers the success / failure result in the "number of successes" field of the specified case record 60. In this case, if the post-checks using all assert statements in one post-check information are successful, the action execution unit 53 marks the success / failure result as "success" and increments the number of successes by one. On the other hand, if the post-checks using at least one assert statement in one post-check information fail, the action execution unit 53 marks the success / failure result as "failure" and increments the number of failures by one.

[0070] If the process is successful, the action execution device 50 can automatically take recovery measures for the new fault location. On the other hand, if the process is unsuccessful, the action execution unit 53 calls the registration processing unit 54 via the registration determination unit 52 and outputs the failure information and action-related information to the registration processing unit 54. Note that the action execution unit 53 may directly call the registration processing unit 54, obtain the fault information from the registration determination unit 52, and output the fault information and the already obtained action-related information to the registration processing unit 54.

[0071] <Registration Processing Unit> The registration processing unit 54 transmits a notification to the input device 80. As a result, the operator y operates the input device 80 to perform pre-check, action, and post-check for the new failure location. Then, the operator y inputs pre-check information, action information, and post-check information in the same format as that recorded in the action-related information management DB 61 described above, and transmits them to the action execution device 50. As a result, the registration processing unit 54 receives the pre-check information, action information, and post-check information.

[0072] The registration processing unit 54 also requests the monitoring and collection server 30 to provide context information relating to the new failure location, and acquires the context information relating to the new failure location from the monitoring and collection server 30 .

[0073] Furthermore, the registration processing unit 54 associates the information (fault information and abstracted fault information) acquired from the registration determination unit 52, the information received from the input device 80 (pre-check information, action information, and post-check information), and the context information (case context) received from the monitoring and collection server 30 with the new record ID to register as a new record (action-related information) in the action-related information management DB 61. In this case, as described above, the pre-check information, action information, and post-check information may be combined into one, and a "script" column indicating the definition of the built-in function or built-in variable may be added.

[0074] [Processing According to the Embodiment] Next, processing according to the embodiment will be described with reference to FIGS.

[0075] <Processing by the Failure Identification Unit> FIG. 12 is a sequence diagram showing the processing by the failure identification unit to acquire failure information including the location and type of failure.

[0076] S11: Because a failure has occurred in the communication device 20, the communication device 20 transmits alarm information to the monitoring and collection server 30. Note that the monitoring and collection server 30 may transmit a request for alarm information to each communication device 20 at regular intervals, and each communication device 20 may transmit alarm information to the monitoring and collection server 30 if a failure has occurred.

[0077] S12: Meanwhile, the failure identification unit 51 periodically transmits a request for alarm information to the monitoring and collection server 30.

[0078] S13: In response to step S12, the monitoring and collection server 30 transmits the single or multiple pieces of alarm information received in step S11 to the fault identification unit 51. As a result, the fault identification unit 51 receives the single or multiple pieces of alarm information.

[0079] S14: The fault identification unit 51 acquires alarm information from the monitoring and collection server 30 and determines whether or not a fault has occurred in the communication device 20 within the LAN 90 in the special zone X. Note that this process S14 may be omitted.

[0080] S15: If the fault identification unit 51 determines that a fault has occurred, it sends a request to the RCA server 70 for fault information (fault location, fault type) related to the new (current) fault. In this case, the required fault information sent is a predetermined period during which the fault occurred (for example, 10:00-11:00). Note that if step S14 is omitted, the fault identification unit 51 will send a request to the RCA server 70 for fault information (fault location, fault type) related to the new (current) fault based on the alarm information received in step S13.

[0081] S16: The RCA server 70 sends a request to the monitoring and collection server 30 for alarm information for a predetermined period.

[0082] S17: In response to step S16, the monitoring and collection server 30 transmits alarm information for a predetermined period to the RCA server 70.

[0083] S18: The RCA server 70 analyzes the cause of alarm information for a predetermined period to determine the location and type of the failure.

[0084] S19: The RCA server 70 transmits the derived failure information (failure location, failure type) to the failure identification unit 51. As a result, the failure identification unit 51 receives the failure information (failure location, failure type) and identifies the failure information (failure location, failure type).

[0085] S20: The fault identification unit 51 outputs the identified fault information to the registration determination unit 52 either as is or converted into a data format that can be processed by the registration determination unit 52.

[0086] The above steps S11 to S20 are repeatedly executed.

[0087] <Processing of Registration Determination Unit> FIG. 13 is a flowchart showing processing performed by the registration determination unit.

[0088] S31: The registration determination unit 52 generates abstracted fault information from the fault information by classifying the fault information acquired from the fault identification unit 51 into groups that perform predetermined similar processing (perform the same role).

[0089] S32: The registration determination unit 52 searches the measure-related information management DB 61 using the generated abstracted fault information as a search key, thereby attempting to extract a corresponding case record (primary search).

[0090] S33: The registration determination unit 52 determines whether or not a single or multiple corresponding case records have been extracted in the primary search.

[0091] S34: In process S33, if the registration determination unit 52 determines that multiple or a single corresponding case record has been extracted (YES), it requests context information related to the new failure location from the monitoring and collection server 30 and acquires the context information related to the new failure location from the monitoring and collection server 30. Then, the registration determination unit 52 uses this context information to execute the pre-check for each case code.

[0092] S35: The registration determination unit 52 determines whether there is one or more case records for which the advance check has been successful (normally completed).

[0093] S36: If there is one or more case records for which the preliminary check has been successful (S35; YES), the registration determination unit 52 extracts a single predetermined case record (measure-related information) from among them in accordance with one of the above-mentioned predetermined rules α, β, and γ (secondary search), and then proceeds to processing S51, which will be described later.

[0094] On the other hand, if there is no corresponding case record at all (NO) in process S33, or if there is no case record for which the advance check has been successful (NO) in process S35, the process proceeds to process S71 described below.

[0095] <Processing of the Measure Executing Unit> FIG. 14 is a flowchart showing the processing performed by the measure executing unit.

[0096] S51: The measure execution unit 53 acquires predetermined measure-related information from the registration determination unit 52.

[0097] S52: Based on the predetermined action-related information acquired from the registration determination unit 52, the action execution unit 53 requests context information related to the new failure location from the monitoring and collection server 30, and acquires the context information related to the new failure location from the monitoring and collection server 30. The action execution unit 53 also uses the acquired context information to execute action content corresponding to the new failure location, thereby taking recovery action. Furthermore, the action execution unit 53 uses the acquired context information to execute post-check content using a command sequence corresponding to the new failure location.

[0098] S53: The measure execution unit 53 executes a post-check based on the executed post-check content, and registers the success or failure result in the "number of successes or failures" field in the predetermined case record 60.

[0099] S54: The corrective action execution unit 53 determines whether the post-checks using all assert statements in one piece of post-check information have been successful. If successful (YES), the corrective action execution device 50 automatically implements recovery measures for the new fault location. On the other hand, if unsuccessful (NO), the process proceeds to step S71.

[0100] <Processing of Registration Processing Unit> FIG. 15 is a flowchart showing the processing performed by the registration processing unit.

[0101] S71: The registration processing unit 54 transmits a notification to the input device 80 that measures are required for the new failure location. As a result, the operator y performs a pre-check, measures, and post-check for the new failure location by operating the input device 80. The operator y then inputs the pre-check information, measures, and post-check information in the same format as that recorded in the above-mentioned measure-related information management DB 61, and transmits them to the measure execution device 50.

[0102] S72: The registration processing unit 54 receives the pre-confirmation information, the action information, and the post-confirmation information from the input device 80.

[0103] S73: The registration processing unit 54 requests the monitoring and collection server 30 for context information relating to the new failure location, and acquires the context information relating to the new failure location from the monitoring and collection server 30.

[0104] S74: The registration processing unit 54 registers a new record (measure-related information) in the measure-related information management DB 61 by associating the information (fault information and abstracted fault information) obtained from the registration determination unit 52, the information received from the input device 80 (pre-confirmation information, measure information, and post-confirmation information), and the context information (case context) received from the monitoring and collection server 30 with the new record ID.

[0105] As a result of the above, even if the measure execution device 50 is unable to automatically perform recovery measures for the failed location, the NOC can have the measure execution device 50 perform recovery measures for the failed location.

[0106] [Use Case] ​​Next, an example use case of this embodiment will be described. In this use case, the same type of failure occurs three times. A hypothetical scenario and the corresponding processing will be described below. For simplicity, the following description will be given of a case in which the action execution device 50 automatically takes recovery measures for one suspected communication device. The context information used in this use case includes traffic volume, memory usage, OS version, and time (of failure occurrence).

[0107] <First failure occurrence> (Scenario) It is now 11:00, and buffer overflow alarm information has been observed multiple times for L3SW#3 in Figure 7. Traffic volume threshold exceedance alarms for L3SW#3 were observed consecutively between 10:00 and 11:00.

[0108] (Processing Contents) The fault identification unit 51 generates fault information "Buffer overflow of L3SW#3." Next, the registration determination unit 52 generates abstracted fault information "Buffer overflow of AccessL3SW." In the primary search by the registration determination unit 52, no record matching this abstracted fault information is registered in the action-related information management DB 61, so the process proceeds to the registration processing unit 54.

[0109] (Scenario) Operator y manually performs the following process. Operator y checks the traffic volume of L3SW#3 and finds that it did indeed exceed the threshold when the failure occurred, but that it has now subsided. However, Operator y finds that the memory usage of L3SW#3 remains high. Operator y also checks the OS version of L3SW#3 and finds that it is version 3.7, which has been reported to have a problem.

[0110] L3SW#3 is allowed to be stopped between 12:00 and 13:00, and for five hours at any other time. Restarting the system (a five-minute interruption in communication) will temporarily resolve the problem, but if possible, upgrading the OS (a 40-minute interruption in communication) is preferable.

[0111] Since it is now 11:00, operator y takes into consideration the allowable communication interruption time and takes the measure of restarting L3SW#3. Then, as a follow-up check, operator y sends a ping from L3SW#3 to the adjacent device and confirms communication.

[0112] (Processing content) The registration processing unit 54 creates and defines an embedded function as shown in Figure 10, and registers it as a new case record c1 in the measure-related information management DB 61 using the pre-confirmation information, measure information, and post-confirmation information created by operator y.

[0113] <Second failure occurrence> (Scenario) It is currently 12:15. Between 10:00 and 11:00, alarm information about buffer overflow was observed multiple times in L3SW#6 in Figure 10. Between 10:00 and 11:00, alarm information about traffic volume threshold being exceeded was observed consecutively in L3SW#6, but this has now subsided.

[0114] (Processing Contents) The fault information output by the fault identification unit 51 is "Buffer overflow of L3SW#6." Then, the registration determination unit 52 generates abstracted fault information "Buffer overflow of AccessL3SW." The primary search finds a case record c1 with matching abstracted fault information, but the pre-check assert statement fails in the secondary search, so the process proceeds to the registration processing unit 54.

[0115] (Scenario) Operator y manually performs the following process. That is, just like when the first failure occurred, the traffic volume of L3SW#6 did indeed exceed the threshold when the failure occurred, but has now subsided. However, memory usage of L3SW#6 remains high. Operator y checks the OS version of L3SW#6 and finds that it is the faulty version 3.7. As mentioned above, there are two options for action: reboot or upgrade the OS. Since the current time is 12:15, operator y takes into account the allowable communication interruption time and upgrades the OS, and then performs a post-event check using ping, as mentioned above.

[0116] (Processing content) The registration processing unit 54 creates and defines an embedded function as shown in Figure 10, and registers it as a new case record c2 in the measure-related information management DB 61 using the pre-confirmation information, measure information, and post-confirmation information created by operator y.

[0117] <Third failure occurrence> (Scenario) It is currently 12:15. Between 10:00 and 11:00, buffer overflow alarm information for L3SW#5 was observed multiple times. Other circumstances are the same as when the second failure occurred.

[0118] (Processing details) The same abstracted fault information is generated as when the second fault occurred, and case records c1 and c2 are extracted in the primary search. Furthermore, in the secondary search, the assert statement in the pre-check of case record c1 fails, while all assert statements in the pre-check of case record c2 succeed. Therefore, case record c2 is passed to the action execution unit 53. Then, the action execution unit 53 automatically implements the action for case record c2, i.e., an OS upgrade, and then performs a post-check and registers the result in the success / failure count field.

[0119] [Major Effects of the Embodiment] As described above, according to the present embodiment, the measure execution device 50 has the effect of being able to take measures against a newly occurring event by referring to similar events (such as the occurrence of a failure) that have occurred in the past.

[0120] [Supplementary Note] The present invention is not limited to the above-described embodiment, and may have the following configurations or processes (operations), for example.

[0121] (1) The measure execution device 50 can be realized by a computer and a program, but this program can also be recorded on a (non-transitory) recording medium or provided via a network such as the WAN 100.

[0122] (2) The processor 1004, which is hardware, may be a single processor or multiple processors.

[0123] (3) In the above embodiment, the "fault" is an example of an "event." Therefore, fault information is an example of event information, and abstracted fault information is an example of abstracted event information.

[0124] (4) The registration determination unit 52 can also be referred to as an abstraction unit, a two-stage search unit, or a pre-check unit. The action execution unit 53 can also be referred to as a post-check unit or a success / failure result registration unit.

[0125] REFERENCE SIGNS LIST 10 Communication system 30 Monitoring and collection server 50 Action execution device 51 Fault identification unit 52 Registration determination unit 53 Action execution unit 54 Registration processing unit 60 Storage unit 61 Action-related information management DB (an example of a management unit) 70 RCA server 80 Input device

Claims

1. A measure execution device that executes measures against events that occur in a communication device in a specified network, comprising: a management unit that associates and manages abstracted event information containing abstracted content of events that occurred in the past with measure information indicating measures against the abstracted content; and a measure execution unit that executes measures against the newly occurred event by using specified measure information corresponding to the specified abstracted event information indicating the abstracted content of a newly occurred event in the management unit.

2. The measure execution device according to claim 1, wherein the measure execution unit executes the measure according to the measure content corresponding to the newly occurring event based on situation information indicating the situation when the event newly occurred.

3. A measure execution device as claimed in claim 1 or 2, wherein the management unit further associates and manages pre-confirmation information indicating the content of pre-confirmation to be executed before the measure is taken, a registration determination unit which uses predetermined pre-confirmation information corresponding to the abstracted event information in the management unit to execute a pre-confirmation to determine whether the execution of the measure will be successful before executing the measure for the newly occurred event, thereby determining whether new measure information needs to be registered in the management unit, and a registration processing unit which, when the measure cannot be executed due to the pre-confirmation and therefore new measure information needs to be registered in the management unit, registers new measure information created by an operator in the management unit.

4. A measure execution device as claimed in claim 1 or 2, wherein the management unit further associates and manages post-confirmation information indicating the content of post-confirmation to be performed after the measure, and the measure execution unit, after performing a measure for the newly occurring event using predetermined post-confirmation information corresponding to the abstracted event information in the management unit, performs a post-confirmation as to whether the execution of the measure was successful, and further associates and registers the success or failure result of the measure in the management unit.

5. A method of executing measures by a measure execution device that executes measures against an event that occurs in a communication device in a specified network, wherein the measure execution device executes measures against the newly occurring event by using specified measure information corresponding to the specified abstract event information that indicates the abstract content of a newly occurring event in a management unit that associates and manages abstracted event information including abstracted content of events that occurred in the past and measure information that indicates the measures that are taken against the abstracted content.

6. A program for causing a computer to execute the method according to claim 5.

Images