Model creation device, method and program

Abstract

To provide a model creation apparatus, a method thereof, and a program therefor which can easily create a monitoring system verification model.SOLUTION: A model creation apparatus according to an embodiment of the present invention has first and second conversion processing means, and first to third creation means. The first conversion processing means converts a first definition file of a monitoring system into an alert model. The second conversion processing means converts a second definition file relating to a server apparatus and a third definition file relating to an application program to a network model. The first creation means creates an inspection formula based on the third definition file and inspection formula information. The second creation means creates state transition information based on a state transition rule, the alert model, and the network model. The third creation means creates a monitoring system verification model formed of the alert model, the network model, the inspection formula, and the state transition information.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] An embodiment of the present invention relates to a model creation device, a model creation method, and a program. [Background technology]

[0002] 2. Description of the Related Art In recent years, it has become known that various services can be provided by running a specific application program on a server device, for example.

[0003] In this case, a monitoring system is often operated that monitors the operation of the above-mentioned server device to discover failures in the IT infrastructure (infrastructure in the IT field) that realizes the server device and in the application programs that run on the server device, and outputs an alert regarding the failure. Note that an IT infrastructure failure refers to a state that affects the operation of a program, such as a depletion of resources in the IT infrastructure (server device) or a malfunction of the IT infrastructure itself. Also, an application program failure refers to a state in which the application program does not operate as expected and responds abnormally, for example.

[0004] Here, in order to realize the provision of stable services, the above-mentioned monitoring system needs to operate appropriately, and the operation of the monitoring system can be verified by inspecting a monitoring system verification model created by modeling the above-mentioned monitoring system, IT infrastructure, application programs, etc. However, it is difficult to create such a monitoring system verification model. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 5966890 Summary of the Invention [Problem to be solved by the invention]

[0006] Therefore, an object of the present invention is to provide a model creation device, method, and program that can easily create a model for verifying a surveillance system. [Means for solving the problem]

[0007] A model creation device according to an embodiment includes a first conversion processing means, a second conversion processing means, a first creation means, a second creation means, and a third creation means. The first conversion processing means executes a process of converting a first definition file of a monitoring system that monitors the operation of a server device into an alert model in which settings related to an alert output from the monitoring system are modeled. The second conversion processing means executes a process of converting a second definition file in which information about the server device is described and a third definition file in which information about an application program running on the server device is described into a network model in which settings related to a network connecting the server device and the monitoring system are modeled. The first creation means creates a check expression based on check expression information based on the third definition file and the specifications of the monitoring system. The second creation means creates state transition information indicating a plurality of states of the monitoring system that transition based on a rule related to state transition of the monitoring system, the alert model, and the network model. The third creation means creates a monitoring system verification model consisting of the alert model, the network model, the check expression, and the state transition information. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a monitoring system. [Diagram 2] FIG. 1 is a diagram showing an example of a cloud environment in which a monitoring system is deployed. [Diagram 3] FIG. 1 is a block diagram showing an example of a functional configuration of a model creating device according to an embodiment. [Figure 4]FIG. 2 is a diagram illustrating an example of a hardware configuration of a model creating apparatus. [Diagram 5] 11 is a flowchart showing an example of a processing procedure of the model creating device. [Figure 6] 13 is a flowchart showing an example of a processing procedure of an alert model conversion process. [Figure 7] FIG. 4 is a diagram showing an example of a first definition file. [Figure 8] FIG. 11 is a diagram showing an example of information acquired from a first definition file. [Figure 9] FIG. 13 is a diagram showing an example of an alert condition expression. [Figure 10] A diagram showing the results of lexical analysis performed on an alert condition expression. [Figure 11] FIG. 13 is a diagram showing an example of an alert model to which an alert condition expression has been added. [Figure 12] FIG. 13 is a diagram showing an example of a metric type. [Figure 13] A diagram showing an example of an alert model with a metric type added. [Figure 14] FIG. 13 illustrates an example of an alert model to which a service has been added. [Figure 15] FIG. 13 is a diagram showing an example of an alert model to which a threshold value has been added. [Figure 16] FIG. 13 illustrates an example of an alert model with a comparison added. [Figure 17] 11 is a flowchart showing an example of a processing procedure for a network model conversion process. [Figure 18] FIG. 11 is a diagram showing an example of a second definition file. [Figure 19] FIG. 11 is a diagram showing an example of a third definition file. [Figure 20] A diagram showing an example of aws_vpc information. [Figure 21] A diagram showing an example of aws_subnet information. [Figure 22] A diagram showing an example of aws_network_acl information. [Diagram 23] FIG. 4 is a diagram showing an example of a network table. [Figure 24]A diagram showing an example of aws_network_interface information. [Diagram 25] FIG. 4 is a diagram showing an example of instance information. [Figure 26] FIG. 11 is a diagram showing an example of information acquired from a third definition file. [Figure 27] FIG. 11 is a diagram showing an example of docker information. [Figure 28] FIG. 13 is a diagram showing an example of information indicating the state of an acl created from instance information. [Figure 29] FIG. 1 is a diagram showing an example of an instance model. [Diagram 30] FIG. 1 is a diagram showing an example of a network model. [Diagram 31] 11 is a flowchart showing an example of a processing procedure for a check-formula generating process. [Diagram 32] FIG. 4 is a diagram showing an example of a data structure of a check equation information storage unit. [Diagram 33] FIG. 13 is a diagram showing an example of check equation information acquired from a check equation information storage unit. [Diagram 34] FIG. 13 is a diagram showing an example of a check formula. [Diagram 35] FIG. 2 is a diagram showing state elements and properties of a monitoring system. [Diagram 36] FIG. 4 is a diagram showing states of the monitoring system that transition based on a first state transition rule. [Figure 37] FIG. 11 is a diagram showing states of the monitoring system that transition based on a second state transition rule. [Figure 38] FIG. 11 is a diagram showing states of the monitoring system that transition based on a third state transition rule. [Figure 39] FIG. 4 is a diagram for explaining an overview of state transition information. [Diagram 40] FIG. 1 is a diagram for explaining an overview of model checking. [Diagram 41] FIG. 1 is a diagram showing an example of a counterexample found in model checking. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment will be described with reference to the drawings. The model creation device according to this embodiment is a device having a function of creating a surveillance system verification model used to verify the operation of a surveillance system.

[0010] First, a monitoring system will be described with reference to Fig. 1. The monitoring system 10 shown in Fig. 1 is a system configured to monitor, for example, the operation of a server device 20 and output an alert regarding the operation of the server device 20 (such as a failure in an IT infrastructure that realizes the server device 20 and an application program that runs on the server device 20). The alert may be notified to a system administrator by, for example, email.

[0011] Here, the monitoring system 10 can be constructed manually, but it is generally constructed automatically by construction automation software based on a definition file. The construction automation software assumed in this embodiment includes infrastructure management software and container management software.

[0012] The infrastructure management software is software that automatically constructs an IT infrastructure (e.g., a network between the monitoring system 10 and the server device 20) for realizing the above-mentioned server device 20. Note that the server device 20 realized by the IT infrastructure may be a virtually constructed server device.

[0013] Container management software is software that uses container-based virtualization to manage containers built on an operating system (OS). Container-based virtualization is a method of computer virtualization in which a part of a running operating system is separated to prepare a dedicated area isolated from the rest, and software (application programs, etc.) is run in that area (container).

[0014] The monitoring system 10 can be automatically constructed by the above-mentioned infrastructure management software and container management software.

[0015] In the following description, it is assumed that Terraform (registered trademark) is used as the infrastructure management software, and docker-compose is used as the container management software. In this case, the definition file of the infrastructure management software (Terraform) is terraform.tf, and the definition file of the container management software (docker-compose) is docker-compose.yml. For example, terraform.tf corresponds to a definition file in which information about the server device 20 is described, and docker-compose.yml corresponds to a definition file in which information about an application program that runs on the server device 20 is described.

[0016] Here, a container environment is constructed on the server device 20, and in the container (environment), an application program 21 to be monitored, a monitoring agent (container) 22, and a monitoring agent (server) 23 run.

[0017] The application program 21 is software that runs on the server device 20 to provide a given service.

[0018] The monitoring agent publishes metrics data so that the integrated monitoring software 12 can collect the metrics data. The metrics data of the monitoring agent includes resource data and application data. The monitoring agent 22 (container) publishes the resource usage status of container applications. The monitoring agent (server) 23 publishes the resource status of EC2 instances. If an application has a monitoring agent function, it also publishes metrics data.

[0019] The monitoring agent (server) 23 is software (application program) for collecting resource data of the server device 20. In addition to resource monitoring, the monitoring agent (server) 23 may also monitor services. In this case, the monitoring agent (server) 23 publishes metrics data. The metrics data published by the monitoring agent (server) 23 includes resource data and the operating status of services. In recent years, applications themselves have the function of publishing metrics data.

[0020] The monitoring agent (container) 22 and the monitoring agent (server) 23 operate as parts of the monitoring system 10 in order to realize monitoring by the monitoring system 10.

[0021] On the other hand, a service monitoring agent 11 operates in a service monitoring environment on the monitoring system 10. The service monitoring agent 11 is a type of monitoring agent such as a monitoring agent (container) 22 and a monitoring agent (server) 23. The service monitoring agent 11 is software (application program) for collecting data on the operating status of a service provided by the operation of the above-mentioned application program 21.

[0022] Furthermore, to realize monitoring by the monitoring system 10, integrated monitoring software 12 is required. The integrated monitoring software 12 operates in cooperation with a monitoring database 13 that searches for monitoring data obtained via each of the monitoring agents described above in the monitoring system environment and stores the monitoring data. Specifically, the integrated monitoring software 12 has a function of accessing each monitoring agent to collect data, a function of managing time-series data (monitoring data), a function of defining rules related to alerts, and the like.

[0023] In the following explanation, it is assumed that Prometheus (registered trademark) is used as the integrated monitoring software 12. In this case, the definition file of the integrated monitoring software 12 (a definition file in which rules related to alerts in monitoring are defined) is rules.yml. Note that this rules.yml corresponds to the definition file of the monitoring system 10 (a formal file that defines the monitoring targets of the monitoring system, thresholds for outputting alerts, and the like).

[0024] In this embodiment, it is assumed that the monitoring system 10 is arranged in a cloud environment such as an AWS (Amazon Web Service) (registered trademark) network, for example, as shown in Fig. 2. Specifically, in the example shown in Fig. 2, it is assumed that the server device 20 shown in Fig. 1 includes a server device 20a that operates in a private environment and a server device 20b that operates in a public environment. The private environment refers to a network environment that cannot be accessed from an external network, and the public environment refers to a network environment that can be accessed from an external network.

[0025] Incidentally, in order to verify the operation of the monitoring system 10 as described above, it is useful to create a monitoring system verification model; however, the system administrator of the monitoring system 10 often does not understand the configuration of the increasingly large-scale cloud system (a system realized by the monitoring system 10, IT infrastructure, and application program 21, etc.), making it difficult to extract the components or constraints of the cloud system required to verify the operation of the monitoring system 10 and create a monitoring system verification model.

[0026] In addition, in order to create a model for verifying the monitoring system, it is necessary to analyze the operation of the cloud system, which takes time. Furthermore, if a defect is discovered during the actual operation of the monitoring system 10, rework costs will be incurred.

[0027] Therefore, in this embodiment, a model creating device capable of automatically creating a surveillance system verification model is provided.

[0028] Fig. 3 is a block diagram showing an example of a functional configuration of the model creating device according to this embodiment. As shown in Fig. 3, the model creating device includes an alert model conversion processing unit 31, a network model conversion processing unit 32, a check-expression information storage unit (check-expression information DB) 33, a check-expression creating unit 34, a state transition rule storage unit (state transition rule DB) 35, a state transition creating unit 36, a model creating unit 37, a check-expression information registration unit 38, and a state transition rule registration unit 39.

[0029] The alert model conversion processing unit 31 acquires a definition file of the monitoring system 10 (integrated monitoring software 12). The definition file of the monitoring system 10 is, for example, a Prometheus yml file (rules.yml). The definition file of the monitoring system 10 is acquired, for example, from an external device different from the model creation device 30, but may be managed within the model creation device 30. Also, rules.yml is a file in which alert rules are described, but the definition file of the monitoring system 10 further includes a file in which a monitoring system application as shown in FIG. 19 is described and a file in which the infrastructure of the monitoring system 10 as shown in FIG. 18 is described.

[0030] The alert model conversion processing unit 31 executes a process of converting the acquired definition file of the monitoring system 10 into an alert model. Note that the alert model is created, for example, by modeling the setting contents related to the alerts output from the monitoring system 10.

[0031] The network model conversion processing unit 32 acquires the above-mentioned infrastructure management software definition file (definition file describing information related to the server device 20) and container management software definition file (definition file describing information related to an application program running on the server device 20). The infrastructure management software definition file is, for example, a Terraform tf file (terraform.tf), and the container management software definition file is, for example, a docker-compose yml file (docker-compose.yml). The infrastructure management software definition file and the container management software definition file are acquired from, for example, an external device different from the model creation device 30, but may be managed within the model creation device 30.

[0032] The network model conversion processing unit 32 executes a process of converting the acquired definition file of the infrastructure management software and the definition file of the container management software into a network model. Note that the network model is created by modeling the setting contents related to the network connecting the monitoring system 10 and the server device 20 (the server devices 20a and 20b), for example.

[0033] The check equation information storage unit 33 stores check equation information based on, for example, the specifications (such as functional requirements described in the specifications) of the monitoring system 10. The check equation information storage unit 33 is configured to accumulate check equation information related to past check equations.

[0034] The check-equation creating unit 34 acquires a definition file of the container management software, and creates a check equation based on the definition file and the check equation information stored in the check-equation information storage unit 33. The check equation corresponds to a logical expression that defines the properties of the system based on the specifications of the monitoring system 10.

[0035] The state transition rule storage unit 35 stores, for example, rules (hereinafter referred to as state transition rules) related to state transitions of the monitoring system 10. As described above, the monitoring agent (container) 22 and the monitoring agent (server) 23 operate as parts of the monitoring system 10, and therefore, the state transitions of the monitoring system 10 in this embodiment also include state transitions of the server device (private environment) 20a and the server device (public environment) 20b.

[0036] The state transition creation unit 36 ​​creates state transition information indicating multiple states of the monitoring system 10 that transition based on the state transition rules stored in the state transition rule storage unit 35, the alert model converted from the definition file of the monitoring system 10 by the alert model conversion processing unit 31, and the network model converted from the definition file of the infrastructure management software and the definition file of the container management software by the network model conversion processing unit 32.

[0037] The model creation unit 37 creates a monitoring system verification model consisting of an alert model converted from the definition file of the monitoring system 10 by the alert model conversion processing unit 31, a network model converted from the definition file of the infrastructure management software and the definition file of the container management software by the network model conversion processing unit 32, the check equation created by the check equation creation unit 34, and the state transition information created by the state transition creation unit 36. The monitoring system verification model created by the model creation unit 37 is also called a formal verification model.

[0038] The surveillance system verification model created by the model creation unit 37 is stored in the model storage unit 40.

[0039] The check equation information registration unit 38 creates check equation information based on the above-mentioned specification of the monitoring system 10 , and registers the created check equation information in the check equation information storage unit 33 .

[0040] The state transition rule registration unit 39 creates state transition rules based on, for example, the specifications of the monitoring system 10 and the check expression information stored in the check expression information storage unit 33, and registers the created state transition rules in the state transition rule storage unit 35.

[0041] The (information about) the specifications of the above-mentioned monitoring system 10 may be managed, for example, in an external device different from the model creating device 30, or may be managed within the model creating device 30.

[0042] When the monitoring system verification model is stored in the model storage unit 40 as described above, the model checking unit 50 checks the monitoring system verification model. The inspection of the monitoring system verification model is performed using a model checking tool for comprehensively checking the state of the model and confirming the properties of the system. By checking the monitoring system verification model in this manner, the operation of the monitoring system 10 can be verified. The inspection results by the model checking unit 50 (e.g., the malfunction state of the monitoring system 10, etc.) are stored (registered) in the inspection result storage unit 60.

[0043] 3, for example, the check-equation information registration unit 38 and the state transition rule registration unit 39 have been described as being included in the model creating device 30, but a part or all of the check-equation information registration unit 38 and the state transition rule registration unit 39 may be arranged outside the model creating device 30. Also, a configuration may be adopted in which the check-equation information storage unit 33 is arranged in an external device of the model creating device 30, and the check-equation creation unit 34 acquires the check-equation information from the external device. Similarly, a configuration may be adopted in which the state transition rule storage unit 35 is arranged in an external device of the model creating device 30, and the state transition creation unit 36 ​​acquires the state transition rules from the external device.

[0044] Furthermore, in this embodiment, it is assumed that the model storage unit 40, the model checking unit 50, and the inspection result storage unit 60 are located outside the model creation device 30, but some or all of the model storage unit 40, the model checking unit 50, and the inspection result storage unit 60 may be included in the model creation device 30.

[0045] Fig. 4 shows an example of a hardware configuration of the model creating device 30 shown in Fig. 3. As shown in Fig. 4, the model creating device 30 includes a CPU 301, a non-volatile memory 302, a main memory 303, a communication device 304, and the like.

[0046] The CPU 301 is a processor that controls the operation of each component in the model creating device 30. The CPU 301 executes various programs loaded from the non-volatile memory 302, which is a storage device, to the main memory 303. These programs include an operating system and a program for creating the above-mentioned monitoring system verification model (hereinafter referred to as a model creating program), etc.

[0047] The communication device 304 is, for example, a device configured to perform communication with an external device other than the model creation device 30 .

[0048] In FIG. 4, only a CPU 301, a non-volatile memory 302, a main memory 303, and a communication device 304 are shown, but the model creation device 30 may further include other storage devices such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive), or may further include other devices.

[0049] In this embodiment, some or all of the alert model conversion processing unit 31, network model conversion processing unit 32, check equation creation unit 34, state transition creation unit 36, check equation information registration unit 38, and state transition rule registration unit 39 included in the model creating device 30 are realized by the CPU 301 (i.e., the model creating device 30) executing a model creation program, that is, by software. Some or all of these units 31, 32, 34, 36, 38, and 39 may be realized by hardware such as an IC (Integrated Circuit), or may be realized by a combination of software and hardware.

[0050] In this embodiment, the check equation information storage unit 33 and the state transition rule storage unit 35 included in the model generating device 30 are realized by the non-volatile memory 302 or other storage devices.

[0051] 3 is included in the model creating device 30, the model checking unit 50 may be realized by executing a model creation program by the CPU 301. Also, when the model storage unit 40 and the inspection result storage unit 60 shown in FIG. 3 are included in the model creating device 30, the model storage unit 40 and the inspection result storage unit 60 may be realized by the non-volatile memory 302 or other storage devices.

[0052] An example of a processing procedure of the model creating device 30 according to this embodiment will be described below with reference to the flowchart of FIG.

[0053] First, when constructing the monitoring system 10, a system administrator (developer) of the monitoring system 10 creates a design document for the monitoring system 10. The design document for the monitoring system 10 corresponds to a document in which the design of the IT infrastructure and system configuration diagram of the monitoring system 10, for example, is described.

[0054] Generally, the monitoring system 10 is constructed based on a design document for the monitoring system 10 prepared by the above-mentioned system administrator, but in this embodiment, after the design document for the monitoring system 10 is prepared, and before the monitoring system 10 is constructed, a monitoring system verification model is created to verify the operation of the monitoring system 10.

[0055] In this case, the alert model conversion processing unit 31 inputs a definition file of the monitoring system 10 based on the design document of the monitoring system 10 described above, and executes the alert model conversion processing (step S1). When the processing of step S1 is executed, the alert model conversion processing unit 31 outputs an alert model. The details of the alert model conversion processing will be described later.

[0056] The network model conversion processing unit 32 also inputs a definition file of the infrastructure management software and a definition file of the container management software, and executes a network model conversion process (step S2). When the process of step S2 is executed, the network model conversion processing unit 32 outputs a network model. The network model conversion process will be described in detail later.

[0057] The order of the processes of steps S1 and S2 may be changed, and the processes of steps S1 and S2 may be executed in parallel.

[0058] Next, the check-equation creating unit 34 inputs the definition file of the container management software and the check-equation information stored in the check-equation information storage unit 33, and executes a check-equation creating process (step S3). When the process of step S3 is executed, the check-equation creating unit 34 outputs a check equation. The check-equation creating process will be described later in detail.

[0059] The state transition creation unit 36 ​​inputs the alert model output from the alert model conversion processing unit 31, the network model output from the network model conversion processing unit 32, and the state transition rules stored in the state transition rule storage unit 35, and executes a state transition creation process (step S4). When the process of step S4 is executed, the state transition creation unit 36 ​​outputs state transition information. The state transition creation process will be described in detail later.

[0060] The order of the processes in steps S3 and S4 may be changed, and the processes in steps S3 and S4 may be executed in parallel.

[0061] Next, the model creation unit 37 creates a monitoring system verification model consisting of the alert model output from the alert model conversion processing unit 31, the network model output from the network model conversion processing unit 32, the check formula output from the check formula creation unit 34, and the state transition information output from the state transition creation unit 36 ​​(step S5). Note that in step S5, a process is executed to automatically convert the above-mentioned alert model, network model, check formula, and state transition information into a model description language.

[0062] The monitoring system verification model created in step S5 described above is inspected to verify the operation of the monitoring system 10. In this embodiment, based on the inspection result of this monitoring system verification model, automatic construction of the monitoring system 10 or re-creation of a design document for the monitoring system 10 is performed. Specifically, if it is confirmed that the monitoring system 10 operates normally as a result of inspecting the monitoring system verification model, automatic construction of the monitoring system 10 may be performed, and if it is confirmed that the monitoring system 10 does not operate normally (a defect occurs), a design document for the monitoring system 10 may be re-created by a system administrator. Details of the inspection of this monitoring system verification model (model inspection) will be described later.

[0063] Next, an example of the processing procedure of the above-mentioned alert model conversion processing (the processing of step S1 shown in FIG. 5) will be described with reference to the flowchart of FIG.

[0064] The alert model conversion process in this embodiment is a process of converting a definition file of the monitoring system 10 into an alert model in which the setting contents related to an alert (warning, notification, etc.) output from the monitoring system 10 are modeled.

[0065] First, the alert model conversion processing unit 31 inputs a Prometheus yml file (rule.yml) as a definition file of the monitoring system 10 (step S11). Note that Fig. 7 shows an example of the Prometheus yml file input in step S11. Hereinafter, the Prometheus yml file shown in Fig. 7 is referred to as a first definition file.

[0066] Next, the alert model conversion processing unit 31 searches for, for example, a rule from the first definition file input in step S11 (step S12). Since the rule is defined in yml format, the data is managed as nested structure data using a hash expressed in the format of "key:value". In this case, the alert model conversion processing unit 31 uses the rules in the first definition file as a key to obtain information (value) of the rules. Note that, according to the example shown in FIG. 7, there are multiple rules in the first definition file, but here, the explanation will be given assuming that the information shown in FIG. 8 has been obtained.

[0067] The alert model conversion processing unit 31 acquires an alert condition formula from the acquired information shown in Fig. 8 (step S13). Fig. 9 shows the alert condition formula acquired from the information shown in Fig. 8.

[0068] The alert model conversion processing unit 31 performs lexical analysis on the alert condition formula acquired in step S13 (step S14). Note that Fig. 10 shows the result of lexical analysis performed in step S14. When the process of step S14 is executed, the alert condition formula is divided into a plurality of tokens (the smallest unit of a character string).

[0069] The alert model in this embodiment is created based on the result of lexical analysis performed on the above-mentioned alert condition formula. In this case, an alert model to which an alert condition formula as shown in FIG. 11 is added is prepared.

[0070] Next, the alert model conversion processing unit 31 acquires (a token containing) a metric type from the result of the lexical analysis performed in step S14 (step S15). The metric type corresponds to, for example, a type of measured value obtained by monitoring by the monitoring system 10. The metric type acquired in step S15 is assumed to be predetermined, for example, as shown in FIG. 12. That is, in step S15, a metric type that matches the metric type shown in FIG. 12 is acquired from the result of the lexical analysis performed in step S14. The predetermined metric type (information on) shown in FIG. 12 may be managed within the model creation device 30, or may be acquired from an external device different from the model creation device 30.

[0071] The metric type obtained in step S14 is added to the alert model shown in Fig. 11. In many cases, a metric type is created as a combination of an agent type and a monitored object. According to the example shown in Fig. 10, "node_filesystem_free_bytes" is obtained as the metric type in step S15, and an alert model shown in Fig. 13 with this metric type added is created. Here, it is assumed that the metric type is the agent name and monitored object, but the metric type may be only the monitored object name.

[0072] Next, the alert model conversion processing unit 31 acquires the service (the token in which the service is described) from the result of the lexical analysis performed in step S14 (step S16).

[0073] The service acquired in step S16 is added to the alert model shown in Fig. 13. According to the example shown in Fig. 10, "gitlab" is acquired as a service in step S16, and the alert model shown in Fig. 14 with the service added is created. Note that, as described above, a service is provided by the operation of an application program, and in step S16, the application program name is acquired as the service.

[0074] Moreover, the alert model conversion processing unit 31 acquires the threshold value (the token in which the threshold value is written) from the result of the lexical analysis performed in step S14 (step S17).

[0075] The threshold value acquired in step S17 is added to the alert model shown in Fig. 14. According to the example shown in Fig. 10, "1024*1024*1024*1.5 (=1.5 GB)" is acquired as the threshold value from the result of lexical analysis performed in step S14, and the alert model shown in Fig. 15 with the threshold value added is created.

[0076] Furthermore, the alert model conversion processing unit 31 acquires (a token containing) comparison from the result of the lexical analysis performed in step S14 (step S18).

[0077] The comparison obtained in step S18 is added to the alert model shown in Fig. 15. According to the example shown in Fig. 10, "<" is obtained as a comparison from the result of lexical analysis performed in step S14, and the alert model shown in Fig. 16 with the comparison added is created.

[0078] As described above, the processes of steps S15 to S18 are executed based on the result of the lexical analysis performed in step S14, and an alert model shown in FIG. 16 is created (step S19).

[0079] When the process of step S19 is executed, it is determined whether or not the processes from step S12 onwards have been executed for all the rules in the first definition file (step S20).

[0080] If it is determined that the process has not been executed for all rules (NO in step S20), the process returns to step S12 and is repeated. In this case, a rule for which the above-mentioned process has not been executed is searched for in step S12, and the process from step S13 onward is executed.

[0081] On the other hand, if it is determined that the processing has been performed for all the rules (YES in step S20), the alert model conversion processing is terminated.

[0082] Next, an example of the processing procedure of the above-mentioned network model conversion processing (the processing of step S2 shown in FIG. 5) will be described with reference to the flowchart of FIG.

[0083] The network model conversion process in this embodiment is a process of converting the definition file of the infrastructure management software and the definition file of the container management software into a network model in which the setting contents related to the network connecting the monitoring system 10 and the server device 20 are modeled.

[0084] First, the network model conversion processing unit 32 inputs a Terraform tf file (terraform.tf) as a definition file of the infrastructure management software (step S21). FIG. 18 shows an example of the Terraform tf file input in step S21. According to the Terraform tf file shown in FIG. 18, information such as the network environment, memory, and disk capacity can be obtained from an EC2 (Amazon Elastic Compute Cloud) instance definition. Hereinafter, the Terraform tf file shown in FIG. 18 is referred to as a second definition file.

[0085] The network model conversion processing unit 32 also inputs a docker-compose yml file (docker-compose.yml) as a definition file of the container management software (step S22). FIG. 19 shows an example of the docker-compose yml file input in step S22. The docker-compose yml file shown in FIG. 19 indicates that cadvidsor, node_expoter, and blackbox expoter are used as monitoring agents of the monitoring system 10. Hereinafter, the docker-compose yml file shown in FIG. 19 is referred to as a third definition file.

[0086] Next, the network model conversion processing unit 32 acquires information on aws_vpc from the second definition file input in step S21 (step S23). The information on aws_vpc indicates, for example, the AWS network space in which the monitoring system 10 is placed. According to the example shown in Fig. 18, for example, the information on aws_vpc shown in Fig. 20 is acquired in step S23.

[0087] The network model conversion processing unit 32 also acquires information about aws_subnet from the second definition file input in step S21 (step S24). The information about aws_subnet represents a partial network space (subnetwork) in the AWS network described above. According to the example shown in FIG. 18, the information about aws_subnet shown in FIG. 21 is acquired in step S24.

[0088] Furthermore, the network model conversion processing unit 32 acquires information on aws_network_acl from the second definition file input in step S21 (step S25). The information on aws_network_acl indicates a port (port number) that can access the network space. According to FIG. 18, in step S25, for example, the information on aws_network_acl shown in FIG. 22 is acquired.

[0089] Next, the network model conversion processing unit 32 creates a network table based on the aws_subnet information acquired in step S24 and the aws_network_acl information acquired in step S25 (step S26).

[0090] The network table corresponds to a table that defines whether or not access between subnetworks is possible. In step S26, a network table is created that holds the information of aws_network_acl acquired in step S25 for each combination of subnetworks represented by the information of aws_subnet acquired in step S24. Note that FIG. 23 shows an example of the network table created in step S26.

[0091] Next, the network model conversion processing unit 32 acquires instance information from the second definition file input in step S21 (step S27).

[0092] Here, FIG. 18 above shows a part of the second definition file (Terraform tf file), and the second definition file further includes the information of aws_network_interface shown in FIG. 24.

[0093] In this case, the network model conversion processing unit 32 acquires the information of aws_network_interface shown in FIG. 24 from the second definition file, and acquires the information of aws_instance having the same ID as the instance of the attachment in the acquired information (aws_instance.ec203.id in this case).

[0094] Furthermore, the network model conversion processing unit 32 acquires, as the instance information, the information of the security group having the same ID as the ID of vpc_security_group_ids (here, aws_security_group.gitlab.id) in the acquired aws_instance information. FIG. 25 is an example of the instance information (security group information) acquired in step S27. This instance information may be included in the second definition file, or may be information included in a file other than the second definition file (that is, the information of aws_network_interface shown in FIG. 24). Note that an instance has various information such as a virtual CPUU, a disk, and an IP address, but in this embodiment, information on the network to which the instance belongs and the security group of the instance is used. A security group is capable of controlling access and a port of an output destination. An instance may have multiple security groups. A security group has a concept similar to a firewall.

[0095] Next, the network model conversion processing unit 32 acquires docker information from the third definition file input in step S22 (step S28).

[0096] In this case, the network model conversion processing unit 32 acquires the information shown in Fig. 26 from the third definition file. When the labels monitor.service and monitor.network are present in the information acquired from the third definition file as shown in Fig. 26, the network model conversion processing unit 32 can acquire the value of the monitor.service (gitlab in this case) and the value of the monitor.network (aws-public in this case) as information on the monitoring target. In addition, the network model conversion processing unit 32 can acquire the value of the labels monitor.agent in the information acquired from the third definition file as information on the monitoring agent. Here, cadvisor and node_exporter operating in the container environment of the server device 20 are acquired as information on the monitoring agent.

[0097] Although the third definition file (docker-compose yml file) shown in Figure 19 above does not describe (information about) labels, docker-compose has a function that allows you to describe (add) labels. Adding such labels can facilitate model conversion (network model conversion processing). The labels of an application describe information about the monitoring agent and the monitoring system. The agent of the monitoring system describes information about the monitored object and the monitoring system. This makes it easy to link the monitored object with the monitoring system.

[0098] As a result, the network model conversion processing unit 32 can acquire information indicating the state of the Docker network shown in Fig. 27 as Docker information. The Docker information acquired in this way includes information on the monitored object (gitlab and aws-public) and information on the monitoring agent (cadvisor and node_exporter). In addition, in the information indicating the state of the Docker network, for example, the port (number) for accessing the application program (the monitored application program and the monitoring agent) described in the "to" column is described in the "from" column.

[0099] Next, the network model conversion processing unit 32 creates an instance model based on the instance information (security group information) acquired in step S27 and the Docker information (information indicating the Docker network state) acquired in step S28 (step S29). In this case, the network model conversion processing unit 32 creates information indicating the ACL state (a set of ACL states) from the instance information as shown in FIG. 28, and creates an instance model including the information indicating the ACL state and the above-mentioned information indicating the Docker network state (Docker information). The instance model further includes information regarding exporters (a set of exporters). This information regarding exporters includes monitoring agents (cadvisor and node_exporter), metric types and thresholds corresponding to the monitoring agents, etc.

[0100] Fig. 29 shows an example of an instance model created in step S29. The information about the exporter shown in Fig. 29 includes nulls corresponding to the monitoring agents and metric types, and if it is found that the monitoring system can access the monitoring agents, the nulls are updated to thresholds corresponding to the monitoring agents and metric types. Whether or not access is possible can be determined by the network model.

[0101] Next, the network model conversion processing unit 32 creates a network model (AWS network model) including the network table created in step S26 and the instance model created in step S29 (step S30).

[0102] Here, Fig. 30 shows an example of a network model created in step S30. The network model includes an instance model created for each EC2 instance (application program). Specifically, an instance model is created for each coded EC2.

[0103] Next, an example of the processing procedure of the above-mentioned check-formula generating process (the processing of step S3 shown in FIG. 5) will be described with reference to the flowchart in FIG.

[0104] First, the test-formula creation unit 34 inputs a docker-compose yml file as a definition file of the container management software (step S31). Here, it is assumed that the docker-compose yml file shown in FIG. 19 with the above-mentioned label information added thereto (third definition file) has been input.

[0105] Next, the check-expression creation unit 34 acquires the information shown in Fig. 26 from the third definition file input in step S31. The check-expression creation unit 34 acquires the value of monitor.service (gitlab in this case) from the information acquired from the third definition file in this way as information on the monitoring target, and acquires the values ​​of monitor.agent (cadvisor and node_exporter) from the information as information on the monitoring agent (step S32).

[0106] Here, in the check-equation creation process, a check equation is created based on the check-equation information stored in the check-equation information storage unit 33. It is assumed that the check-equation information has been created (registered) in advance based on the specifications of the monitoring system 10.

[0107] In this embodiment, the specifications for the monitoring system 10 include descriptions of service monitoring items (e.g., port, network path and process monitoring, etc.) for grasping the operating status of services provided by the operation of the application program 21, and resource monitoring items (e.g., CPU usage, memory usage, load average, disk I / O usage, disk usage and network bandwidth usage, etc.) for monitoring changes in resources. The service monitoring items depend on the service being monitored. On the other hand, the resource monitoring items depend on the virtual hardware, but are often determined in advance, for example.

[0108] Here, Fig. 32 shows an example of a data structure of the check equation information storage unit 33. As shown in Fig. 32, a plurality of pieces of check equation information are stored in the check equation information storage unit 33, and each of the plurality of pieces of check equation information includes a monitoring type for classifying a monitoring target, an agent type representing a monitoring agent used, a service type representing an application program of the monitoring target (a service provided by the operation of the application program), a metric type determined by the monitoring type and the monitoring agent, and a threshold value and a comparison depending on the monitoring target.

[0109] Returning to FIG. 31 again, the check-equation creating unit 34 acquires check equation information from the check equation information storage unit 33 based on the information on the monitoring target and the monitoring agent acquired in step S32 (step S33).

[0110] According to the example shown in Fig. 26, gitlab is acquired as the information on the monitoring target, and cadvisor and node_exporter are acquired as the information on the monitoring agent in step S32. The check-formula creation unit 34 acquires the check-formula information shown in Fig. 33 from the check-formula information storage unit 33 shown in Fig. 32 based on the information on the monitoring target and the information on the monitoring agent. Here, it is assumed that the specifications of the monitoring system 10 require acquisition of check-formula information whose monitoring types are memory, httpt, and disk.

[0111] In addition, when the check equation information shown in FIG. 33 is obtained, it can be seen that the memory value can be obtained from cadvisor, and the memory and disk values ​​can be obtained from node_exporter.

[0112] Next, the check-equation creating unit 34 creates a check equation based on the check equation information acquired in step S33 (step S34). Note that Fig. 34 shows an example of a check equation created based on the check equation information shown in Fig. 33. The check equation shown in Fig. 34 is created by expressing the check equation information shown in Fig. 33 in temporal logic.

[0113] Next, the above-mentioned state transition creation process (the process of step S4 shown in FIG. 5) will be described. First, FIG. 35 is a diagram showing elements and properties that the monitoring system 10 has.

[0114] 35, the elements of the monitoring system 10 include a service monitoring agent 11 that operates in a service monitoring environment constructed on the monitoring system 10, and integrated monitoring software 12 that operates in a monitoring system environment constructed on the monitoring system 10. The elements of the monitoring system 10 also include a monitoring agent (container) 22 and a monitoring agent (server) 23 that operate in a container environment constructed on a server device 20 (server device 20a that operates in a private environment or server device 20b that operates in a public environment).

[0115] The properties correspond to the state of the monitoring system 10, and the properties of the service monitoring agent 11 include "over threshold", "within threshold" and "unacquirable". Unacquirable is a state in which the monitoring system cannot connect to the monitoring agent, and the metric value becomes null. For "over threshold" and "within threshold", the metric values ​​are determined by the alert rule. The same is true for the properties of the monitoring agent (container) 22 and the monitoring agent (server) 23. Meanwhile, the properties of the integrated monitoring software 12 include "alert present" and "no alert present".

[0116] Here, in the state transition creation process, state transition information is created based on the state transition rules stored in the state transition rule storage unit 35. Note that the state transition rule information is assumed to be created (registered) in advance based on, for example, the check expression information stored in the check expression information storage unit 33 and the specifications of the audit system 10.

[0117] The state transition rules will be described below. Here, the first to third state transition rules will be described.

[0118] First, in the first state transition rule, when the monitoring system 10 can access the monitoring agent, a state transition (transition) can occur between the property "over threshold" and the property "within threshold."

[0119] FIG. 36 shows the state of the monitoring system 10 (here, the monitoring agent (container) 22) that transitions based on the first state transition rule.

[0120] Specifically, in step S41, for example, it is indicated that the state of the monitoring agent (container) 22 transitions from the property "within threshold" to the property "over threshold". Furthermore, in step S42, for example, it is indicated that the state of the monitoring agent (container) 22 transitions from the property "over threshold" to the property "over threshold". Moreover, in step S43, for example, it is indicated that the state of the monitoring agent (container) 22 transitions from the property "over threshold" to the property "within threshold".

[0121] On the other hand, in step S44, for example, it is indicated that the state of the monitoring agent (container) 22 transitions from the property "within threshold" to the property "within threshold". Furthermore, in step S45, for example, it is indicated that the state of the monitoring agent (container) 22 transitions from the property "within threshold" to the property "over threshold". Also, in step S46, for example, it is indicated that the state of the monitoring agent (container) 22 transitions from the property "within threshold" to the property "within threshold".

[0122] Next, in the second state transition rule, when the monitoring system 10 cannot access the monitoring agent, the state transitions from the property "unacquirable" to the property "unacquirable".

[0123] FIG. 37 shows the state of the monitoring system 10 (here, the monitoring agent (container) 22) that transitions based on the second state transition rule.

[0124] Specifically, in step S51, for example, it is indicated that the state of the monitoring agent (container) 22 transitions from the property "unacquirable" to the property "unacquirable".

[0125] Furthermore, in the third state transition rule, the state (alert) of the monitoring system 10 (integrated monitoring software 12) can transition (change) between the property "alert present" and the property "no alert present."

[0126] FIG. 38 shows the states of the integrated monitoring software 12 that transition based on the third state transition rule.

[0127] Specifically, in step S61, for example, a transition of the state of the monitoring agent (container) 22 from the property "within threshold" to the property "over threshold" indicates that the state of the integrated monitoring software 12 transitions from the property "no alert" to the property "alert present". Furthermore, in step S62, for example, a transition of the state of the monitoring agent (container) 22 from the property "over threshold" to the property "over threshold" indicates that the state of the integrated monitoring software 12 transitions from the property "alert present" to the property "alert present". In addition, in step S63, for example, a transition of the state of the monitoring agent (container) 22 from the property "over threshold" to the property "within threshold" indicates that the state of the integrated monitoring software 12 transitions from the property "alert present" to the property "no alert".

[0128] On the other hand, in step S64, for example, a transition of the state of the monitoring agent (container) 22 from the property "within threshold" to the property "within threshold" indicates that the state of the integrated monitoring software 12 transitions from the property "no alert" to the property "no alert". Furthermore, in step S65, for example, a transition of the state of the monitoring agent (container) 22 from the property "within threshold" to the property "over threshold" indicates that the state of the integrated monitoring software 12 transitions from the property "no alert" to the property "alert". Also, in step S66, for example, a transition of the state of the monitoring agent (container) 22 from the property "within threshold" to the property "within threshold" indicates that the state of the integrated monitoring software 12 transitions from the property "no alert" to the property "no alert".

[0129] Although an example of a state transition of the monitoring system 10 has been described in Figures 36 to 38, the state transition creation unit 36 ​​can create state transition information indicating various states (all patterns of the states) of the monitoring system 10 (the service monitoring agent 11, the integrated monitoring software 12, the monitoring agent (container) 22, and the monitoring agent (server) 23) that transition based on the above-mentioned first to third state transition rules, as shown, for example, in Figure 39.

[0130] In this embodiment, the results of the above-mentioned alert model conversion process, network model conversion process, check formula creation process, and state transition creation process are automatically converted into a model description language, thereby making it possible to create a monitoring system verification model (a model described in a model description language) consisting of an alert model, a network model, a check formula, and state transition information.

[0131] In this embodiment, the operation of the monitoring system 10 can be verified by checking the monitoring system verification model created as described above. Below, a brief outline of checking the monitoring system verification model (model checking) will be given.

[0132] Model checking is performed based on the alert model, network model, check formula, and state transition information that constitute the monitoring system verification model, and as shown in Fig. 40, in this model checking, a state obtained from a check formula based on the specifications of the monitoring system 10 is compared with a state obtained from an alert model and a network model based on a definition file (design document) of the monitoring system 10 according to (each state indicated by) the state transition information. This makes it possible to discover a state (counterexample) that does not match between the specifications and design document of the monitoring system 10.

[0133] Whether or not the monitoring agents (service monitoring agent 11, monitoring agent (container) 22, and monitoring agent (server) 23) can be accessed can be determined based on the above-mentioned network model. Also, the threshold value can be defined from the above-mentioned alert model and check formula. Furthermore, the state (alert state) of the integrated monitoring software 12 can be determined based on the above-mentioned alert model.

[0134] Here, Fig. 41 shows an example of a counterexample found in model checking. In the example shown in Fig. 41, the specifications (checking formula) of the monitoring system 10 require that 2 GB or more of disk space be reserved at all times, but the definition file (alert model) of the monitoring system 10 is designed to output an alert when the capacity is less than 1.5 GB (i.e., there are cases where an alert is not output even if the capacity is less than 2 GB), so a counterexample was found.

[0135] In this embodiment, when a counterexample is found in this way, the system administrator is notified of the discovery of the counterexample (i.e., a malfunction of the monitoring system 10). In this case, the system administrator can re-create the design document for the monitoring system 10 according to the notified counterexample.

[0136] On the other hand, if no counterexample is found in model checking, construction automation software (infrastructure management software and container management software) is launched, for example, in response to instructions from a system administrator, and the monitoring system 10 is automatically constructed based on the design document for the monitoring system 10 created by the system administrator.

[0137] As described above, in this embodiment, an alert model conversion process is executed to convert a definition file (first definition file) of the monitoring system 10 that monitors the operation of the server device 20 (i.e., the application program 21) into an alert model in which the setting contents related to the alert output from the monitoring system 10 are modeled, and a network model conversion process is executed to convert a definition file (second definition file) in which information about the server device 20 is described and a definition file (third definition file) in which information about the application programs (application program 21, monitoring agents 22 and 23, etc.) that operate on the server device 20 into a network model in which the setting contents related to the network connecting the server device 20 and the monitoring system 10 are modeled. Also, in this embodiment, a check formula is created based on the check formula information based on the above-mentioned third definition file and the specifications of the monitoring system 10, and state transition information indicating a plurality of states of the monitoring system 10 that transition based on a state transition rule, an alert model, and a network model related to the state transition of the monitoring system 10 is created. In this embodiment, a monitoring system verification model consisting of the above-mentioned alert model, network model, check formula, and state transition information is automatically created. The surveillance system verification model is created, for example, by decoding a model description language.

[0138] In this embodiment, the above-described configuration makes it possible to easily create a surveillance system verification model.

[0139] Incidentally, although a monitoring system verification model (formal verification model) is useful for verifying the operation of the monitoring system 10, the creation of the monitoring system verification model generally involves a personal element. In other words, a wide variety of monitoring system verification models are created depending on the idea and target system, and the operation of the monitoring system 10 is verified (software verification) using the monitoring system verification model (and constraints) devised by an individual, so the verification depends on the ability of the individual.

[0140] In contrast, in this embodiment, a model for verifying the monitoring system can be automatically created from a format file (definition file) that describes the system configuration, so that verification of the monitoring system 10 can be realized using a common model regardless of the system administrator (engineer).

[0141] Furthermore, while it is generally necessary to analyze the operation of a cloud system to create a model for monitoring system verification, in this embodiment, by realizing automatic creation of a model for monitoring system verification, it is possible to reduce the cost (e.g., time cost, etc.) associated with creating the model.

[0142] Furthermore, in this embodiment, by focusing on the fact that there are generally many similarities among requirements to be verified in the monitoring system 10, a check formula is automatically created by using information on past check formulas (check formula information stored in the check formula information storage unit 33). This can reduce the cost of creating a check formula and avoid relying on an individual to create the check formula.

[0143] The model creation device 30 according to this embodiment may be configured to at least create a monitoring system verification model, but may also be configured to verify the monitoring system verification model based on the above-mentioned alert model, network model, verification equation, and state transition information. This makes it possible to verify the operation of the monitoring system 10 using the monitoring system verification model.

[0144] Furthermore, the model creating device 30 according to this embodiment may be configured to automatically create (register) check equation information used to create a check equation based on the specifications of the monitoring system 10, and may be configured to automatically create (register) a state transition rule based on the specifications and check equation information of the monitoring system 10. This eliminates the need for a system administrator to prepare check equation information and state transition rules, for example, and thus makes it possible to further reduce time costs and the like.

[0145] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0146] 10... monitoring system, 11... service monitoring agent, 12... integrated monitoring software, 13... monitoring database, 20, 20a, 20b... server device, 21... application program, 22... monitoring agent (container), 23... monitoring agent (server), 30... model creation device, 31... alert model conversion processing unit, 32... network model conversion processing unit, 33... check expression information storage unit, 34... check expression creation unit, 35... state transition rule storage unit, 36... state transition creation unit, 37... model creation unit, 38... check expression information registration unit, 39... state transition rule registration unit, 40... model storage unit, 50... model checking unit, 60... check result storage unit.

Claims

1. a first conversion processing means for converting a first definition file of a monitoring system that monitors an operation of a server device into an alert model in which setting contents related to an alert output from the monitoring system are modeled; a second conversion processing means for converting a second definition file describing information about the server device and a third definition file describing information about an application program running on the server device into a network model in which settings about a network connecting the server device and the monitoring system are modeled; a first generating means for generating a check formula based on the third definition file and check formula information based on a specification of the monitoring system; a second generating means for generating state transition information indicating a plurality of states of the monitoring system that transition based on a rule regarding state transition of the monitoring system, the alert model, and the network model; a third creation means for creating a monitoring system verification model composed of the alert model, the network model, the check formula, and the state transition information; A model creation device comprising:

2. 2. The model creating device according to claim 1, wherein the monitoring system verification model is described in a model description language.

3. 2. The model creating device according to claim 1, further comprising: a checking means for checking said monitoring system verification model based on said alert model, said network model, said checking formula, and said state transition information.

4. 2. The model generating device according to claim 1, further comprising a fourth generating means for generating the check equation information based on a specification of the monitoring system.

5. 2. The model creating device according to claim 1, further comprising a fifth creating means for creating rules relating to said state transitions based on the specifications of said monitoring system and said check expression information.

6. A method executed by a model generating device, comprising: Executing a process of converting a first definition file of a monitoring system that monitors an operation of a server device into an alert model in which setting contents related to an alert output from the monitoring system are modeled; executing a process of converting a second definition file describing information related to the server device and a third definition file describing information related to an application program running on the server device into a network model in which setting contents related to a network connecting the server device and the monitoring system are modeled; creating a check formula based on the third definition file and check formula information based on a specification of the monitoring system; creating state transition information indicating a plurality of states of the monitoring system that transition based on a rule regarding state transition of the monitoring system, the alert model, and the network model; creating a monitoring system verification model that is configured from the alert model, the network model, the verification formula, and the state transition information; A method comprising the steps of:

7. A program executed by a computer of a model creating device, The computer includes: Executing a process of converting a first definition file of a monitoring system that monitors an operation of a server device into an alert model in which setting contents related to an alert output from the monitoring system are modeled; executing a process of converting a second definition file describing information related to the server device and a third definition file describing information related to an application program running on the server device into a network model in which setting contents related to a network connecting the server device and the monitoring system are modeled; creating a check formula based on the third definition file and check formula information based on a specification of the monitoring system; creating state transition information indicating a plurality of states of the monitoring system that transition based on a rule regarding state transition of the monitoring system, the alert model, and the network model; creating a monitoring system verification model that is configured from the alert model, the network model, the verification formula, and the state transition information; A program for executing the above.

Images