System verification device, system verification method, and program

Abstract

A system verification device 10 includes: a performance prediction equation generation unit 11 that, in an embodied system configuration embodying system requirements including an abstract portion, generates, on the basis of pre-set information for quantitative requirements for constituent elements included in an embodied portion embodying the abstract portion, a performance prediction to be used to determine whether a performance defined in the abstract portion is satisfied by the performance of the embodied portion; and a verification results determination unit 12 that calculates a performance prediction value for a quantitative requirement by using a performance prediction equation and a performance prediction value used in the performance prediction equation, and determines whether the calculated performance prediction value satisfies the performance defined in the abstract portion.

Description

System verification device, system verification method, and computer-readable recording medium

[0001] The present disclosure relates to a system verification device and a system verification method used for verifying a system, and further to a computer-readable recording medium on which a program for realizing these is recorded.

[0002] When designing an ICT (Information Communication Technology) system, the first step is to define the requirements. The designer creates information (system requirements) that summarizes the customer's requirements and requests and represents the configuration of the ICT system, including both concrete and abstract elements.

[0003] The configuration of an ICT system can be represented as a graph based on concepts such as IBN (Intent-based networking). A graph represents the elements (components) included in the configuration of an ICT system using nodes or edges. A node is a component that represents, for example, a device or an application. An edge is a component that represents the connection between two nodes.

[0004] Next, based on the concretization rules created in advance, the abstract parts included in the system requirements are concretized, and information (system concrete configuration) that represents the configuration of the ICT system in a deployable state is derived.

[0005] The concretization rules are information used to convert an abstract part into a concrete part by gradually concretizing it, although an abstract part may be converted into a concrete part in a single conversion.

[0006] The concrete part represents a part or configuration that is determined to be actually used in an ICT system, while the abstract part represents an undetermined part or configuration whose function is determined but whose actual use in the system is not yet concretely determined.

[0007] As a related technique, Patent Literature 1 discloses a system configuration derivation device that uses machine learning to reduce the number of steps required for designing an ICT system. According to the system configuration derivation device of Patent Literature 1, when specific system configuration information (specific system configuration) is generated from abstract configuration information (system requirements), a reward value is assigned to each of the generated system configuration and the concretization rules applied in the generation process, and AI (artificial intelligence) is made to learn the system configuration.

[0008] As a result, AI can simulate the system design knowledge of engineers (design knowledge). In addition, by learning designs and reward values ​​for a wide variety of requirements, it becomes possible to design system configurations faster and with higher reliability.

[0009] Patent No. 6989014

[0010] However, the system configuration derivation device of Patent Document 1 builds a verification environment equivalent to the specific system configuration to confirm that the designed specific system configuration satisfies the system requirements. The system configuration derivation device of Patent Document 1 also creates a verification program corresponding to each component that makes up the system requirements. Furthermore, the system configuration derivation device of Patent Document 1 must execute the verification program in the built verification environment for verification. Therefore, it is expected that building the verification environment, creating the verification program, and verifying it will take a long time.

[0011] Furthermore, in the system configuration derivation device of Patent Document 1, when AI performs machine learning to determine system performance indicators, a huge amount of training data related to quantitative requirements such as performance is required. Furthermore, in order to obtain a huge amount of training data related to quantitative requirements such as performance, it is necessary to repeat design and verification, and therefore verification is expected to take a long time.

[0012] One example of an objective of the present disclosure is to reduce the time required to verify an ICT system.

[0013] In order to achieve the above object, a system verification device according to one aspect of the present disclosure comprises: a performance prediction formula generation unit that generates a performance prediction formula used to determine whether performance of a concrete part that embodies a system requirement including an abstract part satisfies the performance defined in the abstract part, based on information that is preset for quantitative requirements of components included in the concrete part, in a concrete system configuration that embodies the abstract part; and a verification result determination unit that calculates a performance prediction value for the quantitative requirement using the performance prediction formula and performance measurement values ​​used in the performance prediction formula, and determines whether the calculated performance prediction value satisfies the performance defined in the abstract part.

[0014] Furthermore, in order to achieve the above object, a system verification method according to one aspect of the present disclosure is characterized in that a computer generates a performance prediction formula used to determine whether the performance of a concrete part that embodies a system requirement including an abstract part satisfies the performance defined in the abstract part, based on information that is preset for quantitative requirements of components included in the concrete part, calculates a performance prediction value for the quantitative requirement using the performance prediction formula and performance measurement values ​​used in the performance prediction formula, and determines whether the calculated performance prediction value satisfies the performance defined in the abstract part.

[0015] Furthermore, in order to achieve the above object, a computer-readable recording medium according to one aspect of the present disclosure is characterized in that it causes a computer to: generate a performance prediction formula used to determine whether or not the performance of a concrete part that embodies a system requirement including an abstract part satisfies the performance defined in the abstract part, based on information previously set for quantitative requirements of components included in the concrete part; calculate a performance prediction value for the quantitative requirement using the performance prediction formula and performance measurement values ​​used in the performance prediction formula; and determine whether or not the calculated performance prediction value satisfies the performance defined in the abstract part.

[0016] As described above, according to the present disclosure, it is possible to reduce the time required to verify an ICT system.

[0017] FIG. 1 is a diagram illustrating an example of a system verification apparatus according to a first embodiment. FIG. 2 is a diagram illustrating an example of system requirements. FIG. 3 is a diagram illustrating an example of a specific system configuration. FIG. 4 is a diagram illustrating an example of generating a performance prediction formula. FIG. 5 is a diagram illustrating an example of a data structure of performance measurement information. FIG. 6 is a diagram illustrating an example of a system including the system verification apparatus according to the first embodiment. FIG. 7 is a diagram illustrating an example of searching for an execution subject and parameters. FIG. 8 is a diagram illustrating an example of the operation of the search. FIG. 9 is a diagram illustrating an example of verification information. FIG. 10 is a diagram illustrating an example of a verification result. FIG. 11 is a diagram illustrating an example of a user interface of a design and evaluation tool. FIG. 12 is a diagram illustrating an example of the operation of the system verification apparatus according to the first embodiment. FIG. 13 is a diagram illustrating an example of a system verification apparatus according to a second embodiment. FIG. 14 is a diagram illustrating an example of the operation of a component extraction unit. FIG. 15 is a diagram illustrating an example of the operation of an update determination unit. FIG. 16 is a diagram illustrating an example of the operation of the system verification apparatus according to the second embodiment. FIG. 17 is a diagram illustrating an example of the operation of the system verification apparatus according to the second embodiment. FIG. 18 is a diagram illustrating an example of a computer that realizes the system verification device according to the first and second embodiments.

[0018] (Embodiment 1) Hereinafter, an embodiment will be described with reference to the drawings. In the drawings described below, elements having the same or corresponding functions are denoted by the same reference numerals, and repeated description thereof may be omitted.

[0019] The configuration of a system verification device 10 according to the first embodiment will be described below. Fig. 1 is a diagram for explaining an example of the system verification device according to the first embodiment.

[0020] 1 is a device for reducing the time required to verify an ICT system. The system verification device 10 includes a performance prediction formula generation unit 11 and a verification result determination unit 12.

[0021] The system verification device 10 uses the system requirements and the specific system configuration to verify the performance of the specific part of the specific system configuration that embodies the abstract part of the system requirements.

[0022] The system requirements are definitions of the requirements that the user of the system verification device 10 expects from the ICT system. The definition of the system requirements is, for example, information (such as programming code) that describes the definitions of the element types of the OS (Operating System) and the server computer.

[0023] Specifically, the definition of system requirements is information (programming code) that includes, for example, information about the type of elements of the ICT system, attribute values ​​of configuration information such as application configuration information and OS configuration information, as well as information that needs to be verified (information about verification items).

[0024] In the definition of the system requirements, there is an inheritance relationship between the elements (components) that make up the ICT system and the types of relationships between the components, and the abstract types of the inheritance source may be included.

[0025] Specifically, for example, the windows type representing the Windows OS, which is a type of OS, and the ubuntu type representing the Ubuntu Linux (registered trademark) OS are in an inheritance relationship in which the abstract os type representing the OS is the parent class and the windows type and ubuntu type are the child classes, and when the user does not specify the type of OS, the components of the os type can be included as elements in the definition of the system requirements.

[0026] Figure 2 is a diagram for explaining an example of system requirements. A in Figure 2 shows a graph representing the configuration (components) of the system requirements and quantitative requirements for the abstract parts. B in Figure 2 shows information (programming code) that defines the system requirements of A in Figure 2.

[0027] In the graph in Figure 2A, an abstract relationship (dashed arrow) is defined that indicates that a clientapp type element "app1", which represents a client application, can access a webapp type element "app2", which represents a web application running on a web server, at a bandwidth greater than 100 Mbps (megabits per second; the same applies hereinafter) and with a latency of less than 10 ms (milliseconds; the same applies hereinafter).

[0028] The programming code in FIG. 2B is data representing a definition of system requirements equivalent to the graph in FIG. 2A, and may be created in a data structure format such as YAML or JSON (JavaScript Object Notation).

[0029] The programming code for B in Figure 2 consists of three fields: "components," which defines the system elements that make up the requirements; "relationships," which defines the relationships between the elements; and "constraints," which defines performance and other aspects.

[0030] "[app1, app2, connTo]" in "relationships" represents a connTo type relationship that represents an abstract connection relationship from web application app1 to web application app2.

[0031] The "(app1, app2, connTo)::bandwidth > 100" (verification item) in "constraints" indicates that the quantitative requirement "(app1, app2, connTo)::bandwidth" representing the bandwidth between the applications app1 and app2 is greater than 100 Mbps.

[0032] Similarly, the "constraints" "(app1, app2, connTo)::delay < 10" (verification item) indicates that the quantitative requirement "(app1, app2, connTo)::delay", which represents the delay between applications app1 and app2, is less than 10 ms.

[0033] The quantitative requirements are not limited to the bandwidth and delay described above. Furthermore, the performance (non-functional requirements) for each quantitative requirement is not limited to the bandwidth performance and delay performance described above.

[0034] One or more system configurations are generated for each system requirement. The system configuration described above is one of the one or more system configurations generated.

[0035] In the first embodiment, a system requirement and a specific system configuration that embodies the system requirement are input as a set to the system verification device 10 .

[0036] Fig. 3 is a diagram for explaining an example of a specific system configuration. Fig. 3A is a graph representing the configuration of the specific system configuration. In other words, it is a graph without any abstract parts. Fig. 3B shows information (programming code) representing the definition of the specific system configuration.

[0037] The definition of the specific system configuration may include, for example, attribute values ​​of configuration information such as information about the type of element, application setting information, and OS setting information.

[0038] Specifically, an ubuntu-type component representing the Ubuntu Linux OS includes an attribute value "version" that specifies the OS version, and one of the strings such as "20.04," "21.10," or "22.04," which are the versions of the OS, is defined as the value of "version."

[0039] The graph in FIG. 3A shows a configuration in which, to realize communication between applications app1 and app2, each application runs on an OS on an independent physical machine, and the physical machines are connected by a router.

[0040] The applications app1 and app2 are connected to the OS-type components os1 and os2, respectively, by a wire:OS-type relationship (solid arrow) that indicates the hosting relationship of the applications on the OS. Additionally, the attribute value "osType: ubuntu" is stored in os1 and os2, indicating that Ubuntu Linux (registered trademark) is used as the OS type.

[0041] Similarly, the OS type components os1 and os2 are connected to the machine type components machine1 and machine2, which represent physical machines, by wire:machine type relationships (solid arrows) that represent the OS running on the machines.The physical machines machine1 and machine2 are also connected to the router type component router1, which represents a router, by wire:router type relationships (solid arrows) that represent a network connection to the router.

[0042] The performance prediction formula generator 11 generates a performance prediction formula to be used to determine whether the performance of a concrete part (verification item) that embodies an abstract part in a specific system configuration that embodies system requirements including an abstract part satisfies the performance defined in the abstract part, based on information that is preset for the quantitative requirements of the components included in the concrete part.

[0043] Specifically, the performance prediction formula generator 11 first acquires a set of system requirements and a specific system configuration from a storage device, and then generates a performance prediction formula to be used for verifying the performance of a specific part (verification item) of the specific system configuration, which is a concrete version of the abstract part of the system requirements.

[0044] Next, the performance prediction formula generation unit 11 refers to performance measurement information that has been generated in advance and is stored in a storage device (described later), and acquires performance measurement values ​​to be used in the performance prediction formula from the performance measurement information. However, the performance measurement values ​​to be used in the performance prediction formula may also be acquired by the verification result determination unit 12.

[0045] The generation of a performance prediction formula will now be described. Fig. 4 is a diagram for explaining an example of the generation of a performance prediction formula. In the graph A of Fig. 4, it is defined as a system requirement that the communication bandwidth from a camera-type component camera1 representing a camera to a face-app-type component app1 representing a face recognition application is greater than 100 [Mbps].

[0046] The graph in Fig. 4B is one of the concrete system configurations generated by concretizing the abstract part of the system requirements shown in Fig. 4A. Note that the concreteization of the abstract part can be performed using, for example, the learning-type system automatic design technology disclosed in the above-mentioned Patent Document 1.

[0047] Specifically, the graph in Figure 4B has a machine-type component machine1, a machine-type component machine2, and a router-type component router1. The machine-type component machine1 represents a physical machine connected to a camera. The machine-type component machine2 represents a physical machine running a face recognition application. The router-type component router1 represents a router connecting the two physical machines.

[0048] 4B has a relationship (solid arrow) between the machine-type component machine1 and the router-type component router1, and a relationship (solid arrow) between the router-type component router1 and the machine-type component machine2. Note that this is shown as a result of concretizing the abstract relationship (dashed arrow) between the camera and face recognition application, which are system requirements.

[0049] Furthermore, when an abstract part is made concrete (when a concretization process is executed), information that associates information representing the abstract part with information representing the concrete part (information representing the association between the components and the relationships) is recorded in the storage device as a concretization history, which serves as a history of concretization of the abstract part.

[0050] The instantiation history is information indicating which abstract configuration elements and relationships were used to generate the components and relationships included in the system configuration during the process from system requirements to the design of the system configuration, and is recorded for each component and relationship. In the example of Figure 4B, the relationship (app1, app2, connTo) is recorded as the instantiation history for each of the components machine1, machine2, and router1. This indicates that the three components were generated by instantiating the abstract relationship (app1, app2, connTo).

[0051] Furthermore, the performance of the abstract portion (dashed line) of the graph in Figure 4A is defined as a bandwidth greater than 100 Mbps ("Bandwidth > 100 Mbps"). The graph in Figure 4B shows the performance prediction formula "Bandwidth: X = Min(X1, X2, X3) > 100 Mbps" for predicting the performance of the concrete portion corresponding to the abstract portion (dashed line) described above.

[0052] In the example of Figure 4, the performance prediction formula generator 11 first refers to the above-mentioned instantiation history, detects the component type after instantiation, and determines that the abstract relationship between camera1 and app1 is realized by three components: machine1, machine2, and router1.

[0053] Next, the performance prediction formula generation unit 11 generates a performance prediction formula for finding a performance prediction value X such that the smallest value among the performance measurement values ​​related to the bandwidth of machine 1, the performance measurement values ​​related to the bandwidth of router 1, and the performance measurement values ​​related to the bandwidth of machine 2 becomes the communication bandwidth between camera 1 and app 1.

[0054] 4, the quantitative requirement is bandwidth, so the formula to be used when generating the performance prediction formula for bandwidth is selected from the determination rules pre-stored in the storage device. The determination rules are information that associates the quantitative requirement with a function.

[0055] 4, the performance prediction formula generator 11 selects the function Min() as the formula for selecting the smallest value because the quantitative requirement is a bandwidth. Note that the formula to be selected differs depending on the type of quantitative requirement.

[0056] Other types of functions that may be used include Max(), which selects the largest value among the arguments, Sum(), which calculates the sum of the argument values, and Average(), which calculates the average value of the argument values.

[0057] The mathematical formula for the performance prediction formula takes the performance values ​​of the components and relationships included in the system specific configuration as arguments, and the type of function is determined by the type of quantitative requirement. The components and relationships that use the performance values ​​as arguments of the function are determined by recursively searching for components and relationships that include the same components or relationships in their instantiation history from the components or relationships for which quantitative requirements are defined, and adding components and relationships that match the type predefined for each type of quantitative requirement.

[0058] In the example of B in Figure 4, an element whose instantiation history includes a relationship (app1, app2, connTo) that defines a quantitative bandwidth requirement is searched for, and variables X1, X2, and X3 that represent the performance values ​​of machine1, router1, and machine2, which are machine-type and router-type elements defined as targets for adding arguments in predicting bandwidth performance, are added to the arguments of the function Min() used to predict bandwidth performance, thereby generating a function Min(X1, X2, X3) as a performance prediction formula for bandwidth X between applications app1 and app2.

[0059] In this way, the performance prediction formula generator 11 generates the performance prediction formula "X = Min(X1, X2, X3) > 100 [Mbps]". In the performance prediction formula, X1 is a variable representing the performance measurement value related to the bandwidth of machine1. X2 is a variable representing the performance measurement value related to the bandwidth of machine2. X3 is a variable representing the performance measurement value related to the bandwidth of router1.

[0060] The verification result determination unit 12 acquires performance measurement values ​​used in the performance prediction formula, calculates a performance prediction value of the quantitative requirements using the acquired performance measurement values ​​and the performance prediction formula, and determines whether the calculated performance prediction value satisfies the performance defined in the abstract part.

[0061] Thereafter, the verification result determination unit 12 outputs the specific system configuration, the verification items, the performance prediction values, and the determination results to the output information generation unit 16 in order to generate the verification results.

[0062] The performance measurement values ​​used in the performance prediction formula may be acquired by the verification result determination unit 12 or by the performance prediction formula generation unit 11 .

[0063] Specifically, the verification result determination unit 12 first obtains, from the performance measurement information, the measured value of the bandwidth performance bandwidth of the machine-type component, the measured value of the bandwidth performance bandwidth of the router-type component, and the measured value of the bandwidth performance bandwidth of the machine-type component, which are to be substituted for the variables X1, X2, and X3 included in the performance prediction formula "X = Min(X1, X2, X3) > 100 [Mbps]".

[0064] Next, the verification result determination unit 12 substitutes the measured values ​​into variables X1, X2, and X3 to calculate the performance prediction values.

[0065] 5 is a diagram illustrating an example of the data structure of performance measurement information. In the example of FIG. 5, the performance measurement information is information in which components, quantitative requirements, and measurement values ​​are associated with each other. Specifically, the performance measurement information is information in which information indicating the type of component, information indicating the type of quantitative requirement, and measurement values ​​are associated with each other.

[0066] 5, a quantitative requirement "bandwidth" representing a bandwidth and a performance measurement value of the bandwidth of 990 [Mbps] are associated with a "machine" type element representing a physical machine. However, the unit may be omitted.

[0067] 5, a quantitative requirement "delay" representing a delay and a performance measurement value of the delay of 0.1 [ms] are associated with the "machine" type element representing a physical machine. However, the unit may be omitted.

[0068] In the example of FIG. 5, the machine-type element representing a physical machine is associated with a quantitative requirement "availability" representing availability and 0.9 representing a performance measurement value of availability.

[0069] In this way, in embodiment 1, the performance prediction value of the quantitative requirements is calculated using a performance prediction formula and performance measurements for each component of the ICT system that have been stored (accumulated) in advance in a storage device, and it is possible to determine whether the calculated performance prediction value satisfies the performance defined in the abstract part.

[0070] Therefore, according to the first embodiment, the construction of a verification environment, the creation of a verification program, and the verification can be omitted, thereby shortening the time required for performance verification. Furthermore, the working time of engineers can be reduced (the burden can be reduced).

[0071] [System Configuration] Next, the configuration of the system verification device 10 in the embodiment will be described in more detail with reference to Fig. 6. Fig. 6 is a diagram for explaining an example of a system including the system verification device of the first embodiment.

[0072] The system 100 includes at least a system verification device 10, a storage device 20, an input device 30, and an output device 40. The system verification device 10, the storage device 20, the input device 30, and the output device 40 are communicatively connected via a network.

[0073] The system verification device 10 is, for example, an information processing device such as a CPU (Central Processing Unit), a programmable device such as an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or a circuit equipped with one or more of these, a server computer, a personal computer, or a mobile terminal.

[0074] The storage device 20 is a database, a server computer, a circuit having a memory, etc. The storage device 20 stores at least the information described below. The storage device 20 stores, for example, at least information such as system requirements, specific system configurations, type definitions of system components and relationships, instantiation rules, judgment rules, instantiation history, and templates for generating a verification program.

[0075] 6, the storage device 20 is provided outside the system verification device 10, but it may also be provided inside the system verification device 10. Furthermore, the storage device 20 may be configured from multiple storage devices, and the above-mentioned information may be stored in a distributed manner.

[0076] The input device 30 is, for example, a keyboard, a mouse, a touch panel, etc. The input device 30 is used when operating the system verification device 10, the output device 40, etc.

[0077] The output device 40 acquires output information (to be described later) converted into an outputtable format by the output information generation unit 16 (to be described later), and outputs generated images, sounds, etc. based on the output information. The output device 40 is, for example, an image display device using a liquid crystal, an organic electroluminescence (EL) display, or a cathode ray tube (CRT). Furthermore, the image display device may also include an audio output device such as a speaker. The output device 40 may also be a printing device such as a printer.

[0078] The communication network is a general network constructed using communication lines such as the Internet, a LAN (Local Area Network), a dedicated line, a telephone line, an in-house network, a mobile communication network, Bluetooth (registered trademark), and Wi-Fi (Wireless Fidelity).

[0079] (System Verification Apparatus) The system verification apparatus will now be described in detail. A system verification apparatus 10 according to the first embodiment includes a performance prediction formula generation unit 11, a verification result determination unit 12, a verification environment construction unit 13, a verification program generation unit 14, a verification program execution unit 15, and an output information generation unit 16.

[0080] Since the performance prediction formula generating unit 11 and the verification result determining unit 12 have been described above, the description of the performance prediction formula generating unit 11 and the verification result determining unit 12 will be omitted.

[0081] If the performance prediction formula generating unit 11 cannot acquire the performance measurement value from the performance measurement information, the verification environment constructing unit 13 constructs a verification environment defined in the specific system configuration.

[0082] Specifically, the verification environment construction unit 13 first obtains from the performance prediction formula generation unit 11 a determination result indicating whether or not the performance measurement information includes performance measurement values ​​contained in the concrete part corresponding to the abstract part generated by the performance prediction formula generation unit 11 when generating the performance prediction formula.

[0083] Next, if the determination result indicates that the performance measurement value is not included in the performance measurement information, the verification environment construction unit 13 constructs a verification environment based on the definition of the specific system configuration in order to measure the performance of the performance measurement value that could not be obtained. The verification environment is preferably constructed in, for example, a virtual environment.

[0084] Verification program generation unit 14 generates a verification program that executes performance measurement processing in the verification environment constructed by verification environment construction unit 13. Verification program generation unit 14 generates the verification program using, for example, the technology disclosed in Japanese Patent Application Laid-Open No. 2021-165930, etc. However, the technology is not limited to the above-mentioned technology.

[0085] Specifically, the verification program generation unit 14 acquires the quantitative requirements and the specific system configuration, refers to a template corresponding to the type of quantitative requirement that has been set in advance, and searches for the graph structure of the proposed configuration to adjust the execution entity and parameters of the verification program, thereby generating a verification program to be executed in the verification environment.

[0086] The template defines the type of quantitative requirement as a key, the type of command required to verify the quantitative requirement, and the execution subject and parameter configuration plan search procedure in the graph structure required to execute the command.

[0087] 7 is a diagram illustrating an example of a search for an execution subject and parameters. The search procedure shown in FIG. 7 is composed of a search start point written in the symbols "<>" and search rule 1 and search rule 2 written separated by the symbol "+".

[0088] The search is executed in the order of search rule 1 and search rule 2. In the example of FIG. <app1>"(HostedOn, OS) + (Join, LAN)" means that for the graph structure of the specific system configuration, the node of the component app1 is used as the starting point of the search, the edges of the relationship HostedOn are traced one or more times, and the node of the component OS type that is reached first is used as the starting point, the edges of the relationship Join are traced one or more times, and the node of the component LAN type that is reached first is the search result.

[0089] 8 is a diagram for explaining an example of a search operation. A method for searching for an execution subject of a component app1 of an application included in a specific system configuration and a search operation in the structure of the specific system configuration will be explained using FIG. 8.

[0090] In the example of FIG. 8, the search procedure definition " <app1>Starting from the node of the component app1 according to "(HostedOn, OS)", the search is followed through "Wire:MW" and "Wire:OS", which are relationship edges inherited from the HostedOn type, and ends at the OS1 node, which is an OS type component, and outputs that node as the search result.

[0091] 9 is a diagram illustrating an example of verification information. The verification program is programming code including information required for verification (verification information) as shown in FIG. The verification information stores quantitative requirements, the order in which the quantitative requirements are executed, an execution entity, and a script for verifying the quantitative requirements in association with each other.

[0092] Verification program execution unit 15 executes the verification program generated by verification program generation unit 14 and outputs information (performance measurement results) representing the results of executing the verification program. Specifically, verification program execution unit 15 first executes the verification program in the verification environment and generates, as a result of the execution, performance measurement results that associate components, quantitative requirements of the components, and performance measurement values ​​of the quantitative requirements.

[0093] Next, the verification program execution unit 15 adds (stores) the output performance measurement results to the performance measurement information in the storage device 20. The output performance measurement results are used by the verification result determination unit 12 to calculate a performance prediction value.

[0094] The output information generation unit 16 acquires the information necessary to generate the verification results from the verification result judgment unit 12 (acquiring at least the specific system configuration, verification items, performance prediction values, and judgment results), generates output information for outputting the verification results to the output device 40, and outputs it to the output device 40.

[0095] In addition to the verification results, the output information generating unit 16 generates, as output information, for example, a user interface screen of an ICT system design and evaluation tool, and outputs it to the output device 40.

[0096] 10 is a diagram for explaining an example of a verification result. The verification result describes the verification results of a specific system configuration. FIG. 10 shows information in which verification items, predicted performance values ​​for the verification items, and judgment results for the verification items are associated with each other for each of a plurality of specific system configurations generated by specifying the system requirements shown in FIG. 2.

[0097] The programming code in the first stage of the "specific system configuration" in Fig. 10 corresponds to the specific system configuration shown in Fig. 3. The programming code in the first stage is associated with a verification item "(app1, app2, connTo)::bandwidth > 100" that indicates a bandwidth constraint and a verification item "(app1, app2, connTo)::delay < 10" that indicates a delay constraint.

[0098] The verification item representing the bandwidth constraint is associated with the verification result "990 [Mbps]" and the judgment result "PASS" indicating that the bandwidth constraint condition has been met. Also, the verification item representing the delay constraint is associated with the verification result "0.3 [ms]" and the judgment result "PASS" indicating that the delay constraint condition has been met.

[0099] Next, the programming code in the next stage of the "Specific System Configuration" in Figure 10 represents a configuration in which physical machine machine1 running application app1 and physical machine machine2 running application app2 are connected via WAN-type component wan1, which represents an Internet line. Although the constraint for the verification item "(app1, app2, connTo)::delay < 10" is met, the verification result for the verification item "(app1, app2, connTo)::bandwidth > 100" is 93.7 [Mbps], which does not meet the constraint, and the verification item is judged as "FAIL," meaning that the verification item is not satisfied.

[0100] (Design and Evaluation Tool) The design and evaluation tool will be described with reference to Fig. 11. The design and evaluation tool (software program) provided in the system verification device 10 is used when generating a specific system configuration from the above-mentioned system requirements.

[0101] 11 is a diagram illustrating an example of a user interface of a design and evaluation tool. A screen display G1 in FIG. 11 represents a screen of a graphical user interface (GUI). The user is, for example, a designer who designs a system.

[0102] The screen display G1 is made up of a library display section G2, an input form G3, a design button G4, an output form G5, and a sub-window (detailed information display section) G6.

[0103] The library display section G2 displays, for example, the types of components of ICT systems used in the system verification device 10, system design (learning-based automatic system design technology shown in Patent Document 1), etc., arranged by category such as App (application), OS, Machine, etc.

[0104] The input form G3 is used to generate a graph representing the system requirements. The user operates the input device 30, for example, to create the system requirement graph shown in the input form G3 in Figure 11. Specifically, the user selects component nodes from the library display section G2 and places the selected nodes in the input form G3 to create the system requirement graph.

[0105] When the design button G4 is pressed after the system requirements have been created in the input form G3, a specific system configuration is generated using the above-mentioned learning-type automatic system design technology.

[0106] The output form G5 displays the specific system configuration that is the design result. When the user operates the input device 30, for example, to select a node (component) or an edge (relationship) in the graph of the system requirements displayed in the input form G3 or the graph of the specific system configuration displayed in the output form G5, detailed information about the selected component or relationship is displayed in the sub-window G6.

[0107] The example of FIG. 11 shows that the relationship (app1, app2, connTo) included in the specific system configuration shown in FIG. 3, which is the result of designing the system requirements shown in FIG. 2, has been selected.

[0108] Sub-window G6 displays information about the components included in the system requirements and the specific system configuration, and the relationships between the components. Specifically, the "id" field displays the ID of the selected component, and the "type" field displays the type name of the selected component. The "resolved" field is a flag indicating that the selected component has been replaced with a specific configuration in the design process of the learning-based system automatic design technology and does not exist in the actual configuration.

[0109] The "properties" field displays information about the attribute values ​​of the selected element, and the "constraints" field displays verification items including quantitative requirements and information equivalent to the verification results shown in FIG.

[0110] In the example of Figure 11, the verification result of the verification item "(app1, app2, connTo)::bandwidth > 100" representing the bandwidth constraint between applications is "PASS" registered as the value of "result" indicating whether the verification item is satisfied.

[0111] The value of "estimation," which represents the predicted performance value of the component, is registered as "950 [Mbps]" calculated by the system verification device 10, and the value of "formula," which represents the performance prediction formula, "Min(machine1, machine2, router1)," indicates that the smallest value among the performance measurement values ​​of the components of the physical machines machine1 and machine2 and router router1 in the specific system configuration is used as the predicted performance value.

[0112] [Device Operation] Next, the operation of the system verification device 10 in the first embodiment will be described with reference to FIG. 12. FIG. 12 is a diagram for explaining an example of the operation of the system verification device in the first embodiment. In the following description, reference will be made to the diagram as appropriate. In the first embodiment, the system verification method is performed by operating the system verification device. Therefore, the description of the system verification method in the first embodiment will be replaced by the following description of the operation of the system verification device.

[0113] First, the performance prediction formula generator 11 acquires a set of system requirements and a specific system configuration from the storage device 20 (step A1).

[0114] Next, the performance prediction formula generator 11 extracts concrete parts (verification items) of the specific system configuration that embody the abstract parts of the system requirements (step A2), and then generates a performance prediction formula to be used to verify the performance of each verification item (step A3).

[0115] Next, the performance prediction formula generator 11 refers to the performance measurement information stored in the storage device 20, and if the performance measurement information stores performance measurements to be used in the performance prediction formula (step A4: Yes), it acquires the performance measurements to be used in the performance prediction formula from the performance measurement information (step A5). However, the performance measurements to be used in the performance prediction formula may also be acquired by the verification result determiner 12.

[0116] Next, the verification result determining unit 12 obtains performance measurements used in the performance prediction formula, and calculates a performance prediction value of the quantitative requirement using the obtained performance measurements and the performance prediction formula (step A6). Next, the verification result determining unit 12 determines whether the calculated performance prediction value satisfies the performance defined in the abstract part (determines whether the performance measurements satisfy the constraints) (step A7).

[0117] Thereafter, the verification result determination unit 12 outputs at least the specific system configuration, the verification items, the performance prediction values, and the determination results to the output information generation unit 16 in order to generate the verification results.

[0118] Next, the output information generating unit 16 acquires information necessary for generating the verification result from the verification result determining unit 12, generates output information to be output to the output device 40, and outputs it to the output device 40 (step A8).

[0119] Furthermore, the performance prediction formula generation unit 11 refers to the performance measurement information stored in the storage device 20, and if the performance measurement values ​​to be used in the performance prediction formula are not stored in the performance measurement information (step A4: No), the verification environment construction unit 13 constructs an environment defined in the specific system configuration (step A9).

[0120] Next, verification program generator 14 generates a verification program for executing a performance measurement process in the environment constructed by verification environment constructor 13 (step A10).

[0121] Next, the verification program execution unit 15 executes the verification program generated by the verification program generation unit 14 (step A11), outputs information indicating the results of the execution of the verification program (performance measurement results), and adds (stores) the output performance measurement results to the performance measurement information (step A12).Then, the process proceeds to step A4.

[0122] [Effects of embodiment 1] According to embodiment 1, a performance prediction value of a quantitative requirement is calculated using a performance prediction formula and performance measurements for each component of an ICT system that have been stored (accumulated) in advance in a storage device, and it is possible to determine whether the calculated performance prediction value satisfies the performance defined in the abstract part.

[0123] Therefore, since the construction of a verification environment, the creation of a verification program, and the verification can be omitted, performance verification can be completed in a short time. Furthermore, the working time of engineers can be reduced (the burden can be reduced).

[0124] [Program] The program in the first embodiment may be a program that causes a computer to execute steps A1 to A12 shown in Fig. 12. By installing and executing this program on a computer, the system verification apparatus and system verification method in the first embodiment can be realized. In this case, the processor of the computer functions as a performance prediction formula generator 11, a verification result determiner 12, a verification environment builder 13, a verification program generator 14, a verification program executor 15, and an output information generator 16, and performs the processes.

[0125] The program in the first embodiment may be executed by a computer system constructed by a plurality of computers, in which case, for example, each computer may function as one of the performance prediction formula generator 11, verification result determiner 12, verification environment builder 13, verification program generator 14, verification program executor 15, and output information generator 16.

[0126] Second Embodiment A configuration of a system verification device 10' according to a second embodiment will be described below. Fig. 13 is a diagram showing an example of the system verification device according to the second embodiment.

[0127] [Device Configuration] A system verification device 10′ shown in FIG. 13 includes a performance prediction formula generation unit 11, a verification result determination unit 12, a verification environment construction unit 13, a verification program generation unit 14, a verification program execution unit 15, an output information generation unit 16, a component extraction unit 17, and an update determination unit 18.

[0128] The component extracting unit 17 first acquires the specific system configuration acquired by the performance prediction formula generating unit 11 with a certain probability, and extracts one of the components included in the acquired specific system configuration.

[0129] Specifically, for each specific system configuration acquired by the performance prediction formula generation unit 11, a determination is made as to whether it should be a target for component extraction with a predefined probability (e.g., 5%, 10%, etc.), and one component is extracted from each specific system configuration determined to be a target for extraction.

[0130] Next, the component extraction unit 17 refers to the performance measurement information in the storage device 20, obtains a list of quantitative requirements related to the extracted components, and outputs information associating the obtained system specific configuration with the list of quantitative requirements to the verification environment construction unit 13.

[0131] 14 is a diagram illustrating an example of the operation of the component extractor 17. In the example of Fig. 14, the component extractor 17 first acquires a specific system configuration 141 (equivalent to the specific system configuration of Fig. 6) from the performance prediction formula generator 11 with a certain probability.

[0132] Next, the component extractor 17 randomly extracts components included in the system specific configuration 141. In the example of Fig. 14, router1 of the router type component 142 is extracted.

[0133] Next, the component extraction unit 17 refers to the performance measurement information 143 of the storage device 20 and obtains the quantitative requirements to be verified, namely, the bandwidth quantitative requirement "bandwidth", the delay quantitative requirement "delay", and the availability quantitative requirement "availability", which correspond to the extracted router type component 142.

[0134] The component extraction unit 17 outputs the system specific configuration 141, the components 142, and the list of quantitative requirements 144 acquired and extracted in the above-described procedure, and inputs them to the verification environment construction unit 13.

[0135] The update determination unit 18 first acquires the performance measurement results executed by the verification program execution unit 15. The performance measurement results are the execution results of a verification program aimed at verifying the quantitative requirements of the configuration elements. The performance measurement results are also sets of the configuration elements, the quantitative requirements, and the performance measurement values.

[0136] Next, the update determination unit 18 determines whether the performance measurement values ​​corresponding to the quantitative requirements of the acquired component and the performance measurement values ​​corresponding to the quantitative requirements of the same component as the acquired component contained in the performance measurement information pre-stored in the memory device 20 match the pre-defined update criteria.

[0137] Next, if the update criteria are met, the update determination unit 18 updates the performance measurement values ​​corresponding to the quantitative requirements of the same component as the acquired component contained in the performance measurement information stored in the memory device 20 with the performance measurement values ​​corresponding to the quantitative requirements of the acquired component.

[0138] In addition, if the performance measurement values ​​corresponding to the acquired components and quantitative requirements are not recorded in the performance measurement information of the storage device 20, the performance measurement values ​​corresponding to the acquired components are added (recorded) to the performance measurement information.

[0139] Fig. 15 is a diagram illustrating an example of the operation of the update determination unit, in which A in Fig. 15 represents the operation of comparing "measured value X" with "measured value Y" for the component "router" and the quantitative requirements "bandwidth," "delay," and "availability."

[0140] "Measurement value X" represents a value already stored in storage device 20. "Measurement value Y" represents the execution result verified using verification program generation unit 14 and verification program execution unit 15 in the verification environment generated by verification environment construction unit 13.

[0141] The update criterion is a condition for updating (overwriting) the measurement value X with a new measurement value Y. In the example of FIG.

[0142] In the example of A in FIG. 15, for the quantitative requirements bandwidth and availability, 990 [Mbps] and 0.99 are stored as measured value X, and 995 [Mbps] and 0.999 are stored as measured value Y, respectively. If measured value Y is greater than measured value X, the update criterion "X < Y" is satisfied, which indicates that the performance measured value is updated.

[0143] As shown in the performance measurement information in B of Fig. 15, only the performance measurements of the quantitative requirements bandwidth and availability are updated. On the other hand, for the quantitative requirement delay, 0.1 [ms] is stored as the measured value X and 0.15 [ms] as the measured value Y, which does not satisfy the update criterion "X > Y," so the performance measurement values ​​in the performance measurement information are not updated.

[0144] 16 and 17 are diagrams for explaining an example of the operation of the system verification device of embodiment 2. In the following description, reference will be made to the diagrams as appropriate. In embodiment 2, the system verification method is implemented by operating the system verification device. Therefore, the explanation of the system verification method in embodiment 2 will be substituted for the explanation of the operation of the system verification device below.

[0145] In FIG. 16, the same steps as those in FIG. 12 of the first embodiment are denoted by the same reference numerals, and detailed description of the steps will be omitted.

[0146] First, the performance prediction formula generator 11 acquires a set of system requirements and a specific system configuration from the storage device 20 (step A1). Next, the component extractor 17 acquires the specific system configuration input to the performance prediction formula generator 11 with a certain probability (step B1).

[0147] Next, when the component extractor 17 acquires the specific system configuration (step B1: Yes), the performance measurement values ​​are updated in the process of step B2 (step B2). Note that the process of updating the performance measurement values ​​in step B2 may be executed asynchronously with the processes from step A2 onwards.

[0148] If the component extractor 17 does not acquire the specific system configuration (step B1: No), the process proceeds to step A2, and the processes from step A2 onwards (steps A2 to A12 and B3) are executed.

[0149] Next, after the verification program is executed in step A11, the update determination unit 18 determines whether to update the performance measurement information based on the execution result of the verification program (step B3).

[0150] Specifically, in step B3, the update determination unit 18 first obtains the performance measurement results (a set of components, quantitative requirements, and performance measurement values, which are the execution results of a verification program aimed at verifying the quantitative requirements of the components) executed by the verification program execution unit 15.

[0151] Next, in step B3, the update determination unit 18 compares the acquired performance measurement value of the component with the performance measurement value of the same component stored in the storage device 20.

[0152] Next, if an update is to be performed in step B3 (step B3: Yes), the update determination unit 18 updates the performance measurement values ​​corresponding to the current components that are the same as the acquired components using predefined update criteria for each type of quantitative requirement and the performance measurement values ​​corresponding to the acquired components (step A12).

[0153] In addition, when updating in step B3 (step B3: Yes), if the performance measurement values ​​corresponding to the acquired components and quantitative requirements are not recorded in the performance measurement information of the storage device 20, the update determination unit 18 adds (records) the performance measurement values ​​to the performance measurement information (step A12).

[0154] If no update is required in step B3 (step B3: No), the process proceeds to step A4 and continues.

[0155] The details of the process of step B2 (the process of updating the performance measurement values) are now explained. The component extractor 17 first acquires the specific system configuration input to the performance prediction formula generator 11 with a certain probability, and extracts one of the components included in the acquired specific system configuration (step C1).

[0156] Next, the component extraction unit 17 refers to the performance measurement information 143 in the storage device 20 and obtains a list of quantitative requirements related to the extracted components (step C2), and outputs information associating the obtained system specific configuration 141 with the list of quantitative requirements to the verification environment construction unit 13 (step C3).

[0157] Next, the verification environment construction unit 13 constructs an environment defined in the specific system configuration (step A9).

[0158] Next, verification program generator 14 generates a verification program for executing a performance measurement process in the environment constructed by verification environment constructor 13 (step A10).

[0159] Next, the verification program execution unit 15 executes the verification program generated by the verification program generation unit 14 (step A11).

[0160] Next, if an update is to be performed in step C4 (step C4: Yes), the update determination unit 18 updates the performance measurement values ​​corresponding to the current components that are the same as the acquired components using predefined update criteria for each type of quantitative requirement and the performance measurement values ​​corresponding to the acquired components (step A12).

[0161] In addition, when updating in step C4 (step C4: Yes), if the performance measurement value corresponding to the quantitative requirement of the acquired component is not recorded in the performance measurement information of the storage device 20, the update determination unit 18 adds (records) the performance measurement value to the performance measurement information (step A12).

[0162] If no update is required in step B3 (step C4: No), the process proceeds to step A4.

[0163] [Effects of the Second Embodiment] According to the second embodiment, a performance prediction value of a quantitative requirement is calculated using a performance prediction formula and performance measurements for each component of an ICT system that have been stored (accumulated) in advance, and it is possible to determine whether the calculated performance prediction value satisfies the performance defined in the abstract part.

[0164] Therefore, since the construction of a verification environment, the creation of a verification program, and the verification can be omitted, performance verification can be completed in a short time. Furthermore, the working time of engineers can be reduced (the burden can be reduced).

[0165] Furthermore, according to the second embodiment, the performance measurement values ​​of the components in the performance measurement information can be automatically updated by constructing a verification environment and performing performance measurement with a certain probability, thereby improving the accuracy of the performance prediction values ​​of the quantitative requirements calculated by the performance prediction formula, and enabling more accurate system performance verification.

[0166] [Program] The program in the second embodiment may be a program that causes a computer to execute steps A1 to A12 and B1 to B3 shown in Fig. 16 and steps C1 to C4 and A9 to A12 shown in Fig. 17. By installing and executing this program on a computer, the system verification apparatus and system verification method of the first embodiment can be realized. In this case, the processor of the computer functions as a performance prediction formula generator 11, a verification result determiner 12, a verification environment builder 13, a verification program generator 14, a verification program executor 15, an output information generator 16, a component extractor 17, and an update determiner 18, and performs the processes.

[0167] The program in the first embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as one of the performance prediction formula generator 11, verification result determiner 12, verification environment builder 13, verification program generator 14, verification program executor 15, output information generator 16, component extractor 17, and update determiner 18.

[0168] [Physical Configuration] A computer that realizes the system verification device by executing the programs according to the first and second embodiments will now be described with reference to Fig. 18. Fig. 18 is a diagram for explaining an example of a computer that realizes the system verification device according to the first and second embodiments.

[0169] 18 , the computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be able to communicate data with each other. Note that the computer 110 may include a GPU or an FPGA in addition to or instead of the CPU 111.

[0170] The CPU 111 loads the program (code) of this embodiment stored in the storage device 113 into the main memory 112 and executes it in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory). The program of this embodiment is provided in a state stored in a computer-readable recording medium 120. The program of this embodiment may be distributed over the Internet connected via the communication interface 117. The recording medium 120 is a non-volatile recording medium.

[0171] Specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to a display device 119 and controls the display on the display device 119.

[0172] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads programs from the recording medium 120, and writes processing results from the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

[0173] Specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as flexible disks, or optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0174] The system verification apparatus in the first and second embodiments can be realized by using hardware corresponding to each unit, rather than a computer on which a program is installed. Furthermore, the system verification apparatus may be partially realized by a program and the remaining unit by hardware.

[0175] [Supplementary Note] The following supplementary note is further disclosed regarding the above-described Embodiments 1 and 2. Some or all of the above-described embodiments can be expressed by (Supplementary Note 1) to (Supplementary Note 12) described below, but are not limited to the following descriptions.

[0176] (Supplementary Note 1) A system verification device comprising: a performance prediction formula generation unit that generates, in a specific system configuration that embodies system requirements including an abstract part, a performance prediction formula used to determine whether the performance of a specific part that embodies the abstract part satisfies the performance defined in the abstract part, based on information that is preset for quantitative requirements of components included in the specific part; and a verification result determination unit that calculates a performance prediction value for the quantitative requirement using the performance prediction formula and performance measurement values ​​used in the performance prediction formula, and determines whether the calculated performance prediction value satisfies the performance defined in the abstract part.

[0177] (Supplementary Note 2) The system verification device according to Supplementary Note 1 further comprises: a verification environment construction unit that constructs a verification environment defined in a specific system configuration when the performance prediction formula generation unit is unable to acquire performance measurements; a verification program generation unit that generates a verification program that executes a performance measurement process in the verification environment; and a verification program execution unit that outputs a result of executing the verification program.

[0178] (Supplementary Note 3) The system verification device according to Supplementary Note 2 further comprises: a component extraction unit that acquires the specific system configuration acquired by the performance prediction formula generation unit with a certain probability and extracts one of the components included in the acquired specific system configuration; and an update determination unit that, when a relationship between the performance measurement value corresponding to the quantitative requirement of the component included in the execution result of the verification program executed by the verification program execution unit and the performance measurement value corresponding to the quantitative requirement of the component included in performance measurement information in advance matches a predefined update criterion, updates the performance measurement value included in the performance measurement information with the performance measurement value corresponding to the quantitative requirement of the component.

[0179] (Supplementary Note 4) The system verification device according to any one of Supplementary Notes 1 to 3, further comprising: an output information generation unit that outputs to an output device a user interface having an input form for creating the system requirements, an output form that displays the specific system configuration that embodies the system requirements, and a detailed information display unit that displays information representing the components included in the system requirements and the specific system configuration and the relationships between the components.

[0180] (Supplementary Note 5) A system verification method in which a computer generates a performance prediction formula used to determine whether the performance of a concrete part that embodies a system requirement including an abstract part satisfies the performance defined in the abstract part, based on information previously set for quantitative requirements of components included in the concrete part, calculates a performance prediction value for the quantitative requirement using the performance prediction formula and performance measurement values ​​used in the performance prediction formula, and determines whether the calculated performance prediction value satisfies the performance defined in the abstract part.

[0181] (Appendix 6) The system verification method described in Appendix 5, wherein the computer further: if the performance measurement value cannot be obtained, creates a verification environment defined in the specific system configuration; generates a verification program that executes a performance measurement process in the verification environment; and outputs the results of executing the verification program.

[0182] (Supplementary Note 7) The system verification method described in Supplementary Note 6, wherein the computer further acquires the acquired specific system configuration with a certain probability, extracts one of the components included in the acquired specific system configuration, and, if the relationship between the performance measurement value corresponding to the quantitative requirement of the component included in the execution result of the verification program executed by the verification program execution unit and the performance measurement value corresponding to the quantitative requirement of the component previously included in performance measurement information matches a predefined update criterion, updates the performance measurement value included in the performance measurement information with the performance measurement value corresponding to the acquired quantitative requirement of the component.

[0183] (Appendix 8) The system verification method described in any one of Appendices 5 to 7, wherein the computer further outputs to an output device a user interface having an input form for creating the system requirements, an output form for displaying the specific system configuration that embodies the system requirements, and a detailed information display unit for displaying information representing the components included in the system requirements and the specific system configuration and the relationships between the components.

[0184] (Supplementary Note 9) A computer-readable recording medium having recorded thereon a program including instructions to cause a computer to: generate, based on information preset for quantitative requirements of components included in a specific system configuration that embodies system requirements including an abstract part, a performance prediction formula used to determine whether the performance of a specific part that embodies the abstract part satisfies the performance defined in the abstract part; calculate a performance prediction value for the quantitative requirement using the performance prediction formula and performance measurement values ​​used in the performance prediction formula; and determine whether the calculated performance prediction value satisfies the performance defined in the abstract part.

[0185] (Supplementary Note 10) A computer-readable recording medium according to Supplementary Note 9, having recorded thereon a program further including instructions to cause the computer to: if the performance measurement value cannot be obtained, construct a verification environment defined in the specific system configuration; generate a verification program that executes a performance measurement process in the verification environment; and output the results of executing the verification program.

[0186] (Supplementary Note 11) A computer-readable recording medium as described in Supplementary Note 10, having recorded thereon a program further including instructions for the program to the computer to: acquire the acquired specific system configuration with a certain probability, extract one of the components included in the acquired specific system configuration, and, if the relationship between the performance measurement value corresponding to the quantitative requirements of the component included in the execution results of the verification program executed by the verification program execution unit and the performance measurement value corresponding to the quantitative requirements of the component previously included in performance measurement information matches a predefined update criterion, update the performance measurement value included in the performance measurement information with the performance measurement value corresponding to the acquired quantitative requirement of the component.

[0187] (Supplementary Note 12) A computer-readable recording medium according to any one of Supplementary Notes 9 to 11, having recorded thereon a program further including instructions for causing the computer to output to an output device a user interface having an input form for creating the system requirements, an output form for displaying the specific system configuration that embodies the system requirements, and a detailed information display section for displaying information representing the components included in the system requirements and the specific system configuration and the relationships between the components.

[0188] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0189] According to the above description, it is possible to reduce the time required for verifying an ICT system, and the present invention is useful in the field of automatic design of ICT systems.

[0190] 10 System verification device 11 Performance prediction formula generation unit 12 Verification result judgment unit 13 Verification environment construction unit 14 Verification program generation unit 15 Verification program execution unit 16 Output information generation unit 17 Component extraction unit 18 Update judgment unit 20 Storage device 30 Input device 40 Output device 110 Computer 111 CPU 112 Main memory 113 Storage device 114 Input interface 115 Display controller 116 Data reader / writer 117 Communication interface 118 Input device 119 Display device 120 Recording medium 121 Bus

Claims

1. a performance prediction formula generating means for generating a performance prediction formula used to determine whether or not a performance of a concrete part embodying a system requirement including an abstract part satisfies a performance defined in the abstract part, based on information preset for quantitative requirements of components included in the concrete part, in a concrete system configuration embodying the system requirement including an abstract part; a verification result determination means for calculating a performance prediction value for the quantitative requirement by using the performance prediction formula and a performance measurement value used in the performance prediction formula, and determining whether or not the calculated performance prediction value satisfies the performance defined in the abstract part; A system verification device having the above configuration.

2. a verification environment construction means for constructing a verification environment defined in a specific system configuration when the performance prediction formula generation means cannot obtain a performance measurement value; a verification program generating means for generating a verification program for executing a performance measurement process in the verification environment; a verification program execution means for outputting a result of execution of the verification program; The system verification apparatus of claim 1 further comprising:

3. a component extracting means for acquiring the specific system configuration acquired by the performance prediction formula generating means with a certain probability and extracting one of the components included in the acquired specific system configuration; an update determination means for updating the performance measurement value included in the performance measurement information with the performance measurement value corresponding to the acquired quantitative requirement of the component when a relationship between the performance measurement value included in the execution result of the verification program executed by the verification program execution means and the performance measurement value corresponding to the quantitative requirement of the component included in the performance measurement information matches a predefined update criterion; The system verification apparatus of claim 2 further comprising:

4. an output information generating means for outputting to an output device a user interface having an input form for creating the system requirements, an output form for displaying the specific system configuration that embodies the system requirements, and a detailed information display section for displaying information representing the components and relationships between the components included in the system requirements and the specific system configuration; 4. The system verification device according to claim 1, further comprising:

5. The computer In a specific system configuration that embodies a system requirement including an abstract part, a performance prediction formula used to determine whether or not the performance of a specific part that embodies the abstract part satisfies the performance defined in the abstract part is generated based on information that is preset for quantitative requirements of components included in the specific part; calculating a performance prediction value for the quantitative requirement using the performance prediction formula and a performance measurement value used in the performance prediction formula, and determining whether or not the calculated performance prediction value satisfies the performance defined in the abstract part; System validation methodology.

6. The computer further comprises: If the performance measurements cannot be obtained, construct a verification environment defined in the specific system configuration; generating a verification program for executing a performance measurement process in the verification environment; outputting a result of executing the verification program; 6. The system verification method of claim 5.

7. The computer further comprises: The specific system configuration is acquired with a certain probability, and one of the components included in the acquired specific system configuration is extracted; when a relationship between the performance measurement value corresponding to the quantitative requirement of the component included in the execution result of the verification program and the performance measurement value corresponding to the quantitative requirement of the component included in the performance measurement information in advance matches a predefined update criterion, the performance measurement value included in the performance measurement information is updated with the performance measurement value corresponding to the acquired quantitative requirement of the component; 7. The system verification method of claim 6.

8. The computer further comprises: outputting, to an output device, a user interface having an input form for creating the system requirements, an output form for displaying the specific system configuration that embodies the system requirements, and a detailed information display section for displaying information representing the components and relationships between the components included in the system requirements and the specific system configuration; A system verification method according to any one of claims 5 to 7.

9. On the computer, In a specific system configuration that embodies a system requirement including an abstract part, a performance prediction formula used to determine whether or not the performance of a specific part that embodies the abstract part satisfies the performance defined in the abstract part is generated based on information that is preset for quantitative requirements of components included in the specific part; calculating a performance prediction value for the quantitative requirement using the performance prediction formula and a performance measurement value used in the performance prediction formula, and determining whether or not the calculated performance prediction value satisfies the performance defined in the abstract part; program.

10. The computer includes: If the performance measurements cannot be obtained, a verification environment defined in the specific system configuration is constructed; generating a verification program for executing a performance measurement process in the verification environment; outputting a result of executing the verification program; The program according to claim 9.

Images