System Verification Device, System Verification Method, and Program

Abstract

A system verification device 10 includes: a performance prediction equation generation unit 11 that, in an embodied system configuration embodying system requirements including an abstract portion, generates, on the basis of pre-set information for quantitative requirements for constituent elements included in an embodied portion embodying the abstract portion, a performance prediction to be used to determine whether a performance defined in the abstract portion is satisfied by the performance of the embodied portion; and a verification results determination unit 12 that calculates a performance prediction value for a quantitative requirement by using a performance prediction equation and a performance prediction value used in the performance prediction equation, and determines whether the calculated performance prediction value satisfies the performance defined in the abstract portion.

Description

Technical Field

[0001] The present disclosure relates to a system verification apparatus, a system verification method used for verifying a system, and further, a program for realizing these. to the extent It relates to.

Background Art

[0002] When designing an ICT (Information Communication Technology) system, first, in requirement definition, a designer creates information (system requirements) representing the configuration of an ICT system including specific elements and abstract elements that summarize the requirements and desires of customers.

[0003] The configuration of an ICT system can be represented as a graph based on concepts such as IBN (Intent-based networking). A graph represents the elements (components) included in the configuration of an ICT system using nodes or edges. A node is a component representing, for example, a device, an application, etc. An edge is a component representing a connection relationship between two nodes.

[0004] Next, based on the pre-created concretization rules, the abstract parts included in the system requirements are concretized, and information (system specific configuration) representing the configuration of a deployable ICT system is derived.

[0005] The concretization rules are information used to concretize abstract parts step by step and convert them into specific parts. However, an abstract part may be converted into a specific part in a single conversion.

[0006] Specific parts represent components or configurations that are determined to be actually used in an ICT system. Abstract parts represent undetermined components or configurations whose functions are determined but the components or configurations actually used in the system are not specifically determined.

[0007] As a related technology, Patent Document 1 discloses a system configuration derivation device that uses machine learning to reduce the man-hours required for the design of an ICT system. According to the system configuration derivation device of Patent Document 1, when generating specific system configuration information (system specific configuration) from abstract configuration information (system requirements), a reward value is given to each of the generated system configuration and the concretization rules applied in the generation process, and it is learned by AI (Artificial Intelligence).

[0008] As a result, the AI can pseudo-acquire the knowledge related to the system design of engineers (design knowledge). Also, by performing learning on designs and reward values for various requirements, it becomes possible to speed up and enhance the reliability of the system configuration design.

Prior Art Documents

Patent Documents

[0009]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0010] However, in the system configuration derivation device of Patent Document 1, in order to confirm that the designed system specific configuration satisfies the system requirements, a verification environment equivalent to the system specific configuration is constructed. Also, in the system configuration derivation device of Patent Document 1, a verification program corresponding to each component constituting the system requirements is created. Furthermore, in the system configuration derivation device of Patent Document 1, the verification program must be executed and verified in the constructed verification environment. Therefore, it is assumed that a long time is required for constructing the verification environment, creating the verification program, and verification.

[0011] In addition, in the system configuration derivation device of Patent Document 1, when AI machine-learns the performance indicators of a system, a huge amount of teacher data regarding quantitative requirements such as performance is required. Further, in order to acquire a huge amount of teacher data regarding quantitative requirements such as performance, it is necessary to repeat design and verification, and thus it is assumed that verification takes a long time.

[0012] An example of the object of the present disclosure is to shorten the time required for verification of an ICT system.

Means for Solving the Problems

[0013] To achieve the above object, a system verification device according to one aspect of the present disclosure In a system specific configuration in which system requirements including an abstract part are materialized, a performance prediction formula used to determine whether the performance of a specific part that materializes the abstract part satisfies the performance defined in the abstract part is generated based on information preset for quantitative requirements of components included in the specific part; a performance prediction formula generation unit; Using the performance prediction formula and the performance measurement value used in the performance prediction formula, calculate a performance prediction value for the quantitative requirement, and a verification result determination unit that determines whether the calculated performance prediction value satisfies the performance defined in the abstract part; It is characterized by having.

[0014] Further, to achieve the above object, a system verification method according to one aspect of the present disclosure A computer In a system specific configuration in which system requirements including an abstract part are materialized, a performance prediction formula used to determine whether the performance of a specific part that materializes the abstract part satisfies the performance defined in the abstract part is generated based on information preset for quantitative requirements of components included in the specific part, Calculate a performance prediction value for the quantitative requirement using the performance prediction formula and the performance measurement value used in the performance prediction formula, and determine whether the calculated performance prediction value meets the performance defined in the abstract part. It is characterized by this.

[0015] Furthermore, in order to achieve the above object, in one aspect of the present disclosure program is Cause a computer to In the system specific configuration that embodies the system requirements including the abstract part, generate a performance prediction formula used to determine whether the performance of the specific part that embodies the abstract part meets the performance defined in the abstract part, based on the information set in advance for the quantitative requirements of the components included in the specific part, Calculate a performance prediction value for the quantitative requirement using the performance prediction formula and the performance measurement value used in the performance prediction formula, and determine whether the calculated performance prediction value meets the performance defined in the abstract part. It is characterized by this.

Advantages of the Invention

[0016] As described above, according to the present disclosure, the time required for verifying the ICT system can be shortened.

Brief Description of the Drawings

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

[0018] (Embodiment 1) Hereinafter, embodiments will be described with reference to the drawings. In the drawings described below, elements having the same function or corresponding functions are denoted by the same reference numerals, and repeated descriptions thereof may be omitted.

[0019] The configuration of the system verification apparatus 10 in Embodiment 1 will be described. FIG. 1 is a diagram for explaining an example of the system verification apparatus of Embodiment 1.

[0020] [Device Configuration] The system verification apparatus 10 shown in FIG. 1 is an apparatus that shortens the time required for verifying an ICT system. The system verification apparatus 10 includes a performance prediction formula generation unit 11 and a verification result determination unit 12.

[0021] The system verification apparatus 10 verifies the performance of specific parts of a system specific configuration in which abstract parts of system requirements are embodied, using the system requirements and the system specific configuration.

[0022] The system requirements are those defined by the user of the system verification apparatus 10 for the ICT system. The definition of the system requirements is, for example, information (such as programming code) that describes definitions regarding the type of elements of an OS (Operation System), the type of elements of a server computer.

[0023] Specifically, the definition of the system requirements is, for example, information (programming code) that includes, in addition to the attribute values of configuration information such as information regarding the type of elements of the ICT system, application setting information, OS setting information, etc., information that needs to be verified (information regarding verification items).

[0024] In the definition of the system requirements, there is an inheritance relationship in the types of elements (constituent elements) that make up the ICT system and the relationships between the constituent elements, and the abstract type of the inheritance source may be included.

[0025] Specifically, for example, the windows type representing Windows OS, which is a type of OS, and the ubuntu type representing Ubuntu Linux (registered trademark) OS have an inheritance relationship with the abstract os type representing the OS as the parent class and the windows type and ubuntu type as the child classes. In the definition of the system requirements, when the user does not specify the type of OS, the constituent elements of the os type can be included as elements.

[0026] Figure 2 is a diagram for explaining an example of system requirements. In A of Figure 2, a graph representing the configuration (components) of the system requirements and the quantitative requirements of the abstract part are shown. In B of Figure 2, information (programming code) representing the definition of the system requirements in A of Figure 2 is shown.

[0027] In the graph of A in Figure 2, an abstract relationship (dashed arrow) is defined, indicating that the clientapp type element "app1" representing the client application can access the webapp type element "app2" representing the web application operating on the web server with a bandwidth greater than 100 [Mbps] (megabits per second; the same applies hereinafter) and a latency less than 10 [ms] (milliseconds per second; the same applies hereinafter).

[0028] The programming code in B of Figure 2 is data representing the definition of system requirements equivalent to the graph in A of Figure 2, and may be created in a data structure format such as YAML or JSON (JavaScript Object Notation).

[0029] The programming code in B of Figure 2 consists of three fields: "components" for defining the elements of the system that make up the requirements, "relationships" for defining the relationships between the elements, and "constraints" for defining aspects such as performance.

[0030] "[app1, app2, connTo]" in "relationships" client represents a connTo type relationship representing an abstract connection relationship from application app1 to web application app2.

[0031] The "constraints" verification item "(app1, app2, connTo)::bandwidth > 100" represents that the quantitative requirement "(app1, app2, connTo)::bandwidth" representing the bandwidth between applications app1 and app2 is greater than 100 [Mbps].

[0032] Similarly, the "constraints" verification item "(app1, app2, connTo):: delay < 10" represents that the quantitative requirement "(app1, app2, connTo)::delay" representing the delay between applications app1 and app2 is less than 10 [ms].

[0033] Note that the quantitative requirements are not limited to the above-mentioned bandwidth and delay. Also, the performance (non-functional requirement) for each quantitative requirement is not limited to the performance of the above-mentioned bandwidth and delay.

[0034] One or more system specific configurations are generated for the system requirements. The above-mentioned system specific configuration is one of the one or more generated system specific configurations.

[0035] In Embodiment 1, a system requirement and one of the system specific configurations that embody the system requirement are input to the system verification apparatus 10 as a set.

[0036] FIG. 3 is a diagram for explaining an example of a system specific configuration. A in FIG. 3 is a graph showing the configuration of the system specific configuration. That is, it is a graph without an abstract part. B in FIG. 3 shows the information (programming code) representing the definition of the system specific configuration.

[0037] The definition of the system specific configuration may include, for example, attribute values of configuration information such as information regarding the type of elements, application setting information, and OS setting information.

[0038] Specifically, the ubuntu-type component representing the Ubuntu Linux OS includes an attribute value "version" for specifying the OS version, and one of the strings such as "20.04", "21.10", "22.04", etc., which is the version of the OS, is defined as the value of "version".

[0039] The graph of A in FIG. 3 shows a configuration in which each application operates on an OS on an independent physical machine and the physical machines are connected by a router to realize communication between applications app1 and app2.

[0040] Applications app1 and app2 are respectively connected to the OS-type components os1 and os2 representing the OS by a wire:OS-type relationship (solid arrow) representing the host relationship of the application on the OS. Also, the attribute value "osType: ubuntu", which indicates that Ubuntu Linux (registered trademark) is used as the OS type, is stored in os1 and os2.

[0041] Similarly, the OS-type components os1 and os2 are respectively connected to the machine-type components machine1 and machine2 representing the physical machines by a wire:Machine-type relationship (solid arrow) representing the operation of the OS on the machine. And the physical machines machine1 and machine2 are respectively connected to the router-type component router1 representing the router by a wire:Router-type relationship (solid arrow) representing the network connection to the router.

[0042] The performance prediction formula generation unit 11 generates a performance prediction formula used to determine whether the performance of the specific part (verification item) obtained by materializing the abstract part in the system specific configuration that materializes the system requirements including the abstract part meets the performance defined in the abstract part, based on the information set in advance for the quantitative requirements of the components included in the specific part.

[0043] Specifically, the performance prediction formula generation unit 11 first acquires a set of system requirements and the specific system configuration from the storage device. Next, the performance prediction formula generation unit 11 generates a performance prediction formula used to verify the performance of the specific part (verification item) of the system specific configuration, which is the concretization of the abstract part of the system requirements.

[0044] Next, the performance prediction formula generation unit 11 refers to the performance measurement information stored in the storage device, which is generated in advance and will be described later, and acquires the performance measurement values used in the performance prediction formula from the performance measurement information. However, the verification result determination unit 12 may acquire the performance measurement values used in the performance prediction formula.

[0045] The generation of the performance prediction formula will be described. FIG. 4 is a diagram for explaining an example of the generation of the performance prediction formula. In the graph of A in FIG. 4, as a system requirement, it is defined that the communication bandwidth from the camera type component camera1 representing a camera to the face - app type component app1 representing a face recognition application is greater than 100 [Mbps].

[0046] The graph of B in FIG. 4 is one of the system specific configurations generated by concretizing the abstract part of the system requirements shown in A of FIG. 4. Note that the concretization of the abstract part can be executed using, for example, the learning - type system automatic design technology disclosed in Patent Document 1 described above.

[0047] Specifically, the graph of B in FIG. 4 has a machine - type component machine1, a machine - type component machine2, and a router - type component router1. The machine - type component machine1 represents a physical machine connected to the camera. The machine - type component machine2 represents a physical machine on which the face recognition application is operating. The router - type component router1 represents a router connecting the two physical machines.

[0048] In addition, the graph of B in FIG. 4 has the relationship (solid arrow) between the machine - type component machine1 and the router - type component router1, and the relationship (solid arrow) between the router - type component router1 and the machine - type component machine2. Note that it is shown as a result of concretizing the abstract (dashed arrow) relationship between the camera and the face recognition application of the system requirements.

[0049] Also, when the abstract part is concretized (when the concretization process is executed), as the history of concretizing the abstract part, information associating the information representing the abstract part and the information representing the concrete part (information representing the relationship between components and relationships) is recorded in the storage device as the concretization history.

[0050] The concretization history is information indicating whether the components and relationships included in the system concrete configuration are generated by applying the concretization rules to any abstract configuration element and relationships in the process from the system requirements to the design of the system concrete configuration, and is recorded for each of the components and relationships. In the example of B in FIG. 4, the relationships (app1, app2, connTo) are recorded as the concretization history of each of the components machine1, machine2, and router1. This indicates that the three - point components are generated by the concretization of the abstract relationships (app1, app2, connTo).

[0051] In addition, the performance of the abstract part (dashed line) of the graph of A in FIG. 4 is defined as the bandwidth being greater than 100 [Mbps] ("bandwidth > 100 [Mbps]"). In the graph of B in FIG. 4, "bandwidth: X = Min(X1, X2, X3) > 100 [Mbps]" is shown as a performance prediction formula for predicting the performance of the concrete part corresponding to the above - mentioned abstract part (dashed line).

[0052] In the example of FIG. 4, the performance prediction formula generation unit 11 first refers to the above-described concretization history, detects the component type after concretization, and determines that the abstract relationship between camera1 and app1 is realized by three components: machine1, machine2, and router1.

[0053] Next, the performance prediction formula generation unit 11 generates a performance prediction formula for obtaining a performance prediction value X, which is the minimum value among the performance measurement values related to the bandwidth of machine1, the performance measurement values related to the bandwidth of router1, and the performance measurement values related to the bandwidth of machine2, and which is the communication bandwidth between camera1 and app1.

[0054] Specifically, in the example of FIG. 4, since the quantitative requirement is the bandwidth, the formula used when generating the performance prediction formula related to the bandwidth is selected from the determination rules stored in advance in the storage device. The determination rule is information in which the quantitative requirement and the function are associated.

[0055] In the example of FIG. 4, since the quantitative requirement is the bandwidth, the performance prediction formula generation unit 11 selects the function Min() as the formula for selecting the minimum value. Note that the selected formula differs depending on the type of the quantitative requirement.

[0056] For other function types, Max() for selecting the largest value among the arguments, Sum() for calculating the sum of the values of the arguments, Average() for calculating the average value of the values of the arguments, etc. may be used.

[0057] The formula of the performance prediction formula takes the performance values of the components and relationships included in the specific system configuration as arguments, and the type of the function is determined by the type of the quantitative requirement. The components and relationships that use the performance value as an argument of the function are determined by recursively searching for the same components or relationships from the components or relationships in which the quantitative requirement is defined to the components and relationships included in the concretization history, and adding the components and relationships that match the type defined in advance for each type of the quantitative requirement.

[0058] In the example of B in Figure 4, the bandwidth quantification requirement searches for elements in the realization history that embody the defined relationships (app1, app2, connTo), and adds the variables X1, X2, and X3, which represent the performance values of the machine type and router type elements machine1, router1, and machine2 defined as the targets for adding arguments in the prediction of bandwidth performance, as arguments to the function Min() used for predicting the bandwidth performance respectively, thereby generating the function Min(X1, X2, X3) as the performance prediction formula for the bandwidth X between the applications app1 and app2.

[0059] In this way, the performance prediction formula generation unit 11 generates the performance prediction formula "X = Min(X1, X2, X3) > 100 [Mbps]". In the performance prediction formula, X1 is a variable representing the performance measurement value related to the bandwidth of machine1. X2 is a variable representing the performance measurement value related to the bandwidth of machine2. X3 is a variable representing the performance measurement value related to the bandwidth of router1.

[0060] The verification result determination unit 12 acquires the performance measurement values used in the performance prediction formula, calculates the performance prediction value of the quantification requirement using the acquired performance measurement values and the performance prediction formula, and determines whether the calculated performance prediction value meets the performance defined in the abstract part.

[0061] After that, for generating the verification result, the verification result determination unit 12 outputs the system specific configuration, the verification item, the performance prediction value, and the determination result to the output information generation unit 16.

[0062] Note that the acquisition of the performance measurement values used in the performance prediction formula may be performed by the verification result determination unit 12 or the performance prediction formula generation unit 11.

[0063] Specifically, the verification result determination unit 12 first obtains, from the performance measurement information, the measured values of the bandwidth performance of the machine-type components, the measured values of the bandwidth performance of the router-type components, and the measured values of the bandwidth performance of the machine-type components, which are substituted into the variables X1, X2, and X3 included in the performance prediction formula "X = Min(X1, X2, X3) > 100 [Mbps]".

[0064] Next, the verification result determination unit 12 calculates the performance prediction values by substituting the measured values into the variables X1, X2, and X3.

[0065] FIG. 5 is a diagram for explaining an example of the data structure of the performance measurement information. In the example of FIG. 5, the performance measurement information is information in which components, quantitative requirements, and measured values are associated. Specifically, the performance measurement information is information that associates information representing the type of component, information representing the type of quantitative requirement, and the measured value.

[0066] In the example of FIG. 5, the machine-type component representing the physical machine is associated with the quantitative requirement bandwidth representing the bandwidth and the performance measurement value of the bandwidth of 990 [Mbps]. However, the unit may not be necessary.

[0067] Also, in the example of FIG. 5, the machine-type component representing the physical machine is associated with the quantitative requirement delay representing the delay and the performance measurement value of the delay of 0.1 [ms]. However, the unit may not be necessary.

[0068] Also, in the example of FIG. 5, the machine-type component representing the physical machine is associated with the quantitative requirement availability representing the availability and the performance measurement value of the availability of 0.9, respectively.

[0069] In this way, in Embodiment 1, the performance prediction value of the quantitative requirement is calculated using the performance prediction formula and the performance measurement values for each component of the ICT system previously stored (accumulated) in the storage device, and it can be determined whether the calculated performance prediction value meets the performance defined in the abstract part.

[0070] Therefore, according to Embodiment 1, the construction of the verification environment, the creation of the verification program, and the verification can be omitted, so the performance verification can be completed in a short time. Furthermore, the working hours of engineers can be shortened (the burden can be reduced).

[0071] [System Configuration] Subsequently, with reference to FIG. 6, the configuration of the system verification apparatus 10 in the embodiment will be described in more detail. FIG. 6 is a diagram for explaining an example of a system having the system verification apparatus of Embodiment 1.

[0072] The system 100 includes at least a system verification apparatus 10, a storage device 20, an input device 30, and an output device 40. The system verification apparatus 10, the storage device 20, the input device 30, and the output device 40 are communicably connected via a network.

[0073] The system verification apparatus 10 is an information processing apparatus such as a programmable device such as a CPU (Central Processing Unit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or a circuit, a server computer, a personal computer, or a mobile terminal equipped with any one or more of them.

[0074] The storage device 20 is a database, a server computer, a circuit having a memory, etc. The storage device 20 stores at least the information described later. The storage device 20 stores, for example, at least information such as system requirements, system specific configurations, type definitions of system components and relationships, concretization rules, determination rules, concretization history, and templates for generating verification programs.

[0075] In the example of FIG. 6, the storage device 20 is provided outside the system verification device 10, but it may also be provided inside the system verification device 10. Further, the storage device 20 may be composed of a plurality of storage devices, and the above-described information may be distributed and stored.

[0076] The input device 30 is, for example, a device such as a keyboard, a mouse, or a touch panel. The input device 30 is used when operating the system verification device 10, the output device 40, and the like.

[0077] The output device 40 acquires output information, which has been converted into an outputable format by an output information generation unit 16 described later, and outputs generated images, sounds, etc. based on the output information. The output device 40 is, for example, an image display device using liquid crystal, organic EL (Electro Luminescence), CRT (Cathode Ray Tube), etc. Further, the image display device may be provided with a sound output device such as a speaker. Note that the output device 40 may be a printing device such as a printer.

[0078] The communication network is, for example, a general network constructed using communication lines such as the Internet, LAN (Local Area Network), dedicated line, telephone line, enterprise internal network, mobile communication network, Bluetooth (registered trademark), WiFi (Wireless Fidelity).

[0079] (System Verification Device) The system verification device will be described in detail. The system verification device 10 in Embodiment 1 includes a performance prediction formula generation unit 11, a verification result determination unit 12, a verification environment construction unit 13, a verification program generation unit 14, a verification program execution unit 15, and an output information generation unit 16.

[0080] Note that the performance prediction formula generation unit 11 and the verification result determination unit 12 have been described above, so the description of the performance prediction formula generation unit 11 and the verification result determination unit 12 will be omitted.

[0081] When the performance prediction formula generation unit 11 cannot obtain the performance measurement value from the performance measurement information, the verification environment construction unit 13 constructs the verification environment defined in the specific system configuration.

[0082] Specifically, the verification environment construction unit 13 first obtains from the performance prediction formula generation unit 11 a determination result indicating whether the performance measurement value included in the specific part corresponding to the abstract part generated when the performance prediction formula generation unit 11 generates the performance prediction formula is included in the performance measurement information.

[0083] Next, when the determination result indicates that the performance measurement value is not included in the performance measurement information, the verification environment construction unit 13 constructs a verification environment based on the definition in the specific system configuration in order to perform the performance measurement of the unobtainable performance measurement value. The verification environment is preferably constructed, for example, on a virtual environment or the like.

[0084] The verification program generation unit 14 generates a verification program for executing the performance measurement process in the verification environment constructed by the verification environment construction unit 13. The verification program generation unit 14 generates a verification program using, for example, the technology disclosed in Japanese Patent Application Laid-Open No. 2021-165930. However, it is not limited to the above-described technology.

[0085] Specifically, the verification program generation unit 14 obtains the quantitative requirements and the specific system configuration, refers to a template corresponding to the type of the preset quantitative requirements, and explores the graph structure of the configuration plan to adjust the execution subject and parameters of the verification program, thereby generating a verification program for execution on the verification environment.

[0086] The template has the type of quantitative requirements as a key, and defines the type of command required for verifying the quantitative requirements, and the search procedure in the graph structure of the configuration plan of the execution subject and parameters required for executing the command.

[0087] FIG. 7 is a diagram for explaining an example of the search for an execution entity and parameters. The search procedure described in FIG. 7 is composed of a starting point of the search described in the symbol "<>" and search rules 1 and 2 described separately by the symbol "+".

[0088] The search is executed in the order of search rule 1 and search rule 2. In the example of FIG. 7, the search procedure " <app1>"(HostedOn, OS) + (Join, LAN)" means that, for the graph structure of the specific system configuration, starting from the node of component app1, the edge of the relationship HostedOn is repeatedly traversed one or more times, and starting from the node of the OS-type component first reached, the edge of the relationship Join is repeatedly traversed one or more times, and the node of the LAN-type component first reached is taken as the search result.

[0089] Figure 8 is a diagram for explaining an example of the search operation. The method of searching for the execution entity of component app1 of the application included in the specific system configuration and the search operation in the structure of the specific system configuration will be described with reference to Figure 8.

[0090] In the example of Figure 8, the definition of the search procedure " <app1>In accordance with "(HostedOn, OS)", starting from the node of component app1, search along the edges of the relationships inherited from the HostedOn type, namely "Wire:MW" and "Wire:OS", and end the search at the node of OS1, which is an OS type component, and output the node as the search result.

[0091] FIG. 9 is a diagram for explaining an example of verification information. The verification program is a programming code including information (verification information) necessary for verification as shown in FIG. 9. The verification information is stored in association with a quantitative requirement, the execution order of the quantitative requirement, the execution entity, and a script for verifying the quantitative requirement.

[0092] The verification program execution unit 15 executes the verification program generated by the verification program generation unit 14 and outputs information (performance measurement result) representing the result of executing the verification program. Specifically, the verification program execution unit 15 first executes the verification program in the verification environment, and as a result of the execution, generates a performance measurement result associating a component, the quantitative requirement of the component, and the performance measurement value of the quantitative requirement.

[0093] Next, the verification program execution unit 15 adds (stores) the output performance measurement result to the performance measurement information in the storage device 20. Note that the output performance measurement result is used by the verification result determination unit 12 for calculating the performance prediction value.

[0094] The output information generation unit 16 acquires information necessary for generating the verification result from the verification result determination unit 12 (at least acquires the system specific configuration, the verification item, the performance prediction value, and the determination result), generates output information for outputting the verification result to the output device 40, and outputs it to the output device 40.

[0095] Note that in addition to the verification result, the output information generation unit 16 generates, for example, a screen of the user interface of the design / evaluation tool of the ICT system as output information and outputs it to the output device 40.

[0096] FIG. 10 is a diagram for explaining an example of verification results. The verification results describe the verification results of the specific system configuration. FIG. 10 shows information in which, for each of a plurality of specific system configurations generated by embodying the system requirements shown in FIG. 2, a verification item, a predicted performance value for the verification item, and a determination result of the verification item are associated with each other.

[0097] The programming code at the first stage of the "specific system configuration" in FIG. 10 corresponds to the specific system configuration shown in FIG. 3. Further, in the programming code at the first stage, a verification item "(app1, app2, connTo)::bandwidth > 100" representing a bandwidth constraint and a verification item "(app1, app2, connTo):: delay < 10" representing a delay constraint are associated with each other.

[0098] The verification result "990 [Mbps]" and the determination result "PASS" indicating that the bandwidth constraint condition is satisfied are associated with the verification item representing the bandwidth constraint. Further, the verification result "0.3 [ms]" of the verification item representing the delay constraint and the determination result "PASS" indicating that the delay constraint condition is satisfied are associated with each other.

[0099] Subsequently, the programming code at the next stage of the "specific system configuration" in FIG. 10 represents a configuration in which the physical machine machine1 on which the application app1 operates and the physical machine machine2 on which the application app2 operates are connected via a wan type component wan1 representing an Internet line. Although the constraint condition is satisfied for the verification item "(app1, app2, connTo):: delay < 10", the verification result of the verification item "(app1, app2, connTo)::bandwidth > 100" is 93.7 [Mbps] and does not satisfy the constraint condition, and it is determined as "FAIL" meaning insufficiency of the verification item.

[0100] (Design and Evaluation Tool) Using FIG. 11, the design and evaluation tool will be described. The design and evaluation tool (software program) provided in the system verification apparatus 10 is used when generating the specific system configuration from the above-described system requirements.

[0101] FIG. 11 is a diagram for explaining an example of the user interface of the design and evaluation tool. The screen display G1 in FIG. 11 represents the screen of a graphical user interface (GUI). The user is, for example, a designer who performs system design.

[0102] The screen display G1 is composed of a library display section G2, an input form G3, a design button G4, an output form G5, and a sub-window (detailed information display section) G6.

[0103] In the library display section G2, for example, the types of components of the ICT system used in the system verification apparatus 10, system design (learning-type system automatic design technology shown in Patent Document 1), etc. are arranged and displayed according to categories such as App (application), OS, and Machine.

[0104] The input form G3 is used to generate a graph representing the system requirements. The user operates the input device 30, for example, to create a graph of the system requirements shown in the input form G3 of FIG. 11. Specifically, the nodes of the components are selected from the library display section G2, and the selected nodes are arranged in the input form G3 to create a graph of the system requirements.

[0105] When the design button G4 is pressed after the system requirements are created in the input form G3, the specific system configuration is generated using the above-described learning-type system automatic design technology, etc.

[0106] The output form G5 is the specific system configuration that is the design result with It is shown. When a user operates the input device 30, for example, to select a node (component) or an edge (relationship) of the graph of system requirements displayed in the input form G3 or the graph of the specific system configuration displayed in the output form G5, detailed information on the selected component or relationship is displayed in the sub-window G6.

[0107] The example in Fig. 11 shows that the relationship (app1, app2, connTo) included in the specific system configuration shown in Fig. 3, which is the result of designing the system requirements shown in Fig. 2, has been selected.

[0108] The sub-window G6 displays information on the components included in the system requirements and the specific system configuration and the relationships between the components. Specifically, the id of the selected element is displayed in the "id" field, and the type name of the selected element is displayed in the "type" field. The "resolved" field is a flag indicating that the selected component has been replaced with a specific configuration in the design process of the learning-based system automatic design technology and does not exist in the actual configuration.

[0109] In the "properties" field, information on the attribute values of the selected element is displayed, and in the "constraints" field, verification items including quantitative requirements and information equivalent to the verification results shown in Fig. 10 are displayed.

[0110] In the example of Fig. 11, as the verification result of the verification item "(app1, app2, connTo)::bandwidth > 100" representing the bandwidth constraint between applications, "PASS" is registered in the value of "result" indicating whether the verification item is satisfied.

[0111] In addition, the value of "estimation" representing the predicted performance value of the component is "950 [Mbps]" calculated by the system verification device 10, and the value of "formula" representing the performance prediction formula, "Min(machine1, machine2, router1)", indicates that the minimum value among the performance measurement values of the components of the physical machines machine1, machine2, and router1 in the specific system configuration is used as the performance prediction value.

[0112] [Device Operation] Next, the operation of the system verification device 10 in Embodiment 1 will be described with reference to FIG. 12. FIG. 12 is a diagram for explaining an example of the operation of the system verification device of Embodiment 1. In the following description, the drawings will be referred to as appropriate. Also, in Embodiment 1, the system verification method is implemented by operating the system verification device. Therefore, the description of the system verification method in Embodiment 1 will be replaced by the following description of the operation of the system verification device.

[0113] First, the performance prediction formula generation unit 11 acquires a set of system requirements and the specific system configuration from the storage device 20 (step A1).

[0114] Next, the performance prediction formula generation unit 11 extracts the specific parts (verification items) of the specific system configuration that embody the abstract part of the system requirements (step A2). Next, the performance prediction formula generation unit 11 generates a performance prediction formula to be used for verifying the performance of each verification item (step A3).

[0115] Next, when the performance prediction formula generation unit 11 refers to the performance measurement information stored in the storage device 20 and the performance measurement value used in the performance prediction formula is stored in the performance measurement information (step A4: Yes), the performance prediction formula generation unit 11 acquires the performance measurement value used in the performance prediction formula from the performance measurement information (step A5). However, the verification result determination unit 12 may acquire the performance measurement value used in the performance prediction formula.

[0116] Next, the verification result determination unit 12 acquires the performance measurement values used in the performance prediction formula, and calculates the performance prediction value of the quantitative requirement using the acquired performance measurement values and the performance prediction formula (step A6). Next, the verification result determination unit 12 determines whether the calculated performance prediction value satisfies the performance defined in the abstract part (whether the performance predicted value satisfies the constraints) (step A7).

[0117] After that, the verification result determination unit 12 outputs at least the system specific configuration, the verification item, the performance prediction value, and the determination result to the output information generation unit 16 in order to generate the verification result.

[0118] Next, the output information generation unit 16 acquires the information necessary to generate the verification result from the verification result determination unit 12, generates output information for outputting to the output device 40, and outputs it to the output device 40 (step A8).

[0119] Furthermore, when the performance prediction formula generation unit 11 refers to the performance measurement information stored in the storage device 20 and the performance measurement values used in the performance prediction formula are not stored in the performance measurement information (step A4: No), the verification environment construction unit 13 constructs the environment defined in the system specific configuration (step A9).

[0120] Next, the verification program generation unit 14 generates a verification program for executing the performance measurement process in the environment constructed by the verification environment construction unit 13 (step A10).

[0121] Next, the verification program execution unit 15 executes the verification program generated by the verification program generation unit 14 (step A11), outputs information (performance measurement result) representing the result of executing the verification program, and adds (stores) the output performance measurement result to the performance measurement information. After that, the process proceeds to step A4.

[0122] [Effects of Embodiment 1] According to Embodiment 1, a performance prediction value of quantitative requirements can be calculated using a performance prediction formula and performance measurement values for each component of the ICT system previously stored (accumulated) in a storage device, and it can be determined whether the calculated performance prediction value satisfies the performance defined in the abstract part.

[0123] Therefore, construction of a verification environment, creation of a verification program, and verification can be omitted, so that performance verification can be performed in a short time. Furthermore, the working hours of engineers can be shortened (the burden can be reduced).

[0124] [Program] The program in Embodiment 1 may be a program that causes a computer to execute steps A1 to A12 shown in FIG. 12. By installing and executing this program on a computer, the system verification apparatus and the system verification method in Embodiment 1 can be realized. In this case, the processor of the computer functions as a performance prediction formula generation unit 11, a verification result determination unit 12, a verification environment construction unit 13, a verification program generation unit 14, a verification program execution unit 15, and an output information generation unit 16, and performs processing.

[0125] Also, the program in Embodiment 1 may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as any one of a performance prediction formula generation unit 11, a verification result determination unit 12, a verification environment construction unit 13, a verification program generation unit 14, a verification program execution unit 15, and an output information generation unit 16.

[0126] (Embodiment 2) The configuration of the system verification apparatus 10' in Embodiment 2 will be described. FIG. 13 is a diagram showing an example of the system verification apparatus of Embodiment 2.

[0127] [Device Configuration] The system verification device 10' shown in FIG. 13 includes a performance prediction formula generation unit 11, a verification result determination unit 12, a verification environment construction unit 13, a verification program generation unit 14, a verification program execution unit 15, an output information generation unit 16, a component extraction unit 17, and an update determination unit 18.

[0128] The component extraction unit 17 first obtains the system specific configuration acquired by the performance prediction formula generation unit 11 with a certain probability, and extracts one of the components included in the acquired system specific configuration.

[0129] Specifically, for each system specific configuration acquired by the performance prediction formula generation unit 11, it is determined whether to be a component extraction target with a predefined probability (e.g., 5%, 10%, etc.). For the system specific configuration determined to be an extraction target, one component is extracted.

[0130] Next, the component extraction unit 17 refers to the performance measurement information of the storage device 20, obtains a list of quantitative requirements related to the extracted component, and outputs the information associating the acquired system specific configuration with the list of quantitative requirements to the verification environment construction unit 13.

[0131] FIG. 14 is a diagram for explaining an example of the operation of the component extraction unit. In the example of FIG. 14, the component extraction unit 17 first obtains the system specific configuration 141 (equivalent to the system specific configuration in the figure 4 from the performance prediction formula generation unit 11 with a certain probability.

[0132] Next, the component extraction unit 17 randomly extracts a component included in the system specific configuration 141. In the example of FIG. 14, router1 of the router type component 142 is extracted.

[0133] Next, the component extraction unit 17 refers to the performance measurement information 143 of the storage device 20, and acquires, as quantitative requirements corresponding to the extracted router-type components 142, the quantitative requirement of bandwidth "bandwidth", the quantitative requirement of delay "delay", and the quantitative requirement of availability "availability" as the quantitative requirements to be verified.

[0134] The component extraction unit 17 outputs the system specific configuration 141, the components 142, and the list 144 of quantitative requirements acquired and extracted by the above-described procedure, and inputs them to the verification environment construction unit 13.

[0135] The update determination unit 18 first is the execution result of the verification program acquires the performance measurement results executed by the verification program execution unit 15. The performance measurement results are the execution results of the verification program for the purpose of verifying the quantitative requirements of the components. Also, it is a set of a component, a quantitative requirement, and a performance measurement value.

[0136] Next, the update determination unit 18 determines whether the performance measurement value corresponding to the quantitative requirement of the acquired component matches the performance measurement value corresponding to the quantitative requirement of the same component included in the performance measurement information stored in advance in the storage device 20 and defined in advance update criteria.

[0137] Next, when the update determination unit 18 matches the update criteria, it updates the performance measurement value corresponding to the quantitative requirement of the same component as the acquired component included in the performance measurement information stored in the storage device 20 with the performance measurement value corresponding to the quantitative requirement of the acquired component.

[0138] In addition, when the performance measurement value corresponding to the acquired component and the quantitative requirement is not recorded in the performance measurement information of the storage device 20, the performance measurement value corresponding to the acquired component is added (recorded) to the performance measurement information.

[0139] FIG. 15 is a diagram for explaining an example of the operation of the update determination unit. A in FIG. 15 represents an operation of comparing "measurement value X" and "measurement value Y" with respect to the component router and the quantitative requirements bandwidth, delay, and availability.

[0140] "Measurement value X" represents a value already stored in the storage device 20. "Measurement value Y" represents an execution result verified using the verification program generation unit 14 and the verification program execution unit 15 in the verification environment generated by the verification environment construction unit 13.

[0141] The update criterion is a condition for updating (overwriting) measurement value X with the new measurement value Y. In the example of FIG. 15, it is expressed as an inequality using the literal expressions X and Y.

[0142] In the example of A in FIG. 15, with respect to the quantitative requirements bandwidth and availability, 990 [Mbps] and 0.99 are stored in measurement value X respectively, and 995 [Mbps] and 0.999 are stored in measurement value Y. When measurement value Y is greater than measurement value X, the update criterion "X < Y" for updating the performance measurement value is satisfied.

[0143] As shown in the performance measurement information of B in FIG. 15, only the performance measurement values of the quantitative requirements bandwidth and availability are updated. On the other hand, with respect to the quantitative requirement delay, 0.1 [ms] is stored in measurement value X and 0.15 [ms] is stored in measurement value Y. Since the update criterion "X > Y" is not satisfied, the performance measurement value of the performance measurement information is not updated.

[0144] [Device Operation] FIGS. 16 and 17 are diagrams for explaining an example of the operation of the system verification device according to Embodiment 2. In the following description, the figures will be referred to as appropriate. Also, in Embodiment 2, the system verification method is implemented by operating the system verification device. Therefore, the description of the system verification method in Embodiment 2 will be replaced by the following description of the operation of the system verification device.

[0145] In addition, in FIG. 16, the same steps as those in FIG. 12 of Embodiment 1 are denoted by the same reference numerals, and detailed description of the steps is omitted.

[0146] First, the performance prediction formula generation unit 11 acquires a set of system requirements and system specific configurations from the storage device 20 (step A1). Next, the component extraction unit 17 acquires the system specific configuration input to the performance prediction formula generation unit 11 with a certain probability (step B1).

[0147] Next, when the component extraction unit 17 acquires the system specific configuration (step B1: Yes), in the process of step B2, the performance measurement value is updated (step B2). Note that the process of updating the performance measurement value in step B2 may be executed asynchronously with the processes after step A2.

[0148] Also, when the component extraction unit 17 does not acquire the system specific configuration (step B1: No), the process proceeds to step A2, and the processes after step A2 (the processes from step A2 to A12 and B3) are executed.

[0149] Next, after the verification program is executed in step A11, the update determination unit 18 determines whether to update the performance measurement information based on the execution result of the verification program (step B3).

[0150] Specifically, in step B3, the update determination unit 18 first is the execution result of the verification program acquires the performance measurement result (a set of a component, a quantitative requirement, and a performance measurement value, which is the execution result of the verification program for verifying the quantitative requirement of the component).

[0151] Next, in step B3, the update determination unit 18 compares the performance measurement value of the acquired component with the performance measurement value of the same component stored in the storage device 20.

[0152] Next, when an update is to be performed in step B3 (step B3: Yes), the update determination unit 18 updates the performance measurement value corresponding to the same current component as the acquired component, using the predefined update criteria and the performance measurement value corresponding to the acquired component, for each type of quantitative requirement (step A12).

[0153] Note that when an update is to be performed in step B3 (step B3: Yes), if the performance measurement value corresponding to the acquired component and the quantitative requirement is not recorded in the performance measurement information of the storage device 20, the update determination unit 18 adds (records) the performance measurement value to the performance measurement information (step A12).

[0154] Also, when no update is performed in step B3 (step B3: No), the process proceeds to step A4 to continue the processing.

[0155] The details of the process of step B2 (performance measurement value update process) will be described. The component extraction unit 17 first acquires the specific system configuration input to the performance prediction formula generation unit 11 with a certain probability, and extracts one of the components included in the acquired specific system configuration (step C1).

[0156] Next, the component extraction unit 17 refers to the performance measurement information 143 of the storage device 20, acquires a list of quantitative requirements related to the extracted component (step C2), and outputs information associating the acquired specific system configuration 141 and the list of quantitative requirements to the verification environment construction unit 13 (step C3).

[0157] Next, the verification environment construction unit 13 constructs the environment defined in the specific system configuration (step A9).

[0158] Next, the verification program generation unit 14 generates a verification program for executing performance measurement processing in the environment constructed by the verification environment construction unit 13 (step A10).

[0159] Next, the verification program execution unit 15 executes the verification program generated by the verification program generation unit 14 (step A11).

[0160] Next, when an update is to be performed in step C4 (step C4: Yes), the update determination unit 18 updates the performance measurement value corresponding to the same current component as the acquired component by using the predefined update criteria for each type of quantitative requirement and the performance measurement value corresponding to the acquired component (step A12).

[0161] Note that when an update is to be performed in step C4 (step C4: Yes), if the performance measurement value corresponding to the quantitative requirement of the acquired component is not recorded in the performance measurement information of the storage device 20, the update determination unit 18 adds (records) the performance measurement value to the performance measurement information (step A12).

[0162] Also, when no update is performed in step C4 (step C4: No), the process proceeds to step A4.

[0163] [Effect of Embodiment 2] According to Embodiment 2, the performance prediction value of the quantitative requirement can be calculated by using the performance prediction formula and the performance measurement values of each component of the ICT system stored (accumulated) in advance, and it can be determined whether the calculated performance prediction value satisfies the performance defined in the abstract part.

[0164] Therefore, the construction of the verification environment, the creation of the verification program, and the verification can be omitted, so that the performance verification can be performed in a short time. Furthermore, the working hours of engineers can be shortened (the burden can be reduced).

[0165] Also, according to Embodiment 2, regarding the performance measurement values of the components of the performance measurement information, the construction of the verification environment and the performance measurement are executed with a certain probability, and the performance measurement values of each quantitative requirement corresponding to the components can be automatically updated. Therefore, the accuracy of the performance prediction value of the quantitative requirement calculated by the performance prediction formula is improved, and a more accurate system performance verification becomes possible.

[0166] [Program] The program in Embodiment 2 may be any program that causes a computer to execute steps A1 to A12, B1 to B3 shown in FIG. 16, and steps C1 to C4, A9 to A12 shown in FIG. 17. By installing and executing this program on a computer, the system verification apparatus and the system verification method in Embodiment 1 can be realized. In this case, the processor of the computer functions as a performance prediction formula generation unit 11, a verification result determination unit 12, a verification environment construction unit 13, a verification program generation unit 14, a verification program execution unit 15, an output information generation unit 16, a component extraction unit 17, and an update determination unit 18, and performs processing.

[0167] Also, the program in Embodiment 1 may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as any one of a performance prediction formula generation unit 11, a verification result determination unit 12, a verification environment construction unit 13, a verification program generation unit 14, a verification program execution unit 15, an output information generation unit 16, a component extraction unit 17, and an update determination unit 18.

[0168] [Physical Configuration] Here, a computer that realizes the system verification apparatus by executing the programs in Embodiments 1 and 2 will be described with reference to FIG. 18. FIG. 18 is a diagram for explaining an example of a computer that realizes the system verification apparatus in Embodiments 1 and 2.

[0169] As shown in FIG. 18, the computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be capable of data communication. Note that the computer 110 may include a GPU or an FPGA in addition to or instead of the CPU 111.

[0170] The CPU 111 expands the program (code) in the present embodiment stored in the storage device 113 into the main memory 112 and executes these in a predetermined order to perform various operations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory). Also, the program in the present embodiment is provided in a state stored in a computer-readable recording medium 120. Note that the program in the present embodiment may be distributed on the Internet connected via the communication interface 117. Note that the recording medium 120 is a non-volatile recording medium.

[0171] In addition, specific examples of the storage device 113 include, in addition to a hard disk drive, a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to the display device 119 and controls the display on the display device 119.

[0172] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, and executes reading of a program from the recording medium 120 and writing of a processing result in the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

[0173] In addition, specific examples of the recording medium 120 include general-purpose semiconductor memory devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as a flexible disk, or optical recording media such as a CD-ROM (Compact Disk Read Only Memory).

[0174] Note that the system verification apparatuses in Embodiments 1 and 2 can also be realized by using hardware corresponding to each part, rather than a computer installed with a program. Further, part of the system verification apparatus may be realized by a program and the remaining part may be realized by hardware.

[0175] [Appendix] Regarding the above Embodiments 1 and 2, the following appendices are further disclosed. Part or all of the above-described embodiments can be expressed by (Appendix 1) to (Appendix 12) described below, but are not limited to the following description.

[0176] (Appendix 1) A performance prediction formula generation unit that generates a performance prediction formula used to determine whether the performance of a specific part that embodies an abstract part satisfies the performance defined in the abstract part, based on information set in advance for the quantitative requirements of the components included in the specific part, in a system specific configuration that embodies system requirements including an abstract part; A verification result determination unit that calculates a performance prediction value for the quantitative requirement by using the performance prediction formula and the performance measurement value used in the performance prediction formula, and determines whether the calculated performance prediction value satisfies the performance defined in the abstract part; A system verification apparatus having the above.

[0177] (Appendix 2) A verification environment construction unit that constructs a verification environment defined in the system specific configuration when the performance prediction formula generation unit cannot obtain a performance measurement value; A verification program generation unit that generates a verification program that executes a performance measurement process in the verification environment; A verification program execution unit that outputs the result of executing the verification program; The system verification apparatus according to Appendix 1, further having the above.

[0178] (Appendix 3) A component extraction unit that obtains the specific system configuration acquired by the performance prediction formula generation unit with a certain probability and extracts one of the components included in the acquired specific system configuration; When the relationship between the performance measurement value corresponding to the quantitative requirement of the component included in the execution result of the verification program executed by the verification program execution unit and the performance measurement value corresponding to the quantitative requirement of the component included in the performance measurement information in advance matches the predefined update criterion, an update determination unit that updates the performance measurement value included in the performance measurement information with the performance measurement value corresponding to the quantitative requirement of the acquired component; The system verification device according to Appendix 2, further comprising:

[0179] (Appendix 4) An output information generation unit that outputs to an output device a user interface having an input form for creating the system requirements, an output form for displaying the specific system configuration that embodies the system requirements, and a detailed information display unit for displaying information representing the components included in the system requirements and the specific system configuration and the relationships between the components; The system verification device according to any one of Appendices 1 to 3, further comprising:

[0180] (Appendix 5) A computer In a specific system configuration that embodies system requirements including an abstract part, a performance prediction formula used to determine whether the performance of the specific part that embodies the abstract part meets the performance defined in the abstract part is generated based on information set in advance for the quantitative requirements of the components included in the specific part; Using the performance prediction formula and the performance measurement values used in the performance prediction formula, calculate a performance prediction value for the quantitative requirement, and determine whether the calculated performance prediction value meets the performance defined in the abstract part; System verification method.

[0181] (Appendix 6) The computer further When the performance measurement value cannot be obtained, construct the verification environment defined in the specific system configuration, generate a verification program that executes performance measurement processing in the verification environment, output the result of executing the verification program, The system verification method described in Appendix 5.

[0182] (Appendix 7) The computer further obtains the specific system configuration obtained at a certain probability, extracts one of the components included in the obtained specific system configuration, when the relationship between the performance measurement value corresponding to the quantitative requirement of the component included in the execution result of the verification program executed by the verification program execution unit and the performance measurement value corresponding to the quantitative requirement of the component included in the performance measurement information in advance matches the predefined update criterion, update the performance measurement value included in the performance measurement information with the performance measurement value corresponding to the quantitative requirement of the obtained component, The system verification method described in Appendix 6.

[0183] (Appendix 8) The computer further outputs to the output device a user interface having an input form for creating the system requirements, an output form for displaying the specific system configuration that embodies the system requirements, and a detailed information display section for displaying information representing the relationship between the components included in the system requirements and the specific system configuration, The system verification method described in any one of Appendices 5 to 7.

[0184] (Appendix 9) To the computer, In a system specific configuration that embodies system requirements including abstract parts, a performance prediction formula used to determine whether the performance of the specific parts that embody the abstract parts meets the performance defined in the abstract parts is generated based on information preset for the quantitative requirements of the components included in the specific parts. Using the performance prediction formula and the performance measurement values used in the performance prediction formula, calculate a performance prediction value for the quantitative requirements, and determine whether the calculated performance prediction value meets the performance defined in the abstract parts. Program including instructions to the extent

[0185] (Appendix 10) When the program causes the computer to If the performance measurement value cannot be obtained, construct the verification environment defined in the system specific configuration. Generate a verification program that executes performance measurement processing in the verification environment. Output the result of executing the verification program. attachment As described in Note 9 program .

[0186] (Appendix 11) When the program causes the computer to Acquire the obtained system specific configuration with a certain probability, and extract one of the components included in the obtained system specific configuration. cause to , When the relationship between the performance measurement value corresponding to the quantitative requirement of the component included in the execution result of the verification program executed by the verification program execution unit and the performance measurement value corresponding to the quantitative requirement of the component included in the performance measurement information in advance matches the predefined update criterion, update the performance measurement value included in the performance measurement information with the performance measurement value corresponding to the quantitative requirement of the obtained component. cause to . attachment As described in Note 10 program .

[0187] (Appendix 12) The program causes the computer to output to an output device a user interface having an input form for creating the system requirements, an output form for displaying the system specific configuration that embodies the system requirements, and a detailed information display section for displaying information representing the components included in the system requirements and the system specific configuration and the relationships between the components. attachment any one of Claims 9 to 11 program .

[0188] Although the invention has been described with reference to the embodiments, the invention is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the invention within the scope of the invention.

Industrial Applicability

[0189] According to the above description, the time required for verifying an ICT system can be shortened. It is also useful in the field of automatically designing an ICT system.

Explanation of Reference Numerals

[0190] 10 System verification device 11 Performance prediction formula generation unit 12 Verification result determination unit 13 Verification environment construction unit 14 Verification program generation unit 15 Verification program execution unit 16 Output information generation unit 17 Component extraction unit 18 Update determination unit 20 Storage device 30 Input device 40 Output device 110 Computer 111 CPU 112 Main memory 113 Memory device 114 Input interface 115 Display controller 116 Data reader / writer 117 Communication interface 118 Input device 119 Display device 120 Recording medium 121 Bus

Claims

1. In a system specific configuration that embodies system requirements including an abstract part, a performance prediction formula generation means for generating, based on information preset for quantitative requirements of components included in the specific part, a performance prediction formula used to determine whether the performance of the specific part that embodies the abstract part meets the performance defined in the abstract part; A verification result determination means for calculating a performance prediction value for the quantitative requirements using the performance prediction formula and a performance measurement value used in the performance prediction formula, and determining whether the calculated performance prediction value meets the performance defined in the abstract part; A system verification apparatus having the above.

2. A verification environment construction means for constructing a verification environment defined in the system specific configuration when the performance prediction formula generation means cannot obtain a performance measurement value; A verification program generation means for generating a verification program that executes a performance measurement process in the verification environment; A verification program execution means for outputting the result of executing the verification program; The system verification apparatus according to claim 1, further having the above.

3. A component extraction means for obtaining the system specific configuration acquired by the performance prediction formula generation means with a certain probability and extracting one of the components included in the acquired system specific configuration; An update determination means for updating the performance measurement value included in the performance measurement information with the performance measurement value corresponding to the quantitative requirement of the component when the relationship between the performance measurement value corresponding to the quantitative requirement of the component included in the execution result of the verification program executed by the verification program execution means and the performance measurement value corresponding to the quantitative requirement of the component included in the performance measurement information in advance matches a predefined update criterion; The system verification apparatus according to claim 2, further having the above.

4. An output information generation means for outputting to an output device a user interface having an input form for creating the system requirements, an output form for displaying the system specific configuration that embodies the system requirements, and a detailed information display section for displaying information representing the components included in the system requirements and the system specific configuration and the relationship between the components; The system verification apparatus according to any one of claims 1 to 3, further having the above.

5. A computer, In a system specific configuration that embodies system requirements including an abstract part, a performance prediction formula used to determine whether the performance of the specific part that embodies the abstract part meets the performance defined in the abstract part is generated based on information preset for the quantitative requirements of the components included in the specific part, A performance prediction value for the quantitative requirements is calculated using the performance prediction formula and the performance measurement value used in the performance prediction formula, and it is determined whether the calculated performance prediction value meets the performance defined in the abstract part. System verification method.

6. The computer further When the performance measurement value cannot be obtained, constructs a verification environment defined in the system specific configuration, Generates a verification program that executes performance measurement processing in the verification environment, Outputs the result of executing the verification program. The system verification method according to claim 5.

7. The computer further Obtains the system specific configuration with a certain probability, extracts one of the components included in the obtained system specific configuration, When the relationship between the performance measurement value corresponding to the quantitative requirement of the component included in the execution result of the verification program and the performance measurement value corresponding to the quantitative requirement of the component included in the performance measurement information in advance matches the predefined update criterion, updates the performance measurement value included in the performance measurement information with the performance measurement value corresponding to the quantitative requirement of the obtained component. The system verification method according to claim 6.

8. The computer further Outputs to an output device a user interface having an input form for creating the system requirements, an output form for displaying the system specific configuration that embodies the system requirements, and a detailed information display section for displaying information representing the components included in the system requirements and the system specific configuration and the relationships between the components. The system verification method according to any one of claims 5 to 7.

9. To a computer, In a system specific configuration that embodies system requirements including an abstract part, a performance prediction formula used to determine whether the performance of the specific part that embodies the abstract part meets the performance defined in the abstract part is generated based on information preset for the quantitative requirements of the components included in the specific part. Using the performance prediction formula and the performance measurement values used in the performance prediction formula, a performance prediction value for the quantitative requirements is calculated, and it is determined whether the calculated performance prediction value meets the performance defined in the abstract part. Program.

10. For the computer, When the performance measurement value cannot be obtained, the computer is caused to construct a verification environment defined in the system specific configuration. A verification program for executing performance measurement processing in the verification environment is generated. The result of executing the verification program is output. The program according to claim 9.

Images