Test support device, test support method and program

Abstract

To enable a system satisfying test requirements to be comparatively easily generated.SOLUTION: A test support device includes test pattern generation means for generating one or more test patterns in which the value of a parameter of test object related information is determined on the basis of the test object related information showing requirements of a test determined according to a test object, including the parameter, and system requirement generation means for generating system requirements of a system satisfying the requirements of the test shown by the test patterns.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a test support device, a test support method, and a program. [Background technology]

[0002] Non-Patent Document 1 describes a technology for automatically designing a system. The technology described in Non-Patent Document 1 receives input of system requirements, generates and outputs a system configuration that satisfies the input requirements. The input system requirements are described by a combination of multiple basic requirements, and the design is performed to generate a system configuration that satisfies these multiple requirements. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] T. Kuroda, T. Kuwahara, T. Maruyama, K. Satoda, H. Shimonishi, T. Osaki, and K. Matsuda, “Weaver: A Novel Configuration Designer for IT / NW Services in Heterogeneous Environments”, in Proceedings of IEEE GLOBECOM 2019, pp. 1-6, December 2019. Summary of the Invention [Problem to be solved by the invention]

[0004] When performing performance evaluation tests of application programs that run on a system, a system (a system for evaluation tests) may be used as a test environment. In such cases, it is preferable to be able to generate a system that satisfies the requirements of the test as easily as possible.

[0005] An object of the present invention is to provide a test support device, a test support method, and a program that can solve the above-mentioned problems. [Means for solving the problem]

[0006] According to a first aspect of the present invention, a test support device is provided that supports a test by detecting a test requirement determined according to a test object. As The system further comprises a test pattern generation means for generating one or more test patterns in which values ​​of the parameters of the test target related information are determined based on the test target related information including parameters, and a system requirement generation means for generating system requirements of a system that satisfy the test requirements indicated by the test patterns.

[0007] According to a second aspect of the present invention, a test support method includes: As The method includes generating one or more test patterns in which values ​​of the parameters of the test subject related information are determined based on the test subject related information including the parameters, and generating system requirements for a system that satisfies the test requirements indicated by the test patterns.

[0008] According to a third aspect of the present invention, a program is provided for causing a computer to execute a test, the test As The program is for causing a system to execute the following operations: generating one or more test patterns in which values ​​of the parameters of the test target related information are determined based on the test target related information including parameters; and generating system requirements for a system that satisfies the test requirements indicated by the test patterns. Effect of the Invention

[0009] According to the present invention, a system that meets test requirements can be generated relatively easily. [Brief description of the drawings]

[0010] [Figure 1]1 is a schematic block diagram showing a functional configuration of a test support device according to a first embodiment. [Diagram 2] 3A to 3C are diagrams illustrating an example of data input / output in the test support device according to the first embodiment. [Diagram 3] FIG. 4 is a diagram showing an example of test-target related information according to the first embodiment; [Figure 4] 5A and 5B are diagrams illustrating examples of test pattern groups according to the first embodiment. [Diagram 5] 5A to 5C are diagrams illustrating an example of display of a test pattern group according to the first embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of system requirements according to the first embodiment. [Figure 7] 4 is a flowchart illustrating an example of a procedure of a process performed by the test support device according to the first embodiment. [Figure 8] FIG. 11 is a schematic block diagram showing an example of the configuration of a test support device according to a second embodiment. [Figure 9] 13A to 13C are diagrams illustrating an example of data input / output in the test support device according to the second embodiment. [Figure 10] FIG. 11 is a diagram showing an example of a system requirement template according to the second embodiment. [Figure 11] FIG. 13 is a diagram showing an example of an instantiation template according to the second embodiment. [Figure 12] FIG. 11 is a diagram illustrating an example of a test environment according to the second embodiment. [Figure 13] 13 is a flowchart illustrating an example of a procedure of a process performed by a test support device according to a second embodiment. [Figure 14] FIG. 13 is a schematic block diagram illustrating a configuration example of a test support system according to a third embodiment. [Figure 15] 13A to 13C are diagrams illustrating an example of data input and output in the test support system according to the third embodiment. [Figure 16] FIG. 13 is a diagram showing an example of a test result group according to the third embodiment. [Figure 17] 13 is a flowchart showing an example of a procedure of a process performed by a test support system according to the third embodiment. [Figure 18]FIG. 13 is a diagram illustrating an example of the configuration of a test support device according to a fourth embodiment. [Figure 19] 23 is a flowchart showing an example of a processing procedure in a test support method according to the fifth embodiment. [Figure 20] FIG. 1 is a schematic block diagram illustrating a configuration of a computer according to at least one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The following describes embodiments of the present invention, but the following embodiments do not limit the scope of the invention. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0012] First Embodiment (Configuration Description) Fig. 1 is a schematic block diagram showing the functional configuration of a test support device according to a first embodiment. In the configuration shown in Fig. 1, the test support device 100 includes a communication unit 110, a display unit 120, an operation input unit 130, a storage unit 180, and a control unit 190. The control unit 190 includes a test pattern generation unit 191 and a system requirement generation unit 192.

[0013] The test support device 100 supports the construction of a test environment for a test subject. The test subject here can be various things that are tested in a system as a test environment, and is not limited to a specific one. For example, the test subject may be a system or a device. Alternatively, the test subject may be hardware, software, or a combination of hardware and software used in the system or device.

[0014] The test environment here refers to the operating environment of the test subject. The test environment may be a system including the test subject. Alternatively, the test environment may be a combination of the system including the test subject and its surrounding environment. The surrounding environment of the system here may be the conditions during the operation of the system, such as the room temperature or humidity of the room in which the system is installed. The system used as the test environment is not limited to a specific type of system, for example, an ICT (Information Communication Technology) system may be used as the test environment, but is not limited thereto.

[0015] The test support device 100 may be configured using a computer such as a personal computer (PC) or a workstation. The test for which the test support device 100 provides support can be various tests performed by constructing a system, and is not limited to a specific one. For example, the test for which the test support device 100 provides support can be a performance test or an operation check test. Hereinafter, a test for which support is provided by the test support device 100 will also be simply referred to as a test.

[0016] The test support device 100 receives test target related information and outputs a group of system requirements. Test subject related information is information that indicates test requirements that are determined according to the test subject. Test subject related information indicates information required for evaluation of the test subject in an abstract form. A system requirements group is information that includes one or more system requirements. A system requirement is a requirement of a system to be used as a test environment, or information that indicates such a requirement. The system requirements here include processes such as measurements or judgments performed for testing, such as measuring CPU usage.

[0017] In some cases, such as when the test target is an application program that runs on a system, comprehensive testing may be performed using various systems as a test environment. In this case, the test support device 100 outputs a system requirement group including the system requirements of each of the multiple systems, so that comprehensive testing can be performed using each of the multiple systems as a test environment. In the following, a case will be described in which the test subject is a face recognition application program and a face recognition system is constructed as a test environment. However, as described above, the test subject and test environment handled by the test support device 100 are not limited to a specific one. The application program is also simply called an application.

[0018] The communication unit 110 communicates with other devices. For example, the communication unit 110 may transmit a group of system requirements generated by the test support device 100 to a system design device that receives the system requirements and performs automatic design of the system. The display unit 120 has a display screen such as a liquid crystal panel or an LED (Light Emitting Diode) panel, and displays various images. For example, the display unit 120 may display various information such as a group of system requirements generated by the test support device 100 in response to a user operation.

[0019] The operation input unit 130 includes input devices such as a keyboard and a mouse, and receives user operations. For example, when a user creates test target related information, the operation input unit 130 may receive a user operation for creating the test target related information. However, the method by which the test support device 100 acquires the test target related information is not limited to the method by which the user creates the information. For example, the designer of the test target may create test target related information according to the test target and register it in a database, and the communication unit 110 may receive the test target related information from the database.

[0020] The storage unit 180 stores various types of information. The storage unit 180 is configured using a storage device included in the test support apparatus 100. The control unit 190 performs various processes by controlling each unit of the test support device 100. The functions of the test support device 100 are executed, for example, by a CPU (Central Processing Unit) included in the test support device 100 reading a program from the storage unit 180 and executing it.

[0021] FIG. 2 is a diagram showing an example of data input / output in the test support device 100. As shown in FIG. The test pattern generation unit 191 receives test target related information and outputs a test pattern group. The test pattern group is information including one or more test patterns. The test pattern is information indicating requirements for tests to be performed in individual test environments.

[0022] Specifically, the test pattern generation unit 191 acquires test target related information that indicates test requirements determined according to the test target, including parameters, and generates individual test patterns by determining the values ​​of the parameters of the test target related information. The parameters referred to here can be regarded as variables that the requirements adopted when testing the test object may take. The same applies to the following embodiments. For example, the test pattern generating unit 191 selects one of the options for the parameter value of the test target related information. The test pattern generating unit 191 corresponds to an example of a test pattern generating means.

[0023] The system requirement generation unit 192 receives a test pattern group and outputs a system requirement group. Specifically, the system requirement generation unit 192 receives a test pattern group generated by the test pattern generation unit 191 and generates a system requirement for each test pattern included in the test pattern group. The system requirement generation unit 192 generates and outputs a system requirement group that compiles the generated system requirements.

[0024] The system requirement generating unit 192 may generate data including only the header data of the system requirement group as the initial data of the system requirement group, for example. Then, each time the system requirement generating unit 192 generates a system requirement, it may add the system requirement to the system requirement group to generate a system requirement group that compiles the system requirements. The system requirement generating unit 192 corresponds to an example of a system requirement generating means.

[0025] Fig. 3 is a diagram showing an example of test subject related information. In the example of Fig. 3, product data of the face recognition application product "app-face" that is the test subject is shown as the test subject related information. In the example of FIG. 3, the test subject related information includes a test subject ID (Identifier), one or more evaluation axes, and one or more evaluation items.

[0026] (Test ID) The test subject ID is identification information for identifying a test subject. In the example of Figure 3, the test subject ID "app-face" identifies the face recognition application product "app-face" that is the test subject. For example, the storage unit 180 may store test subject related information for each of a plurality of test subjects. In this case, the test pattern generation unit 191 can read out the test subject related information corresponding to the test subject from the storage unit 180 by using the test subject ID.

[0027] (Evaluation axis) The evaluation axis is information that indicates the variations (options) in the test. In the example of Figure 3, the test subject related information includes three evaluation axes: “FPS”, “resolution”, and “number of cameras”. Each evaluation axis indicates a variation in the test. The evaluation axis "FPS" indicates the frame rate of the camera (FPS: Frames per Second). On the evaluation axis "FPS", the target item "camera" indicated by the label "target:" indicates that the target of "FPS" is the camera. The values ​​"5", "10", and "15" indicated by the label "value:" indicate the frame rate options.

[0028] The evaluation axis "resolution" indicates the test video to be projected onto the camera. The target item "video" of the evaluation axis "resolution" indicates that the target of "resolution" is video. The values ​​of the evaluation axis "resolution" "VGA:file:people_vga.mp4", "FHD:file:people_fhd.mp4" and "4K:file:people_4k.mp4" indicate the options for the video format (e.g. resolution) and its file name.

[0029] The evaluation axis "Number of cameras" indicates options for the number of cameras, the additional cameras and video files that need to be deployed depending on the number, and information on the connection configuration of these elements. The target item "camera" for the evaluation axis "Number of cameras" indicates that the target of "Number of cameras" is a camera. The values ​​"1", ..., and "5" for the evaluation axis "Number of cameras" indicate options for the number of cameras.

[0030] The redundancy information indicated by the label "redundancy:" indicates a selection of the relationship between the cameras and the videos indicated by the label "nodes:". In this way, the evaluation axis may include a list of values ​​indicated by the label "value:" and information on the elements in the test environment that are the target of the values ​​indicated by the label "target:", as well as information on the scope of the effects that the values ​​have on the test environment, such as file configuration or network configuration.

[0031] A set of evaluation axes is also called an evaluation axis group. An evaluation axis group can be said to be a group of parameters that make up the variation of a test. In the example of Figure 3, the set of three evaluation axes, "FPS," "resolution," and "number of cameras," is an example of an evaluation axis group.

[0032] (Evaluation items) An evaluation item is an item to be evaluated in a test, or information indicating that item. The evaluation here may be, for example, determining a quantitative or qualitative evaluation value by measurement, calculation, or judgment. The example in Figure 3 shows two evaluation items, "CPU usage" and "Memory usage," which represent the load placed on the computer by the OS (Operating System) on which the application program being tested runs.

[0033] A set of evaluation items is also called an evaluation item group. In the example of Figure 3, the set of two evaluation items, "CPU usage" and "Memory usage", is an example of an evaluation item group. The target item "OS" for each evaluation item indicates that the evaluation target is the OS. The unit "percent" indicated by the label "unit:" indicates that the evaluation value is expressed as a percentage. Note that the specific content of the test target related information is not limited to the example shown in FIG.

[0034] Fig. 4 is a diagram showing an example of a test pattern group. A test pattern group includes one or more test patterns. In the example of Fig. 4, individual test patterns are indicated with labels "1:", "2:", ... One test pattern includes values ​​of a group of evaluation axes indicating requirements of one system as a test environment, and a group of evaluation items in a test using the test environment. The test pattern indicates the requirements of one system by determining the values ​​of each evaluation axis included in the group of evaluation axes. As described above, the test pattern generation unit 191 generates one test pattern by determining the values ​​of each evaluation axis in the test target related information. An evaluation axis whose value is not uniquely determined can be regarded as a parameter. Determining the value of an evaluation axis can be regarded as determining the value of a parameter.

[0035] For example, the test pattern indicated by the label “1:” in the example of Figure 4 can be obtained by determining the value of the evaluation axis “FPS” as “5”, the value of the evaluation axis “Resolution” as “VGA: file: people_vga.mp4”, and the value of the evaluation axis “Number of cameras” as “1” in the test subject related information in Figure 3. The evaluation axis "resolution" of this test pattern indicates that the video file "people_vga.mp4" needs to be uploaded and played. The evaluation axis "number of cameras" indicates that one camera and one video file should be installed, the video file should be projected onto the camera, and the camera image should be sent to the application "app-face".

[0036] The method by which the test pattern generation unit 191 determines the evaluation axis values ​​in the test target related information is not limited to a specific method. For example, the test pattern generation unit 191 may generate a test pattern for each possible combination of the evaluation axis values. Alternatively, the test pattern generation unit 191 may randomly select values ​​from options for the evaluation axis values. Alternatively, the test pattern generation unit 191 may display options or ranges of the evaluation axis values ​​on the display unit 120 and determine the evaluation axis values ​​according to a user operation to specify a value.

[0037] FIG. 5 is a diagram showing an example of displaying a test pattern group. Fig. 5 shows an example in which the display unit 120 displays in a table format the test pattern group exemplified in Fig. 4. For simplicity, the display format in Fig. 5 omits abstract information for constructing evaluation requirements associated with each evaluation axis, and information on the evaluation target and value format (unit) of each evaluation item. It should be noted that the specific contents and display format of the test pattern group are not limited to the examples shown in FIGS.

[0038] Fig. 6 is a diagram showing an example of system requirements. Fig. 6 shows an example of displaying an example of system requirements in a graph format on the display unit 120. Specifically, the test pattern is shown in a graph structure in which system elements are represented by nodes and connections between the elements are represented by edges. System requirements abstractly represent the elements required to build a system as a test environment. For example, by specifying the system requirements element by element, it is possible to design a system configuration that is specified to a level that can be deployed.

[0039] In the example of FIG. 6, attribute value information is added to some of the nodes and edges. For example, the attribute value information of nodes and edges indicates that the application "app-face" runs on a server that uses Ubuntu Linux (Ubuntu and Linux are each registered trademarks) as its OS. Specifically, the attribute value of the "OS" node indicates the type of OS. The connection relationship from the "app-face" node to the OS1 node is represented by a "wire:OS" edge, and the connection relationship from the OS1 node to the "machine1" node is represented by a "wire:Machine" edge. The "wire:OS" edge indicates the connection between the application and the OS. The "wire:Machine" edge indicates the connection between the OS and the server.

[0040] In addition, the "camera1" node and the "camera2" node, which are the nodes representing the cameras, are connected to a hasSource edge from a video type node, which represents a video file, indicating that the node is the projection source of the video. The "video1" node and the "video2" node are provided as video type nodes.

[0041] In the example of Figure 6, the connection relationship that connects each of the "camera1" node and the "camera2" node to the "app-face" node with a "sendVideo" edge is indicated by the description of the evaluation axis "number of cameras" of the test pattern illustrated in Figure 4. Specifically, the description of "(Video,Camera,hasSource)" in "edges:" of the "number of cameras" in the test pattern indicates that the "Video" node and the "Camera" node are connected by the "hasSource" edge. Also, the description of the value of the redundancy "redundancy" "value:" indicates the number of cameras connected to the "app-face" node.

[0042] 6 shows an example in which the value "value:" of the redundancy "redundancy" is "2." Based on this value, the system requirement generator 192 provides two "Camera" nodes, a "Camera1" node and a "Camera2" node, and connects each node to the "app-face" node via a "sendVideo" edge.

[0043] Regarding the method of acquiring the evaluation items of CPU usage and memory usage, these two evaluation items are added to the "wire:OS edge" from the "Agent1" node, which is the execution subject of the evaluation program, to the "OS1" node. This expresses that the "Agent1" node executes the evaluation program targeting the "OS1" node. Note that the solid arrows represent edges of concrete connection relationships, and the dashed arrows represent edges of abstract connection relationships. Note that the specific contents and display format of the evaluation requirements are not limited to the example shown in FIG.

[0044] (Explanation of operation) FIG. 7 is a flowchart showing an example of a procedure of processing performed by the test support device 100. In the process of FIG. 7, the test pattern generating unit 191 acquires test target related information (step S1). Then, the test pattern generation unit 191 generates a test pattern group based on the information on the evaluation axis group and the information on the evaluation item group included in the acquired test object related information (step S2). Specifically, the test pattern generation unit 191 generates one test pattern by determining the value of each evaluation axis of the evaluation axis group included in the test object related information. For example, the test pattern generation unit 191 repeats the generation of test patterns for all combinations of the values ​​of the evaluation axis group, and organizes the obtained test patterns into a test pattern group.

[0045] The test pattern generation unit 191 may generate data including only header data of the test pattern group as initial data of the test pattern group, for example. Then, every time the test pattern generation unit 191 generates a test pattern, the test pattern generation unit 191 may add the test pattern to the test pattern group to generate a test pattern group including the test patterns.

[0046] Next, the system requirement generating unit 192 generates a system requirement group based on the test pattern group generated by the test pattern generating unit 191 in step S2 (step S3). Specifically, the system requirement generating unit 192 generates a system requirement for each test pattern included in the test pattern group, and organizes the obtained system requirements into a system requirement group.

[0047] Then, the system requirement generating unit 192 outputs the group of system requirements generated in step S3 (step S4). For example, the system requirement generating unit 192 may transmit the group of system requirements generated to an automatic system design device via the communication unit 110. After step S4, the test support apparatus 100 ends the process of FIG.

[0048] In this way, in the test support device 100, the test pattern generation unit 191 generates a test pattern group based on the information on the evaluation axis group and the information on the evaluation item group indicated in the test object related information. Then, the system requirement generation unit 192 generates and outputs a system requirement group corresponding to the test pattern group.

[0049] (Explanation of effect) As described above, the test pattern generation unit 191 generates one or more test patterns in which the parameter values ​​of the test target related information are determined based on the test target related information including the parameters, and the system requirement generation unit 192 generates system requirements for a system that meets the test requirements indicated by the test pattern generated by the test pattern generation unit 191.

[0050] In this way, the test support device 100 generates a test pattern group and a corresponding system requirement group from the information on the evaluation axes and evaluation items shown in the input test target related information. This allows the test support device 100 to automatically or semi-automatically determine (define requirements) one or more system requirements as a comprehensive test environment group (one or more test environments) required in the test process of a test target such as a product. In this respect, the test support device 100 can relatively easily generate a system that meets the test requirements.

[0051] The test subject related information also includes a group of evaluation axes that indicate the requirements of the test environment in the test in a selectable manner, including parameters, and a group of evaluation items in the test. This allows the test pattern generating unit 191 to generate a test pattern by determining the values ​​of parameters included in the test target related information.

[0052] Depending on the test subject, tests may be performed in multiple test environments. For example, in the development of products such as applications or hardware that are components of an ICT system, comprehensive evaluation tests must be performed in system environments with various configurations to verify that the product satisfies functional and non-functional requirements in the environment in which the user uses it. Examples of such tests include load tests of the computers on which the applications run for application products, and performance measurement tests such as effective throughput for network equipment products.

[0053] In this way, when tests are performed in multiple test environments, it is necessary to design and build a system for each test environment, which places a large burden on the person in charge of man-hours. In particular, the specific configuration of the network, files, and other elements of the system as the test environment differs for each test. If the details of the system configuration information used in each test are determined manually in advance, the system configuration information used in one test cannot be reused for other tests, and the amount of description becomes enormous. On the other hand, if the system configuration information is determined abstractly, it is necessary to concretize the abstract elements and define the requirements for each test. In either case, manual work is required, which places a large burden on the person in charge.

[0054] In contrast, the test support device 100 can automatically or semi-automatically define requirements as described above, reducing the burden on the person in charge. In particular, the test pattern generation unit 191 generates test patterns for all combinations of the values ​​of the evaluation axes included in the test target related information, so that the system requirement generation unit 192 can automatically and comprehensively generate system requirements that indicate the requirements of the system as a test environment.

[0055] Second Embodiment (Configuration Description) FIG. 8 is a schematic block diagram showing an example of the configuration of a test support device according to the second embodiment. 8, the test support device 200 includes a storage unit 280 and a control unit 290. The storage unit 280 includes a system requirement template storage unit 281 and an instantiation template storage unit 282. The control unit 290 includes a test environment instantiation unit 293. Among the components shown in FIG. 8, those having similar functions to those corresponding to the components shown in FIG. 1 are given the same reference numerals (110, 120, 130, 191, 192) and will not be described in detail here.

[0056] The configuration of the test support device 200 shown in Fig. 8 differs from the configuration of the test support device 100 shown in Fig. 1 in that the storage unit 280 includes a system requirement template storage unit 281 and a concretization template storage unit 282, and in that the control unit 290 includes a test environment concretization unit 293. Moreover, the processing of the system requirement generation unit 192 in the second embodiment corresponds to a specific example of the processing described in the first embodiment. In other respects, the test support device 200 is similar to the test support device 100 . The system requirement template storage unit 281 stores a system requirement template for each product. The system requirement template is information indicating a method for generating a system requirement for an input of a test pattern. In the system requirement template, information corresponding to the evaluation axis of the test target related information is indicated by a parameter in the system requirement. The system requirement generation unit 192 generates the system requirement by determining the value of this parameter as the value indicated in the test pattern.

[0057] The instantiation template storage unit 282 stores instantiation templates. An instantiation template is information indicating an instantiation method for partially and stepwise replacing a system configuration including abstract parts such as system requirements with a concrete configuration. Each piece of information indicating this instantiation method is called an instantiation rule. Information including all instantiation rules used in the design (concretization) of at least one system is called an instantiation template.

[0058] The test environment instantiation unit 293 designs a system configuration that satisfies each system requirement generated by the system requirement generation unit 192. Specifically, the test environment instantiation unit 293 repeats instantiation using the instantiation rules for the system requirements. In this way, the test environment instantiation unit 293 converts the system requirements into a system configuration that is instantiated to a deployable level. The test environment instantiation section 293 corresponds to an example of a test environment instantiation means.

[0059] The system configuration here refers to the configuration of a system used as a system environment, or information indicating that configuration. The system configuration and the system requirements differ in the degree to which the system requirements or configuration are specified. The information generated by the system requirement generation unit 192 is called the system requirements, and information indicating a system whose requirements or configuration are specified more specifically than the system requirements is called the system configuration. A system configuration group is information that includes one or more system configurations.

[0060] FIG. 9 is a diagram showing an example of data input / output in the test support device 200. As shown in FIG. Data input / output in the test pattern generating unit 191 is similar to that in FIG. The system requirement generating unit 192 receives a test pattern group and outputs a system requirement group. In the example of Fig. 9, the system requirement generating unit 192 reads out a system requirement template corresponding to a test target from the system requirement template storage unit 281. The system requirement generating unit 192 then generates system requirements for each test pattern by determining the values ​​of parameters included in the system requirement template as values ​​indicated in the test pattern. The system requirement generating unit 192 generates system requirements for all test patterns included in the test pattern group, and organizes the obtained system requirements into a system requirement group. As described above, the system requirement generation unit 192 may generate initial data for a group of system requirements, and add each system requirement to the group of system requirements as it is generated, thereby consolidating the system requirements into a group of system requirements.

[0061] The test environment instantiation unit 293 receives a group of system requirements and outputs a group of system configurations. Specifically, for each system requirement generated by the system requirement generation unit 192, the test environment instantiation unit 293 detects a part to which the instantiation rule can be applied among the system requirements or the system configuration obtained by applying the instantiation rule to the system requirement, to which the instantiation rule can be applied. The part to which the instantiation rule can be applied is a part that has a matching instantiation rule. Then, the test environment instantiation unit 293 applies the instantiation rule to the detected part to perform partial instantiation. The test environment instantiation unit 293 repeats the instantiation using the instantiation rule until a system configuration at a deployable level is obtained.

[0062] Fig. 10 is a diagram showing an example of a system requirement template, in which the display unit 120 displays an example of the system requirement template in a graph format. Fig. 10 shows a system requirement template corresponding to the test subject related information of Fig. 2. The system requirement template of Fig. 10 shows a method of generating system requirements corresponding to a test pattern obtained from the test subject related information of Fig. 2, such as the test pattern exemplified in Fig. 4. Specifically, the system requirement generating unit 192 generates system requirements by determining the values ​​of parameters included in the system requirement template to be values ​​indicated in the test pattern.

[0063] The system requirement template is expressed in a graph format similar to the system requirements illustrated in Fig. 6. As in the case of system requirements, attribute value information may be added to some nodes and edges of the system requirement template. Also, unlike the case of system requirements, parameters of evaluation axes may be further added as attribute value information to the nodes of the system requirement template.

[0064] The system requirements template also shows how to partially replace the graph according to the value of each evaluation axis. For example, the system requirements template for the application product "app-face" shown in Figure 10 shows the file names corresponding to each value of the evaluation axis "resolution" as a replacement method for the "video" node, which represents the image projected on the camera.

[0065] For example, when the system requirement generating unit 192 determines the parameter value of the evaluation axis "resolution" to be "VGA", it sets the attribute value information of the "video" node so that the file "vga.mp4" is uploaded. Alternatively, when the system requirement generating unit 192 determines the parameter value of the evaluation axis "resolution" to be "FHD", it sets the attribute value information of the "video" node so that the file "fhd.mp4" is uploaded. Alternatively, when the system requirement generating unit 192 determines the parameter value of the evaluation axis "resolution" to be "4K", it sets the attribute value information of the "video" node so that the file "4k.mp4" is uploaded.

[0066] After setting the attribute value information of the "video" node, the system requirement generation unit 192 creates subgraphs of the A11 portion in the number corresponding to the value of the evaluation axis "number of cameras." The subgraph of the A11 portion is a subgraph consisting of the "camera" node, the "video" node, the "hasSource" edge connecting these nodes, and the "sendVideo" edge connecting the "camera" node and the "app-face" node.

[0067] For the evaluation items “CPU usage” and “memory usage”, the attribute values ​​attached to the “wire:OS” edge connecting these nodes indicate that the evaluation is to be performed from the “agent1” node to the “os1” node.

[0068] Fig. 11 is a diagram showing an example of an instantiation template, in which the display unit 120 displays an example of the instantiation template in a graph format. The instantiation rule T11 shows a method for instantiating an abstract HTTP type edge that represents HTTP (Hyper Text Transfer Protocol) communication between applications. In this instantiation rule, the test environment instantiation unit 293 hosts the application type nodes "app1" and "app2", which are the two end points of the HTTP type edge, on the OS type nodes "os1" and "os2", respectively, and connects them with a wire:OS type edge. Then, the test environment instantiation unit 293 connects the two OS type nodes "os1" and "os2" with a TCP type edge that represents a TCP (Transmission Control Protocol) connection.

[0069] The instantiation rule T12 shows a method for instantiating the OS type node "os1". In this instantiation rule, the test environment instantiation unit 293 adds a machine type node representing a server for starting the OS, and connects the "os1" node and the added node with a wire:OS type edge. The instantiation rule T13 shows a method for instantiating the machine type node "machine1". In this instantiation rule, the test environment instantiation unit 293 adds a router type node "router" representing a router, and connects the "machine1" node and the "router" node with a "wire:router" edge. The test environment instantiation unit 293 repeats the instantiation of the system requirements using the instantiation rules sequentially until the target graph structure does not contain any abstract elements. In other words, the test environment instantiation unit 293 repeats the instantiation of the system requirements or the system configuration until a system configuration that is instantiated to a deployable level is obtained.

[0070] Fig. 12 is a diagram showing an example of a test environment. Fig. 10 shows an example in which the display unit 120 displays, in the form of a graph, an example of a system configuration that has been embodied to a level that can be deployed as a test environment. In this way, the test environment referred to here is information that represents a specific configuration of a system required to perform a test on a test subject such as a product. The test environment referred to here includes detailed information such as the OS and applications running on the server, file placement, etc., in addition to a network configuration including the connection relationship between the server, routers, etc.

[0071] In the test environment illustrated in Fig. 12, the "sendVideo" edge and "hasSource" edge, which were abstract elements in the system requirements shown in Fig. 5, have been made concrete. In addition, the "machine1" node, which is a server, and two camera nodes are connected to the "router" node, which is the same router. Furthermore, the "os2" node, which is the OS for playing the video files "video1" and "video2", and the "machine2" node, which is the server on which that OS runs, are additionally deployed and connected to the "router" node via the "wire:GW" edge.

[0072] Furthermore, the type of evaluation test to be performed in the test environment, and the evaluation test execution body and test target are specified in the example of Fig. 12. In the example of Fig. 12, the attribute value of the edge indicates that the evaluation is to be performed from the "agent1" node to the "os1" node for the evaluation item group "CPU usage" and "memory usage". The specific contents and display format of the system configuration indicating the test environment are not limited to the example shown in FIG.

[0073] (Explanation of operation) Fig. 13 is a flowchart showing an example of a procedure of processing performed by the test support device 200. Among the processing steps in Fig. 13, the same processing steps as those in Fig. 7 are given the same reference numerals (S1, S2) and detailed description thereof will be omitted here.

[0074] Steps S1 and S2 in Fig. 13 are the same as those in Fig. 7. After step S2, the system requirement generating unit 192 refers to the system requirement template storage unit 281 and acquires a system requirement template corresponding to the test target (step S11). Then, the system requirement generating unit 192 generates a system requirement group based on the test pattern group generated in step S2 and the system requirement template acquired in step S11 (step S12). Specifically, the system requirement generating unit 192 generates a system requirement corresponding to each test pattern included in the test pattern group, and organizes the generated system requirements into a system requirement group.

[0075] Next, the test environment instantiation unit 293 refers to the instantiation template storage unit 282 and acquires an instantiation template corresponding to the test target such as a product (step S13). Then, the test environment instantiation unit 293 generates a system configuration group using the system requirements group generated in step S12 and the instantiation template acquired in step S13 (step S14). Specifically, the test environment instantiation unit 293 generates a system configuration by instantiating each system requirement included in the system requirements group using the instantiation rule. Then, the test environment instantiation unit 293 organizes the generated system configurations into a system configuration group. Then, the test environment instantiation unit 293 outputs the generated system configuration group (step S15). After step S15, the test support device 200 ends the process of FIG.

[0076] In this way, in the test support device 200, the test pattern generation unit 191 generates a test pattern group based on the information of the evaluation axis group and the information of the evaluation item group indicated in the test target related information. Then, the system requirement generation unit 192 generates a system requirement group corresponding to the test pattern group based on the system requirement template acquired from the system requirement template storage unit 281. The test environment instantiation unit 293 reads the instantiation template from the instantiation template storage unit 282, and instantiates each system requirement included in the system requirement group using the instantiation rule. As a result, the test environment instantiation unit 293 generates and outputs a system configuration group as a test environment group corresponding to the system requirement group.

[0077] (Explanation of effect) As described above, the system requirement generating unit 192 generates one or more system requirements by determining the parameter values ​​of a system requirement template in which a part of the system requirement is indicated by a parameter. This allows the system requirement generation unit 192 to generate a system requirement template through a relatively simple process of determining the parameter values ​​of the system requirement template based on the test pattern. Furthermore, the test support device 200 can automatically or semi-automatically determine (define) one or more system requirements as a comprehensive test environment group required in the test process of a test target such as a product. In this respect, the test support device 200 can relatively easily generate a system that satisfies the test requirements.

[0078] Moreover, the test environment instantiation unit 293 instantiates the system requirements, thereby generating a system configuration that satisfies the system requirements. In this way, the test environment realization unit 293 can automatically design a specific system configuration as a test environment based on the system requirements. In this respect, the test environment realization unit 293 can reduce the burden on the person in charge.

[0079] Moreover, the test environment instantiation unit 293 generates a system configuration by repeating partial instantiation of the system requirements. In this way, the test environment instantiation unit 293 can automatically design a specific system configuration as a test environment by repeating the instantiation of the system requirements. In this respect, the test environment instantiation unit 293 can reduce the burden on the person in charge.

[0080] In this way, according to the test support device 200, in addition to the effects described for the test support device 100, by concretizing each system requirement in the test environment concretization unit 293, a comprehensive group of test environments can be automatically generated without manual work. Furthermore, in the test support device 200, when a test environment is automatically generated from the test subject related information, a method for generating a group of system requirements based on the test subject related information is shown as a system requirement template. Also, in the test support device 200, a method used for concretizing a test environment from system requirements is shown as a concretization template, so that when an engineer such as a user of the test support device 200 uses the test support device 200 to build a test environment for a new test subject such as a new product, the engineer can relatively easily generate a concretization template for the new test subject by referring to an existing concretization template. The test support device 100 may include a concretization template storage unit 282 and a test environment concretization unit 293. As a result, similar to the case of the test support device 200, the test environment concretization unit 293 can automatically generate a system configuration group as a test environment group from a system requirement group.

[0081] <Third embodiment> (Configuration Description) FIG. 14 is a schematic block diagram showing an example of the configuration of a test support system according to the third embodiment. 14, the test support system 300 includes a test support device 310, a test execution device 320, and a test result recording device 330. The test result recording device 330 includes a test result storage unit 331.

[0082] The test support device 310 and the test execution device 320 are capable of communicating with each other. The test execution device 320 and the test result recording device 330 are capable of communicating with each other. The test support device 200 may be used as the test support device 310. Alternatively, the test support device 100 including the instantiation template storage unit 282 and the test environment instantiation unit 293 may be used as the test support device 310.

[0083] The test support device 310, the test execution device 320, and the test result recording device 330 may be configured as one device. For example, the functions of the test support device 310, the test execution device 320, and the test result recording device 330 may be executed by the same computer. Alternatively, any two of the test support device 310, the test execution device 320, and the test result recording device 330 may be configured as one device.

[0084] FIG. 15 is a diagram showing an example of data input and output in the test support system 300. As shown in FIG. The test support device 310 receives test target related information and outputs a system configuration group. The test execution device 320 receives a system configuration group and outputs a test result group. Specifically, the test execution device 320 performs a test for each system configuration included in the system configuration group, using the system based on that system configuration as a test environment. The test execution device 320 organizes the test results into a test result group.

[0085] The test result here is information indicating the results of the test in each test environment, specifically, the test result is information indicating the values ​​of the evaluation items in the test in each test environment. A test result set is information that contains one or more test results. The test result recording device 330 stores the test results output by the test execution device 320 in a test result storage section 331 .

[0086] Fig. 16 is a diagram showing an example of a test result group. Fig. 16 shows an example in which the test execution device 320 displays an example of the test result group in a table format. Alternatively, the test support device 310 or the test result recording device 330 may display the test result group. The evaluation result group information is information that indicates the values ​​of each evaluation item obtained by the test execution device 320 performing tests for each test pattern.

[0087] The display example of the test result group shown in Fig. 16 corresponds to an example in which the CPU utilization rate and memory utilization rate values ​​for each test pattern in the test results by the test execution device 320 are filled in and displayed in the display example of the test pattern group in Fig. 5. Therefore, in the example of Fig. 16, the CPU utilization rate and memory utilization rate values ​​of the server on which the application "app-face" runs, which were measured in the evaluation test of this application, are displayed as evaluation results corresponding to each evaluation pattern. The specific contents and display format of the evaluation results are not limited to the example shown in FIG.

[0088] (Explanation of operation) Fig. 17 is a flowchart showing an example of the procedure of processing performed by the test support system 300. Fig. 17 shows an example of processing when the test support device 200 is used as the test support device 310. Among the processing steps in Fig. 17, the same processing steps as those in Fig. 13 are given the same reference numerals (S1, S2, S11, S12, S13, S14, S15) and detailed description thereof will be omitted here.

[0089] Steps S1 to S15 in Fig. 17 are the same as those in Fig. 13. After step S15, the test execution device 320 receives the system configurations generated and output by the test support device 310, executes tests for each system configuration, and outputs an evaluation result group (step S21). The test execution device 320 may automatically construct a system as a test environment based on the system configuration. Alternatively, the test execution device 320 may display the system configuration, and the user may construct the system according to the system configuration, or the system may be manually constructed as a test environment.

[0090] The test execution unit 320 performs tests for each system configuration, and outputs the test results in a test result group. Then, the test result group output by the test execution device 320 is acquired by the test result recording device 330 and stored in the test result storage unit 331 (step S22). After step S22, the test support system 300 ends the process of FIG.

[0091] In this way, in the test support system 300, the test support device 310 outputs a system configuration group based on the test target related information indicating the evaluation axis group and the evaluation item group. Then, the test execution device 320 performs a test for each system configuration included in the system configuration group, and outputs the test results as an evaluation result group. The test result recording device 330 acquires the evaluation result group output by the test execution device 320, and stores it in the test result storage unit 331.

[0092] (Explanation of effect) Thus, in addition to the effects described for the test support device 100 and the test support device 200, the test support system 300 can perform tests automatically or semi-automatically by the test execution device 320, and can automatically store evaluation results by the test result recording device 330. In this respect, the burden on the person in charge can be further reduced.

[0093] <Fourth embodiment> 18 is a diagram showing an example of the configuration of a test support device according to the fourth embodiment. In the configuration shown in FIG.

[0094] In this configuration, the test pattern generation unit 601 generates one or more test patterns in which the values ​​of the parameters of the test target related information are determined based on the test target related information including the parameters, and the system requirement generation unit 602 generates system requirements of a system that satisfy the test requirements indicated by the test pattern. The test pattern generating unit 601 corresponds to an example of a test pattern generating means, and the system requirement generating unit 602 corresponds to an example of a system requirement generating means.

[0095] In this way, the test support device 600 generates a test pattern group and a corresponding system requirement group from the information on the evaluation axes and evaluation items shown in the input test target related information. This allows the test support device 600 to automatically or semi-automatically determine (define requirements) one or more system requirements as a comprehensive test environment group required in the test process of a test target such as a product. In this respect, the test support device 600 can relatively easily generate a system that meets the test requirements.

[0096] <Fifth embodiment> FIG. 19 is a flowchart showing an example of a processing procedure in a test support method according to the fifth embodiment. The process shown in FIG. 19 includes generating a test pattern (step S601) and generating a system requirement (step S602).

[0097] In generating a test pattern (step S601), one or more test patterns are generated in which values ​​of parameters of the test subject related information are determined based on test subject related information indicating test requirements determined according to the test subject, including parameters. In generating a system requirement (step S602), system requirements of a system that meets the test requirements indicated by the test pattern are generated.

[0098] In the process shown in Fig. 19, a test pattern group and a corresponding system requirement group are generated from the evaluation axis group and evaluation item group information shown in the input test target related information. As a result, the process shown in Fig. 19 can automatically or semi-automatically determine (requirements definition) one or more system requirements as a comprehensive test environment group required in the test process of a test target such as a product. In this respect, the process shown in Fig. 19 makes it relatively easy to generate a system that meets the test requirements.

[0099] FIG. 20 is a schematic block diagram illustrating a configuration of a computer according to at least one embodiment. In the configuration shown in FIG. 20, a computer 700 includes a CPU 710, a main memory device 720, an auxiliary memory device 730, and an interface 740.

[0100] Any one or more of the above-mentioned test support device 100, test support device 200, and test support device 600 may be implemented in a computer 700. In this case, the operation of each of the above-mentioned processing units is stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, loads it in the main storage device 720, and executes the above-mentioned processing according to the program. The CPU 710 also secures a storage area corresponding to each of the above-mentioned storage units in the main storage device 720 according to the program.

[0101] When the test support device 100 is implemented in a computer 700, the operations of the control unit 190 and each of its units are stored in the form of a program in an auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing in accordance with the program. Furthermore, the CPU 710 allocates a storage area corresponding to the storage unit 180 in the main storage device 720 in accordance with the program. The communication by the communication unit 110 is performed by the interface 740 having a communication function and performing communication under the control of the CPU 710. Display by the display unit 120 is performed by the interface 740 having a display screen and displaying various images under the control of the CPU 710. Reception of a user operation by the operation input unit 130 is performed by the interface 740 having an input device, receiving the user operation, and outputting a signal indicating the received user operation to the CPU 710.

[0102] When the test support device 200 is implemented in a computer 700, the operations of the control unit 290 and each of its units are stored in the form of a program in an auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it in the main storage device 720, and executes the above-mentioned processing according to the program. Furthermore, the CPU 710 allocates storage areas in the main memory 720 corresponding to the storage unit 280 and each of its units in accordance with the program. The communication by the communication unit 110 is performed by the interface 740 having a communication function and performing communication under the control of the CPU 710. Display by the display unit 120 is performed by the interface 740 having a display screen and displaying various images under the control of the CPU 710. Reception of a user operation by the operation input unit 130 is performed by the interface 740 having an input device, receiving the user operation, and outputting a signal indicating the received user operation to the CPU 710.

[0103] When the test support device 600 is implemented in a computer 700, the operations of the test pattern generation unit 601 and the system requirement generation unit 602 are stored in the form of a program in an auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it in the main storage device 720, and executes the above-mentioned processing according to the program.

[0104] Note that a program for implementing all or part of the functions of the test support device 100, the test support device 200, and the test support device 600 may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to process each part. The term "computer system" here includes hardware such as an OS (operating system) and peripheral devices. "Computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs (Read Only Memory), and CD-ROMs (Compact Disc Read Only Memory), as well as storage devices such as hard disks built into computer systems. The above-mentioned program may be for realizing part of the above-mentioned functions, or may be capable of realizing the above-mentioned functions in combination with a program already recorded in the computer system.

[0105] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like that do not deviate from the gist of the present invention are also included. [Explanation of symbols]

[0106] 100, 200, 310, 600 Test Support Equipment 110 Communications Department 120 Display section 130 Operation input section 180, 280 storage section 190, 290 Control unit 191, 601 Test pattern generation unit 192, 602 System Requirements Generation Division 281 System requirements template storage section 282 Instantiation template storage unit 293 Test Environment Specifications Department 300 Test Support System 320 Test Execution Device 330 Test result recording device 331 Test result storage unit

Claims

1. a test pattern generating means for generating one or more test patterns in which values ​​of parameters of the test subject related information are determined based on test subject related information including the parameters as test requirements determined according to the test subject; a system requirement generating means for generating system requirements for a system that satisfies the requirements of the test indicated by the test pattern; A test support device comprising:

2. The test target related information includes a group of evaluation axes that selectably indicate requirements of the test environment in the test, including parameters, and a group of evaluation items that indicate items to be evaluated in the test. The test support device according to claim 1.

3. The system requirement generating means generates one or more of the system requirements by determining values ​​of the parameters of a system requirement template in which a part of the system requirement is represented by the parameters.

3. The test support device according to claim 1 or 2.

4. A test environment instantiation means for instantiating the system requirements to generate a system configuration that satisfies the system requirements. The test support device according to claim 1 , further comprising:

5. the test environment instantiation means generates the system configuration by repeating partial instantiation of the system requirements; 5. The test support device according to claim 4.

6. generating one or more test patterns in which values ​​of parameters of the test subject related information are determined based on test subject related information including the parameters as test requirements determined according to the test subject; generating system requirements for a system that satisfies the test requirements indicated by the test patterns; A test support method including:

7. On the computer, generating one or more test patterns in which values ​​of parameters of the test subject related information are determined based on test subject related information including the parameters as test requirements determined according to the test subject; generating system requirements for a system that satisfies the test requirements indicated by the test patterns; A program for executing the above.

Images