Test assistance system and control method therefor

The test support system addresses the issue of deleted test cases by updating and adding necessary tests to system tests based on unit test results, ensuring comprehensive validation and preventing defects in monitoring systems.

WO2025004232A1PCT designated stage expired Publication Date: 2025-01-02MITSUBISHI ELECTRIC BUILDING SOLUTIONS CORP +1
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2023/024036
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-06-28
Publication Date
2025-01-02

AI Technical Summary

Technical Problem

Existing automatic test execution systems may inadvertently delete necessary test cases during unit testing, leading to undetected defects that surface during system testing, due to duplication elimination, and inadequate sharing of test items between unit and system testing teams, resulting in potential product defects post-delivery.

Method used

A test support system that conducts confirmation tests by storing and updating test contents, adding tests to the system test based on differences between unit and system test contents, ensuring all necessary tests are performed, particularly for continuously variable input/output values.

Benefits of technology

Prevents defects in the confirmation test by ensuring all necessary tests are executed, improving test efficiency and reducing rework, and ensuring comprehensive validation of input and output values in monitoring systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2023024036_02012025_PF_FP_ABST
    Figure JP2023024036_02012025_PF_FP_ABST
Patent Text Reader

Abstract

The processor (411) executes update processing for updating the content of a plurality of tests. The monitoring system executes internal processing on the basis of a supplied input value and outputs an output value as a result. A confirmation test includes a first confirmation test and a second confirmation test performed after the first confirmation test. In the update processing, the processor (411) adds a test to the second confirmation test on the basis of the difference between the content of the first confirmation test and the content of the second confirmation test.
Need to check novelty before this filing date? Find Prior Art

Description

Test support system and control method thereof

[0001] The present disclosure relates to a test support system that supports a verification test to confirm the validity of input / output values ​​input and output in a monitoring system that includes equipment and a server device that monitors the equipment via a controller.

[0002] Japanese Patent Laid-Open Publication No. 2009-217664 (Patent Document 1) discloses an automatic test execution system that analyzes the source code of the test target and tests the target program by inputting various values ​​into the program. This automatic test execution system analyzes the conditional expressions defined in the method, extracts boundary values, and generates all test data and return values ​​of the test data as a test execution result table.

[0003] Then, by embedding a tracing function that tracks the processing flow in the source code to be tested, the trace information is extracted and linked to the test execution result table mentioned above, and duplicate test cases to be executed are eliminated. By eliminating duplicate test cases, test cases that do not contribute to the coverage rate, which is an index showing the percentage of a program that has been tested, can be excluded from the test targets.

[0004] Japanese Patent Application Laid-Open No. 2009-217664

[0005] However, in the automatic test execution system described in Patent Document 1, deleting duplicate tests (test cases) can result in the elimination of tests that are necessary but do not contribute to improving the coverage rate. For example, there is a case where tests that check continuously variable input and output values ​​are performed consecutively. In such a case, by performing these two tests, it is possible to check the change in the output value. However, if the latter test is deleted, this check cannot be performed.

[0006] In a monitoring system that includes equipment and a server device that monitors the equipment via a controller such as a PLC (Programmable Logic Controller), a testing format is envisioned in which, for example, a unit test that checks the input / output data of the PLC is performed as a confirmation test, and then a system test (combination test) that tests the entire monitoring system is performed as a confirmation test for the next process.

[0007] In this case, by deleting duplicate tests in unit testing in order to reduce the number of tests, defects that should have been detected in unit testing may not be detected in the unit testing, but may be detected in the next process, system testing. In such cases, after correcting the software defects, the software must be retested again, which reduces the efficiency of testing.

[0008] Furthermore, if the test items for system testing are not sufficiently selected and the tests that should be performed are not carried out, defects may occur in the product when it is in operation at the customer's site after delivery. This can happen when different testers are responsible for unit testing and system testing, and the necessary information is not shared between them. For example, even if the test items for unit testing are sufficiently selected, the contents of these test items may not be fully reflected in the system testing, resulting in test items that should be performed in the system testing being missing.

[0009] The present disclosure has been made to solve the above-mentioned problems, and its purpose is to provide a test support system and a control method thereof that can prevent defects that should be detected in confirmation tests from occurring at the customer's site.

[0010] A test support system according to the present disclosure is a system that supports a confirmation test that confirms the validity of input / output values ​​input and output in a monitoring system that includes equipment and a server device that monitors the equipment via a controller. The test support system includes a memory and a processor. The memory stores the contents of multiple tests to be performed in the confirmation test. The processor executes an update process that updates the contents of the multiple tests. The monitoring system executes internal processing in response to given input values ​​and outputs output values ​​as a result. The confirmation test includes a first confirmation test and a second confirmation test that is performed after the first confirmation test. In the update process, the processor adds a test to the second confirmation test based on a difference between the contents of the first confirmation test and the contents of the second confirmation test.

[0011] A control method according to the present disclosure is a control method for a test support system that supports a confirmation test that confirms the validity of input / output values ​​input and output in a monitoring system including equipment and a server device that monitors the equipment via a controller. The control method includes a step of storing the contents of multiple tests to be performed in the confirmation test and a step of updating the contents of the multiple tests. The monitoring system executes internal processing in response to given input values ​​and outputs output values ​​as a result. The confirmation tests include a first confirmation test and a second confirmation test that is performed after the first confirmation test. The updating step further includes a step of adding a test to the second confirmation test based on a difference between the contents of the first confirmation test and the contents of the second confirmation test.

[0012] According to the present disclosure, tests are added to the confirmation test of the next process based on the difference between the contents of the confirmation test of the previous process and the contents of the confirmation test of the next process, thereby preventing defects that should be detected in the confirmation test from occurring at the customer's site.

[0013] FIG. 1 is a diagram showing an example of the overall configuration of a monitoring system and a test support system in embodiment 1. FIG. 2 is a diagram showing an example of the hardware configuration of the monitoring system and the test support system. FIG. 3 is a functional block diagram of the test support system. FIG. 4 is a diagram showing control of the rotation speed of a pump. FIG. 5 is a diagram showing an example of a test specification for a unit test. FIG. 6 is a diagram showing an example of a test specification for a system test. FIG. 7 is a flowchart of processing executed by the test support system. FIG. 8 is a diagram showing control of the rotation speed of a pump in embodiment 2. FIG. 9 is a diagram showing an example of a test specification for a unit test. FIG. 10 is a diagram showing an example of a test specification for a system test. FIG. 11 is a flowchart of processing executed by the test support system.

[0014] Hereinafter, embodiments will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. The names and functions of these components are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0015] 1 is a diagram showing an example of the overall configuration of a monitoring system 1 and a test support system 400 according to a first embodiment.

[0016] The monitoring system 1 (also referred to as an “air conditioning instrumentation system”) includes a server device 100 , a PLC (Programmable Logic Controller) 200 as a controller, and equipment (PLC) 300 .

[0017] The monitoring system 1 is a system that manages various facilities installed in a building. The server device 100 monitors and controls the facility 300 via the PLC 200. Signals output by the facility 300 are acquired by the server device 100 via the PLC 200 and can be displayed (monitored) on a display unit 421 (FIG. 2). In addition, a verification test implementer 5 (administrator of the monitoring system 1) can transmit control signals from the server device 100 to the facility 300 via the PLC 200 by operating the input unit 420 (FIG. 2).

[0018] The equipment (PLC) 300 includes air conditioning equipment installed in the building. The equipment (PLC) 300 may also include other equipment, such as lighting equipment, installed in the building. The PLC 200 relays and transmits input / output signals between the equipment (PLC) 300 and the server device 100, and controls the equipment (PLC) 300.

[0019] The test support system 400 is a system that supports a verification test that verifies the validity of input / output values ​​(input / output signals) input and output in the monitoring system 1 .

[0020] The confirmation tests include a "unit test" as a first confirmation test and a "system test" as a second confirmation test that is performed after the first confirmation test. The unit test is a test that verifies input / output data input to and output from some of the devices included in the monitoring system 1. In the first embodiment, this "some of the devices" is the PLC 200, but it may also be the equipment (PLC) 300.

[0021] The system test is a test (combination test) that comprehensively tests the behavior of the entire monitoring system 1. First, a unit test is performed as a confirmation test, and then a system test is performed as a confirmation test in the next step.

[0022] In embodiment 1, a case is described in which a PLC 300, which is a device installed to simulate equipment and inputs and outputs signals in place of the equipment, is connected to the monitoring system 1, but confirmation tests may also be performed by connecting actual equipment to the monitoring system 1.

[0023] The verification test can be automatically executed using a test support system 400. The test support system 400 includes a support tool 401, a browser 402, a console 403, an operation flow 414, a test specification 415, an input / output test program 416, and an input / output test code 417.

[0024] The server device 100 can be accessed via a browser 402. The browser 402 is displayed on the screen of a display unit 421 (FIG. 2), and values ​​acquired by the server device 100 (for example, various values ​​acquired from the air conditioning equipment) are displayed on the browser 402. In addition, the equipment (PLC) 300 can be controlled via the browser 402 by operating the input unit 420 (FIG. 2). For example, the set temperature of the air conditioning equipment can be set on the browser 402.

[0025] The support tool 401 is a tool that automatically executes the operation of the browser 402 and automatically executes the confirmation test. The support tool 401 controls the browser 402 and the console 403 in accordance with the procedure described in a preset operation flow 414 to execute the confirmation test.

[0026] A procedure for automatically operating the browser 402 is created using the support tool 401. The created procedure is stored in the operational flow 414.

[0027] The input / output test program 416 generates input / output test code 417. The support tool 401 starts the input / output test program 416 from the console 403. The contents of the tests to be executed in the confirmation test are described in a test specification 415. The input / output test program 416 reads the test specification 415 and generates the input / output test code 417 based on the test specification 415.

[0028] The result of the verification test is determined by executing the input / output test code 417. The support tool 401 starts and executes the input / output test code 417 from the console 403. The console 403 is a console screen that can be started in the test support system 400. By executing commands on the console 403, various software that can be executed on the test support system 400 can be executed.

[0029] The test support system 400 is connected to the PLC 200 and the equipment (PLC) 300 via signal lines so that the input / output signals of the PLC 200 and the equipment (PLC) 300 can be acquired. When the input / output test code 417 is executed, the input / output signals of the PLC 200 and the equipment (PLC) 300 are acquired, and the test results (validation of the input / output data) are determined based on the test content of the test specification 415. Specifically, the values ​​described in the test specification 415 are compared with the actually acquired values ​​to determine whether the values ​​are valid. The results are recorded in the test specification 415.

[0030] With the above configuration, the test support system 400 automatically executes the confirmation test. Note that the above example is merely an example, and any other means may be used to realize the automatic execution of the confirmation test, the tester 5 may perform the confirmation test by directly operating the browser 402, the tester 5 may visually confirm the results of the confirmation test, or the test support system 400 may be configured to automatically execute part of the confirmation test.

[0031] The support tool 401 is one of the so-called no-code tools that do not require programming, and the test support system 400 may be constructed using any commonly distributed no-code tool, or, without using such a tool, the process of controlling the operation of the browser 402 may be implemented by a program to automatically execute the confirmation test.

[0032] Furthermore, in the first embodiment, the input / output test program 416 is configured to be able to execute an update process that updates (deletes or adds) the test content of the test specification 415. The update process includes a process that updates the test content of a unit test and a process that updates the test content of a system test. The input / output test program 416 can delete or add test content of the test specification 415 and update the test content of the test specification 415 based on the test content of the test specification 415 and the program executed by the device to be tested (e.g., the PLC 200) of the monitoring system 1 (this will be described in detail later with reference to FIG. 3 and subsequent figures).

[0033] As described above, the update process is implemented as a process executed by the input / output test program 416 running on the server device 100, but the update process may be implemented in any format as long as it is implemented as a process that can be executed on the test support system 400. For example, the update process may be provided as a plug-in for application software that runs on the server device 100, or as an add-on that can run on a browser, or the update process may be implemented so as to be executed on a server device installed in a remote location separate from the server device 100.

[0034] 2 is a diagram showing an example of the hardware configuration of the monitoring system 1 and the test support system 400. As described above, the monitoring system 1 includes the server device 100, the PLC 200, and the PLC 300. The server device 100 and the PLC 300 are connected by a communication line (signal line) via the PLC 200, and are configured to be able to communicate with each other.

[0035] The test support system 400 is also connected to the PLC 300 and the PLC 200 by communication lines (signal lines) and is configured to be able to acquire signals input to and output from the PLC 300 and the PLC 200. The test support system 400 is configured to be able to communicate with the server device 100 via a browser 402.

[0036] The server device 100, which is made up of at least one or more units, includes a processor 111, a memory 112, and a communication unit 113. The processor 111 is, for example, a central processing unit (CPU). The memory 112 is, for example, configured to include read-only memory (ROM), random access memory (RAM), and a storage device. The storage device is a non-volatile storage device such as an HDD or SSD.

[0037] The processor 111 controls the entire server device 100. The processor 111 executes programs stored in the memory 112. The ROM stores programs that describe the processing procedures of the server device 100. The RAM serves as a work area when the CPU executes the programs, and temporarily stores programs, data used for executing the programs, and the like. The communication unit 113 is an interface for communicating with the PLC 200 and the test support system 400.

[0038] The PLC 200, which is made up of at least one unit, includes a processor 211, a memory 212, a communication unit 213, and an input / output terminal 214. The processor 211 is, for example, a CPU. The memory 212 is, for example, configured to include a ROM, a RAM, and a storage device. The processor 211 performs overall control of the entire PLC 200. The processor 211 executes programs stored in the memory 212. The communication unit 213 is an interface for communicating with the server device 100, the PLC 300, and the test support system 400. The communication unit 213 transmits and receives input / output signals to and from the PLC 300 via the input / output terminal 214.

[0039] The PLC 300, which is configured with at least one unit, includes a processor 311, a memory 312, a communication unit 313, and an input / output terminal 314. The processor 311 is, for example, a CPU. The memory 312 is, for example, configured to include a ROM, a RAM, and a storage device. The processor 311 performs overall control of the entire PLC 300. The processor 311 executes programs stored in the memory 312. The communication unit 313 is an interface for communicating with the PLC 200 and the test support system 400. The communication unit 313 transmits and receives input / output signals to and from the PLC 200 via the input / output terminal 314. The input / output terminal 214 of the PLC 200 and the input / output terminal 314 of the PLC 300 are connected by multiple signal lines. Electrical signals (input / output signals) are input and output between the PLC 200 and the PLC 300 via multiple signal lines connected between the input / output terminal 214 and the input / output terminal 314.

[0040] The test support system 400 includes a processor 411, a memory 412, a communication unit 413, an input unit 420, and a display unit 421. The processor 411 is, for example, a CPU. The memory 412 is, for example, configured to include a ROM, a RAM, and a storage device. The processor 411 performs overall control of the entire test support system 400. The processor 411 executes programs stored in the memory 412. The communication unit 413 is an interface for communicating with the server device 100, the PLC 200, and the PLC 300.

[0041] The display unit 421 is a display that displays various information images. The input unit 420 accepts input from the implementer 5. The input unit 420 is, for example, a keyboard, a mouse, or a touch panel.

[0042] 1, the support tool 401, browser 402, console 403, input / output test program 416, and input / output test code 417 are each stored in memory 412 and executed by a processor 411. In addition, an operation flow 414 and a test specification 415 are stored in memory 412. The test support system 400 may be a personal computer or the like equipped with a display unit 421 and an input unit 420.

[0043] 3 is a functional block diagram of a test support system 400. The test support system 400 includes a test specification 415, a test data generation unit 432, test data 433, a test data execution unit 434, a test data tracing unit 435, a tracing result 436, a test result output unit 437, a test result 438, a test attribute 440, and a test data complementation unit 441.

[0044] The above function is executed by the input / output test program 416, and updates the test contents of the test specification 415 based on the test contents of the test specification 415 and the program executed by the device (such as PLC 200) to be tested in the monitoring system 1.

[0045] The test specification 415, the test data 433, the tracing result 436, the test result 438, and the test attribute 440 are stored in the memory 412 of the test support system 400. The test data generation unit 432, the test data execution unit 434, the test data tracing unit 435, the test result output unit 437, and the test data complementation unit 441 are executed by the processor 411 of the test support system 400.

[0046] The test data generation unit 432 reads the test specifications 415 for the unit test and generates test data 433. The test data 433 is data obtained by extracting input / output data to be input to and output from the monitoring system 1 based on the test specifications 415.

[0047] The test data execution unit 434 inputs the test data 433 to the monitoring system 1 and runs a program executed in the monitoring system 1. The monitoring system 1 executes internal processing (processing executed by a program) according to the given input value, and outputs an output value as a result.

[0048] The test data tracing unit 435 acquires the source code of the program executed in the monitoring system 1. When the test data 433 is input to the monitoring system 1 and the program is run, the test data tracing unit 435 identifies the processing performed by the program executed in the monitoring system 1 and saves it as a tracing result 436.

[0049] For example, assume that the target for inputting and outputting data is the PLC 200. In this case, assume that when input data X is given to the PLC 200, output data Y is output. The test data tracing unit 435 analyzes the source code from the set of input data X and output data Y to identify which internal process has been executed in the program running on the PLC 200.

[0050] For example, in a program running on the PLC 200, when input data X is given, processing of function A is executed with input data X given as an argument, and output data Y is output as a return value of function A, the test data tracing unit 435 identifies "function A" as the internal processing to be executed. In cases where processing A1 in function A is executed when input data is X1, and processing A2 in function A is executed when input data is X2, "processing A1 of function A" and "processing A2 of function A" may be identified as the internal processing, respectively.

[0051] The process of analyzing the source code of the test data execution unit 434 and the test data tracing unit 435 may be realized using a known technique such as that disclosed in Japanese Patent Application Laid-Open No. 2009-217664.

[0052] The process of identifying which internal process has been executed in a program running on PLC 200 is not limited to the above-described process of identifying by analyzing the source code, but may be identified from test specifications 415. For example, if test specifications 415 describe test items, program functions when these items are executed, input data (arguments) to be input to the functions, and output data (return values) to be output from the functions, the internal process to be executed may be identified based on these descriptions.

[0053] The test data complementing unit 441 determines test attributes based on the test data 433 and the tracing result 436 and saves them as test attributes 440. The test data complementing unit 441 executes an update process to update (delete or add) the contents of multiple tests. In the update process, the test data complementing unit 441 targets test data of unit tests with overlapping internal processes for deletion. Based on the test data 433 and the test attributes, the test data complementing unit 441 determines whether to exclude tests targeted for deletion from the list of tests targeted for deletion. In the update process, the test data complementing unit 441 ultimately deletes the test data of the unit tests targeted for deletion. Furthermore, in the update process, the test data complementing unit 441 adds test items missing from the system test to the test specification for the system test based on the difference between the contents of the unit test and the contents of the system test.

[0054] The test result output unit 437 saves the test results of the test data of the unit test updated in the update process as test results 438 , and also reflects the test results of the unit test in the test specification 415 .

[0055] The update process executed by the test data complementing unit 441 will be specifically described below. Fig. 4 is a diagram showing the control of the pump rotation speed. For example, assume that an air conditioning system is installed in a building, and the PLC 300 is installed as a device that simulates the air conditioning system. The air conditioning system is a device that lowers or raises the temperature of outdoor air taken in from outside by heat exchange with hot or cold water, and supplies the air indoors.

[0056] Here, the temperature of the outside air taken in from outside is measured as the "outside air temperature." The indoor temperature set in the monitoring system 1 is acquired as the "indoor temperature." The temperature of the outside air that is heat exchanged and supplied to the indoors is measured as the "outlet temperature." PLC 200 controls the pump rotation speed by issuing a command to PLC 300 based on the outside air temperature, the indoor temperature, and the outlet temperature.

[0057] FIG. 4 shows the pump rotation speed [%] (vertical axis) controlled relative to the outlet temperature [°C] (horizontal axis) when the set temperature is 23°C and the outside air temperature is 10°C. When the pump is rotating at the maximum rotation speed, the pump rotation speed is 100%. When the pump is rotating at K% of the maximum rotation speed, the pump rotation speed is K%. The minimum rotation speed of the pump is 50% of the maximum rotation speed.

[0058] In this example, when the outlet temperature is "t1," the pump rotation speed is controlled to "50% (minimum rotation speed)." As the pump rotation speed is subsequently increased, it is controlled to "65%" at an outlet temperature of "10°C" (Test 1 in FIG. 5), to "75%" at an outlet temperature of "20°C" (Test 2 in FIG. 5), to "85%" at an outlet temperature of "25°C" (Test 3 in FIG. 5), and to "100%" at an outlet temperature of "t2°C."

[0059] 5 is a diagram showing an example of a test specification 91 for a unit test (corresponding to the test specification 415 in FIG. 3). The test specification 91 describes the test contents (tests 1 to 3, etc.) of the unit test as a confirmation test.

[0060] Test 1 is a test to confirm whether a pump rotation speed of 65% is output from PLC 200 when an outlet temperature of 10°C, a set temperature of 23°C, and an outside air temperature of 10°C are given as input values ​​to PLC 200. As shown in "Test 1" in Figure 4, the pump rotation speed gradually increases, and it is expected that "65%" will be detected as the pump rotation speed when the outlet temperature reaches 10°C.

[0061] Test 2 is a test to check whether a pump rotation speed of 75% is output as an output value from PLC 200 when an outlet temperature of 20°C, a set temperature of 23°C, and an outside air temperature of 10°C are given as input values ​​to PLC 200. As shown in "Test 2" in Figure 4, the pump rotation speed gradually increases, and it is expected that "75%" will be detected as the pump rotation speed when the outlet temperature reaches "20°C."

[0062] Test 3 is a test to check whether a pump rotation speed of 85% is output from PLC 200 when an outlet temperature of 25°C, a set temperature of 23°C, and an outside air temperature of 10°C are given as input values ​​to PLC 200. As shown in "Test 3" in Figure 4, the pump rotation speed gradually increases, and it is expected that "85%" will be detected as the pump rotation speed when the outlet temperature reaches 25°C.

[0063] Tests 1 to 3 are all the same internal process because the "processing to increase the pump rotation speed" is executed as an internal process of the PLC 200. In such a case, as described with reference to FIG. 3, the test data complementing unit 441 selects tests 2 and 3 as deletion targets.

[0064] On the other hand, the input and output values, namely, the outlet temperature, the set temperature, the outside air temperature, and the pump rotation speed, are continuous values ​​(also referred to as "continuous analog values") that can change continuously. In Tests 1 to 3, not only is it possible to confirm what pump rotation speed is output when the outlet temperature, the set temperature, and the outside air temperature are given, but it is also possible to confirm whether the pump rotation speed is increasing in a state in which an increase in the pump rotation speed is expected. In such a case, the test data complementing unit 441 sets the test attribute of Tests 1 to 3 to "continuous analog value" and excludes Tests 1 to 3 whose test attribute is set to "continuous analog value" from the deletion targets.

[0065] As a result, tests 2 and 3 would conventionally be deleted from the unit test, but in embodiment 1, they are excluded from the deletion targets and all tests 1 to 3 are executed. Note that tests 2 and 3 may be deleted from the unit test and executed in the system test.

[0066] After the unit test is completed, a system test is then performed. Figure 6 shows an example of a test specification 93 for a system test. The test specification 93 initially describes tests Y1 to Y3. Tests Y4 and Y5 are added by the process shown in the flowchart of Figure 7, which will be described later.

[0067] Test Y1 is a test to check the minimum pump rotation speed. The minimum pump rotation speed is 50%. Test Y2 is a test to check the maximum pump rotation speed. The maximum pump rotation speed is 100%. Test Y3 is a test to check a pump rotation speed other than the maximum and minimum. In this case, a pump rotation speed of 75% is checked.

[0068] The following description will be given using a flowchart. Fig. 7 is a flowchart of the process executed by the test support system 400. This process is an update process executed by the test data complementing unit 441, etc. Hereinafter, "step" may also be simply referred to as "S".

[0069] Steps S101 to S105 are processes related to the unit test specification 91. When this process starts, in step S101, the processor 411 acquires a plurality of unit tests and their corresponding tracing results from the test data 433 and the tracing results 436.

[0070] In S102, the processor 411 sets, based on the tracing result, tests among the plurality of tests whose internal processes match as tests to be deleted. In the example shown in FIG. 5 , the plurality of tests includes test 1 as the first test and tests 2 and 3 as the second tests. Tests 2 and 3 are set as tests to be deleted because their internal processes match those of test 1. Here, if test 1 and test 2 are tests that are performed consecutively, test 2 may be set as the test to be deleted, and if test 2 and test 3 are tests that are performed consecutively, test 3 may be set as the test to be deleted.

[0071] The processor 411 determines the test attribute for each of the plurality of tests based on the type of input value and the type of output value. Specifically, in S103, the processor 411 determines that the test attribute is a continuous value (continuous analog value) if the input value and the output value are continuously variable for each of the plurality of tests.

[0072] A continuously variable value is a value that differs from a value that is set to one of two values, such as 0 or 1, ON or OFF. Also, when the input value is ON or OFF and the output value is a continuously variable value, the test attribute may be determined to be a continuous value (continuous analog value).

[0073] The processor 411 excludes the second test (Tests 2 and 3) from the deletion targets based on the test attribute of the first test (Test 1) and the test attribute of the second test (Test 2). Specifically, the processor 411 excludes from the deletion targets those tests set as deletion targets in S104 that have a test attribute of continuous analog values. As a result, even if the processor 411 determines that the internal processing of the first test (Test 1) and the second test (Tests 2 and 3) is the same, if it determines that the test attribute of the first test (Test 1) and the test attribute of the second test (Tests 2 and 3) are both continuous values, the processor 411 excludes the second test (Tests 2 and 3) from the deletion targets.

[0074] In S105, the processor 411 deletes the test set as the deletion target, updates the test specification 91 of the unit test (test specification 415 in FIG. 3), and outputs the test result 438.

[0075] S106 is a process related to the test specification 93 for the system test. In S106, the processor 411 adds a test to the test specification for the system test based on the difference between the content of the unit test and the content of the system test. Adding a test to the system test can be achieved, for example, by the following method.

[0076] Test Y1 of the system tests illustrated in FIG. 6 is a test to check the pump rotation speed at "50%," Test Y2 is a test to check the pump rotation speed at "100%,," and Test Y3 is a test to check the pump rotation speed at "75%." In contrast, Test 1 of the unit tests illustrated in FIG. 5 is a test to check the pump rotation speed at "65%,," Test 2 is a test to check the pump rotation speed at "75%,," and Test 3 is a test to check the pump rotation speed at "85%." Of these, Test 2 and Test Y3 have the same pump rotation speed (75%). Meanwhile, the output values ​​of Tests 1 and 3 of the unit tests do not match the output values ​​of any of the system tests. For this reason, for example, Test Y4, which checks the output value of Test 1 (pump rotation speed "65%), and Test Y5, which checks the output value of Test 3 (pump rotation speed "85%)), are added to the system tests. In this case, after test Y3 that checks for a pump speed of 75%, test Y4 that changes the output value to the output value of test 1 (pump speed of 65%) (i.e., a test that checks for a decrease in the pump speed) may be added, and further test Y5 that changes the output value to the output value of test 3 (pump speed of 85%) (i.e., a test that checks for an increase in the pump speed) may be added. Note that a more detailed method for adding tests to the system tests will be described in embodiment 2 using Figures 8 to 11.

[0077] As described above, the test support system 400 is a system that supports a confirmation test that confirms the validity of input / output values ​​input and output in the monitoring system 1, which includes the equipment (PLC) 300 and the server device 100 that monitors the equipment via the PLC 200 as a controller. The test support system 400 includes a memory 412 and a processor 411. The memory 412 stores the contents of multiple tests to be performed in the confirmation test. The processor 411 executes an update process that updates the contents of the multiple tests. The monitoring system 1 (PLC 200) executes internal processing in response to given input values ​​and outputs output values ​​as the result. The confirmation test includes a unit test as a first confirmation test and a system test as a second confirmation test that is performed after the unit test. In the update process, the processor 411 adds a test to the system test based on the difference between the contents of the unit test and the contents of the system test.

[0078] If the test items for system testing are not sufficiently extracted and the tests that should have been performed are not carried out, defects may occur in the product when it is in operation at the customer's site after delivery. This can occur when different testers are responsible for unit testing and system testing, and the necessary information is not shared between them. For example, even if sufficient test items for unit testing are extracted, the contents of these test items may not be fully reflected in the system testing, resulting in test items that should have been performed in the system testing being missing. With the above configuration, tests can be added to the system testing based on a comparison with the contents of the unit testing, preventing defects that should have been detected in the system testing from occurring at the customer's site.

[0079] In the update process, the processor 411 deletes a test set as a deletion target from the unit test. When the input and output values ​​of each of the multiple tests are continuously variable, the processor 411 determines that the test attribute is a continuous value (continuous analog value). The multiple tests in the unit test include a first test (Test 1) and a second test (Tests 2 and 3). When the processor 411 determines that the internal processing of the first test (Test 1) and the second test (Tests 2 and 3) matches, the processor 411 sets the second test (Tests 2 and 3) as a deletion target. Even when the processor 411 determines that the internal processing of the first test (Test 1) and the second test (Tests 2 and 3) matches, the processor 411 excludes the second test (Tests 2 and 3) from the deletion target if it determines that both the test attribute of the first test (Test 1) and the test attribute of the second test (Tests 2 and 3) are continuous values ​​(continuous analog values).

[0080] In unit testing, deleting redundant tests (e.g., Tests 2 and 3 in the above example) to reduce the number of tests can result in defects that should have been detected in the unit testing being overlooked and instead detected in the next process, the system testing. In such cases, after correcting the software defects, the unit testing and system testing must be repeated, reducing the efficiency of testing. As described above, if the internal processing is consistent and the input / output values ​​are continuously variable, excluding them from the deletion targets in the update process allows the unit testing to check the change state of the output value (the continuity of the output value). This prevents defects that should have been detected in the previous process (the unit testing) from being detected in the next process (the system testing). Furthermore, as described above, test items that are missing in the system testing can be added based on the unit testing, which not only prevents defects that should have been detected in the unit testing from being detected in the system testing, but also prevents defects that should have been detected in the system testing from occurring at the customer's site.

[0081] The update process for unit testing may be performed by the test support system 400 as described above, or may be performed manually in whole or in part. For example, the implementer 5 may visually check the test specifications 91 and decide whether to exclude a test from deletion. Furthermore, a test that is selected for deletion in unit testing may be executed in system testing rather than in unit testing. Furthermore, a test that is selected for deletion may be executed in both unit testing and system testing. In the first embodiment, tests are added or deleted from multiple tests recorded in the test specifications 415 prepared in advance. However, the test support system 400 may generate all test items and record them in the test specifications 415.

[0082] Second Embodiment Next, a test support system 400 according to a second embodiment will be described. In the first embodiment, the processor 411 is configured to add a test to the system test based on the difference between the contents of the unit test and the contents of the system test in the update process. Furthermore, the processor 411 is configured to exclude a unit test from deletion targets in the update process when the internal processing matches and the input / output values ​​are continuously variable values ​​in the unit test.

[0083] In the second embodiment, a test is added to the system test by the method described below. Note that in the second embodiment, only the differences from the first embodiment will be described, and the description of the commonalities between the first embodiment and the second embodiment will be omitted.

[0084] Fig. 8 is a diagram showing control of the rotation speed of a pump in embodiment 2. The content of Fig. 8 is the same as that described in Fig. 4. Fig. 9 is a diagram showing an example of a test specification 92 for a unit test. Fig. 10 is a diagram showing an example of a test specification 93 for a system test. The content of Fig. 10 is the same as that described in Fig. 6.

[0085] The content of the unit test shown in Fig. 9 is substantially the same as the content of the unit test shown in Fig. 5. Test 1 in Fig. 5 corresponds to tests X1 to X3 in Fig. 9, test 2 in Fig. 5 corresponds to tests X4 to X6 in Fig. 9, and test 3 in Fig. 5 corresponds to tests X7 to X9 in Fig. 9.

[0086] For example, Test X1 tests whether a pump rotation speed of "65%" is output when the outlet temperature is "10°C," Test X2 tests whether a pump rotation speed of "65%" is output when the set temperature is "23°C," and Test X3 tests whether a pump rotation speed of "65%" is output when the outside air temperature is "10°C." When the outlet temperature is "10°C" and the set temperature is "23°C," as shown in FIG. 8, the outlet temperature and pump rotation speed continuously increase over time, so that, as in FIG. 5, a pump rotation speed of "75%" is measured for an outlet temperature of "20°C," and a pump rotation speed of "85%" is measured for an outlet temperature of "25°C." As a result, Tests X4 to X9 can be performed.

[0087] In the test specification 92 for the unit test, the test attribute for tests X1 to X9 is set to be a continuous analog value, and a "circle" is added to indicate that the results of tests X1 to X9 are valid.

[0088] The test specification 93 for the system test to be performed after the unit test is as shown in Fig. 10. Tests Y1 to Y3 are initially described in the test specification 93. Tests Y4 and Y5 are added by the process shown in the flowchart in Fig. 11, which will be described later.

[0089] Test Y1 is a test to check the minimum pump rotation speed. As shown in FIG. 8, the minimum pump rotation speed is 50%. Test Y2 is a test to check the maximum pump rotation speed. As shown in FIG. 8, the maximum pump rotation speed is 100%. Test Y3 is a test to check a pump rotation speed other than the maximum and minimum. As shown in FIG. 8, in this case, a pump rotation speed of 75% is checked.

[0090] 11 is a flowchart of the process executed by the test support system 400. In this process, a test is added to the system test based on the difference between the contents of the unit test and the contents of the system test. This process may be executed by the input / output test program 416 after the test result output unit 437 has finished processing, for example.

[0091] When this process starts, in S201, the processor 411 acquires the contents of multiple unit tests recorded in the unit test specification 92 and the contents of multiple system tests recorded in the system test specification 93.

[0092] In S202, the processor 411 extracts tests from the unit tests that have a test attribute of continuous values ​​(continuous analog values) and matching internal processing. For example, in the example of FIG. 9, tests X3, X6, and X9 that have a test attribute of continuous analog values ​​and matching internal processing are extracted.

[0093] In S203, the processor 411 classifies the extracted tests into tests whose output values ​​match the output values ​​of the system tests and tests whose output values ​​do not match the output values ​​of the system tests. Test Y1 of the system tests is a test to check the pump rotation speed at "50%", test Y2 is a test to check the pump rotation speed at "100%," and test Y3 is a test to check the pump rotation speed at "75%". Test X3 of the unit tests is a test to check the pump rotation speed at "65%," test X6 is a test to check the pump rotation speed at "75%," and test X9 is a test to check the pump rotation speed at "85%". Of these, test X6 matches test Y3 in terms of pump rotation speed (75%), but tests X3 and X9 do not match the output values ​​of any of the system tests.

[0094] In S204, the processor 411 adds to the system tests a test that changes the output value from a matched output value to a mismatched output value, and a further test that changes the output value to another mismatched output value. In the above example, after test Y3, which confirms that the pump speed is 75%, whose output matches that of test X6, test Y4 is added, which changes the output value to the output value of test X3 (pump speed 65%) (i.e., a test that confirms a decrease in the pump speed), and further adds test Y5, which changes the output value to the output value of test X9 (pump speed 85%) (i.e., a test that confirms an increase in the pump speed).

[0095] As explained above, if it is determined that the internal processing of the first test (Test X6) and the second test (Test X3) in the unit test is consistent, and if it is determined that the test attributes of the first test (Test X6) and the second test (Test X3) are both continuous analog values, and if the system test does not include a test to check the output value of the second test (Test X3) (pump rotation speed "65%)), then in the update process, an additional test Y4 to check the output value of the second test (Test X3) is added to the system test.

[0096] In particular, if the system test includes a test to confirm the output value (pump speed "75%) of the first test (test X6), an additional test Y4 may be added to confirm that the output value (pump speed "75%) of the first test (test X6) changes to the output value (pump speed "65%) of the second test (test X3) (pump speed decreases).

[0097] This allows the results of unit testing and system testing to be compared, and the test contents of the system testing can be supplemented with test contents that have been performed in the unit testing but not in the system testing, thereby preventing defects that should be detected in the system testing from occurring at the customer's site.

[0098] The embodiments disclosed herein are intended to be combined as appropriate within the scope of any technical inconsistency. The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The technical scope of the present disclosure is defined by the claims, not the description of the above-mentioned embodiments, and is intended to include all modifications within the meaning and scope of the claims.

[0099] 1 Monitoring system, 5 Implementer, 91-93 Test specification, 100 Server device, 111, 211, 311, 411 Processor, 112, 212, 312, 412 Memory, 113, 213, 313, 413 Communication unit, 214, 314 Input / output terminal, 420 Input unit, 421 Display unit, 200 PLC, 300 Equipment (PLC), 400 Test support system, 401 Support tool, 402 Browser, 403 Console, 414 Operation flow, 415 Test specification, 416 Input / output test program, 417 Input / output test test code, 432 Test data generation unit, 433 Test data, 434 Test data execution unit, 435 Test data tracing unit, 436 Tracing result, 437 Test result output unit, 438 Test result, 440 Test attributes, 441 Test data complement.

Claims

1. A test support system that supports a confirmation test to confirm the validity of input / output values ​​input and output in a monitoring system having equipment and a server device that monitors the equipment via a controller, comprising: a memory that stores the contents of multiple tests to be performed in the confirmation test; and a processor that executes an update process to update the contents of the multiple tests, wherein the monitoring system executes internal processing in response to given input values ​​and outputs an output value as a result, the confirmation tests include a first confirmation test and a second confirmation test that is performed after the first confirmation test, and wherein the processor, in the update process, adds a test to the second confirmation test based on the difference between the contents of the first confirmation test and the contents of the second confirmation test.

2. The test support system of claim 1, wherein the processor, in the update process, deletes a test set as a deletion target from the first confirmation test; determines that a test attribute is a continuous value if the input value and output value in each of the multiple tests are values ​​that can change continuously; the multiple tests in the first confirmation test include a first test and a second test; and, when the processor determines that the internal processing of the first test and the second test match, sets the second test as the deletion target; and, even when the processor determines that the internal processing of the first test and the second test match, excludes the second test from the deletion target when it is determined that the test attribute of the first test and the test attribute of the second test are both the continuous value.

3. The test support system of claim 2, wherein when the internal processing of the first test and the second test is determined to be consistent, and when the test attributes of the first test and the test attributes of the second test are both determined to be continuous values, and when the second confirmation test does not include a test to confirm the output value of the second test, in the update process, the processor adds an additional test to the second confirmation test to confirm the output value of the second test.

4. A test support system as described in claim 3, wherein when the second confirmation test includes a test to confirm the output value of the first test, the additional test is a test to confirm the change from the output value of the first test to the output value of the second test.

5. A control method for a test support system that supports a confirmation test to confirm the validity of input / output values ​​input and output in a monitoring system having equipment and a server device that monitors the equipment via a controller, the control method comprising: a step of storing contents of multiple tests to be performed in the confirmation test; and a step of updating the contents of the multiple tests, the monitoring system executes internal processing in accordance with given input values ​​and outputs an output value as a result, the confirmation tests include a first confirmation test and a second confirmation test performed after the first confirmation test, and the updating step further comprises a step of adding a test to the second confirmation test based on a difference between the contents of the first confirmation test and the contents of the second confirmation test.

Citation Information

Patent Citations

  • Method for generating test specification

    JP1993240745A

  • Automatic test execution system

    JP2009217664A

  • Test device

    JP2014229168A

  • Test device for monitoring device and test program

    JP2018018126A