Verification System and Verification Method

Abstract

Provided is a verification system for executing a test relating to an electronic control device, wherein: the verification system comprises an acceptance management unit that accepts a request for the test and test information relating to the test, an execution sequence determination unit that determines the execution sequence of a plurality of tests accepted by the acceptance management unit, and a test execution unit that executes the test in accordance with the execution sequence determined by the execution sequence determination unit; and the execution sequence determination unit compares configuration values of configuration items of the verification system that are included in the test information and determines the execution sequence of the tests such that tests for which the test information is similar are executed in succession.

Description

Technical Field

[0001] The present invention relates to a verification system and a method for executing tests of a verification apparatus.

Background Art

[0002] In recent years, with the increasing sophistication and complexity of vehicle control, the scale of the software of an ECU (Electric Control Unit) has been growing, and an environment for efficiently testing this software has become necessary.

[0003] One of the conventional software verification environments is a HILS verification system that verifies by linking an electronic control unit (ECU) incorporated in an actual system and a Hardware-in-the-Loop Simulation (HILS) apparatus. The HILS verification system models a plant to be verified and executes software. The verification system using HILS has advanced in automation and network compatibility, enabling remote verification (see Non-Patent Document 1).

[0004] Furthermore, with the progress of network technology, it is essential to share and use a verification environment among multiple people. Sharing resources among multiple people and performing job execution, termination, reporting, etc. can apply the technology of a job management system used in general-purpose computers and server systems (see Non-Patent Document 2).

[0005] In job management used in a computer, since calculation settings before executing a job for calculation resources are unnecessary or completed in a short time, the setting time is not a problem. On the other hand, in the HILS verification system, prior settings are required before executing a test, such as rewriting the ROM in which the software to be verified is written or changing the connection between the ECU and the HILS.

Prior Art Documents

Non-Patent Documents

[0006]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0007] In order to efficiently execute tests using a verification system that can automatically and continuously execute tests, if the next test can be executed immediately after the completion of the previous test, the efficiency of test execution can be improved. However, when executing the next test, if time is required to change the settings of the verification system, the test execution efficiency of the verification system cannot be improved. For example, it takes time to change the settings, such as swapping the software to be verified between the previously executed test and the next test to be executed, or replacing it with a different plant model.

[0008] The present invention aims to shorten the setup time of the test environment and improve the test efficiency of the verification system by changing the execution order of the tests.

Means for Solving the Problems

[0009] A typical example of the invention disclosed in the present application is as follows. That is, a verification system that executes tests related to an electronic control device, comprising: a reception management unit that receives test requirements of the tests and test information related to the tests; an execution order determination unit that determines the execution order of a plurality of tests received by the reception management unit; and a test execution unit that executes the tests according to the execution order determined by the execution order determination unit, wherein the execution order determination unit compares the set values of the setting items of the verification system included in the test information, and determines the execution order of the tests so as to continuously execute tests with similar test information.

Advantages of the Invention

[0010] According to one aspect of the present invention, the time for setting the test environment can be shortened, and the test efficiency of the verification system can be improved. Problems, configurations, and effects other than those described above will be clarified by the description of the following embodiments.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Modes for Carrying Out the Invention

[0012] First, an overview of the embodiments of the present invention will be described. In a verification system capable of automatically and continuously executing tests, if the settings of the immediately preceding test and the settings of the next test to be executed are the same, there is no need to change the settings, and the setting change time becomes zero. Also, for some setting changes, the setting time can be shortened. To achieve this, it is advisable to select a test with the same settings as the immediately preceding test or a test with similar settings from the tests waiting to be executed. The setting content of the verification system at the time of executing the immediately preceding test can be obtained from the files required for setting each device. Then, compare the setting files of each device of the verification system corresponding to the test waiting to be executed with the setting file used in the immediately preceding test, determine whether the setting values of the two setting files match or do not match, and select the test whose setting values of all setting files match as the next test to be executed. If there is no test whose setting values of all setting files match, select a test whose setting values of some setting files match. If the selection condition is only file comparison, there may be multiple tests as selection candidates. When multiple tests are selection candidates, it is advisable to preferentially execute the test with the longest waiting time among the candidate tests.

[0013] Hereinafter, with reference to the drawings, the configuration and processing of an embodiment of the present invention will be described. In the following description, unless otherwise specified, the same elements are referred to by the same reference numerals throughout the drawings. It should be understood that the configuration and processing described here are described as an example, and there is no intention to limit the technical scope of the present invention to this example.

[0014] FIG. 1 is a diagram showing the overall configuration of the simulation system of this embodiment.

[0015] In the simulation system of this embodiment, test execution requests from the computers 301, 302, and 303 of each verifier are registered with the test management unit 100. The test management unit 100 controls the test execution order to be executed and outputs the test execution requests to the HILS verification environment 200. After receiving the test execution requests from the test management unit 100, the HILS verification environment 200 sends the test execution results to the test management unit 100. The test management unit 100 reports the test execution to the computers 301, 302, and 303 of the verifier who is the test requester. The test management unit 100 is implemented by software executed on a computer. The HILS verification environment 200 is composed of electronic devices, measuring instruments, computers, and the like.

[0016] Next, the processing of each part will be described.

[0017] The test management unit 100 is implemented on a computer having a processor, a memory, an auxiliary storage device, and a communication interface. The processor is an arithmetic unit that executes programs stored in the memory. By the processor executing various programs, the functions provided by the test management unit 100 (the reception management unit 110, the test execution management unit 120, the test execution client 170) are realized. Note that a part of the processing performed by the processor executing the program may be executed by another arithmetic unit (for example, hardware such as an ASIC or an FPGA). The memory includes a non-volatile storage element ROM and a volatile storage element RAM. The ROM stores unchangeable programs (for example, BIOS). The RAM is a high-speed and volatile storage element such as a DRAM (Dynamic Random Access Memory), and temporarily stores the programs executed by the processor and the data used during the execution of the programs. The auxiliary storage device is a large-capacity and non-volatile storage device such as a magnetic storage device, and stores the programs executed by the processor and the data used by the processor during the execution of the programs. The communication interface is a network interface device that controls communication with other devices according to a predetermined protocol.

[0018] The computers 301, 302, and 303 of each verifier and the test management unit 100 are connected by a network. The reception management unit 110 of the test management unit 100 receives test execution requests and test information (e.g., test conditions) required for the test from the computers 301, 302, and 303 of each verifier, and notifies the execution order control unit 130. In addition, the reception management unit 110 transfers the received test information to the test information storage unit 160. Also, the reception management unit 110 notifies the test results to the computers 301, 302, and 303 of the verifier that registered the test execution request.

[0019] The test execution management unit 120 has an execution order control unit 130, a test information storage unit 160, a test execution client 170, and HILS verification environment setting information 180.

[0020] The execution order control unit 130 enqueues the test execution requests received from the reception management unit 110 into the execution queue 140. Also, different from a general queue, the execution order control unit 130 dequeues an element (test execution request) from the execution queue 140 according to the priority judgment determined by the execution order determination unit 150, and not the head element of the execution queue 140, and notifies the test execution client 170. The execution order control unit 130 dequeues an element from the execution queue 140 at the timing when there is an element in the execution queue 140 and the processing of the HILS verification environment 200 is idle.

[0021] The test information storage unit 160 stores the test information received from the reception management unit 110 for each test execution request. The test information stores the test scenario at the time of test execution, the expected value, the setting file required for the setting of the HILS environment, the data to be written to the ROM, and the PLANT model data, etc. The execution order determination unit 150 determines the priority of the test execution request by referring to the test information. The test information storage unit 160 extracts the test information associated with the dequeued element at the timing when an element (test execution request) is dequeued from the execution queue 140, and sends it to the test execution client 170 and the execution order determination unit 150.

[0022] The test execution client 170 receives a test execution request from the execution order control unit 130 and test information from the test information storage unit 160. Next, the test execution client 170 transmits the test information together with a test execution instruction to the test execution flow control unit 211 of the execution server 210 in the HILS verification environment 200 via the network. The test execution flow control unit 211 executes a series of tests according to the received test execution instruction and then reports test completion. When receiving the report of test completion, the test execution client 170 reports the test result to the reception management unit 110.

[0023] The HILS verification environment setting information 180 stores test information passed when instructing the HILS verification environment 200 from the test execution client 170. The test information stored in the HILS verification environment setting information 180 is used as comparison reference information when the execution order determination unit 150 determines the priority of the test execution request.

[0024] The HILS verification environment 200 includes an execution server 210 that controls the overall operation, an ECU board 220 necessary for executing tests, an interface board 230 including circuits necessary for connecting between the ECU board 220 and the HILS device 240, and a HILS device 240 for executing a plant model and a circuit model.

[0025] The test execution flow control unit 211 communicates with the test execution client 170 and instructs the execution of processing to the setting change processing unit 212, the test execution control unit 213, and the expected value check unit 214.

[0026] The setting change processing unit 212 rewrites the ROM mounted on the ECU board 220 via the measurement & programming tool 250 from the setting information received from the test execution client 170, switches the signal line connection switch of the interface board 230, and changes the settings of the HILS verification environment 200 such as replacing the plant model of the HILS device 240.

[0027] The test execution control unit 213 sets the data necessary for test execution in the ECU board 220, the interface board 230, the HILS device 240, and the measurement & programming tool 250 according to the test scenario received via the test execution flow control unit 211, executes the test, and then acquires the test result.

[0028] The expected value check unit 214 compares the test result acquired by the test execution control unit 213 with the expected value of the test result received via the test execution flow control unit 211, and reports the comparison result to the test execution client 170 via the test execution flow control unit 211.

[0029] Figure 2 is a flowchart of the processes executed by the execution order determination unit 150.

[0030] After the processing start (151), the execution order determination unit 150 waits (152) until an element waiting for order is stored in the execution waiting queue 140. When an element is stored in the execution waiting queue 140, the element at the head of the execution waiting queue 140 is extracted (153), and it is determined whether the waiting time of the extracted element in the execution waiting queue 140 is equal to or longer than a predetermined maximum time (154). If the waiting time of the extracted element in the execution waiting queue 140 is equal to or longer than the maximum waiting time, the test of the head element is executed (157). By this processing, the elapsed time from the reception time of the test is prevented from becoming long. On the other hand, if the waiting time of the extracted element in the execution waiting queue 140 is shorter than the maximum waiting time, the comparison score is calculated by comparing the current setting state of the HILS verification environment 200 with the setting values of all the elements in the execution waiting queue 140 (155). The current setting state of the HILS verification environment 200 is a set value set according to the test information of the currently executing test or the test immediately executed before. Then, among the results compared in step 155, the element with the highest comparison score and the youngest order in the execution waiting queue 140 is extracted (156). Then, the test execution client 170 is instructed to execute a series of tests (157). After the test execution instruction, the test execution client 170 waits for a processing completion report of a series of tests (158). When the processing completion report is received, it is determined that the test is completed, and the process returns to step 152, and waits until an element waiting for order in the execution waiting queue 140 is stored. In this way, the process is repeatedly executed.

[0031] FIG. 3 is a diagram showing the verification environment setting items of the HILS verification environment 200 and their setting times.

[0032] Under certain conditions, it takes 50 seconds to rewrite the ECU board ROM, 10 seconds to switch the I / F setting, 20 seconds to replace the PLANT model, and 15 seconds to set the measuring instrument. If the rewriting of the ECU board ROM can be omitted, the time required for setting change can be reduced to about half.

[0033] FIG. 4 is a table in which the comparison scores are calculated from the respective settings of Verifier B and Verifier C, using the verification environment setting items of the HILS verification environment 200 in FIG. 3 and their setting times as weights.

[0034] For the test execution requests registered by Verifier B, among the setting items No1 to No4, the set values defined in the setting file in the setting items No2 to No4 match the set values defined in the setting file of the immediately preceding executed test, and the comparison score obtained by summing the weights of the items with matching set values is 45. For the test execution requests registered by Verifier C, the set values defined in the setting file match in all the setting items No1 to No4, and the comparison score is 95. The test execution request registered by Verifier C with the larger calculated comparison score has a higher similarity to the current setting state of the HILS verification environment 200. Thus, when the tests of Verifier B and Verifier C are waiting to be executed, the next test to be executed is the test execution request registered by Verifier C with a higher comparison score (i.e., higher similarity).

[0035] Figure 5 is a time chart of the processing from when the test management unit 100 issues a test execution instruction to the HILS verification environment 200 until the test completion report.

[0036] When the test execution client 170 receives a series of test execution instructions from the test execution management unit 120, it issues a test execution instruction to the test execution flow control unit 211. When the test execution flow control unit 211 receives the test execution instruction, it issues a setting change request instruction to the setting change processing unit 212. When the setting change processing unit 212 receives the setting change request instruction, if the setting is different from the setting of the HILS verification environment 200, it changes the setting and reports the completion of the setting change to the test execution flow control unit 211. Next, the test execution flow control unit 211 issues a test execution instruction to the test execution control unit 213. The test execution control unit 213 executes the test according to the received test execution instruction and reports the test completion to the test execution flow control unit 211. Next, the test execution flow control unit 211 performs an expected value check to compare the result of the test executed under the test execution control with the expected value of the test, and reports the generated test result to the test execution flow control unit 211. The test execution flow control unit 211 transmits the test completion report including the received test result to the test execution client 170.

[0037] FIG. 6 is a diagram showing a plurality of test requests and the swapping of the execution order.

[0038] There are test execution requests from verifier A (301), verifier B (302), and verifier C (303), and the test execution requests from verifier A (301) and verifier C (303) utilize the same HILS environment settings. The test execution request order is verifier A, verifier B, verifier C. After receiving the test result for the first time, verifier A registers an additional test execution request. In this state, the dequeue of elements from the execution waiting queue 140 is executed in the order of verifier A, verifier C, verifier A, verifier B. Executing in this order can omit the test environment settings in the second processing of verifier C and verifier A, improving the test execution efficiency.

[0039] As described above, in the simulation system of the embodiment of the present invention, when continuously executing tests that require verification environment settings before the test, by preferentially executing tests with the same or approximate setting contents as the setting contents of the verification environment set immediately before, the time for changing the verification environment settings can be made zero or shortened. As a result, the time available for executing tests in an environment using HILS becomes longer, and consequently, the test execution processing ability of the verification system can be improved.

[0040] Also, since the setting items that require a long time for change are weighted with a large coefficient to calculate the similarity (comparison score) of the setting contents, the test execution order with a high time-saving effect can be accurately determined.

[0041] Note that the present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the scope of the appended claims. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and the present invention is not necessarily limited to those having all the configurations described. Also, a part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Also, the configuration of another embodiment may be added to the configuration of one embodiment. Also, for a part of the configuration of each embodiment, addition, deletion, or replacement with other configurations may be made.

[0042] In addition, each of the above-described configurations, functions, processing units, processing means, etc. may be realized in hardware by designing part or all of them, for example, by means of an integrated circuit, or may be realized in software by a processor interpreting and executing a program for realizing each function.

[0043] Information such as programs, tables, and files for realizing each function can be stored in a storage device such as a memory, a hard disk, an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

[0044] Also, the control lines and information lines shown are those considered necessary for explanation, and do not necessarily show all the control lines and information lines required for implementation. In practice, it may be considered that almost all the components are interconnected.

Description of Reference Numerals

[0045] 100: Test Management Unit 110: Reception Management Unit 120: Reception Execution Management Unit 130: Execution Order Control Unit 140: Execution Waiting Queue 150: Execution Order Determination Unit 160: Test Information Storage Unit 170: Test Execution Client 180: HILS Verification Environment Setting Information 200: HILS Verification Environment 210: Execution Server 211: Test Execution Flow Control Unit 212: Setting Change Processing Unit 213: Test Execution Control Unit 214: Expected Value Check Unit 220: ECU Board 230: Interface Board 240: HILS Device 250: Measurement & Programming Tool

Claims

1. A verification system for executing tests related to an electronic control device, comprising: a reception management unit that receives a request for the test and test information related to the test; an execution order determination unit that determines an execution order of a plurality of tests received by the reception management unit; a test execution unit that executes the test according to the execution order determined by the execution order determination unit, wherein the execution order determination unit determines the execution order of the tests so as to continuously execute tests with similar test information by comparing setting values of setting items of the verification system included in the test information. The verification system is characterized by this.

2. The verification system according to claim 1, wherein the execution order determination unit determines the execution order of the tests in the order of similarity between the setting values of the setting items of the current verification system and the setting values of the setting items of a plurality of tests received by the reception management unit. The verification system is characterized by this.

3. The verification system according to claim 2, wherein the test information includes a plurality of setting items with different times required for changes, and the execution order determination unit weights a setting item with a long time required for change with a large coefficient to calculate the similarity of the test information. The verification system is characterized by this.

4. A verification method for a verification system to execute tests related to an electronic control device, wherein the verification system has an arithmetic device that executes predetermined arithmetic processing and a storage device connected to the arithmetic device, and the verification method includes: a reception management procedure in which the arithmetic device receives a request for the test and test information related to the test; an execution order determination procedure in which the arithmetic device determines an execution order of a plurality of tests received in the reception management procedure; a test execution procedure in which the arithmetic device executes the test according to the execution order determined in the execution order determination procedure, wherein in the execution order determination procedure, the arithmetic device determines the execution order of the tests so as to continuously execute tests with similar test information by comparing setting values of setting items of the verification system included in the test information. The verification method is characterized by this.

Images