Test system and test execution method

Abstract

This test system, in which a plurality of electronic devices to be tested can be connected to a test execution unit, comprises: a test management unit that manages the execution of tests on the electronic devices; a test overall flow control unit that controls the entire test execution unit; and a switch mechanism that switches and connects electronic devices on which tests are to be executed. Upon receiving a test execution request for a first electronic device and a second electronic device that are connected to the test execution unit, the test management unit transmits the received test execution request to the test overall flow control unit. The test overall flow control unit executes a test on the first electronic device according to the received test execution request, instructs the switch mechanism to switch the test target to the second electronic device after the test on the first electronic device is completed, and starts a test on the second electronic device according to the received test execution request.

Description

Test system and test execution method

[0001] The present invention relates to a test system and a test execution method.

[0002] In recent years, as vehicle control has become more sophisticated and complex, the scale of software for electronic control units (ECUs) has increased, creating a need for an environment in which this software can be tested efficiently.

[0003] One conventional software verification environment is a Hardware-in-the-Loop Simulation (HILS) verification system that verifies the electronic control unit (ECU) incorporated into the actual system in conjunction with a Hardware-in-the-Loop Simulation (HILS) device. Operational verification of the ECU board is performed by connecting an MCD (Measurement, Calibration and Diagnostics) tool to the ECU board and verifying the software installed on the ECU board. HILS verification systems have become more automated and network-compatible, making remote verification possible (see Non-Patent Document 1).

[0004] Furthermore, with the advancement of network technology, it is becoming increasingly necessary for multiple people to share the verification environment. Job management system technology used in general-purpose computers and server systems can be applied to enable multiple people to share resources and execute, finish, and report jobs (see Non-Patent Document 2).

[0005] A HILS verification system requires a HILS device, an ECU board, an MCD tool, and a PC to control each device, which makes it difficult to secure the initial cost and a location to install the system, and it is therefore necessary to improve the operating rate of a single HILS verification system.

[0006] MCD tools are multifunctional and some allow you to choose from multiple methods for connecting to the ECU board, so you can select the functions and connection method of the MCD tool to suit the ECU board of the verification target.

[0007] "Test Automation Software", [online], February 2020, dSPACE GmbH, Internet <URL:https: / / www.dspace.com / shared / data / pdf / 2020 / dSPACE_TestAutomation_SW_Product-information_2020_01_200804_J_ebook.pdf> "Quick Guide to LSF [User Edition]", [online], October 1, 2009, HPC Systems, Inc., Internet<URL:https: / / www.hpc.co.jp / images / pdf / manual / support_002_091001.pdf>

[0008] In the background art described above, the connections and settings between devices that make up a verification system using a HILS device are fixed, limiting the scope of possible verification. No means has been proposed for remotely changing the connections and settings between devices that make up the verification system.

[0009] A representative example of the invention disclosed in the present application is as follows: That is, a test system capable of connecting a plurality of electronic devices to be tested to a test execution unit, comprising: a test management unit that manages execution of tests on the electronic devices; an overall test flow control unit that controls the entire test execution unit; and a switch mechanism that switches and connects the electronic devices to be tested, wherein upon receiving a test execution request for a first electronic device and a second electronic device connected to the test execution unit, the test management unit transmits the received test execution request to the overall test flow control unit, the overall test flow control unit executes a test on the first electronic device in accordance with the received test execution request, and after testing of the first electronic device is completed, instructs the switch mechanism to switch the test target to the second electronic device, and starts testing of the second electronic device in accordance with the received test execution request.

[0010] According to one aspect of the present invention, it is possible to expand the verification range of a test system using a HILS device. Objects, configurations, and effects other than those described above will become apparent from the following description of the embodiments.

[0011] It is a diagram showing the overall configuration of the test system of this embodiment. It is a diagram showing the configuration of the test overall flow control unit of this embodiment. It is a sequence diagram of the processing executed by the test system of this embodiment. It is a diagram showing the configuration of the switch mechanism of this embodiment. It is a diagram showing an example of the configuration of the request table of this embodiment.

[0012] The configuration and processing of one embodiment of the present invention will be described below with reference to the drawings. In the following description, unless otherwise specified, the same elements will be referred to by the same reference numerals throughout the drawings. It should be understood that the configuration and processing described here are described as one embodiment, and that it is not intended to limit the technical scope of the present invention to this embodiment.

[0013] FIG. 1 is a diagram showing the overall configuration of the test system of this embodiment.

[0014] In the test system of this embodiment, the test management unit 100 registers test execution requests from the verifier's computers 301, 302, and 303, controls the test execution order, and outputs the test execution request to the test execution unit 200. The test execution unit 200 references the equipment information stored in the DB 400 to determine whether the verification configuration specified in the test execution request is configurable. If it determines that the configuration is configurable, it executes the test and returns the test execution results to the test management unit 100. The test management unit 100 reports the test environment (how the HILS device 221, tools 231-233, and ECU boards 251-253 are connected) and the test results to the verifier's computers 301-303 that issued the test request and the system administrator's computer 304. The test management unit 100 also obtains the test execution status (e.g., completed, running, incomplete) from the test execution unit 200 and transmits it to the verifier's computers 301-303 that issued the test request and the system administrator's computer 304. The test management unit 100 is implemented as software executed on a computer. The test execution unit 200 is composed of electronic devices, measuring instruments, computers, and the like.

[0015] The test management unit 100 and the test execution unit 200 are implemented on a computer having a processor, memory, auxiliary storage device, and a communication interface. The processor is a computing device that executes programs stored in memory. The processor executes various programs to realize the functions provided by the test management unit 100 (the reception management unit 110, the test execution management unit 120, and the test execution client 130). Note that some of the processing performed by the processor executing the programs may be executed by another computing device (e.g., hardware such as an ASIC or FPGA). The memory includes ROM, which is a non-volatile storage element, and RAM, which is a volatile storage element. ROM stores unchanging programs (e.g., BIOS). RAM is a high-speed, volatile storage element such as DRAM (Dynamic Random Access Memory) that temporarily stores programs executed by the processor and data used during program execution. The auxiliary storage device is a large-capacity, non-volatile storage device such as a magnetic storage device that stores programs executed by the processor and data used during program execution. The communication interface is a network interface device that controls communication with other devices according to a predetermined protocol.

[0016] Next, the processing of each unit will be described.

[0017] The test management unit 100, the verifier's computers 301 to 303, and the system administrator's computer 304 are connected via a network. When the test management unit 100 receives a test execution request from the system administrator's computer 304 or the verifier's computers 301 to 303, it refers to the request table 500 to determine whether processing of the received type of request is permitted, and if processing of the received type of request is permitted, it executes the processing in accordance with the received test execution request.

[0018] The usage request table 500 shown in Figure 5 is implemented in the reception management unit 110 and determines whether processing of the usage request is permitted in the test system for each combination of the type of user of the test system (system administrator, verifier) ​​and the type of usage request.

[0019] If the request from the system administrator's computer 304 is a system configuration change request, the reception management unit 110 changes the system configuration information managed in the database 400 and reports the change results to the system administrator's computer 304 that made the request. The system administrator sends a system configuration change request when changing the equipment used by the test execution unit 200. Furthermore, if the request from the verifier's computers 301-303 or the system administrator's computer 304 is a configuration inquiry request, the reception management unit 110 queries the database 400 for configuration information of the test execution unit 200 and reports the results obtained by the inquiry to the requesting computers 301-303, 304. Furthermore, if the request from the verifier's computers 301-303 or the system administrator's computer 304 is a test execution request, the reception management unit 110 notifies the test execution management unit 120 of the test execution. Furthermore, when the test execution is completed, the reception management unit 110 receives a notification and the test execution result from the test execution management unit 120, and reports the received test execution result to the computer that made the request.

[0020] The test execution management unit 120 manages test execution. That is, the test execution management unit 120 queues test execution requests and information required for test execution from the reception management unit 110, determines the test execution order according to the reception order and priority of the queued execution requests so that tests with execution requests with high priority and earliest reception are executed first, and notifies the test execution client 130 of the test request with the highest priority. The priority is included and indicated in the test execution request. The test execution management unit 120 receives test completion notifications, test results, and test execution instruction source information from the test execution client 130, and reports them to the reception management unit 110.

[0021] The test execution client 130 connects the test management unit 100 and the test execution unit 200 via a network, and issues test execution requests from the test execution management unit 120 to the test execution unit 200. In addition, when the test execution unit 200 completes test execution, the test execution client 130 receives the test results and reports the received test results to the test execution management unit 120.

[0022] The test execution unit 200 connects the devices required for test execution, configures each device, executes the test according to the test scenario, and reports the execution results to the test management unit 100.

[0023] The overall test flow control unit 210 is a program executed by a computer, and controls the entire test execution unit 200. When the overall test flow control unit 210 receives a test execution request, it checks whether the devices can be connected according to the device configuration included in the test execution request, and if the connections are possible, it instructs the switch mechanism 241 to connect the devices via the switch control PC 240. Next, it changes and sets the settings of the devices to be tested (for example, the controlled model) via the HILS control PC 220.

[0024] The tool control PC 230 sets the tools 231 to 233 and the ECU boards 251 to 253 used in the test.

[0025] After the settings for each device are completed, the overall test flow control unit 210 executes a test by issuing instructions to each device via the control PCs 220, 230, and 240. The test results of the executed tests are obtained via the control PCs 220, 230, and 240, and the obtained test results are reported to the test management unit 100.

[0026] FIG. 2 is a diagram showing the configuration of the overall test flow control unit 210. As shown in FIG.

[0027] The overall test flow control unit 210 is made up of a test flow control unit 211 , a state management unit 212 , a setting change unit 213 , and an execution processing unit 214 .

[0028] FIG. 3 is a sequence diagram of the process executed by the test system of this embodiment.

[0029] When the test flow control unit 211 receives a test execution request and registration information from the test management unit 100, it outputs requested test environment configuration information and an environment check instruction to the status management unit 212 (S10). The registration information includes the requested test environment configuration, the test scenario, and the test execution source information.

[0030] When the status management unit 212 receives an environment check instruction, it acquires test environment information of the device implemented in the test execution unit 200 from the database 400. The status management unit 212 executes an environment check to compare the requested test environment configuration with the test environment information (S11), and returns the result of determining whether the requested test environment can be configured by the test execution unit 200 to the test flow control unit 211. The test flow control unit 211 judges the confirmation result of the status management unit 212, and if the configuration is possible by the test execution unit 200, it sends the requested test environment configuration information and a test environment configuration instruction to the setting change unit 213. On the other hand, if the configuration is not possible by the test execution unit 200, it returns to the test management unit 100 a message indicating that the test environment cannot be configured.

[0031] When the setting change unit 213 receives the test environment setting instruction, it issues a setting clear instruction to each of the control PCs 220, 230, 240 to clear the settings of each device (HILS device 221, tools 231 to 233, switch mechanism 241, ECU boards 251 to 253) (S12).

[0032] After receiving the test environment setting instruction, the setting change unit 213 obtains the setting information required for each device from the database 400 (S13), selects the tools required for the received test environment setting instruction, determines the combinations of the tools 231-233 and the ECU boards 251-253, and transmits the setting information and setting instructions to each of the control PCs 220, 230, and 240 (S14). After completing the setting of all devices, the setting change unit 213 reports a setting completion report to the test flow control unit 211. If the test flow control unit 211 receives a normal setting completion report from the setting change unit 213, it transmits a test execution instruction to the execution processing unit 214. On the other hand, if the test flow control unit 211 receives an abnormal setting completion report from the setting change unit 213, it replies to the test management unit 100 that the setting of the devices has ended abnormally.

[0033] When the execution processing unit 214 receives the test execution instruction, it causes each device to execute a test according to the test scenario via each control PC 220, 230, 240 (S15). The HILS device 221 verifies the functions of the software implemented on the ECU boards 251 to 253. When the test is completed, the execution processing unit 214 collects the test results from each control PC 220, 230, 240 (S16) and reports them to the test flow control unit 211.

[0034] When the test flow control unit 211 collects the test results, it reports the test results, the configured test environment, and test execution instruction source information to the test management unit 100 (S17). When one test is completed, the test flow control unit 211 determines whether there are any unexecuted test execution requests, and if there are any unexecuted test execution requests, it configures the environment for the next test to be executed and executes the next test.

[0035] The processing of the test execution unit 200 in Fig. 1 will be explained based on the operation of the test flow control unit 211 in Fig. 2. The details of the switch mechanism 241 are shown in Fig. 4.

[0036] Before starting a test, the test execution unit 200 performs a clear process for each device. The settings of each device (HILS device 221, tools 231 to 233, switch mechanism 241, ECU boards 251 to 253) are cleared from the overall test flow control unit 210 (setting change unit 213) via the control PCs 220, 230, and 240.

[0037] Next, the test execution unit 200 connects the devices (HILS device 221, tools 231 to 233, switch mechanism 241, ECU boards 251 to 253). For example, the switch mechanism 241 connects the devices according to instructions from the overall test flow control unit 210 via a switch control PC 240. The switch mechanism 241 has an FPGA (Field Programmable Gate Array) and analog switches for switching connections, and driver devices and receiver devices for various interface signals.

[0038] The test execution unit 200 includes a HILS device 221, multiple ECU boards (ECU_A board 251, ECU_B board 252, ECU_C board 253), and multiple tools (tool 1 (231), tool 2 (232), tool 3 (233)). The ECU boards 251 to 253 are electronic devices that simulate electronic control units (ECUs), and the electronic control units themselves may also be used. The tools 231 to 233 are MCD (Measurement, Calibration and Diagnostics) tools that access the ECU boards 251 to 253 and change the settings of the ECU boards 251 to 253.

[0039] For example, a verification environment can be configured by connecting the HILS device 221, the ECU_A board 251 to be verified, tool 1 (231), and tool 2 (232). In response to an instruction from the overall test flow control unit 210, the switch control PC 240 instructs the digital / analog signal switch 242 to switch via a switch instruction unit 245, thereby connecting the HILS device 221 and the ECU_A board 251. The ECU_A board 251 and tool 1 (231) are connected by a communication interface such as a Controller Area Network (CAN) or a Local Interconnect Network (LIN) of an in-vehicle network. Upon receiving an instruction from the test overall flow control unit 210, the switch control PC 240 instructs the communication IF switch 243 to switch via the switch instruction unit 245, thereby connecting the ECU_A board 251 and tool 1 (231).

[0040] Upon receiving an instruction from the test overall flow control unit 210 , the switch control PC 240 instructs the JTAG switch 244 to switch via the switch instruction unit 245 , and connects the ECU_A board 251 and tool 2 ( 232 ) via the JTAG switch 244 .

[0041] Each switch, the digital / analog signal switch 242, the communication IF switch 243, and the JTAG switch 244, is composed of a selector circuit that selects the signal input from each device, and an enable control circuit that selects the output destination device and outputs it, and the control signal to the circuit is input from the switch instruction unit. Each switch circuit has a different signal level, so it is composed of a circuit that is suitable for the signal level.

[0042] The overall test flow control unit 210 sets each device via each control PC.

[0043] The HILS control PC 220 rewrites the HILS control program and the control target model to be executed, and changes the settings of the HILS device 221, upon receiving an instruction from the overall test flow control unit 210.

[0044] The tool control PC 230, upon receiving an instruction from the overall test flow control unit 210, rewrites the execution script for each tool and changes the settings of tool 1 (231) and tool 2 (232).

[0045] In response to an instruction from the overall test flow control unit 210, the tool control PC 230 acquires ROM data, which is the software to be verified and is installed in ECU_A, via tool 2 (232) and the switch mechanism 241, and rewrites the ROM data on the ECU_A board 251 to change the settings of the ECU_A board 251. The tool control PC 230 reports the completion of setting each device to the overall test flow control unit 210.

[0046] Next, the overall test flow control unit 210 sets a test scenario in the HILS device 221 via the HILS control PC 220, and sets the test scenario in tool 1 (231) and tool 2 (232) via the tool control PC 230. After completing the setting of the test scenario, the overall test flow control unit 210 sends execution instructions to each of the devices 221, 231 to 233, 241, 251 to 253 via the control PCs 220, 230, and 240, and executes a series of tests.

[0047] The overall test flow control unit 210 collects test completion reports from each control PC, and when it determines that the test is complete, it acquires the logs of each device 221, 231 to 233, 241, 251 to 253 via each control PC 220, 230, 240, and completes the test.

[0048] The database 400 manages information on the devices 221 , 231 to 233 , 241 , 251 to 253 used by the test execution section 200 , data for operating the devices 221 , 231 to 233 , 241 , 251 to 253 , and status information of the test execution section 200 .

[0049] As described above, according to the embodiment of the present invention, it is possible to expand the verification scope of a test system using the HILS device 221. In other words, by using the HILS device 221 to switch between multiple verifiable ECU boards 251 to 253 and multiple devices (tools 231 to 233) used for verification, it becomes possible to configure a test verification environment required by each verifier, thereby expanding the test scope of the test system.

[0050] In addition, by using a test system that uses HILS equipment, the ECU board to be verified and the equipment used for testing can be switched according to the verifier's instructions, making testing possible even in a remote environment.

[0051] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added, deleted, or replaced with other configurations.

[0052] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by a processor interpreting and executing a program that realizes each function.

[0053] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, hard disk, or SSD (Solid State Drive), or in a recording medium such as an IC card, SD card, or DVD.

[0054] In addition, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily represent all the control lines and information lines that are necessary for implementation. In reality, it can be assumed that almost all components are interconnected.

Claims

1. A test system comprising: a test execution unit that executes tests on the electronic device; a test management unit that manages the execution of tests on the electronic device; an overall test flow control unit that controls the entire test execution unit; and a switch mechanism that switches the electronic device that executes the test and connects it to the test execution unit, wherein when the test management unit receives a test execution request for a first electronic device and a second electronic device connected to the test execution unit, it sends the received test execution request to the overall test flow control unit, and the overall test flow control unit executes a test on the first electronic device in accordance with the received test execution request, and after testing of the first electronic device is completed, it instructs the switch mechanism to switch the test target to the second electronic device, and starts testing of the second electronic device in accordance with the received test execution request.

2. A test system according to claim 1, wherein said test management unit executes tests in order of the electronic devices that receive the test execution request first.

3. A test system according to claim 1, wherein the test management section determines the execution order of the tests of the electronic device according to the priority of the tests to be executed.

4. A test system according to claim 1, comprising a plurality of test tools for changing the settings for the test, wherein the overall test flow control unit selects the test tool in response to the received test execution request.

5. A test system as described in claim 4, comprising a database for storing information on the test execution unit including setting information on the electronic devices, setting information on the test tool, and connection information by the switch mechanism, wherein the test overall flow control unit compares the required test environment configuration registered by the verifier with the test environment information, and when it is determined that the required test environment can be configured in the test execution unit 200, the test execution unit determines the combination of the electronic devices in accordance with the required test environment configuration.

6. A test system according to claim 1, characterized in that a plurality of communication interfaces for connecting to the electronic device are implemented, and the overall test flow control unit switches to a communication interface selected in response to the test execution request.

7. A test system according to claim 1, wherein the test execution request verifies the functionality of software implemented in the electronic device.

8. A test system according to claim 1, wherein the overall test flow control unit outputs a test environment in which the test of the electronic device is executed.

9. A test system as described in claim 4, comprising a database that stores information about the test environment, including setting information for the electronic device, setting information for the test tool, and connection information by the switch mechanism, and the test overall flow control unit refers to the database to determine whether the test environment included in the test setting request can be configured.

10. A test system according to claim 1, wherein said test management unit outputs a test execution status.

11. A test system as described in claim 9, characterized in that the test overall flow control unit determines the combination of the electronic device, the test tool, and the control target model to be used in the test execution unit based on information obtained from the database, and configures the test environment.

12. A test execution method for an electronic device executed by a test system, wherein the test system is composed of a computer having an arithmetic unit that executes predetermined arithmetic processing and a storage device connected to the arithmetic unit, and comprises: a test execution unit that executes tests of the electronic device; a test management unit that manages the execution of tests of the electronic device; an overall test flow control unit that controls the entire test execution unit; and a switch mechanism that switches and connects the electronic device that executes the test, wherein the test execution method comprises the steps of: when the test management unit receives a test execution request for a first electronic device and a second electronic device connected to the test execution unit, sending the received test execution request to the overall test flow control unit; the overall test flow control unit executing a test of the first electronic device in accordance with the received test execution request; the overall test flow control unit instructing the switch mechanism to switch the test target to the second electronic device after testing of the first electronic device is completed; and the overall test flow control unit starting a test of the second electronic device in accordance with the received test execution request.

Images