Automatic test system of vehicle-mounted software

Abstract

The invention provides an automatic test system for vehicle-mounted software, and the system comprises a test case management unit which is used for determining a to-be-executed target test case and sending the target test case to a compiling unit, and the target test case comprises a target test code file and a target configuration file; the compiling unit is used for compiling the target test code file and a source code file involved in the target test code according to the target configuration file to obtain an executable file corresponding to the target test case; the execution unit is used for executing the executable file after establishing connection with the vehicle-mounted controller installed with the vehicle-mounted software, reading an actual execution result of the test variable from a test array pre-established by the vehicle-mounted controller, and feeding back the actual execution result to the test case management unit; and the test case management unit is used for comparing the actual execution result of the test variable with the expected execution result to determine a test result for the test variable.

Description

Technical Field

[0001] This application relates to the technical field of vehicle testing, and more particularly, to an automated testing system for in-vehicle software. Background Art

[0002] In order to ensure the quality and reliability of the software R & D and version upgrade of intelligent connected vehicle products, and to timely discover and improve their problem points, continuous testing is required. However, the current in-vehicle software testing method is manually completed by testers throughout the process, which is time-consuming and laborious, with high labor costs, and has strict requirements for testers. Once there is negligence, the software quality cannot be guaranteed. Especially in the field of automotive electronics, low-quality in-vehicle software will also affect the driving safety of vehicles. Summary of the Invention

[0003] This application provides an automated testing system for in-vehicle software, which can not only implement the automated testing of in-vehicle software, but also quickly read and determine whether the values of the variables in the code under test meet the expectations, thereby further improving the test coverage and completeness.

[0004] The specific technical solutions are as follows:

[0005] In a first aspect, an embodiment of this application provides an automated testing system for in-vehicle software, and the system includes: a test case management unit, a compilation unit, and an execution unit;

[0006] The test case management unit is used to determine the target test case to be executed and send the target test case to the compilation unit, where the target test case includes a target test code file and a target configuration file;

[0007] The compilation unit is used to compile the target test code file and the source code files involved in the target test code according to the target configuration file to obtain an executable file corresponding to the target test case;

[0008] The execution unit is used to execute the executable file after establishing a connection with the vehicle controller installed with the in-vehicle software, and read the actual execution result of the test variable from the test array pre-established by the vehicle controller, and feedback the actual execution result to the test case management unit, where the test array includes the mapping relationship between the test case name, the test variable name, and the actual execution result, and the values of the test case name and the test variable name are stored before the execution unit executes the executable file, and the value of the actual execution result is added after the execution unit executes the executable file;

[0009] The test case management unit is used to determine the test result for the test variable by comparing the actual execution result and the expected execution result of the test variable.

[0010] In a possible implementation manner, the compilation unit is used to, when the target configuration file is the same as the configuration files of at least one executed test case, copy the compiled target test project to the folder where the target test case is located, replace the test code file in the compiled test project with the target test code file, and use a preset incremental compilation method to only compile the target test code file to generate an executable file corresponding to the target test case, where the compiled target test project is the test project corresponding to the test case belonging to the configuration file that is the same as the target configuration file.

[0011] In a possible implementation manner, the test case management unit is used to compare the lengths of the actual execution result and the expected execution result of the test variable;

[0012] When the lengths are inconsistent, determine that the test result for the test variable is a test error;

[0013] When the lengths are consistent, compare each element in the actual execution result and the expected execution result one by one, and when any element is different, stop the comparison operation and determine that the test result for the test variable is a test error. When all elements are the same, determine that the test result for the test variable is a test pass.

[0014] In a possible implementation manner, the execution unit is used to perform at least one of the following actions on the vehicle-mounted controller:

[0015] Memory erasure, code burning, start control, debug setting;

[0016] The memory erasure is used to erase the historical data in the preset storage space to make the preset storage space in a writable state;

[0017] The code burning is used to write the compiled program or firmware into the non-volatile memory of the vehicle-mounted controller so that it can run according to the expectations of the latest test case;

[0018] The start control is used to control the process of the vehicle-mounted controller from reset to system initialization so that the tested code is correctly loaded and executed;

[0019] The debug setting includes at least one of capturing variable status, capturing function call stack, assisting in analyzing the cause of exceptions, pausing program execution, checking code logic line by line, and register changes.

[0020] In a possible implementation, the system further includes:

[0021] A communication unit, configured to receive a request message sent by the test case management unit, send the request message to the vehicle-mounted controller, receive an actual response message sent by the vehicle-mounted controller, and feedback the actual response message to the test case management unit;

[0022] The test case management unit is configured to perform statistics on the actual response message according to the message statistical algorithm corresponding to the message expected result, obtain the actual message result, and determine the test result of the vehicle-mounted controller in terms of the performance of receiving and sending messages by comparing the message expected result with the actual message result.

[0023] In a possible implementation, when the message expected result includes an expected transmission period, the message statistical algorithm corresponding to the message expected result includes statistically calculating the difference between the timestamps carried in two adjacent actual response messages as the actual message result; and / or,

[0024] When the message expected result includes the expected message content, the message statistical algorithm corresponding to the message expected result includes reading the content of the actual response message and using the content of the actual response message as the actual message result; and / or,

[0025] When the message expected result includes the expected number of responses, the message statistical algorithm corresponding to the message expected result includes statistically calculating the number of actual response messages as the actual message result.

[0026] In a possible implementation, the communication unit is further configured to test whether the receiving and sending of actual messages by the vehicle-mounted controller has changed by sending multiple irrelevant messages to the vehicle-mounted controller.

[0027] In a possible implementation, the execution unit includes a power management module;

[0028] The power management module is configured to power off the vehicle-mounted controller first and then power it on again after the execution unit receives the power-on and power-off test instruction issued by the test case management unit;

[0029] The test case management unit is configured to test whether the receiving and sending of actual messages by the vehicle-mounted controller has changed through the communication unit after the vehicle-mounted controller is powered on again.

[0030] In a possible implementation, the power management module is further configured to perform fault injection on the data transmission bus;

[0031] The test case management unit is configured to test whether the transceiver status of the actual messages of the vehicle-mounted controller has changed through the communication unit after introducing a bus fault.

[0032] In a possible implementation, the system further includes:

[0033] A log unit, configured to record the execution processes and results of other units in the system in the form of log files, and save communication files of messages in the case of test cases related to message transceiver in the other units, where the communication files are used to store test records.

[0034] In a possible implementation, the system further includes:

[0035] A test report management unit, configured to generate a test report by statistically analyzing the test results of each test case, and output the test report.

[0036] In a second aspect, an embodiment of the present application provides an electronic device, where the electronic device includes the system according to any implementation of the first aspect.

[0037] As can be seen from the above, the automated test system for vehicle-mounted software provided by the embodiment of the present application can first determine a target test case to be executed by the test case management unit, and then the compilation unit compiles the target test code file and the source code files involved in the target test code according to the target configuration file in the target test case to obtain an executable file corresponding to the target test case. After the execution unit establishes a connection with the vehicle-mounted controller installed with the vehicle-mounted software, the execution unit executes the executable file, directly reads the actual execution results of the test variables from a test array pre-established in the vehicle-mounted controller, and hands them over to the test case management unit to determine the test results for the test variables by comparing the actual execution results and the expected execution results of the test variables. It can be seen that the test system provided by the embodiment of the present application can not only realize the automated test function of vehicle-mounted software without relying on manual labor, thereby improving the test efficiency, but also pre-store the actual execution results of the test variables in the test array, so that the execution unit can directly and quickly read the actual execution results from the test array, without spending a lot of time searching for the actual execution results from all the execution results of the executable file, thereby realizing the rapid test of each test variable, making the test granularity more detailed, and improving the test coverage and completeness.

[0038] In addition, the technical effects that can also be achieved by the embodiments of the application include:

[0039] 1. In the embodiment of the present application, the source code files in the test cases corresponding to the same configuration file are compiled only once, and the test code files corresponding to the same configuration file are compiled by using the incremental compilation method, thereby greatly reducing the repeated compilation of the source code files and improving the compilation efficiency.

[0040] 2. Compared with directly and blindly comparing each actual execution result with the expected execution result element by element, the embodiment of the present application can quickly draw a conclusion that the two execution results are different by comparing the lengths of the execution results, thereby improving the efficiency of result comparison.

[0041] 3. The communication unit provided by this test system supports reading aspects such as the message content, quantity, and period for the test case management unit to judge. Since the interval for the in-vehicle controller to send messages is generally between dozens of milliseconds or even a few milliseconds, applying this communication unit can enable the test case management unit to judge the message attributes without missing any message, making up for the limitations of manual testing.

[0042] 4. The communication unit realizes a kind of fault injection by sending multiple irrelevant messages to the in-vehicle controller, and then determines whether this fault injection affects the in-vehicle controller by checking whether the transceiver situation of the valid messages in the communication between the communication unit and the in-vehicle controller has changed, thereby determining the reliability of the in-vehicle software.

[0043] 5. In the embodiment of the present application, the power management module powers on and off the in-vehicle controller and injects faults into the data transmission bus to test the influence of power on / off and the data bus on the in-vehicle controller, thereby judging the reliability of the in-vehicle software.

[0044] 6. This test system records the execution process and results of other units in the system in the form of log files through the log unit, and in the case of test cases involving message transceiver, saves the communication files of the messages, and generates and outputs test reports through the test report management unit for testers to view.

[0045] Of course, implementing any product or method of the present application does not necessarily need to achieve all the above-mentioned advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for description in the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0047] Figure 1An architecture diagram of an automated test system for in-vehicle software provided by an embodiment of the present application;

[0048] Figure 2 A schematic diagram of an automated test interaction process for in-vehicle software provided by an embodiment of the present application;

[0049] Figure 3 Another architecture diagram of an automated test system for in-vehicle software provided by an embodiment of the present application. Detailed implementation manners

[0050] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0051] It should be noted that the terms "including" and "having" in the embodiments of the present application and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0052] Figure 1 An architecture diagram of an automated test system for in-vehicle software provided by an embodiment of the present application. This system can be applied to an electronic device. Before testing, software for configuration and management, such as Configurator software, compiler, debugging software, communication driver, and this automated test system, has been installed in the electronic device. This system can be implemented in programming languages such as Python and Java. This system mainly includes: a test case management unit 110, a compilation unit 120, and an execution unit 130;

[0053] The test case management unit 110 is used to determine the target test case to be executed and send the target test case to the compilation unit 120, where the target test case includes a target test code file and a target configuration file;

[0054] The compilation unit 120 is used to compile the target test code file and the source code files involved in the target test code according to the target configuration file to obtain an executable file corresponding to the target test case;

[0055] An execution unit 130, which is configured to execute an executable file after establishing a connection with a vehicle-mounted controller that installs vehicle-mounted software, read the actual execution result of a test variable from a test array pre-established by the vehicle-mounted controller, and feed back the actual execution result to a test case management unit 110. The test array includes the mapping relationship between the test case name, the test variable name, and the actual execution result. Before the execution unit 130 executes the executable file, the values of the test case name and the test variable name are stored, and after the execution unit 130 executes the executable file, the value of the actual execution result is added;

[0056] The test case management unit 110 is configured to determine the test result for the test variable by comparing the actual execution result and the expected execution result of the test variable.

[0057] In the embodiment of the present application, the test case management unit 110 can complete the file management work of test codes, configuration files, test scripts, etc. in each test case to ensure that the correct files are used when the use case is executed. It can also screen test cases before each test execution and start execution in a predetermined order. The test case management unit 110 can determine the test case corresponding to the test case name input by the tester as the target test case to be executed, or can sequentially determine each test case as the target test case to be executed according to the order of the test cases described in the test sequence. Each test case includes a configuration file and a test code file, so the target test case includes a target test code file and a target configuration file. The configuration file includes test objectives, test environments, preconditions, expected results, etc. The configuration file can be an arxml file.

[0058] As Figure 2 shown, a compilation unit 120 is responsible for preparing a test project and compiling an executable file. First, start the Configurator software, open the target configuration file provided by the test case management unit 110, generate configuration codes, and complete the update of the configuration file in the test project. Then, according to specific compilation rules, start multi-threaded compilation of the source files involved in the target test code file by means of command line calls, and link the intermediate files to form a binary executable file. The compilation rules may include, but are not limited to, makefile, cmakelist, etc. If a compilation error occurs, interrupt the execution of this use case and output the corresponding compilation error information to a log file for testers to view and debug.

[0059] As Figure 2As shown, the execution unit 130 can operate in an asynchronous programming manner. After passing in the executable file path and establishing a connection with the vehicle-mounted controller, the execution unit 130 successively performs actions such as memory erasure, code burning, start control, and debug (fault elimination) setting on the vehicle-mounted controller, so that the latest code of the current test case is run each time on the vehicle-mounted controller. If any of the above actions encounters an error, the execution of the test case is interrupted, and the corresponding error code is written into the running log file.

[0060] Among them, memory erasure is used to erase the historical data in the preset storage space, so that the preset storage space is in a writable state. This is a prerequisite for writing the code under test, to avoid program conflicts or logical errors caused by residual data. Among them, the preset storage space includes Flash, EEPROM, etc.

[0061] Code burning is used to write the compiled program or firmware into the non-volatile memory of the vehicle-mounted controller, so that it can operate as expected by the latest test case.

[0062] Start control is used to control the process of the vehicle-mounted controller from reset to system initialization, so that the code under test is correctly loaded and executed.

[0063] The debug settings include at least one of the settings such as (real-time) capturing the variable state, (real-time) capturing the function call stack, assisting in analyzing the cause of exceptions, pausing the program execution, checking the code logic and register changes line by line.

[0064] After the executable file runs for a specified time, the test case management unit 110 will send an instruction to the execution unit 130 to indicate reading the test variable. The execution unit 130 will read the actual execution result of the test variable from the vehicle-mounted software and feedback it to the test case management unit 110 for further judgment of the test result. Specifically, the execution unit 130 obtains the distribution of the code and data in the memory, as well as their positions and sizes in the executable file, etc. by reading the map file during the code compilation process. After screening out the memory address of a certain test variable, it then reads the value of the test variable (i.e., the actual execution result) from the vehicle-mounted controller according to this address and feedbacks it to the use case management unit to complete the assertion of the actual result and the expected result. Of course, if the debugger supports directly reading the value according to the variable name, the execution unit 130 can omit the process of parsing the map file, directly read the variable value, and feedback it to the use case management unit.

[0065] Among them, when the actual execution result of the test variable is the same as the expected execution result, it is determined that there is no problem with the in-vehicle software in calculating this test variable. When the actual execution result of the test variable is different from the expected execution result, it is determined that there is a problem with the in-vehicle software in calculating this test variable. The test result can be saved in the form of a log for testers to view.

[0066] In addition, starting from the test code in the in-vehicle controller, the Test_Common.c file can be introduced therein. By calling interfaces such as TestResult_Set_8BitData, TestResult_Set_16BitData, and TestResult_Set_32BitData inside the file, key variables are written into a test array (such as the TestResult array), and finally the array value is obtained through the execution unit 130.

[0067] The execution unit 130 can also record the execution time of each test case. For example, relevant test code can be added in the test code file to record the start and end times of code execution. After taking the difference between the two, the TestResult_Set_32BitData interface in Test_Common.c is called to write it into the TestResult array, and then the execution unit 130 reads this time according to the TestResult variable address and records it in the log file.

[0068] The automated test system for in-vehicle software provided by the embodiments of the present application can first have the test case management unit determine the target test case to be executed, and then the compilation unit compiles the target test code file and the source code files involved in the target test code according to the target configuration file in the target test case to obtain the executable file corresponding to the target test case. After the execution unit establishes a connection with the in-vehicle controller where the in-vehicle software is installed, it executes this executable file, directly reads the actual execution result of the test variable from the test array pre-established in the in-vehicle controller, and hands it over to the test case management unit to determine the test result for the test variable by comparing the actual execution result and the expected execution result of the test variable. It can be seen that the test system provided by the embodiments of the present application can not only realize the automated test function of in-vehicle software without relying on manual labor, thereby improving the test efficiency, but also pre-store the actual execution result of the test variable in the test array, enabling the execution unit to directly and quickly read the actual execution result from the test array without spending a lot of time searching for the actual execution result from all the execution results of the executable file, thus realizing the quick test of each test variable, making the test granularity more detailed, and improving the test coverage and completeness.

[0069] In a possible implementation, when a large number of test cases are executed, if all source code files of each test case are compiled, it is easy to cause the problems of repeated compilation and long time consumption, which will inevitably hinder the implementation of the subsequent automated test process. Therefore, in order to improve the compilation efficiency, the embodiment of the present application makes the following improvements to the compilation unit 120:

[0070] The compilation unit 120 is configured to, when the target configuration file is the same as the configuration file of at least one executed test case, copy the compiled target test project to the folder where the target test case is located, replace the test code file in the compiled test project with the target test code file, and use a preset incremental compilation method to only compile the target test code file to generate an executable file corresponding to the target test case, where the compiled target test project is the test project corresponding to the test case to which the configuration file identical to the target configuration file belongs.

[0071] Among them, in order to improve the efficiency of comparing configuration files, it is possible to directly judge whether two configuration files are the same by calculating the hash values of the two configuration files. Specifically, calculate the hash value of the target configuration file and the hash value of the configuration file of each executed test case respectively. When it is determined that there is a configuration file with the same hash value as the target configuration file, it is determined that this configuration file is the same as the target configuration file. Among them, the algorithms for calculating hash values include but are not limited to SHA256, MD5, etc.

[0072] In addition, the preset incremental compilation method can be the incremental compilation method in makefile.

[0073] Actual tests found that the compilation time of a test project is between 30s and 120s, while by the method of the current compilation unit 120, the compilation between test cases with the same configuration only takes 3s, and the efficiency is increased by more than 90%.

[0074] The embodiment of the present application only compiles the source code files in the test cases corresponding to the same configuration file once, and uses the incremental compilation method to compile the test code files corresponding to the same configuration file, thereby greatly reducing the repeated compilation of source code files and improving the compilation efficiency.

[0075] In a possible implementation, the test case management unit 110 is configured to compare the lengths of the actual execution result and the expected execution result of the test variable;

[0076] In the case of inconsistent lengths, determine that the test result for the test variable is a test error;

[0077] When the lengths are the same, each element in the actual execution result and the expected execution result is compared one by one. When any element is different, the comparison operation is stopped, and the test result for the test variable is determined to be a test error. When all elements are the same, the test result for the test variable is determined to be a test success.

[0078] Compared with directly and blindly comparing each element of each actual execution result with the expected execution result one by one, the embodiments of the present application can quickly draw a conclusion that the two execution results are different by comparing the lengths of the execution results, thereby improving the efficiency of result comparison.

[0079] In a possible implementation manner, as Figure 2 and Figure 3 shown, the system may further include:

[0080] A communication unit 140, configured to receive a request message sent by the test case management unit 110, send the request message to the vehicle-mounted controller, receive an actual response message sent by the vehicle-mounted controller, and feed back the actual response message to the test case management unit 110;

[0081] A test case management unit 110, configured to perform statistics on the actual response message according to the message statistical algorithm corresponding to the message expected result, obtain the message actual result, and determine the test result of the vehicle-mounted controller in terms of message sending and receiving performance by comparing the message expected result with the message actual result.

[0082] In the embodiments of the present application, when the message expected result includes an expected sending period, the message statistical algorithm corresponding to the message expected result includes statistically calculating the difference between the timestamps carried in two adjacent actual response messages as the message actual result; and / or,

[0083] When the message expected result includes the message expected content, the message statistical algorithm corresponding to the message expected result includes reading the content of the actual response message and using the content of the actual response message as the message actual result; and / or,

[0084] When the message expected result includes an expected response quantity, the message statistical algorithm corresponding to the message expected result includes statistically calculating the quantity of the actual response messages as the message actual result.

[0085] In an automated test system, when the communication unit 140 starts up, it is necessary to ensure that the execution unit 130 has started correctly and is running continuously. It can complete the sending and receiving of network messages while the in-vehicle software is running, and can use various network protocols such as CAN (Controller Area Network), CANFD (CAN with Flexible Data rate), LIN (Local Interconnect Network), and Ethernet. The communication unit 140 supports reading aspects such as message content, quantity, and period for the test case management unit 110 to judge.

[0086] Since the interval for the in-vehicle controller to send messages is generally between dozens of milliseconds or even a few milliseconds, using this communication unit 140 can enable the test case management unit 110 to judge the message attributes without missing any message, making up for the limitations of manual testing.

[0087] In a possible implementation, the communication unit 140 is also used to test whether the sending and receiving of actual messages by the in-vehicle controller has changed by sending multiple irrelevant messages to the in-vehicle controller.

[0088] Specifically, the communication unit 140 realizes a kind of fault injection by sending multiple irrelevant messages to the in-vehicle controller, and then determines whether this fault injection has an impact on the in-vehicle controller by checking whether the sending and receiving of valid messages in the communication between the communication unit 140 and the in-vehicle controller has changed, so as to determine the reliability of the in-vehicle software.

[0089] In a possible implementation, as Figure 3 shown, the execution unit 130 may include a power management module 131;

[0090] The power management module 131 is used to cut off the power supply of the in-vehicle controller first and then power it on again after the execution unit 130 receives the power-on and power-off test instruction issued by the test case management unit 110.

[0091] The test case management unit 110 is used to test whether the sending and receiving of actual messages by the in-vehicle controller has changed through the communication unit 140 after the in-vehicle controller is powered on again.

[0092] In the embodiment of the present application, the power management module 131 powers on and off the in-vehicle controller to test the impact of power-on and power-off on the in-vehicle controller, so as to judge the reliability of the in-vehicle software.

[0093] In addition, the power management module 131 can also be used to inject faults into the data transmission bus;

[0094] The test case management unit 110 is used to test whether the transceiver situation of the in-vehicle controller for the actual message has changed through the communication unit 140 after introducing a bus fault.

[0095] For example, the power supply principle module can control CAN_H (i.e., the high-voltage line) and CAN_L (i.e., the low-voltage line) in the CAN bus to implement operations such as short circuit and open circuit of the CAN bus, so as to test the impact of the bus fault on the in-vehicle controller.

[0096] In a possible implementation manner, as Figure 3 shown, the system may further include:

[0097] The log unit 150 is used to record the execution process and execution results of other units in the system in the form of a log file.

[0098] The log unit 150 can be nested in the remaining units, can identify the test case number, test case name, file path, compilation result, execution information, and network message, and execute with locking among multiple processes, and output the identified information to the log file in sequence to realize the automatic saving of the test record.

[0099] Specifically, the log unit 150 can save the execution log information of the entire automated test system as a log file. For test cases involving message transceiver, communication files of the messages will also be saved, such as asc files, blf files, pcap files, etc., to retain the test records for testers to consult.

[0100] In a possible implementation manner, as Figure 3 shown, the system may further include:

[0101] The test report management unit 160 is used to generate a test report by statistically analyzing the test results of each test case and output the test report.

[0102] The test report management unit 160 can complete tasks such as collecting assertion results in the test script, statistically analyzing execution information, and drawing the test report. The statistical analysis of the test results runs through the entire test case execution process. If all the above steps are executed successfully, information such as the test case number, test case name, and execution time will be statistically written into the test report. If any error occurs, the execution of the test case can be interrupted and the test case will be marked as failed.

[0103] When multiple test cases need to be executed, the above steps can be repeated continuously. It should be noted that regardless of whether the current test case passes or fails, the next test case can continue to be executed. After all test cases are executed, all test results will be statistically analyzed, and the test report will be drawn according to information such as the tester and test description passed in when the system is started.

[0104] It should be added that the automated test system for in-vehicle software can test any module in the Autosar Classic Platform architecture. In addition to diagnostics and network management which require communication modules, the Autosar Classic Platform architecture also contains many management modules (such as BswM, EcuM, ComM), intermediate modules (such as Crc, PduR, SoAd), etc. For the testing of these modules, network packets do not need to be sent and received. For such tests, the embodiments of the present application can freely choose whether to use the communication unit 140, thus greatly increasing the scope of automated testing applicable to the embodiments of the present application, and making it applicable to the automated testing of each module under the Autosar Classic Platform architecture.

[0105] Taking the BswM module in the system service layer under the Autosar Classic Platform architecture as an example, the above testing process is described as follows:

[0106] [1] BswM is short for Basic Software Mode Manager, and it is a module in the system service layer under the Autosar Classic Platform architecture. Its responsibility is to arbitrate mode requests from the application layer SW-C or other BSW modules according to simple rules and perform operations according to the arbitration results. Briefly understanding its function is arbitration and execution. It itself does not have the function of sending and receiving network packets.

[0107] [2] Before the test starts, it should be ensured that the test scripts, test codes, test configurations and other files related to this module are correctly stored in the automated test system.

[0108] [3] The automated test system has the test functions of all functional modules under the Autosar Classic Platform architecture. Testers can screen the BswM module and the test cases under this module according to actual needs, and select all or part of the test cases to be executed in this test.

[0109] [4] The test case management unit can automatically find the test code file, configuration file and other test files of this test case saved in the specified folder according to the name information of the test case, and send this information to the compilation unit.

[0110] [5]Based on the above information, the compilation unit obtains the test code file and configuration file of this test case, copies and replaces the test code into the test project. Then it calls the third-party interface of the Configurator tool chain to open the configuration file and generate configuration code into the test project. Thus, according to the test requirements and test configuration of this test case, the two jointly form a test project.

[0111] [Note] For this test case, when the current configuration arbitration state is NO_COMMUNICATION, it will trigger the BswM module to perform arbitration. In the test code, only the BswM_CanSM_CurrentState() interface of the BswM module needs to be called, and then the arbitration state can be observed.

[0112] [6] The compilation unit writes relevant compilation rules and configurations, starts multi-threaded compilation by means of command-line invocation, generates an executable file that can run on the in-vehicle controller, and obtains the result after compilation. If a compilation error occurs, the execution of this test case is interrupted, and the corresponding compilation error information is output to the log file for testers to consult and debug.

[0113] [7] According to the generated executable file above, the execution unit burns it into the in-vehicle controller through a series of operations such as erasing the memory, burning the code, and running the code, and feeds back the code running status to the test case management unit.

[0114] [9] During the test, it is also necessary to compare the values in the TestResult array. After the code runs for the specified time, the test case management unit will send a signal to the execution unit. The execution unit will read the actual execution result of this variable from the in-vehicle software and feed it back to the test case management unit, enabling it to compare the actual execution result and the expected execution result of the test variable to determine the test result for the test variable.

[0115]

[10] After the test execution is completed, the test report management unit will collect the execution information of each test case, write information such as the test case number, test case name, and execution time into the test report, completing a fully automated test process.

[0116]

[11] For easy understanding, a simple test case is given below

[0117] Init:

[0118] Make

[0119] Begin:

[0120] Run

[0121] assert TestResultCompare == 0

[0122] End

[0123]

[12] The test case distribution of the test case management unit, and operations such as the copying of the compilation unit test code and the generation of configuration code are all implicit in the Init operation, minimizing repetitive labor and improving efficiency. After the initialization operation is completed, the test project has been prepared according to the test requirements of different test cases and the configuration file. This test project already contains the call to the BswM_CanSM_CurrentState() interface described above.

[0124] The Make statement calls the compilation unit to compile the test project to obtain an executable file.

[0125] After running for a specified time, the execution unit will obtain the value of TestResultCompare, which is judged by the test case management unit to determine whether it matches the expected execution result.

[0126] Finally, the test report management unit will collect the case execution information, output the log to the log file, and write information such as the case number, case name, and execution time into the test report.

[0127]

[13] It should be noted that all test cases for modules that do not need to send and receive messages can apply this test script, which greatly reduces the workload of script development and the difficulty of using this automated test system.

[0128] In addition, when testing the Diagnostic Communication Management (DCM) module under the Autosar Classic Platform architecture, etc., the communication unit is also involved. The test case management unit distributes the message information to the communication unit according to the actual test needs to complete the sending of relevant CAN messages. After the communication unit receives the response message, it feeds it back to the test case management unit for further judgment on attributes such as the message content, period, and quantity. Among them, in the AUTOSAR Classic Platform architecture, the full name of the Dcm module is Diagnostic Communication Manager, which is a module for diagnostic communication management and is located in the communication service layer. Its main role is to be responsible for diagnostic data flow and management of diagnostic states, including diagnostic sessions, security states, and diagnostic service allocation, etc.

[0129] Exemplarily, when testing the DCM module, the test cases used can include:

[0130] Init:

[0131] Make

[0132] Begin:

[0133] Run

[0134] Send 10 03

[0135] assert received 50 03

[0136] assert TestResultCompare===0

[0137] End

[0138] The test system can also be applied to the transmission and reception of LIN network protocol messages. The communication unit can automatically convert the sent and received messages into the header and response format according to the communication mode of the LIN network protocol, transmit and receive LIN messages, and judge the message content and subsequent TestResultCompare value.

[0139] For example, when testing the communication mode of the LIN network protocol, the test cases used may include:

[0140] Init:

[0141] Make

[0142] Begin:

[0143] Run

[0144] Send 3C Header

[0145] Send 3C Response 10 03

[0146] Send 3D Header

[0147] assertreceived3DResponse 50 03

[0148] assert TestResultCompare===0

[0149] End

[0150] The above test system can also be applied to the Ethernet network protocol for message sending and receiving. The communication unit can automatically send and receive Ethernet messages according to the communication mode of the Ethernet protocol, and judge the message content and the subsequent TestResultCompare value. The message sending and receiving interfaces adopted in this embodiment and the DCM module are the same because the network protocol is encapsulated in the communication unit, so that the external interfaces exposed by CAN communication and Ethernet communication are the same, which minimizes the user's usage cost and simplifies the subsequent maintenance steps.

[0151] Exemplarily, when testing the communication mode of the Ethernet network protocol, the test cases used may include: Init:

[0152] Make

[0153] Begin:

[0154] Run

[0155] Send 10 03

[0156] assert received 50 03

[0157] assert TestResultCompare==0

[0158] Based on the above method embodiments, another embodiment of the present application provides an electronic device, and the electronic device includes the automated test system of the in-vehicle software described in any of the above embodiments.

[0159] The above virtual device embodiments and entity device embodiments correspond to the method embodiments respectively, and have the same technical effects as the method embodiments. For specific descriptions, please refer to the method embodiments. The device embodiments are obtained based on the method embodiments. For specific descriptions, please refer to the method embodiment part and will not be repeated here. Those of ordinary skill in the art can understand that the drawings are only schematic diagrams of an embodiment, and the modules or processes in the drawings are not necessarily essential for implementing the present application.

[0160] Those of ordinary skill in the art can understand that the modules in the device in the embodiments can be distributed in the devices in the embodiments according to the descriptions in the embodiments, or can be correspondingly changed and located in one or more devices different from the present embodiments. The modules in the above embodiments can be combined into one module, or can be further split into multiple sub-modules.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An automated test system for in-vehicle software, characterized in that, The system includes: a test case management unit, a compilation unit, and an execution unit; The test case management unit is configured to determine a target test case to be executed and send the target test case to the compilation unit, where the target test case includes a target test code file and a target configuration file; The compilation unit is configured to compile the target test code file and the source code files involved in the target test code according to the target configuration file to obtain an executable file corresponding to the target test case; The execution unit is configured to, after establishing a connection with a vehicle-mounted controller installed with vehicle-mounted software, execute the executable file, read the actual execution result of a test variable from a test array pre-established by the vehicle-mounted controller, and feedback the actual execution result to the test case management unit, where the test array includes a mapping relationship between a test case name, a test variable name, and an actual execution result, and values of the test case name and the test variable name are stored before the execution unit executes the executable file, and a value of the actual execution result is added after the execution unit executes the executable file; The test case management unit is configured to determine a test result for the test variable by comparing the actual execution result and the expected execution result of the test variable.

2. The system according to claim 1, wherein The compilation unit is configured to, when the target configuration file is the same as the configuration files of at least one executed test case, copy the compiled target test project to the folder where the target test case is located, replace the test code file in the compiled test project with the target test code file, and use a preset incremental compilation method to only compile the target test code file to generate an executable file corresponding to the target test case, where the compiled target test project is the test project corresponding to the test case with the same configuration file as the target configuration file.

3. The system according to claim 1, wherein The test case management unit is configured to compare the lengths of the actual execution result and the expected execution result of the test variable; In the case of inconsistent lengths, determine that the test result for the test variable is a test error; In the case of consistent lengths, compare each element in the actual execution result and the expected execution result one by one, and stop the comparison operation when any element is different, determine that the test result for the test variable is a test error, and when all elements are the same, determine that the test result for the test variable is a test pass.

4. The system according to claim 1, wherein The execution unit is configured to perform at least one of the following actions on the vehicle-mounted controller: Memory erasure, code burning, start control, debug setting; The memory erasure is used to erase historical data in a preset storage space to make the preset storage space in a writable state; The code burning is used to write the compiled program or firmware into the non-volatile memory of the vehicle-mounted controller so that it can run according to the expectations of the latest test case; The startup control is used to control the process of the vehicle-mounted controller from reset to system initialization, so that the code under test is correctly loaded and executed; The debug settings include at least one of capturing variable states, capturing function call stacks, assisting in analyzing the causes of exceptions, pausing program execution, checking code logic line by line, and register changes.

5. The system according to claim 1, wherein The system further includes: A communication unit, configured to receive a request message sent by the test case management unit, send the request message to the vehicle-mounted controller, receive an actual response message sent by the vehicle-mounted controller, and feedback the actual response message to the test case management unit; The test case management unit is configured to perform statistics on the actual response message according to the message statistical algorithm corresponding to the message expected result, obtain the actual result of the message, and determine the test result of the vehicle-mounted controller in terms of message sending and receiving performance by comparing the message expected result with the actual result of the message.

6. The system according to claim 5, characterized in that, When the message expected result includes an expected sending period, the message statistical algorithm corresponding to the message expected result includes statistically calculating the difference between timestamps carried in two adjacent actual response messages as the actual result of the message; and / or, When the message expected result includes the expected message content, the message statistical algorithm corresponding to the message expected result includes reading the content of the actual response message and using the content of the actual response message as the actual result of the message; and / or, When the message expected result includes the expected number of responses, the message statistical algorithm corresponding to the message expected result includes statistically calculating the number of actual response messages as the actual result of the message.

7. The system according to claim 5, wherein The execution unit includes a power management module; The power management module is configured to power off the vehicle-mounted controller first and then power it on again after the execution unit receives the power on / off test instruction issued by the test case management unit; The test case management unit is configured to test whether there is a change in the sending and receiving of actual messages by the vehicle-mounted controller through the communication unit after the vehicle-mounted controller is powered on again.

8. The system according to claim 7, wherein The power management module is further configured to perform fault injection on the data transmission bus; The test case management unit is configured to test whether there is a change in the sending and receiving of actual messages by the vehicle-mounted controller through the communication unit after a bus fault is introduced.

9. The system according to claim 1, wherein The system further includes: A log unit, configured to record the execution process and execution results of other units in the system in the form of a log file, and save the communication file of the message in the case of test cases involving message sending and receiving in other units, where the communication file is used to store test records.

10. The system according to any one of claims 1-9, characterized in that, The system further includes: A test report management unit, configured to generate a test report by performing statistics on the test results of each test case and output the test report.

Images