Automated testing system for in-vehicle software

Abstract

The present application provides an automated testing system for in-vehicle software. The system comprises: a test case management unit, configured to determine a target test case to be executed and send the target test case to a compiling unit, the target test case comprising a target test code file and a target configuration file; the compiling unit, configured to, on the basis of the target configuration file, compile the target test code file and a source code file involved in a target test code, to obtain an executable file corresponding to the target test case; and an execution unit, configured to establish a connection with an in-vehicle controller on which in-vehicle software is installed and then execute the executable file, read an actual execution result of a test variable from a test array established in advance by the in-vehicle controller, and feed back the actual execution result to the test case management unit. The test case management unit is also configured to determine a test result for the test variable by comparing the actual execution result of the test variable with an expected execution result.

Description

Automated testing system for in-vehicle software This application claims priority to Chinese Patent Application No. 202510341561.6, filed on March 21, 2025, entitled "Automatic Testing System for Vehicle Software", the entire contents of which are incorporated herein by reference. Technical Field This application relates to the field of vehicle testing technology, and more specifically, to an automated testing system for in-vehicle software. Background Technology To ensure the quality and reliability of software development and version upgrades for intelligent connected vehicle products, and to promptly identify and improve upon issues, continuous testing is necessary. However, current in-vehicle software testing methods rely entirely on manual testing by personnel, which is time-consuming, labor-intensive, and resource-intensive. Furthermore, it places stringent demands on testers, and any oversight can compromise software quality. Especially in the automotive electronics field, low-quality in-vehicle software can negatively impact driving safety. Summary of the Invention This application provides an automated testing system for in-vehicle software, which can not only realize automated testing of in-vehicle software, but also quickly read and determine whether the values ​​of the variables in the tested code meet expectations, thereby further improving the test coverage and completeness. The specific technical solution is as follows: In a first aspect, embodiments of this application provide an automated testing system for vehicle-mounted software, the system comprising: a test case management unit, a compilation unit, and an execution unit; The test case management unit is used to determine the target test cases to be executed and send the target test cases to the compilation unit, wherein the target test cases include target test code files and target configuration files; The compilation unit is used to compile the target test code file and the source code files involved in the target test code according to the target configuration file to obtain the executable file corresponding to the target test case; The execution unit is configured to execute the executable file after establishing a connection with the vehicle controller that has the vehicle software installed, and read the actual execution results of the test variables from the test array pre-established by the vehicle controller, and feed back the actual execution results to the test case management unit. The test array includes a mapping relationship between test case names, test variable names and actual execution results, and stores the values ​​of test case names and test variable names before the execution unit executes the executable file, and adds the values ​​of the actual execution results after the execution unit executes the executable file. The test case management unit is used to determine the test result for the test variable by comparing the actual execution result and the expected execution result of the test variable. In one possible implementation, the compilation unit is configured to, when the target configuration file is identical to the configuration file of at least one executed test case, copy the compiled target test project to the folder where the target test case is located, replace the test code file in the compiled test project with the target test code file, and use a preset incremental compilation method to compile only the target test code file to generate the executable file corresponding to the target test case, wherein the compiled target test project is the test project corresponding to the test case to which the configuration file identical to the target configuration file belongs. In one possible implementation, the test case management unit is used to compare the length of the actual execution result of the test variable with the length of the expected execution result; In cases where the lengths are inconsistent, the test result for the test variable is determined to be a test error; If the lengths are the same, compare each element in the actual execution result and the expected execution result one by one. If any element is different, stop the comparison operation and determine that the test result for the test variable is a test error. If all elements are the same, determine that the test result for the test variable is a test correct. In one possible implementation, the execution unit is configured to perform at least one of the following actions on the vehicle controller: Memory erasure, code burning, boot control, debug settings; The memory erasure is used to erase historical data in the preset storage space, so that the preset storage space is in a writable state; The code burning is used to write the compiled program or firmware into the non-volatile memory of the vehicle controller, so that it can run as expected according to the latest test cases; The startup control is used to control the process of the vehicle controller from reset to system initialization, so that the code under test is correctly loaded and executed; The debug settings include at least one of the following: capturing variable states, capturing function call stacks, assisting in analyzing the cause of exceptions, pausing program execution, and checking code logic and register changes line by line. In one possible implementation, the system further includes: The communication unit is configured to receive request messages sent by the test case management unit and send the request messages to the vehicle controller, receive actual response messages sent by the vehicle controller, and feed back the actual response messages to the test case management unit. The test case management unit is used to perform statistics on the actual response messages according to the message statistics algorithm corresponding to the expected message results, obtain the actual message results, and determine the test results of the vehicle controller in terms of message transmission and reception performance by comparing the expected message results with the actual message results. In one possible implementation, when the expected message result includes an expected transmission period, the message statistics algorithm corresponding to the expected message result includes calculating the difference between the timestamps carried in two adjacent actual response messages as the actual message result; and / or, When the expected message result includes expected message content, the message statistics algorithm corresponding to the expected message result includes reading the content of the actual response message and using the content of the actual response message as the actual message result; and / or, When the expected message result includes the expected number of responses, the message statistics algorithm corresponding to the expected message result includes counting the number of actual response messages as the actual message result. In one possible implementation, the communication unit is further configured to test whether the vehicle controller's transmission and reception of actual messages has changed by sending multiple irrelevant messages to the vehicle controller. In one possible implementation, the execution unit includes a power management module; The power management module is used to first power off the vehicle controller and then power it back on after the execution unit receives the power-on / off test command issued by the test case management unit. The test case management unit is used to test, through the communication unit, whether the vehicle controller's actual message transmission and reception has changed after the vehicle controller is powered on again. In one possible implementation, the power management module is also used to inject faults into the data transmission bus; The test case management unit is used to test whether the vehicle controller's actual message transmission and reception has changed after a bus fault is introduced, through the communication unit. In one possible implementation, the system further includes: The log unit is used to record the execution process and results of other units in the system in the form of log files, and to save the communication file of the messages in the case of test cases involving message sending and receiving in the other units. The communication file is used to store test records. In one possible implementation, the system further includes: The test report management unit is used to generate and output test reports by statistically analyzing the test results of each test case. Secondly, embodiments of this application provide an electronic device, which includes the system described in any embodiment of the first aspect. As described above, the automated testing system for vehicle-mounted software provided in this application embodiment can first have the test case management unit determine the target test cases to be executed, and then the compilation unit compiles the target test code file and the source code files involved in the target test code according to the target configuration file in the target test case to obtain the executable file corresponding to the target test case. After establishing a connection with the vehicle controller where the vehicle-mounted software is installed, the execution unit executes the executable file and directly and quickly reads the actual execution results of the test variables from the test array pre-established by the vehicle controller. The test case management unit then compares the actual execution results of the test variables with the expected execution results to determine the test results for the test variables. Therefore, the testing system provided in this application embodiment can not only achieve automated testing of vehicle-mounted software without relying on manual intervention, thereby improving testing efficiency, but also, by pre-storing the actual execution results of the test variables in the test array, allow the execution unit to quickly read the actual execution results directly from the test array without spending a lot of time searching for the actual execution results from all the execution results of the executable file. This achieves rapid testing of each test variable, making the testing granularity more detailed and improving the test coverage and completeness. Furthermore, the technical effects that can be achieved by the application embodiments include: 1. In this embodiment of the application, the source code files in the test cases corresponding to the same configuration file are compiled only once, while the test code files corresponding to the same configuration file are compiled incrementally, thereby greatly reducing the repeated compilation of source code files and improving compilation efficiency. 2. Compared with directly and blindly comparing each actual execution result with the expected execution result element by element, the embodiments of this application can quickly draw the conclusion that the two execution results are different by comparing the length of the execution results, thereby improving the efficiency of result comparison. 3. The communication unit provided by this testing system supports reading message content, quantity, and period, which can be used by the test case management unit for judgment. Since the interval between messages sent by the vehicle controller is generally between tens of milliseconds or even a few milliseconds, using this communication unit allows the test case management unit to judge message attributes without missing any messages, thus overcoming the limitations of manual testing. 4. The communication unit implements a fault injection by sending multiple irrelevant messages to the vehicle controller. Then, by checking whether the sending and receiving of valid messages between the communication unit and the vehicle controller have changed, it is determined whether the fault injection has an impact on the vehicle controller, thereby determining the reliability of the vehicle software. 5. In this embodiment of the application, the power management module is used to power on and off the vehicle controller and inject faults into the data transmission bus to test the impact of power on and off and the data bus on the vehicle controller, thereby determining the reliability of the vehicle software. 6. This testing system records the execution process and results of other units in the system in the form of log files through the log unit. In the case of test cases involving message sending and receiving, it saves the communication files of the messages and generates and outputs test reports through the test report management unit for testers to view. Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort. Figure 1 is an architecture diagram of an automated testing system for vehicle software provided in an embodiment of this application; Figure 2 is a schematic diagram of an automated testing interaction process for vehicle software provided in an embodiment of this application; Figure 3 is an architecture diagram of another automated testing system for vehicle software provided in an embodiment of this application. Detailed Implementation The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. Figure 1 is an architecture diagram of an automated testing system for vehicle-mounted software provided in an embodiment of this application. This system can be applied to electronic devices. Before testing, the electronic device has already installed software for configuration and management, such as Configurator software, compiler, debugging software, communication driver, and the automated testing system. This system can be implemented using programming languages ​​such as Python and Java. The system mainly includes: a test case management unit 110, a compilation unit 120, and an execution unit 130. The test case management unit 110 is used to determine the target test cases to be executed and send the target test cases to the compilation unit 120. The target test cases include target test code files and target configuration files. The compilation unit 120 is used to compile the target test code file and the source code files involved in the target test code according to the target configuration file to obtain the executable file corresponding to the target test case; The execution unit 130 is used to execute the executable file after establishing a connection with the vehicle controller with the installed vehicle software, and read the actual execution results of the test variables from the test array pre-established by the vehicle controller, and feed back the actual execution results to the test case management unit 110. The test array includes the mapping relationship between the test case name, the test variable name and the actual execution result. The values ​​of the test case name and the test variable name are stored before the execution unit 130 executes the executable file, and the values ​​of the actual execution results are added after the execution unit 130 executes the executable file. The test case management unit 110 is used to determine the test result for the test variable by comparing the actual execution result and the expected execution result of the test variable. In this embodiment, the test case management unit 110 can manage the test code, configuration files, test scripts, and other files in each test case, ensuring that the correct files are used when the test cases are executed. It can also filter test cases before each test execution and start execution in a predetermined order. The test case management unit 110 can determine the test case corresponding to the test case name entered by the tester as the target test case to be executed, or it can determine each test case as the target test case to be executed sequentially according to the order of the test cases described in the test sequence. Each test case includes a configuration file and a test code file, so the target test case includes a target test code file and a target configuration file. The configuration file includes the test objective, test environment, preconditions, expected results, etc. The configuration file can be an arxml file. As shown in Figure 2, compilation unit 120 is responsible for preparing the test project and compiling the executable file. First, it starts the Configurator software, opens the target configuration file provided by test case management unit 110, generates configuration code, and updates the configuration file within the test project. Then, following specific compilation rules, it begins multi-threaded compilation of the source files involved in the target test code file via command-line invocation, and links intermediate files to form a binary executable file. The compilation rules may include, but are not limited to, makefiles, cmakelists, etc. If a compilation error occurs, the execution of the test case is interrupted, and the corresponding compilation error information is output to the log file for testers to review and debug. As shown in Figure 2, the execution unit 130 can run asynchronously. The executable file path is passed in. After establishing a connection with the vehicle controller, the execution unit 130 sequentially performs actions such as memory erasure, code burning, startup control, and debug (fault elimination) settings on the vehicle controller, ensuring that the vehicle controller always runs the latest code for the current test case. If any of the above actions fails, the execution of the test case is interrupted, and the corresponding error code is written to the runtime log file. The memory erasure function is used to erase historical data in the preset storage space, making the preset storage space writable. This is a prerequisite for writing the code under test and avoids program conflicts or logical errors caused by residual data. The preset storage space includes Flash, EEPROM, etc. Code burning is used to write the compiled program or firmware into the non-volatile memory of the vehicle controller, enabling it to run as expected according to the latest test cases. Startup control is used to control the process from reset to system initialization of the vehicle controller, so that the code under test is correctly loaded and executed. Debug settings include at least one of the following: (real-time) capture of variable states, (real-time) capture of function call stacks, assistance in analyzing the cause of exceptions, pausing program execution, and line-by-line inspection of code logic and register changes. After the executable file has run for a specified period, the test case management unit 110 sends an instruction to the execution unit 130 to read the test variable. The execution unit 130 reads the actual execution result of the test variable from the vehicle software and feeds it back to the test case management unit 110 for further evaluation of the test result. Specifically, the execution unit 130 reads the map file generated during the code compilation process to obtain information such as the distribution of code and data in memory, as well as their location and size in the executable file. It then selects the memory address of a specific test variable, reads the value of that test variable (i.e., the actual execution result) from the vehicle controller based on that address, and feeds it back to the test case management unit, thus completing the assertion between the actual and expected results. Of course, if the debugger supports reading values ​​directly by variable name, the execution unit 130 can omit the map file parsing process, directly read the variable value, and feed it back to the test case management unit. Specifically, when the actual execution result of the test variable is the same as the expected execution result, it is determined that the vehicle software has no problem in calculating the test variable. When the actual execution result of the test variable is different from the expected execution result, it is determined that there is a problem in the vehicle software in calculating the test variable. The test result can be saved in the form of a log for testers to view. Alternatively, one can start with the test code in the vehicle controller, import the Test_Common.c file, and write key variables into the test array (such as the TestResult array) by calling the TestResult_Set_8BitData, TestResult_Set_16BitData, TestResult_Set_32BitData interfaces inside the file. Finally, the array value is obtained through the execution unit 130. Execution unit 130 can also record the execution time of each test case. For example, relevant test code can be added to the test code file to record the start and end times of code execution. After subtracting the two times, the TestResult_Set_32BitData interface in Test_Common.c is called to write the result to the TestResult array. Then, execution unit 130 reads the time from the address of the TestResult variable and records it in the log file. The automated testing system for vehicle-mounted software provided in this application embodiment first allows the test case management unit to determine the target test cases to be executed. Then, the compilation unit compiles the target test code file and the source code files involved in the target test code according to the target configuration file in the target test case, obtaining the executable file corresponding to the target test case. After establishing a connection with the vehicle controller where the vehicle-mounted software is installed, the execution unit executes the executable file, directly and quickly reading the actual execution results of the test variables from a pre-established test array on the vehicle controller. The test case management unit then compares the actual execution results of the test variables with the expected execution results to determine the test results for the test variables. Therefore, the testing system provided in this application embodiment not only achieves automated testing of vehicle-mounted software without relying on manual intervention, thereby improving testing efficiency, but also allows the execution unit to quickly read the actual execution results directly from the test array by pre-storing the actual execution results of the test variables, without spending a lot of time searching for the actual execution results from all the execution results of the executable file. This enables rapid testing of each test variable, resulting in more granular testing and improved test coverage and completeness. In one possible implementation, when a large number of test cases are executed, compiling all source code files for each test case can easily lead to repetitive compilation and long processing times, which will inevitably hinder the implementation of subsequent automated testing processes. Therefore, to improve compilation efficiency, the compilation unit 120 in this embodiment of the application has been improved as follows: The compilation unit 120 is used to copy the compiled target test project to the folder where the target test case is located when the target configuration file is the same as the configuration file of at least one executed test case, replace the test code file in the compiled test project with the target test code file, and use a preset incremental compilation method to compile only the target test code file to generate the executable file corresponding to the target test case. The compiled target test project is the test project corresponding to the test case of the configuration file that is the same as the target configuration file. To improve the efficiency of configuration file comparison, the hash values ​​of the two configuration files can be directly calculated to determine if they are identical. Specifically, the hash value of the target configuration file and the hash value of the configuration file for each executed test case are calculated. When a configuration file with the same hash value as the target configuration file is found, it is determined that the configuration file is identical to the target configuration file. The hash value calculation algorithms include, but are not limited to, SHA256 and MD5. In addition, the default incremental compilation method can be the incremental compilation method in the makefile. Actual testing revealed that the compilation time for a test project ranged from 30 to 120 seconds. However, by using the current compilation unit 120 method, the compilation time for test cases with the same configuration was only 3 seconds, resulting in an efficiency improvement of over 90%. This application embodiment greatly reduces the repeated compilation of source code files and improves compilation efficiency by compiling the source code files in the test cases corresponding to the same configuration file only once, while using incremental compilation for the test code files corresponding to the same configuration file. In one possible implementation, the test case management unit 110 is used to compare the length of the actual execution result of the test variable with the expected execution result. In cases where the lengths are inconsistent, the test result for the test variable is determined to be a test error; If the lengths are the same, compare each element in the actual execution result with the expected execution result one by one. If any element is different, stop the comparison operation and determine that the test result for the test variable is a test error. If all elements are the same, determine that the test result for the test variable is a test correct. Compared to directly and blindly comparing each actual execution result with the expected execution result element by element, the embodiments of this application can quickly draw the conclusion that the two execution results are different by comparing the length of the execution results, thereby improving the efficiency of result comparison. In one possible implementation, as shown in Figures 2 and 3, the system may further include: The communication unit 140 is used to receive request messages sent by the test case management unit 110 and send the request messages to the vehicle controller, receive actual response messages sent by the vehicle controller, and feed back the actual response messages to the test case management unit 110. The test case management unit 110 is used to statistically analyze the actual response messages according to the message statistical algorithm corresponding to the expected message result, obtain the actual message result, and determine the test result of the vehicle controller in terms of message transmission and reception performance by comparing the expected message result with the actual message result. In this embodiment of the application, when the expected message result includes the expected sending period, the message statistics algorithm corresponding to the expected message result includes calculating the difference between the timestamps carried in two adjacent actual response messages as the actual message result; and / or, When the expected message result includes the expected message content, the message statistics algorithm corresponding to the expected message result includes reading the content of the actual response message and using the content of the actual response message as the actual message result; and / or, When the expected result of a message includes the expected number of responses, the message statistics algorithm corresponding to the expected result of the message includes counting the actual number of response messages as the actual result of the message. In the automated testing system, the communication unit 140 must ensure that the execution unit 130 has started correctly and is running continuously when it starts. It can send and receive network messages while the onboard software is running, and can use various network protocols such as CAN (Controller Area Network), CANFD (CAN with Flexible Data Rate), LIN (Local Interconnect Network), and Ethernet. The communication unit 140 supports reading message content, quantity, and cycle time for the test case management unit 110 to make judgments. Since the interval between messages sent by the vehicle controller is generally between tens of milliseconds or even a few milliseconds, the application of this communication unit 140 allows the test case management unit 110 to judge the message attributes without missing any messages, thus making up for the limitations of manual testing. In one possible implementation, the communication unit 140 is also used to test whether the vehicle controller’s transmission and reception of actual messages has changed by sending multiple irrelevant messages to the vehicle controller. Specifically, the communication unit 140 implements a fault injection by sending multiple irrelevant messages to the vehicle controller. Then, by checking whether the sending and receiving of valid messages between the communication unit 140 and the vehicle controller have changed, it determines whether the fault injection has an impact on the vehicle controller, thereby determining the reliability of the vehicle software. In one possible implementation, as shown in FIG3, the execution unit 130 may include a power management module 131; The power management module 131 is used to first power off the vehicle controller and then power it back on after the execution unit 130 receives the power-on / off test command issued by the test case management unit 110. The test case management unit 110 is used to test whether the vehicle controller’s actual message transmission and reception has changed after the vehicle controller is powered on again via the communication unit 140. In this embodiment of the application, the power management module 131 is used to power on and off the vehicle controller to test the impact of power-on and power-off on the vehicle controller, thereby determining the reliability of the vehicle software. In addition, the power management module 131 can also be used to inject faults into the data transmission bus; The test case management unit 110 is used to test whether the vehicle controller’s actual message transmission and reception changes after a bus fault is introduced, through the communication unit 140. For example, the power supply principle module can control CAN_H (i.e., high voltage line) and CAN_L (i.e., low voltage line) in the CAN bus to perform short circuit, open circuit and other operations on the CAN bus, so as to test the impact of bus faults on the vehicle controller.

[0096] In one possible implementation, as shown in Figure 3, the system may further include: Log unit 150 is used to record the execution process and results of other units in the system in the form of a log file. Log unit 150 can be nested within other units. It can identify test case number, test case name, file path, compilation result, execution information, and network packets. It can also lock and execute across multiple processes and output the identified information to the log file in sequence, thus achieving automatic saving of test records. Specifically, the log unit 150 can save the execution log information of the entire automated testing system as a log file. If test cases involve message sending and receiving, it will also save the message communication files, such as .asc files, .blf files, .pcap files, etc., to retain test records for testers to review. In one possible implementation, as shown in Figure 3, the system may further include: The test report management unit 160 is used to generate and output test reports by statistically analyzing the test results of each test case. The test report management unit 160 can complete tasks such as collecting assertion results in test scripts, statistically analyzing execution information, and generating test reports. Test result statistics are performed throughout the entire test case execution process. If all the above steps are executed successfully, information such as the test case number, test case name, and execution time is written into the test report. If any error occurs, the execution of the test case can be interrupted, and the test case execution will be marked as failed. When multiple test cases need to be executed, the above steps can be repeated continuously. It should be noted that regardless of whether the current test case passes or fails, the next test case can be executed. After all test cases have been executed, all test results are statistically analyzed, and a test report is generated according to the testers, test descriptions, and other information passed in when the system was started. It should be added that the automated testing system for in-vehicle software can test any module in the Autosar Classic Platform architecture. Besides diagnostics and network management modules that require communication, the Autosar Classic Platform architecture also contains many management modules (such as BswM, EcuM, and ComM), intermediate modules (such as CRC, PduR, and SoAd), etc. Testing these modules no longer requires sending and receiving network packets. For such tests, this application embodiment can freely choose whether to use the communication unit 140, thereby greatly increasing the scope of automated testing applicable to this application embodiment, making it suitable for automated testing of every module under the Autosar Classic Platform architecture. The following section uses the BswM module of the system service layer under the Autosar Classic Platform architecture as an example to illustrate the above testing process: [1] BswM is short for Basic Software Mode Manager, a module in the system service layer of the Autosar Classic Platform architecture. Its responsibility is to arbitrate mode requests from the application layer SW-C or other BSW modules according to simple rules, and to execute operations based on the arbitration results. Simply put, its function is arbitration and execution. It does not have the function of sending and receiving messages itself. [2] Before the test begins, ensure that the automated test system contains the test scripts, test code and test configuration files for the relevant test cases of the module. [3] The automated testing system has testing functions for all functional modules under the Autosar Classic Platform architecture. Testers can filter the BswM module and the test cases under the module according to actual needs, and select all or part of the test cases to be executed in this test. [4] The test case management unit can automatically find the test code file and configuration file of the test case saved in the specified folder according to the name information of the test case, and send this information to the compilation unit. [5] Based on the above information, the compilation unit obtains the test code file and configuration file for the test case, and copies and replaces the test code into the test project. Then, it calls the third-party interface of the Configurator toolchain to open the configuration file and generate the configuration code into the test project. Thus, based on the test requirements and test configuration of this test case, the two together form a test project. [Note] This test case assumes that the BswM module will be triggered to arbitrate when the current arbitration status is NO_COMMUNICATION. In the test code, you only need to call the BswM_CanSM_CurrentState() interface of the BswM module and then observe the arbitration status. [6] The compilation unit writes the relevant compilation rules and configurations, starts multi-threaded compilation by calling the command line, generates an executable file that can run on the vehicle controller, and obtains the result after compilation. If a compilation error occurs, the execution of the test case is interrupted, and the corresponding compilation error information is output to the log file for testers to review and debug. [7] The execution unit burns the executable file generated above into the vehicle controller through a series of operations such as erasing memory, burning code, and running code, and then reports the code running status to the test case management unit. [8] The test also needs to compare the values ​​of the TestResult array. After the code runs for a specified time, the test case management unit will send a signal to the execution unit. The execution unit will read the actual execution result of the variable from the vehicle software and feed it back to the test case management unit so that it can determine the test result for the test variable by comparing the actual execution result of the test variable with the expected execution result. [9] After the test is completed, the test report management unit will collect the execution information of each test case and write the test case number, test case name, execution time and other information into the test report to complete the fully automated test process.

[0010] To make it easier to understand, here's a simple test case. Init: Make Begin: Run assert TestResultCompare==0 End

[0011] The test case management unit's test case distribution, compilation unit test code copying, and configuration code generation are all implicitly included in the Init operation, minimizing repetitive work and improving efficiency. After the initialization operation is completed, the test project is ready according to the test requirements and configuration files of different test cases. This test project already includes the previously mentioned call to the BswM_CanSM_CurrentState() interface. The Make statement calls the compilation unit to compile the test project and obtain the executable file. After running for a specified time, the execution unit will obtain the value of TestResultCompare, which will be judged by the test case management unit to see if it matches the expected execution result. Finally, the test report management unit collects test case execution information, outputs logs to the log file, and writes information such as test case number, test case name, and execution time into the test report.

[0012] It is worth noting that this test script can be used for all module test cases that do not require sending or receiving messages, which greatly reduces the workload of script development and lowers the difficulty of using this automated testing system. Furthermore, when testing the Diagnostic Communication Manager (DCM) module under the AUTOSAR Classic Platform architecture, the communication unit is also required. The test case management unit sends message information to the communication unit according to actual testing needs, completing the transmission of relevant CAN messages. After receiving the response message, the communication unit feeds it back to the test case management unit for further judgment on attributes such as message content, period, and quantity. In the AUTOSAR Classic Platform architecture, the DCM module, short for Diagnostic Communication Manager, is a module used for diagnostic communication management, located in the communication service layer. Its main function is to manage diagnostic data flow and diagnostic status, including diagnostic sessions, security status, and diagnostic service allocation. For example, when testing the DCM module, the test cases used may include: Init: Make Begin: Run Send 1003 assert received 5003 assert TestResultCompare==0 End The aforementioned testing system can also be applied to the transmission and reception of LIN network protocol messages. The communication unit can automatically convert the transmitted and received messages into Header and Response formats according to the LIN network protocol's communication method, perform LIN message transmission and reception, and evaluate the message content and subsequent TestResultCompare values. For example, when testing the communication methods of the LIN network protocol, the test cases used may include: Init: Make Begin: Run Send 3C Header Send 3C Response 1003 Send 3D Header assertreceived3DResponse 5003 assert TestResultCompare==0 End The aforementioned testing system can also be applied to Ethernet network protocol for message transmission and reception. The communication unit can automatically transmit and receive Ethernet messages according to the Ethernet protocol's communication method, and evaluate the message content and subsequent TestResultCompare values. This implementation uses the same message transmission and reception interface as the DCM module because the network protocol is encapsulated in the communication unit, making the external interfaces exposed for CAN communication and Ethernet communication identical, minimizing user costs and simplifying subsequent maintenance steps. For example, when testing the communication methods of the Ethernet network protocol, the test cases used may include: Init: Make Begin: Run Send 1003 assert received 5003 assert TestResultCompare==0 Based on the above method embodiments, another embodiment of this application provides an electronic device, which includes the automated testing system for vehicle software described in any of the above embodiments. The above-described virtual device embodiments and physical device embodiments correspond to the method embodiments and have the same technical effects. For detailed descriptions, please refer to the method embodiments. The device embodiments are derived based on the method embodiments; detailed descriptions can be found in the method embodiments section, and will not be repeated here. Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An automated testing system for vehicle-mounted software, characterized in that, The system includes: a test case management unit, a compilation unit, and an execution unit; The test case management unit is used to determine the target test cases to be executed and send the target test cases to the compilation unit, wherein the target test cases include target test code files and target configuration files; The compilation unit is used to compile the target test code file and the source code files involved in the target test code according to the target configuration file to obtain the executable file corresponding to the target test case; The execution unit is configured to execute the executable file after establishing a connection with the vehicle controller that has the vehicle software installed, and read the actual execution results of the test variables from the test array pre-established by the vehicle controller, and feed back the actual execution results to the test case management unit. The test array includes a mapping relationship between test case names, test variable names and actual execution results, and stores the values ​​of test case names and test variable names before the execution unit executes the executable file, and adds the values ​​of the actual execution results after the execution unit executes the executable file. The test case management unit is used to determine the test result for the test variable by comparing the actual execution result and the expected execution result of the test variable.

2. The system according to claim 1, characterized in that, The compilation unit is configured to, when the target configuration file is identical to the configuration file of at least one executed test case, copy the compiled target test project to the folder where the target test case is located, replace the test code file in the compiled test project with the target test code file, and use a preset incremental compilation method to compile only the target test code file to generate the executable file corresponding to the target test case, wherein the compiled target test project is the test project corresponding to the test case to which the configuration file identical to the target configuration file belongs.

3. The system according to claim 1, characterized in that, The test case management unit is used to compare the length of the actual execution result and the expected execution result of the test variable; In cases where the lengths are inconsistent, the test result for the test variable is determined to be a test error; If the lengths are the same, compare each element in the actual execution result and the expected execution result one by one. If any element is different, stop the comparison operation and determine that the test result for the test variable is a test error. If all elements are the same, determine that the test result for the test variable is a test correct.

4. The system according to claim 1, characterized in that, The execution unit is configured to perform at least one of the following actions on the vehicle controller: Memory erasure, code burning, boot control, debug settings; The memory erasure is used to erase historical data in the preset storage space, so that the preset storage space is in a writable state; The code burning is used to write the compiled program or firmware into the non-volatile memory of the vehicle controller, so that it can run as expected according to the latest test cases; The startup control is used to control the process of the vehicle controller from reset to system initialization, so that the code under test is correctly loaded and executed; The debug settings include at least one of the following: capturing variable states, capturing function call stacks, assisting in analyzing the cause of exceptions, pausing program execution, and checking code logic and register changes line by line.

5. The system according to claim 1, characterized in that, The system also includes: The communication unit is configured to receive request messages sent by the test case management unit and send the request messages to the vehicle controller, receive actual response messages sent by the vehicle controller, and feed back the actual response messages to the test case management unit. The test case management unit is used to perform statistics on the actual response messages according to the message statistics algorithm corresponding to the expected message results, obtain the actual message results, and determine the test results of the vehicle controller in terms of message transmission and reception performance by comparing the expected message results with the actual message results.

6. The system according to claim 5, characterized in that, When the expected message result includes the expected sending period, the message statistics algorithm corresponding to the expected message result includes calculating the difference between the timestamps carried in two adjacent actual response messages as the actual message result; And / or, When the expected message result includes the expected message content, the message statistics algorithm corresponding to the expected message result includes reading the content of the actual response message and using the content of the actual response message as the actual message result. And / or, When the expected message result includes the expected number of responses, the message statistics algorithm corresponding to the expected message result includes counting the number of actual response messages as the actual message result.

7. The system according to claim 5, characterized in that, The execution unit includes a power management module; The power management module is used to first power off the vehicle controller and then power it back on after the execution unit receives the power-on / off test command issued by the test case management unit. The test case management unit is used to test, through the communication unit, whether the vehicle controller's actual message transmission and reception has changed after the vehicle controller is powered on again.

8. The system according to claim 7, characterized in that, The power management module is also used to inject faults into the data transmission bus; The test case management unit is used to test whether the vehicle controller's actual message transmission and reception has changed after a bus fault is introduced, through the communication unit.

9. The system according to claim 1, characterized in that, The system also includes: The log unit is used to record the execution process and results of other units in the system in the form of log files, and to save the communication files of the messages in the case of test cases involving message sending and receiving in the other units. The communication files are used to store test records.

10. The system according to any one of claims 1-9, characterized in that, The system also includes: The test report management unit is used to generate and output test reports by statistically analyzing the test results of each test case.

Images