Automatic vehicle testing method and device, electronic equipment and storage medium

Abstract

The invention provides a vehicle automatic test method and device, electronic equipment and a storage medium, and the method comprises the steps: a test piece periodically sends or receives a load message according to a routing relation, and records a receiving and transmitting record corresponding to a message identifier based on the message identifier of the sent or received load message, obtaining performance test results corresponding to different message identifiers based on the receiving and transmitting records corresponding to each message identifier, uploading the performance test results to an upper computer, receiving a first diagnosis message corresponding to a test case sent by the upper computer, and converting the first diagnosis message into a test message corresponding to the test case, the test message is sent to the tested piece, so that the tested piece executes a test case based on the test message and returns a response message, the response message is converted into a second diagnosis message, the second diagnosis message is sent to the upper computer, and a case test result is obtained; through the method, the technical problems of high cost and low efficiency of testing the electronic controller in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of automated testing technology, and in particular to a vehicle automated testing method, device, electronic equipment and storage medium. Background Art

[0002] CAN (Controller Area Network) is a communication protocol for vehicle electronic control systems. It is a serial communication protocol based on message passing, mainly used for real-time transmission of data and control signals. ECU (Electronic Control Unit) is the core component in the automotive electronic system, responsible for performing specific functions such as engine control, brake system control, etc. ECU usually contains one or more microcontrollers, as well as input and output interfaces connected to sensors and actuators.

[0003] In related technologies, testing the vehicle's ECU usually requires the use of professional software such as CANOE and vehicle spy to capture CAN messages and then perform performance and functional tests on them, but these devices are very expensive, ranging in price from hundreds of thousands of yuan. In addition, the functional testing of each module usually needs to be done manually, and after completing the functional testing, manual statistics and report filling are required, which is a complicated process and prone to errors. In addition, since the meaning of the signal value of the CAN message in the CAN protocol needs to be viewed with the help of professional software and definition tables, the test efficiency is low. Summary of the invention

[0004] In view of the shortcomings of the related technologies mentioned above, the present application provides a vehicle automated testing method, device, electronic device and storage medium to solve the technical problems of high cost and low efficiency in testing vehicle electronic controllers in the prior art.

[0005] The present application provides a vehicle automation testing method, which includes: the test piece periodically sends or receives a load message according to a routing relationship, and each time the test piece sends or receives the load message, the sending and receiving record corresponding to the message identifier of the sent or received load message is recorded based on the message identifier of the load message; performance test results corresponding to different message identifiers are obtained based on the sending and receiving records corresponding to each message identifier, and the performance test results are uploaded to a host computer; a first diagnostic message corresponding to a test case sent by the host computer is received, the first diagnostic message is converted into a test message corresponding to the test case, and the test message is sent to the device under test, so that the device under test executes the test case based on the test message and returns a response message; the response message is converted into a second diagnostic message, and the second diagnostic message is uploaded to the host computer, so that the host computer obtains the actual result of the test case based on the second diagnostic message, and the actual result of the test case is compared with the preset expected result corresponding to the test case to obtain the case test result of the test case.

[0006] In one embodiment of the present application, the test piece periodically sends or receives load messages according to the routing relationship, including: the test piece receives a database file sent by the host computer, and parses the database file to generate the routing relationship with the device under test, the test piece is a vehicle electronic controller with message sending and receiving functions and Ethernet communication functions, the device under test is other electronic controllers connected to the test piece, and the host computer communicates with the test piece through the Ethernet communication function; the test piece periodically sends the load message to the device under test based on the routing relationship, and the device under test forwards the load message to the test piece after receiving the load message.

[0007] In one embodiment of the present application, each time the test piece sends or receives the load message, the sending and receiving record corresponding to the message identifier is recorded based on the message identifier of the sent or received load message, including: each time the test piece sends the load message, a sending timestamp is added to the message identifier corresponding to the sent load message and the number of sending times is increased, and the sending and receiving record includes the sending timestamp and the number of sending times; each time the test piece receives a load message, a receiving timestamp is added to the message identifier corresponding to the received load message and the number of receiving times is increased, and the sending and receiving record includes the receiving timestamp and the number of receiving times.

[0008] In one embodiment of the present application, performance test results corresponding to different message identifiers are obtained based on the sending and receiving records corresponding to each message identifier and the performance test results are uploaded to the host computer, including: for load messages with the same message identifier, the packet loss rate corresponding to the message identifier is calculated based on the number of sending times, the number of receiving times and the period adjustment coefficient, the period adjustment coefficient is obtained based on the receiving period and the sending period, the receiving period is obtained based on the sending and receiving records, the sending period is obtained based on the routing relationship, and the performance test results include the packet loss rate and delay; for the same load message, the delay of the load message is calculated based on the receiving timestamp and the sending timestamp of the load message; the maximum delay and the average delay corresponding to the message identifier are obtained according to the delays of all load messages with the same message identifier; the packet loss rate, the maximum delay and the average delay corresponding to each message identifier are uploaded to the host computer as performance test results.

[0009] In one embodiment of the present application, the test piece periodically sends or receives load messages according to the routing relationship, and further includes: receiving the load message frequency sent from the upper computer, and adjusting the sending period of the load message based on the load message frequency to adjust the load of the test piece; and recording the sending and receiving records under different loads of the test piece to obtain the performance test results under different loads.

[0010] In one embodiment of the present application, after receiving the first diagnostic message corresponding to the test case sent by the host computer, it includes: the host computer captures all diagnostic messages sent and received by the Ethernet interface, and if the test result of the test case is a test failure, all the diagnostic messages during the execution of the test case are saved and recorded in the test log; if the test result of the test case is a success, the diagnostic messages during the execution of the test case are not saved.

[0011] In one embodiment of the present application, the actual result of the test case is compared with the preset expected result corresponding to the test case to obtain the case test result of the test case, including: if the actual result is disconnection, the case test result is a test failure and the failure level of the case test result is a smoke level; if the actual result is inconsistent with the preset expected result, the case test result is a test failure and the failure level of the case test result is an intermediate level; if the actual result is other errors beyond expectations, the case test result is a test failure and the failure level of the case test result is an advanced level; the host computer establishes a test visualization panel, and the test visualization panel includes a test result display section and a test configuration section; the performance detection results and the case test results of different failure levels are visualized through the test result display section; and different database files and test cases are configured through the test configuration panel.

[0012] An embodiment of the present application also provides a vehicle automation test device, which includes: a message recording module, which is used for the test piece to periodically send or receive load messages according to the routing relationship, and each time the test piece sends or receives the load message, the sending and receiving records corresponding to the message identifier of the sent or received load message are recorded based on the message identifier of the load message; a performance testing module, which is used to obtain performance test results corresponding to different message identifiers based on the sending and receiving records corresponding to each message identifier and upload the performance test results to a host computer; a use case receiving module, which is used to receive a first diagnostic message corresponding to the test case sent by the host computer, convert the first diagnostic message into a test message corresponding to the test case, and send the test message to the device under test, so that the device under test executes the test case based on the test message and returns a response message; a use case testing module, which is used to convert the response message into a second diagnostic message, and upload the second diagnostic message to the host computer, so that the host computer obtains the actual result of the test case based on the second diagnostic message, and compares the actual result of the test case with the preset expected result corresponding to the test case to obtain the use case test result of the test case.

[0013] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the vehicle automation testing method as described in any of the above embodiments.

[0014] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor of a computer, the computer is enabled to execute the vehicle automation testing method as described in any one of the above embodiments.

[0015] Beneficial effects of the present application: The embodiments of the present application provide a vehicle automation testing method, device, electronic device and storage medium. The method periodically sends or receives load messages according to the routing relationship through a test piece, records the sending and receiving records corresponding to the message identifier based on the message identifier of the sent or received load message, obtains performance test results corresponding to different message identifiers based on the sending and receiving records corresponding to each message identifier and uploads the performance test results to a host computer, receives a first diagnostic message corresponding to a test case sent by the host computer, converts the first diagnostic message into a test message corresponding to the test case, and sends the test message to the device under test, so that the device under test executes the test case based on the test message and returns a response message, converts the response message into a second diagnostic message, sends the second diagnostic message to the host computer, obtains the test case test result, and uses a low-cost electronic controller as a test piece instead of expensive professional equipment, thereby greatly saving hardware costs. By automatically performing performance tests and function tests on vehicle electronic controllers, the efficiency of test automation is improved and labor costs are reduced.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic diagram of an implementation environment of a vehicle automated testing method shown in an exemplary embodiment of the present application;

[0018] Figure 2 is a flow chart of a vehicle automated testing method shown in an exemplary embodiment of the present application;

[0019] Figure 3 is a testware software architecture diagram shown in an exemplary embodiment of the present application;

[0020] Figure 4 is a diagram of the upper computer software architecture shown in an exemplary embodiment of the present application;

[0021] Figure 5 is a vehicle automation performance test flow chart shown in an exemplary embodiment of the present application;

[0022] Figure 6 is a vehicle automation use case test flow chart shown in an exemplary embodiment of the present application;

[0023] Figure 7is a schematic diagram of a visualization interface shown in an exemplary embodiment of the present application;

[0024] Figure 8 is a block diagram of a vehicle automated testing device shown in an exemplary embodiment of the present application;

[0025] Fig. 9 It is a structural schematic diagram of an electronic device shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0026] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0027] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0028] It should be noted that in this application, "first", "second", etc. are only used to distinguish similar objects, and are not used to limit the order or precedence of similar objects. The variations of "including", "having", etc. described above indicate that the scope covered by the subject of the word is not exclusive except for the examples shown by the word.

[0029] It is understood that the various numbers, step numbers, and other reference numerals recorded in this application are distinguished for the convenience of description and are not intended to limit the scope of this application. The size of the reference numerals in this application does not mean the order of execution. The execution order of each process should be determined by its function and internal logic.

[0030] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0031] First of all, it should be noted that CAN (Controller Area Network) is a communication protocol for vehicle electronic control systems. It is a serial communication protocol based on message passing, mainly used for real-time transmission of data and control signals. ECU (Electronic Control Unit) is a core component in automotive electronic systems, responsible for performing specific functions such as engine control, brake system control, etc. ECU usually contains one or more microcontrollers, as well as input and output interfaces connected to sensors and actuators.

[0032] The embodiments of the present application respectively propose a vehicle automation testing method, a vehicle automation testing device, an electronic device, a computer-readable storage medium, and a computer program product, which will be described in detail below.

[0033] See also Figure 1 , Figure 1 It is a schematic diagram of an implementation environment of a vehicle automation testing method shown in an exemplary embodiment of the present application.

[0034] like Figure 1 As shown, the implementation environment may include a host computer, a test piece ECU and a device under test ECU (dut), wherein the host computer is a computer that can directly issue control commands, and may be a computer, a mobile phone, a tablet, a touch screen, etc. The test piece ECU is a microcontroller unit that supports CAN message sending and receiving and Ethernet communication functions, and the device under test ECU is an electronic controller connected to the test piece ECU for executing test cases.

[0035] In one embodiment of the present application, the host computer can be connected to the test device ECU via a 100M Ethernet 100BASE-TX, and the test device ECU is connected to the device under test ECU via CAN.

[0036] See also Figure 2 , Figure 2 is a flow chart of a vehicle automated testing method shown in an exemplary embodiment of the present application. The method can be applied to Figure 1 The implementation environment shown is as follows. The method may also be applicable to other exemplary implementation environments and be specifically executed by devices in other implementation environments. This embodiment does not limit the implementation environment to which the method is applicable.

[0037] like Figure 2 As shown, in an exemplary embodiment, the vehicle automated testing method includes at least steps S210 to S240, which are described in detail as follows:

[0038] Step S210: The test piece periodically sends or receives a load message according to the routing relationship. Each time the test piece sends or receives a load message, the test piece records a sending and receiving record corresponding to the message identifier based on the message identifier of the sent or received load message.

[0039] In one embodiment of the present application, the test piece periodically sends or receives load messages according to the routing relationship, including: the test piece receives a database file sent by a host computer, and parses the database file to generate a routing relationship with the device under test, the test piece is a vehicle electronic controller with message sending and receiving functions and Ethernet communication functions, the device under test is other electronic controllers connected to the test piece, and the host computer communicates with the test piece through the Ethernet communication function; the test piece periodically sends load messages to the device under test based on the routing relationship, and the device under test forwards the load messages to the test piece after receiving the load messages.

[0040] In one embodiment of the present application, a routing relationship is generated based on a database file of a host computer, and the routing relationship records relevant routing information of CAN message sending and receiving between the test piece and the device under test. The routing information includes a sending period corresponding to each message identifier, so that the test piece sends a load message corresponding to the message identifier based on a preset sending period.

[0041] In one embodiment of the present application, each time the test piece sends or receives a load message, a sending and receiving record corresponding to the message identifier is recorded based on the message identifier of the sent or received load message, including: each time the test piece sends a load message, a sending timestamp is added to the message identifier corresponding to the sent load message and the number of sending times is increased, and the sending and receiving record includes the sending timestamp and the number of sending times; each time the test piece receives a load message, a receiving timestamp is added to the message identifier corresponding to the received load message and the number of receiving times is increased, and the sending and receiving record includes the receiving timestamp and the number of receiving times.

[0042] Step S220, obtaining performance test results corresponding to different message identifiers based on the sending and receiving records corresponding to each message identifier and uploading the performance test results to the host computer.

[0043] In one embodiment of the present application, performance test results corresponding to different message identifiers are obtained based on the sending and receiving records corresponding to each message identifier and the performance test results are uploaded to a host computer, including: for load messages with the same message identifier, the packet loss rate corresponding to the message identifier is calculated based on the number of sending times, the number of receiving times and the period adjustment coefficient, the period adjustment coefficient is obtained based on the receiving period and the sending period, the receiving period is obtained based on the sending and receiving records, the sending period is obtained based on the routing relationship, and the performance test results include the packet loss rate and delay; for the same load message, the delay of the load message is calculated based on the receiving timestamp and the sending timestamp of the load message; the maximum delay and the average delay corresponding to the message identifier are obtained according to the delay of all load messages with the same message identifier; the packet loss rate, maximum delay and average delay corresponding to each message identifier are uploaded to the host computer as the performance test results.

[0044] In one embodiment of the present application, when the test piece sends a CAN load message each time, the number of transmissions txCount corresponding to the CAN ID (message identifier) ​​of the load message is increased once, and the corresponding transmission timestamp txTime is added to the message. When the test piece captures a CAN load message each time, the number of receptions rxCount corresponding to the CAN ID (message identifier) ​​of the load message is increased once, and the corresponding reception timestamp rxTime is added to the message. Calculating the packet loss rate corresponding to each message identifier includes: lost = rxCount-txCount*txPeriod / rxPeriod, where lost is the number of packet losses, rxCount is the number of receptions corresponding to the message identifier, txCount is the number of transmissions corresponding to the message identifier, txPeriod is the transmission period, rxPeriod is the receiving period, and the period adjustment coefficient is obtained by the ratio of the transmission period to the receiving period. After adjusting the number of transmissions based on the period adjustment coefficient, the number of packet losses is obtained by subtracting the adjusted number of transmissions from the number of receptions, and then the packet loss rate is obtained based on the number of packet losses. The sending period in this embodiment is obtained based on the routing relationship, and the receiving period can be calculated based on the number of receptions and the receiving timestamp corresponding to the message identifier in the sending and receiving records.

[0045] In one embodiment of the present application, the test piece periodically sends or receives load messages according to the routing relationship, and further includes: receiving the load message frequency sent from the upper computer, and adjusting the load message sending and receiving frequency based on the load message frequency to adjust the load of the test piece; recording the sending and receiving records under different loads of the test piece to obtain performance test results under different loads.

[0046] Step S230, receiving a first diagnostic message corresponding to the test case sent by the host computer, converting the first diagnostic message into a test message corresponding to the test case, and sending the test message to the device under test, so that the device under test executes the test case based on the test message and returns a response message.

[0047] In one embodiment of the present application, the first diagnostic message is a DOIP message. The host computer sends the DOIP message corresponding to the test case of this test to the test piece. The test piece converts the first diagnostic message (DOIP message) into a test message (CAN message) corresponding to the test case, and sends the test message to the device under test, so that the device under test processes the test message of this test case and returns a response message (CAN message). After receiving the response message, the test piece converts the response message into a second diagnostic message (DOIP message) and uploads the second diagnostic message to the host computer.

[0048] In one embodiment of the present application, the host computer includes multiple test cases, and the host computer sends the first diagnostic messages corresponding to the test cases to be tested to the test piece one by one, so that the test piece executes the multiple test cases one by one based on the method described in the above embodiment, and uploads the second diagnostic messages corresponding to each test case to the host computer one by one.

[0049] In one embodiment of the present application, after receiving the first diagnostic message corresponding to the test case sent by the host computer, it includes: the host computer captures all diagnostic messages sent and received by the Ethernet interface, and if the test case test result is a test failure, all diagnostic messages during the execution of the test case are saved and recorded in the test log; if the test case test result is a success, the diagnostic messages during the execution of the test case are not saved.

[0050] In one embodiment of the present application, after the host computer sends the first diagnostic message (DOIP message), when the test case is tested, the host computer captures all diagnostic messages of the Ethernet interface. If the test result of the test case is failure, all diagnostic messages (including at least the first diagnostic message and the second diagnostic message) of the test case during the test are saved, and recorded in the test log, so that the engineer can diagnose the cause of the failure. If the test result of this case is a successful test, the capture of the diagnostic message is stopped and the diagnostic message during the test case test is not saved. Until the next test case starts testing again, continue to repeat the process of capturing and recording the above-mentioned diagnostic message.

[0051] Step S240, convert the response message into a second diagnostic message, and upload the second diagnostic message to the host computer, so that the host computer obtains the actual result of the test case based on the second diagnostic message, and compares the actual result of the test case with the preset expected result corresponding to the test case to obtain the test case test result of the test case.

[0052] In one embodiment of the present application, the actual result of the test case is compared with the preset expected result corresponding to the test case to obtain the use case test result of the test case, including: if the actual result is a disconnection, the use case test result is a test failure and the failure level of the use case test result is a smoke level; if the actual result is inconsistent with the preset expected result, the use case test result is a test failure and the failure level of the use case test result is an intermediate level; if the actual result is other errors beyond expectations, the use case test result is a test failure and the failure level of the use case test result is an advanced level; the host computer establishes a test visualization panel, the test visualization panel includes a test result display section and a test configuration section; the performance test results and use case test results of different failure levels are visualized through the test result display section; different database files and test cases are configured through the test configuration panel.

[0053] In one embodiment of the present application, if a socket.error exception occurs during the execution of a test case, it means that the connection is disconnected, which is a serious error and belongs to the smoke level. In this case, the test case test result of the test case is judged to be a failure, and the failure level is the smoke level, and this exception is captured and assigned to the cell corresponding to the test report as "smoking" and a red background. If an AssertionError exception occurs during the execution of a test case, it means that the assertion fails, that is, the received message value (actual result) does not meet the preset expected result. This usually means that there is a logical error or the data returned by the device is incorrect. Since this modification of the return value is well implemented in the code, it is not a serious error, but is defined as an intermediate level problem. In this case, the test case test result of the test case is judged to be a failure, and the failure level is intermediate, and this exception is captured and assigned to the cell corresponding to the test report as "intermediate" and a green background. If other exceptions other than socket.error and AssertionError occur during the execution of the test case, this may be a high-level error that exceeds the expected range. In this case, the test case test result of the test case is judged to be failure, and the failure level is advanced. This exception is captured and the corresponding cell in the test report is assigned a value of "advanced" and a yellow background.

[0054] See also Figure 3 , Figure 3It is a test piece software architecture diagram shown in an exemplary embodiment of the present application. In one embodiment of the present application, the software in the test piece includes a DBC parsing module, a CAN-DOIP module, a CAN performance test module, a CAN load control module, a CAN transceiver module, a CAN driver module, and an ETH driver module. The CAN driver module and the ETH driver module are used for upper-layer application calls. The DBC parsing module is used to receive and parse the database file (DBC file) passed in by the host computer, and generate a routing relationship based on the parsed DBC matrix. The CAN transceiver module is used to call the CAN driver module to send and receive CAN messages. The CAN performance test module is used to calculate and count the packet loss rate and delay, and the packet loss rate and delay corresponding to each message identifier (CANID) are calculated by recording the message transceiver record of each message identifier (CANID). The CAN load control module is used to control the CAN bus load of each CAN path by sending a load message to test the performance of the tested piece under different CAN bus load rates, and to increase or decrease the bus load of the CAN by adjusting the sending frequency of the load message through the host computer. The CAN-DOIP module is used to realize the mutual conversion between CAN messages and DOIP messages, and calls the CAN driver and ETH driver to send and receive corresponding messages and perform conversion.

[0055] See also Figure 4 , Figure 4 It is a diagram of the upper computer software architecture shown in an exemplary embodiment of the present application. In one embodiment of the present application, the software in the upper computer includes a graphical interface, DBC file parsing, a test case automatic execution module, a log module, a test report automatic generation module and a test progress visual display module. Among them, the upper computer uses python to develop a graphical interface to add buttons to control the test piece to execute the corresponding test content by clicking the corresponding button. The test content includes performance testing and case testing (functional testing). The graphical interface is used to display the CAN message information sent and received, such as CAN id, number of sending and receiving times, message length, message data and message type energy data. The DBC file parsing module is used to parse the database file (DBC file) stored locally in the upper computer, and send the parsed information to the test piece.

[0056] In one embodiment of the present application, the upper computer parses the DBC file, including: inputting the DBC file storage path through the upper computer, and the DBC parsing program reads the file path of the DBC file of each CAN in the corresponding folder; using the open() function to open the DBC file of each CAN, and using the readline method to read each line of the file line by line to match the content of each read line, obtain different content according to the flag at the beginning of each line, and obtain the following content according to the DBC file format: obtain the CAN channel corresponding to the DBC file, obtain the name, MsgID, message length and whether it is received or sent on the channel of all messages sent and received by the channel, obtain all signal information of the MsgID, and obtain the sending cycle of each message.

[0057] In one embodiment of the present application, obtaining the CAN channel corresponding to the DBC file includes: using the if(line.startswith("BU_")): statement to filter the lines starting with "BU_", and the CAN channel is the content after the keyword "BU_", such as BU_:Node_1GW represents the Node_1 channel.

[0058] In one embodiment of the present application, obtaining the name, MsgID, message length and whether the message is sent or received on the channel includes: using the if(line.startswith("BO_")): statement to filter the line starting with "BO_", the content after the keyword "BO_" is canid, followed by the message name, followed by the message length, and followed by the send / receive flag. For example, the content parsed from BO_261VCU1_vehicleTorque_105:8Node_1 is: canid=261, message name VCU1_vehicleTorque_105, message length=8, and the message is sent on the CAN channel of Node_1. If the message is received by the channel, the final send / receive standard is "GW". And these information are separated by spaces when saved, so you can use the substrings = line.split ("") command to split the line into multiple strings, and get the corresponding information in the string according to the index of the information. For example, MsgID corresponds to the second string, its index is 1 (starting from 0), and its value is substrings [1]. When traversing to a new line, it corresponds to a new message, and you need to create a new list to save the signal, signal start bit, and signal length.

[0059] In one embodiment of the present application, obtaining all signal information of the MsgID includes: after a certain number of lines below a message, all signal information will be indented by one grid and listed, with each piece of information on one line. It starts with the keyword "SG_", so the if(line.startswith("SG_")): statement is used to filter the lines starting with "SG_", the keyword "SG_" is at the end, such as SG_EMS_engFuelCutoff:7|1@0+(1,0)[0|1]""Vector__XXX; represents that the signal name is EMS_engFuelCutoff, the actual bit of the signal is 7, and the signal length is 1. Similarly, its value is accessed by cutting it into a string.

[0060] In one embodiment of the present application, obtaining the sending cycle of each message includes: using the if(line.startswith("BA_")): statement to filter the lines starting with "BA_", the last content of the keyword "BA_", such as BA_"GenMsgCycleTime"BO_324 10; representing that the message GenMsgCycleTime cycle is 10ms.

[0061] In one embodiment of the present application, a test case automatic execution module is used to send a first diagnostic message and receive a second diagnostic message. Since the first diagnostic message and the second diagnostic message are both DOIP messages, the sending and receiving of DOIP messages to the test piece is implemented through the python socket library. The test piece converts the DOIP message (first diagnostic message) into a CAN message through the CAN-DOIP module and then sends it to the device under test. After performing the corresponding use case test, the test piece receives the CAN message returned by the device under test and converts the CAN message into a DOIP message (second diagnostic message) through the CAN-DOIP module, and sends the DOIP message to the test case automatic execution module of the host computer.

[0062] In one embodiment of the present application, the test case automatic execution module is also used to manage test cases, count all test cases that need to be executed according to the functional requirements of the vehicle ECU in accordance with national standards and enterprise standards, and then cover all test cases by writing different test case scripts. When the test case needs to be executed, it is automatically executed by running the corresponding script.

[0063] In one embodiment of the present application, the test case automatic execution module is also used to judge the test result of the test case. When each test case sends a DOIP message, the CAN message data (expected result) returned by the device under test that the test case expects to receive is judged according to national standards and enterprise standards, and the actual result returned by the device under test is compared with the preset expected result to determine whether the actual result is consistent with the preset expected result. If not, the test case test result of the test case is failed and recorded in the log module, and all DOIP messages about this failed case are saved to analyze the cause of the failure.

[0064] In one embodiment of the present application, the log module is used to save DOIP messages, and all DOIP messages of the use case whose test result is a failure. When a test case starts to execute, the wrpcap tool is used to start capturing all DOIP messages sent and received by the Ethernet port. If the use case test result of the test case is a failure, the DOIP messages in the entire test case process are saved so that engineers can analyze the problem. When saving, the file of the message is automatically named after the test case. If the use case test result of the test case is a success, the capture of DOIP messages is stopped at the end of the test case, and no saving is required.

[0065] In one embodiment of the present application, the test report automatic generation module can dynamically fill in the test report during the execution of the test case, and after the test is completed, count the pass rate of the test cases, summarize the items that failed the test, and output the test report in the form of Excel or Word text.

[0066] In one embodiment of the present application, when the test report automatic generation module generates a test report, the test report layout design - a template containing all test cases is designed according to the test case, and the test case, test description, and test type are the template content of the test report; the test results, levels, and test dates are automatically and dynamically filled in by the test report automatic generation module python script according to the results of the test case execution of the test case automatic execution module, and the test results show whether the test case passes or not, and if it fails, the severity level of the failed item is filled in the level and the cell is filled with the corresponding color through the oppyxl fill function to make it more obvious.

[0067] In one embodiment of the present application, if the failure level of the use case test result is a smoke level, the cell is filled with red; if the failure level of the use case test result is a high level, the cell is filled with yellow; if the failure level of the use case test result is a medium level, the cell is filled with green.

[0068] See also Figure 5 , Figure 5is a flow chart of vehicle automation performance testing shown in an exemplary embodiment of the present application, such as Figure 5 As shown, when performing vehicle automation performance testing, the user inputs a performance test command on the host computer. After receiving the command, the test piece enters the performance test program. The host computer selects a local DBC file (database file) and sends it to the test piece. The test piece parses the selected DBC file, obtains the routing relationship, and calculates the packet loss rate and delay based on it. The test piece sends load messages on each CAN channel according to the routing relationship. After receiving the load message, the device under test will forward these messages. The load message is sent and forwarded periodically during the test time. The CAN performance test module of the test piece calculates the packet loss rate and delay based on the received and sent load messages and sends the result as the performance test result to the host computer. The user can adjust the load message frequency on the host computer. When the user sends the end command on the host computer, the test piece ends the cyclic sending and receiving of the load message, and the user can view the performance test results on the host computer.

[0069] See also Figure 6 , Figure 6 is a vehicle automation use case test flow chart shown in an exemplary embodiment of the present application, such as Figure 6 As shown, when performing vehicle automation case test (functional test), the user inputs the test case automatic test command on the host computer, and the test piece enters the test case automatic test program after receiving the command. The host computer selects the local DBC file (database file), and the test piece parses the selected DBC file and obtains the routing relationship. The host computer reads all test cases through the test case management module, and sends the read test cases to the test piece one by one. For each received test case, the test piece performs the following actions: the test piece sends the test message of the test case, and the device under test receives and processes these messages and returns a response message to the test piece. The test piece judges the received response message (CAN message) and converts it into a second diagnostic message (DOIP message) and sends it to the host computer, so that the host computer obtains the test case test result of the test case based on the second diagnostic message and saves the test log based on the test case test result, fills in the test report, and visualizes the test result. When all test cases are sent and tested, the host computer forms a final test report based on the test case test results of each test case.

[0070] See also Figure 7 , Figure 7 is a schematic diagram of a visualization interface shown in an exemplary embodiment of the present application. In one embodiment of the present application, Figure 7As shown, by configuring the grafana visualization panel to display the real-time CAN bus load, CPU utilization, CAN message packet loss rate, CAN routing delay, fault reporting times and information, test case pass rate, test case execution rate and other key test indicators during the test, it is convenient and intuitive to view the test situation. Among them, the CPU utilization is reported by the CPU in real time, the fault reporting times and information are the number of times the test case test results are failed and related information, and the CAN bus load is load = (32 + 8 * DLC) * number of messages / baud rate. For example, assuming the baud rate is 500kbps, the DLC is 8, and 10 messages are transmitted per second, then the CAN bus load is: load = (32 + 8 * 8) * 10 / 500,000 = 0.96

[0071] In one embodiment of the present application, the process of building a visualization panel includes: 1. Install and configure the influxdb database on the host PC, install Prometheus and configure the connection to influxdb. Develop the influxdb database dot API in the test case automatic execution module, and call these APIs in real time in each step of the test case to record the real-time CAN bus load, CPU utilization, CAN message packet loss rate, CAN routing delay, fault reporting times and information, test case pass rate, test brave execution rate and other key test indicators during the test. Connect Prometheus on Grafana, create a visualization panel, and configure the test data indicator visualization panel. The visualization panel also includes the implementation of security measures, including user authentication and authorization. Users are divided into administrators and ordinary users. Administrators can view and configure the data indicators of concern at any time, and users can view these indicators at any time. Open the Grafana data visualization panel URL.

[0072] In one embodiment of the present application, the configuration process of the visualization panel includes: creating a new data source for the first time and creating a new dashboard for the first time. When creating a new data source, select the influxdb database address where the system stores data, enter the address, account number, and password of the database; create a new blank dashboard, and the URL of the dashboard is the page where data visualization will be performed. The panel belongs to a certain dashboard, so we need to create a dashboard first. Click + in the sidebar to switch to Dashboard and start creating a dashboard. There is an empty panel in the new dashboard created by default. Click Add an empty panel to start adding a panel: create panel. Display the selected chart type in the panel part, such as pie charts, bar charts, line charts, etc. You can also configure PromQL statements and enter the corresponding PromQL syntax statements for the desired chart effect, such as the dial configuration for displaying the test case pass rate: the number of test case pass points and / total number of points, and the real-time CPU utilization configuration: display the CPU utilization information carried by each point. You can also personalize the properties and appearance of the dial. Each data of interest has a dial. One panel can be configured with multiple dials. For other data of interest, follow the above process.

[0073] See also Figure 8 , Figure 8 is a block diagram of a vehicle automated testing device shown in an exemplary embodiment of the present application. The device can be applied to Figure 1 The implementation environment shown is as follows. The device may also be applicable to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0074] like Figure 8 As shown, the exemplary vehicle automation testing device includes: a message recording module 801 , a performance testing module 802 , a use case receiving module 803 and a use case testing module 804 .

[0075] The message recording module 801 is used for the test piece to periodically send or receive load messages according to the routing relationship. Each time the test piece sends or receives a load message, the sending and receiving records corresponding to the message identifier are recorded based on the message identifier of the sent or received load message.

[0076] The performance test module 802 is used to obtain the performance test results corresponding to different message identifiers based on the sending and receiving records corresponding to each message identifier and upload the performance test results to the host computer;

[0077] The use case receiving module 803 is used to receive a first diagnostic message corresponding to the test case sent by the host computer, convert the first diagnostic message into a test message corresponding to the test case, and send the test message to the device under test, so that the device under test executes the test case based on the test message and returns a response message;

[0078] The use case testing module 804 is used to convert the response message into a second diagnostic message, and upload the second diagnostic message to the host computer, so that the host computer obtains the actual result of the test case based on the second diagnostic message, and compares the actual result of the test case with the preset expected result corresponding to the test case to obtain the use case test result of the test case.

[0079] Fig. 9 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Fig. 9 The computer system 900 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0080] like Fig. 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 902 or the program loaded from the storage part 908 to the random access memory (RAM) 903, such as executing the method described in the above embodiment. In the RAM 903, various programs and data required for system operation are also stored. The CPU 901, ROM 902 and RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0081] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed so that a computer program read therefrom is installed into the storage section 908 as needed.

[0082] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 909, and / or installed from a removable medium 911. When the computer program is executed by a central processing unit (CPU) 901, various functions defined in the system of the present application are executed.

[0083] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0084] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0085] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0086] Another aspect of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes the vehicle automated testing method as described above. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0087] Another aspect of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the vehicle automation test method provided in each of the above embodiments.

[0088] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A vehicle automated testing method, characterized in that: The vehicle automation testing method comprises: The test piece periodically sends or receives a load message according to the routing relationship. Each time the test piece sends or receives the load message, the test piece records a sending and receiving record corresponding to the message identifier based on the message identifier of the sent or received load message. Obtaining performance test results corresponding to different message identifiers based on the sending and receiving records corresponding to each message identifier and uploading the performance test results to the host computer; Receive a first diagnostic message corresponding to the test case sent by the host computer, convert the first diagnostic message into a test message corresponding to the test case, and send the test message to the device under test, so that the device under test executes the test case based on the test message and returns a response message; The response message is converted into a second diagnostic message, and the second diagnostic message is uploaded to the host computer so that the host computer obtains the actual result of the test case based on the second diagnostic message, and compares the actual result of the test case with the preset expected result corresponding to the test case to obtain the case test result of the test case.

2. The vehicle automated testing method according to claim 1, characterized in that: The test piece periodically sends or receives load messages based on the routing relationship, including: The test piece receives the database file sent by the host computer, and parses the database file to generate the routing relationship with the device under test, the test piece is a vehicle electronic controller with message receiving and sending functions and Ethernet communication functions, the device under test is another electronic controller connected to the test piece, and the host computer communicates with the test piece through the Ethernet communication function; The test piece periodically sends the load message to the device under test based on the routing relationship, and the device under test forwards the load message to the test piece after receiving the load message.

3. The vehicle automated testing method according to claim 1, characterized in that: Each time the test piece sends or receives the load message, the sending and receiving records corresponding to the message identifier of the sent or received load message are recorded based on the message identifier of the sent or received load message, and include: When the test piece sends the load message once, a sending timestamp is added to the message identifier corresponding to the sent load message and the number of sending times is increased once, and the sending and receiving record includes the sending timestamp and the number of sending times; Each time the test piece receives a load message, a receiving timestamp is added to the message identifier corresponding to the received load message and the number of receiving times is increased once. The sending and receiving record includes the receiving timestamp and the number of receiving times.

4. The vehicle automated testing method according to claim 3, characterized in that: Obtaining performance test results corresponding to different message identifiers based on the sending and receiving records corresponding to each message identifier and uploading the performance test results to the host computer includes: For the load message with the same message identifier, the packet loss rate corresponding to the message identifier is calculated based on the number of transmissions, the number of receptions and the cycle adjustment coefficient, the cycle adjustment coefficient is obtained based on the receiving cycle and the sending cycle, the receiving cycle is obtained based on the sending and receiving records, the sending cycle is obtained based on the routing relationship, and the performance test result includes the packet loss rate and delay; Calculating the delay of the load message based on the receiving timestamp and the sending timestamp of the load message for the same load message; Obtaining the maximum delay and average delay corresponding to the message identifier according to the delays of all load messages with the same message identifier; The packet loss rate, the maximum delay and the average delay corresponding to each message identifier are uploaded to the host computer as performance test results.

5. The vehicle automated testing method according to any one of claims 1 to 3, characterized in that: The test piece periodically sends or receives load messages according to the routing relationship and also includes: Receiving a load message frequency sent from the host computer, and adjusting a sending period of the load message based on the load message frequency to adjust the load of the test piece; The sending and receiving records are recorded under different loads of the test piece to obtain the performance test results under different loads.

6. The vehicle automated testing method according to any one of claims 1 to 3, characterized in that: After receiving the first diagnostic message corresponding to the test case sent by the host computer, the method includes: The host computer captures all diagnostic messages received and sent by the Ethernet interface, and if the test result of the test case is a test failure, all the diagnostic messages during the execution of the test case are saved and recorded in a test log; If the test result of the test case is successful, the diagnostic message during the execution of the test case is not saved.

7. The vehicle automated testing method according to any one of claims 1 to 3, characterized in that: Comparing the actual result of the test case with the preset expected result corresponding to the test case to obtain the test case test result of the test case includes: If the actual result is disconnection, the test result of the use case is a test failure and the failure level of the test result of the use case is a smoke level; If the actual result is inconsistent with the preset expected result, the test result of the use case is a test failure and the failure level of the test result of the use case is a medium level; If the actual result is other errors beyond expectations, the test result of the use case is a test failure and the failure level of the test result of the use case is a high level; The host computer establishes a test visualization panel, which includes a test result display section and a test configuration section; Visually display the performance test results and the test results of use cases with different failure levels through the test result display section; The different database files and the test cases are configured through the test configuration panel.

8. A vehicle automated testing device, characterized in that: The vehicle automation test device comprises: A message recording module, used for the test piece to periodically send or receive load messages according to the routing relationship, and each time the test piece sends or receives the load message, the sending and receiving record corresponding to the message identifier is recorded based on the message identifier of the sent or received load message; A performance test module, used to obtain performance test results corresponding to different message identifiers based on the sending and receiving records corresponding to each message identifier and upload the performance test results to the host computer; A use case receiving module, used to receive a first diagnostic message corresponding to the test case sent by the host computer, convert the first diagnostic message into a test message corresponding to the test case, and send the test message to the device under test, so that the device under test executes the test case based on the test message and returns a response message; A use case testing module is used to convert the response message into a second diagnostic message, and upload the second diagnostic message to the host computer, so that the host computer obtains the actual result of the test case based on the second diagnostic message, and compares the actual result of the test case with the preset expected result corresponding to the test case to obtain the use case test result of the test case.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the electronic device to implement the vehicle automation testing method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is enabled to execute the vehicle automation testing method as described in any one of claims 1 to 7.

Images