LLM-based controller can communication test method and system

Abstract

The application discloses a kind of controller CAN communication test method and system based on LLM, DBC file is converted into configuration file, VSS and CAN signal corresponding relationship file is read, as input to LLM;Call LLM model, according to prompt word generation test case;Based on behavior-driven development BDD mode, build automation framework to realize execution function;Correlation generated test case and execution function in automation framework, build complete test suite, run test suite and execute CAN bus communication test.The application is based on the controller that has realized VSS technology to carry out the check of VSS path and CAN Signal, using LLM generates the test case of BDD natural language, test case is directly used as test script, saves the step of manual writing test case, improves test efficiency, reduces test cost.

Description

TECHNICAL FIELD

[0001] The present application relates to CAN communication test technology, in particular to a controller CAN communication test method and system based on LLM. BACKGROUND

[0002] With the upgrading iteration of automobile electronic architecture, a new generation of automobile electronic and electrical architecture is introduced, and different vehicle models gradually tend to the electronic and electrical architecture scheme of central computing unit (CCU) and 2-4 regional controller (Zone Controller Unit, ZCU) in terms of ECU hardware configuration. The vehicle functions are becoming more and more rich, and the signal quantity of CAN bus is also becoming larger and larger. As a core component in the vehicle, the CAN signal quantity supported by the regional controller is also increasing, and the CAN signal value check as the basic communication link is particularly important. The CAN signal names, quantities, values, etc. in the DBC files of different manufacturers are different, the CAN signal is converted to VSS Path through the VSS standard, the different CAN signals at the bottom layer can be ignored, and the technical scheme is also being implemented.

[0003] Based on the technology of converting CAN signal to VSS Path, the CAN communication link check is realized on this basis, and the initial value, the maximum value, the minimum value and the random value are checked. Compared with the prior art, only the VSS standard needs to be concerned, the CAN signal physical value is obtained through the VSS Path, the test case of the standard template is generated through the LLM, the test case is directly imported into the BDD framework, the automatic test is realized, and the test verification efficiency is improved. SUMMARY

[0004] In order to solve the problems in the prior art, the purpose of the present application is to provide a controller CAN communication test method and system based on LLM.

[0005] In order to achieve the purpose of the present application, the technical scheme adopted by the present application is:

[0006] A controller CAN communication test method based on LLM, comprising the steps of:

[0007] (1) converting the DBC file into a configuration file, reading the VSS and CAN signal corresponding relationship file as input to the LLM;

[0008] (2) calling the LLM model, generating a test case according to the prompt word;

[0009] (3) constructing an automatic framework to realize an execution function based on a behavior-driven development BDD mode;

[0010] (4) Associate the generated test cases with the execution functions in the automation framework, build a complete test suite, and run the test suite to perform CAN bus communication testing.

[0011] Further, in step (1), the configuration file is in units of messages, and each message includes:

[0012] Message type: "CAN";

[0013] Message ID: uniquely identifies each CAN message;

[0014] Message name: describes the function of the message;

[0015] Byte size: standard CAN: 8 bytes; CAN FD: 8-64 bytes;

[0016] Signal list: each signal contains position, data type, and range details.

[0017] Further, in step (2), prompt words are written according to specific test requirements, and test cases are automatically generated using templated prompt words based on an LLM model. The LLM model reviews whether the test cases meet the requirements, and the user makes the final confirmation.

[0018] Further, prompt words are written according to specific communication test procedures, which include:

[0019] From top to bottom, the test node controls the VSS Client to issue VSS Path and corresponding values. The VSS Client subscribes to the VSS Service through a dds topic. The VSS Service is deployed on the CCU, maps the VSS Signal to the VSS Path through the VSS Service, and converts the VSS Signal values according to the DBC rules to convert the physical values to raw values. The ZCU converts the VSS Signal to the CAN Signal and connects with the real-time simulator through the CAN-BUS. The test node controls the real-time simulator to obtain the CAN Signal.

[0020] From bottom to top, the test node controls the real-time simulator to send the CAN Signal. The CAN Signal is forwarded to the ZCU through the CAN-BUS. The ZCU converts the CAN Signal to the VSS Signal and forwards it to the CCU through DDS. The VSS Service processes the VSS Signal to map it to different VSS paths. The test node controls the VSS Client to obtain the VSS Path and corresponding values, and checks whether the CAN Signal and the VSS path values are consistent.

[0021] Further, convert the DBC file to signal.json, read the VSS and CAN signal correspondence file mapping.json;

[0022] According to the input mapping file mapping.json, parse the VSS signal and VSS path mapping relationship, obtain the CAN signal value according to the configuration file signal.json, use the LLM model to understand the content of the mapping file, and generate a structured test case according to the format requirements.

[0023] Further, the test case format includes:

[0024] Feature: a feature represents a function;

[0025] Scenario: used to describe a use case;

[0026] Step: contains Given, When, Then, used to define the steps of the scenario;

[0027] Given: the environment required by the given scenario, a prerequisite condition;

[0028] When: a user event;

[0029] Then: define the verification result, the verification point in the usual test, and the assertion.

[0030] Further, in step (3), the execution function includes:

[0031] CAN communication function: encapsulates CAN bus data transmission and reception, and parsing logic;

[0032] VSS Client calling function: realizes the interaction with the vehicle signal specification VSS server;

[0033] VSS path and CAN signal verification function: defines signal path verification and CAN signal value verification rules;

[0034] Given / Then BDD syntax function: standardizes test step description, and associates test preconditions and assertion logic.

[0035] A controller CAN communication test case generation method based on LLM, comprising the steps of:

[0036] (1) Convert the DBC file to a configuration file, read the VSS and CAN signal correspondence file, and input it to the LLM;

[0037] (2) Call the LLM model to generate test cases according to the prompt words; the model self-evaluates whether the test cases meet the requirements, the user makes the final confirmation, and the test cases are converted into executable automated test scripts.

[0038] An LLM-based controller CAN communication test case generation system, comprising a file parsing module, an LLM test case generation module, and an automated script generation module;

[0039] The file parsing module performs parsing operations on the input related files and extracts key information; the parsed information is transferred to the LLM test case generation module, which automatically generates test cases for controller CAN communication with the aid of a large language model (LLM) based on the parsed content and preset prompt words; the test cases output by the LLM test case generation module are passed to the automated script generation module to convert the test cases into executable automated test scripts.

[0040] The LLM-based controller CAN communication test method of the present application has the advantages that, compared with the prior art, the controller based on the VSS technology is used to verify the VSS Path and CAN Signal, which can verify the connectivity of the CAN communication link and the Ethernet communication link and the correctness of the CAN signal transmission, and only needs to focus on the VSS standard and does not need to focus on the changes of the underlying CAN signal. The LLM is used to generate test cases in the natural language of BDD, and the test cases are directly used as test scripts, which saves the step of manually writing test cases and a large amount of development of automated test code, improves the test efficiency, and reduces the test cost. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is a LLM-based controller CAN communication test flowchart;

[0042] Figure 2 is a LLM-based controller CAN communication test case generation method flowchart;

[0043] Figure 3 is a signal.json format schematic diagram;

[0044] Figure 4 is a prompt word example diagram;

[0045] Figure 5 is a LLM-based controller CAN communication test method flowchart;

[0046] Figure 6 is a LLM-based controller CAN communication test case generation system schematic diagram. DETAILED DESCRIPTION

[0047] The technical solutions of the present application are further described below in combination with the drawings and examples. The following examples are only used to more clearly illustrate the technical solutions of the present application, and cannot limit the protection scope of the present application.

[0048] As shown in Figure 1 The controller CAN communication test flow of the present application based on LLM, the test node can be deployed on a PC or other device containing a central processor.

[0049] From top to bottom, the test node is responsible for controlling the VSS Client to issue VSS Path and corresponding values, the VSS Client can be deployed on a device, on the test node, or on the CCU. The VSS Client subscribes to the VSS Service through the dds topic, and sets the VSS Path and corresponding values through the set interface. The VSS Service is deployed on the CCU, maps the VSS Signal (signal) with the VSS Path (path) through the VSS Service, and converts the VSS Signal value according to the DBC rule to convert the physical value to the original value. The ZCU converts the VSS Signal to the CAN Signal, connects with the real-time emulator through the CAN-BUS, and the test node controls the real-time emulator to obtain the CAN Signal. The real-time emulator simulates various vehicle-mounted sensors, ECUs, and other CAN messages.

[0050] From bottom to top, the test node controls the real-time emulator to send the CAN Signal, the CAN Signal is forwarded to the ZCU through the CAN-BUS, the ZCU converts the CAN Signal to the VSS Signal and forwards it to the CCU through the DDS, the VSS Service processes the VSS Signal to map it to different VSS paths, the test node controls the VSS Client to obtain the VSS Path and corresponding values, and checks whether the CAN Signal and the VSS path values are consistent.

[0051] As shown in Figure 2 The test case generation method of the controller CAN communication of the present application based on LLM includes the following steps:

[0052] First, convert the DBC file to signal.json, and the converted format is as shown in Figure 3 Read the VSS and CAN signal correspondence file mapping.json, and use the two json files as input to the LLM;

[0053] According to the DBC file analysis into signal.json, list all the CAN message under the CAN signal, list id, name, startBit, bitSize, dataType, etc., according to factor and offset to calculate the physical value range of CAN signal.

[0054] As Figure 3 The file structure is in units of messages (Message), and each message contains the following key information:

[0055] Message type (type): "CAN";

[0056] Message ID (id): uniquely identifies each CAN message;

[0057] Message name (name): describes the function of the message;

[0058] Byte size (byteSize): standard CAN: 8 bytes; CAN FD: 8~64 bytes;

[0059] Signal list (signals): each signal contains position, data type, range, etc. Detailed parameters.

[0060] Second, build a test case automatic generation system based on large language model, according to the specific needs of writing prompt words, prompt word example as Figure 4 shown;

[0061] Users can define LLM as different roles according to the prompt words, and test case content can be given through prompt words. Test case text format can also be given through prompt words. And you can use different models to generate test cases to find the best test case. Test the test case to see if it meets the requirements. If the generated test case does not meet the requirements, you can modify the prompt words. LLM model reviews test cases for compliance, and users can confirm.

[0062] Based on LLM, use templated prompt words to automatically generate test cases, parse VSS signals and VSS path mapping relationships in the input mapping.json, get the initial value, maximum value, and minimum value of the CAN signal according to signal.json, use large language model to understand the content of the JSON mapping file, generate structured test cases according to the format requirements, support Markdown format output, and facilitate direct use.

[0063] The test case format generated by the prompt words in this example is consistent with the behavior description of BDD:

[0064] Feature (feature): a feature represents a function;

[0065] Scenario (scenario): used to describe a use case;

[0066] Step: contains Given, When, Then, used to define the steps of the scenario;

[0067] Given (given): the environment required by the scenario, a precondition;

[0068] When (when): a user event, such as clicking, inputting, etc.

[0069] Then (then): define the verification result, the verification point in the usual test, assertion.

[0070] Using LLM to generate test cases, there is no need for additional logic to implement the association of VSS Path and CAN Signal, and there is no need for additional logic to extract the CAN Signal value. These tasks are all completed by LLM. Users only need to provide appropriate prompt words and select appropriate AI models.

[0071] The test cases generated by LLM can be well adapted to BDD automated test framework. Using BDD test framework, a subset of Gherkin language is used to define test scenarios, features are broken down into individual scenarios, and each step of the scenario is described using natural language. The natural language of each step in the scenario is recognized, parsed, and converted into code language.

[0072] As shown in Figure 5 The controller CAN communication test method based on LLM of the present application comprises the following steps:

[0073] (1) Import DBC file, convert DBC file to signal.json, read VSS and CAN signal corresponding relationship file mapping.json, and input the two json files to LLM;

[0074] Here, the CAN database file (Database CAN, DBC) is generally a DBC file provided by the vehicle manufacturer, has a standard format, and can also be a custom DBC file, mainly including message definitions and signal definitions, and can be in.dbc,.xml, or other formats. By extracting this file, information such as message type, message ID, message name, byte size, signal list (each signal contains position, data type, range, and other detailed parameters) is obtained, and then a data configuration file is generated based on this information. The configuration file generated in the example is in json format, and other formats can also be generated. The mapping.json file corresponding to the VSS and CAN signal relationship can also be in other formats, depending on whether the LLM supports it, and currently the LLM on the market supports multiple file format types.

[0075] (2) Call the LLM model to generate test cases according to the prompt words;

[0076] (3) Based on the behavior-driven development (BDD) mode, an automation framework is constructed to realize in turn:

[0077] CAN communication function: encapsulates CAN bus data transmission and reception, and parsing logic;

[0078] VSS Client calling function: realizes interaction with the vehicle signal specification (VSS) server;

[0079] VSS Path and CAN Signal verification function: defines signal path verification and CAN signal value verification rules;

[0080] Given / Then BDD syntax function: standardizes test step description and associates preconditions and assertion logic.

[0081] (4) Associate the generated test cases with the execution functions in the automation framework to build a complete test suite; run the test suite to perform CAN bus communication testing; collect test data to generate a test report containing case execution status and signal verification results.

[0082] This process realizes efficient construction and execution of CAN bus communication testing by automatically parsing, AI-assisted case generation, and BDD framework combination, covering the complete test cycle from input processing to result output.

[0083] As Figure 6 shown, the LLM-based controller CAN communication test case generation system of the present application includes three function modules connected in turn, a file parsing module, an LLM test case generation module, and an automation script generation module.

[0084] Firstly, the file analysis module is responsible for analyzing the input related files (such as DBC files, etc.) and extracting key information; then, the parsed information is transferred to the LLM test case generation module, which generates test cases for controller CAN communication by means of large language model (LLM) according to the parsed content and preset prompt words; finally, the test cases output by the LLM test case generation module are transmitted to the automatic script generation module, which converts the test cases into executable automatic test scripts, so as to realize the modular operation of the LLM-based controller CAN communication test related process.

[0085] The application has the advantages that, compared with the prior art, the LLM-based controller CAN communication test method can verify the VSS Path and CAN Signal based on the controller that has realized the VSS technology, can verify the connectivity of the CAN communication link and the Ethernet communication link, and can verify the correctness of the CAN signal transmission, only needs to focus on the VSS standard, and does not need to focus on the change of the underlying CAN signal. The LLM is used to generate test cases conforming to the BDD natural language, the test cases are directly used as test scripts, the step of manually writing test cases is saved, a large amount of automatic test code development is saved, the test efficiency is improved, and the test cost is reduced.

[0086] The applicant of the present application has made a detailed description and explanation of the embodiments of the present application in combination with the drawings of the specification, but those skilled in the art should understand that the above embodiments are only preferred embodiments of the present application, and the detailed description is only to help the reader better understand the spirit of the present application, and is not a limitation on the protection scope of the present application, on the contrary, any improvement or modification based on the spirit of the present application should fall within the protection scope of the present application.

Claims

1. A method for testing CAN communication of an LLM-based controller, characterized in that, The method comprises the steps of: (1) converting the DBC file into a configuration file, reading a VSS and CAN signal correspondence file as input to the LLM; (2) calling the LLM model, writing prompt words according to specific test requirements, automatically generating test cases based on the LLM model using templated prompt words, and the LLM model automatically reviewing whether the test cases meet the requirements, and the user making a final confirmation; Specifically, the DBC file is converted into signal.json, and the VSS and CAN signal correspondence file mapping.json is read; the VSS signal and VSS path mapping relationship is parsed according to the input mapping file mapping.json, the CAN signal value is obtained according to the configuration file signal.json, the LLM model is used to understand the content of the mapping file, and the structured test case is generated according to the format requirement; (3) based on the behavior-driven development BDD mode, an automatic framework is constructed to implement an execution function; The execution function includes: a CAN communication function, a VSS Client calling function, a VSS Path and CAN Signal verification function, and a Given / Then BDD syntax function; (4) associating the generated test case with the execution function in the automatic framework, constructing a complete test suite, and running the test suite to execute CAN bus communication testing.

2. The LLM-based controller CAN communication test method of claim 1, wherein, In step (1), the configuration file is in units of messages, and each message includes: Message type: "CAN"; Message ID: uniquely identifies each CAN message; Message name: describes the function of the message; Byte size: standard CAN: 8 bytes; CAN FD: 8~64 bytes; Signal list: each signal contains position, data type, and range detail parameters.

3. The LLM-based controller CAN communication test method of claim 1, wherein, According to the specific communication test process, the prompt words are written, and the communication test process includes: From top to bottom, the test node controls the VSS Client to issue the VSS Path and the corresponding value, the VSS Client subscribes to the VSS Service through ddstopic, the VSS Service is deployed on the CCU, the VSS Signal is mapped with the VSS Path through the VSS Service, and the VSS Signal value is converted according to the DBC rule to convert the physical value into the original value; the ZCU converts the VSS Signal into the CAN Signal, connects with the real-time simulation machine through the CAN-BUS, and the test node controls the real-time simulation machine to obtain the CAN Signal. From bottom to top, the test node controls the real-time simulator to send CAN signals, the CAN signals are forwarded to the ZCU through the CAN-BUS, the ZCU converts the CAN signals into VSS signals and forwards them to the CCU through the DDS, the VSS Service processes the VSS signals to map them into different VSS paths, the test node controls the VSS Client to obtain the VSS paths and corresponding values, and it is verified whether the CAN signals and the VSS path values are consistent.

4. The LLM-based controller CAN communication test method of claim 1, wherein, The test case format includes: Feature: a feature represents a function; Scenario: used to describe a use case; Step: contains Given, When, and Then, used to define the steps of the scenario; Given: the environment required by the scenario, a precondition; When: a user event; Then: defines the verification result, the verification point in the usual test, assertion.

5. The LLM-based controller CAN communication test method of claim 1, wherein, In step (3), the execution function includes: CAN communication function: encapsulates CAN bus data transmission and reception, analysis logic; VSS Client calling function: realizes interaction with the vehicle signal specification VSS server; VSS Path and CAN Signal verification function: defines signal path verification and CAN signal value verification rules; Given / Then BDD syntax function: standardizes test step description, associates test preconditions and assertion logic.

Images