Multi-dimensional intent control test system and method based on large language model

Abstract

The application discloses a kind of multi-dimension intention control test system and method based on large language model, including test case generation module, parse the configuration file of intention control system, generate test case according to different scene type;Atomic service call verification module is different according to different scene processing logic, capture function call information from intention control system log, compare with the expected result in test case, for single scene, verify complete match, for combined scene, carry out logic and and logic or operation, calculate confidence degree;Vehicle state consistency verification module obtains current actual control car state by RESTful interface, compare with function call output, verify one-to-one correspondence;Finally enter semantic verification module, different scene selects different semantic verification branch.The application judges the correctness of intention control system by multi-dimension verification mechanism, significantly improves the accuracy and reliability of verification.

Description

Technical Field

[0001] This invention relates to the field of vehicle control, and in particular to a multi-dimensional intent control testing system and method based on a large language model. Background Technology

[0002] With the rapid development of intelligent connected vehicle technology, dynamically configured in-vehicle intent control systems are increasingly being applied in the field of vehicle control. This system understands the user's natural language commands through a large language model, matches the corresponding intent configuration, and invokes in-vehicle atomic services to complete vehicle control operations. However, existing technologies have the following technical problems:

[0003] The results of vehicle-side intent control are inherently uncertain, and a single verification method is insufficient: Vehicle-side intent control systems involve multiple stages, including natural language understanding, intent matching, and atomic service invocation, each of which carries inherent uncertainty. The diversity of natural language expressions and the uncertainty of large language model responses make it difficult to ensure that the model correctly understands the user's intent and provides the expected response. For fuzzy instructions (such as "go down a little" or "drive a little"), it is difficult to verify whether the parameters parsed by the model match the user's true intent. Figure 1 To be continued. Existing verification methods often rely on a single semantic judgment or a single vehicle control service judgment, which cannot comprehensively and accurately verify the correctness of the intent control system. Single semantic verification cannot verify whether the atomic service call is reasonable, and single vehicle control service verification cannot verify whether the semantic understanding is accurate. Both have limitations and make it difficult to ensure the accuracy of the verification results.

[0004] The end-to-end verification process is incomplete: existing verification methods are mostly manual testing or simple functional testing, lacking an end-to-end automated verification process from user input to the final change in vehicle state, and cannot systematically verify the correctness of each link in the entire intent control chain.

[0005] Therefore, there is an urgent need for a multi-dimensional intent control testing system and method based on a large language model, which can automatically and systematically verify the semantic understanding accuracy, atomic service call rationality, and parameter parsing correctness of the intent control system, thereby improving verification efficiency and reliability. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the present invention aims to provide a multi-dimensional intent control testing system and method based on a large language model.

[0007] To achieve the objectives of this invention, the technical solution adopted is as follows:

[0008] A multi-dimensional intent control testing system based on a large language model includes test nodes, an intent control system, a ZCU, and a real-time simulator;

[0009] The test node is responsible for executing test cases and verification processes; the intent control system is the target system being verified, which contains a CCU and is responsible for coordinating the entire vehicle intent control process; the ZCU is responsible for vehicle control within the area; the real-time simulator is used to simulate the actual hardware environment of the vehicle and provide realistic vehicle status feedback.

[0010] WebSocket connections are used for real-time bidirectional communication. Test nodes send user natural language commands to the intent control system via WebSocket and receive responses from the intent control system.

[0011] The RESTful interface connection is used to query vehicle status information. The test node calls the getData interface of the intent control system through the RESTful interface to obtain the current vehicle control status and verify whether the atomic service has actually performed the vehicle control operation.

[0012] The CCU and ZCU communicate to transmit control commands and status information.

[0013] The ZCU and the real-time simulator communicate via the CAN-BUS protocol. The ZCU converts control commands into CAN signals and sends them to the real-time simulator. The real-time simulator simulates the vehicle hardware response and returns vehicle status information via CAN-BUS.

[0014] Furthermore, each intent-controlled WebSocket connection maintains an independent session history and supports context-dependent instruction verification.

[0015] Furthermore, the test node deploys four sequentially connected functional modules: test case generation module, atomic service call verification module, vehicle status consistency verification module, and semantic verification module.

[0016] The test case generation module parses the configuration file of the intent control system and generates test cases according to different scenario types. The expected result of each test case is given by a specific atomic service and parameter value.

[0017] The atomic service call verification module processes different logics for different scenarios. It captures function call information from the intent control system log and compares it with the expected results in the test cases. For a single scenario, it verifies a perfect match. For combined scenarios, it performs logical AND and logical OR operations and calculates the confidence level.

[0018] If the atomic service call verification module passes, the vehicle state consistency verification module will be executed. This module has multiple branches depending on the scenario. It obtains the current actual vehicle control state through a RESTful interface and compares it with the function call output to verify the one-to-one correspondence. For combined scenarios, the logic of state changes will be verified. Before the vehicle control command is executed, the current actual vehicle control state is obtained and compared with the state after the vehicle control command is executed to verify whether the changes in the vehicle control state before and after are logical.

[0019] Finally, we enter the semantic verification module, where different semantic verification branches are selected for different scenarios.

[0020] Furthermore, the semantic verification branch of the semantic verification module includes: basic semantic verification, which uses a large language model to perform simple semantic comparison; and enhanced semantic verification, which uses multi-round verification and context enhancement methods to improve accuracy, with the large language model selecting the model that is good at semantic recognition.

[0021] A multi-dimensional intent control testing method based on a large language model includes the following steps:

[0022] (1) Parse the configuration file of the intent control system, expand the synonyms based on the intent description using the large language model, generate test cases according to different scenario types, and give the expected results of each test case with a definite atomic service and parameter value;

[0023] (2) Verify atomic service calls, capture function call information from the intention control system log, compare it with the expected results in the test cases, verify a perfect match for a single scenario, and perform logical AND and logical OR operations for combined scenarios to calculate the confidence level;

[0024] (3) If the atomic service call verification is successful, the vehicle state consistency verification is performed. Depending on the scenario, there are multiple branches. The current actual vehicle control state is obtained through the RESTful interface and compared with the function call output to verify the one-to-one correspondence. For combined scenarios, the state change logic verification is required. Before the vehicle control command is executed, the current actual vehicle control state is obtained and compared with the state after the vehicle control command is executed to verify whether the changes in the vehicle control state before and after are logical.

[0025] (4) Finally, semantic verification is performed. Different semantic verification branches are selected for different scenarios. Basic semantic verification uses a large language model to perform simple semantic comparison. Enhanced semantic verification uses multi-round verification and context enhancement methods to improve accuracy. The large language model selected is a model that is good at semantic recognition.

[0026] Further, the test case generation steps:

[0027] Test cases are automatically generated based on core inputs, which include intent configuration files, user command examples, expected vehicle control results, and vehicle control calls to atomic services.

[0028] Parse the intent configuration to extract the intent name, description, and execution configuration information; expand the synonyms based on the intent description using a large language model to generate test cases not in the intent configuration; classify the test cases according to the expected vehicle control results and generate corresponding test case formats according to the scenario type.

[0029] Furthermore, the single-scenario verification process:

[0030] Step 1: Check the intent control system to see if the function call information of the large language model matches the expectation. If they match, proceed to the next step; otherwise, return failure.

[0031] Step 2: Call the RESTful interface to obtain the current actual vehicle control status and compare it with the function call output of the intent control system. If they match, proceed to the next step; otherwise, return failure.

[0032] Step 3: The semantic consistency between user commands and intents and the control system response is compared. If consistent, the verification passes; if inconsistent, the verification returns failure.

[0033] Furthermore, the combined scenario verification process:

[0034] Step 1: Compare the expected atomic service to be called with the function calls of the large language model recorded in the intent control system log; the combined scenario is expected to have multiple atomic services. When comparing with the actual function calls of the intent control system, perform logical AND and logical OR operations to calculate the confidence score of the match. If the confidence score is ≥50%, proceed to the next step; if the confidence score is <50%, return failure.

[0035] Step 2: Call the RESTful interface to obtain the current actual vehicle control status and compare it with the function call output of the intent control system. If they match, proceed to the next step; otherwise, return failure.

[0036] Step 3: Compare the changes in vehicle control status before and after execution to verify whether the changes in status are logical; verify whether the parameters of the relative command are reasonable; verify the logic of the changes in status; verify the understanding of historical context; if the changes in vehicle control status before and after execution are logical, proceed to the next step; if they are not logical, return failure.

[0037] Step 4: Compare the semantic consistency between the user commands and the response commands of the large language model recorded in the intent control system log. Use an enhanced semantic verification method. If they are consistent, the verification passes; if they are inconsistent, the verification returns failure.

[0038] The beneficial effects of this invention are that, compared with existing technologies, it solves the problem of uncertainty in intent control results and improves verification accuracy. By comprehensively judging the correctness of the intent control system through a multi-dimensional verification mechanism (semantic verification, atomic service verification, and state verification), it solves the problem that single semantic judgment or single vehicle control service judgment is not accurate enough. Compared with single verification methods, this invention can comprehensively evaluate the correctness of the system from multiple dimensions, significantly improving the accuracy and reliability of verification. Traditional test cases are directly divided based on vehicle control function modules. This invention, based on vehicle control function modules, innovatively divides them according to different scenarios, categorizing test scenarios into multiple scenarios according to the expected vehicle control results. It innovatively adopts a RESTful interface for business function verification, which is faster to learn than CAN bus interfaces such as python-can, eliminating the need to learn the CAN protocol and python-can library, directly verifying business functions, and reducing development threshold and learning costs. It achieves end-to-end automated verification from user input to vehicle state changes without manual intervention. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the multi-dimensional intent control testing system based on a large language model according to the present invention;

[0040] Figure 2 This is a structural diagram of the multi-dimensional intent control testing module based on a large language model according to the present invention;

[0041] Figure 3 This invention provides a flowchart for generating multi-dimensional intent control test cases based on a large language model.

[0042] Figure 4 This is a flowchart of a single scenario for the multi-dimensional intent control testing method based on a large language model, as described in this invention.

[0043] Figure 5 This is a flowchart of a combined scenario for a multi-dimensional intent control testing method based on a large language model, as presented in this invention. Detailed Implementation

[0044] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of this application.

[0045] like Figure 1 As shown, the multi-dimensional intent control test system based on a large language model described in this invention includes a test node, an intent control system, a ZCU, and a real-time simulator.

[0046] The test node is the core component of the system, responsible for executing test cases and verifying the process. The intent control system is the target system being verified, containing functional modules such as intent matching, ODC (Operational Design Condition) rule checking, and intent execution. Internally, this system includes a CCU (Central Control Unit), responsible for coordinating the entire vehicle-side intent control process. The ZCU (Zone Control Unit) is responsible for vehicle control within its designated area. The real-time simulator is used to simulate the actual hardware environment of the vehicle, providing realistic vehicle status feedback.

[0047] WebSocket connections are used for real-time bidirectional communication. Test nodes send user natural language commands to the intent control system via WebSocket and receive responses from the intent control system. Each intent control WebSocket connection maintains an independent session history and supports context-sensitive command verification.

[0048] The RESTful interface connection is used to query vehicle status information. The test node calls the getData interface of the intent control system through the RESTful interface to obtain the current vehicle control status and verify whether the atomic service has actually performed the vehicle control operation.

[0049] The CCU and ZCU communicate to transmit control commands and status information. The ZCU and the real-time simulator communicate via the CAN-BUS protocol. The ZCU converts control commands into CAN signals and sends them to the real-time simulator. The real-time simulator simulates the vehicle hardware response and returns vehicle status information via CAN-BUS.

[0050] This architecture enables a complete verification chain from test nodes to the intent control system and then to the actual vehicle hardware, verifying the correctness of the intent control system throughout the entire control chain.

[0051] like Figure 2 As shown, the test node deploys four sequentially connected functional modules: test case generation module, atomic service call verification module, vehicle status consistency verification module, and semantic verification module.

[0052] First, the test case generation module parses the intent control system's configuration file (intent.json), expands the synonyms based on the intent description using a large language model, and generates test cases according to different vehicle control functions and scenario types. Each test case provides a defined atomic service and parameter value as the expected result. The generated test cases are divided according to different scenarios. The atomic service call verification module has different branches and processing logic depending on the scenario. This module captures function call information from the intent control system logs and compares it with the expected results in the test cases. For single scenarios, it verifies a perfect match; for combined scenarios, it performs logical AND and OR operations to calculate the confidence level. If the atomic service call verification module passes, the vehicle state consistency verification module is executed. This module also has multiple branches depending on the scenario. It obtains the current actual vehicle control state through a RESTful interface and compares it with the function call output to verify a one-to-one correspondence. For combined scenarios, further state change logic verification is required. Before executing the vehicle control command, the current actual vehicle control state is obtained and compared with the state after the command is executed to verify whether the changes in vehicle control state before and after are logically consistent. Finally, we enter the semantic verification module. Different semantic verification branches are selected for different scenarios. Basic semantic verification uses a large language model to perform simple semantic comparisons, while enhanced semantic verification uses methods such as multi-round verification and context enhancement to improve accuracy. For the large language model, a model that excels in semantic recognition is selected. These four modules work together to realize a complete testing process from test case generation to multi-dimensional verification.

[0053] This invention constructs a complete end-to-end automated verification process from test case generation to state verification. By building a HIL test bench, it realizes a complete verification link from test node → intent control system → CCU → ZCU → real-time simulator. It uses a RESTful interface for business function verification, eliminating the need to additionally acquire CAN bus interface data, and achieving end-to-end automated verification from user input to vehicle state changes without manual intervention.

[0054] This invention provides automated generation, execution, and result analysis functions for test cases, supporting batch verification and automatic generation of verification reports. Verification results are categorized and statistically analyzed by scenario type, generating Excel-formatted verification reports, significantly improving verification efficiency and coverage.

[0055] like Figure 3 As shown, this describes the test case generation process and scenario classification method. Core inputs include: intent configuration file (intent.json), user command examples, expected vehicle control results, and atomic services invoked during vehicle control.

[0056] Test case generation is based on core input and automatically generates test cases. The generation process includes: parsing the intent configuration and extracting information such as intent name, description, and execution configuration; expanding the vocabulary of synonyms based on the intent description using a large language model to generate test cases not in the intent configuration; and generating the corresponding test case format according to the scenario type.

[0057] Test cases are categorized by scenario based on the expected vehicle control outcome, not by instruction type. The categories include:

[0058] Scenario 1: A single instruction for the vehicle control atomic service is expected to be uniquely determined, but the parameters are uncertain;

[0059] Example: Input "open the window a little", expected to call 'control_window', but the parameters are uncertain.

[0060] Scenario 2: A single, uniquely determined atomic command for vehicle control, with defined parameters;

[0061] Example: Input "Turn on red ambient light", expected to call 'control_ambient_light', with explicit parameters (red, A graphic, turn on).

[0062] Scenario 3: Two definite vehicle control atomic service commands are expected, with determined parameters;

[0063] Example: Input "Open the driver's side window 50%, open the passenger side window halfway", expected to call 'control_window' twice, parameters to be determined.

[0064] Scenario 4: Two definite vehicle control atomic service commands are expected, but the parameters are uncertain;

[0065] Example: Input "Open the window, adjust the seat", expected to call 'control_window' and 'control_seat', but the parameters are uncertain.

[0066] Scenario N: Expected multi-atomic service collaborative invocation, parameters are determined;

[0067] Example: Input "Turn on yellow ambient light, turn off light, turn on blue B graphic", expected to call 'control_ambient_light' multiple times, parameters to be determined.

[0068] Scenario N+1: Expected multi-atomic service collaborative invocation, with uncertain parameters;

[0069] Example: Input "Open the window, adjust the seat, turn on the yellow A-shaped ambient light; turn off the light; it's a bit dark and stuffy", expected to call multiple atomic services, parameters are uncertain.

[0070] This classification method can systematically cover various test scenarios, ensuring the comprehensiveness of the verification.

[0071] Different verification processes are designed for different scenario types. A three-step verification process is used for single scenarios: atomic service call verification → vehicle state consistency verification → basic semantic verification. A four-step verification process is used for combined scenarios: atomic service call verification (confidence calculation) → vehicle state consistency verification (one-to-one correspondence) → state change logic verification → enhanced semantic verification. The verification process is automatically selected based on the scenario complexity.

[0072] Figure 4 The verification process for Scenario 1 (a single instruction for the expected unique vehicle control atomic service with fixed parameters) is demonstrated. Figure 5 The verification process for scenario N+1 (expected multi-atomic service collaborative call, parameters are uncertain) is demonstrated.

[0073] Input Scenario 1: Uniquely identify the user command and the expected function to be called; Example: Input "Turn on the red ambient light", the expected function to be called is 'control_ambient_light', and the parameters are clear.

[0074] Step 1: Check the intent control system to see if the function call information of the large language model matches the expectation. If they match, proceed to the next step; otherwise, return failure.

[0075] Step 2: Call the RESTful interface to obtain the current actual vehicle control status and compare it with the function call output of the intent control system. If they match, proceed to the next step; otherwise, return failure.

[0076] Step 3: Compare the semantic consistency of user commands and intents with the control system response (basic semantic verification). If consistent, the verification passes; if inconsistent, it returns a failure.

[0077] Input scenario N+1: Multiple user commands and expected function calls; Example: Input "Open the window, adjust the seat, turn on the yellow A-shaped ambient light; turn off the light; it's a bit dark, a bit stuffy". Expected calls to multiple atomic services: 'control_window', 'control_seat', 'control_ambient_light', etc., with uncertain parameters.

[0078] Step 1: Compare the expected atomic service call with the function calls of the large language model recorded in the intent control system log. Combinatorial scenarios are expected to involve multiple atomic services. When comparing with the actual function calls of the intent control system, perform logical AND and logical OR operations, rather than simple exact matching. Calculate the confidence score of the match, considering partial matching, sequential matching, etc.; proceed to the next step if the confidence score is ≥50%, otherwise return failure if the confidence score is <50%.

[0079] Step 2: Call the RESTful interface to obtain the current actual vehicle control status and compare it with the function call output of the intent control system. If they match, proceed to the next step; otherwise, return failure.

[0080] Step 3: Compare the changes in vehicle control status before and after execution to verify if the changes are logical. Verify the reasonableness of the parameters of the relative command (e.g., does "open a little" actually increase the opening degree); verify the logic of the status changes (e.g., does the light status actually turn off after "turn off the lights"); verify the understanding of historical context (e.g., is "a bit dark, a bit stuffy" a reasonable adjustment based on the previous state). If the changes in vehicle control status before and after execution are logical, proceed to the next step; otherwise, return failure.

[0081] Step 4: Compare the semantic consistency of user commands and the large language model response commands recorded in the system's logs. Use enhanced semantic verification methods, including multi-round verification and context enhancement, to improve verification accuracy. Verification passes if consistent; otherwise, it fails.

[0082] The beneficial effects of this invention are that, compared with existing technologies, it solves the problem of uncertainty in intent control results and improves verification accuracy. By comprehensively judging the correctness of the intent control system through a multi-dimensional verification mechanism (semantic verification, atomic service verification, and state verification), it solves the problem that single semantic judgment or single vehicle control service judgment is not accurate enough. Compared with single verification methods, this invention can comprehensively evaluate the correctness of the system from multiple dimensions, significantly improving the accuracy and reliability of verification. Traditional test cases are directly divided based on vehicle control function modules. This invention, based on vehicle control function modules, innovatively divides them according to different scenarios, categorizing test scenarios into multiple scenarios according to the expected vehicle control results. It innovatively adopts a RESTful interface for business function verification, which is faster to learn than CAN bus interfaces such as python-can, eliminating the need to learn the CAN protocol and python-can library, directly verifying business functions, and reducing development threshold and learning costs. It achieves end-to-end automated verification from user input to vehicle state changes without manual intervention.

[0083] The applicant of this invention has provided a detailed description of the embodiments of the invention in conjunction with the accompanying drawings. However, those skilled in the art should understand that the above embodiments are merely preferred embodiments of the invention. The detailed description is only intended to help readers better understand the spirit of the invention and is not intended to limit the scope of protection of the invention. On the contrary, any improvements or modifications made based on the inventive spirit of the invention should fall within the scope of protection of the invention.

Claims

1. A multi-dimension intent control test system based on a large language model, characterized in that, This includes test nodes, intent control systems, ZCUs, and real-time simulators. The test node is responsible for executing test cases and verification processes; The intent control system is the target system being verified. The system contains a CCU, which is responsible for coordinating the entire vehicle-side intent control process. ZCU is responsible for vehicle control within the area; the real-time simulator is used to simulate the actual hardware environment of the vehicle and provide realistic vehicle status feedback. WebSocket connections are used for real-time bidirectional communication. Test nodes send user natural language commands to the intent control system via WebSocket and receive responses from the intent control system. The RESTful interface connection is used to query vehicle status information. The test node calls the getData interface of the intent control system through the RESTful interface to obtain the current vehicle control status and verify whether the atomic service has actually performed the vehicle control operation. The CCU and ZCU communicate to transmit control commands and status information. The ZCU and the real-time simulator communicate via the CAN-BUS protocol. The ZCU converts control commands into CAN signals and sends them to the real-time simulator. The real-time simulator simulates the vehicle hardware response and returns vehicle status information via CAN-BUS. The test node deploys four sequentially connected functional modules: test case generation module, atomic service call verification module, vehicle status consistency verification module, and semantic verification module. The test case generation module parses the configuration file of the intent control system and generates test cases according to different scenario types. The expected result of each test case is given by a specific atomic service and parameter value. The atomic service call verification module processes different logics for different scenarios. It captures function call information from the intent control system log and compares it with the expected results in the test cases. For a single scenario, it verifies a perfect match. For combined scenarios, it performs logical AND and logical OR operations and calculates the confidence level. If the atomic service call verification module passes, the vehicle state consistency verification module will be executed. This module has multiple branches depending on the scenario. It obtains the current actual vehicle control state through a RESTful interface and compares it with the function call output to verify the one-to-one correspondence. For combined scenarios, the logic of state changes will be verified. Before the vehicle control command is executed, the current actual vehicle control state is obtained and compared with the state after the vehicle control command is executed to verify whether the changes in the vehicle control state before and after are logical. Finally, we enter the semantic verification module, where different semantic verification branches are selected for different scenarios.

2. The multi-dimensional intent control testing system based on a large language model according to claim 1, characterized in that, Each intent controls a WebSocket connection and maintains an independent session history, supporting context-dependent instruction verification.

3. The multi-dimensional intent control testing system based on a large language model according to claim 1, characterized in that, The semantic verification branch of the semantic verification module includes: basic semantic verification, which uses a large language model to perform simple semantic comparison; and enhanced semantic verification, which uses multi-round verification and context enhancement methods to improve accuracy, and selects a large language model that is good at semantic recognition.

4. A multi-dimensional intent control testing method based on a large language model, based on the multi-dimensional intent control testing system based on a large language model as described in any one of claims 1-3, characterized in that, Including the following steps: (1) Parse the configuration file of the intent control system, expand the synonyms based on the intent description using the large language model, generate test cases according to different scenario types, and give the expected results of each test case with a definite atomic service and parameter value; (2) Verify atomic service calls, capture function call information from the intention control system log, compare it with the expected results in the test cases, verify a perfect match for a single scenario, and perform logical AND and logical OR operations for combined scenarios to calculate the confidence level; (3) If the atomic service call verification is successful, the vehicle state consistency verification is performed. Depending on the scenario, there are multiple branches. The current actual vehicle control state is obtained through the RESTful interface and compared with the function call output to verify the one-to-one correspondence. For combined scenarios, the state change logic verification is required. Before the vehicle control command is executed, the current actual vehicle control state is obtained and compared with the state after the vehicle control command is executed to verify whether the changes in the vehicle control state before and after are logical. (4) Finally, semantic verification is performed. Different semantic verification branches are selected for different scenarios. Basic semantic verification uses a large language model to perform simple semantic comparison. Enhanced semantic verification uses multi-round verification and context enhancement methods to improve accuracy. The large language model selected is a model that is good at semantic recognition.

5. The multi-dimensional intent control testing method based on a large language model according to claim 4, characterized in that, Test case generation steps: Test cases are automatically generated based on core inputs, which include intent configuration files, user command examples, expected vehicle control results, and vehicle control calls to atomic services. Parse the intent configuration to extract the intent name, description, and execution configuration information; expand the synonyms based on the intent description using a large language model to generate test cases not in the intent configuration; classify the test cases according to the expected vehicle control results and generate corresponding test case formats according to the scenario type.

6. The multi-dimensional intent control testing method based on a large language model according to claim 4, characterized in that, Single-scenario verification process: Step 1: Check the intent control system to see if the function call information of the large language model matches the expectation. If they match, proceed to the next step; otherwise, return failure. Step 2: Call the RESTful interface to obtain the current actual vehicle control status and compare it with the function call output of the intent control system to see if they are consistent; If the agreement is consistent, proceed to the next step; otherwise, return failure. Step 3: The semantic consistency between user commands and intents and the control system response is compared. If consistent, the verification passes; if inconsistent, the verification returns failure.

7. The multi-dimensional intent control testing method based on a large language model according to claim 4, characterized in that, Combined scenario verification process: Step 1: Compare the expected atomic service to be called with the function calls of the large language model recorded in the intent control system log; the combined scenario is expected to have multiple atomic services. When comparing with the actual function calls of the intent control system, perform logical AND and logical OR operations to calculate the confidence score of the match. If the confidence score is ≥50%, proceed to the next step; if the confidence score is <50%, return failure. Step 2: Call the RESTful interface to obtain the current actual vehicle control status and compare it with the function call output of the intent control system to see if they are consistent; If the agreement is consistent, proceed to the next step; otherwise, return failure. Step 3: Compare the changes in vehicle control status before and after execution to verify whether the changes in status are logical; verify whether the parameters of the relative command are reasonable; verify the logic of the changes in status; verify the understanding of historical context; if the changes in vehicle control status before and after execution are logical, proceed to the next step; if they are not logical, return failure. Step 4: Compare the semantic consistency between the user commands and the response commands of the large language model recorded in the intent control system log. Use an enhanced semantic verification method. If they are consistent, the verification passes; if they are inconsistent, the verification returns failure.

Images