A Test Case Generation Method and System Based on Large Language Model and Rule Collaboration

Abstract

This invention belongs to the field of vehicle software testing technology, specifically involving a test case generation method and system based on a large language model and rule collaboration. The method generates structured test requirements by parsing a requirement document and mapping it to a fault ontology model; it generates multiple sets of fault verification conditions using the large language model based on equivalence class partitioning and boundary value analysis; it retrieves reference test cases from a historical test case library using a multi-dimensional hybrid retrieval strategy; and it generates the pre-construction environment settings, serialized operation instructions, and expected responses for test cases in stages, combining verification conditions, reference test cases, and a standardized information library. Finally, it ensures test case quality through collaborative verification using the large language model and a rule engine. This invention effectively solves the problems of traditional methods, such as reliance on manual labor, low coverage, error susceptibility, and difficulty in reusing historical knowledge, significantly improving the automation, accuracy, and completeness of BMS test case generation.

Description

Technical Field

[0001] This invention belongs to the field of vehicle software testing technology, specifically relating to a test case generation method and system based on large language models and rule collaboration. Background Technology

[0002] As the core control unit of new energy vehicles and energy storage systems, the Battery Management System (BMS) is a crucial link in ensuring the safe and reliable operation of the system through testing and verification. Currently, the construction of BMS test cases mainly relies on the manual summarization and writing by test engineers, or parameterized generation based on fixed templates. Some automated methods use scripts or model-driven technologies to generate basic test scenarios by parsing keywords in requirements documents. These traditional methods improve the efficiency of test design to some extent, but their level of intelligence is limited, their automation level is low, and they heavily rely on the domain knowledge accumulated by engineers. Furthermore, existing technologies typically lack effective mechanisms for mining and reusing historical test case data, failing to form a systematic test knowledge management system.

[0003] However, existing technologies have significant drawbacks: First, manual coding is inefficient and prone to omissions, especially in effectively covering complex test scenarios and boundary conditions with multiple coupled parameters, resulting in insufficient test coverage. Second, template-based generation methods lack flexibility, struggle to adapt to dynamically changing test requirements, and are prone to logical conflicts or mismatched error codes. Furthermore, traditional methods have low utilization of historical test data, failing to achieve effective knowledge transfer and reuse, leading to a waste of test assets. These shortcomings make it difficult for existing technologies to meet the high-complexity, high-accuracy, and full lifecycle compliance requirements of BMS testing. Summary of the Invention

[0004] The purpose of this invention is to provide a test case generation method and system based on the collaboration of large language models and rules, so as to solve the problems of existing technologies where BMS test case generation relies on manual labor, has low coverage and is prone to errors, and cannot effectively reuse historical test knowledge.

[0005] The present invention achieves the above objectives through the following technical solutions:

[0006] Firstly, this invention proposes a test case generation method based on the collaboration of a large language model and rules, comprising the following steps:

[0007] S1. Respond to the BMS test case generation request, parse and extract the target parameter information from the request document, map the target parameter information to the predefined fault ontology model, and generate structured test requirement information; the target parameter information includes fault name, enabling condition, triggering condition, recovery condition, fault level, and fault manifestation;

[0008] S2. Receive the test requirement information, and based on the triggering condition and the recovery condition, generate multiple sets of fault verification conditions for verifying fault triggering and fault recovery using equivalence class partitioning and boundary value analysis.

[0009] S3. Based on the fault verification conditions, a preset multi-dimensional hybrid retrieval strategy is used to retrieve multiple historical test cases from the historical test case library as reference test cases.

[0010] S4. Combining the fault verification conditions, the reference test cases, and the preset standardized information database, the pre-environment settings, serialization operation instructions, and expected responses of the test cases are generated in stages using the large language model, and the BMS test cases are generated and output.

[0011] Furthermore, step S1 includes:

[0012] S1.1 Receive BMS test case generation request and obtain unstructured BMS test request document;

[0013] S1.2. Use natural language processing technology to perform semantic parsing on the request document, identify and extract key entity values ​​that characterize the target parameter information of the fault features;

[0014] S1.3 Classify the key entity values ​​and populate them into the corresponding fields of the fault ontology model used to describe the fault entities of the BMS and the relationships between their attributes, so as to form machine-readable structured test requirement information.

[0015] Furthermore, in step S2, the generation of multiple sets of fault verification conditions for verifying fault triggering and fault recovery using equivalence class partitioning and boundary value analysis includes:

[0016] Using a large language model, invalid equivalence class verification conditions, valid equivalence class verification conditions, and boundary disturbance class verification conditions are generated for continuous parameters in fault triggering or recovery conditions.

[0017] The invalid equivalence class verification condition is a parameter value that does not meet the fault triggering or recovery condition; the valid equivalence class verification condition is a parameter value that meets the fault triggering or recovery condition; and the boundary disturbance class verification condition is a parameter value that is offset at or near the fault triggering or recovery threshold boundary.

[0018] For scenarios involving multiple combined fault triggering conditions, a decision table method is used to generate verification conditions that cover all combinations of conditions in the scenario.

[0019] Furthermore, prior to step S3, the method further includes:

[0020] Multi-dimensional annotations are performed on BMS test cases of historical projects, including annotations on test type, test phase, test name, and test time;

[0021] The Sentence-BERT model is used to vectorize the labeled test cases in order to build a structured historical test case library.

[0022] Furthermore, step S3 includes:

[0023] S3.1. Based on the test item name and test stage to which the fault verification condition belongs, a preliminary screening is performed from the historical test case library to obtain a candidate test case set;

[0024] S3.2. Based on semantic vector similarity, the candidate test case set is finely screened, the cosine similarity between the fault verification condition and the semantic embedding vector of each historical test case in the candidate test case set is calculated, and historical test cases with similarity higher than a preset threshold are selected.

[0025] S3.3. Based on the time decay factor, the historical test cases after fine screening are weighted and sorted to select the test cases with more recent generation time;

[0026] S3.4 Filter the weighted sorted results based on domain tag matching degree, and select the N highest-ranked historical test cases from the filtered results as the reference test cases, where N is a natural number greater than 1, and the domain tags include vehicle platform and battery cell type.

[0027] Furthermore, step S4 includes:

[0028] S4.1 Based on the name of the current test item, retrieve the corresponding standardized information block from the preset standardized information library. The standardized information block includes standard pre-action, standard power-on operation command, standard fault manifestation and standard error code.

[0029] S4.2. Based on the large language model and the aforementioned standard pre-actions, generate the standardized pre-environment settings for the current test case;

[0030] S4.3. Combining the fault verification conditions and fault triggering logic, and referring to the operation sequence in the reference use case, use the large language model to generate time-sequential test operation steps to form the serialized operation instructions.

[0031] S4.4 Based on the generated serialized operation instructions, the standard fault manifestations and standard error codes, use the large language model to infer the corresponding expected response results and complete the construction of test cases.

[0032] Furthermore, the preset standardized information database includes:

[0033] A precondition template library is used to store standardized parameter templates related to test environment initialization;

[0034] A message protocol library for storing standardized simulation message templates based on CAN, LIN, or Ethernet protocols;

[0035] The fault feature knowledge base is used to store standardized fault performance descriptions, recovery performance descriptions, and fault level information that uniquely correspond to fault codes.

[0036] The information blocks in the standardized information database are associated and indexed through the fault names or fault codes in the fault ontology model.

[0037] Furthermore, before generating and outputting BMS test cases, the method further includes: performing semantic verification on the generated test cases using a large language model, including: verification of precondition matching degree, verification of consistency between operation steps and fault triggering logic, comparison of fault feature matching degree, and verification of single test item test case quantity constraints; performing deterministic verification on the test cases using a rule engine, including: verification of numerical compliance, verification of value rationality, and detection of conflicts between multi-level faults; and directly outputting test cases that pass the verification.

[0038] Test cases that fail verification are manually reviewed and intervened upon, and then either corrected and adopted or eliminated.

[0039] Secondly, this invention proposes a test case generation system based on a large language model and rule collaboration, used to implement the steps of the test case generation method described above. The system includes:

[0040] The requirement parsing module is used to respond to BMS test case generation requests, parse and extract the target parameter information from the request document, map the target parameter information to a predefined fault ontology model, and generate structured test requirement information. The target parameter information includes fault name, enabling conditions, triggering conditions, recovery conditions, fault level, and fault manifestation.

[0041] The condition generation module is used to receive the test requirement information and, based on the triggering and recovery conditions therein, generate multiple sets of fault verification conditions using a large language model through equivalence class partitioning and boundary value analysis.

[0042] The test case retrieval module is used to retrieve multiple historical test cases from the historical test case library as reference test cases based on the fault verification conditions and using a preset multi-dimensional hybrid retrieval strategy.

[0043] The test case generation module is used to combine the fault verification conditions, the reference test cases, and the preset standardized information library, and use a large language model to generate the pre-environment settings, serialization operation instructions and expected responses of test cases in stages, and generate and output BMS test cases.

[0044] The collaborative verification module is used to verify and filter the generated BMS test cases using a large language model and rule engine;

[0045] The test case library is used to store the historical test cases, the preset standardized information library, and the finally generated BMS test cases.

[0046] The beneficial effects of this invention are as follows:

[0047] This invention, based on a fault ontology model and natural language processing technology, achieves automatic parsing and structured mapping of unstructured requirements, overcoming the shortcomings of traditional methods that heavily rely on human experience and suffer from incomplete information extraction. Through equivalence class partitioning, boundary value analysis, and decision table methods, it automatically generates multi-dimensional test conditions, effectively covering single-parameter boundaries and multi-parameter combination scenarios, solving the problem of incomplete test case coverage in complex fault scenarios. A multi-dimensional hybrid retrieval strategy enables accurate reuse of historical test cases, avoiding knowledge waste and redundant development. Combining a standardized information database with a phased generation strategy ensures the consistency of test case logic and the temporality of operations. Finally, through a rule engine and a large language model collaborative verification mechanism, it achieves automated verification of numerical compliance, logical conflicts, and fault feature matching, significantly improving the accuracy and reliability of test cases. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating a test case generation method based on the collaboration of a large language model and rules, as shown in one embodiment of this application.

[0049] Figure 2 This is another flowchart illustrating a test case generation method based on the collaboration of a large language model and rules, as shown in one embodiment of this application.

[0050] Figure 3 This is a flowchart illustrating step S1 of a test case generation method based on large language model and rule collaboration, as shown in one embodiment of this application.

[0051] Figure 4 This is a flowchart illustrating step S3 of a test case generation method based on the collaboration of a large language model and rules, as shown in one embodiment of this application.

[0052] Figure 5 This is a flowchart illustrating step S4 of a test case generation method based on large language model and rule collaboration, as shown in one embodiment of this application.

[0053] Figure 6 This is a schematic diagram illustrating the structure of a test case generation system based on the collaboration of a large language model and rules, as shown in another embodiment of this application. Detailed Implementation

[0054] The present application will now be described in further detail with reference to the accompanying drawings. It should be noted that the following specific embodiments are only used to further illustrate the present application and should not be construed as limiting the scope of protection of the present application. Those skilled in the art can make some non-essential improvements and adjustments to the present application based on the above application content.

[0055] It is worth noting that although some existing research attempts to improve testing efficiency using model-driven or script-based methods, these methods still suffer from insufficient flexibility and poor maintainability, and cannot effectively understand and process unstructured natural language requirements documents. How to achieve intelligent and automated generation of test cases and ensure high quality has become a pressing technical challenge in this field.

[0056] Example 1

[0057] Please combine Figure 1 and Figure 2 To address the aforementioned issues, a specific embodiment of this application proposes a test case generation method based on the collaboration of a large language model and rules. The main steps of the method include:

[0058] S1. Respond to the BMS test case generation request, parse and extract the target parameter information (such as "discharge cell overvoltage level 1 trigger threshold Vmax≥3700mV") from the request document, map the target parameter information to the predefined fault ontology model, and generate structured test requirement information; the target parameter information includes fault name, enabling condition, triggering condition, recovery condition, fault level and fault manifestation.

[0059] Specifically, the system receives user-submitted test requirement documents (such as Word and PDF formats) and automatically extracts key parameters such as fault name, enabling conditions, triggering conditions, recovery conditions, fault level, and fault manifestation by integrating the key entity recognition and relation extraction capabilities of Natural Language Processing (NLP) technology. These parameters are then structured and populated into a predefined fault ontology model, forming machine-readable standardized data objects, providing a consistent and unambiguous input basis for subsequent processing.

[0060] S2. Receive test requirement information, and based on triggering and recovery conditions, use equivalence class partitioning and boundary value analysis to generate multiple sets of fault verification conditions for verifying fault triggering and fault recovery.

[0061] This step leverages the logical reasoning and test design capabilities of the large language model to automatically generate invalid equivalence classes, valid equivalence classes, and boundary disturbance test conditions for continuous parameters (such as voltage, current, and temperature), ensuring comprehensive coverage of normal ranges, abnormal conditions, and critical values. For multi-condition combination scenarios (such as "temperature > 50℃ and SOC < 20%), a decision table method is used to automatically enumerate all condition combinations and generate corresponding test condition sets to verify the system behavior under complex logic.

[0062] S3. Based on the fault verification conditions, a pre-defined multi-dimensional hybrid retrieval strategy is used to retrieve multiple historical test cases from the historical test case library as reference test cases. The retrieval process comprehensively considers semantic similarity, temporal proximity, and domain relevance: First, similarity is calculated based on semantic vectors to initially screen relevant test cases; then, a time-decay factor is introduced to prioritize recent test cases to ensure the timeliness of the testing strategy; finally, matching and filtering are performed based on domain tags such as vehicle model and battery cell type, ultimately outputting several historical test cases most relevant to the current test scenario, providing empirical reference for test generation.

[0063] S4. Combining fault verification conditions, reference test cases, and a pre-defined standardized information library, the system utilizes a large language model to generate test case pre-environment settings, serialized operation instructions, and expected responses in stages, generating and outputting BMS test cases. During the generation process, pre-defined pre-actions, message templates, fault manifestations, and other resources are retrieved from the standardized information library to ensure the standardization and consistency of basic operations. Then, based on test conditions and fault logic, operation steps with temporal relationships are dynamically combined and generated. Finally, based on the operation sequence and fault feature library, the corresponding expected response results are derived and generated, completing the automated construction of complete test cases.

[0064] It is understood that the "fault ontology model" refers to a structured data framework used to standardize the description of BMS fault entities, attributes and their relationships, which usually includes fields such as fault name, enabling conditions, triggering / recovery conditions, and fault level; while the "multi-dimensional hybrid retrieval strategy" refers to a method that integrates multi-dimensional information such as semantic similarity, temporal factors and domain tags to accurately retrieve similar use cases from the historical use case library.

[0065] In this application, the "large language model" specifically refers to a generative artificial intelligence model that has undergone domain adaptation and hinting engineering optimization and is capable of understanding BMS testing terminology and fault diagnosis logic. Its specific functions and applications in this application are as follows:

[0066] First, in the requirements analysis and structuring (S1) phase, the large language model, relying on its powerful natural language understanding capabilities, is used to perform semantic parsing on the unstructured BMS test requirements document. It can accurately identify and extract key entities and parameters representing fault characteristics in the text (such as "3700mV" and "duration ≥4000ms"), and understand the logical relationships between these parameters (such as the threshold inequality relationship between triggering conditions and recovery conditions), providing accurate input for subsequent mapping to the fault ontology model.

[0067] Secondly, in the test condition generation (S2) phase, the core role of the large language model is to perform logical reasoning and test design based on test theory. Based on the input structured requirement information, the model automatically applies test case design methods such as equivalence class partitioning and boundary value analysis to generate invalid equivalence classes, valid equivalence classes, and boundary disturbance test conditions. For scenarios with multiple condition combinations, such as "temperature > 50℃ and SOC < 20%", the model can automatically derive and generate test items covering all condition combinations based on the decision table method, thereby achieving complete coverage of complex fault triggering logic.

[0068] Finally, in the test case generation (S4) phase, the large language model combines the fault verification conditions, the retrieved reference test cases, and modules in the standardized information base (such as standard pre-actions and standard fault manifestations) to generate a sequence of test operation steps (Test Steps.Action) with temporal relationships, and infers the corresponding expected results (Test Result) based on the fault triggering and release logic.

[0069] Optionally, before generating and outputting BMS test cases, the method further includes: performing semantic verification on the generated test cases using a large language model, including: verification of precondition matching degree, verification of consistency between operation steps and fault triggering logic, comparison of fault feature matching degree, and verification of single test item test case quantity constraints; performing deterministic verification on the test cases using a rule engine, including: verification of numerical compliance, verification of value rationality, and detection of conflicts between multi-level faults; directly outputting test cases that pass the verification; and manually reviewing and intervening on test cases that fail the verification, correcting them before adoption or elimination.

[0070] In practice, the steps for verifying test cases mentioned above include the following:

[0071] (1) Semantic verification of large models

[0072] Based on the prompt-based engineering construction verification template, the user inputs the original requirements, test conditions, and generated test case triples. LLM's logical reasoning capabilities then complete the following: ① Precondition verification: Checking the matching degree between the test environment configuration and the trigger threshold; ② Operation step verification: Verifying the consistency between the test steps and the fault triggering logic; ③ Fault feature comparison: Confirming the matching degree between the expected results and preset features through comparison of messages, fault codes, fault trigger behaviors, and fault release behaviors; ④ Combination constraints: Controlling the upper limit of the number of test cases for a single test item (e.g., SOC estimation deviation test ≤ 5 test cases).

[0073] (2) Deterministic verification of the rule engine

[0074] Construct a multi-dimensional verification rule base to implement hard constraints: ① Numerical compliance: Verify that the parameter values ​​meet the inequality constraints (e.g., Vmax=3700mV>recovery threshold 3650mV); ② Value rationality: Check that the threshold and time interval for fault triggering and recovery meet the standard requirements; ③ Conflict detection: Identify the correlation and boundaries between multiple levels of faults in the same test item to avoid conflicts.

[0075] (3) Manual intervention verification

[0076] After collaborative verification using a large model and rules, the semantic verification of the large model and the deterministic judgment of the rule engine complement each other, filtering out all problematic generated test cases. At this point, manual review is performed. Test cases with minor deviations are manually modified and adopted, while severely conflicting or erroneous test cases are automatically eliminated, ultimately forming a valid and standardized test case set.

[0077] Table 1 below shows an example of BMS test cases generated collaboratively based on a large language model and rules. It includes core elements such as test item name, application conditions, pre-actions, test operations, and expected results. The table verifies the feasibility of the method through a specific scenario (such as a discharge cell undervoltage level 2 trigger / non-trigger), demonstrating the structured design and timing logic control capabilities of the automated test case generation. The generated test cases cover fault trigger thresholds, recovery conditions, and multi-level fault responses, and are completely consistent with the phased generation strategy, standardized information database call, and verification mechanism in the invention, providing a highly accurate and complete empirical reference for BMS testing.

[0078] Table 1

[0079] ;

[0080] ;

[0081] Based on the above main scheme, the automated generation method in this application also includes the following sub-steps:

[0082] Please combine Figure 3Preferably, step S1 includes:

[0083] S1.1 Receive BMS test case generation request. This request is usually submitted by the test engineer through the integration interface or API interface to obtain the unstructured BMS test request document. The content contains test requirements described in natural language, such as "When the single cell voltage is lower than 2800mV for 4 seconds in the discharge state, trigger the secondary undervoltage fault".

[0084] S1.2. Natural Language Processing (NLP) techniques are used to semantically parse the request document, identifying and extracting key entity values ​​representing fault characteristics and target parameter information; including but not limited to: entities representing fault type (e.g., "overvoltage," "undervoltage"), parameter entities (e.g., "voltage," "temperature," "SOC"), numerical entities (e.g., "2800," "3700"), unit entities (e.g., "mV," "℃," "ms"), and logical relation entities (e.g., "and," "or," "greater than," "continuous"). This process converts unstructured text into discrete but meaningful data units.

[0085] S1.3 Classify the key entity values ​​and populate them into the corresponding fields of the fault ontology model used to describe the fault entities of the BMS and the relationships between their attributes, so as to form machine-readable structured test requirement information.

[0086] As an example, the execution process of the test case is further illustrated using the "discharge cell undervoltage level 2" fault test. Under the enabled condition, the trigger threshold is Vmin≤2800mV and the duration is≥4000ms, which triggers a level 4 fault; the recovery threshold is Vmin≥2900mV and the duration is≥4000ms, and the fault recovers to level 3. The message information shows bms_state as discharge state 0x4 and fault state 0x5. In the level 3 fault, both discharge power and feedback power linearly decrease to 75% of the map table (3KW per second); in the level 4 fault, both decrease to a limp power of 15KW (3KW per second). During the test operation, this test case can be verified by simulating voltage changes, etc.

[0087] Preferably, in step S2, equivalence class partitioning and boundary value analysis are used to generate multiple sets of fault verification conditions for verifying fault triggering and fault recovery. This includes: using a large language model, generating invalid equivalence class verification conditions, valid equivalence class verification conditions, and boundary disturbance class verification conditions for continuous parameters in the fault triggering or recovery conditions; wherein, invalid equivalence class verification conditions are parameter values ​​that do not meet the fault triggering or recovery conditions; valid equivalence class verification conditions are parameter values ​​that meet the fault triggering or recovery conditions; boundary disturbance class verification conditions are parameter values ​​that are offset at or near the fault triggering or recovery threshold boundary; for multi-condition combination fault triggering scenarios, a decision table method is used to generate verification conditions covering all condition combinations in the scenario.

[0088] Specifically, test item conditions are generated based on the requirements information. Following the software testing methods specified in relevant national standards for BMS, the triggering and recovery conditions of the test requirements are analyzed to obtain fault verification conditions. Taking the discharge cell overvoltage level 1 trigger threshold Vmax≥3700mV as an example, conditions are generated from different perspectives using a large model. An improved boundary value analysis method is used for equivalence class partitioning optimization, generating invalid equivalence classes (e.g., Vmax = 3699mV), valid equivalence classes (e.g., Vmax = 3700mV, 3750mV), and boundary disturbance classes (e.g., Vmax = 3700±1mV). For multi-condition triggering scenarios, a decision table method is used to achieve combinational logic coverage and generate full combinational test items. The recovery threshold is also obtained in this way to obtain unreleased and released condition test cases.

[0089] As an example, as shown in Table 2 below, for a trigger threshold Vmin≤2800mV and a duration≥4000ms, the following test conditions are generated:

[0090] Table 2

[0091] ;

[0092] As an example, as shown in Table 3 below, for a recovery threshold Vmin ≥ 2900mV and a duration ≥ 4000m, the following test conditions are generated:

[0093] Table 3

[0094] ;

[0095] Preferably, before step S3, the method further includes: annotating the BMS test cases of historical projects in multiple dimensions, including the test type (such as functional test, fault injection test, endurance test, etc.), test phase (such as unit test, integration test, system test, etc.), test name (such as "discharge overvoltage protection test", "charging low temperature self-recovery test", etc.) and test time (time stamp of test case creation or last modification); and using the Sentence-BERT semantic vectorization model to vectorize the annotated test cases in order to build a structured historical test case library.

[0096] Specifically, the text content of each test case (including key text fields such as test name, operation steps, and expected results) is input into a pre-trained Sentence-BERT model to generate a high-dimensional semantic embedding vector (e.g., 1024 dimensions). This vector can numerically represent the deep semantic information of the test case, making semantically similar test cases closer in the vector space. Finally, all vectorized historical test cases are associated and stored with their multi-dimensional annotation information to construct a structured historical test case library. This preprocessing step is a prerequisite for achieving accurate test case retrieval, ensuring that the most relevant reference test cases can be quickly located from historical data.

[0097] Please combine Figure 4 Preferably, step S3 includes:

[0098] S3.1. Based on the fault verification conditions (e.g., "discharge cell undervoltage level 2") and the test phase (e.g., "discharge fault test"), a preliminary screening is performed from the historical test case library to obtain a candidate test case set. Using the multi-dimensional metadata (e.g., test type, test phase, etc.) pre-annotated to the historical test cases, obviously irrelevant test cases are quickly filtered out to narrow the search scope and form an initial candidate test case set.

[0099] S3.2. Based on semantic vector similarity, the candidate test case set is finely screened. The cosine similarity between the fault verification condition and the semantic embedding vector of each historical test case in the candidate test case set is calculated, and historical test cases with similarity higher than a preset threshold are selected. Only historical test cases with similarity higher than the threshold are retained to ensure that the retrieved test cases are highly relevant to the current requirements in terms of functional intent and test logic.

[0100] S3.3. Based on the time decay factor, the historical test cases after fine screening are weighted and sorted, and the test cases generated more recently are selected.

[0101] S3.4. Filter the weighted ranking results based on domain label matching. Select the top N historical test cases from the filtered results as reference test cases, where N is a natural number greater than 1. Domain labels include vehicle platform and battery cell type. Specifically, based on the vehicle platform (e.g., SUV, sedan) and battery cell type (e.g., lithium iron phosphate, ternary lithium) labels specific to the current project, filter test cases from the ranking list that have a completely matching or highly compatible domain configuration. Finally, select the top N historical test cases from the filtered results as reference test cases for generating new test cases using the large language model.

[0102] Please combine Figure 5 Preferably, step S4 includes:

[0103] S4.1 Based on the name of the current test item (e.g., "Discharge cell undervoltage level 2"), retrieve the corresponding standardized information block from the preset standardized information library. This information block serves as a reusable basic component and includes: standard pre-action (e.g., "System power-on +12V", "Main relay disconnected for more than 1s"), standard power-on operation instructions (e.g., CANoe simulation message template "0x460:01 00 00 00 00 00 00 00"), standard fault manifestations (e.g., "Discharge power linearly drops to 75% of the map table"), and standard error codes (e.g., "Bms_ErrorCode=14").

[0104] S4.2 Based on the large language model and standard pre-actions, generate standardized pre-environment settings for the current test case; the large language model parameterizes and logically combines standard actions according to the specific context of the current test item (such as the required initial voltage and temperature values), generating specific and executable environment preparation steps such as "set the battery pack temperature to 25℃; set the individual cell voltage to 3.3V; meet the pre-charge conditions".

[0105] S4.3. Combining fault verification conditions (such as "Vmin=2800mV, duration=4000ms") with fault triggering logic, and referring to the operation sequences of similar scenarios in the reference test cases, use a large language model to generate time-sequential test operation steps to form serialized operation instructions; for example: "1. Set ON position to valid: KeyON=ON, low voltage wakes up BMS; 2. Confirm BMS enters discharge state bms_state=0x4; 3. Set Vmin=2800mV, duration=4000ms; 4. Listen and confirm whether relevant messages are sent correctly", thus forming a complete and automatically executable serialized operation instruction.

[0106] S4.4. Based on the generated serialized operation instructions, standard fault manifestations, and standard error codes, use a large language model to infer the corresponding expected response results, thus completing the construction of the test case. For example: "1. High voltage successfully applied to BMS; 2. Trigger discharge cell low voltage 2 fault; 3. bms_state=0x4, alarm handling; discharge power reduced to limp power 15KW (3KW per second); 4. Bms_ErrorCode=14, Bms_ErrorLevel=level4". At this point, a complete BMS test case, including pre-setting environment, serialized operation instructions, and expected response results, is constructed.

[0107] Table 4 below illustrates (taking the discharge cell overvoltage level 1 fault triggering test as an example) the following steps: The standardized information library and test item names preset in S1 are input into the large language model to generate pre-test actions (i.e., steps S4.1 and S4.2); the application conditions generated in S2, the reference test cases selected in S3, and the pre-test actions in S4.2 are input into the large language model to generate serialized test operation instructions (i.e., step S4.3); the structured test requirement information in S1, the standardized information blocks in S4.1, and the test operations in S4.3 are input into the large language model to generate the corresponding expected test results (i.e., step S4.4). This embodiment fully presents the interaction relationship between the modules of the test case generation system, verifies the technical effect of the phased generation strategy and standardized information library calling mechanism of this invention, and achieves fully automated generation of test item names, pre-test environment settings, serialized operation instructions, and expected response results. The generated test cases meet the usability requirements in terms of logical coherence, operational timing accuracy, and result verifiability.

[0108] Table 4

[0109] ;

[0110] Preferably, the pre-defined standardized information base includes: a precondition template library for storing standardized parameter templates related to test environment initialization; a message protocol library for storing standardized simulation message templates based on CAN, LIN, or Ethernet protocols; and a fault feature knowledge base for storing standardized fault performance descriptions, recovery performance descriptions, and fault level information that uniquely correspond to fault codes. Among these, the information blocks in the standardized information base are associated and indexed through fault names or fault codes in the fault ontology model.

[0111] According to the above embodiments, this invention achieves automated generation of BMS test cases based on a collaborative mechanism of a large language model and a rule engine. First, unstructured requirement documents are parsed using natural language processing technology to extract key parameters and map them to a predefined fault ontology model, forming structured test requirements. Then, equivalence class partitioning and boundary value analysis are applied using the large language model to generate multiple sets of fault verification conditions covering normal, abnormal, and boundary scenarios. A multi-dimensional hybrid retrieval strategy is combined to accurately match reference test cases from a historical test case library, integrating a pre-defined standardized information library (including preconditions, message protocols, and fault characteristic knowledge) to construct the pre-setting environment, time-sequential operation instructions, and expected response results of the test cases in stages. Finally, semantic verification using the large language model and deterministic verification using the rule engine ensures the accuracy, compliance, and logical consistency of the generated test cases.

[0112] Example 2

[0113] Please combine Figure 6 Another embodiment of this application proposes a test case generation system based on large language model and rule collaboration, which is used to implement the steps of the test case generation method in Embodiment 1. The system includes a requirement parsing module, a condition generation module, a test case retrieval module, a test case generation module, and a collaborative verification module.

[0114] The system comprises the following modules: The Requirements Parsing module responds to BMS test case generation requests, parses and extracts target parameter information from the request document, maps this information to a predefined fault ontology model, and generates structured test requirements. Target parameter information includes fault name, enabling conditions, triggering conditions, recovery conditions, fault level, and fault manifestation. The Condition Generation module receives the test requirements and, based on the triggering and recovery conditions, uses a large language model to generate multiple sets of fault verification conditions through equivalence class partitioning and boundary value analysis. The Test Case Retrieval module retrieves multiple historical test cases from the historical test case library as reference test cases based on the fault verification conditions and employs a pre-defined multi-dimensional hybrid retrieval strategy. The Test Case Generation module combines the fault verification conditions, reference test cases, and a pre-defined standardized information library, using a large language model to generate the pre-set environment, serialized operation instructions, and expected responses for test cases in stages, generating and outputting BMS test cases. The Collaborative Verification module uses a large language model and a rule engine to verify and filter the generated BMS test cases.

[0115] The system also includes a built-in test case library, which stores historical test cases, a pre-set standardized information library, and the final generated BMS test cases.

[0116] For specific limitations regarding the test case generation system based on large language models and rule collaboration, please refer to the limitations of the test case generation method based on large language models and rule collaboration mentioned above, which will not be repeated here. It should be noted that each module in the above generation system corresponds to a step in implementing the above generation method. Multiple modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1 above.

[0117] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of any of the above-described test case generation methods based on large language models and rule collaboration.

[0118] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the steps of the test case generation method based on the large language model and rule collaboration in the above embodiments.

[0119] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0120] In addition, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0121] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A test case generation method based on large language model and rule collaboration, characterized in that, Includes the following steps: S1. Respond to the BMS test case generation request, parse and extract the target parameter information in the request document, map the target parameter information to the predefined fault ontology model, and generate structured test requirement information. The target parameter information includes fault name, enabling condition, triggering condition, recovery condition, fault level, and fault manifestation; S2. Receive the test requirement information, and based on the triggering condition and the recovery condition, generate multiple sets of fault verification conditions for verifying fault triggering and fault recovery using equivalence class partitioning and boundary value analysis. In step S2, the generation of multiple sets of fault verification conditions for verifying fault triggering and fault recovery using equivalence class partitioning and boundary value analysis includes: Using a large language model, invalid equivalence class verification conditions, valid equivalence class verification conditions, and boundary disturbance class verification conditions are generated for continuous parameters in fault triggering or recovery conditions. The invalid equivalence class verification condition is a parameter value that does not meet the fault triggering or recovery condition; the valid equivalence class verification condition is a parameter value that meets the fault triggering or recovery condition; and the boundary disturbance class verification condition is a parameter value that is offset at or near the fault triggering or recovery threshold boundary. For scenarios involving multiple combined fault triggering conditions, a decision table method is used to generate verification conditions that cover all combinations of conditions in the scenario. Multi-dimensional annotation of BMS test cases for historical projects, including annotation of test type, test phase, test name and test time; The Sentence-BERT model is used to vectorize the labeled test cases in order to build a structured historical test case library. S3. Based on the fault verification conditions, a preset multi-dimensional hybrid retrieval strategy is used to retrieve multiple historical test cases from the historical test case library as reference test cases. Step S3 includes: S3.

1. Based on the test item name and test stage to which the fault verification condition belongs, a preliminary screening is performed from the historical test case library to obtain a candidate test case set; S3.

2. Based on semantic vector similarity, the candidate test case set is finely screened, the cosine similarity between the fault verification condition and the semantic embedding vector of each historical test case in the candidate test case set is calculated, and historical test cases with similarity higher than a preset threshold are selected. S3.

3. Based on the time decay factor, the historical test cases after fine screening are weighted and sorted to select the test cases with more recent generation time; S3.4 Filter the weighted sorted results based on domain tag matching degree, and select the N highest-ranked historical test cases from the filtered results as the reference test cases, where N is a natural number greater than 1, and the domain tags include vehicle platform and battery cell type; S4. Combining the fault verification conditions, the reference test cases, and the preset standardized information database, the pre-environment settings, serialization operation instructions, and expected responses of the test cases are generated in stages using the large language model, and the BMS test cases are generated and output.

2. The test case generation method based on large language model and rule collaboration according to claim 1, characterized in that, Step S1 includes: S1.1 Receive BMS test case generation request and obtain unstructured BMS test request document; S1.

2. Use natural language processing technology to perform semantic parsing on the request document, identify and extract key entity values ​​that characterize the target parameter information of the fault features; S1.3 Classify the key entity values ​​and populate them into the corresponding fields of the fault ontology model used to describe the fault entities of the BMS and the relationships between their attributes, so as to form machine-readable structured test requirement information.

3. The test case generation method based on large language model and rule collaboration according to claim 1, characterized in that, Step S4 includes: S4.1 Based on the name of the current test item, retrieve the corresponding standardized information block from the preset standardized information library. The standardized information block includes standard pre-action, standard power-on operation command, standard fault manifestation and standard error code. S4.

2. Based on the large language model and the aforementioned standard pre-actions, generate the standardized pre-environment settings for the current test case; S4.

3. Combining the fault verification conditions and fault triggering logic, and referring to the operation sequence in the reference use case, use the large language model to generate time-sequential test operation steps to form the serialized operation instructions. S4.4 Based on the generated serialized operation instructions, the standard fault manifestations and standard error codes, use the large language model to infer the corresponding expected response results and complete the construction of test cases.

4. The test case generation method based on large language model and rule collaboration according to claim 3, characterized in that, The preset standardized information database includes: A precondition template library is used to store standardized parameter templates related to test environment initialization; A message protocol library for storing standardized simulation message templates based on CAN, LIN, or Ethernet protocols; The fault feature knowledge base is used to store standardized fault performance descriptions, recovery performance descriptions, and fault level information that uniquely correspond to fault codes. The information blocks in the standardized information database are associated and indexed through the fault names or fault codes in the fault ontology model.

5. The test case generation method based on large language model and rule collaboration according to claim 1, characterized in that, Before generating and outputting BMS test cases, the method further includes: performing semantic verification on the generated test cases using a large language model, including: verification of precondition matching degree, verification of consistency between operation steps and fault triggering logic, comparison of fault feature matching degree, and verification of single test item test case quantity constraints; performing deterministic verification on the test cases using a rule engine, including: verification of numerical compliance, verification of value rationality, and detection of conflicts between multi-level faults; and directly outputting test cases that pass the verification. Test cases that fail verification are manually reviewed and intervened upon, and then either corrected and adopted or eliminated.

6. A test case generation system based on large language model and rule collaboration, used to implement the steps of the test case generation method as described in any one of claims 1-5, characterized in that, The system includes: The requirement parsing module is used to respond to BMS test case generation requests, parse and extract the target parameter information from the request document, map the target parameter information to a predefined fault ontology model, and generate structured test requirement information. The target parameter information includes fault name, enabling conditions, triggering conditions, recovery conditions, fault level, and fault manifestation. The condition generation module is used to receive the test requirement information and, based on the triggering and recovery conditions therein, generate multiple sets of fault verification conditions using a large language model through equivalence class partitioning and boundary value analysis. The test case retrieval module is used to retrieve multiple historical test cases from the historical test case library as reference test cases based on the fault verification conditions and using a preset multi-dimensional hybrid retrieval strategy. The test case generation module is used to combine the fault verification conditions, the reference test cases, and the preset standardized information library, and use a large language model to generate the pre-environment settings, serialization operation instructions and expected responses of test cases in stages, and generate and output BMS test cases. The collaborative verification module is used to verify and filter the generated BMS test cases using a large language model and rule engine; The test case library is used to store the historical test cases, the preset standardized information library, and the finally generated BMS test cases.

Images