Intelligent automobile test case automatic generation and evaluation system, device and product

Abstract

The invention provides an intelligent automobile test case automatic generation and evaluation system, device and product, and relates to the technical field of intelligent automobile test.According to the system, domain knowledge support is provided through a test knowledge base, and a demand understanding module is combined with a fine-tuning large model to deeply analyze demands; function points are extracted, and hidden scenes such as sensor faults and multi-system cooperation are identified; the test case generation module is based on a special prompt template and few-sample learning, a complete case containing elements such as purposes, steps and data is generated by fusing a vehicle model configuration library, and the specialty and the performability are ensured; and the evaluation module realizes objective quality evaluation through multi-dimensional quantitative scoring of demand coverage rate, scene integrity, risk coverage degree, normalization and the like. All the modules cooperate to realize intelligent generation and system evaluation of test cases, and the automation level, coverage range and reliability of intelligent automobile testing are improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent vehicle testing technology, and in particular to an automatic generation and evaluation system, device and product for intelligent vehicle test cases. Background Technology

[0002] In the R&D of intelligent vehicles, test case design is a crucial step in ensuring the functional safety and reliability of the system. Traditional test case generation mainly relies on manual writing, with experience-driven test design based on requirements documents. This approach suffers from insufficient test coverage, low efficiency, and inconsistent quality. Due to the complexity of intelligent vehicle functions, involving highly integrated systems such as multi-sensor fusion, autonomous driving algorithms, and vehicle-to-everything (V2X) communication, requirements documents often contain numerous implicit scenarios and boundary conditions. Manual analysis is prone to misunderstandings or omissions, resulting in incomplete coverage of key test scenarios, particularly weak defect detection capabilities in anomaly handling, multi-system interactions, and security boundaries. Although some automated testing tools have attempted to generate test cases using rule engines or model-driven methods, they rely on preset templates and fixed logic, lacking deep semantic understanding of natural language requirements, making it difficult to identify implicit requirements, and limiting their ability to adapt to new scenarios. Summary of the Invention

[0003] The purpose of this invention is to provide an automatic generation and evaluation system, device and product for intelligent vehicle test cases, in order to solve one or more technical problems existing in the prior art, or at least provide a beneficial option or create conditions.

[0004] The solution to the technical problem of this invention is: This invention provides an automatic generation and evaluation system for intelligent vehicle test cases, comprising: The test knowledge base is configured to store automotive testing terminology, test design patterns, historical defect cases, and sensor configurations and control logic for different vehicle models. The requirements understanding module is configured to: combine the test knowledge base and use a pre-trained large language model that has been fine-tuned with corpus in the automotive testing field to perform semantic parsing on the requirements document of intelligent vehicle functions, and extract the set of function points, implicit test requirements and functional dependencies. The implicit testing requirements include sensor fault handling logic and multi-system collaborative scenarios. The test case generation module is configured to generate test case sets based on automotive-specific prompt templates and few-shot learning technology, combined with a vehicle configuration library. The test case set includes at least the test objective, preconditions, executable test steps, expected results, and test data. The test case evaluation module is configured to: perform multi-dimensional quality evaluation on the test case set, calculate the requirement coverage, scenario integrity score, risk coverage and test case standardization score, and integrate the scores of each dimension according to preset weights to obtain a comprehensive quality score.

[0005] Furthermore, the requirement understanding module is also configured as follows: The multi-task learning framework simultaneously performs test element identification, test type classification, and test priority prediction. Test element identification is used to extract test objects, test conditions, and expected results from the requirements document. Test type classification is used to determine whether the test type corresponding to the requirement is functional testing, performance testing, or exception testing. Test priority prediction is used to predict test priorities based on the importance and security level of the requirement.

[0006] Furthermore, in the test case generation module: The automotive testing-specific prompt template includes a description of functional requirements, test type, sensor configuration and control logic of the target vehicle model, and generation requirements; The few-shot learning technique retrieves historical test cases similar to the current requirement from the test knowledge base as examples, embeds the examples into the prompt template, and guides the large language model to generate test cases that conform to domain specifications.

[0007] Furthermore, in the test case evaluation module: The requirement coverage is obtained by identifying atomic requirement points in the requirement document through semantic analysis and calculating the ratio of the number of atomic requirement points covered by the test case set to the total number of atomic requirement points. The scenario integrity score is calculated by weighting the coverage of normal scenarios, boundary scenarios, abnormal scenarios, failure scenarios, and multi-system interaction scenarios according to preset weights. The risk coverage is obtained by weighting test cases that cover high-risk scenarios based on the frequency and severity of occurrence of each risk scenario in historical defect cases.

[0008] Furthermore, the system also includes a closed-loop learning mechanism, configured to collect user feedback on generated test cases and test execution result data, update the test knowledge base based on the feedback and test execution result data, and incrementally fine-tune the pre-trained large language model to continuously optimize the quality of generated test cases and the accuracy of evaluation.

[0009] Furthermore, the closed-loop learning mechanism is specifically configured as follows: new defect information discovered during test execution is stored as new historical defect cases in the test knowledge base; based on user feedback on test case modification suggestions, the generation requirements or example set in the automotive test-specific prompt template are adjusted to optimize the generation strategy.

[0010] Furthermore, the vehicle configuration library includes sensor types, quantities, installation locations, sensing ranges, and corresponding vehicle control logic for different vehicle models. When generating test cases, the test case generation module calls the configuration information of the target vehicle model in the vehicle configuration library to generate executable test steps that match the actual vehicle hardware.

[0011] Furthermore, the test case evaluation module is also configured to output specific optimization suggestions when the overall quality score is lower than a preset score threshold. The optimization suggestions include supplementing missing test scenarios, refining test steps, or adding high-risk coverage test cases.

[0012] On the other hand, this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned intelligent vehicle test case automatic generation and evaluation system.

[0013] On the other hand, this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned intelligent vehicle test case automatic generation and evaluation system.

[0014] The beneficial effects of this invention are as follows: This application provides an automatic generation and evaluation system for intelligent vehicle test cases. By constructing a test knowledge base containing domain knowledge and vehicle configuration, and combining it with a domain-fine-tuned large language model, it achieves deep semantic understanding of requirement documents. This enables accurate extraction of functional points and identification of implicit test requirements, including sensor fault handling and multi-system collaboration, significantly improving the comprehensiveness of test coverage. Furthermore, by utilizing dedicated prompt templates and few-shot learning technology, the system can automatically generate high-quality test case sets with complete structure and executable steps, greatly improving test design efficiency. Simultaneously, by introducing a quantitative evaluation mechanism covering multiple dimensions such as requirement coverage, scenario completeness, risk coverage, and standardization, it achieves an objective and systematic evaluation of test case quality, ensuring the sufficiency and reliability of generated test cases. Overall, it solves the problems of low coverage, poor efficiency, and unstable quality in traditional methods for testing complex intelligent vehicle systems. This application also provides corresponding equipment and products. The beneficial effects of the equipment and products are the same as those of the system, and will not be elaborated here.

[0015] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0016] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

[0017] Figure 1 This is a structural diagram of the intelligent vehicle test case automatic generation and evaluation system provided in this application; Figure 2 This is a structural diagram of the electronic device provided in this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] The present application will be further described below with reference to the accompanying drawings and specific embodiments. The described embodiments should not be considered as limitations on the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present application.

[0020] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0022] With the rapid development of intelligent connected vehicle technology, automobiles are evolving from traditional mechanical vehicles into highly intelligent, automated, and software-defined mobile terminals. The widespread application of Advanced Driver Assistance Systems (ADAS), autonomous driving (Level 2 and above), smart cockpits, and vehicle-to-everything (V2X) technologies has led to an exponential increase in the complexity of automotive systems. These systems typically consist of numerous sensors (such as cameras, radar, and lidar), controllers (ECUs), actuators, and complex software algorithms, with functional logic involving multiple levels such as multimodal perception, decision-making and planning, control execution, and human-machine interaction. Against this backdrop, ensuring the safety, reliability, and robustness of intelligent vehicle functions has become a core challenge in the research and development process, while the testing and verification of software and systems accounts for a significant portion of the entire development cycle.

[0023] As the basic unit of testing activities, the design quality of test cases directly determines the effectiveness of testing and the ability to detect defects. Traditionally, test case design mainly relies on test engineers manually analyzing and writing test cases based on requirements documents (such as functional specifications and system design documents). This process usually follows classic test design methods, such as equivalence partitioning, boundary value analysis, cause-effect graphs, and state transition graphs, aiming to systematically cover normal processes, boundary conditions, and abnormal scenarios. However, in the highly complex system environment of intelligent vehicles, this manual-led test design model has revealed many insurmountable shortcomings.

[0024] Furthermore, existing test case generation technologies mainly include rule-based automated tools, model-driven testing (MBT), search-based test generation, and optimization methods based on historical data. While these technologies have improved testing efficiency to some extent, they still have significant drawbacks: they generally lack deep semantic understanding of natural language requirements, making it difficult to identify implicit test scenarios (such as boundary conditions and multi-system collaborative logic), resulting in insufficient test coverage; the generation methods rely on preset rules or fixed models, resulting in poor flexibility, delayed response to frequent requirement changes, and weak adaptability; they do not adequately cover complex scenarios such as sensor failures, system anomalies, and inter-functional interactions, limiting system-level testing capabilities; at the same time, test case quality assessment lacks quantitative indicators, mainly relying on manual review, which is highly subjective and difficult to guarantee consistency; in addition, test knowledge is mostly in the form of personal experience, making it difficult to effectively accumulate and reuse, and the generated test cases often have problems such as ambiguous steps and poor executability, requiring a lot of manual intervention, which seriously restricts the efficiency and quality of test design under the high complexity of intelligent vehicle systems.

[0025] To address the aforementioned issues, this application proposes an automated test case generation and evaluation system, device, and product for intelligent vehicles. Its core lies in the deep integration of artificial intelligence technology and software testing engineering methods, systematically solving problems such as shallow semantic understanding, incomplete coverage, low efficiency, and difficulty in quality control in traditional test design. By constructing a test knowledge base containing domain terminology, historical defects, and vehicle configurations, and employing a large language model fine-tuned on automotive test corpora, the system can perform deep semantic analysis of natural language requirements, accurately extract functional points, and intelligently infer implicit test requirements such as sensor fault handling and multi-system collaboration. Based on this, combined with dedicated prompt engineering and few-shot learning techniques, the system guides the model to generate high-quality test case sets that are structurally complete, have executable steps, and clear data, improving generation efficiency and standardization. Simultaneously, it innovatively establishes a multi-dimensional quantitative evaluation system covering requirement coverage, scenario completeness, risk coverage, and test case standardization. Through weighted fusion, a comprehensive quality score is generated, achieving objective evaluation and optimization feedback on test adequacy, forming a closed-loop intelligent testing process of "understanding—generation—evaluation—optimization," comprehensively improving the automation, intelligence, and systematization level of intelligent vehicle testing.

[0026] First, the automatic generation and evaluation system for intelligent vehicle test cases provided in this application will be described in detail below with reference to the accompanying drawings.

[0027] Reference Figure 1 The intelligent vehicle test case automatic generation and evaluation system provided in this application includes a test knowledge base, a requirements understanding module, a test case generation module, a vehicle configuration library, and a test case evaluation module.

[0028] First, the test knowledge base is configured to store automotive testing terminology, test design patterns, historical defect cases, and sensor configurations and control logic for different vehicle models.

[0029] Specifically, the test knowledge base, as a core supporting component of the system, provides domain knowledge and contextual information for the entire test case generation and evaluation process. By systematically storing automotive testing terminology, it ensures the system can accurately identify and interpret industry-specific expressions when understanding requirements documents, avoiding semantic ambiguity. The accumulation of test design patterns allows the system to reuse mature testing strategies and methods, improving the scientific rigor and effectiveness of generated test cases. The storage of historical defect cases provides data support for risk-oriented testing, enabling the system to identify and prioritize high-problem areas. Furthermore, the integration of sensor configurations and control logic for different vehicle models ensures that the test knowledge is vehicle-adaptable, providing necessary technical parameters and system architecture information for subsequently generating test cases that match the actual vehicle hardware, thereby guaranteeing the accuracy and executability of the test design.

[0030] Secondly, the requirements understanding module is configured to combine a test knowledge base with a pre-trained large language model fine-tuned from automotive testing corpora to semantically parse the requirements documents for intelligent vehicle functions, extracting sets of functional points, implicit test requirements, and functional dependencies. Implicit test requirements include sensor fault handling logic and multi-system collaborative scenarios.

[0031] Specifically, the core function of the requirements understanding module is to perform deep semantic analysis on the input natural language requirements document. Its significance lies in overcoming the limitations of traditional methods that can only identify explicit requirements. By combining domain knowledge provided by the test knowledge base and utilizing a large language model fine-tuned on automotive test corpora, this module can understand the contextual relationships and deep logic of the requirements, thereby accurately extracting the set of functional points—that is, the core functional units explicitly described in the requirements. More importantly, this module possesses reasoning capabilities, enabling it to identify implicit test requirements that are not directly stated but actually exist within the requirements. Examples include the system's processing logic when sensors fail or signals are abnormal, as well as the interactive behaviors and dependencies of multiple functional systems (such as ACC and AEB) during operation. This mining of implicit requirements significantly improves the comprehensiveness of test coverage, helping to discover potential design flaws and safety risks.

[0032] Furthermore, the vehicle configuration library includes the sensor types, quantities, installation locations, sensing ranges, and corresponding vehicle control logic for different vehicle models. When generating test cases, the test case generation module calls the configuration information of the target vehicle model in the vehicle configuration library to generate executable test steps that match the actual vehicle hardware.

[0033] Specifically, the vehicle configuration library provides the hardware and control parameters for specific vehicle models, ensuring the physical executability of the generated test cases. Different vehicle models differ in sensor configuration (such as the number of cameras, radar layout, and lidar installation location) and perception capabilities (such as detection distance and field of view). These differences directly affect the setting of test conditions and the expected test behavior. By tightly integrating the vehicle configuration library with the test case generation process, the system can dynamically reference the actual parameters of the target vehicle model when generating test steps. For example, it can set the following distance threshold based on the perception range of a vehicle's forward radar, or determine the braking response timing based on its control logic. This customized generation based on real vehicle configurations avoids the problem of test cases being detached from the actual hardware environment, ensuring the accuracy and operability of the test steps, and enabling the generated test cases to realistically reflect the vehicle's behavior in actual operation.

[0034] Then, the test case generation module is configured to generate test case sets based on automotive-specific prompt templates and few-shot learning technology, combined with a vehicle configuration library. Each test case set includes at least the test objective, preconditions, executable test steps, expected results, and test data.

[0035] The test case generation module transforms the analysis results from the requirements understanding phase into concrete, executable test cases. By employing prompt templates specifically designed for automotive testing, this module structurally guides the large language model to generate test content that conforms to industry standards and project requirements, ensuring consistent output format and complete information. The prompt templates include task definitions, output structures, and key element requirements, effectively improving the accuracy and consistency of the generated results. Simultaneously, combined with few-shot learning technology, the system can reference high-quality test case examples from historical projects during the generation process, quickly adapting to new requirement scenarios and improving generation efficiency and quality. The final generated test case set includes necessary elements such as test objectives, preconditions, executable test steps, expected results, and test data, forming a complete, clear, and directly usable document set for test execution, achieving automated transformation from requirements to test design.

[0036] Finally, the test case evaluation module is configured to perform multi-dimensional quality evaluation on the test case set, calculate the requirement coverage, scenario integrity score, risk coverage and test case standardization score, and integrate the scores of each dimension according to preset weights to obtain a comprehensive quality score.

[0037] Specifically, the significance of the test case evaluation module lies in establishing an objective and quantitative test quality evaluation system, addressing the problem of traditional evaluations relying on subjective experience. This module comprehensively reviews the generated test case set from multiple dimensions: requirement coverage measures the extent to which test cases cover each atomic requirement point in the original requirements, ensuring functional integrity; scenario integrity score assesses the balanced coverage of normal, boundary, abnormal, performance, and combined scenarios, improving test comprehensiveness; risk coverage analyzes the coverage of historically frequent defects and high-risk scenarios, enhancing defect prevention capabilities; and test case standardization score checks the clarity, logic, and format consistency of test steps, ensuring test case executability. Finally, the scores from each dimension are integrated according to preset weights to form a comprehensive quality score, providing a unified quantitative indicator for test design quality and supporting the scientific judgment and continuous optimization of test adequacy.

[0038] In some embodiments of this application, the requirement understanding module is further configured to simultaneously perform test element identification, test type classification, and test priority prediction using a multi-task learning framework. Specifically, test element identification is used to extract test objects, test conditions, and expected results from the requirement document; test type classification is used to determine whether the test type corresponding to the requirement is functional testing, performance testing, or exception testing; and test priority prediction is used to predict test priorities based on the importance and security level of the requirement.

[0039] Specifically, test element identification is responsible for extracting test objects, test conditions, and expected results from the requirements text, transforming unstructured descriptions into actionable test input elements, clarifying "what to test," "under what conditions to test," and "what the expected results are." Test type classification determines whether the requirements belong to functional testing, performance testing, or exception testing, selecting appropriate test design strategies and scenario construction methods for the subsequent generation process, ensuring the professionalism and relevance of test case design. Test priority prediction automatically assesses the test priority based on the functional importance and security level of the requirements, ensuring that high-risk and high-security-critical functions are prioritized for coverage and verification, thereby optimizing test resource allocation and supporting early defect discovery and risk prevention. These three tasks together constitute a comprehensive understanding of test requirements, laying a solid foundation for generating high-quality, focused, and executable test cases.

[0040] Therefore, the requirements understanding module enables collaborative and efficient deep analysis of requirements documents. This framework leverages the semantic relationships between multiple test-related tasks, performing various analyses in parallel based on a shared underlying semantic representation. This avoids the information fragmentation and computational redundancy caused by traditional serial processing, improving not only the overall accuracy of model understanding and predictive consistency but also significantly enhancing the processing efficiency of requirements analysis. This provides comprehensive and structured input support for the intelligent generation of subsequent test cases.

[0041] In some embodiments of this application, the test case generation module includes: a dedicated prompt template for automotive testing containing a functional requirement description, test type, sensor configuration and control logic of the target vehicle model, and generation requirements; and a few-shot learning technique that retrieves historical test cases similar to the current requirements from the test knowledge base as examples and embeds these examples into the prompt template to guide the large language model in generating test cases that conform to domain specifications.

[0042] Specifically, the automotive testing-specific prompt template includes a functional requirement description, test type, target vehicle sensor configuration and control logic, and generation requirements. Its purpose is to provide structured, domain-customized generation instructions for large language models, ensuring that the output test cases conform to professional specifications and actual engineering needs for intelligent vehicle testing. This prompt template not only clarifies the core content of the function to be tested but also incorporates test type information to guide scenario design (e.g., boundary value testing for performance requirements, fault injection for anomaly testing) and introduces the specific hardware configuration and control logic of the target vehicle, enabling the generated test steps to match the physical capabilities of real vehicles. Simultaneously, the generation requirements section clearly defines the elements that test cases must include (e.g., preconditions, executable steps, expected results, etc.) and format specifications, effectively constraining the model's output behavior and avoiding the generation of ambiguous, logically disjointed, or formatted test cases, thereby improving the accuracy, consistency, and executability of the generated results.

[0043] Furthermore, few-shot learning technology retrieves historical test cases similar to the current requirement from the test knowledge base as examples and embeds these examples into prompt templates to guide the large language model in generating test cases that conform to domain specifications. The significance of this technology lies in using high-quality historical test cases as "demonstrations," enabling the model to quickly understand test design patterns and expression styles in specific scenarios even in the absence of a large amount of labeled data. By using semantic similarity matching to find historical cases similar to the current requirement in terms of functional type, system interaction, or risk characteristics, the system inputs these examples along with the current requirement into the model, achieving in-context learning. This allows the model to imitate the structure, terminology, and test logic of the examples to generate new test cases. This approach significantly improves the model's adaptability to new requirements, enhances the professionalism and standardization of generated test cases, and supports the continuous reuse and accumulation of testing experience, giving the system stronger generalization ability and domain adaptability.

[0044] In some embodiments of this application, in the test case evaluation module: the requirement coverage rate is obtained by identifying atomic requirement points in the requirement document through semantic analysis and calculating the ratio of the number of atomic requirement points covered by the test case set to the total number of atomic requirement points; the scenario integrity score is obtained by weighting the coverage of normal scenarios, boundary scenarios, abnormal scenarios, failure scenarios and multi-system interaction scenarios according to preset weights; the risk coverage rate is obtained by weighting the test cases covering high-risk scenarios in the test case set based on the frequency and severity of occurrence of each risk scenario in historical defect cases.

[0045] Among them, the requirement coverage rate uses a large language model to perform deep semantic analysis on the requirement documents, identify all atomic requirement points (i.e. the smallest testable functional unit), and calculate the ratio of the number of atomic requirement points covered by the test case set to the total number, thereby objectively measuring the coverage breadth of the test cases in terms of functional integrity and ensuring that all explicit requirements are effectively verified.

[0046] Optionally, the formula for calculating the demand coverage rate is: Demand Coverage Rate = (Number of covered demand points / Total number of demand points) × 100%, where the identification of demand points is achieved through semantic analysis. For example, for the demand "ACC can be activated within the vehicle speed range of 30-150km / h", the system will identify three demand points: the minimum activation speed is 30km / h, the maximum operating speed is 150km / h, and normal activation is possible within the speed range.

[0047] The scenario integrity score further evaluates the distribution balance and logical depth of test cases in different types of scenarios. By dividing test scenarios into five categories—normal scenarios, boundary scenarios, abnormal scenarios, failure scenarios, and multi-system interaction scenarios—and setting preset weights based on the technical importance and risk level of each type of scenario, the coverage of each type of scenario is weighted and calculated to comprehensively reflect the sufficiency of test design in terms of main process, extreme conditions, fault tolerance, fault recovery, and system integration, and to prevent testing from focusing too much on routine processes and neglecting critical edge cases.

[0048] Optionally, the calculation formula for scene integrity assessment is: Scene integrity score = ,in Represents the number of scene types. Representing the Scenario-specific weights Representing the The coverage score for each scenario (range 0-1). Specific scenario types and weight assignments are shown in Table 1.

[0049] Table 1. Examples of Scene Types and Weight Allocation

[0050] Risk coverage incorporates historical defect cases as an assessment basis. It weights each risk scenario based on its frequency of occurrence and severity in past projects, and calculates the extent to which current test cases cover these high-risk scenarios. This achieves a test adequacy evaluation guided by practical experience, ensuring that vulnerable links that are prone to errors and have a significant impact are given priority coverage, thereby improving the foresight and pertinence of defect prevention.

[0051] Optionally, the risk coverage assessment is based on historical defect data, and the calculation formula is: Risk Coverage = ,in For the number of risk points, For the first Risk level (1-5) of each risk point. For the first The coverage indicator for each risk point (0 indicates no coverage, 1 indicates coverage).

[0052] Furthermore, the overall quality score is calculated by integrating multiple dimensions of indicators, using the formula: Overall Quality Score = 0.3 × Requirement Coverage + 0.3 × Scenario Completeness + 0.25 × Risk Coverage + 0.15 × Use Case Standardization. Taking the ACC functional test as an example, the evaluation results are shown in Table 2.

[0053] Table 2. Example of ACC functional test evaluation results

[0054] In some embodiments of this application, the test case evaluation module is further configured to output specific optimization suggestions when the overall quality score is lower than a preset score threshold. The optimization suggestions include supplementing missing test scenarios, refining test steps, or adding high-risk coverage test cases.

[0055] Specifically, when the system determines that the overall quality of the test case set does not meet the standard, it can automatically analyze the weaknesses in each evaluation dimension and proactively generate targeted optimization suggestions: For issues of insufficient scenario completeness, it suggests supplementing missing test scenarios such as extreme environments and multi-system interactions to improve the breadth and depth of test coverage; for issues of poor test case standardization and ambiguous steps, it prompts for refining the specific conditions, parameters, and execution processes in test operations to enhance the readability and executability of test cases and ensure the consistency and repeatability of the testing process; for situations with low risk coverage, it suggests adding test cases for historically high-incidence defects or safety-critical scenarios to strengthen the verification capability of potential failure modes and improve the security and reliability of the system. This mechanism not only avoids the limitations of traditional evaluation but also significantly reduces the cost of manual analysis, guides continuous iterative optimization of test design, and effectively ensures the quality and effectiveness of test cases.

[0056] In some embodiments of this application, the system further includes a closed-loop learning mechanism, configured to collect user feedback on generated test cases and test execution result data, update the test knowledge base based on the feedback and test execution result data, and incrementally fine-tune the pre-trained large language model to continuously optimize the quality of generated test cases and the accuracy of evaluation. Specifically, the closed-loop learning mechanism is configured to: store new defect information discovered during test execution as new historical defect cases in the test knowledge base; and adjust the generation requirements or example set in the automotive testing-specific prompt template based on user feedback on test case modification suggestions, thereby optimizing the generation strategy.

[0057] Specifically, the closed-loop learning mechanism proactively collects user feedback on generated test cases (such as modification suggestions, omission scenario annotations, and step executability evaluations) and actual result data during test execution (such as test pass / fail records, defect discovery status, and execution logs). This valuable runtime information is then injected back into the system's core components as learning signals. Based on this feedback data, the system can dynamically update the test knowledge base and optimize the generation capability of large language models. This allows test design capabilities to continuously improve with project accumulation and experience, achieving a shift from "one-time generation" to "continuous evolution," ensuring that test case generation and evaluation capabilities always remain synchronized with actual engineering needs.

[0058] Storing newly discovered defects as new historical defect cases in the test knowledge base is significant for achieving dynamic accumulation of test knowledge assets and continuous updating of the risk database. Each newly discovered defect represents a weak link or an insufficiently covered scenario in the system design or implementation. Recording them structurally as historical defect cases not only enriches the risk pattern library in the knowledge base but also provides important reference for testing similar functions in the future. These new cases will be considered in future risk coverage assessments and test case generation, enabling the system to automatically strengthen testing of similar problems and prevent the recurrence of the same or similar defects in different projects. This forms a positive cycle of "defect discovery—knowledge accumulation—prevention of recurrence," significantly improving the foresight and effectiveness of testing.

[0059] Based on user feedback and suggestions for test case modification, the generation requirements or example sets in the automotive-specific prompt templates are adjusted to optimize the generation strategy. This aims to achieve refined tuning of the generated model and improve its domain adaptability. The modification suggestions from test engineers during actual use (such as "test steps should be more detailed," "addition of takeover condition descriptions is needed," and "the description of expected results is unclear") directly reflect the gap between the generated test cases and actual engineering specifications. By analyzing this feedback, the system makes targeted adjustments to the generation requirements in the prompt templates (such as adding output constraints and refining field descriptions) or expanding / replacing the example sets in few-shot learning, enabling the large language model to mimic expressions and design logic that better conform to team standards in subsequent generation. This strategy optimization based on real user feedback allows the system to quickly adapt to the testing styles and specifications of different teams, projects, or vehicle platforms, continuously enhancing the practicality, standardization, and acceptability of the generated results, and promoting the deep integration of test automation systems and engineering practices.

[0060] In some embodiments of this application, taking the generation of test cases for the adaptive cruise control (ACC) function as an example, in response to the input requirement that "the adaptive cruise control system should be able to work within a speed range of 30-150km / h, support a following distance setting of 1-3 seconds, have a maximum deceleration not exceeding 3.5m / s², and be able to identify vehicles ahead and maintain a safe distance", the system identifies key information such as function name, speed range, following distance setting, performance constraints, and core capabilities during the requirement understanding phase. Subsequently, it generates a set of 45 test cases, including functional tests (15, such as ACC activation / deactivation tests), boundary tests (8, such as 29 / 30 / 150 / 151km / h speed boundary tests), performance tests (5, such as braking response time tests), anomaly tests (10, such as sensor obstruction handling), and integration tests (7, such as tests in collaboration with the AEB system). The typical use case, "ACC Speed ​​Boundary Activation Test" (ACC_BVT_001), was conducted under the preconditions of normal vehicle operation, no fault codes in the ACC system, and no obstacles ahead. The activation logic was verified by sequentially stabilizing the vehicle speed at 29 km / h (unable to activate, instrument panel indicates low speed), 30 km / h (normal activation and display of set speed), 150 km / h (maintaining operation), and 151 km / h (automatic exit or speed limit). Test speed points of [29, 30, 31, 149, 150, 151] km / h were used (each held for ≥10 seconds, acceleration ≤0.5 m / s²). Quality assessment showed a requirement coverage of 91.7% (covering 11 / 12 requirement points), scenario completeness of 87.5% (weighted calculation of normal / boundary / abnormal / performance / combined scenarios), and an overall quality score of 89.3% (0.3 × coverage + 0.3 × scenario completeness + 0.25 × risk coverage + 0.15 × use case standardization).

[0061] In some embodiments of this application, taking the generation of test cases for intelligent parking assistance functions as an example, the system generates test scenarios based on the input requirement that "the intelligent parking assistance system supports parallel and perpendicular parking modes, can identify effective parking spaces with a length greater than 1.2 times the vehicle body, parking speed does not exceed 7km / h, and supports manual intervention at any time." These scenarios are divided into three categories: basic function tests (including parallel parking, perpendicular parking, parking space recognition, and trajectory planning), environmental adaptation tests (including lighting conditions, ground markings, slope effects, and rain / snow weather), and safety tests (including emergency braking, obstacle detection, manual intervention, and pedestrian protection). This comprehensively covers the scenarios of function implementation, environmental robustness, and safety assurance.

[0062] In some embodiments of this application, taking test case optimization and iteration as an example, the initially generated intelligent parking test case steps are "activate parking function → select parking mode → start parking → complete parking," which has the problem of simplified steps and lack of operational details. After optimization based on user feedback, the test steps are refined to: "1. Reduce vehicle speed to below 10km / h, press the parking assist button and observe the instrument prompts; 2. After the system scans for a valid parking space, select the target parking space according to the voice prompts; 3. Shift into R gear, release the brake, the system takes over steering, and the driver controls the accelerator and brake; 4. Shift gears according to the system prompts until the voice prompt 'parking complete'; 5. Shift into P gear and confirm that the vehicle position is centered and the front and rear distances are uniform." By supplementing vehicle speed conditions, human-computer interaction prompts, and operational specifications, the executability and accuracy of the test cases are improved.

[0063] Secondly, refer to Figure 2 This application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned intelligent vehicle test case automatic generation and evaluation system.

[0064] Furthermore, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the aforementioned intelligent vehicle test case automatic generation and evaluation system.

[0065] In summary, the intelligent vehicle test case automatic generation and evaluation system, equipment, and products provided in this application have the following technical effects.

[0066] This application's embodiments achieve deep semantic understanding of natural language requirements by constructing a test knowledge base containing domain knowledge and vehicle configuration, and combining it with a large language model fine-tuned from automotive test corpora. This enables accurate extraction of functional points and inference of implicit test requirements such as sensor fault handling and multi-system collaboration, significantly improving the comprehensiveness and accuracy of test coverage. Based on automotive-specific prompt templates and few-shot learning technology, the system can automatically generate high-quality test case sets that are structurally complete, feature-rich, and have executable steps, greatly improving test design efficiency and effectively addressing the challenges of frequent requirement changes. By introducing a vehicle configuration library, it ensures that the generated test steps are consistent with the actual vehicle's sensor layout and sensing capabilities. By matching the scope of knowledge with control logic, the physical executability and engineering applicability of test cases are enhanced. An innovative multi-dimensional quantitative evaluation system covering requirement coverage, scenario completeness, risk coverage, and test case standardization is established. When the overall quality score falls below a threshold, specific optimization suggestions such as supplementing test scenarios, refining test steps, and adding high-risk test cases are output, achieving objective evaluation and closed-loop optimization of test quality. Furthermore, through a closed-loop learning mechanism, user feedback and test execution results are continuously collected, the test knowledge base is dynamically updated, prompt templates and generation strategies are optimized, and the language model is incrementally fine-tuned, enabling the system to continuously evolve, accumulate testing experience, and improve generation quality.

[0067] Overall, this application achieves full automation and intelligence in the entire process of intelligent vehicle test cases, from requirements understanding and test case generation to quality assessment and optimization. It significantly improves test coverage, defect detection rate and test design efficiency, reduces labor costs and security risks, promotes the digital accumulation and reuse of test knowledge, and provides strong technical support for the safe and reliable verification of highly complex intelligent vehicle systems.

[0068] It should be noted that in all specific embodiments of this application, all data processing activities related to user identity or personal characteristics, such as user information, user behavior data, historical data, and location information, will be conducted in accordance with the principles of legality, legitimacy, and necessity. All data collection, use, storage, and processing will be subject to compliance with applicable national and regional laws, regulations, and industry standards, and informed consent from users will be obtained in a clear and explicit manner before processing. For the processing of sensitive personal information, separate consent from users will be obtained through prominent means such as pop-up prompts and independent confirmation pages. If any processing conflicts with laws and regulations, the laws and regulations will prevail, and necessary data processing will only be carried out within the scope permitted by laws and regulations, ensuring that all data-based applications, analyses, and technical implementations are conducted within the scope permitted by laws and regulations.

[0069] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0070] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of ordinary skill of an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary skill. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0071] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several programs to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0072] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequential list of executable programs for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, a program execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can retrieve and execute a program from or in conjunction with such a program execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit a program for use by or in conjunction with a program execution system, apparatus, or device.

[0073] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Additionally, computer-readable media can even be paper or other suitable media on which programs can be printed, for example, by optically scanning the paper or other media, then editing, interpreting, or, if necessary, processing it in a suitable manner to obtain the program electronically, and then storing it in computer memory.

[0074] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable program execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0075] In the foregoing description of this specification, the reference to terms such as "one embodiment / implementation," "another embodiment / implementation," or "certain embodiments / implementations," etc., indicates that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in an embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0076] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0077] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. An intelligent automatic generation and evaluation system for test cases of an automobile, characterized in that, The system comprises: a test knowledge base configured to store automotive test field professional terms, test design patterns, historical defect cases, and sensor configurations and control logic of different vehicle models; a requirement understanding module configured to, in combination with the test knowledge base, use a pre-trained large language model fine-tuned on automotive test field corpus to perform semantic analysis on a requirement document of an intelligent vehicle function, and extract a set of function points, implicit test requirements, and function dependency relationships; wherein the implicit test requirements include sensor fault handling logic and multi-system coordination scenarios; a test case generation module configured to generate a test case set based on automotive test-specific prompt templates and few-shot learning techniques in combination with a vehicle model configuration library; wherein the test case set at least includes test purposes, preconditions, executable test steps, expected results, and test data; a test case evaluation module configured to perform multi-dimensional quality evaluation on the test case set, calculate requirement coverage, scenario completeness score, risk coverage, and test case specification score, and fuse the scores of each dimension according to a preset weight to obtain a comprehensive quality score.

2. The intelligent test case auto-generation and evaluation system for automobiles as claimed in claim 1 wherein, The requirement understanding module is further configured to: perform test element identification, test type classification, and test priority prediction simultaneously through a multi-task learning framework; wherein the test element identification is used to extract test objects, test conditions, and expected results in the requirement document; the test type classification is used to determine whether the test type corresponding to the requirement is functional testing, performance testing, or abnormal testing; and the test priority prediction is used to predict the test priority based on the importance and safety level of the requirement.

3. The intelligent test case auto-generation and evaluation system for automobiles as claimed in claim 1 wherein, In the test case generation module: the automotive test-specific prompt templates include function requirement descriptions, test types, sensor configurations and control logic of target vehicle models, and generation requirements; the few-shot learning technique retrieves historical test cases similar to the current requirement from the test knowledge base as examples, embeds the examples in the prompt templates to guide the large language model to generate test cases that meet the field specifications.

4. The intelligent test case auto-generation and evaluation system for automobiles as claimed in claim 1, wherein, In the test case evaluation module: the requirement coverage is obtained by calculating the ratio of the number of atomic requirement points covered by the test case set to the total number of atomic requirement points through semantic analysis; the scenario completeness score is obtained by calculating the weight of normal scenarios, boundary scenarios, abnormal scenarios, failure scenarios, and multi-system interaction scenarios according to the preset weight; the risk coverage is obtained by weighting and counting the test cases in the test case set that cover high-risk scenarios based on the occurrence frequency and severity level of each risk scenario in the historical defect cases.

5. The intelligent test case auto-generation and evaluation system for automobiles as claimed in claim 1 wherein, The system further comprises a closed-loop learning mechanism configured to collect user feedback on the generated test cases and test execution result data, update the test knowledge base based on the feedback and test execution result data, and incrementally fine-tune the pre-trained large language model to continuously optimize the generation quality of test cases and the accuracy of evaluation.

6. The intelligent test case auto-generation and evaluation system for automobiles as claimed in claim 5, wherein, The closed-loop learning mechanism is specifically configured to store new defect information found during test execution as new historical defect cases in the test knowledge base; and adjust the generation requirements or example sets in the vehicle test prompt template according to the test case modification suggestions fed back by the user, and optimize the generation strategy.

7. The intelligent test case auto-generation and evaluation system for automobiles as claimed in claim 1 wherein, The vehicle model configuration library includes sensor types, numbers, installation positions, sensing ranges and corresponding vehicle control logics of different vehicle models, and the test case generation module calls configuration information of a target vehicle model in the vehicle model configuration library to generate executable test steps matching actual vehicle hardware when generating test cases.

8. The intelligent test case auto-generation and evaluation system for automobiles as claimed in claim 1 wherein, The test case evaluation module is further configured to output specific optimization suggestions when the comprehensive quality score is lower than a preset score threshold, the optimization suggestions including supplementing missing test scenarios, refining test steps or adding high-risk coverage cases.

9. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the intelligent automobile test case automatic generation and evaluation system of any one of claims 1 to 8 when executing the computer program.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the intelligent automobile test case automatic generation and evaluation system of any one of claims 1 to 8.

Images