Defect self-checking driven dynamic reduction AI test case automatic generation system and method

Abstract

The application provides a kind of defect self-checking drive dynamic simplified AI test case automatic programming system and method;System includes the multimodal data acquisition module connected in turn, demand hierarchical analysis module, initial case generation module, AI case self-checking module, case intelligent optimization module, result output archiving module, adopts double big model isolation running architecture;Method includes multimodal demand acquisition preprocessing, hierarchical business disassembly, initial case generation, full-dimensional defect self-checking, dynamic optimization rectification, standardized archiving output six major steps.The beneficial effects of the present application are: it can independently complete the case defect detection and semantic level redundancy reduction, actively excavate implicit test scene, avoid single model self-checking blind area, effectively reduce the cost of artificial review, improve test coverage and case preparation efficiency, adapt to multi-category software test scene, have good versatility and practical value.

Description

Technical Field

[0001] This invention mainly relates to the fields of software testing technology, natural language processing, and large language model artificial intelligence technology, specifically to a defect self-inspection driven dynamic simplification AI test case automatic writing system and method. Background Technology

[0002] Software testing is a core component in ensuring the quality of software products before they go live. Test cases, as the basis for software testing, directly determine the quality and efficiency of the testing work due to their completeness, standardization, and conciseness. Currently, test case writing in the industry mainly falls into two categories: The first is the traditional manual writing mode, where dedicated test engineers write test cases item by item according to the requirements document. The advantage is that the test cases are closely aligned with the business and the logic is accurate, but it has drawbacks such as long writing cycles, high labor costs, inconsistent quality of test cases written by newcomers, easy omission of boundary scenarios, and cumbersome reuse of regression test cases, making it unsuitable for the current rapid iteration pace of software development. The second is the existing AI-assisted test case generation mode, which relies on a large language model to quickly generate batches of test cases by inputting the requirements text, solving the problem of low efficiency in manual writing. Currently, similar patents and commercial tools on the market all adopt a single large model unidirectional generation architecture, which can only complete a single process of "requirement input → test case output".

[0003] After comprehensive patent searches and market research, all existing AI test case writing-related patents and products suffer from several unresolved technical shortcomings. Firstly, they lack self-inspection capabilities. Existing AI generation models can only generate test cases but cannot independently determine whether they contain vulnerabilities, requiring manual review and correction, thus failing to truly reduce the workload of testing personnel. Secondly, they lack coverage of implicit scenarios. Traditional AI can only identify explicit textual content in requirement documents and cannot uncover unlabeled implicit anomalies or boundary scenarios, resulting in consistently lower test coverage than manually written test cases. Thirdly, test case redundancy is extremely high. Batch-generated test cases contain a large amount of semantic repetition and homogenized steps, making subsequent manual removal of redundant test cases time-consuming and laborious. Fourthly, there is a single-model detection blind spot. Using the same large model for generation and self-inspection means that the model's inherent cognitive defects cannot be self-checked, resulting in extremely low defect detection accuracy.

[0004] Currently, the publicly available patent database contains only a small number of technical solutions based on keyword matching and single-model test case generation. There are no solutions covering the entire process of dual-model isolation architecture, AI-driven test case defect detection, automatic simplification of semantically redundant test cases, and proactive mining of implicit test scenarios. There are significant technological gaps in related technical fields, making it difficult to meet the needs of high-quality and efficient test case development in scenarios of rapid software iteration. Summary of the Invention

[0005] 1. The technical problem that the invention aims to solve: This invention provides a defect self-inspection-driven dynamic simplification AI test case automatic writing system and method, realizing fully unmanned operation from requirement import, test case generation, defect self-inspection, automatic optimization to report output, completely eliminating the dependence on human experience in test case writing, and comprehensively improving the quality and efficiency of test case writing.

[0006] 2. Technical Solution: To achieve the above objectives, the present invention adopts the following independent dual-model isolation architecture technical solution: The method for automatically writing dynamic and simplified AI test cases driven by defect self-inspection includes the following steps: S1. Multimodal test requirement data collection and preprocessing: Collect four types of raw data corresponding to the software under test: structured requirement documents, unstructured business process diagrams, interface messages, and historical defect data. Unify the data format and clean up invalid and redundant text to build a complete test requirement dataset. S2, Layered Intelligent Decomposition of Business Requirements: Input the pre-processed requirement dataset into the deep analysis model of requirements, and decompose the business according to four dimensions: main process branch, abnormal interruption branch, boundary value branch, and permission verification branch, and output a standardized test decomposition list. S3. Fully Automatic Generation of Initial Test Cases: Based on the test breakdown list, combined with the built-in software test case writing specification library, equivalence class partitioning algorithm, and boundary value analysis method, it automatically generates a complete set of initial test cases, including preconditions, operation steps, expected results, assertion parameters, test priorities, and test types. S4. AI-driven full-dimensional test case defect detection: Input the initial test case set into an independent test case quality evaluation model, and detect it line by line from five dimensions: missing scenario coverage, logical conflict of test case steps, duplicate and redundant test cases, missing assertion results, and unreasonable equivalence class partitioning. Mark all defect points and generate a detailed defect ledger. S5. Dynamic optimization and closed-loop rectification of test cases: Based on the defect details log, automatically supplement missing boundary scenarios and abnormal branch test cases, merge redundant test cases with homogeneous content, correct test steps with logical conflicts, and fill in missing assertion content to complete the fully automatic optimization of the initial test cases. S6. Standardized Test Case Output and Version Archiving: Outputs final test cases that conform to industry-standard test templates, along with accompanying reports, and archives versions of test cases generated in each round for future reuse.

[0007] Furthermore, the hierarchical intelligent decomposition of business requirements in step S2 specifically includes: fully decomposing the normal business forward process, while actively mining hidden abnormal scenarios that are not explicitly marked in the requirement documents, making up for the shortcomings of manual decomposition and traditional AI models that ignore hidden test scenarios.

[0008] Furthermore, in step S4, the large-scale test case quality evaluation model is specifically trained with a massive number of defective test case samples. The training dataset contains hundreds of thousands of low-quality, high-defect real industrial test cases, which can accurately identify hidden logical vulnerabilities that are difficult for humans to discover.

[0009] Furthermore, in step S5, the redundant test cases are merged as follows: based on semantic similarity algorithm rather than text keyword matching, homogeneous test cases with consistent operation logic and only different descriptions are identified, the optimal test steps are retained, and duplicate content is deleted to achieve test case simplification without information loss.

[0010] Furthermore, in step S3, the initial test cases are generated based on the built-in software test case writing specification library, equivalence class partitioning algorithm and boundary value analysis method. Each test case contains all execution elements, including preconditions, operation steps, expected results, assertion parameters, test priority and test type.

[0011] The defect self-inspection-driven dynamic simplification AI test case automatic writing system includes a multimodal data acquisition module, a requirement layering parsing module, an initial test case generation module, an AI test case self-inspection module, a test case intelligent optimization module, and a result output archiving module, which are connected in sequence. The multimodal data acquisition module is used to collect raw data for various test requirements and to perform data cleaning and format unification. The requirement layering and parsing module has a built-in deep requirement parsing model, which is used to complete the multi-dimensional business branch decomposition and output a standardized test decomposition list. The initial test case generation module has a built-in standardized test specification library and classic test algorithms, which can generate complete initial test cases with one click; The AI ​​test case self-inspection module has a built-in independent test case quality evaluation model, which is used to detect various defects in the initial test cases in all aspects and locate the defect locations. The intelligent test case optimization module is used to automatically complete test case completion, error correction, deduplication and simplification based on the detection results; The result output archiving module is used to output standardized test cases and accompanying reports, and at the same time complete test case version management and archiving storage.

[0012] Furthermore, the system is equipped with a dual-model isolation operation mechanism. The large model for in-depth requirement analysis and the large model for test case quality evaluation are independent of each other and do not share parameters, thus avoiding blind spots in detection caused by the integration of a single model generation and self-testing.

[0013] Furthermore, the system also has an interface for connecting to external testing platforms to achieve data interoperability with mainstream test management platforms and supports one-click export of test cases in multiple formats.

[0014] Furthermore, the accompanying reports output synchronously in step S6 include test case coverage reports and defect rectification reports. Version archiving supports subsequent version difference comparisons and rapid reuse of regression test cases.

[0015] 3. Beneficial effects: Compared with the prior art, the technical solution provided by this invention has the following advantages: We construct a physically isolated dual-model collaborative architecture, which is different from the industry's conventional solution of single-model self-generation and self-verification. It completely avoids the inherent cognitive blind spots of a single model, realizes fully automatic closed-loop self-checking and self-correction of test cases, and greatly reduces the workload of manual review, which can reduce the manual test case review time by more than 80%. By leveraging algorithms for uncovering implicit demand scenarios, this approach breaks through the limitations of traditional AI, which can only identify explicit textual demands. It proactively captures implicit anomalies and boundary scenarios not marked in demand documents, effectively improving the overall test coverage completeness. The test coverage rate is more than 25% higher than existing AI solutions. Instead of the traditional keyword matching method, a semantic similarity matching mechanism is used to remove duplicate test cases. This can accurately identify homogeneous test cases with different wording but consistent test logic. Without reducing effective test points, 30%-40% of invalid and redundant test cases are compressed, reducing the load on subsequent test execution. Adapted to agile software iteration and development models, the entire process requires no manual intervention, from the import of original requirements to the final output of compliant use cases, and runs automatically. It has good versatility and compatibility, and can be adapted to all types of software testing work, including mobile terminals, web terminals, and backend interfaces. It supports connection to mainstream test management platforms on the market, with low format adaptation costs and low barriers to implementation.

[0016] It should be noted that the structures not described in this invention are not related to the design points and improvement directions of this invention, and are the same as or can be implemented using existing technologies, and will not be elaborated here. Attached Figure Description

[0017] Figure 1 A schematic diagram of the AI ​​test case writing method of the present invention; Figure 2 Write a system overall architecture module diagram for the AI ​​test cases of this invention; Figure 3 This is a schematic diagram of the dual-model isolated operation architecture of the present invention. Detailed Implementation

[0018] To facilitate understanding of the present invention, a more complete description of the invention will be given below with reference to the accompanying drawings, which illustrate several embodiments of the invention. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of the invention will be more thorough and complete.

[0019] See attached document Figure 1-3 This invention is structured into a six-layer architecture: data layer, parsing layer, generation layer, quality inspection layer, optimization layer, and output layer. The two main models operate independently throughout the entire process, with no parameter cross-interference, achieving complete automation of test case writing. The method flow executes unidirectionally and serially according to business logic. The defect detection stage can reverse-engineer defect data back to the upstream generation stage, forming a closed-loop optimization mechanism to ensure the effectiveness of test case iterative optimization.

[0020] Example 1: E-commerce payment interface test scenario This embodiment uses an e-commerce platform payment interface as the test object, and fully executes the AI ​​test case automatic writing method described in this invention. The specific steps are as follows: S1. Multimodal test requirement data collection and preprocessing: Import four types of raw data from the e-commerce payment module: structured requirement documents, unstructured payment business process diagrams, interface request messages, and historical payment defect work orders. The system automatically cleans up invalid language and redundant expressions, extracts and retains core business logic fields, and constructs a complete payment test requirement dataset after unifying the data format.

[0021] S2. Layered Intelligent Decomposition of Business Requirements: The pre-processed payment requirement dataset is input into a deep requirement analysis model. The model decomposes the business requirements according to four dimensions: main process branch, exception interruption branch, boundary value branch, and permission verification branch. The main process branch is broken down into a complete forward payment process; The abnormal interruption branch is broken down into the insufficient balance abnormal process and the network timeout interruption process; The process of breaking down boundary value branches to extract the upper and lower limits of the amount; The permission verification branch is broken down into a duplicate submission permission verification process; at the same time, the model actively discovers hidden abnormal scenarios such as "exiting the page during payment" that are not explicitly stated in the requirements document, and adds them to the decomposition list, ultimately outputting a standardized payment business test decomposition list.

[0022] S3. Initial Test Case Generation: Based on the payment business test breakdown list, combined with the built-in software test case writing specification library, equivalence class partitioning algorithm, and boundary value analysis, the system automatically generates an initial test case set. Each test case includes all elements such as preconditions, interface input parameters, operation steps, expected return value, assertion fields, test priority, and test type. This embodiment generates a total of 42 initial test cases.

[0023] S4, AI-driven, multi-dimensional test case defect detection, inputs the generated initial test case set into an independent test case quality evaluation model. The model scans and detects defects line by line from five dimensions: missing scenario coverage, logical conflicts in test case steps, duplicate and redundant test cases, missing assertion results, and unreasonable equivalence class partitioning. A total of 11 defects were detected: including 3 redundant and duplicate test cases, 2 missing boundary value assertions, 1 case with reversed step logic, and 4 missing implicit scenario test cases. The model marks all defect locations and generates a detailed ledger of payment test case defects.

[0024] S5. Dynamic optimization and closed-loop rectification of test cases are based on the defect detail ledger, and the system automatically performs closed-loop rectification: Remove 3 redundant test cases with identical semantics, and keep the version with the optimal test logic; Complete the two missing boundary value assertions; Correct the order of one logically reversed test step; Four additional test cases were added for implicit scenarios; after rectification, a final set of 39 high-quality, defect-free test cases was obtained.

[0025] S6. Standardized test case output and version archiving: Output the final payment interface test cases in Excel format, along with a test coverage report and a defect rectification details report; At the same time, archive and store the test cases generated in this round to support the comparison of test case differences in subsequent iterations and the rapid reuse of regression test cases.

[0026] Example 2: UI Functionality Test Scenario for Backend Management System This embodiment uses an enterprise backend user permission management page as the test object to execute the system and method described in this invention. The specific implementation process is as follows: Import the page prototype diagram and functional requirement document of the user permission management backend into the system's multimodal data acquisition module. After the module completes data cleaning and format unification, it transmits the requirement data to the requirement layering and parsing module.

[0027] The built-in deep requirement analysis model in the requirement layering and parsing module breaks down page functions into layers, covering four major business branches: page buttons, pop-up interactions, input box validation, and permission switching. At the same time, it uncovers implicit test scenarios such as permission cross-operations and outputs a standardized UI function test breakdown checklist.

[0028] The initial test case generation module automatically generates a full-dimensional set of initial test cases for page functionality based on the breakdown list and UI test writing specifications.

[0029] The AI ​​test case self-inspection module performs full-dimensional defect detection on the initial test cases, identifies logical vulnerabilities and other defects in the cross-permission test cases, and generates a detailed ledger.

[0030] The intelligent test case optimization module automatically repairs logical vulnerabilities, merges redundant test cases, and fills in missing assertions based on the ledger, thus completing test case optimization.

[0031] The final result output archiving module interfaces with an external testing platform, outputting standardized test cases that can be directly written to the MeterSphere testing platform, simultaneously generating accompanying test reports and completing version archiving. This embodiment requires no manual editing or modification by testers and can be directly deployed for testing.

[0032] This system follows a modular operating architecture that prioritizes data flow and business processing. After the raw requirement data enters from the input end, it sequentially completes the entire process of parsing, generating, quality inspection, optimization, and archiving. Each module has clear functional boundaries and works collaboratively. It also supports the reverse feedback of defect detection results, forming a closed-loop optimization capability. The specific functions and implementation mechanisms of each module are as follows: Multimodal data acquisition module: As the data entry point of the system, it supports the import of original requirement data in multiple formats such as documents, images, messages, and flowcharts, adapts to various native requirement carriers in the R&D process, and simultaneously completes the format unification of original data and the cleaning of invalid and redundant content, providing standardized input for subsequent processing.

[0033] The dual-model core module serves as the core processing unit of the system, comprising a large-scale requirement analysis model and a large-scale test case quality evaluation model. The large-scale requirement analysis model focuses on understanding business logic and generating test cases, while the large-scale test case quality evaluation model focuses on detecting test case defects. The two models are physically isolated through model isolation boundaries, and their training parameters, weights, and training datasets are completely separate, interacting only through business data flows. This eliminates blind spots in single-model self-checking from an architectural perspective.

[0034] Test Case Optimization Engine: As the system's optimization execution unit, it has three core algorithms built-in: semantic similarity algorithm, boundary value supplementation algorithm, and logic error correction algorithm. It can automatically perform test case completion, logic error correction, and redundancy deduplication operations based on defect detection results, supporting fully automatic test case rectification and optimization.

[0035] External testing platform interface: Deployed at the system output end, it supports one-click export of mainstream test management platforms and document formats such as Git, Jira, Metersphere, and Excel, achieving seamless compatibility between this system and existing test tool systems, and reducing the cost of implementation and adaptation.

[0036] Results output archiving module: It is responsible for the final output and asset accumulation, and is responsible for outputting standardized test cases and supporting analysis reports. It also has test case version management capabilities, supporting historical version query, version difference comparison and rapid reuse of regression test cases.

[0037] The above-described embodiments are merely illustrative of certain implementations of the present invention, and are described in a relatively specific and detailed manner. However, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements are all within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for automatically writing dynamic, simplified AI test cases driven by defect self-inspection, characterized in that: Includes the following steps: S1. Multimodal test requirement data collection and preprocessing: Collect four types of raw data corresponding to the software under test: structured requirement documents, unstructured business process diagrams, interface messages, and historical defect data. Unify the data format and clean up invalid and redundant text to build a complete test requirement dataset. S2, Layered Intelligent Decomposition of Business Requirements: Input the pre-processed requirement dataset into the deep analysis model of requirements, and decompose the business according to four dimensions: main process branch, abnormal interruption branch, boundary value branch, and permission verification branch, and output a standardized test decomposition list. S3. Fully Automatic Generation of Initial Test Cases: Based on the test breakdown list, combined with the built-in software test case writing specification library, equivalence class partitioning algorithm, and boundary value analysis method, it automatically generates a complete set of initial test cases, including preconditions, operation steps, expected results, assertion parameters, test priorities, and test types. S4. AI-driven full-dimensional test case defect detection: Input the initial test case set into an independent test case quality evaluation model, and detect it line by line from five dimensions: missing scenario coverage, logical conflict of test case steps, duplicate and redundant test cases, missing assertion results, and unreasonable equivalence class partitioning. Mark all defect points and generate a detailed defect ledger. S5. Dynamic optimization and closed-loop rectification of test cases: Based on the defect details log, automatically supplement missing boundary scenarios and abnormal branch test cases, merge redundant test cases with homogeneous content, correct test steps with logical conflicts, and fill in missing assertion content to complete the fully automatic optimization of the initial test cases. S6. Standardized Test Case Output and Version Archiving: Outputs final test cases that conform to industry-standard test templates, along with accompanying reports, and archives versions of test cases generated in each round for future reuse.

2. The method for automatically writing dynamically simplified AI test cases driven by defect self-inspection according to claim 1, characterized in that, Step S2, the hierarchical intelligent decomposition of business requirements, specifically includes: fully decomposing the normal business forward process, while actively mining hidden abnormal scenarios that are not explicitly marked in the requirement documents, making up for the shortcomings of manual decomposition and traditional AI models that ignore hidden test scenarios.

3. The method for automatically writing dynamically simplified AI test cases driven by defect self-inspection according to claim 1, characterized in that: In step S4, the large-scale test case quality evaluation model is trained with a massive number of defective test case samples. The training dataset contains hundreds of thousands of low-quality, high-defect real industrial test cases, which can accurately identify hidden logical vulnerabilities that are difficult for humans to find.

4. The method for automatically writing dynamically simplified AI test cases driven by defect self-inspection according to claim 1, characterized in that, In step S5, the redundant test cases are merged by using a semantic similarity algorithm instead of text keyword matching to identify homogeneous test cases with consistent operational logic but different wording. The optimal test steps are retained, and duplicate content is deleted to achieve test case simplification without information loss.

5. The method for automatically writing dynamically simplified AI test cases driven by defect self-inspection according to claim 1, characterized in that: In step S3, the initial test cases are generated based on the built-in software test case writing specification library, equivalence class partitioning algorithm and boundary value analysis method. Each test case contains all execution elements, including preconditions, operation steps, expected results, assertion parameters, test priority and test type.

6. A defect self-inspection-driven, dynamically simplified AI test case automatic writing system, characterized in that: The method for writing AI test cases according to any one of claims 1-5 includes a multimodal data acquisition module, a requirement layering parsing module, an initial test case generation module, an AI test case self-checking module, a test case intelligent optimization module, and a result output archiving module connected in sequence. The multimodal data acquisition module is used to collect raw data for various test requirements and to perform data cleaning and format unification. The requirement layering and parsing module has a built-in deep requirement parsing model, which is used to complete the multi-dimensional business branch decomposition and output a standardized test decomposition list. The initial test case generation module has a built-in standardized test specification library and classic test algorithms, which can generate complete initial test cases with one click; The AI ​​test case self-inspection module has a built-in independent test case quality evaluation model, which is used to detect various defects in the initial test cases in all aspects and locate the defect locations. The intelligent test case optimization module is used to automatically complete test case completion, error correction, deduplication and simplification based on the detection results; The result output archiving module is used to output standardized test cases and accompanying reports, and at the same time complete test case version management and archiving storage.

7. The defect self-inspection driven dynamic simplification AI test case automatic writing system according to claim 6, characterized in that: The system internally employs a dual-model isolation mechanism, where the large model for in-depth requirement analysis and the large model for test case quality evaluation are independent of each other and do not share parameters, thus avoiding blind spots in detection caused by the integration of a single model generation and self-testing.

8. The defect self-inspection driven dynamic simplification AI test case automatic writing system according to claim 6, characterized in that: The system also has an interface for connecting to external testing platforms, enabling data exchange with mainstream test management platforms and supporting one-click export of test cases in multiple formats.

9. The defect self-inspection driven dynamic simplification AI test case automatic writing system according to claim 6, characterized in that: The accompanying reports output in step S6 include a test case coverage report and a defect remediation report. Version archiving supports subsequent version difference comparisons and rapid reuse of regression test cases.

Images