End-to-end automated test code generation system and method

Abstract

The technical problem solved by the present application is: how to automatically, accurately and verifiably convert the business test intention described by product managers or testers in natural language (such as Excel spreadsheets or comma-separated value files (CSV, Comma-Separated Values)) into executable end-to-end test code that meets project specifications without increasing the coding burden of test engineers. In order to solve the above technical problem, the first aspect of the technical solution of the present application discloses an end-to-end automated test code generation system based on test intermediate representation and AI self-correction closed loop. The second aspect of the technical solution of the present application discloses an end-to-end automated test code generation method based on test intermediate representation and AI self-correction closed loop.

Description

Technical Field

[0001] This invention relates to an end-to-end (E2E) automated test code generation system and method based on intermediate test representation and AI self-correction closed loop, belonging to the fields of software engineering and artificial intelligence (AI). In this invention, "end-to-end" refers to a test method that verifies the complete business process of a software system from the user interface to the backend data by simulating real user operations (such as clicking, filling out forms, and switching pages) through automated programs. Background Technology

[0002] End-to-end testing is currently the mainstream method for quality assurance in web front-end software. Frameworks such as Playwright, Selenium, and Cypress are widely used end-to-end testing tools that verify the correctness of system functions by controlling a real browser to execute test steps and check the page state. The usage of these frameworks involves test engineers manually writing two types of code files: one is the "Page Object (PO)" class—a code abstraction layer used to encapsulate the underlying positioning logic of each interactive element on the page (buttons, input boxes, dropdown menus, etc.) into reusable methods; the other is the "Test Script"—a test program that calls the page object methods, describes the operation steps, and verifies the results. Writing these two types of files specifically requires completing the following:

[0003] (1) Element location (selector writing): In the HTML document object model (DOM, which is the browser's memory representation of the page's HTML structure), write precise location expressions for each operable element. Common element location methods include: locating by the data-testid attribute (a test-specific identifier marked on the HTML tag by the developer), locating by the element's semantic role, or locating by XPath (XML path language, a query language that describes the location of document nodes). When the front-end code is modified, these location expressions must be updated manually in sync, which is extremely costly to maintain. (2) Page Object Writing: Encapsulate business operations such as "Place an order", "Cancel an order", and "Switch accounts" into code methods. Engineers are required to have a deep understanding of the DOM structure of the system under test and the naming convention of data-testid. (3) Test script writing: Translate the business test cases described by the product or testers into the framework application programming interface (API) call sequence, including technical details such as assertion (i.e., automatic verification logic for expected results), asynchronous waiting strategy and test data parameterization.

[0004] The above tasks have high technical barriers and are tightly coupled with the specific implementation of the user interface (UI): whenever a UI component changes (for example, by modifying the data-testid of a button or adjusting the page layout), the relevant page objects and test scripts must be manually modified in sync, and each system iteration may trigger large-scale test code maintenance work.

[0005] In recent years, several solutions have emerged that use artificial intelligence (AI) to assist in generating test code, but they all suffer from the following fundamental flaws: (1) Recording and replay tools (such as Playwright Codegen, Cypress Studio): Automatically generate test scripts by recording the developer's actual operation behavior in the browser. The drawback is that the generated scripts directly depend on the DOM snapshot at the moment of recording (i.e. the HTML structure of the page at that time). Once the UI structure changes, the scripts will immediately become invalid, and they cannot understand and generate assertion logic at the business semantic level (such as "after clicking to buy, verify that a new record is added to the order list"). (2) Large Language Model (LLM) directly generates test code: Large Language Model is a type of AI model that is trained on massive amounts of text and can understand and generate natural language and code (such as GPT, Claude, etc.). Although LLM can generate test code based on natural language descriptions, it has the following fundamental defects: (a) Hallucination problem: When generating test code, LLM cannot perceive the actual list of page objects (POs) and methods in the project under test. Therefore, it often generates method names that do not exist in the project, or call code with incorrect number and type of parameters. This phenomenon of AI model "confidently generating non-existent content" is called "hallucination". Such code will immediately report an error at runtime, and engineers must manually review and eliminate it line by line. (b) Lack of self-verification capability: After the LLM generates test code, it cannot perform verification on its own. It needs to be manually reviewed line by line before it can be used, which is time-consuming and labor-intensive. (c) Unstable quality: The same test requirement may produce code with significant quality differences in different generation rounds, lacking an engineering mechanism to ensure stable output quality; (d) Inability to reuse existing test infrastructure: LLM does not understand the project's existing fixture system, utility function library and other test infrastructure, and the generated code cannot be integrated with it, forming isolated "disposable" code.

[0006] In summary, the existing technical solutions have the following unresolved technical gaps, and the technical solutions disclosed in this invention can fill these technical gaps: (1) There is a lack of a UI-agnostic test semantic intermediate representation layer suitable for AI processing. Existing tools either directly record DOM operations (strongly coupled with UI) or directly generate code from natural language (skipping the semantic verification process, making it difficult to guarantee quality). (2) Lack of a structured knowledge base that is automatically built based on real project code and can be accurately queried by AI - LLM cannot automatically perceive the existing Page Object (PO) methods in the project, which makes the illusion problem impossible to eradicate; (3) Lack of automated execution verification and closed-loop error correction mechanism after AI generates test code - whether the generated code can run and whether the running results meet expectations still depends on manual verification; (4) Lack of interactive tools to integrate the above capabilities into a complete visual workflow - engineers lack a smooth working interface between test requirement input, AI processing, result review and code application. Summary of the Invention

[0007] The technical problem this invention aims to solve is: how to automatically, accurately, and verifiably convert business test intentions described by product managers or testers in natural language (such as Excel spreadsheets or comma-separated values ​​(CSV) format) into executable end-to-end test code that conforms to project specifications without increasing the coding burden on test engineers.

[0008] To address the aforementioned technical problems, the first aspect of the present invention discloses an end-to-end automated test code generation system based on intermediate test representation and an AI self-correction closed loop. This system comprises four subsystems: a three-layer knowledge base system, an AI pipeline, an AI self-correction service, and a test execution infrastructure. These four subsystems work collaboratively, exchanging information through clearly defined data formats to form a complete test code generation pipeline. The three-layer knowledge base system is used to store and maintain the structured knowledge of the project under test, which can be queried by the test code generation AI model at each stage of test code generation. Specifically, it includes the page index file index.json, the business interaction definition file page_behavior.json, and the page object method metadata file meta.json. The AI ​​pipeline is used to perform two-stage AI processing on CSV / Excel format test case documents, and finally output executable test code in TypeScript format. The two AI processing stages are: a test intent parser, which is used to convert the original test cases into a test intermediate layer representation TestIR that is independent of the UI implementation; and a test specification generator, which is used to convert the test intermediate layer representation TestIR into executable test code in TypeScript format. When generating executable test code, the test specification generator only calls the methods recorded in the page object method metadata file meta.json. The AI ​​self-correction service uses the model context protocol to expose the control capabilities of the real browser to the test code generation AI model, enabling the test code generation AI model to execute the generated test code in the real browser, observe the actual execution results, and autonomously correct errors in the code until the code passes verification. The test execution infrastructure consists of the Playwright test framework, the layered fixture dependency injection system, and the TestUtils atomic operation toolset, and is the underlying environment upon which the final generated test code depends for execution.

[0009] Preferably, the page index file index.json is used to maintain an index table of all implemented page object classes in the project under test. Each record in the index table includes the page ID, Chinese semantic name, corresponding TypeScript file path, and the file paths of the associated business interaction definition file page_behavior.json and the page object method metadata file meta.json. The business interaction definition file page_behavior.json is used to describe what the current page "can do" at the business level. Each business interaction record in the business interaction definition file page_behavior.json includes: a unique interaction identifier, a triggering method, a parameter list, and an operation result. The meta.json file, representing the method metadata of the page object, is automatically generated by the TypeScript abstract syntax tree parser. It records the complete signature information of all implemented code methods in each page object class, including: method name, parameter list, return type, and JSDoc documentation comment summary.

[0010] Preferably, all page object classes have the @PageObject decorator added above their class definitions, and the cnNames parameter declares a list of Chinese business semantic names corresponding to the current class. The Chinese names in the cnNames parameter correspond one-to-one with the semantic names in the page index file index.json. The TypeScript abstract syntax tree parsing tool is used to analyze the abstract syntax tree to extract structural information from the code without executing code or manually writing parsing rules. It extracts metadata using the following steps to generate the page object method metadata file meta.json: Step 101: Call the TypeScript compiler application programming interface to create a compiler and perform a full TypeScript abstract syntax tree traversal on all .ts files in the pages / directory; Step 102: Scan the class declaration node in the TypeScript abstract syntax tree of each file and check if its decorator list contains a calling expression named "PageObject". Step 103: For the identified @PageObject class, parse the cnNames parameter value of the decorator, establish a mapping relationship of "Chinese semantic name -> class name -> file path", and write it into the page index file index.json; Step 104: Traverse all method declaration nodes of the current class and extract the method name, parameter list, and return type; Step 105: Read the JSDoc block above each method declaration node and extract the text content of the @description, @param, and @returns tags; Step 106: Serialize the complete method signature information extracted in steps 104 and 105 into JSON format and write it into the meta.json file of the corresponding page object method metadata file.

[0011] Preferably, the test intermediate layer represents TestIR including: TestAction describes a single operation step in a test case. It includes the action field, which represents the action type; the page field, which represents the semantic page name; the target field, which represents the semantic element name; the value field, which represents the operation value; the assertions field, which represents the assertion list attached to the current step; the confidence field, which represents the confidence level of the semantic parsing of the current step; and the tbd field, which represents the placeholder list to be confirmed. An assertion is used to describe the expected state verification of a page element. It includes a type field to represent the assertion type, a target field to represent the semantic name of the element being asserted, and an expected field to represent the expected value. The placeholder TBDPlaceholder is inserted when there is missing information in the test step description. It includes a type field to indicate the type of missing information, a key field to indicate the identifier key of the missing information, and a description field to indicate the text description of the missing information. The test middle layer represents the TestIR body, which includes an id field that is a unique identifier for test cases, a title field that represents the name of the test case, a precondition field that represents the preconditions, an actions field that represents the list of TestActions, and an overallConfidence field that represents the overall confidence level.

[0012] Preferably, the confidence field for each test action is calculated by weighting the following four sub-dimensions: Sub-dimension 1, semantic matching degree, with a weight of 40%, is used to represent the word overlap rate between the original text of the calculation steps and the business interaction description in the business interaction definition file page_behavior.json. Word overlap rate = number of overlapping words / number of words in the union of the two text segments; Sub-dimension 2, parameter completeness, with a weight of 25%, is used to calculate the proportion of required parameters that have been successfully extracted from the original text of the steps. The percentage of successful extraction of all required parameters is 1.0. For each missing parameter, the percentage is reduced proportionally. Placeholders for unextracted parameters are inserted at the corresponding positions. Sub-dimension 3, Page Location Reliability, with a weight of 20%, is calculated based on the number of candidate pages matched in the page index.json in the original steps: a unique match is scored as 1.0, matching N candidate pages is scored as 1 / N, and no match is scored as 0.0. Sub-dimension 4, Action Type Clarity, with a weight of 15%, is used to determine whether the step verbs precisely correspond to the trigger events defined in the current business interaction definition file page_behavior.json: a precise correspondence is scored as 1.0, while using ambiguous verbs is scored as 0.4.

[0013] Preferably, when there is semantic missingness in the test step description, the test intent parser does not infer the missing value or skip the current step, but instead inserts a structured placeholder for confirmation at the corresponding position and records the reason for the missing value. When the downstream test specification generator encounters a placeholder that needs to be confirmed, it generates test.fixme() instead of attempting to complete it, and at the same time, it explains the missing information in the comments.

[0014] Preferably, in the AI ​​self-correction service, the test specification generator and the test intent parser implement the self-correction closed loop using the following steps: Step 201: Generate test code using the AI ​​model. Generate an initial version of TypeScript test code based on the TestIR (test intermediate layer representation) and the meta.json (page object method metadata file). Step 202: Test code generation AI model. The AI ​​model is simulated to execute the operation sequence described in the test code in a real browser using the Model Context Protocol tool; Step 203: After each operation step is executed, the test code generates an AI model and collects the execution results through the Model Context Protocol tool; Step 204: Test code generation AI model compares the actual state observed in step 203 with the expected state defined in TestIR, representing the test intermediate layer, to determine if there is a difference. If there is a difference, the root cause is analyzed and a code correction patch is generated. Step 205: Re-execute steps 202-204 of the corrected code, looping for a maximum of N rounds. After all steps have passed verification, output the final code in a unified difference format. If any step still fails after reaching the maximum number of loops, add a summary of errors that still require manual processing to the difference file.

[0015] Preferably, in the layered fixture dependency injection system, a bottom-up pyramid-shaped dependency chain is adopted, including: a bottom layer config.ts, used to load test configuration; a middle layer login.ts, which depends on the bottom layer config.ts, performs login operations, and provides the logged-in browser page context to the upper layer; a menu layer menu.ts, which depends on the middle layer login.ts, provides navigation capabilities between pages and a common pop-up object; and a functional layer fixtures, which depends on the menu layer menu.ts, with one file for each business module, providing the corresponding page object instance. Furthermore, in the hierarchical fixture dependency injection system, the following rules are enforced: All test files must import the test object from the fixtures file of the functional layer that is closest to the business logic; direct import from @playwright / test is prohibited. Directly writing element location expressions is prohibited in test files and function layer fixtures files. All element access must be done through the encapsulated methods of the page object to ensure centralized management of location logic. Navigation between pages must be handled uniformly through MenuPage; hardcoding URLs in functional pages is prohibited.

[0016] Preferably, the TestUtils atomic operation toolkit contains five atomic operation functions, which are injected into all tests through a layered fixture dependency injection system to ensure the consistency and correctness of asynchronous processing. Specifically, these include: The pollUntil function polls until the condition is met: internally, it calls the expect.poll() API of the Playwright testing framework, repeatedly calling the asynchronous function fn at specified intervals, and passing the return value of the asynchronous function fn to the condition judgment function predicate, until the condition judgment function predicate returns true or a timeout occurs; The asynchronous retry assertion function expectEventually continuously calls the asynchronous function fn, passing the return value of the asynchronous function fn to the function containing the expect(...) call, and retryes continuously within the timeout period until the assertion passes or an error occurs. The atomic combination function `actAndWait` combines operations and waits: first, the asynchronous function that triggers the operation is executed; then, the function `pollUntil` polls until the condition is met, and the function `read` polls until the condition is met; The retry function `retry` will retry the asynchronous function `fn` a maximum of `times` times, catching any exceptions and re-executing it each time. The pollWithLog function has the same functionality as the pollUntil function, which polls until a condition is met, but it additionally prints the current return value to the test standard output stream with each poll.

[0017] The second aspect of the technical solution of this invention discloses an end-to-end automated test code generation method based on intermediate test representation and AI self-correction closed loop. Employing the aforementioned end-to-end automated test code generation system, the method includes the following steps: Step 1: Test engineers upload test case documents in CSV or Excel format through the visual pipeline front-end interface; Step 2: The system automatically parses the document and maps each line of test cases to the original test case data structure. Engineers can manually correct parsing errors in the front-end table. Step 3: The test intent parser reads the knowledge base, converts the natural language descriptions in the original test cases into a structured test intermediate layer representation TestIR, and labels the semantic parsing confidence of each step; Step 4: Engineers review the TestIR (Test Intermediate Layer) on the front-end interface of the visual pipeline. After confirming that the semantic meaning of each test step is correct, the next stage is triggered. Step 5: The test specification generator reads the list of page object methods recorded in the page object method metadata file meta.json, maps each semantic step in the test intermediate layer representation TestIR to the corresponding code method call, and generates TypeScript test code; Step 6: The MCP self-correction service executes the generated code step by step in the real browser and feeds back the execution results to the test code to generate the AI ​​model. The test code generates the AI ​​model to correct the code accordingly. This process is repeated until all steps are executed successfully. Step 7: Finally, the verified test code is output in a unified difference format. After the engineer previews the changes in the visual interface, he can write the code into the code repository with one click.

[0018] To achieve the goal of automatically, accurately, and verifiably converting business test intentions described in natural language by product managers or testers into executable end-to-end test code that conforms to project specifications without increasing the coding burden on test engineers, the technical solution disclosed in this invention solves the following problems: (1) Semantic representation problem: This invention designs a test semantic intermediate representation between natural language and code, so that it can be intuitively reviewed by human test engineers and reliably structured by AI; (2) Eliminate illusions: When generating test code, the AI ​​will automatically detect and only reference the methods of the actual page objects (POs) in the project, thus eliminating the generation of illusion code; (3) Automatic error correction: After the AI ​​generates the test code, it can automatically complete the code feasibility verification and multiple rounds of error correction without human intervention; (4) Uncertainty handling: When there are semantic gaps or ambiguities in the test intent description, the uncertainty of AI is exposed to the engineers in a structured way, rather than silently generating error code; (5) Timing stability issues: Reduce timing instability issues in asynchronous UI environment testing from the code generation mechanism level (known in the industry as "unstable testing", Flaky Test).

[0019] Therefore, the technical solution disclosed in this invention has the following beneficial effects compared with the prior art: (1) It fundamentally eliminates the illusion of large language models and significantly improves the usability of generated code. Through a three-layer knowledge base system (especially the meta.json automatically generated by the TypeScript AST), the test specification generator (AI Skill generate-test-spec) only calls the actual methods recorded in meta.json during the code generation phase, without relying on the "guessing" of the large language model. This fundamentally eliminates the illusion that the large language model generates methods without names. Compared with existing direct generation solutions using large language models, the direct runnability rate of the generated code is significantly improved, and the cost of manual review is greatly reduced. (2) Achieve multi-round automatic error correction without human intervention, and fully automate the debugging process. The MCP self-correction closed loop employed in this invention enables AI to autonomously correct and generate code based on the actual execution results of a real browser (screenshots, console logs, DOM snapshots), completely automating the process that traditional solutions require engineers to repeatedly debug, modify, and rerun. In particular, it can capture and correct errors that traditional static code reviews cannot detect, such as insufficient waiting time and runtime issues like discrepancies between actual and expected UI text. (3) Lower the technical threshold for test writing so that non-technical QA personnel can participate in test review. The TestIR middleware introduced in this invention completely decouples the "business test intent" from the "Playwright test framework technical details". Test engineers and quality assurance (QA) personnel only need to review the Chinese semantic step descriptions in the TestIR middleware representation in the visual interface, without needing to understand the specific usage of TypeScript code or Playwright test framework API, to complete the entire process of conversion from business test cases to executable test code. (4) Decoupling from UI implementation to minimize test code maintenance costs. The TestIR and knowledge base system work together to ensure that when the testId of a UI component, the DOM structure, or the method signature of a page object changes, you only need to rerun pnpm pageobject-meta to update meta.json. There is no need to re-parse natural language test cases or modify the TestIR data. The impact of the change is precisely limited to the corresponding Page Object file and meta.json, minimizing the test maintenance workload caused by UI changes. (5) Systematically reduce unstable tests from the code generation mechanism level. The TestUtils atomic operation toolkit provided by this invention converges the asynchronous UI waiting mode into five semantically clear functions. Combined with the IR-driven method recommendation mechanism, it ensures that each test step generated by AI uses the correct waiting strategy, avoiding the two root causes of Flaky Tests: "premature assertion" and "hard waiting". Compared with the practice of randomly selecting waiting methods when writing tests manually, the code generated by the system has a consistent and correct asynchronous processing mode. (6) The knowledge base and code are automatically synchronized, eliminating the cost and errors of manual document maintenance. The TypeScript AST automatic extraction mechanism provided by this invention ensures automatic synchronization between meta.json and the code implementation: after each modification of the page object class, the engineer can update the knowledge base by running pnpm pageobject-meta. The whole process does not require manual editing of any documents. Compared with the traditional method of manually maintaining test documents, it eliminates the risk of inconsistency between documents and code, and also eliminates the problem of AI querying outdated method information due to document lag. (7) The test intermediate layer represents TestIR as a multi-purpose semantic asset, which has reusability value that goes beyond test code generation. TestIR, the Test Intermediate Layer Representation, is a structured semantic representation of test intent. Its value is not limited to driving the Playwright testing framework to generate test code. The same TestIR can also be used to: generate Markdown format test documents (for product managers to read and accept); compare with the requirements document to verify the test case coverage of business functions; drive the generation of API Testing scripts (mapping "click submit" to the corresponding HTTP request); and serve as a benchmark for test case semantic version management (quickly detect whether the test intent has changed by diffing TestIR). Attached Figure Description

[0020] Figure 1 The system architecture diagram shows the data flow arrows, calling relationships, and data format annotations (RawTestCase, TestIR, TypeScript code, unified diff, etc.) between the four subsystems (three-layer knowledge base system, AI pipeline, MCP AI self-correction service, and test execution infrastructure). Figure 2This is a two-stage AI pipeline workflow diagram, illustrating the complete processing flow from CSV / Excel file input to AI Skill directly writing to test files. It labels the input / output data format and the knowledge base files used at each stage: CSV / Excel -> RawTestCase -> test-intent-parser (querying index.json + page_behavior.json) -> TestIR -> generate-test-spec (querying meta.json) -> TypeScript test code -> MCP self-correction (including sample restoration in step 0) -> Validation passed -> Directly written to tests / .spec.ts; Figure 3 The TestIR data structure diagram shows the fields and inter-reference relationships of the four data structures: TestIR, TestAction, Assertion, and TBDPlaceholder. It also marks the position and grading threshold of the Confidence field in TestAction. Figure 4 The diagram shows the hierarchical relationship of the three files index.json, page_behavior.json, and meta.json, as well as the information fields they store, and which knowledge base the two AI Skills test-intent-parser and generate-test-spec query respectively. Figure 5 The flowchart for extracting TypeScript AST metadata illustrates the complete algorithm steps from triggering the pnpm pageobject-meta command to writing to meta.json, including: the TypeScript compiler creating a Program, traversing the AST node tree, recognizing the @PageObject decorator, extracting the MethodDeclaration signature, reading JSDoc comments, and serializing to JSON at each processing node; Figure 6 The flowchart for the MCP self-correction closed loop illustrates the complete self-correction process: Step 0 (MCP example step restoration) replays the TestIR operation sequence and collects ground truth snapshots, then enters the closed loop of "Step 1 Generate initial code -> Step 2 MCP execution -> Step 3 AI observation (screenshot / log / DOM) -> Step 4 Difference analysis -> Patch correction -> Re-execution", marking the name of the MCP tool used in each step and the AI ​​decision branch (pass / correct / exceeded number of attempts to report an error). Figure 7This diagram illustrates the dependency chain of hierarchical fixtures, showing a pyramid-shaped unidirectional dependency relationship from config.ts to login.ts to menu.ts to functional fixtures (t0-order.ts, etc.), as well as the object types provided by each layer of fixture (configuration object, page context, MenuPage, business page object). Figure 8 To visualize the pipeline front-end interface, the interface layout of the three interactive stages (FileUpload / RawDataEditor / TestIRViewer) is shown. The confidence level color warning (yellow / red), TBD placeholder highlighting in TestIRViewer, and the work endpoint where the "Export IR" button triggers AISkill to directly write to the test file are highlighted. Detailed Implementation

[0021] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0022] The technical solutions disclosed in the embodiments of this invention involve the following core technologies: (1) Test Intermediate Representation (TestIR) – a semantic structure between natural language test cases and executable test code; (2) A three-tiered structured knowledge base system; (3) Automatic extraction of TypeScript Abstract Syntax Tree (AST) metadata - automatically reading the project code structure using compiler-level static code analysis technology; (4) Fixture dependency injection – a mechanism for modular management of test preconditions; (5) AI self-correction closed loop based on Model Context Protocol (MCP); (6) Multi-stage visualized pipeline front end.

[0023] Based on the aforementioned core technologies, the first aspect of this invention is to disclose an end-to-end automated test code generation system based on intermediate test representation and AI self-correction closed loop, which consists of four collaborative subsystems. Each subsystem transmits information to the other through a clearly defined data format to form a complete test code generation pipeline.

[0024] (I) The first subsystem is a three-layer knowledge base system. This invention constructs a three-layer structured knowledge base specifically for optimizing AI model queries in test code generation. This is a core technical means to eliminate the LLM illusion problem. The three-layer knowledge base system is used to store and maintain the structured knowledge of the project under test, which is used by the test code generation AI model at each stage of test code generation. Specifically, it includes the page index file index.json, the business interaction definition file page_behavior.json, and the page object method metadata file meta.json. These are queried sequentially in the test code generation AI model processing flow. If any layer is missing, a degradation process is triggered, such as inserting a TBD placeholder or generating a stub comment.

[0025] The first layer of the three-layer knowledge base system is the page index (ai_index / pages / index.json): This uses a JSON (JavaScript Object Notation, a lightweight data interchange format) file to maintain an index table of all implemented page object classes in the project under test. Each record in the index table includes: page ID, Chinese semantic name (e.g., "T0 Order Page"), the corresponding TypeScript file path, and the paths to the associated page_behavior.json and meta.json files. When parsing each test step, the test intent parser (AI Skill test-intent-parser) first queries this JSON file to confirm the existence of the target page.

[0026] The second layer of the three-layer knowledge base system is the business interaction definition (page_behavior.json): Each page in the project under test corresponds to a JSON file, manually maintained by the test engineer, used to describe what the current page "can do" (rather than "how to do") at the business level. Each business interaction record includes: a unique identifier (id), a trigger method (including event type and trigger element), a parameter list (params, including parameter name, whether it is required, and description), and the operation result (including UI changes and system state changes). The test intent parser (AISkill test-intent-parser) queries this JSON file to ensure that each test action (TestAction) maps to a real business interaction.

[0027] The third layer of the three-layer knowledge base system is the page object method metadata (meta.json): Each page in the project under test corresponds to a JSON file, automatically generated by the TypeScript (a statically typed JavaScript superset programming language) Abstract Syntax Tree (AST) parser. This file records the complete signature information of all implemented code methods in each page object class, including: method name, parameter list (including parameter names and TypeScript types), return type, and a summary of JSDoc (JavaScript Documentation Comments) documentation comments. When generating test code, the test specification generator (AI Skill generate-test-spec) only calls the methods recorded in this file, without relying on the test code to generate "speculations" for the AI ​​model, fundamentally eliminating illusionary code. If a business operation does not have a corresponding method in the page object method metadata file meta.json, a method stub is generated, prompting engineers to complete the implementation before rerunning the metadata extraction.

[0028] In a preferred embodiment of this invention, all page object classes have the `@PageObject` decorator added above their class definitions, and the list of Chinese business semantic names corresponding to the current class is declared in their `cnNames` parameter, for example: `cnNames: ["T0 Order Page", "Underlying Warehouse Enhanced Transaction Page"]`. The Chinese names in the `cnNames` parameter correspond one-to-one with the semantic names in the page index file `index.json`, serving as the sole bridge between the test code generation AI model and the "Chinese test steps" and the "TypeScript class". The decorator is a syntax feature provided by the TypeScript language that allows annotations starting with "@" to be added above the class definition, thereby attaching additional meta-information or behavior.

[0029] In another preferred embodiment of the present invention, the TypeScript Abstract Syntax Tree parsing tool is used to analyze the Abstract Syntax Tree (AST) and can accurately extract structural information from the code without executing the code or manually writing parsing rules. The Abstract Syntax Tree is a tree-like data structure generated by the compiler after parsing the source code, and each node represents a syntactic unit in the code (class declaration, method declaration, parameter list, etc.).

[0030] After executing the command `pnpm pageobject-meta`, the TypeScript abstract syntax tree parser extracts metadata using the following steps to generate the page object method metadata file `meta.json`: Step 1: Call the TypeScript compiler application programming interface (ts.createProgram) to create a compiler and perform a full AST traversal of all .ts files in the pages / directory; Step 2: Scan the class declaration node in the AST of each file and check if its decorator list contains a call expression named "PageObject". One specific implementation is to check if the expression field of the node is an identifier and if the text value is equal to "PageObject".

[0031] Step 3: For the identified @PageObject class, parse the cnNames parameter value of the decorator, establish a mapping relationship of "Chinese semantic name -> class name -> file path", and write it into the page index file index.json; Step 4: Traverse all method declaration nodes (MethodDeclaration) of the current class, and extract the method name (text value of the method identifier), parameter list (traverse the parameters array and extract the name and text representation of the type annotation for each parameter), and return type (extract the text representation of the type node). Step 5: Read the JSDoc block above each method declaration node and extract the text content of the @description, @param, and @returns tags; Step 6: Serialize the complete method signature information extracted in Steps 4 and 5 into JSON format and write it into the meta.json file of the corresponding page object method (overwrite if the file already exists).

[0032] In another preferred embodiment of the present invention, an incremental update mechanism is set up: each time pnpmpageobject-meta is executed, the script compares the file system modification timestamp (mtime) of each .ts file with the timestamp recorded during the last extraction, and only re-executes steps 2 to 6 for files with updated timestamps, so as to ensure the execution speed in large projects.

[0033] (ii) The second subsystem is the AI ​​pipeline, which performs two-stage AI processing on the CSV / Excel format test case documents uploaded by test engineers, and finally outputs executable test code in TypeScript format. The two AI processing stages are: Test Intent Parser (AI Skill test-intent-parser), which is used to convert the original test cases into TestIR (Test Intermediate Layer Representation); and Test Specification Generator (AI Skill generate-test-spec), which is used to convert TestIR into executable test code in TypeScript format.

[0034] In existing technical solutions, when test code generation AI models directly generate code from natural language test cases, they must simultaneously undertake two tasks: first, understanding the business semantics of the test steps (what does "clicking the buy button" mean); and second, generating code that conforms to project specifications (which method to call and what parameters to pass). Combining these two tasks into a single AI call means that errors in either will lead to code errors, which are difficult to locate.

[0035] This invention splits the two tasks mentioned above into two independent stages and introduces a test intermediate layer representation, TestIR, which is independent of the UI implementation, between the two stages as a "translation buffer" between natural language test intent and executable test code. TestIR uses structured data (rather than code) to describe "what to do," allowing engineers to directly review and modify the test intent before generating code, and to confirm the semantic correctness before proceeding to the code generation stage.

[0036] The Test Intermediate Layer (TIIL) consists of the following four data structures: (1) Test Action: Used to describe a single operation step in the test case, including the following fields: action field (action type, enumeration values: navigate / click / fill / select / wait / assert), page field (semantic page name, such as "T0 order page", corresponding to the semantic name in the three-layer knowledge base system), target field (semantic element name, such as "buy submit button"), value field (operation value, such as the specific content filled in), assertions field (assertions attached to the current step), confidence field (confidence of semantic parsing of the current step, 0.0~1.0), tbd field (a list of placeholders for To Be Determined (TBD) used to mark the semantically missing parts).

[0037] In a preferred embodiment of the present invention, the confidence field for each test action is calculated by weighting the following four sub-dimensions: (a) Semantic matching degree (weight 40%): Calculate the word overlap rate between the original text of the steps and the business interaction description in the business interaction definition file page_behavior.json. The Jaccard similarity algorithm is used: word overlap rate = number of overlapping words / number of words in the union of the two texts. The higher the overlap rate, the more the step description matches the known business interaction.

[0038] (b) Parameter completeness (weight 25%): Calculate the proportion of required parameters (required=true) that have been successfully extracted from the original text of the steps. 1.0 is obtained when all required parameters are successfully extracted. The value is reduced proportionally for each missing parameter. A TBD placeholder is inserted at the position of the parameter that was not extracted.

[0039] (c)Page positioning confidence (weight 20%): Calculated based on the number of candidate pages matched in the page index index.json according to the step text: 1.0 for a unique match, 1 / N for N candidate pages matched (the more candidates, the lower the certainty), and 0.0 for no match.

[0040] (d)Action type clarity (weight 15%): Determine whether the step verb ("click", "input", "select", "wait") exactly corresponds to the trigger event (trigger.event) of the current business interaction definition in the business interaction definition file page_behavior.json: 1.0 for an exact match, and 0.4 for using a vague verb (such as "operate", "process").

[0041] Confidence grading processing strategy: Score 0.85 and above (high confidence) - directly enter code generation without manual confirmation; 0.70 - 0.85 (medium confidence) - marked with a yellow warning in the front end, and it is recommended that engineers confirm; 0.50 - 0.70 (low confidence) - marked with a red warning in the front end, and engineers must manually confirm before proceeding; below 0.50 (blocking) - the system refuses to generate code for this step and forces engineers to re-describe the step.

[0042] In an embodiment of the present invention, an optional solution is that the calculation of confidence is performed by a dedicated tool function calculate_confidence. The test code generation AI model is responsible for semantic understanding and extracting input values for each dimension, and the tool function performs deterministic numerical calculations. Their responsibilities are strictly separated to ensure that the confidence result can be fully reproduced under the same input and is not affected by the randomness of the model.

[0043] In an embodiment of the present invention, a better implementation is to design a TBD placeholder mechanism. When there is a semantic missing in the test step description (for example, "input amount" but no specific value is given), the test intent parser (AI Skilltest-intent-parser) adopts the following processing method: do not speculate on the missing value, do not skip this step, but insert a structured TBD placeholder (such as "<TBD:param:entrusted price>") at the corresponding position and record the reason for the missing.

[0044] When the downstream test specification generator (AI Skill generate-test-spec) encounters a TBD placeholder, it generates test.fixme() (the "mark as to be fixed" API provided by the Playwright test framework) instead of trying to complete it, and at the same time writes a description of the missing information in the comment to help engineers quickly locate the content that needs to be supplemented. This mechanism converts "AI uncertainty" into a clear action item that engineers can operate on, rather than letting the AI silently generate incorrect code.

[0045] (2) Assertion: Used to describe the expected state verification of a page element, including: type field (assertion type: visible / text / value / state), target field (semantic name of the element being asserted), and expected field (expected value).

[0046] (3) Placeholder to be confirmed (TBDPlaceholder): Inserted when there is missing information in the test step description, including: type field (missing type: param parameter missing / ui interface information missing / assert assertion target missing / page page ownership missing / data test data missing), key field (identification key of missing information), description field (textual description of missing information).

[0047] (4) The test intermediate layer represents the TestIR body: including: id field (unique identifier of test case), title field (test case name), precondition field (precondition), actions field (list of TestAction, arranged in order of steps), overallConfidence field (overall confidence, calculated by weighted average of the confidence of all steps).

[0048] (iii) The third subsystem is the AI ​​self-correction service, which uses the Model Context Protocol (MCP) to expose the control capabilities of the real browser to the test code generation AI model, so that the test code generation AI model can execute the generated test code in the real browser, observe the actual execution results, and autonomously correct the errors in the code until the code passes the verification.

[0049] The Model Context Protocol (MCP) used in this invention is a standardized communication protocol that allows test code generation AI models to call external tools. Through MCP, the test code generation AI model can send call requests (request name and parameters) to external services. After the external service performs the actual operation, it returns the result to the test code generation AI model, which then continues reasoning and decision-making based on the result. This invention utilizes the chrome-devtools MCP service to expose the control capabilities of the Google Chrome browser to the test code generation AI model, enabling the test code generation AI model to operate and observe the system under test in a real browser, just like a human test engineer.

[0050] In a preferred embodiment of the present invention, the test specification generator (AISkill generate-test-spec) and the test intent parser (AI Skill test-intent-parser) in the AI ​​self-correction service jointly implement a self-correction closed loop based on the following five-step cycle: Step 1 (Generation): The AI ​​model generates initial TypeScript test code based on the TestIR (test intermediate layer representation) and the meta.json (page object method metadata file).

[0051] Step 2 (Execution): The AI ​​model generates test code and simulates the sequence of operations described in the test code in a real browser using the MCP tool. Specifically, it calls: `mcp__chrome-devtools__navigate_page` to navigate to the target page URL; for each operation step, it sequentially calls tools such as `mcp__chrome-devtools__click` (clicking an element) and `mcp__chrome-devtools__fill` (filling in an input box), and uses `mcp__chrome-devtools__wait_for` to wait for the asynchronous state to stabilize.

[0052] Step 3 (Observation): After each operation step is executed, the test code-generated AI model collects the execution results using the following tools: mcp__chrome-devtools__take_screenshot (captures the current page image; the AI ​​analyzes the page state using multimodal vision capabilities); mcp__chrome-devtools__get_console_message (reads browser console logs and captures JavaScript runtime errors); mcp__chrome-devtools__evaluate_script (executes JavaScript expressions in the browser and reads the actual attribute values ​​or text content of elements); mcp__chrome-devtools__list_network_requests (monitors HTTP network requests and checks whether API calls were successful).

[0053] Step 4 (Analysis and Correction): The test code generation AI model compares the actual state observed in Step 3 with the expected state defined in TestIR, representing the intermediate test layer. If discrepancies exist, the root cause is analyzed and a code correction patch is generated. In this embodiment of the invention, common types of discrepancies and corresponding correction methods include: Actual text value does not match expected assertion value – correct the expected field of the assertion; Element not found (timeout) – check if testId is correct or add a pre-wait; Operation side effects have not yet completed – extend or shorten the polling interval of pollUntil; Network request failed – add necessary system state initialization in the pre-test steps.

[0054] Step 5 (Loop or Output): Re-execute steps 2-4 with the corrected code, looping a maximum of N times (N is determined by the Skill configuration parameter, defaulting to 3 times). After all steps have passed verification, output the final code in a unified diff format. If any step fails after reaching the maximum number of loops, append a summary of errors requiring manual processing to the diff file.

[0055] (iv) The fourth subsystem is the test execution infrastructure, which consists of the Playwright test framework, the layered fixture dependency injection system, and the TestUtils atomic operation toolkit. It is the underlying environment on which the final generated test code depends for execution.

[0056] Fixtures are a test precondition management mechanism provided by the Playwright testing framework: they encapsulate test preparation tasks such as "logging into the system" and "opening a specific page" into modules that can be declaratively referenced by test files. Tests only need to declare "which fixtures I need", and the framework automatically completes the corresponding initialization work in sequence.

[0057] In a preferred embodiment of the present invention, a strictly hierarchical fixture dependency injection system is designed to form a bottom-up pyramid-shaped dependency chain, including: (1) The underlying config.ts is used to load the test configuration: basic environment parameters such as server address and account information; (2) The middle layer login.ts depends on the underlying config.ts, performs the login operation, and provides the logged-in browser page context to the upper layer; (3) The menu layer menu.ts depends on the intermediate layer login.ts, providing navigation capabilities between pages and a common pop-up object; (4) Functional fixes depend on menu.ts. Each business module has one file, which provides the corresponding page object instance. For example, t0-order.ts provides T0OrderPage instance, and split-order.ts provides SplitOrderPage instance.

[0058] In a layered fixture dependency injection architecture, the system enforces the following rules: (1) All test files must import the test object from the fixtures file of the functional layer closest to the business logic. It is forbidden to import it directly from @playwright / test (the original package of the Playwright test framework) - this rule ensures that each test is initialized through a complete dependency chain; (2) Element location expressions (e.g., getByTestId, XPath, etc.) must not be written directly in test files and function layer fixtures files. All element access must be done through the encapsulation methods of the page object to ensure centralized management of the location logic. (3) Page navigation must be handled uniformly through MenuPage, and hardcoding of URLs in functional pages is prohibited.

[0059] The instability of end-to-end testing stems from the fact that modern web application UIs are rendered asynchronously. When a user triggers an action (such as clicking the "submit" button), the change in page state (such as adding a row to the delegate list) does not occur immediately, but rather asynchronously after a delay of several milliseconds to seconds. If the test code executes assertions immediately after the action, the assertions often fail because the page has not yet finished updating. However, the same test may occasionally pass when network conditions are good and server response is fast. This "sometimes fast, sometimes slow" random failure is known as Flaky Testing.

[0060] In another preferred embodiment of the present invention, the asynchronous waiting scenario described above is abstracted into five atomic operation functions in the TestUtils atomic operation toolkit. These functions are then injected into all tests through a layered fixture dependency injection system to ensure the uniformity and correctness of the asynchronous processing method. Specifically, this includes: (1) pollUntil(fn, predicate, options) — Polling until the condition is met: Internally, it calls the Playwright testing framework's expect.poll() API, repeatedly calling fn (an asynchronous function that returns the current value) at intervals specified by options.intervals (incrementing by 100 / 250 / 500 / 1000 milliseconds by default), passing the return value to the condition judgment function predicate, until the condition judgment function predicate returns true or the options.timeout timeout is reached. This invention uses expect.poll instead of a self-implemented loop to ensure that the timeout behavior is completely consistent with Playwright's native timeout semantics, and the test report can correctly display the reason for failure.

[0061] (2) expectEventually(fn, assertion, timeout) – Asynchronous retry assertion: continuously calls fn, passing the return value to assertion (a function containing expect(...) calls), and retryes continuously within the timeout period until the assertion passes or an error occurs. The difference between this function and pollUntil is that this function directly accepts Playwrightexpect-style assertion functions, which is suitable for scenarios where intermediate values ​​do not need to be read (such as "waiting for the element to become visible").

[0062] (3) actAndWait(action, read, predicate) – An atomic combination of operation and wait: First, the action (the asynchronous function that triggers the operation) is executed, and then the read (the function that reads the result) is polled through pollUntil until the condition function predicate is satisfied. The "triggering operation" and "waiting for the side effect to complete" are atomically combined into a single call, eliminating the race condition that may occur between the two steps.

[0063] (4) retry(fn, times) — Operation retry: Retry fn a maximum of times, and re-execute after catching an exception each time. It is specifically designed to deal with occasional UI failures (e.g., pop-up window obstruction causing click failure) and is not used to cover up system errors.

[0064] (5) pollWithLog(fn, predicate) — Polling with log: The function is the same as pollUntil, but each poll additionally prints the current return value to the test standard output stream (stdout) in the format "[poll N]<value>". It is used to troubleshoot failures that are difficult to reproduce in CI (Continuous Integration) environments. In production testing, it is recommended to replace it with pollUntil to reduce log noise.

[0065] In one embodiment of the invention, an optional implementation is that the TestUtils atomic operation toolkit is injected through a hierarchical fixture dependency injection system: in fixtures / utils.ts, the base test object is extended using base.extend<{testUtils:TestUtils}>(), a testUtils Fixture is declared, and a new TestUtils(page) is instantiated in the initialization function of this Fixture, and then injected into the test file through the use() function. The test file can use all atomic operations by declaring the testUtils parameter without manual instantiation.

[0066] The second aspect of this invention discloses an end-to-end automated test code generation method based on intermediate test representation and AI self-correction closed loop. Using the aforementioned end-to-end automated test code generation system, the method specifically includes the following steps: Step 1: Test engineers upload test case documents in CSV or Excel format through the visual pipeline front-end interface.

[0067] Step 2: The system automatically parses the document and maps each line of test cases to the raw test case data structure. Engineers can manually correct parsing errors in the front-end table.

[0068] Step 3: The test intent parser (AI Skill test-intent-parser) reads the knowledge base, converts the natural language description in RawTestCase into a structured test intermediate layer representation TestIR, and labels the semantic parsing confidence of each step (a value from 0.0 to 1.0, representing the degree of certainty that the test code generation AI model is confident in the accuracy of parsing that step).

[0069] Step 4: Engineers review the TestIR (Test Intermediate Layer) on the front-end interface of the visual pipeline. After confirming that the semantic meaning of each test step is correct, they click "Generate Code" to trigger the next stage.

[0070] Step 5: The test specification generator (AI Skill generate-test-spec) reads the list of page object methods recorded in the page object method metadata file meta.json, precisely maps each semantic step in the test intermediate layer representation TestIR to the corresponding code method call, and generates TypeScript test code.

[0071] In a preferred embodiment of the present invention, during the test specification generator phase, the test code generation AI model automatically selects the most suitable TestUtils method and writes it into the generated code based on the type of the action field in each TestAction and the semantics of the assertions list, according to the following rules: If TestAction.action is "wait", or if the assertions describe a polling semantic of "waiting for a certain state to occur" - pollUntil is recommended; Assertions that target a change in the state of a result of an operation, and require the operation to be triggered before waiting for side effects—actAndWait is recommended. Assertions only check the immediate property values ​​(text / visibility, etc.) of the current UI element—expectEventually is recommended; The steps are marked with occasional operation failure scenarios such as "occasional pop-ups" and "occasional obstruction" in the notes - retry is recommended; The intermediate test layer represents the steps marked with debug=true in TestIR—pollWithLog is recommended.

[0072] Step 6: The MCP self-correction service executes the generated code step by step in a real browser and feeds back the execution results (screenshots, logs, actual element values, etc.) to the test code to generate the AI ​​model. The test code generates the AI ​​model to correct the code accordingly, and this process is repeated until all steps are successfully executed.

[0073] Step 7: The final verified test code is output in a unified diff format. After previewing the changes in the visual interface, the engineer can write the code to the code repository with one click.

[0074] In a preferred embodiment of the present invention, the visual pipeline front-end (code directory: apps / pipeline) in the above steps is built based on React (a JavaScript framework for building web user interfaces) + Vite (a modern front-end build tool) + TypeScript + Ant Design (an open-source UI component library from Alibaba), providing a complete visual interface for test code generation workflow, divided into four interactive stages: Phase 1, File Upload: Supports drag-and-drop upload of test case documents in Excel (.xlsx) and CSV formats. The system automatically detects the document's column structure, allowing users to configure mapping relationships such as "which column corresponds to the test case number" and "which column corresponds to the step description" on the interface. After mapping each row of data to a RawTestCase structure, it is displayed in a preview table for user confirmation before proceeding to the next phase.

[0075] Phase Two, Raw Data Editor Phase: All Raw Test Cases are displayed in an editable table format. Users can directly modify content within table cells (inline editing), correcting document parsing errors or adding missing fields. Batch selection is supported: engineers can select only the test case set they wish to process in this instance to generate the AI ​​model from the test code, while the remaining test cases are retained for later processing.

[0076] Phase 3, TestIRViewer Phase: This phase displays the TestIR intermediate layer representation generated by the Test Intent Parser (AI Skill test-intent-parser) as a structured list, with each test step occupying one line. Steps with a confidence level below 0.70 are highlighted in yellow; steps containing TBD placeholders are highlighted in red, and a TBD description is displayed at the end of the line to guide engineers in filling in missing information. Engineers can directly modify fields such as action type and target name on the interface. After confirming the semantics are correct, clicking the "Generate Code" button triggers the Test Specification Generator (AI Skill generate-test-spec).

[0077] Phase Four: Code Preview and Export Phase: The generated code changes are displayed in a unified diff format. Red lines represent content to be deleted from the original file, and green lines represent new content. File-by-file preview is supported. The "Apply Changes" button writes all diff files directly to the corresponding path in the local code repository; the "Copy to Clipboard" option allows manual pasting for application.

[0078] In another preferred embodiment of the present invention, the end-to-end automated test code generation method further includes a step of automatic test failure diagnosis: after a test execution failure in the continuous integration (CI) environment, the test code generation AI model reads the failure report (including screenshot.png and execution trace file trace.zip) generated by the Playwright testing framework, the browser console error log, and the DOM snapshot of the current page, and automatically classifies the failure cause into one of the following four categories and generates corresponding repair suggestions: (1) Assertion value mismatch - Provides a comparison between the actual value and the expected value and suggests corrections for the expected field value; (2) ElementTimeout – Provides suggestions for troubleshooting possible spelling errors in testId; (3) Element not visible - provide a code patch for waiting for the element to become visible; (4) Network error - Provide suggestions for prior status checks.

[0079] In another preferred embodiment of the present invention, the end-to-end automated test code generation method further includes the steps of Flaky Test prediction and annotation: the system collects test execution time sequence data (including historical pass / fail records of each test case, execution time statistical distribution, and environmental snapshots at the time of failure) from multiple continuous integration (CI) runs, trains a lightweight binary classification prediction model, and predicts which test cases have unstable risks in this run based on current environmental indicators (machine load, network latency, etc.) before the start of a new round of continuous integration (CI) execution, automatically adds @flaky tags to these test cases, and allows the continuous integration (CI) strategy to adopt independent execution (isolated execution to avoid mutual influence) or automatic retry (up to N retry times) strategies for the marked test cases.

[0080] In another preferred embodiment of this invention, the end-to-end automated test code generation method further includes a step of test case semantic deduplication and coverage analysis: The imported natural language test cases are semantically vectorized (using an embedding model to convert text into high-dimensional numerical vectors), and the cosine similarity (a value closer to 1 indicates greater semantic similarity) between test cases is calculated. Test case pairs with similarity exceeding a configurable threshold are marked as "suspected duplicates," prompting engineers to merge or differentiate them. The test coverage distribution of each page is displayed in the form of a coverage heatmap, identifying test blind spots, and supporting statistical analysis of test coverage metrics by business function dimension (page / interaction type / assertion type).

Claims

1. An end-to-end automated test code generation system based on intermediate test representation and AI self-correction closed loop, characterized in that, It comprises four subsystems: a three-layer knowledge base system, an AI pipeline, an AI self-correction service, and a test execution infrastructure. These four subsystems work collaboratively, exchanging information with each other through clearly defined data formats to form a complete test code generation pipeline. The three-layer knowledge base system is used to store and maintain the structured knowledge of the project under test, which can be queried by the test code generation AI model at each stage of test code generation. Specifically, it includes the page index file index.json, the business interaction definition file page_behavior.json, and the page object method metadata file meta.json. The AI ​​pipeline is used to perform two-stage AI processing on CSV / Excel format test case documents, and finally output executable test code in TypeScript format. The two AI processing stages are: a test intent parser, which is used to convert the original test cases into a test intermediate layer representation TestIR that is independent of the UI implementation; and a test specification generator, which is used to convert the test intermediate layer representation TestIR into executable test code in TypeScript format. When generating executable test code, the test specification generator only calls the methods recorded in the page object method metadata file meta.json. The AI ​​self-correction service uses the model context protocol to expose the control capabilities of the real browser to the test code generation AI model, enabling the test code generation AI model to execute the generated test code in the real browser, observe the actual execution results, and autonomously correct errors in the code until the code passes verification. The test execution infrastructure consists of the Playwright test framework, the layered fixture dependency injection system, and the TestUtils atomic operation toolset, and is the underlying environment upon which the final generated test code depends for execution.

2. The end-to-end automated test code generation system based on intermediate test representation and AI self-correction closed loop as described in claim 1, characterized in that, The page index file index.json is used to maintain an index table of all implemented page object classes in the project under test. Each record in the index table includes the page ID, Chinese semantic name, corresponding TypeScript file path, and the file paths of the associated business interaction definition file page_behavior.json and the page object method metadata file meta.json. The business interaction definition file page_behavior.json is used to describe what the current page "can do" at the business level. Each business interaction record in the business interaction definition file page_behavior.json includes: a unique interaction identifier, a triggering method, a parameter list, and an operation result; The meta.json file, representing the method metadata of the page object, is automatically generated by the TypeScript abstract syntax tree parser. It records the complete signature information of all implemented code methods in each page object class, including: method name, parameter list, return type, and JSDoc documentation comment summary.

3. The end-to-end automated test code generation system based on intermediate test representation and AI self-correction closed loop as described in claim 2, characterized in that, All page object classes have the @PageObject decorator added above their class definitions, and the cnNames parameter declares a list of Chinese business semantic names corresponding to the current class. The Chinese names in the cnNames parameter correspond one-to-one with the semantic names in the page index file index.json. The TypeScript abstract syntax tree parsing tool is used to analyze the abstract syntax tree to extract structural information from the code without executing code or manually writing parsing rules. It extracts metadata using the following steps to generate the page object method metadata file meta.json: Step 101: Call the TypeScript compiler application programming interface to create a compiler and perform a full TypeScript abstract syntax tree traversal on all .ts files in the pages / directory; Step 102: Scan the class declaration node in the TypeScript abstract syntax tree of each file and check if its decorator list contains a calling expression named "PageObject". Step 103: For the identified @PageObject class, parse the cnNames parameter value of the decorator, establish a mapping relationship of "Chinese semantic name -> class name -> file path", and write it into the page index file index.json; Step 104: Traverse all method declaration nodes of the current class and extract the method name, parameter list, and return type; Step 105: Read the JSDoc block above each method declaration node and extract the text content of the @description, @param, and @returns tags; Step 106: Serialize the complete method signature information extracted in steps 104 and 105 into JSON format and write it into the meta.json file of the corresponding page object method metadata file.

4. The end-to-end automated test code generation system based on intermediate test representation and AI self-correction closed loop as described in claim 1, characterized in that, The test intermediate layer represents TestIR, which includes: TestAction describes a single operation step in a test case. It includes the action field, which represents the action type; the page field, which represents the semantic page name; the target field, which represents the semantic element name; the value field, which represents the operation value; the assertions field, which represents the assertion list attached to the current step; the confidence field, which represents the confidence level of the semantic parsing of the current step; and the tbd field, which represents the placeholder list to be confirmed. An assertion is used to describe the expected state verification of a page element. It includes a type field to represent the assertion type, a target field to represent the semantic name of the element being asserted, and an expected field to represent the expected value. The placeholder TBDPlaceholder is inserted when there is missing information in the test step description. It includes a type field to indicate the type of missing information, a key field to indicate the identifier key of the missing information, and a description field to indicate the text description of the missing information. The test middle layer represents the TestIR body, which includes an id field that is a unique identifier for test cases, a title field that represents the name of the test case, a precondition field that represents the preconditions, an actions field that represents the list of TestActions, and an overallConfidence field that represents the overall confidence level.

5. The end-to-end automated test code generation system based on intermediate test representation and AI self-correction closed loop as described in claim 4, characterized in that, The confidence field for each test action is calculated by weighting the following four sub-dimensions: Sub-dimension 1, semantic matching degree, with a weight of 40%, is used to represent the word overlap rate between the original text of the calculation steps and the business interaction description in the business interaction definition file page_behavior.json. Word overlap rate = number of overlapping words / number of words in the union of the two text segments; Sub-dimension 2, parameter completeness, with a weight of 25%, is used to calculate the proportion of required parameters that have been successfully extracted from the original text of the steps. The percentage of successful extraction of all required parameters is 1.

0. For each missing parameter, the percentage is reduced proportionally. Placeholders for unextracted parameters are inserted at the corresponding positions. Sub-dimension 3, Page Location Reliability, with a weight of 20%, is calculated based on the number of candidate pages matched in the page index.json in the original steps: a unique match is scored as 1.0, matching N candidate pages is scored as 1 / N, and no match is scored as 0.

0. Sub-dimension 4, Action Type Clarity, with a weight of 15%, is used to determine whether the step verbs precisely correspond to the trigger events defined in the current business interaction definition file page_behavior.json: a precise correspondence is scored as 1.0, while using ambiguous verbs is scored as 0.

4.

6. The end-to-end automated test code generation system based on intermediate test representation and AI self-correction closed loop as described in claim 4, characterized in that, When there are semantic gaps in the test step description, the test intent parser does not infer the missing value or skip the current step, but instead inserts a structured placeholder for confirmation at the corresponding position and records the reason for the gap. When the downstream test specification generator encounters a placeholder that needs to be confirmed, it generates test.fixme() instead of attempting to complete it, and at the same time, it explains the missing information in the comments.

7. The end-to-end automated test code generation system based on intermediate test representation and AI self-correction closed loop as described in claim 1, characterized in that, In the AI ​​self-correction service, the test specification generator and the test intent parser implement the self-correction closed loop through the following steps: Step 201: Generate test code using the AI ​​model. Generate an initial version of TypeScript test code based on the TestIR (test intermediate layer representation) and the meta.json (page object method metadata file). Step 202: Test code generation AI model. The AI ​​model is simulated to execute the operation sequence described in the test code in a real browser using the Model Context Protocol tool; Step 203: After each operation step is executed, the test code generates an AI model and collects the execution results through the Model Context Protocol tool; Step 204: Test code generation AI model compares the actual state observed in step 203 with the expected state defined in TestIR, representing the test intermediate layer, to determine if there is a difference. If there is a difference, the root cause is analyzed and a code correction patch is generated. Step 205: Re-execute steps 202-204 of the corrected code, looping for a maximum of N rounds. After all steps have passed verification, output the final code in a unified difference format. If any step still fails after reaching the maximum number of loops, add a summary of errors that still require manual processing to the difference file.

8. The end-to-end automated test code generation system based on intermediate test representation and AI self-correction closed loop as described in claim 1, characterized in that, In the layered fixture dependency injection system, a bottom-up pyramid-shaped dependency chain is adopted, including: the bottom layer config.ts, used to load test configuration; the middle layer login.ts, which depends on the bottom layer config.ts, performs login operations, and provides the logged-in browser page context to the upper layer; the menu layer menu.ts, which depends on the middle layer login.ts, provides navigation capabilities between pages and a common pop-up object; and the functional layer fixtures, which depends on the menu layer menu.ts, with one file for each business module, providing the corresponding page object instance. Furthermore, in the hierarchical fixture dependency injection system, the following rules are enforced: All test files must import the test object from the fixtures file of the functional layer that is closest to the business logic; direct import from @playwright / test is prohibited. Directly writing element location expressions is prohibited in test files and function layer fixtures files. All element access must be done through the encapsulated methods of the page object to ensure centralized management of location logic. Navigation between pages must be handled uniformly through MenuPage; hardcoding URLs in functional pages is prohibited.

9. The end-to-end automated test code generation system based on intermediate test representation and AI self-correction closed loop as described in claim 1, characterized in that, The TestUtils atomic operation toolkit contains five atomic operation functions, which are injected into all tests through a layered fixture dependency injection system to ensure the consistency and correctness of asynchronous processing. Specifically, these include: The pollUntil function polls until the condition is met: internally, it calls the expect.poll() API of the Playwright testing framework, repeatedly calling the asynchronous function fn at specified intervals, and passing the return value of the asynchronous function fn to the condition judgment function predicate, until the condition judgment function predicate returns true or a timeout occurs; The asynchronous retry assertion function expectEventually continuously calls the asynchronous function fn, passing the return value of the asynchronous function fn to the function containing the expect(...) call, and retryes continuously within the timeout period until the assertion passes or an error occurs. The atomic combination function `actAndWait` combines operations and waits: first, the asynchronous function that triggers the operation is executed; then, the function `pollUntil` polls until the condition is met, and the function `read` polls until the condition is met; The retry function `retry` will retry the asynchronous function `fn` a maximum of `times` times, catching any exceptions and re-executing it each time. The pollWithLog function has the same functionality as the pollUntil function, which polls until a condition is met, but it additionally prints the current return value to the test standard output stream with each poll.

10. An end-to-end automated test code generation method based on intermediate test representation and AI self-correction closed loop, employing the end-to-end automated test code generation system as described in claim 1, characterized in that, Includes the following steps: Step 1: Test engineers upload test case documents in CSV or Excel format through the visual pipeline front-end interface; Step 2: The system automatically parses the document and maps each line of test cases to the original test case data structure. Engineers can manually correct parsing errors in the front-end table. Step 3: The test intent parser reads the knowledge base, converts the natural language descriptions in the original test cases into a structured test intermediate layer representation TestIR, and labels the semantic parsing confidence of each step; Step 4: Engineers review the TestIR (Test Intermediate Layer) on the front-end interface of the visual pipeline. After confirming that the semantic meaning of each test step is correct, the next stage is triggered. Step 5: The test specification generator reads the list of page object methods recorded in the page object method metadata file meta.json, maps each semantic step in the test intermediate layer representation TestIR to the corresponding code method call, and generates TypeScript test code; Step 6: The MCP self-correction service executes the generated code step by step in the real browser and feeds back the execution results to the test code to generate the AI ​​model. The test code generates the AI ​​model to correct the code accordingly. This process is repeated until all steps are executed successfully. Step 7: Finally, the verified test code is output in a unified difference format. After the engineer previews the changes in the visual interface, he can write the code into the code repository with one click.

Images