Front-end automatic testing method of large language model based on LLM (Logistics Language Model)

Abstract

The invention provides a front-end automatic testing method of a large language model based on LLM, and belongs to the technical field of front-end automatic testing. The method comprises five core steps of test intention recognition and semantic analysis, page structure analysis and target element positioning, automatic generation of a test operation sequence and assertion logic, test script execution and dynamic regression verification, and test feedback analysis and continuous training. The strong semantic understanding and context reasoning capabilities of the LLM are utilized to perform semantic analysis and simulation on a front-end page structure, a user behavior process and business logic, so that automatic generation and maintenance of a test script are realized. According to the method, the test coverage rate and the generation efficiency are greatly improved, the maintenance cost is remarkably reduced, the readability and the intelligence of the test case are enhanced, and the method is suitable for the modern Web front-end application test of rapid iteration.

Description

Technical Field

[0001] The present invention relates to the technical field of front - end automated testing, and particularly to a front - end automated testing method based on a large language model (LLM). Background Art

[0002] Currently, front - end automated testing has gradually become a key link in ensuring product quality in the actual development process. However, the traditional way of writing test scripts has problems such as high threshold, low coverage rate, and high maintenance cost. Especially when facing frequently changing front - end page structures and interaction logics, manually maintaining test scripts is inefficient. Although in recent years, test generation tools based on recording and playback or low - code have emerged, these methods still have difficulty accurately understanding complex business logics and changing user behaviors.

[0003] With the wide application of large language models (LLMs), their powerful capabilities in natural language understanding, intent recognition, and logical reasoning have provided new solutions for front - end automated testing. Therefore, there is an urgent need for a testing method that integrates the capabilities of large language models, which can drive the testing process with natural language, realize the intelligent generation, dynamic execution, and continuous optimization of test scripts, and improve the intelligent level and coverage of front - end testing. Summary of the Invention

[0004] To solve the above - mentioned technical problems, the present invention provides a front - end automated testing method based on a large language model (LLM), which makes full use of the semantic parsing ability of the model to automatically generate high - quality and executable test scripts. This method combines various technologies such as DOM structure analysis, user behavior abstraction, and test intent recognition to realize the full - process intelligent transformation from "natural language test requirements" to "runnable test code".

[0005] The technical solution of the present invention is as follows: A front - end automated testing method based on a large language model (LLM), which makes full use of the semantic parsing ability of the model to automatically generate high - quality and executable test scripts. This method combines various technologies such as DOM structure analysis, user behavior abstraction, and test intent recognition to realize the full - process intelligent transformation from "natural language test requirements" to "runnable test code". It mainly includes: Test intent recognition and semantic parsing: Based on the LLM, semantic parsing is performed on the natural - language test description input by the user to identify elements such as the user interaction path, expected behavior, and page state change, extract core information such as user intent, operation object, and assertion logic, and convert the semantic information into a structured test intent intermediate representation; Page Structure Analysis and Target Element Location: Automatically obtain the current page DOM structure and component tree, establish an element index by combining the rendering state and attributes (such as ARIA labels, IDs, classes, etc.), and use the LLM to further judge the matching degree between the target element and the context to improve the positioning accuracy.

[0006] Automatically Generate Test Operation Sequences and Assertion Logic: Convert the structured intention into code for the target test framework (such as Cypress, Playwright), automatically fill in user behaviors (click, input, select, etc.) and assertion conditions, support conditional branches and error fallback logic in the test process, and form a standardized test process tree.

[0007] Test Script Execution and Dynamic Regression Verification: Execute the generated script in a multi-browser environment, record the execution logs and results, automatically identify the reasons for failures, provide script repair suggestions, compare UI changes in combination with the snapshot mechanism, identify unexpected rendering problems, monitor the results and analyze abnormal situations.

[0008] Test Feedback Analysis and Continuous Training: Feed the execution data and user feedback back to the LLM fine-tuning mechanism for learning and optimization, improve the accuracy of the next script generation and the test scenario coverage rate, and continuously enhance the script generation accuracy and scenario adaptability.

[0009] The beneficial effects of the present invention are: By introducing the natural language understanding and semantic reasoning capabilities of large language models, the present invention innovatively realizes the transformation of front-end automated testing from script-driven to semantic-driven. This method supports users to describe test intentions in natural language, and the system automatically parses semantics, matches page elements and generates a complete test process, effectively reducing the test threshold. By constructing a process tree-style test structure, it can flexibly cover complex interaction scenarios with multiple branches and paths; at the same time, combined with the automated execution and exception analysis modules, it can achieve efficient test execution and real-time regression verification. Further, by continuously optimizing the model Prompt and strategy parameters with test feedback data, the quality of test cases is continuously enhanced. The overall solution significantly improves the test generation efficiency, execution stability and system adaptability, and is suitable for the high-frequency automated testing requirements of modern Web applications in the context of agile development and continuous integration. Brief Description of the Drawings

[0010] Figure 1 is a schematic diagram of the work flow of the present invention; Figure 2 is a diagram of the specific implementation steps of the present invention. Detailed Embodiments

[0011] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0012] The present invention provides a front-end automated testing method based on a large language model (LLM), as Figure 1 shown, which includes the following steps: (I) Test intention recognition and semantic parsing The specific implementation is as Figure 2 shown. In this implementation, semantic parsing is performed on the natural language test description input by the user based on the LLM to extract core information such as user intention, operation object, and assertion logic. This process includes the following steps: First, the user submits test requirements through natural language. The system first preprocesses the input statement, including word segmentation, part-of-speech tagging, named entity recognition, etc.

[0013] The further preprocessing results will be input into the large language model as a prompt embedding (Prompt). The model performs semantic understanding of the test target based on the pre-trained knowledge.

[0014] Further, key information is extracted from the model, including: test module name, expected user operation sequence, target page status, assertion conditions, etc.

[0015] Further, the system structurally encapsulates the extracted information to generate a standard "test intention object", providing a data basis for the subsequent steps.

[0016] Further, if the user input is ambiguous or incomplete, the system interacts with the user through multiple rounds of questions and answers to complete the test scenario.

[0017] (II) Page structure parsing and target element positioning The specific implementation is as Figure 2 shown. In this implementation, the target page is automatically loaded, DOM structure information is extracted, and semantic matching is performed on the element description in combination with the language model to achieve accurate positioning. The implementation steps of this process are as follows: First, the system loads the target page through a browser automation tool and extracts the DOM structure tree after the page is fully rendered.

[0018] Further, page element information (including tag name, ID, Class, ARIA label, text content, data attribute, etc.) is extracted and organized into a structured DOM object.

[0019] A further system semantically compares and matches the description of the target elements in the "test intent object" with the page DOM information.

[0020] In the further matching process, a large language model is introduced to understand and judge the element semantics, solving the problem that traditional methods cannot accurately identify "implicitly named" components (for example, the description of an "email input box" may not contain standard attributes).

[0021] After further element positioning, the system establishes a mapping table between test actions and specific DOM nodes, which serves as the core basis for subsequent test action generation.

[0022] (3) Automatically generating test operation sequences and assertion logic The specific implementation is as Figure 2 shown. In this implementation manner, according to the recognized intent and page elements, test operation sequences and assertion logic are automatically constructed to form a standardized test process tree. The implementation process is as follows: First, the system analyzes the user operation path based on the test intent and the located page elements to construct a clear interaction process model.

[0023] Furthermore, this process model includes each user behavior step, such as input, click, hover, selection, etc., and clarifies the target of each step.

[0024] Further, the system combines rule templates with a language model to convert each interaction action into a structured test operation description.

[0025] At the same time, the assertion logic is extracted as "state judgment conditions" and associated and bound with the current page state to verify the success or failure of the test.

[0026] Furthermore, these operations and assertions are organized into linear or branched operation flows and form a complete test process tree structure for subsequent unified execution.

[0027] (4) Executing test scripts and dynamically verifying regression The specific implementation is as Figure 2 shown. In this implementation manner, based on an automated test engine, test scripts are executed in a browser, and the results are monitored and abnormal situations are analyzed. The implementation method is as follows: First, the system passes the test process tree into the automated test engine and runs test tasks in parallel in a multi-browser and multi-device environment.

[0028] During the further execution process, the system monitors behaviors such as page state, console output, exception throwing, and screenshot changes to analyze whether the page responds as expected.

[0029] Furthermore, if an error occurs during execution, the system will package the error information (such as element not found, assertion failure, etc.) with the execution context and submit it to the large language model for analysis.

[0030] Furthermore, the model will return possible explanations for the error causes and improvement suggestions, such as "the page structure change causes the element selector to fail" or "the assertion condition is missing in the logical branch", etc.

[0031] Furthermore, the user can choose whether to adopt the model suggestions to update the test intent or operation flow, thus forming an automated "regression repair closed loop".

[0032] (5) Test feedback analysis and continuous optimization training The specific implementation is as Figure 2 shown. In this implementation manner, the test data is fed back to the large language model for learning and optimization to continuously improve the script generation accuracy and scenario adaptability. The specific implementation manner is as follows: First, the system records all test execution results into the test database, including multi-dimensional metrics such as success rate, failure reason, and time-consuming statistics.

[0033] Furthermore, the system will automatically aggregate similar failure cases and build a "failure mode" label system to guide the subsequent response methods of the generation model in similar scenarios.

[0034] Furthermore, based on user feedback or test analysis, the system can generate Prompt adjustment suggestions or sample data for the post-training and adaptability optimization of the model.

[0035] Furthermore, when the user has repaired a certain type of business logic multiple times, the system will identify its general pattern and convert it into a pre-training task to improve the domain ability of the model.

[0036] Furthermore, the entire optimization process forms an automatic enhancement loop of "semantic understanding - test generation - execution feedback - model learning", promoting the continuous evolution of test quality and intelligent level.

[0037] In summary, by introducing the natural language understanding and semantic reasoning capabilities of large language models, the present invention innovatively realizes the transformation of front-end automated testing from script-driven to semantic-driven. This method supports users to describe test intentions in natural language, and the system automatically parses semantics, matches page elements and generates a complete test process, effectively reducing the test threshold. By constructing a process tree-style test structure, it can flexibly cover complex interaction scenarios with multiple branches and paths; at the same time, combined with the automated execution and exception analysis modules, efficient test execution and real-time regression verification can be achieved. Further, by continuously optimizing the model Prompt and policy parameters with the help of test feedback data, the quality of test cases can be continuously enhanced. The overall solution significantly improves the test generation efficiency, execution stability and system adaptability, and is suitable for the high-frequency automated testing requirements of modern Web applications under the background of agile development and continuous integration, with broad promotion value and engineering practice significance.

[0038] The above are only the preferred embodiments of the present invention, which are only used to illustrate the technical solutions of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A front - end automated testing method for large language models based on LLM, characterized in that using the semantic understanding and context reasoning capabilities of LLM, semantic parsing and simulation are performed on the front - end page structure, user behavior flow, and business logic to achieve the automatic generation and maintenance of test scripts; including: test intention recognition and semantic parsing, page structure parsing and target element positioning, automatic generation of test operation sequences and assertion logic, test script execution and dynamic regression verification, test feedback analysis and continuous training.

2. The method according to claim 1, characterized in that Test intention recognition and semantic parsing: Based on LLM, semantic parsing is performed on the natural - language test description input by the user, the user interaction path, expected behavior, and page state changes are recognized, user intention, operation object, and assertion logic are extracted, and the semantic information is converted into a structured test intention intermediate representation.

3. The method according to claim 2, characterized in that Specifically including: The user submits test requirements through natural language, and pre - processes the input statement; The pre - processing result is embedded as a prompt into the large language model, and the model performs semantic understanding of the test target based on pre - trained knowledge; Key information is extracted from the model, including: test module name, expected user operation sequence, target page state, assertion conditions; The extracted information is structurally encapsulated to generate a standard test intention object, providing a data basis for subsequent steps; If the user input is ambiguous or incomplete, interaction with the user is carried out through several rounds of question - and - answer to complete the test scenario.

4. The method according to claim 3, characterized in that Page structure parsing and target element positioning: Automatically obtain the current page DOM structure and component tree, establish an element index in combination with the rendering state and attributes, and use LLM to further judge the matching degree of the target element with the context; Specifically including: Load the target page through a browser automation tool, and extract the DOM structure tree after the page is fully rendered; Page element information is extracted and organized into a structured DOM object; The target element description in the test intention object is semantically compared and matched with the page DOM information; The large language model is introduced during the matching process to understand and judge the element semantics; After the element is located, a mapping table is established between the test action and the specific DOM node, which serves as the core basis for subsequent test action generation.

5. The method according to claim 4, characterized in that Automatically generate test operation sequences and assertion logic: Convert the structured intention into the code of the target test framework, automatically fill in user behavior and assertion conditions, support test process conditional branches and error fallback logic, and form a standardized test process tree.

6. The method according to claim 5, characterized in that Specifically including: According to the test intention and the located page elements, analyze the user operation path and construct a clear interaction process model; This process model includes each user behavior step, and clarifies the target of each step; By combining rule templates and language models, each interaction action is converted into a structured test operation description; Meanwhile, the assertion logic is extracted as a state judgment condition, associated and bound with the current page state to verify the success or failure of the test; These operations and assertions are organized into a linear or branched operation flow and form a complete test process tree structure for subsequent unified execution.

7. The method according to claim 6, wherein Test script execution and dynamic regression verification: Execute the generated script in a multi-browser environment, record the execution logs and results, automatically identify the reasons for failure, provide script repair suggestions, compare UI changes in combination with the snapshot mechanism, identify unexpected rendering problems, monitor the results and analyze abnormal situations.

8. The method according to claim 7, wherein Specifically, it includes: First, pass the test process tree into the automated test engine and run the test tasks in parallel in a multi-browser and multi-device environment; During the execution process, monitor the page state, console output, exception throwing, and screenshot changes to analyze whether the page responds as expected; If an error occurs during execution, package the error information and execution context and submit them to the large language model for request analysis; The model returns possible explanations for the reasons for the error and improvement suggestions; The user can choose whether to adopt the model suggestions to update the test intent or operation flow, thus forming an automated regression repair closed loop.

9. The method according to claim 8, wherein Test feedback analysis and continuous training: Feed the execution data and user feedback back to the LLM fine-tuning mechanism for learning and optimization to improve the accuracy of the next script generation and the test scenario coverage rate, and continuously improve the script generation accuracy and scenario adaptability.

10. The method according to claim 9, wherein Specifically, it includes: First, record all test execution results in the test database, including success rate, reasons for failure, and time-consuming statistics; Automatically aggregate similar failure cases and build a fault mode label system to guide the subsequent response methods of the generation model in similar scenarios; Based on user feedback or test analysis, generate Prompt adjustment suggestions or sample data for the post-training and adaptability optimization of the model; When the user has repaired a certain type of business logic several times, identify its general pattern and convert it into a pre-training task; The entire optimization process forms an automatic enhancement cycle of semantic understanding - test generation - execution feedback - model learning.

Images