WebGUI automatic test method and system based on multi-modal large model, storage medium and equipment
By combining multimodal large models with screenshots and text information for automated testing of WebGUIs, the efficiency and accuracy issues of existing methods in complex interfaces and cross-interface interactions are resolved, achieving efficient and accurate automated testing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-23
- Publication Date
- 2026-05-01
AI Technical Summary
Existing WebGUI automated testing methods are inefficient when dealing with complex interfaces, cross-interface interactions, and multi-control operations. They also have poor generalization ability and are susceptible to the LLM illusion problem, leading to inaccurate test cases.
Interactive elements are labeled using a multimodal large model combined with screenshots and text information. Visual and textual verification is performed using the GPT-4-ACT tool to generate structured test scripts. Assertion verification is performed using a pre-built assertion pool, forming a self-optimizing closed-loop feedback mechanism.
It improved testing efficiency and accuracy, reduced computing resource consumption, enhanced adaptability to dynamic interfaces, reduced logical errors and security vulnerabilities, and achieved self-correction and continuous optimization.
Smart Images
Figure CN121958128A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of software testing, and in particular to an automated testing method, system, storage medium, and device for WebGUI based on a multimodal large model. Background Technology
[0002] Large Language Models (LLMs) represent a significant breakthrough in artificial intelligence in recent years. With their massive parameter scale and complex neural network structures, they are capable of processing vast amounts of text data and performing a variety of complex tasks. As LLM technology matures, its application in automated testing is becoming increasingly widespread, providing a novel solution for automated testing frameworks for Web Graphical User Interfaces (GUIs). In traditional software development and testing processes, Web GUI testing typically relies on manual operation or rule-based scripting, which is not only inefficient but also susceptible to human error. While existing automated testing methods have improved testing efficiency to some extent, they still have many limitations. For example, many tools can only handle a single input pattern (such as HTML), which can be very verbose and dynamically changing; other methods rely on domain-specific training data and lack generalization ability; furthermore, existing methods perform poorly when faced with tasks involving multiple controls or cross-interface interactions and are susceptible to the LLM "illusion" problem, leading to inaccurate test cases.
[0003] Specifically, the existing testing methods and their shortcomings are as follows: 1. Deep Learning-Based Automated Testing of WebGUIs Early WebGUI automated testing tools such as IDEs, UI Recorders, and CukeTest generated scripts by recording tester actions. With the development of machine learning and deep learning, researchers have used algorithms to generate automated test scripts from large datasets that learn software usage patterns, user behavior characteristics, and defect patterns. These methods significantly improve testing efficiency and coverage by simulating human interaction, automatically generating and executing test cases, and reducing manual testing workload. However, model training requires a large amount of labeled data, and the cost of data collection and processing is high, limiting its application in data-scarce scenarios. Simultaneously, training and inference also require significant computational resources, greatly increasing costs. When faced with complex interfaces or non-standard fonts, GNNs may fail to correctly identify interface elements, leading to test failures, i.e., poor generalization ability; furthermore, special fonts require additional training, further increasing costs.
[0004] 2. Automated testing of WebGUI based on Large Language Model (LLM) Large Language Models (LLMs) such as ChatGPT, GitHub-Copilot, and GPT-4 have shown potential in automated testing of Web GUIs. For example, leveraging AI models to generate end-to-end test scripts based on natural language descriptions, particularly in Gherkin-defined test cases, can reduce development time; or, through a pipeline architecture of object detection, text detection, and natural language processing models, test cases can be automatically generated from the UI structure and updated automatically when the UI changes. These methods utilize the semantic understanding and code generation capabilities of LLMs, resulting in highly efficient test script generation, reducing manual development time. When integrated with automation tools, test accuracy and coverage are significantly improved, and they can adapt to dynamic UI changes. However, these methods also have the following limitations: 1) Training data needs to be customized for different testing domains, and the pass rate drops significantly in cross-domain testing; 2) When the test task involves multiple controls or requires cross-interface operation, these methods will encounter serious errors, resulting in a decrease in the pass rate of test cases; 3) The LLM illusion problem causes a large number of generated operations to point to non-existent interface elements, thus causing the generated test cases to become invalid. Summary of the Invention
[0005] The purpose of this invention is to overcome the problems existing in the prior art and to provide a WebGUI automated testing method, system, storage medium and device based on a multimodal large model.
[0006] The objective of this invention is achieved through the following technical solution: Firstly, a WebGUI automated testing method based on a multimodal large model is provided, including the following steps: S1. Construct the multimodal input of the webpage to be tested; S2. Construct operation prompts that include a test task description and the multimodal input; S3. The large model generates a test script based on the operation prompts, performs a syntax check on the generated test script, and if the syntax check is successful, proceeds to S4; otherwise, iterative correction is performed until the syntax check is successful. S4. Execute the test script if the syntax check is successful and generate a test report.
[0007] In some embodiments, constructing the multimodal input for the webpage to be tested includes: Interactive elements are labeled using object detection on screenshots of the webpage under test, and bounding boxes with numerical labels are overlaid on each element. At the same time, JavaScript tools are used to extract the text information of the labeled elements and integrate it into a JSON array.
[0008] In some embodiments, the text information includes XPath, element type, class name, ID, and text content.
[0009] In some embodiments, the construction includes a test task description and operation prompts for the multimodal input, including: Define the system role as Selenium Test Expert, and then integrate the test task description and the multimodal input; The test task is a reasoning chain of target understanding, element location, operation generation, and assertion verification, generating operations within a limited scope. The above operations are formatted using Python code structure templates and special operation prompts are embedded to form structured multimodal test instructions.
[0010] In some embodiments, element positioning includes: Visual and textual verification was performed using the GPT-4-ACT tool in conjunction with XPath and bounding boxes with numerical labels. The operation generation includes: The large model selects operations through a question-and-answer mechanism; The assertion verification includes: Generate validation assertions by selecting templates from a pre-built assertion pool and replacing them with real text content.
[0011] In some embodiments, the syntax detection includes: AST analysis is performed using a Python syntax detector to identify and record structural and identifier errors.
[0012] In some embodiments, the iterative correction includes: Based on the structural and identifier errors, the modified operation prompts are submitted to the large model to regenerate the test script.
[0013] Secondly, a WebGUI automated testing system based on a multimodal large model is provided, including: The environment observation module is used to construct multimodal inputs for the webpage under test; The prompt construction module is used to construct operation prompts that include a description of the test task and the multimodal input; The test script generation module is used to generate test scripts based on the operation prompts using the large model. The generated test scripts are then subjected to syntax checks. If the syntax check is successful, the script execution module is entered; otherwise, iterative correction is performed until the syntax check is successful. The script execution module is used to execute test scripts that successfully perform syntax checks and generate test reports.
[0014] Thirdly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the WebGUI automated testing method based on a multimodal large model as described in the first aspect.
[0015] Fourthly, an electronic device is provided, including a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, and the processor executes the WebGUI automated testing method based on a multimodal large model as described in the first aspect when executing the computer instructions.
[0016] It should be further noted that the technical features corresponding to the above-mentioned options and embodiments can be combined or substituted with each other to form new technical solutions without conflict.
[0017] Compared with the prior art, the beneficial effects of the present invention are: 1. Efficient multimodal information processing: This invention uses screenshots as the primary method and text information as a secondary method, bypassing cumbersome HTML parsing. It does not rely on static HTML structures but combines real-time screenshots with key text information. It can effectively handle scenarios that traditional script recording tools struggle to handle, such as dynamic interface changes, asynchronous loading, and non-standard controls. It significantly reduces the communication token cost with large language models (LLMs) and improves testing efficiency.
[0018] 2. Precise element positioning: By using the GPT-4-ACT tool in combination with XPath and bounding boxes with number labels for two-way visual and textual verification, the accurate positioning of interactive elements (such as buttons and input boxes) is ensured.
[0019] 3. High accuracy of operation generation: LLM generates test scripts based on a limited set of operations (click, input, scroll, drag, return), and the self-questioning and answering mechanism effectively avoids generating illusions and improves the reliability of operations. At the same time, it constrains the output space of the model, making the generated operation intentions clearer and safer.
[0020] 4. Text information optimization: The text is cleaned up, and key text information such as element type, XPath, class name, and ID are extracted and integrated into a JSON array. This eliminates the lengthy original HTML, shortens the text length by about 50%, and reduces the complexity and processing cost of structured text.
[0021] 5. Flexible assertion generation: LLM uses a pre-built assertion pool to select and replace real text content from templates to generate accurate assertions, verify the consistency of test results, and reduce logical errors and security vulnerabilities.
[0022] 6. Efficient resource utilization: Screenshots are uploaded to the server, and LLM accesses them via URL, reducing local computing resource consumption; when errors occur, error information is integrated and operations are regenerated, improving the framework's robustness.
[0023] 7. Adapt to dynamic testing scenarios: Supports various interactive operations (such as clicking, inputting, and scrolling), enabling users to browse complex web pages and complete diverse testing tasks, thereby enhancing test coverage.
[0024] 8. Forming a self-optimizing closed-loop feedback mechanism: When a syntax error is detected, the system can automatically record the error information and reverse the operation prompts, driving the large model to regenerate. This iterative mechanism of "generation-detection-regeneration" gives the system the ability to self-correct and continuously optimize, gradually approaching a correct and executable script without manual intervention, thus improving the framework's autonomy and practicality. Attached Figure Description
[0025] Figure 1 This is a flowchart of an automated testing method for WebGUI based on a multimodal large model according to the present invention; Figure 2 This invention provides an automated GUI testing process based on a large-scale visual language model. Figure 3 This is a schematic diagram of the VL-GATF framework of the present invention. Detailed Implementation
[0026] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0027] It should be noted that the defects in the solutions in the prior art are all the results of the inventors' practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the embodiments of this application in the following text should be the inventors' contributions to this application in the process of invention and creation, and should not be understood as technical content known to those skilled in the art.
[0028] Before describing the embodiments of this application, the technical terms involved are explained: 1. AST is a low-level analysis tool used to ensure the quality of generated code.
[0029] 2. QA is a guided method used to improve the reasoning logic and accuracy of large models.
[0030] 3. XPath is a standard language used to precisely locate web page elements at the code level and combine them with visual information to achieve reliable verification.
[0031] In view of the technical problems pointed out in the background art, the present invention provides the following embodiments: In one exemplary embodiment, a WebGUI automated testing method based on a multimodal large model is provided, such as... Figure 1 As shown, it includes the following steps: S1. Construct the multimodal input of the webpage to be tested; S2. Construct operation prompts that include a test task description and the multimodal input; S3. The large model generates a test script based on the operation prompts, performs a syntax check on the generated test script, and if the syntax check is successful, proceeds to S4; otherwise, iterative correction is performed until the syntax check is successful. S4. Execute the test script if the syntax check is successful and generate a test report.
[0032] Specifically, this invention proposes a WebGUI automated testing framework, VL-GATF, based on a multimodal large model. This framework can autonomously acquire text and image information from web pages to complete testing tasks, without requiring manual intervention. Figure 2 As shown, after a task is given, VL-GATF first collects the visual information (i.e., screenshots) and text information (i.e., HTML elements) of the webpage to be tested, and then sends them to LMM. LMM generates a test script (based on Selenium and Unittest). Based on the task, text and visual information contained in the prompts, it performs a syntax check on the script until there are no syntax errors. Finally, LMM executes the test script and outputs a test report.
[0033] In step S1, interactive elements are labeled in the webpage screenshot using object detection, and a bounding box with a numerical label is overlaid on each element. Simultaneously, JavaScript is used to extract key text information such as XPath, class name, and ID of the labeled elements, constructing concise JSON data. The results are then observed in the environment, providing a dual-modal input of visually labeled images and structured text.
[0034] In step S2, CoT prompts are constructed. First, the system role is defined as Selenium Test Expert. Then, dynamic environment information (test task + multimodal input) is integrated. Next, the complex task is decomposed into an inference chain of "target understanding → element location → operation generation → assertion verification". Finally, the output format is standardized by Python code template and special operation prompts are embedded to form structured multimodal test instructions.
[0035] In step S3, test scripts are generated and verified using an iterative “generate-detect-regenerate” mechanism. First, test scripts based on Selenium and Unittest are generated. Then, AST analysis is performed using a Python syntax detector to identify two types of errors: structural errors and identifier errors.
[0036] In step S4, automated GUI testing is performed, running the final script through the WebDriver agent and recording three key metrics: task pass rate, subtask completion rate, and average execution time. Finally, the test report automatically generates assertion verification results, where assertion content is selected from a preset assertion pool and replaces real text information to avoid false assertions caused by the LLM illusion.
[0037] Furthermore, such as Figure 3 As shown, VL-GATF consists of four parts: environment observation, prompt construction, test script generation, and script execution. The following describes the specific implementation details of the above methods within the VL-GATF framework, mainly including the following parts: 1. Construction of the environmental observation module First, the GPT-4-ACT JavaScript tool is used to locate interactive elements (such as input boxes, buttons, and checkboxes) at the HTML code level, and bounding boxes with numerical labels are overlaid on each element at the image level to assist the Large Language Model (LLM) in element location and operation generation. Second, the text information is cleaned, extracting key content (such as element type, class name, ID, text content, and XPath) and integrating it into a JSON array. Compared to the original tool, the text length is reduced by approximately 50%, lowering the complexity of structured text and token costs. Subsequently, screenshots are uploaded to the server, and the LLM accesses the screenshots via URLs to reduce resource consumption. During the task execution phase, the framework uses the current screenshot, JSON-formatted auxiliary text, and the test task as input to generate test operations. If an error occurs in the detection module script, the framework retrieves the screenshot and text information, integrates the error information as new input, and regenerates the operation. This process, through the collaborative processing of visual and text information, ensures the accuracy of element location and supports the generation of test script assertions, thereby achieving efficient and automated WebGUI testing.
[0038] 2. Definition of actions in the test script This module uses Selenium to implement test scripts, simulating specific operations in manual testing (such as input, clicking, scrolling, etc.). The specific steps are as follows: First, the WebDriver is initialized, creating a WebDriver instance and launching the browser to establish the test environment. Second, element location and manipulation are performed. XPath is used to accurately locate interactive elements at the HTML code level, and bidirectional verification is performed using bounding boxes of numerical labels in visual information to ensure the correspondence between visual and text elements. After location, a Large Language Model (LLM) generates a range of operations, including click, input, scroll, drag, and back. The LLM's self-questioning mechanism selects operations to avoid generating illusions. Next, assertions are executed to verify that the test results are consistent with expectations, preventing logical errors or security vulnerabilities. Assertion generation is based on a pre-built assertion pool. The LLM selects a suitable template and replaces the content in the template with data obtained from text information (such as real text extracted by XPath). If an assertion fails, the test immediately stops and returns an error message for testers to locate the problem.
[0039] 3. Prompt for building the module This module constructs LMM hints for testing. It first accepts text, image, and task information generated by the environment observation module as input. Then, based on a pre-designed hint structure, it generates multimodal large-scale model hints for the task, outputting multimodal hints that the large-scale model can understand. The defined hint structure consists of five parts: 1) System Role: The role is explicitly defined as a GUI testing expert proficient in the Selenium framework, ensuring the generated script is professional and conforms to industry standards. 2) Environment Information: Provides task descriptions, text, and visual data to provide the model with sufficient contextual information for accurate understanding of the testing environment and objectives. 3) Thinking Process: First, the complex task is broken down into multiple sub-tasks, then QA selects the XPath and specific operations for each component in each step. 4) Formatted Answer: Provides a Python code structure template to standardize the script format and ensure the script conforms to Selenium development specifications. 5) Hints: Provides direct guidance for LMM management of complex tasks and hints for special tasks.
[0040] 4. Construction of the test script generation and execution module This module defines the test script generation process as a "generate-detect-regenerate" loop. The final output of this process is the test script based on Selenium and Unittest generated by LMM. Once the hint is constructed, it is submitted to LMM, which then generates the test script based on the task and provided information. Subsequently, a syntax checker examines the script. If errors are found, the environment is re-examined, and error messages and codes are recorded, resulting in a modified hint. The new hint is then resubmitted to LMM to regenerate the test script, iterating until there are no syntax errors, thus proceeding to the next step.
[0041] In the architecture of test scripts, syntax errors can be divided into two categories. The first category includes structural code errors, such as mismatched brackets and improper code block indentation. The second category is identifier-related errors, including misspelled keywords and unconventional naming conventions for identifiers. To effectively detect syntax errors in test scenarios, this invention implements a Python-based syntax detector. This detector first deconstructs the source code into lexical units through a lexical analysis process. The lexical analyzer scans the source code of the WebGUI automated test script, identifying and decomposing lexical units such as keywords and identifiers. Syntactic analysis based on the lexical analysis results involves the construction of the Abstract Syntax Tree (AST). This process mainly focuses on the syntactic structures related to the testing framework (such as Selenium and Unittest). The syntax detector can detect syntax errors in the test script in real time, thereby ensuring the fidelity and robustness of the script.
[0042] The final step is to run the test script in the local environment. During execution, this invention records the results of each assertion in the script to verify whether the operation meets expectations, and finally generates a test report. The report includes a task description, whether it meets expectations, and the execution time. This invention evaluates the test script from both performance and efficiency metrics to verify its effectiveness.
[0043] In another exemplary embodiment, based on the aforementioned VL-GATF framework, a WebGUI automated testing system based on a multimodal large model is provided, comprising: The environment observation module is used to construct multimodal inputs for the webpage under test; The prompt construction module is used to construct operation prompts that include a description of the test task and the multimodal input; The test script generation module is used to generate test scripts based on the operation prompts using the large model. The generated test scripts are then subjected to syntax checks. If the syntax check is successful, the script execution module is entered; otherwise, iterative correction is performed until the syntax check is successful. The script execution module is used to execute test scripts that successfully perform syntax checks and generate test reports.
[0044] In another exemplary embodiment, based on the same inventive concept as the method embodiment, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which, when executed by a processor, implements the WebGUI automated testing method based on a multimodal large model provided in this embodiment of the invention. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0045] In another exemplary embodiment, based on the same inventive concept as the method embodiment, an electronic device is provided, including a memory and a processor. The memory stores computer instructions that can be executed on the processor. When the processor executes the computer instructions, it performs the WebGUI automated testing method based on a multimodal large model provided in the embodiment of the present invention.
[0046] The processor may be a single-core or multi-core central processing unit or a specific integrated circuit, or one or more integrated circuits configured to implement the present invention.
[0047] The embodiments of the subject matter and functional operation described in this specification can be implemented in: tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing device or for controlling the operation of a data processing device. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing device.
[0048] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.
[0049] Suitable processors for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.
[0050] It should be understood that each block in a flowchart or block diagram can represent a module, segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0051] The above detailed embodiments are a description of the present invention. It should not be considered that the specific embodiments of the present invention are limited to these descriptions. For those skilled in the art, several simple deductions and substitutions can be made without departing from the concept of the present invention, and all of these should be considered to fall within the protection scope of the present invention.
Claims
1. A WebGUI automated testing method based on a multimodal large model, characterized in that, Includes the following steps: S1. Construct the multimodal input of the webpage to be tested; S2. Construct operation prompts that include a test task description and the multimodal input; S3. The large model generates a test script based on the operation prompts, performs a syntax check on the generated test script, and if the syntax check is successful, proceeds to S4; otherwise, iterative correction is performed until the syntax check is successful. S4. Execute the test script if the syntax check is successful and generate a test report.
2. The WebGUI automated testing method based on a multimodal large model according to claim 1, characterized in that, The construction of the multimodal input for the webpage to be tested includes: Interactive elements are labeled using object detection on screenshots of the webpage under test, and bounding boxes with numerical labels are overlaid on each element. At the same time, JavaScript tools are used to extract the text information of the labeled elements and integrate it into a JSON array.
3. The WebGUI automated testing method based on a multimodal large model according to claim 2, characterized in that, The text information includes XPath, element type, class name, ID, and text content.
4. The WebGUI automated testing method based on a multimodal large model according to claim 3, characterized in that, The construction includes a test task description and operation prompts for the multimodal input, including: Define the system role as Selenium Test Expert, and then integrate the test task description and the multimodal input; The test task is a reasoning chain of target understanding, element location, operation generation, and assertion verification, generating operations within a limited scope. The above operations are formatted using Python code structure templates and special operation prompts are embedded to form structured multimodal test instructions.
5. The WebGUI automated testing method based on a multimodal large model according to claim 4, characterized in that, The element positioning includes: Visual and textual verification was performed using the GPT-4-ACT tool in conjunction with XPath and bounding boxes with numerical labels. The operation generation includes: The large model selects operations through a question-and-answer mechanism; The assertion verification includes: Generate validation assertions by selecting templates from a pre-built assertion pool and replacing them with real text content.
6. The WebGUI automated testing method based on a multimodal large model according to claim 1, characterized in that, The syntax detection includes: AST analysis is performed using a Python syntax detector to identify and record structural and identifier errors.
7. The WebGUI automated testing method based on a multimodal large model according to claim 6, characterized in that, The iterative correction includes: Based on the structural and identifier errors, the modified operation prompts are submitted to the large model to regenerate the test script.
8. A WebGUI automated testing system based on a multimodal large model, characterized in that, include: The environment observation module is used to construct multimodal inputs for the webpage under test; The prompt construction module is used to construct operation prompts that include a description of the test task and the multimodal input; The test script generation module is used to generate test scripts based on the operation prompts using the large model. The generated test scripts are then subjected to syntax checks. If the syntax check is successful, the script execution module is entered; otherwise, iterative correction is performed until the syntax check is successful. The script execution module is used to execute test scripts that successfully perform syntax checks and generate test reports.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the WebGUI automated testing method based on a multimodal large model as described in any one of claims 1-7.
10. An electronic device comprising a memory and a processor, wherein the memory stores computer instructions executable by the processor, characterized in that, The processor executes the WebGUI automated testing method based on a multimodal large model as described in any one of claims 1-7 when running computer instructions.