Intelligent Web test method and system based on Playwright and AI

Abstract

The invention discloses an intelligent Web testing method and system based on Playwright and AI, and belongs to the technical field of Web testing. According to the method, a test script is executed through a Playwright framework, page elements and operations are packaged, and multi-dimensional information such as a DOM tree, network interaction and screenshot is collected; analyzing the test intention by means of the optimized language model, and generating a structured description; analyzing failure reasons through a multi-modal model in combination with failure moment data and intention description; and searching candidate elements based on an analysis result, generating a repair script, deploying a test node cluster dynamic allocation task and generating a report. According to the scheme, AI and automatic testing technologies are fused, intelligent promotion of the whole testing process is achieved, manual intervention is reduced, the maintenance cost is reduced, the testing accuracy and execution efficiency are improved, the testing coverage range is expanded, and reliable support is provided for quality assurance under Web application rapid iteration.

Description

Technical Field

[0001] This invention relates to the field of web testing technology, and in particular to an intelligent web testing method and system based on Playwright and AI. Background Technology

[0002] As web applications become increasingly feature-rich and complex, web testing, as a crucial element in ensuring application stability and usability, has garnered widespread attention within the industry. Currently, automated testing frameworks are commonly used to execute web testing tasks. By encapsulating page elements and operational logic, these frameworks achieve semi-automated or automated testing processes. Simultaneously, they collect fundamental data such as page structure, network interactions, and execution trajectories during testing, providing support for test result analysis. The application of AI technology in testing is becoming increasingly widespread. Some solutions incorporate language models and computer vision to optimize aspects such as test intent understanding and element location. Multi-browser compatibility testing and cross-domain scenario handling have also become standard requirements for web testing. Test execution efficiency and coverage have become core areas of focus for the industry.

[0003] Current web testing technologies still have significant shortcomings, failing to meet the high-efficiency testing needs of complex applications. Existing testing methods largely rely on fixed test scripts and locators. When page structure and element attributes change, scripts are prone to failure and require manual intervention for repair, leading to high test maintenance costs. Test intent parsing often depends on manual definition or simple rule matching, making it difficult to accurately understand the core objectives and logical steps of test cases, affecting test targeting. After a test fails, manual investigation using logs, screenshots, and other data is required to find the cause, which is inefficient and prone to missing key factors. Furthermore, the comprehensiveness of test data collection is insufficient, lacking integrated analysis of multi-dimensional data, and failing to provide sufficient support for intelligent decision-making. These problems result in a low level of intelligence in web testing, with many manual intervention steps in the testing process, making it difficult to adapt to the testing needs of rapid iteration in web applications and hindering the improvement of testing efficiency and quality. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide an intelligent web testing method and system based on Playwright and AI.

[0005] The objective of this invention is achieved through the following technical solution: A smart web testing method based on Playwright and AI is provided, which includes the following steps: S1. The test script is executed using a browser automation framework that supports multiple browsers, automatic waiting and network control. Page elements and operations are encapsulated according to the page object model. During the test execution, the DOM tree, network requests, network responses, execution trajectory, page screenshots, console logs, performance indicators and information related to the time of failure are collected synchronously, including the DOM at the time of failure, the page screenshot at the time of failure, the execution trajectory at the time of failure and the console log at the time of failure. S2. Based on the test case description, historical execution data, and information collected in S1, analyze the global test objectives and specific operational steps of the test cases to generate a structured intent description; S3. Combining test execution failure information, failure time-related information collected in S1, and structured intent description, analyze the cause of test failure and generate a failure analysis report; S4. Based on the information of the failed element, the DOM of the current page, the screenshot of the current page, and the failure analysis report, find matching candidate elements and generate the repaired test script, deploy the test execution node as a cluster and dynamically allocate test tasks, generate test reports and execute intelligent log recording.

[0006] Furthermore, step S1 includes the following sub-steps: S1.1. An automated framework that supports multiple browsers, automatically waits for elements to load, intercepts and modifies network requests, and has rich selector functions based on element ID, class name, tag name, attribute value, and text content for both location and combination location is selected as the testing tool. S1.2. Encapsulate page elements and their corresponding click, input, or selection operation logic according to the page object model, and clarify the relationship between elements and the order of operation dependencies; S1.3. During the test execution, cross-domain iframe scenarios are handled. Before the execution of key steps, after the execution, at the moment of operation node switching and at the moment of failure, the DOM tree, the DOM at the moment of failure, network requests, network responses, the execution trajectory containing complete operation steps, the execution trajectory at the moment of failure, page screenshots, the page screenshot at the moment of failure, console logs, the console logs at the moment of failure, as well as performance indicators such as page loading time, interface response time and resource loading time are collected synchronously.

[0007] Furthermore, step S2 includes the following sub-steps: S2.1. Obtain three types of historical execution data: test case descriptions in natural language or structured format, past failure records, execution success rate, and historical repair plans, as well as the DOM tree and page screenshots collected by S1; S2.2. Utilize the optimized language model of the prompt engineering, which has been optimized or constructed with intent parsing and prompt templates. The optimization data of this language model comes from the Web test intent-related corpus, which consists of the mapping data between test case descriptions and corresponding intent tags. The prompt template includes test intent parsing task descriptions, input data format requirements, and output structure specifications. The model combines DOM node attributes and hierarchical relationships in static page elements with error points and operation contexts in dynamic execution information to complete information association and semantic analysis. S2.3. Extract the operation sequence, expected results, and related element identification information from the script metadata to form a complementary analysis basis with the test case description. Analyze the global test objectives and the intent of each specific operation step of the test case, and output a structured intent description containing the global objectives and the intent of the step sequence.

[0008] Furthermore, step S3 includes the following sub-steps: S3.1. Collect test execution failure information, including error logs, stack traces, and the operation status when execution is interrupted; S3.2. Obtain the DOM at the time of failure collected by S1, screenshots of the old and new versions of the page at the time of failure, and the intent of the current test step generated by S2; S3.3. Utilize a combination of multimodal language model or computer vision technology and language model. The multimodal language model can simultaneously receive text input and visual input. When processing text input, semantic features are extracted. When processing visual input, image features are extracted. Then, the two types of features are integrated through a feature fusion algorithm. Text input includes error logs, stack traces, and test intents. Visual input includes screenshots of the old and new versions of the page at the time of failure. S3.4. Compare the position, style, text content, and DOM node hierarchy of elements in the old and new versions of the page, and determine the failure type based on the test intent. Failure types include locator failure, invalid value, workflow change, operation logic change, and error propagation. Locator failure corresponds to changes in DOM node attributes or paths, and workflow change corresponds to adjustments in the order of operation steps. Generate a failure analysis report that includes direct causes, root causes, and other causes.

[0009] Furthermore, step S4 includes the following sub-steps: S4.1. Obtain invalid element information, which includes the old DOM, old page screenshot, current page DOM, and current page screenshot; S4.2. Combining DOM structure analysis and visual information analysis, find matching candidate elements in the current page. DOM structure analysis includes traversing and searching parent nodes, child nodes, and sibling nodes. Visual information analysis utilizes computer vision technology or a multimodal language model. The multimodal language model learns the visual features and functional association data of page elements, and can associate the visual performance of elements with functional attributes. The matching candidate elements are sorted according to similarity and functional relevance. S4.3. Output the sorted candidate elements and their positioning information, including the generated selector; S4.4. Using the language model and guided by the test intent generated in S2, combined with the failure analysis report and candidate element information, generate a repaired test script that includes adding, deleting or modifying operation steps, updating the locator, and adjusting data parameters. Generate repair candidates and execute them one by one for verification, and compare the degree of fit between the execution results and the test intent.

[0010] Furthermore, in step S4, the database is optimized by analyzing slow query statements in the database, optimizing query indexes and SQL statement structures, configuring a master-slave separation architecture to achieve read-write separation, building a cache cluster, deploying cache nodes and using master-slave replication or sharding storage to achieve data synchronization between nodes, storing frequently accessed data, query results and commonly used configuration information during the test into the cache cluster, and responding to data query requests during the test through the cache cluster.

[0011] Furthermore, in step S2, a test data set is obtained, which includes input data and expected result data that are manually entered, exported from the database, or obtained from the interface. Combined with the structured intent description, the test data set is associated with the intent of specific operation steps. Data-driven test cases are designed in the form of tabular storage or JSON format encapsulation, and the input parameters, execution step relationships, and expected result judgment criteria of each test case are clarified, so that the test script can execute test operations in batches according to the data-driven test cases.

[0012] Furthermore, in step S3.3, the text input and visual input are preprocessed. The text input preprocessing includes error log formatting, keyframe extraction of stack information, and removal of redundant characters. The visual input preprocessing includes screenshot cropping, noise removal, and key area feature point extraction. Then, a fusion process is performed. The fused information is subjected to feature extraction and semantic analysis through a multimodal language model to generate unified analysis data. Based on the analysis data, the visual differences and DOM differences between the old and new versions of the page are compared.

[0013] Furthermore, in step S1, network requests during the test process are intercepted, and the parameters, request paths, request methods, and response status codes of the network requests are viewed. Some network request parameters or response data are modified, and test environments under different network conditions are simulated, including network latency within a specified time range, phased network interruptions, and network scenarios with limited bandwidth. In the simulated network environment, the test script continues to be executed, and relevant test data such as request success rate, response time, page rendering completion time, and operation execution results are collected. The relevant test data includes request success rate, response time, page rendering completion time, and operation execution results.

[0014] In some embodiments, a Playwright and AI-based intelligent web testing system includes a test execution module, an intent parsing module, a failure diagnosis module, an intelligent repair module, a high-concurrency optimization module, and a report generation module. The test execution module is connected to the intent parsing module and the failure diagnosis module, the intent parsing module is connected to the failure diagnosis module and the intelligent repair module, the failure diagnosis module is connected to the intelligent repair module, the intelligent repair module and the test execution module are both connected to the high-concurrency optimization module, and the high-concurrency optimization module and the test execution module are both connected to the report generation module. The test execution module executes test scripts and collects test-related information such as the DOM tree, network requests, responses, execution paths, screenshots, logs, performance metrics, and the DOM, page screenshots, execution paths, and console logs at the time of failure. The intent parsing module calls the optimized language model to parse the test intent and generate a structured description. The failure diagnosis module calls a multimodal language model or a combination of computer vision technology and a language model to analyze the causes of test failures and generate reports. The intelligent repair module calls the multimodal language model to find candidate elements and generate repair scripts. The high-concurrency optimization module optimizes the high-concurrency test environment. The report generation module generates test reports and implements intelligent log recording.

[0015] The beneficial effects of this invention are: (1) By executing test scripts and collecting multi-dimensional information through Playwright, and combining AI to analyze test intent and failure causes, the entire Web testing process can be automated, thereby improving the accuracy and intelligence of testing. (2) AI is deeply integrated with DOM, vision and other multi-dimensional data to accurately locate the root cause of test failure and generate repair scripts, reduce test interruptions caused by page changes and reduce the manpower and time costs of test maintenance. (3) Integrate technologies such as high-concurrency test environment optimization and data-driven testing to expand the test coverage and improve execution efficiency, providing stable and reliable technical support for Web application quality assurance. Attached Figure Description

[0016] Figure 1 A flowchart illustrating the steps of an intelligent web testing method based on Playwright and AI; Figure 2 This is a flowchart illustrating the specific steps of an intelligent web testing method based on Playwright and AI, provided as an example. Detailed Implementation

[0017] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1 See Figure 1 This embodiment provides an intelligent web testing method based on Playwright and AI, which includes the following steps: S1. The test script is executed using a browser automation framework that supports multiple browsers, automatic waiting and network control. Page elements and operations are encapsulated according to the page object model. During the test execution, the DOM tree, network requests, network responses, execution trajectory, page screenshots, console logs, performance indicators and information related to the time of failure are collected synchronously, including the DOM at the time of failure, the page screenshot at the time of failure, the execution trajectory at the time of failure and the console log at the time of failure. S2. Based on the test case description, historical execution data, and information collected in S1, analyze the global test objectives and specific operational steps of the test cases to generate a structured intent description; S3. Combining test execution failure information, failure time-related information collected in S1, and structured intent description, analyze the cause of test failure and generate a failure analysis report; S4. Based on the information of the failed element, the DOM of the current page, the screenshot of the current page, and the failure analysis report, find matching candidate elements and generate the repaired test script, deploy the test execution node as a cluster and dynamically allocate test tasks, generate test reports and execute intelligent log recording.

[0019] In some embodiments, step S1 includes the following sub-steps: S1.1. An automated framework that supports multiple browsers, automatically waits for elements to load, intercepts and modifies network requests, and has rich selector functions based on element ID, class name, tag name, attribute value, and text content for both location and combination location is selected as the testing tool. S1.2. Encapsulate page elements and their corresponding click, input, or selection operation logic according to the page object model, and clarify the relationship between elements and the order of operation dependencies; S1.3. During the test execution, cross-domain iframe scenarios are handled. Before the execution of key steps, after the execution, at the moment of operation node switching and at the moment of failure, the DOM tree, the DOM at the moment of failure, network requests, network responses, the execution trajectory containing complete operation steps, the execution trajectory at the moment of failure, page screenshots, the page screenshot at the moment of failure, console logs, the console logs at the moment of failure, as well as performance indicators such as page loading time, interface response time and resource loading time are collected synchronously.

[0020] In some embodiments, step S2 includes the following sub-steps: S2.1. Obtain three types of historical execution data: test case descriptions in natural language or structured format, past failure records, execution success rate, and historical repair plans, as well as the DOM tree and page screenshots collected by S1; S2.2. Utilize the optimized language model of the prompt engineering, which has been optimized or constructed with intent parsing and prompt templates. The optimization data of this language model comes from the Web test intent-related corpus, which consists of the mapping data between test case descriptions and corresponding intent tags. The prompt template includes test intent parsing task descriptions, input data format requirements, and output structure specifications. The model combines DOM node attributes and hierarchical relationships in static page elements with error points and operation contexts in dynamic execution information to complete information association and semantic analysis. S2.3. Extract the operation sequence, expected results, and related element identification information from the script metadata to form a complementary analysis basis with the test case description. Analyze the global test objectives and the intent of each specific operation step of the test case, and output a structured intent description containing the global objectives and the intent of the step sequence.

[0021] In some embodiments, step S3 includes the following sub-steps: S3.1. Collect test execution failure information, including error logs, stack traces, and the operation status when execution is interrupted; S3.2. Obtain the DOM at the time of failure collected by S1, screenshots of the old and new versions of the page at the time of failure, and the intent of the current test step generated by S2; S3.3. Utilize a combination of multimodal language model or computer vision technology and language model. The multimodal language model can simultaneously receive text input and visual input. When processing text input, semantic features are extracted. When processing visual input, image features are extracted. Then, the two types of features are integrated through a feature fusion algorithm. Text input includes error logs, stack traces, and test intents. Visual input includes screenshots of the old and new versions of the page at the time of failure. S3.4. Compare the position, style, text content, and DOM node hierarchy of elements in the old and new versions of the page, and determine the failure type based on the test intent. Failure types include locator failure, invalid value, workflow change, operation logic change, and error propagation. Locator failure corresponds to changes in DOM node attributes or paths, and workflow change corresponds to adjustments in the order of operation steps. Generate a failure analysis report that includes direct causes, root causes, and other causes.

[0022] In some embodiments, step S4 includes the following sub-steps: S4.1. Obtain invalid element information, which includes the old DOM, old page screenshot, current page DOM, and current page screenshot; S4.2. Combining DOM structure analysis and visual information analysis, find matching candidate elements in the current page. DOM structure analysis includes traversing and searching parent nodes, child nodes, and sibling nodes. Visual information analysis utilizes computer vision technology or a multimodal language model. The multimodal language model learns the visual features and functional association data of page elements, and can associate the visual performance of elements with functional attributes. The matching candidate elements are sorted according to similarity and functional relevance. S4.3. Output the sorted candidate elements and their positioning information, including the generated selector; S4.4. Using the language model and guided by the test intent generated in S2, combined with the failure analysis report and candidate element information, generate a repaired test script that includes adding, deleting or modifying operation steps, updating the locator, and adjusting data parameters. Generate repair candidates and execute them one by one for verification, and compare the degree of fit between the execution results and the test intent.

[0023] In some embodiments, in step S4, the database is optimized by analyzing slow query statements in the database, optimizing query indexes and SQL statement structures, configuring a master-slave separation architecture to achieve read-write separation, building a cache cluster, deploying cache nodes and using master-slave replication or sharding storage to achieve data synchronization between nodes, storing frequently accessed data, query results and commonly used configuration information during the test into the cache cluster, and responding to data query requests during the test through the cache cluster.

[0024] In some embodiments, in step S2, a test data set is obtained, which includes input data and expected result data that are manually entered, exported from the database, or obtained through an interface. Combined with a structured intent description, the test data set is associated with the intent of specific operation steps. Data-driven test cases are designed in the form of tabular storage or JSON format encapsulation, and the input parameters, execution step relationships, and expected result judgment criteria of each test case are clarified, so that the test script can perform test operations in batches according to the data-driven test cases.

[0025] In some embodiments, in step S3.3, the text input and visual input are preprocessed. The text input preprocessing includes error log formatting, keyframe extraction of stack information, and removal of redundant characters. The visual input preprocessing includes screenshot cropping, noise removal, and key region feature point extraction. Then, a fusion process is performed. The fused information is subjected to feature extraction and semantic analysis through a multimodal language model to generate unified analysis data. Based on the analysis data, the visual differences and DOM differences between the old and new versions of the page are compared.

[0026] In some embodiments, in step S1, network requests during the test process are intercepted, and the parameters, request paths, request methods, and response status codes of the network requests are viewed. Some parameters or response data of the network requests are modified, and test environments under different network conditions are simulated, including network latency within a specified time range, phased network interruptions, and network scenarios with limited bandwidth. In the simulated network environment, the test script continues to be executed and test data related to request success rate, response time, page rendering completion time, and operation execution results are collected. The relevant test data includes request success rate, response time, page rendering completion time, and operation execution results.

[0027] In some embodiments, a Playwright and AI-based intelligent web testing system includes a test execution module, an intent parsing module, a failure diagnosis module, an intelligent repair module, a high-concurrency optimization module, and a report generation module. The test execution module is connected to the intent parsing module and the failure diagnosis module, the intent parsing module is connected to the failure diagnosis module and the intelligent repair module, the failure diagnosis module is connected to the intelligent repair module, the intelligent repair module and the test execution module are both connected to the high-concurrency optimization module, and the high-concurrency optimization module and the test execution module are both connected to the report generation module. The test execution module executes test scripts and collects test-related information such as the DOM tree, network requests, responses, execution paths, screenshots, logs, performance metrics, and the DOM, page screenshots, execution paths, and console logs at the time of failure. The intent parsing module calls the optimized language model to parse the test intent and generate a structured description. The failure diagnosis module calls a multimodal language model or a combination of computer vision technology and a language model to analyze the causes of test failures and generate reports. The intelligent repair module calls the multimodal language model to find candidate elements and generate repair scripts. The high-concurrency optimization module optimizes the high-concurrency test environment. The report generation module generates test reports and implements intelligent log recording.

[0028] Example 2 This embodiment provides a specific implementation process for an intelligent web testing method based on Playwright and AI. It utilizes the Playwright framework to execute test scripts and collect multi-dimensional data, combining AI-related models to achieve test intent parsing, failure diagnosis, candidate element matching, and repair script generation. Simultaneously, it optimizes the high-concurrency testing environment and data storage architecture. Figure 2 As shown, the specific steps are as follows: Step S1. Execute the test script and collect relevant information: S1.1. An automated framework supporting multiple browsers, automatic element loading, network request interception and modification, and rich selector functionality based on element ID, class name, tag name, attribute value, and text content for both location and combined location is selected as the testing tool: This automation framework must be cross-browser compatible, capable of running test scripts in browsers with different rendering engines, without requiring additional adaptation code. The framework's automatic element loading wait function automatically adjusts the timing of operations based on the loading status of page elements, avoiding operation failures due to incomplete element loading. Network request interception and modification functions allow capturing all network requests during testing and adjusting request parameters and response data. The rich selector function supports positioning and combined positioning based on element ID, class name, tag name, attribute value, and text content, allowing selection of appropriate positioning methods based on different page element attributes, ensuring accurate element positioning in complex pages.

[0029] In some embodiments, the rich selector of the automation framework can support multiple positioning methods based on element ID, class name, tag name, attribute value, text content, etc., and allows multiple attributes to be combined for precise positioning, improving the success rate of element positioning in complex pages.

[0030] S1.2. Encapsulate page elements and their corresponding click, input, or selection operation logic according to the page object model, and clarify the relationships between elements and the order of operation dependencies: Abstract each page of the web application into an independent page object. Within each page object, define all the elements contained within that page and their corresponding operational logic, such as clicking a button, entering text, or selecting a dropdown option. During encapsulation, it's crucial to clarify the relationships between elements. For example, the operable state of a button depends on whether a valid input field has been filled in, and the display state of an element is determined by the results of operations performed on other elements. Simultaneously, clearly define the order of operations; for instance, login must be completed before subsequent operations like queries and submissions can be executed, ensuring that the execution logic of test scripts conforms to the web application's workflow.

[0031] In some embodiments, the encapsulation of the page object model can adopt a layered design, separating element positioning from operation logic, and storing element positioning information centrally in a configuration file for easy unified maintenance and modification.

[0032] S1.3. During test execution, cross-domain iframe scenarios are handled. The DOM tree, DOM at the time of failure, network requests, network responses, execution trajectory including complete operation steps, execution trajectory at the time of failure, page screenshots, console logs, and performance metrics such as page load time, API response time, and resource loading time are collected synchronously before and after key steps, at operation node switching moments, and at failure trigger moments. When handling cross-domain iframe scenarios, control of the iframe page is obtained through a dedicated interface provided by the framework, overcoming cross-domain access restrictions and enabling the location and manipulation of elements within the iframe. Key steps include page navigation, core function triggering, and data submission; operation node switching includes switching from one page element to another, from one page to another, and from the main page to the iframe page; the failure trigger moment refers to the moment when an error, exception, or failure to achieve the expected result occurs during the execution of the test script. At these moments, various types of data are collected simultaneously: the DOM tree records the complete structure and element attributes of the current page, retaining core fields such as node ID, class name, tag name, and hierarchical path during extraction; network requests record the parameters, path, method, and request header information of the requests; network responses record the status codes, response headers, and response body data returned by the server; execution traces record the sequence of test operations, operation object identification, and execution duration; page screenshots are stored in a standard image format, covering the core areas involved in the test operations; console logs record prompts, warnings, and error messages during page execution in timestamp order; and performance metrics record key data such as total page loading time, interface response time, and resource loading time.

[0033] In some embodiments, data acquisition can be set with custom trigger conditions. In addition to the default key nodes, specific acquisition trigger points can be added according to testing needs, such as when a specific element appears or after a specific request is responded to.

[0034] S1.4. Intercept network requests during the test process, view the parameters, request paths, request methods, and response status codes of the network requests, modify some network request parameters or response data, simulate test environments under different network conditions, including network latency within a specified time range, intermittent network interruptions, and network scenarios with limited bandwidth, continue to execute the test script in the simulated network environment, and collect test data related to request success rate, response time, page rendering completion time, and operation execution results: By leveraging the network control capabilities of the automation framework, all network requests generated during the testing process are intercepted. Detailed request parameters are examined, including request headers, data in the request body, request path, request method, and the server's response status code. Based on testing requirements, parameters of some network requests are modified, such as adjusting parameter values ​​to test the system's response under different input conditions; or the server's response data is modified to simulate abnormal responses, boundary value responses, and other scenarios. When simulating different network states, network latency (the time interval between sending a request and receiving a response) is set using the framework or system tools; periodic network interruptions are implemented to keep the network disconnected for specific time periods; network bandwidth is limited to reduce data transmission rates, simulating weak network environments. Test scripts are then executed in these simulated environments to collect relevant test data such as request success rate, response time, page rendering completion time, and operation execution results, comprehensively verifying the performance of the web application under different network conditions.

[0035] Step S2. Parse the test intent and generate a structured description: S2.1. Obtain three types of historical execution data: test case descriptions in natural language or structured format, past failure records, execution success rate, and historical repair plans, as well as the DOM tree and page screenshots collected by S1: Test case descriptions can be written by testers in natural language, clearly defining the test objectives, operational steps, expected results, and other information; alternatively, structured formats such as tables or JSON can be used, with test case content standardized according to fixed fields for easy machine reading and parsing. Past failure records include test failures that occurred during historical testing, including the failed operational steps, error messages, and the scenarios in which the failure occurred; execution success rate records the percentage of successful executions for each test case in historical executions; and historical remediation plans record the solutions and modifications taken to address past test failures. These historical execution data, along with the DOM tree (extracting node attributes and hierarchical relationship fields) and page screenshots (retaining core area image data) collected in step S1, are aggregated to provide comprehensive data support for subsequent test intent analysis.

[0036] S2.2. Utilize an optimized language model based on the intent parsing and prompt template. The optimized data for this language model comes from the Web test intent-related corpus, which consists of the mapping data between test case descriptions and corresponding intent tags. The prompt template includes test intent parsing task descriptions, input data format requirements, and output structure specifications. The model combines DOM node attributes and hierarchical relationships in static page elements with error points and operation contexts in dynamic execution information to complete information association and semantic analysis. AI deployment configuration for test intent parsing: Input data: Test case descriptions are in natural language text or JSON format, containing fixed fields such as "test objective," "operation steps," and "expected results"; DOM information extracts core content such as node ID, class name, and hierarchical path, presented in structured text; in dynamic execution information, error points record "operation step ID" and "error type," and the operation context records "preceding operations" and "current page state"; Web test intent-related corpus consists of mapping data between test case descriptions and corresponding intent tags, with intent tags divided into global target tags and step intent tags.

[0037] Output data: The structured intent description is in JSON format and contains three core fields: "Global Goal", "Step Sequence Intent", and "Associated Element Intent". "Step Sequence Intent" is an ordered array, corresponding to the intent of each operation step. "Associated Element Intent" is a key-value pair, where the key is the element identifier and the value is the element's functional intent.

[0038] Integration Logic: The model uses semantic analysis algorithms to associate test case description text with DOM node attributes and hierarchical relationships. For example, it binds the "Enter User Information" step with the "Username Input Box" and "Password Input Box" nodes in the DOM. It combines error points and operation context in dynamic execution information to filter irrelevant information, focuses on the core test intent, and ensures that the parsing results are consistent with the Web test logic, providing a foundation for intent matching in subsequent failure diagnosis.

[0039] In some embodiments, the language model can be optimized by incremental training. After accumulating new corpus related to Web test intent, the model can be trained again based on the original model to continuously improve the parsing accuracy. Setting output structure specifications for prompt templates can require the model to output intent descriptions in the form of key-value pairs to ensure the standardization of output data.

[0040] S2.3. Extract the operation sequence, expected results, and related element identification information from the script metadata to form a complementary analysis basis with the test case description. Analyze the global test objective and the intent of each specific operation step of the test case, and output a structured intent description containing the global objective and the intent of the step sequence: Script metadata contains descriptive information within the test script. Operation sequence records the execution order of various operations within the test script; expected results record the effect that each operation step or the entire test case should achieve, such as redirecting the page to a specified address, displaying specific text, or returning specific data; associated element identification information records the unique identifier of the page element targeted by each operation, such as element ID or custom attribute values. Combining this metadata information with the test case description overcomes the limitations of a single data source, making intent parsing more comprehensive and accurate. Through comprehensive analysis of this data, the global test objective of the test case is determined, i.e., the web application function or business scenario that the entire test case aims to verify; simultaneously, the intent of each specific operation step is parsed, i.e., the role that step plays in the test process and the specific purpose it aims to achieve. Finally, a structured intent description is output, clearly presenting the global objective and the intent of each step.

[0041] In some embodiments, the structured intent description may include data types such as global goals, step sequence intents, and associated element intents, wherein the step sequence intent is an ordered list, and each list item corresponds to an intent for an operation step, for example: {"Global Goal": "Verify user registration function", "Step Sequence Intent": ["Enter Username", "Enter Password", "Confirm Password", "Submit Registration Information"], "Associated Element Intent": {"Username Input Box": "Receive Account Information Entered by User", "Register Button": "Trigger Registration Request"}}.

[0042] S2.4. Obtain the test data set, which includes input data and expected result data obtained manually, exported from the database, or acquired through an interface. Combine this with a structured intent description, associate the test data set with the intent of specific operation steps. Design data-driven test cases using tabular storage or JSON format, clearly defining the input parameters, execution step relationships, and expected result judgment criteria for each test case. This allows the test scripts to execute test operations in batches according to the data-driven test cases. Test data sets can be acquired through several methods, including manual input (testers directly enter the required test data into the testing system), database export (exporting relevant data from the web application's database or a dedicated test database), and API acquisition (retrieving test data by calling the data provider's API). Input data consists of data that needs to be passed into the test script during the testing process, such as user accounts, passwords, and query parameters. Expected result data refers to the results that each operation step or the entire test case should achieve, such as page redirection addresses, the number of query results, and prompt text content. Combined with the structured intent description generated by S2.3, each set of data in the test data set is associated with the corresponding operation step intent, clarifying which operation step's intent the set of data is used to verify. Data-driven test cases are designed and encapsulated in tabular or JSON format. In tabular storage, each row represents a set of test data, and the columns correspond to input parameters, expected results, and associated step intents, respectively. In JSON format encapsulation, each test data set is presented as a key-value pair. The test cases should clearly define the mapping relationship between input parameters and variables in the test script, the order of execution steps (e.g., which steps use this set of data), and the criteria for judging expected results (e.g., equality, inclusion, conformity to a specific format, etc.). When the test script is executed, test data is read in batches from the data-driven test cases and sequentially substituted into the script for execution, thereby achieving automated batch execution of test cases.

[0043] In some embodiments, data-driven test cases can be stored in XML format, with tags defining input parameters, expected results, associated intents, and other information, facilitating compatibility with various testing tools.

[0044] Step S3. Analyze the causes of test failures and generate a failure analysis report: S3.1. Collect test execution failure information, including error logs, stack traces, and the operation status at the time of execution interruption: During test execution, when a test fails, the system automatically captures an error log. The error log contains key information such as the error type, the time of occurrence, and the error description, providing a clear picture of the basic error situation. The stack trace records the function call chain of the program at the time of the error, tracing upwards from the currently executing function to display the function name, line number, and parameter information at each call level, helping to pinpoint the specific location of the error. The operation status at the time of interruption records the test steps being executed, completed steps, the current page element status, network connection status, and other information, providing a basis for reconstructing the error scenario and analyzing the cause of the failure.

[0045] S3.2. Obtain the DOM at the time of failure captured by S1, screenshots of the old and new versions of the page at the time of failure, and the intent of the current test step generated by S2: The DOM record at the moment of failure shows the complete structure and element attributes of the page when the test failed. By comparing it with the DOM in the normal state, it's possible to quickly identify whether elements have changed. Screenshots of the old and new versions of the page at the moment of failure correspond to the normal page visual state before the web application update (or when the test case was written) and the page visual state after the update (when the test failed), respectively. Image comparison allows for a direct observation of differences in page visual layout, element positions, text content, etc. The intent of the current test step generated by S2 clearly defines the purpose that the step wants to achieve. Combining this intent with the analysis of the failure cause can determine whether the problem lies in the execution of the step itself or in the inability to achieve the intent due to page changes.

[0046] S3.3. Utilizing a combination of multimodal language models or computer vision technology and language models, the multimodal language model can simultaneously receive text input and visual input. Semantic features are extracted during text input processing, and image features are extracted during visual input processing. The two types of features are then integrated using a feature fusion algorithm. Text input includes error logs, stack traces, and test intents; visual input includes screenshots of the old and new versions of the page at the time of failure. AI-based failure diagnosis deployment: Input data: Text input is in a structured format. Error logs extract the fields "error type", "error description" and "steps in which it occurred". Stack information extracts the fields "key call chain" and "error line number". Test intent reuses the structured intent description JSON data output by S2. Visual input consists of screenshots of the old and new versions of the page, which are uniformly converted to the same image format to focus on the core area of ​​the test operation.

[0047] Output data: The fused feature vector is a fixed-dimensional numerical array with two types of labels: "semantic feature labels" (such as "error type features" and "test intent features") and "image feature labels" (such as "element location features" and "text content features").

[0048] Fusion Logic: If a multimodal language model is used, semantic analysis algorithms extract key information from the text input, associating error types such as "locator not found" with the "element operation" step in the test intent; image analysis algorithms extract element position, style, and text features from page screenshots, corresponding to DOM structure information; feature fusion algorithms employ an attention mechanism, assigning higher weights to features highly relevant to failure analysis, and the integrated feature vector is directly used for failure type judgment, aligning with the failure diagnosis logic of Web testing. If a combination of computer vision technology and a language model is used, computer vision technology identifies page differences through image comparison algorithms and transforms them into structured text such as "element A position offset," then the language model semantically associates the differing text with error logs and test intents, outputting the association analysis results.

[0049] In some embodiments, the feature fusion algorithm may employ an attention mechanism fusion method, which automatically focuses on features that are more important to failure analysis, assigns them higher weights, and improves the effectiveness of feature fusion; the text description of the difference information extracted by computer vision technology may include types such as "element position offset", "element style change", "element addition", "element missing", and "text content modification".

[0050] S3.4. Preprocess the text and visual inputs. Text input preprocessing includes error log formatting, keyframe extraction from stack information, and removal of redundant characters. Visual input preprocessing includes screenshot cropping, noise removal, and key region feature point extraction. Then, a fusion process is performed. A multimodal language model is used to extract features and perform semantic analysis on the fused information, generating unified analysis data. Based on this analysis data, the visual and DOM differences between the old and new versions of the page are compared. The failure type is determined by combining the test intent. Failure types include locator failure, invalid values, workflow changes, operational logic changes, and error propagation. Locator failure corresponds to changes in DOM node attributes or paths, and workflow changes correspond to adjustments in the order of operational steps. A failure analysis report containing direct causes, root causes, and other causes is generated. During text input preprocessing, error logs are formatted, organizing error type, description, and occurrence time according to a fixed format. Keyframes are extracted from stack traces to filter out function call levels and code lines directly related to the error, removing irrelevant call records. Redundant characters (such as spaces, special symbols, and repeated characters) are removed from all text input to simplify the text content. During visual input preprocessing, page screenshots are cropped to remove areas irrelevant to the test elements. Image denoising algorithms are used to remove noise points from screenshots, improving image clarity. Feature points are extracted from key areas in the screenshots (such as elements involved in the test operation and core functional areas of the page) to highlight key information. After preprocessing, text and visual inputs are fused. A multimodal language model is used to extract the fused features and perform semantic analysis to generate unified analysis data. Based on this analysis data, visual differences (such as changes in element position, style, and text content) and DOM differences (such as changes in node attributes and hierarchical relationships) between the old and new versions of the page are compared, and the failure type is determined in conjunction with the test intent. Locator failure refers to the inability of the original test script to locate the target element due to changes in DOM node attributes or paths; invalid values ​​refer to input data or server-returned data that do not meet test expectations; workflow changes refer to adjustments in the order of operation steps in the web application's process, causing a mismatch between the execution logic of the original test script and the application flow; operation logic changes refer to changes in the execution conditions, results, or internal processing logic of a certain operation; error propagation refers to an error in one module or step causing other related modules or steps to also fail. After the analysis is completed, a failure analysis report is generated, where the direct cause refers to the immediate factor that caused the failure, the root cause refers to the underlying factor that triggered the failure, and other causes refer to secondary factors that may have an impact on the failure.

[0051] In some embodiments, the failure analysis report may include analysis content such as failure type percentage statistics and ranking of common failure causes, providing data reference for the optimization of web applications.

[0052] Step S4. Select candidate elements, generate repair scripts, and optimize the test environment: S4.1. Obtain invalid element information, including the old DOM, old page screenshot, current page DOM, and current page screenshot: The old DOM and screenshot of the old page correspond to the page structure and visual state when the test case was written or during the last successful execution. This includes data such as the location, visual characteristics, and hierarchical relationship of the failing element at that time. The current page DOM and screenshot of the current page correspond to the page structure and visual state when the test failure occurred. This includes information on all elements and their visual appearance on the current page where the failing element is located. By obtaining this information, we can clarify the differences between the historical and current states of the failing element, providing foundational data for subsequently finding matching candidate elements.

[0053] S4.2. Combining DOM structure analysis and visual information analysis, find matching candidate elements in the current page. DOM structure analysis includes traversing and searching parent nodes, child nodes, and sibling nodes. Visual information analysis utilizes computer vision technology or a multimodal language model. The multimodal language model learns the visual features and functional association data of page elements, enabling it to associate the visual appearance of elements with their functional attributes. Matching candidate elements are then sorted according to similarity and functional relevance. AI deployment configuration for candidate element matching: Input data: Extract the "parent node identifier", "child node index", and "sibling node characteristics" fields from the old DOM for invalid elements; extract the "visual feature vector" and "layout relationship with surrounding elements" from the old page screenshot for invalid elements; extract the node attributes and hierarchical relationships of all elements from the current page DOM; extract the visual feature vectors and layout information of all elements from the current page screenshot; the visual features of page elements are associated with functions as "visual feature vector - function label" mapping pairs, where function labels include "input function", "click function", and "select function".

[0054] Output data: The candidate element list is a JSON array, and each element contains the fields of "element identifier", "DOM structure similarity score", "visual feature similarity score", "functional relevance score" and "location information", which are sorted in descending order of comprehensive score; the location information includes the generated selector string and selector type label.

[0055] Fusion Logic: The multimodal language model uses inference algorithms to match the visual features of current page elements with the "visual features-function" mapping relationship to determine whether the element's functional attributes are consistent with those of the invalid element; it also calculates structural similarity by combining the DOM structure analysis results, sorts the two scores, and outputs candidate elements that directly correspond to the Web test element location requirements, providing a precise basis for updating the locator in the repair script.

[0056] In some embodiments, the ranking criteria for candidate elements may include three types: DOM structure similarity, visual feature similarity, and functional relevance. Each type is scored on a 100-point scale, and the final score is calculated by weighted sum of the three. The weights can be adjusted according to the needs of the test scenario.

[0057] S4.3. Output the sorted candidate elements and their positioning information, including the generated selector: The candidate elements sorted in S4.2 are output sequentially, with each candidate element accompanied by corresponding positioning information, represented by a generated selector. Selector generation is based on the current page's DOM structure and element attributes, ensuring that the selector uniquely identifies the corresponding candidate element and is less prone to failure due to minor page changes. The generated selector type can be flexibly selected based on element characteristics, such as ID-based selectors, attribute-based selectors, and hierarchy-based selectors. Multiple selector types can also be combined to improve the stability and accuracy of positioning.

[0058] S4.4. Using the language model and guided by the test intent generated in S2, combined with the failure analysis report and candidate element information, generate a repaired test script that includes adding, deleting, or modifying operation steps, updating the locator, and adjusting data parameters. Generate repair candidates and execute them one by one for verification, comparing the degree of fit between the execution results and the test intent. Fix the AI ​​deployment configuration generated by the script: Input data: The test intent reuses the structured intent description JSON data output by S2; the failure analysis report extracts the fields of "failure type", "direct cause" and "root cause"; the candidate element information extracts the "location information" and "functional relevance score" of the top three candidate elements; the input data is concatenated into model prompt text in a fixed format to clarify the syntax specifications of the repair script and the business requirement of "keeping the original test intent unchanged".

[0059] Output data: The repaired test script is an executable code text, with an attached "Repair Instructions" JSON data, including fields such as "Repair Type", "Corresponding Failure Reason" and "Verification Points"; the repair candidates are an array of multiple script texts, with each script corresponding to a repair approach.

[0060] Integration Logic: The language model uses a code generation algorithm to match repair strategies based on the failure type. For example, if the locator fails, the locator should be updated; if the workflow changes, the step order should be adjusted. The test intent serves as a constraint to ensure that the original test objective is not deviated from. After generating multiple repair candidates, the execution verification algorithm compares the script execution result with the "expected result" in the test intent and outputs the script with the highest degree of fit, which is adapted to the Web test script repair logic.

[0061] In some embodiments, the verification of the repair script can be evaluated from three dimensions: step execution success rate, expected result achievement, and intent matching. Each dimension has a clear judgment criterion, and only the repair script that meets all the judgment criteria is determined as the final repair result.

[0062] S4.5. Optimize the database by analyzing slow queries, optimizing query indexes and SQL statement structure, configuring a master-slave architecture to achieve read-write separation, building a cache cluster, deploying cache nodes, and using master-slave replication or sharding storage to achieve data synchronization between nodes. Store frequently accessed data, query results, and commonly used configuration information during the test into the cache cluster, and use the cache cluster to respond to data query requests during the test. When optimizing a database, first enable the slow query log function to capture slow queries that take longer than a set threshold. By analyzing the execution plan of these slow queries, identify the causes of slowness, such as missing indexes, redundant SQL statements, or overly complex joins. For missing indexes, create appropriate query indexes to improve query efficiency. For redundant SQL statements, simplify the SQL statement structure, removing unnecessary query conditions, joins, and calculation steps. For complex joins, break them down into multiple simpler queries and execute them step by step.

[0063] Configure a master-slave database architecture, setting up one master database and multiple slave databases. The master database handles all write operations (such as data insertion, updating, and deletion), while the slave databases handle all read operations (such as data querying). The master database synchronizes write operation logs to each slave database, and the slave databases update their own data based on the synchronized logs, ensuring data consistency between the master and slave databases. This read-write separation reduces the access pressure on the master database and improves the database's concurrency processing capabilities.

[0064] When building a cache cluster, multiple cache nodes are deployed. Using a master-slave replication approach, one master cache node and multiple slave cache nodes are configured. The master node handles data write and update operations, while the slave nodes synchronize data with the master node and respond to read requests. Using a sharded storage approach, cached data is split into different cache nodes according to preset rules (such as data hash values ​​and types), with each node storing a portion of the data. Frequently accessed data during testing (such as test case information, element location information, and user configuration information), database query results, and commonly used configuration information are stored in the cache cluster. When this data needs to be queried during testing, it is retrieved first from the cache cluster. If the required data exists in the cache, it is directly returned to the testing system without accessing the database, reducing database access pressure and improving data query speed.

[0065] In some embodiments, data synchronization of the cache cluster can be performed in real time, where data updates on the master node are immediately synchronized to the slave nodes or other shard nodes to ensure the consistency of cached data.

[0066] S4.6. Deploy the test execution nodes as a cluster, dynamically allocate test tasks, generate test reports, and perform intelligent logging: Based on testing requirements and resource allocation, multiple test execution nodes are deployed. Each node is configured with the necessary operating system, browser, automation framework, dependency libraries, and other environments for testing. These nodes together constitute a test execution cluster. A cluster management platform is built to manage all test execution nodes uniformly, including monitoring node operating status (such as CPU utilization, memory usage, disk I / O, and network bandwidth), allocating and scheduling test tasks, and collecting and summarizing test results. When dynamically allocating test tasks, the cluster management platform collects load data for each node in real time, calculates the task processing capacity of each node based on the load, and assigns new test tasks to nodes with lower loads and sufficient resources, ensuring load balancing across the nodes in the cluster.

[0067] After test execution, the integrated report generation system collects various data from the testing process, including test case execution results, execution time, failure status, failure analysis reports, repair results, performance metrics, etc. This data is then organized and analyzed to generate a test report containing a test overview, detailed execution steps, result statistics, failure cause analysis, repair suggestions, and performance analysis. Simultaneously, intelligent logging is performed, recording various log information during the testing process, including test execution logs, node operation logs, error logs, database operation logs, and cache operation logs. Log recording supports level-based filtering, allowing users to set different levels based on log importance, enabling them to filter and view logs at specific levels as needed. It also supports automatic time-based log file splitting, allowing splitting by day, hour, or other time dimensions to avoid excessively large single log files and facilitate log management, querying, and backup.

[0068] This embodiment utilizes Playwright and AI-based intelligent web testing methods. The Playwright framework executes test scripts across multiple browser environments, encapsulating elements and operations using a page object model to ensure maintainability and cross-browser compatibility. Multi-dimensional data is collected during testing, providing a comprehensive data source for subsequent AI analysis, ensuring comprehensive and accurate test data collection. Leveraging language models and multimodal language models, and through explicit input / output data configuration and deep integration of algorithms and logic, the system accurately completes test intent parsing, failure diagnosis, candidate element matching, and repair script generation. This overcomes the limitations of traditional automated testing, automatically understanding test objectives, analyzing failure causes, and generating repair solutions, thus enhancing the intelligence level of testing. Database optimization, cache cluster construction, and test node cluster deployment optimize the high-concurrency testing environment, reducing data access latency and improving test task execution efficiency and system stability. Data-driven test case design enables batch execution of test cases, expanding test coverage; network environment simulation allows testing to cover application performance under different network conditions, enhancing test comprehensiveness. The overall solution forms a complete closed loop from test execution, data collection, intelligent analysis, script repair to environment optimization and report generation, reducing manual intervention, lowering test and maintenance costs, and providing reliable technical support for the quality assurance of web applications.

[0069] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A smart web testing method based on Playwright and AI, characterized in that, Includes the following steps: S1. The test script is executed using a browser automation framework that supports multiple browsers, automatic waiting and network control. Page elements and operations are encapsulated according to the page object model. During the test execution, the DOM tree, network requests, network responses, execution trajectory, page screenshots, console logs, performance indicators and information related to the time of failure are collected synchronously, including the DOM at the time of failure, the page screenshot at the time of failure, the execution trajectory at the time of failure and the console log at the time of failure. S2. Based on the test case description, historical execution data, and information collected in S1, analyze the global test objectives and specific operational steps of the test cases to generate a structured intent description; S3. Combining test execution failure information, failure time-related information collected in S1, and structured intent description, analyze the cause of test failure and generate a failure analysis report; S4. Based on the information of the failed element, the DOM of the current page, the screenshot of the current page, and the failure analysis report, find matching candidate elements and generate the repaired test script, deploy the test execution node as a cluster and dynamically allocate test tasks, generate test reports and execute intelligent log recording.

2. The method according to claim 1, characterized in that, Step S1 includes the following sub-steps: S1.

1. An automated framework that supports multiple browsers, automatically waits for elements to load, intercepts and modifies network requests, and has rich selector functions based on element ID, class name, tag name, attribute value, and text content for both location and combination location is selected as the testing tool. S1.

2. Encapsulate page elements and their corresponding click, input, or selection operation logic according to the page object model, and clarify the relationship between elements and the order of operation dependencies; S1.

3. During the test execution, cross-domain iframe scenarios are handled. Before the execution of key steps, after the execution, at the moment of operation node switching and at the moment of failure, the DOM tree, the DOM at the moment of failure, network requests, network responses, the execution trajectory containing complete operation steps, the execution trajectory at the moment of failure, page screenshots, the page screenshot at the moment of failure, console logs, the console logs at the moment of failure, as well as performance indicators such as page loading time, interface response time and resource loading time are collected synchronously.

3. The method according to claim 1, characterized in that, Step S2 includes the following sub-steps: S2.

1. Obtain three types of historical execution data: test case descriptions in natural language or structured format, past failure records, execution success rate, and historical repair plans, as well as the DOM tree and page screenshots collected by S1; S2.

2. Utilize the optimized language model of the prompt engineering, which has been optimized or constructed with intent parsing and prompt templates. The optimization data of this language model comes from the Web test intent-related corpus, which consists of the mapping data between test case descriptions and corresponding intent tags. The prompt template includes test intent parsing task descriptions, input data format requirements, and output structure specifications. The model combines DOM node attributes and hierarchical relationships in static page elements with error points and operation contexts in dynamic execution information to complete information association and semantic analysis. S2.

3. Extract the operation sequence, expected results, and related element identification information from the script metadata to form a complementary analysis basis with the test case description. Analyze the global test objectives and the intent of each specific operation step of the test case, and output a structured intent description containing the global objectives and the intent of the step sequence.

4. The method according to claim 1, characterized in that, Step S3 includes the following sub-steps: S3.

1. Collect test execution failure information, including error logs, stack traces, and the operation status when execution was interrupted; S3.

2. Obtain the DOM at the time of failure collected by S1, screenshots of the old and new versions of the page at the time of failure, and the intent of the current test step generated by S2; S3.

3. Utilize a combination of multimodal language model or computer vision technology and language model. The multimodal language model can simultaneously receive text input and visual input. When processing text input, semantic features are extracted. When processing visual input, image features are extracted. Then, the two types of features are integrated through a feature fusion algorithm. Text input includes error logs, stack traces, and test intents. Visual input includes screenshots of the old and new versions of the page at the time of failure. S3.

4. Compare the position, style, text content, and DOM node hierarchy of elements in the old and new versions of the page, and determine the failure type based on the test intent. Failure types include locator failure, invalid value, workflow change, operation logic change, and error propagation. Locator failure corresponds to changes in DOM node attributes or paths, and workflow change corresponds to adjustments in the order of operation steps. Generate a failure analysis report that includes direct causes, root causes, and other causes.

5. The method according to claim 1, characterized in that, Step S4 includes the following sub-steps: S4.

1. Obtain invalid element information, which includes the old DOM, old page screenshot, current page DOM, and current page screenshot; S4.

2. Combining DOM structure analysis and visual information analysis, find matching candidate elements in the current page. DOM structure analysis includes traversing and searching parent nodes, child nodes, and sibling nodes. Visual information analysis utilizes computer vision technology or a multimodal language model. The multimodal language model learns the visual features and functional association data of page elements, and can associate the visual performance of elements with functional attributes. The matching candidate elements are sorted according to similarity and functional relevance. S4.

3. Output the sorted candidate elements and their positioning information, including the generated selector; S4.

4. Using the language model and guided by the test intent generated in S2, combined with the failure analysis report and candidate element information, generate a repaired test script that includes adding, deleting or modifying operation steps, updating the locator, and adjusting data parameters. Generate repair candidates and execute them one by one for verification, and compare the degree of fit between the execution results and the test intent.

6. The method according to claim 1, characterized in that, In step S4, the database is optimized by analyzing slow query statements, optimizing query indexes and SQL statement structures, configuring a master-slave architecture to achieve read-write separation, building a cache cluster, deploying cache nodes and using master-slave replication or sharding storage to achieve data synchronization between nodes, storing frequently accessed data, query results and commonly used configuration information during the test into the cache cluster, and responding to data query requests during the test through the cache cluster.

7. The method according to claim 1, characterized in that, In step S2, a test data set is obtained, which includes input data and expected result data that are manually entered, exported from the database, or obtained from the interface. Combined with the structured intent description, the test data set is associated with the intent of specific operation steps. Data-driven test cases are designed in the form of tabular storage or JSON format encapsulation. The input parameters, execution step relationships, and expected result judgment criteria of each test case are clarified, so that the test script can execute test operations in batches according to the data-driven test cases.

8. The method according to claim 4, characterized in that, In step S3.3, the text input and visual input are preprocessed. The text input preprocessing includes error log formatting, keyframe extraction of stack information, and removal of redundant characters. The visual input preprocessing includes screenshot cropping, noise removal, and key region feature point extraction. Then, a fusion process is performed. The fused information is subjected to feature extraction and semantic analysis through a multimodal language model to generate unified analysis data. Based on the analysis data, the visual differences and DOM differences between the old and new versions of the page are compared.

9. The method according to claim 1, characterized in that, In step S1, network requests during the test are intercepted, and the parameters, request paths, request methods, and response status codes of the network requests are viewed. Some network request parameters or response data are modified, and test environments under different network conditions are simulated, including network latency within a specified time range, intermittent network interruptions, and network scenarios with limited bandwidth. In the simulated network environment, the test script continues to be executed, and relevant test data such as request success rate, response time, page rendering completion time, and operation execution results are collected. The relevant test data includes request success rate, response time, page rendering completion time, and operation execution results.

10. A smart web testing system based on Playwright and AI, used to execute the method described in any one of claims 1-9, characterized in that, It includes a test execution module, an intent parsing module, a failure diagnosis module, an intelligent repair module, a high-concurrency optimization module, and a report generation module. The test execution module is connected to the intent parsing module and the failure diagnosis module. The intent parsing module is connected to the failure diagnosis module and the intelligent repair module. The failure diagnosis module is connected to the intelligent repair module. The intelligent repair module and the test execution module are both connected to the high-concurrency optimization module. The high-concurrency optimization module and the test execution module are both connected to the report generation module. The test execution module is used to execute test scripts and collect test-related information such as DOM tree, network requests, responses, execution traces, screenshots, logs, performance metrics, DOM at the time of failure, page screenshot at the time of failure, execution trace at the time of failure, and console logs at the time of failure; the intent parsing module is used to call the optimized language model, parse the test intent, and generate a structured description. The failure diagnosis module is used to call a multimodal language model or a combination of computer vision technology and language model to analyze the cause of test failure and generate a report; The intelligent repair module is used to call the multimodal language model to find candidate elements and generate repair scripts; the high-concurrency optimization module is used to optimize the high-concurrency test environment; and the report generation module is used to generate test reports and implement intelligent log recording.

Images