An AI-based automated testing method, electronic device, and storage medium

By using AI-based automated testing methods, combined with multimodal description information and semantic-level result judgment, a closed-loop self-repair mechanism is formed, which solves the problems of fragile element location and rigid result judgment in web page automated testing, and improves test stability and executability.

CN123114953APending Publication Date: 2026-11-17FUZHOU CHANGXIN INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611206094.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-10
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing automated web page testing suffers from fragile element location, poor script stability, lack of closed-loop self-repair mechanism during execution, insufficient executability of generated scripts, and overly mechanical result judgment methods that fail to accurately distinguish the causes of failure.

Method used

An AI-based automated testing method is adopted. By receiving natural language testing requirements, recording the webpage operation process, extracting multimodal description information, generating test scripts, and performing intelligent diagnosis and repair when execution fails, semantic-level result judgment is made by combining information such as DOM snapshots and page screenshots, forming a closed loop of element collection, script generation, execution monitoring, anomaly repair and result judgment.

Benefits of technology

It significantly improves the overall stability of the automated testing chain, overcomes the problem of easy failure of traditional static positioning, reduces the instability of scripts caused by natural language ambiguity, and improves the executability of generated scripts in real environments and the accuracy of result judgment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123114953A_ABST
    Figure CN123114953A_ABST
Patent Text Reader

Abstract

This application relates to the field of automated software testing, specifically an AI-based automated testing method, electronic device, and storage medium. The method includes: requirement input (receiving natural language test requirements, business rules, test data, and environmental information); element acquisition (recording webpage operation processes and extracting multimodal description information of page elements using fixed rules); test intent parsing (converting natural language test requirements into a structured intermediate representation of test intent); script generation (generating test scripts based on the intermediate representation of test intent and element description information); and execution monitoring (executing the generated scripts and collecting real-time information such as exceptions, DOM snapshots, page screenshots, network logs, console logs, and execution trajectory information). This method effectively improves upon the vulnerabilities, high maintenance costs, and rigid result judgment issues of existing webpage automated test scripts.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of automated testing technology, and in particular to an AI-based automated testing method, electronic device, and storage medium. Background Technology

[0002] Automated testing is a software quality assurance method that uses tools to transform manual testing into machine execution under preset normal and abnormal conditions for software and web pages. It is mainly applied to regression testing and performance testing scenarios.

[0003] Current automated web testing is typically implemented using frameworks such as Selenium, Playwright, Puppeteer, and Robot Framework. Existing technologies usually involve testers generating test steps through recording and playback tools or by manually writing code. Test scripts are generated based on static positioning information of page elements, such as XPath, CSS Selector, id, name, and text. During execution, the script sequentially performs operations such as clicking, inputting, redirecting, and validating. During validation, hard-coded assertions are used to precisely compare text, attribute values, status codes, or page nodes to determine whether the test passes or fails.

[0004] However, the existing technical solutions mentioned above do have the following drawbacks: fragile element positioning and poor script stability; lack of a closed-loop self-repair mechanism in the execution phase; insufficient executability of the generated script; overly mechanical result judgment methods; and inability to accurately distinguish the causes of failure. Summary of the Invention

[0005] To help improve the problems of existing web page automated testing scripts being fragile, having high maintenance costs, and having rigid result judgments, this application provides an AI-based automated testing method, electronic device, and storage medium.

[0006] Firstly, this application provides an AI-based automated testing method, employing the following technical solution: Input requirements include natural language testing requirements, business rules, test data, and environment information. Element collection involves recording the webpage operation process and extracting multimodal description information of page elements based on fixed rules. Test intent parsing converts natural language testing requirements into a structured intermediate representation of test intent; Script generation: Generates test scripts based on intermediate representations of test intent and element description information; Perform monitoring, execute the generated scripts, and collect anomaly, DOM snapshots, page screenshots, network logs, console logs, and / or runtime trajectory information in real time; Intelligent diagnosis and repair analyze the cause of failure when execution fails, identify candidate target elements, generate alternative positioning strategies or corrected test steps, and trigger re-execution.

[0007] Secondly, this application provides an electronic device that adopts the following technical solution: An electronic device comprising: At least one processor; Memory; At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application being configured to: execute the AI-based automated testing method as described above.

[0008] Thirdly, this application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the AI-based automated testing method provided in the first aspect.

[0009] In summary, the technical solution of this application connects element acquisition, script generation, execution monitoring, anomaly repair, and result judgment into a unified closed loop, significantly improving the overall stability of the automated testing chain. By integrating DOM, text semantics, visual regions, and action context for element repair, the problem of easy failure of traditional static positioning can be effectively overcome. A "two-stage generation" approach is adopted, first forming an intermediate representation of the test intent, and then generating the specific script, reducing script instability caused by natural language ambiguity. Waiting, retry, anomaly handling, and context control logic are automatically injected into the script, improving the executability of the generated script in a real environment. By replacing simple hard-coded assertions with semantic-level result judgment, more complex scenarios such as visual verification, interaction state verification, and business state verification can be covered. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating an AI-based automated testing method provided in an embodiment of this application. Figure 2 This is a schematic diagram of the composition and data flow of the element acquisition module of an AI-based automated testing method provided in another embodiment of this application; Figure 3 This is a flowchart of a two-stage generation process from natural language to test scripts for an AI-based automated testing method provided in another embodiment of this application. Figure 4 This is a flowchart of intelligent diagnosis and adaptive repair after execution failure of an AI-based automated testing method provided in another embodiment of this application; Figure 5 This is a flowchart illustrating an AI-based automated testing method provided in another embodiment of this application; Figure 6 This is a schematic diagram of a module of an electronic device provided in an embodiment of this application. Detailed Implementation

[0011] To make the purpose, technical solution, and advantages of this application clearer, the following description is provided in conjunction with the appendix. Figure 1-6 The present application will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the application.

[0012] This application provides an AI-based automated testing method for use in a testing system.

[0013] This embodiment uses a web page automated testing system as an application example to specifically illustrate the above method. In actual implementation, the above method can also be used in other automated testing systems, such as H5 pages, hybrid application interfaces, backend management systems, low-code platform pages, or enterprise business inspection scenarios. As long as the target object has collectable page structure features, executable actions, and identifiable business results, it can be implemented with reference to this invention. This embodiment does not limit it in this regard.

[0014] Specific reference Figure 1 As shown, an AI-based automated testing method is characterized by comprising: S10: Requirement Input, receiving natural language test requirements, business rules, test data, and environment information; S20: Element collection, which records the webpage operation process and extracts multimodal description information of page elements based on fixed rules; Specifically, the system invokes the click, input, selection, hover, submit, and / or jump operations performed by testers in the browser to obtain the initial operation sequence and initial element location information; during or after recording, it performs a DOM scan on the target page; and it associates and stores element description information with page identifiers, business scenario identifiers, and test step numbers to form an element knowledge base, which is then queried during script generation and exception repair.

[0015] The element description information includes: Structural features: DOM path, tag name, attribute set; Semantic features: button text, label name, business semantic description; Visual features: element screenshots, region coordinates, color block distribution, layout position; Behavioral characteristics: types of executable actions, preconditions for the actions, and page responses after the actions; Contextual features: the URL of the page containing the element, the page title, the module it belongs to, and its relationship with adjacent controls.

[0016] S30: Test intent parsing, which converts natural language test requirements into a structured intermediate representation of test intent; the intermediate representation of test intent is represented using JSON, YAML, or an internal DSL structure; by first forming a platform-independent semantic layer expression, it then enters the specific code generation stage.

[0017] S40: Script generation, generates test scripts based on intermediate representations of test intent and element description information; Specifically, during script generation, the system generates a primary positioning strategy and at least one backup positioning strategy for each critical operation; the script automatically inserts the following engineering control logic for each step: Page loading waiting logic; element visibility / clickability / inputability detection; timeout control logic; exception handling logic; conditional retry logic; page redirection and pop-up handling logic; test data preparation and cleanup logic; log and screenshot collection logic.

[0018] It should be noted that this embodiment uses Robot Framework + Playwright as the execution framework. However, other automated execution engines such as Selenium, Puppeteer, and Cypress can also be used. As long as the overall approach of natural language parsing, multimodal modeling of elements, adaptive repair during execution, and semantic judgment of results is still adopted, it is not limited to the description in this embodiment.

[0019] S50: Executes monitoring, executes generated scripts, and collects exception, DOM snapshot, page screenshot, network log, console log and / or runtime trajectory information in real time; Specifically, during monitoring, an execution context object is created for each step. The execution context object includes: current step number, current page URL, current page DOM snapshot, current page or partial screenshot, console log, network request and response summary, currently used positioning strategy, historical executed action sequence, current browser state information and / or exception type and exception stack. When the system detects that an element cannot be found, the element is not clickable, the page times out, the assertion fails, the redirect is abnormal, or the data status is abnormal, the execution monitoring module will send the corresponding execution context. S60: Intelligent diagnosis and repair. When execution fails, it analyzes the cause of failure, identifies candidate target elements, generates alternative positioning strategies or corrected test steps, and triggers re-execution. Specifically, the steps involved in intelligent diagnosis and repair include: Failure classification categorizes the causes of failure based on exception information and execution context; When the failure reason is identified as element positioning failure or page structure change, the candidate element retrieval unit retrieves a set of candidate elements from the current page based on the element description information of the original target element. Calculate a comprehensive matching score for each candidate element; when the comprehensive matching score of a candidate element is higher than a preset threshold, the system selects that candidate element to generate an alternative positioning strategy; when the scores of multiple candidate elements are close, a multi-round verification strategy is adopted to determine the final element. Based on the identified candidate elements, the corrected test steps are output, and the operation code or keywords for the current step are regenerated. The corrected steps are sent back to the execution monitoring module to run again. If the retry is successful, the system records the repair path, score and success conditions. If it fails, it proceeds to the next round of diagnosis or reporting.

[0020] It is understood that this solution can perform repairs based solely on DOM structure features, attribute features, and text features, without enabling visual features. However, in scenarios where the interface changes frequently, local visual matching weights, historical repair weights, or form semantic weights can be further increased, which are not limited to the descriptions in this embodiment.

[0021] S70: Intelligent result judgment, based on page screenshots, DOM state, text semantics, network response and expected description, performs semantic-level pass / fail judgment on test results; It should be noted that the intelligent result judgment module is no longer limited to hard-coded assertions, but instead performs intelligent judgments by combining expected semantics and actual execution results. The intelligent result judgment module first converts the expected description into computable semantic judgment conditions. Then, it extracts the corresponding state features from the actual page and, through a semantic matching model, visual understanding model, or rule enhancement model, provides a conclusion of pass, failure, or pending manual confirmation, while also outputting the basis for the conclusion.

[0022] Compared to traditional precise assertions, the intelligent judgment mechanism of this invention can handle ambiguous but real business verification needs, thereby improving the coverage of automated testing.

[0023] It is understood that in this embodiment, the intelligent judgment of results can adopt a hybrid judgment mode of "rule engine + large model", that is, first complete the deterministic screening based on rules, and then perform semantic judgment on complex results that cannot be covered by the rules. However, it can also adopt a pure visual model, a pure NLP model or a multi-model integration method, which is not limited to the description in this embodiment.

[0024] S80: Knowledge feedback, saving repair results, successful location solutions, failed samples and manual confirmation conclusions.

[0025] The knowledge feedback module persistently stores information, which can be used for manual review later. When executing similar test scenarios in the future, the system can prioritize referencing historical successful repair samples and historical judgment samples to improve the accuracy of script generation, repair speed, and consistency of judgment.

[0026] It should be noted that knowledge feedback can store historical execution data in relational databases, vector databases, graph databases, or file systems. It can also build knowledge graphs based on scenarios, pages, elements, and failure types according to business needs to enhance subsequent retrieval and repair.

[0027] This embodiment connects element acquisition, script generation, execution monitoring, anomaly repair, and result judgment into a unified closed loop, significantly improving the overall stability of the automated testing chain. By integrating DOM, text semantics, visual regions, and action context for element repair, it effectively overcomes the problem of easy failure in traditional static positioning. A "two-stage generation" approach is adopted, first forming an intermediate representation of the test intent, and then generating the specific script, reducing script instability caused by natural language ambiguity. Waiting, retrying, anomaly handling, and context control logic are automatically injected into the script, improving the executability of the generated script in a real-world environment. Replacing simple hard-coded assertions with semantic-level result judgment can cover more complex scenarios such as visual verification, interactive state verification, and business state verification.

[0028] In another embodiment of the present invention, the method includes the following steps: S101: Receives natural language testing requirements, testing environment information, and testing data.

[0029] S102: Record the user's operation process on the target webpage using Playwright Codegen to obtain the initial test steps and initial element location information; at the same time, scan the target page to extract the structural features, semantic features, visual features and contextual features of candidate elements to form element description information.

[0030] S103: Convert the natural language test requirements into an intermediate representation of test intent, and generate RobotFramework+Playwright test scripts based on the intermediate representation of test intent and element description information.

[0031] S104: Execute the test script and collect page screenshots, DOM snapshots, logs, network responses, and exception information in real time.

[0032] S105: When a step execution failure is detected, the failure type is determined based on the exception information and execution context; if the failure type is element location failure or page structure change, the candidate element set is retrieved from the current page.

[0033] S106: Based on the attribute similarity, DOM structure similarity, text semantic similarity, visual region similarity, and action matching degree between the original target element and the candidate elements, calculate the comprehensive matching score, select the target candidate element, and generate an alternative positioning strategy.

[0034] S107: Repair and re-execute the failed steps using an alternative positioning strategy; if successful, save the repair results; if unsuccessful, continue diagnosis, switch strategies, or initiate manual confirmation.

[0035] S108: After the test steps are completed, based on the expected description in natural language, page screenshots, DOM state and network response, perform semantic-level intelligent judgment on the test results, and output the pass / fail conclusion and the basis for the conclusion.

[0036] S109: Write the script generation results, execution logs, repair paths, and result judgment conclusions into the knowledge base for reuse in subsequent tasks.

[0037] The above technical solution addresses the problems of fragile webpage automated testing scripts, high maintenance costs, and rigid result judgment by constructing a closed-loop processing chain of "element collection, script generation, execution monitoring, intelligent diagnosis, adaptive repair, result semantic judgment, and knowledge feedback."

[0038] See attached document Figures 2 to 5 As shown, the present invention also proposes an AI-based automated testing system, which includes at least the following modules: 1. Requirement Input Module: This module receives natural language testing requirements, business rules, test data, and environment information.

[0039] 2. Element Acquisition Module: This module records webpage operations using Python and Playwright Codegen, and extracts multimodal description information of page elements based on fixed rules.

[0040] The element acquisition module includes a recording unit, a rule extraction unit, a multimodal modeling unit, and an element library management unit.

[0041] The recording unit calls Playwright Codegen to record the actions of testers in the browser, such as clicking, inputting, selecting, hovering, submitting, and jumping, to obtain the initial operation sequence and initial element positioning information.

[0042] During or after recording, the rule extraction unit performs a DOM scan on the target page to extract one or more of the following attributes from candidate elements: Element tag type; id, name, class, data-* attributes; XPath, CSS path, parent-child hierarchy; visible text, placeholder text, tooltip text, aria attribute; element coordinates, width, height, and hierarchy order in the viewport; title, description text, and tag text of the element's adjacent area; page state changes before and after triggering the element.

[0043] The multimodal modeling unit constructs element description information from the above information. Preferably, the element description information includes at least: Structural features: DOM path, tag name, attribute set; Semantic features: button text, label name, and business semantic description.

[0044] Visual features: element screenshots, region coordinates, color block distribution, layout position.

[0045] Behavioral characteristics: type of action that can be performed, preconditions for the action, and page response after the action.

[0046] Contextual features: the URL of the page containing the element, the page title, the module it belongs to, and its relationship with adjacent controls.

[0047] The element library management unit associates and stores element description information with page identifiers, business scenario identifiers, and test step numbers, forming an element knowledge base that can be queried during subsequent script generation and exception repair.

[0048] 3. Test intent parsing module, used to convert natural language test requirements into a structured intermediate representation of test intent.

[0049] The test intent parsing module receives natural language requirements, such as "Log in to the system with a valid account, enter the order list page, filter orders to be paid, and after clicking the first order, the order details should be displayed and the payment button should be clickable," and parses it into a structured intermediate representation of the test intent. This intermediate representation includes at least: Test objectives; prerequisites; environmental constraints; test data constraints; sequence of operation steps; action type of each step; semantics of the target object of each step; checkpoints and pass criteria; remedial actions or termination conditions after failure.

[0050] Preferably, the intermediate representation uses JSON, YAML, or an internal DSL structure. By first forming a platform-independent semantic layer expression and then proceeding to the specific code generation stage, the impact of natural language ambiguity on the stability of the final script can be reduced.

[0051] 4. Script generation module, used to generate RobotFramework+Playwright test scripts based on intermediate representations of test intent and element description information.

[0052] The script generation module generates RobotFramework+Playwright test scripts based on the intermediate representation of the test intent, the element description information in the element library, and the target execution framework constraints.

[0053] At this stage, the system does not hardcode the script with a single positioning value. Instead, it generates a primary positioning strategy and at least one backup positioning strategy for each critical operation. Preferably, one or more of the following engineering control logics are automatically inserted into the script for each step: Page loading waiting logic; element visibility / clickability / inputability detection; timeout control logic; exception handling logic; conditional retry logic; page redirection and pop-up handling logic; test data preparation and cleanup logic; log and screenshot collection logic.

[0054] Through a two-stage generation mechanism, this invention can simultaneously ensure the business semantic integrity of the test steps and the engineering stability of the execution-level scripts.

[0055] 5. The execution monitoring module is used to execute the generated scripts and collect information such as exceptions, DOM snapshots, page screenshots, network logs, console logs, and execution trajectory information in real time.

[0056] During the execution of the test script, the monitoring module establishes an execution context object for each step. The execution context object includes at least: Current step number; current page URL; current page DOM snapshot; current page screenshot or partial screenshot; console log; network request and response summary; current location strategy; historical sequence of executed actions; current browser state information; exception type and exception stack.

[0057] When the system detects that an element cannot be found, an element is not clickable, a page times out, an assertion fails, a redirect is abnormal, or a data status is abnormal, the execution monitoring module sends the corresponding execution context to the intelligent diagnosis and repair module.

[0058] 6. Intelligent diagnosis and repair module, used to analyze the cause of failure when execution fails, identify candidate target elements, generate alternative positioning strategies or corrected test steps, and trigger re-execution.

[0059] The intelligent diagnosis and repair module includes a failure classification unit, a candidate element retrieval unit, a candidate strategy scoring unit, and a repair execution unit.

[0060] Failure Classification: Based on exception information and execution context, the failure classification unit categorizes failure reasons into one or more of the following categories: Element positioning failure; page not fully loaded or asynchronous rendering not completed; page structure changed but business process unchanged; test data does not meet expected conditions; network error or API response error; abnormal permissions, login status or environment configuration; inconsistency between expected rules and actual page design.

[0061] Candidate element retrieval: When the failure is identified as element location failure or page structure change, the candidate element retrieval unit retrieves a set of candidate elements from the current page based on the element description information of the original target element. The generation of the candidate set can incorporate the following information: The node in the current page's DOM that is closest to the original element's attributes; the node that is closest to the original element's text semantics; the node that is visually located near the original area; and the interactive node that matches the original step's action type.

[0062] Nodes that are identical to the original target in adjacent labels, headings, form groups, table headers, or list structures.

[0063] Candidate Strategy Scoring: The candidate strategy scoring unit calculates a comprehensive matching score for each candidate element. This score can be obtained by weighting one or more of the following dimensions: Attribute similarity; DOM structure similarity; text semantic similarity; visual region similarity; action executability matching degree; page context consistency; history restoration success rate.

[0064] Preferably, when the overall matching score of a candidate element is higher than a preset threshold, the system selects the candidate element to generate an alternative positioning strategy; when the scores of multiple candidate elements are close, the system can adopt a multi-round verification strategy, including trial clicks, trial inputs, and local verification, to determine the final element.

[0065] Repair execution: The repair execution unit outputs the corrected test steps based on the determined candidate elements. Repair methods include, but are not limited to: Replace the original positioning expression; add waiting conditions; rewrite the single positioning as a combined positioning; adjust the execution order of steps; insert page stability verification steps.

[0066] Regenerate the Playwright action code or RobotFramework keyword for the current step.

[0067] The repaired steps are sent back to the execution monitoring module for re-run. If the retry is successful, the system records the repair path, score, and success conditions; if it fails, it proceeds to the next round of diagnosis or reports for manual confirmation.

[0068] Thus, this invention forms a closed-loop feedback mechanism of "execution failure, context collection, intelligent diagnosis, candidate retrieval, location correction, and re-execution".

[0069] 7. The intelligent result judgment module is used to perform semantic-level pass / fail judgment on test results based on page screenshots, DOM state, text semantics, network response, and expected description.

[0070] The intelligent judgment module is no longer limited to hard-coded assertions, but makes intelligent judgments by combining expected semantics and actual execution results.

[0071] Specifically, the result intelligent judgment module receives the following inputs: Natural language expected description; checkpoint definition in the intermediate representation of test intent; current page screenshot, partial screenshot, or comparison results of previous and next frames; page DOM state; visible text, OCR text, and control state; relevant network request and response summary; behavioral trajectory generated during execution.

[0072] The intelligent judgment module first converts the expected description into computable semantic judgment conditions, for example: Does the specified function entry point appear? Is the target data displayed in the expected list format? Is a button clickable? Does the pop-up window appear successfully and contain the core prompt information? Does the page enter the target business state after submission?

[0073] Subsequently, the intelligent result judgment module extracts the corresponding state features from the actual page and, through a semantic matching model, visual understanding model, or rule enhancement model, provides a conclusion of pass, failure, or pending manual confirmation, while also outputting the basis for the conclusion. This basis may include: The text fragments on the page that were hit; the state of the controls that were identified; image evidence that matched the target area; and the consistency between the fields returned by the API and the page presentation.

[0074] Compared to traditional precise assertions, the intelligent judgment mechanism of this invention can handle ambiguous but real business verification needs, thereby improving the coverage of automated testing.

[0075] 8. Knowledge feedback module, used to save repair results, successful location solutions, failed samples and manual confirmation conclusions, for continuous optimization in subsequent generation and repair processes.

[0076] The knowledge feedback module will persistently store the following information: original test requirements and final script; element description information for each step; failure type and failure context; candidate element scores and final repair results; conclusions and basis for intelligent result judgment; and manual review results.

[0077] When performing similar test scenarios in the future, the system can prioritize referencing historical successful repair samples and historical judgment samples to improve the accuracy of script generation, repair speed, and consistency of judgment.

[0078] The modules described above can be deployed on the same server or distributed across test execution nodes, AI inference nodes, and management nodes. Modules interact with each other via API calls, message queues, or shared databases.

[0079] This invention integrates element acquisition, script generation, execution monitoring, anomaly repair, and result judgment into a unified closed loop, significantly improving the overall stability of the automated testing chain. By fusing DOM, text semantics, visual regions, and action context for element repair, it effectively overcomes the problem of easy failure in traditional static positioning. A two-stage generation approach is adopted, first forming an intermediate representation of the test intent, and then generating the specific script, reducing script instability caused by natural language ambiguity. Waiting, retrying, anomaly handling, and context control logic are automatically injected into the script, improving the executability of the generated script in a real-world environment. Semantic-level result judgment replaces simple hard-coded assertions, covering more complex scenarios such as visual verification, interactive state verification, and business state verification. A knowledge feedback mechanism reuses historical repair and judgment experience, continuously reducing script maintenance costs and improving long-term performance. It can be directly applied to enterprise-level regression testing, smoke testing, inspection testing, and low-code testing platforms, demonstrating strong practical value.

[0080] This application also provides an electronic device, such as... Figure 6 As shown, Figure 6 The illustrated electronic device 700 includes a processor 701 and a memory 703. The processor 701 and the memory 703 are connected, for example, via a bus 702. Optionally, the electronic device 700 may also include a transceiver 704. It should be noted that in practical applications, the transceiver 704 is not limited to one type, and the structure of this electronic device 700 does not constitute a limitation on the embodiments of this application.

[0081] Processor 701 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 701 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0082] Bus 702 may include a pathway for transmitting information between the aforementioned components. Bus 702 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 702 can be divided into address bus, data bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0083] The memory 703 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0084] The memory 703 is used to store application code that executes the solution of this application, and its execution is controlled by the processor 701. The processor 701 is used to execute the application code stored in the memory 703 to implement the content shown in the foregoing method embodiments.

[0085] Electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, PDAs (personal digital assistants), and PADs (tablet computers), as well as fixed terminals such as digital TVs and desktop computers. Servers can also be included. Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0086] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed in a computer, causes the computer to perform the AI-based automated testing method provided in the above embodiments.

[0087] It should be understood that although the steps in the flowcharts in the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise expressly stated herein, there is no strict order in which these steps are performed, and they may be performed in other orders.

[0088] The above are only some embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An AI-based automated testing method, characterized in that, The method includes: Input requirements include natural language testing requirements, business rules, test data, and environment information. Element collection involves recording the webpage operation process and extracting multimodal description information of page elements based on fixed rules. Test intent parsing converts natural language testing requirements into a structured intermediate representation of test intent; Script generation: Generates test scripts based on intermediate representations of test intent and element description information; Perform monitoring, execute the generated scripts, and collect anomaly, DOM snapshots, page screenshots, network logs, console logs, and / or runtime trajectory information in real time; Intelligent diagnosis and repair analyze the cause of failure when execution fails, identify candidate target elements, generate alternative positioning strategies or corrected test steps, and trigger re-execution.

2. The AI-based automated testing method according to claim 1, characterized in that, Following diagnosis and repair, the following is also included: The results are intelligently judged based on page screenshots, DOM state, text semantics, network response, and expected descriptions, and the test results are judged semantically as pass / fail. Knowledge feedback includes saving repair results, successful location solutions, failed samples, and confirmation conclusions.

3. The AI-based automated testing method according to claim 1, characterized in that, When collecting elements: Invoke the click, input, selection, hover, submit and / or jump operations performed by the tester in the browser to obtain the initial operation sequence and initial element positioning information; During or after recording, perform a DOM scan on the target page; The element description information is associated with and stored with page identifiers, business scenario identifiers, and test step numbers to form an element knowledge base, which can be queried during script generation and exception repair.

4. The AI-based automated testing method according to claim 3, characterized in that, in, Element description information should include at least: Structural features: DOM path, tag name, attribute set; Semantic features: button text, label name, business semantic description; Visual features: element screenshots, region coordinates, color block distribution, layout position; Behavioral characteristics: types of executable actions, preconditions for the actions, and page responses after the actions; Contextual features: the URL of the page containing the element, the page title, the module it belongs to, and its relationship with adjacent controls.

5. The AI-based automated testing method according to claim 1, characterized in that, The test intent is represented using JSON, YAML, or an internal DSL structure; a platform-independent semantic layer expression is first formed before proceeding to the specific code generation stage.

6. The AI-based automated testing method according to claim 1, characterized in that, When the script is generated, the system generates a primary positioning strategy and at least one backup positioning strategy for each critical operation; wherein, the script automatically inserts one or more of the following engineering control logics for each step: Page load waiting logic; Element visibility / clickability / inputability detection; Timeout control logic; Exception handling logic; Conditional retry logic; Page redirection and pop-up handling logic; Test data preparation and cleaning logic; Log and screenshot collection logic.

7. The AI-based automated testing method according to claim 1, characterized in that, During monitoring, an execution context object is created for each step, and the execution context object includes: Current step number, current page URL, current page DOM snapshot, current page or partial screenshot, console log, network request and response summary, current location strategy, historical sequence of executed actions, current browser state information and / or exception type and exception stack; When the system detects that an element cannot be found, the element is not clickable, the page times out, the assertion fails, the redirect is abnormal, or the data status is abnormal, the execution monitoring module will send the corresponding execution context.

8. The AI-based automated testing method according to claim 2, characterized in that, When performing intelligent diagnosis and repair, the following are included: Failure classification categorizes the causes of failure based on exception information and execution context; When the failure reason is identified as element positioning failure or page structure change, the candidate element retrieval unit retrieves a set of candidate elements from the current page based on the element description information of the original target element. Calculate a comprehensive matching score for each candidate element; when the comprehensive matching score of a candidate element is higher than a preset threshold, the system selects that candidate element to generate an alternative positioning strategy; when the scores of multiple candidate elements are close, a multi-round verification strategy is adopted to determine the final element. Based on the identified candidate elements, the corrected test steps are output, and the operation code or keywords for the current step are regenerated. The corrected steps are sent back to the execution monitoring module to run again. If the retry is successful, the system records the repair path, score and success conditions. If it fails, it proceeds to the next round of diagnosis or reporting.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; Memory; At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, said at least one application being configured to: execute the AI-based automated testing method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed in a computer, the computer is instructed to perform the AI-based automated testing method according to any one of claims 1 to 8.