A UI automatic test script self-healing method and device based on multi-level business intention reasoning, an electronic device and a readable storage medium

Abstract

This invention provides a self-healing method, apparatus, electronic device, and readable storage medium for UI automated test scripts based on multi-level business intent reasoning, relating to the field of software automated testing technology. The method includes: during test execution, detecting whether the locator corresponding to the currently executed operation is invalid; when the locator is invalid, performing multi-level intent reasoning on each candidate element using a knowledge graph, wherein the multi-level intent reasoning includes at least: an element semantic matching level matching the target description of the operation to be executed, a page context matching level matching the historical context, and a business process matching level verifying business process constraints in the knowledge graph, and generating a comprehensive matching score based on the matching results of the three levels; and performing graded handling on the invalid locator according to the comprehensive matching score. This invention, through the business process matching level, ensures that the repaired element is consistent with the original business intent, rather than merely textually similar.

Description

Technical Field

[0001] This invention relates to the field of software automation testing technology, and in particular to a self-healing method, apparatus, electronic device, and readable storage medium for UI automation test scripts based on multi-level business intent reasoning. Background Technology

[0002] UI automated testing is a core method for software quality assurance. Test scripts identify page elements and execute operations through locators. When the front-end page of the system under test is redesigned, the locators frequently fail, causing widespread errors in the test scripts, requiring manual fixing one by one.

[0003] Existing self-healing technologies can be mainly divided into the following three categories: (1) Multi-attribute backup location method: Pre-store multiple sets of attributes (ID, CSS selector, XPath, etc.) for each element, and try backup locators in turn when the main locator fails.

[0004] Drawback: It only solves the problem of "whether the element can be found", without verifying whether the found element carries the same business function. When page reconstruction causes a large-scale change in element attributes, the backup locator may point to an element with completely different functions.

[0005] (2) DOM structure similarity matching method: By comparing structural features such as the element's hierarchical path and sibling relationships in the DOM tree, the most similar candidate elements are found.

[0006] Drawback: Similar DOM structure does not equate to identical business functionality. For example, a "Login" button and a "Register" button may have almost identical structural features in the DOM (siblings, of the same type, and sharing the same parent node), but their business intents are completely different.

[0007] (3) Visual feature matching method: The appearance features of elements can be identified through screenshot comparison or visual AI.

[0008] Drawbacks: Visual similarity cannot guarantee business consistency and is sensitive to changes in page style.

[0009] The common drawback of the above methods is that they operate at the "element location" level and lack the ability to understand the "business intent." This leads to a critical problem—false positive repair: the system successfully locates an element and completes the repair, and the test case shows that it has passed, but the actual element operated on does not match the original business intent, making the test results unreliable.

[0010] Therefore, a self-healing method, device, electronic device, and readable storage medium for UI automation test scripts based on multi-level business intent reasoning are proposed. Summary of the Invention

[0011] This specification provides a self-healing method, device, electronic device, and readable storage medium for UI automation test scripts based on multi-level business intent reasoning. By matching the business process hierarchy, it ensures that the repaired elements match the original business intent. Figure 1 It is not merely textual similarity, but rather a connection between the two.

[0012] This specification provides a self-healing method for UI automation test scripts based on multi-level business intent reasoning, including: Offline construction of a knowledge graph to represent business intent, page elements and their relationships; During test execution, check whether the locator corresponding to the current operation to be executed has failed; When the locator fails, candidate elements are filtered from the current page, and multi-level intent reasoning is performed on each candidate element using the knowledge graph. The multi-level intent reasoning includes at least: element semantic matching level matching with the target description of the operation to be performed, page context matching level matching with the historical context, and business process matching level verifying business process constraints in the knowledge graph. A comprehensive matching score is generated based on the matching results of the three levels. Based on the comprehensive matching score, the failed locator is subjected to tiered processing, which includes at least: when the comprehensive matching score meets the preset high confidence condition, the locator is automatically updated to the attribute of the candidate element with the highest score.

[0013] Optionally, the offline construction of a knowledge graph representing business intent, page elements, and their relationships includes: Analyze historical test cases to extract structured information including operation type, target element description, business intent tag, and expected result; Based on the structured information, a knowledge graph is constructed that includes intent nodes, page nodes, element nodes, and edges used to represent the relationships between nodes. An embedding vector that integrates textual semantic information and graph structure information is generated for the intent node, and the embedding vector is stored in a vector database.

[0014] Optionally, generating an embedding vector that fuses textual semantic information and graph structure information for the intent node includes: The business intent tag and its associated target element description are processed using a pre-trained text embedding model to obtain a text embedding vector; The graph neural network is used to process the neighbor information of the intent node in the knowledge graph to obtain the graph structure embedding vector; The text embedding vector is fused with the graph structure embedding vector to obtain a fused embedding vector.

[0015] Optionally, the element semantic matching level that matches the target description of the operation to be executed, the page context matching level that matches the historical context, and the business process matching level that verifies business process constraints in the knowledge graph, and the comprehensive matching score generated based on the matching results of the three levels, include: Query the business intent tag corresponding to the current operation in the knowledge graph to obtain its preceding intent and coexisting intent; Determine whether all the preceding intentions of the current operation have been satisfied; otherwise, directly determine that the business process matching score is the lowest value. If so, the compatibility between the intent associated with the candidate element and the coexisting intent is further evaluated to generate a business process consistency score.

[0016] Optional, including: When the accumulated relational data about the current business intent in the knowledge graph has not reached a preset threshold, a cold start strategy is adopted to calculate the consistency score of the business process: different preset initial values ​​are assigned according to the semantic or domain consistency between the candidate element's associated intent and the current operation intent; when the accumulated relational data reaches the preset threshold, the dynamic calculation mode based on graph relations is switched.

[0017] Optionally, the step of performing tiered processing on failed locators based on the comprehensive matching score includes: When the comprehensive matching score reaches or exceeds the first preset threshold, it is determined to be of high confidence and repair is automatically performed; When the overall matching score is between the second preset threshold and the first preset threshold, it is determined to be of medium confidence, the test is paused and a repair suggestion for manual confirmation is output; When the overall matching score is lower than the second preset threshold, it is determined to be of low confidence, the current operation is skipped and it is marked as requiring manual processing.

[0018] Optional, also includes: The execution results or manual confirmation results of each repair are collected as feedback data. The knowledge graph is incrementally updated using the feedback data, and incremental training of the model used in the multi-level intent reasoning is triggered when the accumulated amount of feedback data reaches a preset amount.

[0019] This specification provides a self-healing device for UI automation test scripts based on multi-level business intent reasoning, including: The knowledge graph construction module is used to build offline knowledge graphs that represent business intents, page elements and their relationships. The locator monitoring module is used to detect whether the locator corresponding to the current operation to be performed is malfunctioning during the test execution process; The intent reasoning module is used to filter candidate elements from the current page when the locator fails, and to perform multi-level intent reasoning on each candidate element using the knowledge graph. The multi-level intent reasoning includes at least: an element semantic matching level that matches the target description of the operation to be performed, a page context matching level that matches the historical context, and a business process matching level that verifies the business process constraints in the knowledge graph. A comprehensive matching score is generated based on the matching results of the three levels. The feedback update module is used to perform graded processing on the failed locators according to the comprehensive matching score. The graded processing includes at least: when the comprehensive matching score meets the preset high confidence condition, automatically updating the locator to the attribute of the candidate element with the highest score.

[0020] Optionally, the atlas construction module includes: Analyze historical test cases to extract structured information including operation type, target element description, business intent tag, and expected result; Based on the structured information, a knowledge graph is constructed that includes intent nodes, page nodes, element nodes, and edges used to represent the relationships between nodes. An embedding vector that integrates textual semantic information and graph structure information is generated for the intent node, and the embedding vector is stored in a vector database.

[0021] Optionally, generating an embedding vector that fuses textual semantic information and graph structure information for the intent node includes: The business intent tag and its associated target element description are processed using a pre-trained text embedding model to obtain a text embedding vector; The graph neural network is used to process the neighbor information of the intent node in the knowledge graph to obtain the graph structure embedding vector; The text embedding vector is fused with the graph structure embedding vector to obtain a fused embedding vector.

[0022] Optionally, the intent reasoning module includes: Query the business intent tag corresponding to the current operation in the knowledge graph to obtain its preceding intent and coexisting intent; Determine whether all the preceding intentions of the current operation have been satisfied; otherwise, directly determine that the business process matching score is the lowest value. If so, the compatibility between the intent associated with the candidate element and the coexisting intent is further evaluated to generate a business process consistency score.

[0023] Optional, including: When the accumulated relational data about the current business intent in the knowledge graph has not reached a preset threshold, a cold start strategy is adopted to calculate the consistency score of the business process: different preset initial values ​​are assigned according to the semantic or domain consistency between the candidate element's associated intent and the current operation intent; when the accumulated relational data reaches the preset threshold, the dynamic calculation mode based on graph relations is switched.

[0024] Optionally, the feedback update module includes: When the comprehensive matching score reaches or exceeds the first preset threshold, it is determined to be of high confidence and repair is automatically performed; When the overall matching score is between the second preset threshold and the first preset threshold, it is determined to be of medium confidence, the test is paused and a repair suggestion for manual confirmation is output; When the overall matching score is lower than the second preset threshold, it is determined to be of low confidence, the current operation is skipped and it is marked as requiring manual processing.

[0025] Optional, also includes: The execution results or manual confirmation results of each repair are collected as feedback data. The knowledge graph is incrementally updated using the feedback data, and incremental training of the model used in the multi-level intent reasoning is triggered when the accumulated amount of feedback data reaches a preset amount.

[0026] This specification also provides an electronic device, wherein the electronic device includes: A processor; and a memory storing computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.

[0027] This specification also provides a computer-readable storage medium that stores one or more programs that, when executed by a processor, implement any of the methods described above.

[0028] In this invention, by matching business processes at different levels, it is ensured that the repaired elements are consistent with the original business intent. Figure 1 It is consistent with, rather than just similar in text; the three-level scoring mechanism makes each repair decision traceable and explainable, and facilitates manual review; the human-machine collaborative feedback mechanism enables the system to continuously improve its accuracy over time; by changing the knowledge graph and retraining the weights, it can be adapted to different business fields (e-commerce, finance, healthcare, etc.). Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 A schematic diagram illustrating the principle of a self-healing method for UI automated test scripts based on multi-level business intent reasoning, provided in the embodiments of this specification; Figure 2 A schematic diagram of the structure of a UI automated test script self-healing device based on multi-level business intent reasoning, provided as an embodiment of this specification; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification; Figure 4 This is a schematic diagram of a computer-readable medium provided for embodiments of this specification. Detailed Implementation

[0031] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.

[0032] The following is in conjunction with the appendix Figures 1-4 Exemplary embodiments of the invention will be described more fully here. However, exemplary embodiments can be implemented in many forms and should not be construed as limiting the invention to the embodiments set forth herein. Rather, these exemplary embodiments are provided to make the invention more comprehensive and complete, and to facilitate a full communication of the inventive concept to those skilled in the art. The same reference numerals in the figures denote the same or similar elements, components, or parts, and therefore repeated descriptions of them are omitted.

[0033] Subject to the technical concept of this invention, the features, structures, characteristics or other details described in a particular embodiment may be combined in one or more other embodiments in a suitable manner.

[0034] In the description of specific embodiments, the features, structures, characteristics, or other details described in this invention are intended to enable those skilled in the art to fully understand the embodiments. However, it is not excluded that those skilled in the art can practice the technical solutions of this invention without one or more of the specific features, structures, characteristics, or other details.

[0035] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0036] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0037] The terms “and / or” or “and / or” include all combinations of any one or more of the listed items.

[0038] Figure 1 This specification provides a schematic diagram illustrating the principle of a self-healing method for UI automation test scripts based on multi-level business intent reasoning, as illustrated in the embodiments of this specification. The method may include: S110: Offline construction of a knowledge graph to represent business intent, page elements and their relationships; Optionally, S110 includes: Analyze historical test cases to extract structured information including operation type, target element description, business intent tag, and expected result; Based on the structured information, a knowledge graph is constructed that includes intent nodes, page nodes, element nodes, and edges used to represent the relationships between nodes. An embedding vector that integrates textual semantic information and graph structure information is generated for the intent node, and the embedding vector is stored in a vector database.

[0039] In the specific implementation of this specification, the historical test case set is structured and parsed to extract a quadruple from each test step: T=(action,target desc intent label ,expected result ); Where T represents a structured representation of a single test step, containing the following four fields: action: type of operation (click, input, select, hover, etc.); target desc : Natural language description of the target element (such as "login button", "username input box"); intent labelBusiness intent tags, using a three-level naming convention of "domain.action.object" (e.g., ecommerce.auth.login, finance.payment.confirm). expected result Expected result description (e.g., "page redirects to homepage", "payment confirmation pops up").

[0040] The annotation method for intent_label: During the initial deployment, the testing team should manually annotate the seed set based on the business documentation (recommended ≥500 entries). It will be automatically expanded through a feedback mechanism.

[0041] A business intent knowledge graph G=(V,E) is constructed based on the extracted quadruples, where G represents the knowledge graph, V represents the set of all nodes in the graph, and E represents the set of all edges in the graph: Node V includes three types: Intent Node: Each unique intent label For each node, the attributes include intent description, business domain, and a list of associated operation types; Page Node: Each unique page identifier corresponds to one node, and its attributes include the page URL pattern, page title, and the business module to which it belongs; Element Node: Each history locator corresponds to one node, whose attributes include element text, element type, CSS class name, and relative position area on the page (a nine-grid area of ​​top / middle / bottom × left / middle / right).

[0042] Edge E contains four types of relations: BELONGS TO : Element node - Page node (the element belongs to a certain page); SERVES: Element node - Intent node (the element serves a certain business intent); PRECEDES: Intent Nodes – Intent Nodes (Intent A is a prerequisite step for Intent B, such as login PRECEDESview) order ); COEXISTS: Intent Node-Intent Node (Two intents coexist on the same page, such as add_to_cartCOEXISTSview_product_detail).

[0043] Optionally, generating an embedding vector that fuses textual semantic information and graph structure information for the intent node includes: The business intent tag and its associated target element description are processed using a pre-trained text embedding model to obtain a text embedding vector; The graph neural network is used to process the neighbor information of the intent node in the knowledge graph to obtain the graph structure embedding vector; The text embedding vector is fused with the graph structure embedding vector to obtain a fused embedding vector.

[0044] In the specific implementation of this specification, the embedding vector is calculated for each intent node. Specific method: (a) Text embedding: embedding intent label (Business intent tag) and its associated target desc (Target element description) The set is concatenated into a text sequence, which is then input into a large-scale pre-trained text embedding model (preferably GTE-Qwen2-7B-instruct, which is based on the Qwen2 large language model architecture and is trained through multi-stage contrastive learning, significantly outperforming traditional BERT-like models in semantic understanding and cross-lingual text representation) to obtain text vectors: v text ∈R d ; Where v text For text embedding vectors, R d Let represent a d-dimensional real vector space, where d = 3584; (b) Graph Structure Embedding: Run a Graph Attention Network (GAT, 2 layers, 4 attention heads per layer) on the knowledge graph to aggregate the neighbor information of each intent node and obtain the structure vector v. graph ∈R d (where v) graph (This is a graph structure embedding vector with the same dimensions as the text embedding vector). The message passing formula for GAT is as follows: ; in: Let i be the hidden state component of node i at layer l+1 (i.e., the node feature representation after aggregation in the current layer). Let be the hidden state component of node j in the l-th layer (i.e., the feature representation of node j in the previous layer). The activation function is a non-linear activation function (preferably the ELU activation function). Let i be the set of neighboring nodes of node i in the knowledge graph; Let be the attention coefficient of node j to node i in the l-th layer, representing the importance of neighbor node j to the current node i; is the learnable weight matrix of the l-th layer, used to perform linear transformation on the node features.

[0045] ; in, This is a learnable attention parameter vector; This is the transpose of vector a; [·‖·] represents the vector concatenation operation, which connects two vectors end to end; It is a linear rectified activation function with leakage (the slope of the negative half-axis is 0.2). This is the result of applying a linear transformation to the feature vector of node i; , These are the results after applying the same linear transformation to the feature vectors of neighboring nodes j and k, respectively. The summation in the denominator iterates through all neighboring nodes k of node i, implementing softmax normalization to ensure that the sum of the attention coefficients of all neighbors is 1.

[0046] (c) Fusion Embedding: Text embedding and graph structure embedding are concatenated and then dimensionality reduced using a fully connected layer. v intent =FC([v text ;v graph ])∈R d ; in: v intent The embedding vector is fused to the final intent; FC stands for Fully Connected Layer, which is used to map the concatenated high-dimensional vector back to d-dimensional space; [v text ;v graph ] indicates embedding the text into the vector v text And graph structure embedding vector v graph To splice; R d Let represent a d-dimensional real vector space (d=3584).

[0047] All the embedding vectors obtained from the above calculations (including the intent embedding vector v) intentElement semantic attribute embedding vectors and page context feature embedding vectors are uniformly stored in a vector database (Milvus is preferred, but Qdrant, Pinecone, and similar systems can also be used). Specific storage scheme: (a) Collection partitioning: intent embeddings Set: The fused embedding vector v of the stored intent node intent The primary key is intent label Additional fields include the business domain to which it belongs and a list of associated operation types; element embeddings Collection: Stores semantic attribute embedding vectors of historical elements. The primary key is the element's unique identifier (page URL + locator), and additional fields include element text, element type, and location region. context embeddings Collection: Stores embedded vectors of page context features. The primary key is the page identifier, and additional fields include URL pattern and the business module to which it belongs.

[0048] (b) Indexing strategy: IVF is adopted. FLAT The index (inverted file + precise distance calculation) and the nlist parameter are set according to the data size (default nlist=128) to achieve a balance between recall precision and retrieval speed.

[0049] (c) Retrieval process during online inference: When the self-healing process is triggered, the intent inference module uses the real-time embedding vector of the candidate element as the query vector and performs an approximate nearest neighbor (ANN) retrieval in the vector database. k =10, the search results serve as input for multi-level intent reasoning. Compared to full traversal computation, this method can reduce the search latency from seconds to milliseconds when dealing with elements in the tens of thousands or more.

[0050] S120: During test execution, check whether the locator corresponding to the current operation to be executed has failed; S130: When the locator fails, candidate elements are filtered from the current page, and multi-level intent reasoning is performed on each candidate element using the knowledge graph. The multi-level intent reasoning includes at least: element semantic matching level matching with the target description of the operation to be performed, page context matching level matching with the historical context, and business process matching level verifying business process constraints in the knowledge graph. A comprehensive matching score is generated based on the matching results of the three levels. In the specific implementation of this specification, during test execution, when the locator of a certain test step cannot find a matching element in the current page DOM, a self-healing process is triggered. The specific detection conditions are described in Chinese as follows: When the locator searches for the target element in the Document Object Model (DOM) of the current page and returns an empty result (i.e., no matching element was found), and the accumulated number of retries is greater than or equal to the maximum number of retries threshold, the locator is determined to have failed, triggering the self-healing process.

[0051] Expressed as a formula: Triggering self-healing (Locator search result = Φ) ∧ (Number of retries ≥ Maximum number of retries); in: The locator lookup result is the return value of the locator after performing an element search operation in the current page DOM. Φ indicates that no matching element was found. The number of retries is the cumulative number of retries performed since the current locator failed to find the target. The maximum number of retries is the system's preset retry limit, with a default value of 3; ∧ is the logical AND operator, indicating that two conditions must be met simultaneously.

[0052] The retry interval increases exponentially (1 second for the first retry, 2 seconds for the second, and 4 seconds for the third) to eliminate false positives caused by page loading delays.

[0053] Optionally, S130 includes: Query the business intent tag corresponding to the current operation in the knowledge graph to obtain its preceding intent and coexisting intent; Determine whether all the preceding intentions of the current operation have been satisfied; otherwise, directly determine that the business process matching score is the lowest value. If so, the compatibility between the intent associated with the candidate element and the coexisting intent is further evaluated to generate a business process consistency score.

[0054] Optional, including: When the accumulated relational data about the current business intent in the knowledge graph has not reached a preset threshold, a cold start strategy is adopted to calculate the consistency score of the business process: different preset initial values ​​are assigned according to the semantic or domain consistency between the candidate element's associated intent and the current operation intent; when the accumulated relational data reaches the preset threshold, the dynamic calculation mode based on graph relations is switched.

[0055] In the specific implementation of this specification, a candidate element set C is filtered from the current page DOM, with the following filtering conditions: Element visibility (display !=none, visibility !=hidden, opacity>0); The element is interactive (compatible with the original operation type, such as the click operation requiring the element to be clickable); Element type compatibility (e.g., if the original target is a button, the candidates include clickable elements such as button, a, and input[type=submit]).

[0056] For each candidate element c∈C (where C is the set of candidate elements after coarse screening, and c is a single candidate element in the set), perform three levels of intent matching computation: First level – element semantic matching (weight w1): Extract the text content, aria-label, placeholder, and other semantic attributes of candidate elements, and compare them with the target description of the original locator. desc Calculate text semantic similarity: score element (c)=cosine sim (encode(target desc ),encode(semantic attrs (c))).

[0057] in: score element (c) is the semantic matching score of candidate element c; c represents the candidate element currently being evaluated; cosine sim This is the cosine similarity calculation function, which measures the degree of similarity between two vectors in a direction, and its value ranges from [-1, 1]. `encode` is a text embedding encoding function that uses the GTE-Qwen2 text embedding model to convert the input text into a high-dimensional vector representation. target desc Natural language description of the target element in the original test steps (such as "login button"); semantic attrs (c) is the set of semantic attributes of candidate element c, including the visible text of the element, the aria-label attribute value, the placeholder attribute value, etc.

[0058] Second level – Page context matching (weight w2): Extract the contextual features of the page where the candidate element is located, including: The degree of matching between the page URL and the page nodes in the knowledge graph; The consistency between the position of candidate elements on the page and their historical location; The semantic identifier of the parent container of the candidate element (such as the aria-label and data-module attributes of the section).

[0059] ; in: The page context matching score for candidate element c; k is the index of the context feature, and all context feature dimensions are traversed (including URL matching degree, location region consistency, parent container semantic identifier matching degree, etc.). The sub-weight coefficient of the k-th context feature is obtained by performing logistic regression fitting on historical repair data, and represents the degree of contribution of this feature to context matching. The score is the matching degree score between candidate element c and historical records on the k-th context feature dimension, with a value range of [0,1]. This refers to the historical context feature record of the element corresponding to this test step stored in the knowledge graph.

[0060] Third level – Business process matching (weight w3): Query the business intent tag of the current test step in the knowledge graph. label ), obtain its prior intent (incoming edges of the PRECEDES relationship) and coexistence intent (COEXISTS relationship), and verify whether the candidate element meets the business process constraints: ; in: The business process matching score for candidate element c; The business process consistency score for candidate element c is calculated by checking whether the intent associated with the candidate element is compatible with the set of coexisting intents of the current step, and the value range is [0,1]. Initial value settings: During the system cold start phase (when there is not enough PRECEDES and COEXISTS relationship data in the knowledge graph). The following initialization strategy is adopted: When the intent associated with the candidate element label With the intent of the current step label When completely consistent, flow consistency Initial value = 0.90; When the intent associated with the candidate element labelWith the intent of the current step label When they belong to the same business domain (e.g., all belong to ecommerce.auth.*), flow consistency Initial value = 0.50; When the intent associated with the candidate element label With the intent of the current step label When belonging to different business domains, flow consistency Initial value = 0.10; When a candidate element has no associated intent record (new element) in the knowledge graph, flow consistency Initial value = 0.30.

[0061] As the system runs and feedback data accumulates, flow consistency The computation gradually transitions to dynamic computation based on graph relationships, with the transition condition being: the current intent label When the total number of associated PRECEDES and COEXISTS edges is ≥5, switch to dynamic calculation mode.

[0062] Overall rating: final score (c)=w1 score element (c)+w2 score context (c)+w3 score flow (c); in: final score (c) represents the final comprehensive matching score of candidate element c; w1 is the weight coefficient for the first level (element semantic matching); w2 is the weighting coefficient for the second level (page context matching); w3 is the weighting coefficient for the third level (business process matching); score element (c) score context (c) score flow (c) represents the matching scores of candidate element c at the three levels (see above for definition).

[0063] The method for determining weights w1, w2, and w3: The initial values ​​were determined by a grid search on the labeled dataset, with a search range of [0.1, 0.9], a step size of 0.1, and a constraint w1+w2+w3=1; In the experimental environment of this invention (e-commerce test scenario, 3000 labeled samples), the optimal weights are w1=0.35, w2=0.25, w3=0.40; The weights can be retrained and adjusted according to different business areas.

[0064] S140: Perform tiered processing on the failed locator according to the comprehensive matching score. The tiered processing includes at least: when the comprehensive matching score meets the preset high confidence condition, automatically update the locator with the attribute of the candidate element with the highest score.

[0065] Optionally, S140 includes: When the comprehensive matching score reaches or exceeds the first preset threshold, it is determined to be of high confidence and repair is automatically performed; When the overall matching score is between the second preset threshold and the first preset threshold, it is determined to be of medium confidence, the test is paused and a repair suggestion for manual confirmation is output; When the overall matching score is lower than the second preset threshold, it is determined to be of low confidence, the current operation is skipped and it is marked as requiring manual processing.

[0066] In the specific implementation of this specification, based on the final value of the highest-scoring candidate element... score (Final overall matching score) will be used for tiered processing: (a) High confidence (final) score ≥θ high , default θ high =0.85): Where θ high The high confidence threshold represents the lowest score at which the system considers the matching result to be highly reliable.

[0067] The system automatically performs a fix, updating the locator to the candidate element's attribute, and the test continues. Fix records are written to the log but the process is not interrupted.

[0068] (b) Medium confidence level (θ) low ≤final score <θ high , default θ low =0.55): Where θ low The low confidence threshold represents the lowest score at which the system believes the matching result requires human intervention.

[0069] Pause test execution and generate a remediation suggestion report, including: Screenshots of candidate elements are highlighted; Detailed matching scores for each of the three levels; Visualization of the relationships between related intent nodes in a knowledge graph; Recommended repair solutions and risk warnings.

[0070] Waiting for manual confirmation before continuing.

[0071] (c) Low confidence (final) score <θ low ): Mark this test step as "requires manual rewriting" and skip the current step to continue with subsequent steps (if the subsequent steps do not depend on the result of the current step).

[0072] threshold θ high (High confidence threshold) and θ low The low confidence threshold can be configured by the user according to the accuracy requirements of the business.

[0073] Optional, also includes: The execution results or manual confirmation results of each repair are collected as feedback data. The knowledge graph is incrementally updated using the feedback data, and incremental training of the model used in the multi-level intent reasoning is triggered when the accumulated amount of feedback data reaches a preset amount.

[0074] In the specific implementation of this specification, feedback data is collected for each repair (whether automatically or manually confirmed): Automatic repair: Whether all subsequent test steps pass (pass = positive sample, fail = negative sample). Manual confirmation: Testers' confirmation / rejection decisions and the annotation of rejection reasons.

[0075] Update the knowledge graph based on feedback data: Positive samples: If the repair involves new element-intent mappings, add SERVES edges to the graph; Negative samples: Reduce the weight of incorrect mapping relationships (soft deletion) and add failed cases to the negative sample training set; Add new intent: If an intent_label that does not exist in the graph is manually labeled, create a new intent node and establish an associated edge.

[0076] When the accumulated feedback samples reach a preset threshold (default 100), the following is triggered: The GTE-Qwen2 model is fine-tuned on new samples (the learning rate is reduced to 1 / 10 of the initial value and trained for 3 epochs), and the fine-tuned embedding vectors are rewritten into the corresponding set of the vector database. The GAT model is retrained on the updated graph (full training, as changes in graph structure may affect global embeddings). The optimal values ​​for weights w1, w2, and w3 are re-searched on the expanded labeled dataset.

[0077] Taking an e-commerce platform testing scenario as an example: Original description of the test steps: "Click the login button to enter the login page"; Original locator: id="btn-login"; After the page redesign: the login button's ID has been changed to "auth-signin-btn", the text has been changed from "Login" to "Login / Register", and a new "Quick Registration" button has been added to the page.

[0078] Self-healing process: The locator id="btn-login" did not find a matching element on the current page and triggered self-healing after 3 retries.

[0079] The initial screening yields a candidate element set C = {c1: "Login / Register" button, c2: "Quick Registration" button, c3: "Help Center" link}.

[0080] For c1 ("Login / Register" button): First level: encode("Login button") vs encode("Login / Register") → score element =0.82; Second level: Location region = top right corner (consistent with history), parent container data-module="user-auth" → score context =0.90; Third level: intent label =ecommerce.auth.login, the prerequisite step "Open Homepage" has been completed, the coexistence intent set includes ecommerce.auth.register (compatible with "Login / Register") → score flow =0.95; final score =0.35×0.82+0.25×0.90+0.40×0.95=0.287+0.225+0.380=0.892.

[0081] For c2 ("Quick Registration" button): First level: encode("Login button") vs encode("Quick Registration") → score_element=0.41; Second level: Location area = center of page (inconsistent with history), parent container data-module="promotion" → score_context=0.30; Third level: intent label=ecommerce.auth.login is inconsistent with the intent of "Quick Registration" ecommerce.auth.register → score flow =0.20; final score =0.35×0.41+0.25×0.30+0.40×0.20=0.144+0.075+0.080=0.299.

[0082] S4: final of c1 score =0.892≥0.85, automatic repair, update the locator to c1's attributes.

[0083] In this invention, by matching business processes at different levels, it is ensured that the repaired elements are consistent with the original business intent. Figure 1 It is consistent with, rather than just similar in text; the three-level scoring mechanism makes each repair decision traceable and explainable, and facilitates manual review; the human-machine collaborative feedback mechanism enables the system to continuously improve its accuracy over time; by changing the knowledge graph and retraining the weights, it can be adapted to different business fields (e-commerce, finance, healthcare, etc.).

[0084] Figure 2 This specification provides a schematic diagram of a UI automation test script self-healing device based on multi-level business intent reasoning, which may include: The graph construction module 10 is used to build offline knowledge graphs that represent business intents, page elements and their relationships. The locator monitoring module 20 is used to detect whether the locator corresponding to the current operation to be executed is malfunctioning during the test execution process; The intent reasoning module 30 is used to filter candidate elements from the current page when the locator fails, and to perform multi-level intent reasoning on each candidate element using the knowledge graph. The multi-level intent reasoning includes at least: an element semantic matching level that matches the target description of the operation to be performed, a page context matching level that matches the historical context, and a business process matching level that verifies the business process constraints in the knowledge graph, and generates a comprehensive matching score based on the matching results of the three levels. The feedback update module 40 is used to perform graded processing on the failed locator according to the comprehensive matching score. The graded processing includes at least: when the comprehensive matching score meets the preset high confidence condition, automatically updating the locator to the attribute of the candidate element with the highest score.

[0085] Optionally, the map construction module 10 includes: Analyze historical test cases to extract structured information including operation type, target element description, business intent tag, and expected result; Based on the structured information, a knowledge graph is constructed that includes intent nodes, page nodes, element nodes, and edges used to represent the relationships between nodes. An embedding vector that integrates textual semantic information and graph structure information is generated for the intent node, and the embedding vector is stored in a vector database.

[0086] Optionally, generating an embedding vector that fuses textual semantic information and graph structure information for the intent node includes: The business intent tag and its associated target element description are processed using a pre-trained text embedding model to obtain a text embedding vector; The graph neural network is used to process the neighbor information of the intent node in the knowledge graph to obtain the graph structure embedding vector; The text embedding vector is fused with the graph structure embedding vector to obtain a fused embedding vector.

[0087] Optionally, the intent reasoning module 30 includes: Query the business intent tag corresponding to the current operation in the knowledge graph to obtain its preceding intent and coexisting intent; Determine whether all the preceding intentions of the current operation have been satisfied; otherwise, directly determine that the business process matching score is the lowest value. If so, the compatibility between the intent associated with the candidate element and the coexisting intent is further evaluated to generate a business process consistency score.

[0088] Optional, including: When the accumulated relational data about the current business intent in the knowledge graph has not reached a preset threshold, a cold start strategy is adopted to calculate the consistency score of the business process: different preset initial values ​​are assigned according to the semantic or domain consistency between the candidate element's associated intent and the current operation intent; when the accumulated relational data reaches the preset threshold, the dynamic calculation mode based on graph relations is switched.

[0089] Optionally, the feedback update module 40 includes: When the comprehensive matching score reaches or exceeds the first preset threshold, it is determined to be of high confidence and repair is automatically performed; When the overall matching score is between the second preset threshold and the first preset threshold, it is determined to be of medium confidence, the test is paused and a repair suggestion for manual confirmation is output; When the overall matching score is lower than the second preset threshold, it is determined to be of low confidence, the current operation is skipped and it is marked as requiring manual processing.

[0090] Optional, also includes: The execution results or manual confirmation results of each repair are collected as feedback data. The knowledge graph is incrementally updated using the feedback data, and incremental training of the model used in the multi-level intent reasoning is triggered when the accumulated amount of feedback data reaches a preset amount.

[0091] The functions of the apparatus in this embodiment have been described in the above method embodiments. Therefore, for any parts not detailed in this embodiment, please refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.

[0092] Based on the same inventive concept, embodiments of this specification also provide an electronic device.

[0093] The following describes embodiments of the electronic device of the present invention, which can be considered as specific implementations of the methods and apparatus embodiments of the present invention described above. Details described in the embodiments of the electronic device of the present invention should be considered as supplements to the methods or apparatus embodiments described above; details not disclosed in the embodiments of the electronic device of the present invention can be implemented with reference to the methods or apparatus embodiments described above.

[0094] Figure 3 This is a schematic diagram of an electronic device provided as an embodiment of this specification. Refer to the following... Figure 3 The electronic device 300 according to this embodiment of the present invention will be described. Figure 3 The electronic device 300 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0095] like Figure 3 As shown, the electronic device 300 is presented in the form of a general-purpose computing device. The components of the electronic device 300 may include, but are not limited to: at least one processing unit 310, at least one storage unit 320, a bus 330 connecting different system components (including storage unit 320 and processing unit 310), a display unit 340, etc.

[0096] The storage unit stores program code that can be executed by the processing unit 310, causing the processing unit 310 to perform the steps described in the processing method section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 310 can perform, for example... Figure 1 The steps are shown.

[0097] The storage unit 320 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 3201 and / or a cache storage unit 3202, and may further include a read-only memory unit (ROM) 3203.

[0098] The storage unit 320 may also include a program / utility 3204 having a set (at least one) program module 3205, such program module 3205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0099] Bus 330 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0100] Electronic device 300 can also communicate with one or more external devices 400 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable viewers to interact with electronic device 300, and / or with any device that enables electronic device 300 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 350. Furthermore, electronic device 300 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 360. Network adapter 360 can communicate with other modules of electronic device 300 via bus 330. It should be understood that, although... Figure 3 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0101] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described in this invention can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the method described above according to this invention. When the computer program is executed by a data processing device, it enables the computer-readable medium to implement the method described above, i.e.: as... Figure 1 The method shown.

[0102] Figure 4 This is a schematic diagram of a computer-readable medium provided for embodiments of this specification.

[0103] accomplish Figure 1 The computer program of the method shown can be stored on one or more computer-readable media. A computer-readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0104] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0105] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the audience's computing device, partially on the audience's device, as a standalone software package, partially on the audience's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the audience's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0106] In summary, the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that in practice, general-purpose data processing devices such as microprocessors or digital signal processors (DSPs) can be used to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0107] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the present invention is not inherently related to any specific computer, virtual device, or electronic device, and various general-purpose devices can also implement the present invention. The above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0108] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0109] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A self-healing method for UI automated test scripts based on multi-level business intent reasoning, characterized in that, include: Offline construction of a knowledge graph to represent business intent, page elements and their relationships; During test execution, check whether the locator corresponding to the current operation to be executed has failed; When the locator fails, candidate elements are filtered from the current page, and multi-level intent reasoning is performed on each candidate element using the knowledge graph. The multi-level intent reasoning includes at least: element semantic matching level matching with the target description of the operation to be performed, page context matching level matching with the historical context, and business process matching level verifying business process constraints in the knowledge graph. A comprehensive matching score is generated based on the matching results of the three levels. Based on the comprehensive matching score, the failed locator is subjected to tiered processing, which includes at least: when the comprehensive matching score meets the preset high confidence condition, the locator is automatically updated to the attribute of the candidate element with the highest score.

2. The UI automated test script self-healing method based on multi-level business intent reasoning as described in claim 1, characterized in that, The offline construction of the knowledge graph used to represent business intent, page elements, and their relationships includes: Analyze historical test cases to extract structured information including operation type, target element description, business intent tag, and expected result; Based on the structured information, a knowledge graph is constructed that includes intent nodes, page nodes, element nodes, and edges used to represent the relationships between nodes. An embedding vector that integrates textual semantic information and graph structure information is generated for the intent node, and the embedding vector is stored in a vector database.

3. The UI automated test script self-healing method based on multi-level business intent reasoning as described in claim 2, characterized in that, The step of generating an embedding vector that integrates textual semantic information and graph structure information for the intent node includes: The business intent tag and its associated target element description are processed using a pre-trained text embedding model to obtain a text embedding vector; The graph neural network is used to process the neighbor information of the intent node in the knowledge graph to obtain the graph structure embedding vector; The text embedding vector is fused with the graph structure embedding vector to obtain a fused embedding vector.

4. The UI automated test script self-healing method based on multi-level business intent reasoning as described in claim 1, characterized in that, The matching process involves three levels: an element semantic matching level that matches the target description of the operation to be executed, a page context matching level that matches the historical context, and a business process matching level that verifies business process constraints in the knowledge graph. A comprehensive matching score is generated based on the matching results from these three levels, including: Query the business intent tag corresponding to the current operation in the knowledge graph to obtain its preceding intent and coexisting intent; Determine whether all the preceding intentions of the current operation have been satisfied; otherwise, directly determine that the business process matching score is the lowest value. If so, the compatibility between the intent associated with the candidate element and the coexisting intent is further evaluated to generate a business process consistency score.

5. The UI automated test script self-healing method based on multi-level business intent reasoning as described in claim 4, characterized in that, include: When the accumulated relational data about the current business intent in the knowledge graph has not reached a preset threshold, a cold start strategy is adopted to calculate the consistency score of the business process: different preset initial values ​​are assigned according to the semantic or domain consistency between the candidate element's associated intent and the current operation intent; Once the accumulated relational data reaches a preset threshold, the system switches to a dynamic calculation mode based on graph relations.

6. The UI automated test script self-healing method based on multi-level business intent reasoning as described in claim 1, characterized in that, The step of performing tiered handling of failed locators based on the comprehensive matching score includes: When the comprehensive matching score reaches or exceeds the first preset threshold, it is determined to be of high confidence and repair is automatically performed; When the overall matching score is between the second preset threshold and the first preset threshold, it is determined to be of medium confidence, the test is paused and a repair suggestion for manual confirmation is output; When the overall matching score is lower than the second preset threshold, it is determined to be of low confidence, the current operation is skipped and it is marked as requiring manual processing.

7. The UI automated test script self-healing method based on multi-level business intent reasoning as described in claim 1, characterized in that, Also includes: The execution results or manual confirmation results of each repair are collected as feedback data. The knowledge graph is incrementally updated using the feedback data, and incremental training of the model used in the multi-level intent reasoning is triggered when the accumulated amount of feedback data reaches a preset amount.

8. A self-healing device for UI automated test scripts based on multi-level business intent reasoning, characterized in that, include: The knowledge graph construction module is used to build offline knowledge graphs that represent business intents, page elements and their relationships. The locator monitoring module is used to detect whether the locator corresponding to the current operation to be performed is malfunctioning during the test execution process; The intent reasoning module is used to filter candidate elements from the current page when the locator fails, and to perform multi-level intent reasoning on each candidate element using the knowledge graph. The multi-level intent reasoning includes at least: an element semantic matching level that matches the target description of the operation to be performed, a page context matching level that matches the historical context, and a business process matching level that verifies the business process constraints in the knowledge graph. A comprehensive matching score is generated based on the matching results of the three levels. The feedback update module is used to perform graded processing on the failed locators according to the comprehensive matching score. The graded processing includes at least: when the comprehensive matching score meets the preset high confidence condition, automatically updating the locator to the attribute of the candidate element with the highest score.

9. An electronic device, characterized in that, The electronic device includes: A processor; and a memory storing computer-executable instructions, which, when executed, cause the processor to perform the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs that, when executed by a processor, implement the method of any one of claims 1-7.

Images