User interface test method and device, equipment, medium and program product

Abstract

The invention provides a user interface testing method which can be applied to the technical field of artificial intelligence, and relates to application of a large model in the fields of information security and financial science and technology. The user interface testing method comprises the steps that on the basis of a pre-constructed user interface component model library, intention inference is conducted on initial abstract syntax tree nodes of codes to be tested, target abstract syntax tree nodes are obtained, the target abstract syntax tree nodes comprise variable fields, user interface component type fields and service intention fields, and the variable fields are selected from the variable fields, the user interface component type fields and the service intention fields; the business intention field represents the business meaning of the user interface level; based on the target abstract syntax tree node and the to-be-tested code, testing the target user interface, and outputting a test result; and under the condition that the test fails, constructing an execution snapshot graph based on the target abstract syntax tree node and the test data in the test result to obtain a repair scheme. The invention further provides a user interface testing device and equipment, a storage medium and a program product.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, relates to the application of large models in the field of information security and financial technology, and more particularly to a user interface testing method, device, equipment, medium and program product. BACKGROUND

[0002] User interface testing is a testing process of verifying that the layout, style consistency, control position and visual elements of the software interface function module conform to the design specification through simulating user interaction. Traditional user interface testing is mainly implemented by positioning driver element interaction and hard-coded assertion as the core, assisted by image recognition, recording and playback technology, but the high vulnerability and inefficient maintenance problems caused by the strong coupling between test code and user interface implementation in user interface testing have not been solved. Therefore, it is urgent to provide a method that can effectively solve the strong coupling problem between test code and user interface front-end implementation, realize accurate traceability and repair of test failure causes, and improve the adaptability and fault troubleshooting efficiency of user interface testing. SUMMARY

[0003] In view of the above problems, the present application provides a user interface testing method, device, equipment, medium and program product.

[0004] According to a first aspect of the present application, a user interface testing method is provided, comprising: based on a pre-constructed user interface component model library, performing intent inference on an initial abstract syntax tree node of test code to obtain a target abstract syntax tree node, wherein the target abstract syntax tree node includes a variable field, a user interface component type field and a business intent field, and the business intent field represents the business meaning of the user interface level; based on the target abstract syntax tree node and the test code, testing the target user interface and outputting a test result; in the case of test failure, constructing an execution snapshot graph based on the target abstract syntax tree node and the test data in the test result to obtain a repair scheme, wherein the execution snapshot graph includes the target abstract syntax tree node, a document object model node representing a user interface component entity, a state node representing the available state of the user interface component entity, and an edge representing the association between nodes.

[0005] According to the embodiment of the present application, the intention inference is performed on the initial abstract syntax tree node of the code to be tested, and the target abstract syntax tree node is obtained, including: extracting a field in the initial abstract syntax tree node, and taking the field as a variable field; performing word segmentation processing on the field according to a preset naming rule to obtain a plurality of segmented fragments; matching each segmented fragment with a standardized user interface component type in a user interface component model library, taking the segmented fragment that matches successfully as a user interface component type field, and taking the segmented fragment that does not match successfully as a business intention field; and forming the target abstract syntax tree node based on the variable field, the user interface component type field and the business intention field.

[0006] According to the embodiment of the present application, the intention inference is performed on the initial abstract syntax tree node of the code to be tested, and the target abstract syntax tree node is obtained, including: extracting a field in the initial abstract syntax tree node, and taking the field as a variable field; extracting context information of the field, wherein the context information includes a method set called by the field and a locator call type corresponding to the field; analyzing the method set based on the user interface component model library to obtain a candidate user interface component type field; analyzing the locator call type to determine a user interface component type field from the candidate user interface component type field; obtaining a business intention field representing a business meaning of a user interface level according to a business scenario corresponding to the method set and the locator call type; and forming the target abstract syntax tree node based on the variable field, the user interface component type field and the business intention field.

[0007] According to the embodiment of the present application, the intention inference is performed on the initial abstract syntax tree node of the code to be tested, and the target abstract syntax tree node is obtained, including: extracting and standardizing the code to be tested corresponding to the initial abstract syntax tree node to obtain standard data; performing semantic analysis on the standard data by using a pre-trained large model based on the user interface component model library to obtain the target abstract syntax tree node including a variable field, a user interface component type field and a business intention field.

[0008] According to the embodiment of the present application, in the case of test failure, the execution snapshot graph is constructed based on the target abstract syntax tree node and the test data in the test result, and the repair scheme is obtained, including: extracting a problem vector based on the execution snapshot graph, wherein the problem vector represents the cause of the test failure; according to the problem vector, searching in a historical vector library based on vector similarity to obtain a repair scheme, and storing the problem vector and the repair scheme in the historical vector library, wherein the historical vector library contains N historical problem vectors and at least one repair scheme and repair effect evaluation value corresponding thereto, N is a positive integer greater than or equal to 1.

[0009] According to an embodiment of this application, extracting a problem vector based on an execution snapshot graph includes: finding the corresponding target document object model node based on the target abstract syntax tree node corresponding to the test failure; if the target document object model node is not found, determining the test failure reason as locator failure; if the target document object model node is found, checking the target state node corresponding to the target document object model node; if the target state node is abnormal, determining the test failure reason as state error; if the target state node is normal, checking the parent node of the target abstract syntax tree node and the parent node of the target document object model node; and obtaining the problem vector based on the execution snapshot graph and the test failure reason.

[0010] According to an embodiment of this application, the method of obtaining a repair solution based on a problem vector and a historical vector database based on vector similarity includes: retrieving N historical problem vectors similar to the problem vector from the historical vector database based on similarity; obtaining at least one historical repair solution associated with each historical problem vector and its corresponding historical repair effect evaluation value; calculating the expected success rate of each historical repair solution based on its frequency of occurrence and using the corresponding historical repair effect evaluation value as a weight; and comparing the expected success rates of each historical repair solution to obtain a repair solution.

[0011] The second aspect of this application provides a user interface testing apparatus, comprising: an intent inference module, used to infer intent from the initial abstract syntax tree node of the code to be tested based on a pre-built user interface component model library, to obtain a target abstract syntax tree node, wherein the target abstract syntax tree node includes a variable field, a user interface component type field, and a business intent field, the business intent field representing the business meaning at the user interface level; a testing module, used to test the target user interface based on the target abstract syntax tree node and the code to be tested, and output test results; and a repair solution module, used to construct an execution snapshot graph based on the target abstract syntax tree node and the test data in the test results to obtain a repair solution in the event of test failure, wherein the execution snapshot graph includes the target abstract syntax tree node, a document object model node representing a user interface component entity, a state node representing the available state of the user interface component entity, and edges representing the relationships between the nodes.

[0012] A third aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0013] A fourth aspect of this application also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0014] The fifth aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method. Attached Figure Description

[0015] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0016] Figure 1 The illustration schematically depicts application scenarios of user interface testing methods, apparatus, devices, media, and program products according to embodiments of this application;

[0017] Figure 2 A flowchart illustrating a user interface testing method according to an embodiment of this application is shown schematically.

[0018] Figure 3 A flowchart illustrating a method for obtaining a target abstract syntax tree node according to an embodiment of this application is shown schematically.

[0019] Figure 4 A flowchart illustrating another method for obtaining a target abstract syntax tree node according to an embodiment of this application is shown schematically;

[0020] Figure 5 The flowchart schematically illustrates yet another method for obtaining a target abstract syntax tree node according to an embodiment of this application;

[0021] Figure 6 This illustration schematically shows an execution snapshot diagram according to an embodiment of the present application;

[0022] Figure 7 A schematic diagram illustrating the structure of a user interface testing apparatus according to an embodiment of this application is shown; and

[0023] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a user interface testing method according to an embodiment of this application. Detailed Implementation

[0024] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0027] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0028] In the technical solution of this application, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0029] In scenarios involving automated decision-making using personal information, the methods, devices, and systems provided in this application all offer users corresponding entry points for choosing to agree to or reject the automated decision-making results. If the user chooses to reject, the process proceeds to the expert decision-making stage. Here, "automated decision-making" refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests, or economic, health, and credit status through computer programs, and then making a decision. Here, "expert decision-making" refers to the activity of making decisions by personnel who specialize in a particular field, possess specialized experience, knowledge, and skills, and have reached a certain level of professional expertise.

[0030] This application provides a user interface testing method, comprising: inferring intent from the initial abstract syntax tree node of the code to be tested based on a pre-built user interface component model library to obtain a target abstract syntax tree node, wherein the target abstract syntax tree node includes a variable field, a user interface component type field, and a business intent field, the business intent field representing the business meaning at the user interface level; testing the target user interface based on the target abstract syntax tree node and the code to be tested, and outputting test results; in the event of test failure, constructing an execution snapshot graph based on the target abstract syntax tree node and the test data in the test results to obtain a repair solution, wherein the execution snapshot graph includes the target abstract syntax tree node, a document object model node representing the user interface component entity, a state node representing the available state of the user interface component entity, and edges representing the relationships between the nodes. By inferring intent from the initial abstract syntax tree node based on the user interface component model library to generate the target abstract syntax tree node, the test code is decoupled from the implementation details of the user interface (UI) front end, solving the high vulnerability problem caused by the strong coupling between traditional UI test code and locators and UI styles, and improving the stability and maintainability of the test code in scenarios of UI reconstruction and element attribute changes. By testing the target user interface using target abstract syntax tree nodes in conjunction with the code under test, this approach addresses the shortcomings of traditional testing in its insufficient focus on business context. When a test fails, an execution snapshot graph is constructed based on the target abstract syntax tree nodes and test data. This facilitates the visualization of discrepancies between the code's intent and the actual UI state, enabling precise diagnosis of test failure causes. This avoids the inefficiency of traditional manual troubleshooting. Furthermore, the execution snapshot graph analysis generates targeted remediation plans, improving the accuracy of fault repair and ultimately enhancing the adaptability of UI testing to dynamic and complex UI scenarios, as well as improving troubleshooting efficiency.

[0031] Figure 1 The illustration schematically depicts application scenarios of user interface testing methods, apparatus, devices, media, and program products according to embodiments of this application.

[0032] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0033] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0034] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0035] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0036] It should be noted that the user interface testing method provided in this application embodiment can generally be executed by server 105. Correspondingly, the user interface testing device provided in this application embodiment can generally be located in server 105. The user interface testing method provided in this application embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the user interface testing device provided in this application embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0037] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0038] The following will be based on Figure 1 The described scene, through Figures 2-6 A user interface testing method according to embodiments of this application will be described in detail.

[0039] Figure 2 A flowchart illustrating a user interface testing method according to an embodiment of this application is shown.

[0040] like Figure 2 As shown, the user interface testing method of this embodiment includes operations S210 to S230.

[0041] In operation S210, based on the pre-built user interface component model library, intent inference is performed on the initial abstract syntax tree node of the code to be tested to obtain the target abstract syntax tree node. The target abstract syntax tree node includes variable fields, user interface component type fields, and business intent fields. The business intent fields represent the business meaning at the user interface level.

[0042] In the embodiments of this application, the User Interface (UI) can refer to the visual interface through which a user interacts with software or an application, such as the login page of a mobile application or a website. The Abstract Syntax Tree (AST) can refer to the syntactic structure tree-like representation of the code to be tested, breaking down the syntactic logic of the code into individual nodes, such as variable definitions, function calls, and conditional statements. Each node represents a syntactic unit of the code, excluding redundant formatting such as spaces and comments, retaining only the core syntactic logic. The initial AST node can refer to an unprocessed AST node that purely describes the syntactic structure of the code. The target AST node can refer to an AST node that has been appended with key information after intent inference, and can contain three core fields: a variable field, a user interface component type field, and a business intent field.

[0043] In the embodiments of this application, a user interface component model library is pre-built. This library can refer to a pre-built, technology-independent, standardized component database. It contains definitions of common UI components, such as buttons, input boxes, and dropdown menus, as well as identification attributes for each component. Priority can be distinguished; for example, the prompt text of an input box is a strong identification feature, while the style class name is a weak identification feature. This can be used to match and identify components in the UI to be tested. An intent inference module is built to clarify the business intent of an isolated piece of code. Intent inference operations are performed on the initial abstract syntax tree node of the code to be tested, ultimately generating the target abstract syntax tree node.

[0044] For example, the intent inference process extracts UI-related information from the initial AST nodes and combines it with the standardized rules of the UI component model library to determine three key fields: the variable field corresponds to variable names in the code under test; the user interface component type field is the component category determined by matching the extracted information with standardized UI components in the model library, such as matching as a button or input box; and the business intent field is the business meaning of the node at the UI level, derived from the UI component type and code logic, such as login verification or order submission. By integrating these three types of fields, the target abstract syntax tree node is given a UI-level business meaning beyond its purely syntactic structure.

[0045] When operating S220, the target user interface is tested based on the target abstract syntax tree nodes and the code to be tested, and the test results are output.

[0046] In the embodiments of this application, based on the generated target abstract syntax tree nodes and the original code to be tested, test operations are performed on the actual user interface, and explicit test results are output. The test results can be relevant information when the test is successful or error information corresponding to the test failure.

[0047] In operation S230, in the event of a test failure, an execution snapshot graph is constructed based on the target abstract syntax tree nodes and the test data in the test results to obtain a repair solution. The execution snapshot graph includes target abstract syntax tree nodes, document object model nodes representing user interface component entities, state nodes representing the available state of user interface component entities, and edges representing the relationships between the nodes.

[0048] In the embodiments of this application, the execution snapshot graph can refer to the fault scenario association graph constructed when a test fails, containing target AST nodes, DOM nodes, state nodes, and edges connecting nodes. It is used to visually present the correspondence between code intent, actual UI components, and actual UI component states, which can assist in fault location. Document Object Model (DOM) nodes refer to nodes that represent actual UI component entities; they are tree-structured nodes representing the parsed user interface. State nodes can refer to nodes that represent the available state of UI component entities, recording the current state of the UI component, such as enabled, disabled, hidden, and visible.

[0049] For example, when a test fails, an execution snapshot graph is constructed based on the target abstract syntax tree (AST) nodes and test data recorded in the test results, such as error details, component status records, and operation execution logs. The graph is then used to deduce a repair plan. The execution snapshot graph contains edges connecting three types of core nodes to representative nodes: target AST nodes, representing the code intent, carry the code syntax, UI component type, and business intent information of the failed test; DOM nodes, representing the UI entity, represent the component entities present in the actual UI interface at the time of the test failure, including information such as component position, attributes, and hierarchy; and state nodes record the current availability state of the UI component corresponding to the DOM node, such as enabled, disabled, hidden, or visible. Edges between nodes define the association logic between various types of nodes. For example, an edge between a target AST node and a DOM node represents which UI component the code wants to operate on; an edge between a DOM node and a state node represents the current state of the UI component; an edge between target AST nodes represents the logical dependencies between code components; and an edge between DOM nodes represents the hierarchical relationship between UI components.

[0050] According to embodiments of this application, by inferring intent from initial abstract syntax tree nodes based on a user interface component model library, a target abstract syntax tree node is generated. This decouples the test code from the UI front-end implementation details, solving the high vulnerability problem caused by the strong coupling between traditional UI test code and locators and UI styles. It also improves the stability and maintainability of the test code in scenarios involving UI reconstruction and element attribute changes. By testing the target user interface using the target abstract syntax tree node in conjunction with the code under test, the shortcomings of traditional testing in insufficient attention to business context are addressed. Furthermore, by constructing an execution snapshot graph based on the target abstract syntax tree node and test data when a test fails, the difference between the code intent and the actual UI state is readily apparent, enabling accurate diagnosis of the cause of test failure. This avoids the inefficiency of traditional troubleshooting relying on manual intervention. Simultaneously, targeted repair solutions are generated based on the execution snapshot graph analysis, improving the accuracy of fault repair and thus enhancing the adaptability of UI testing to dynamic and complex UI scenarios and the efficiency of fault diagnosis.

[0051] Figure 3 A flowchart illustrating a method for obtaining a target abstract syntax tree node according to an embodiment of this application is shown schematically.

[0052] like Figure 3 As shown, the method for obtaining the target abstract syntax tree node in this embodiment includes operations S310 to S340.

[0053] In operation S310, the fields in the initial abstract syntax tree node are extracted and used as variable fields.

[0054] In the embodiments of this application, the fields related to UI testing are first extracted from the initial abstract syntax tree nodes to obtain variable fields.

[0055] When operating S320, the field is segmented according to the preset naming rules to obtain multiple segmented fragments.

[0056] In the embodiments of this application, the preset naming rules can refer to unified rules for naming UI-related fields defined in advance during UI testing, such as camelCase, underscore separation, and suffix identification. The fields are segmented according to the predefined preset naming rules, breaking down consecutive field names into multiple semantically independent segmented fragments. The purpose of segmentation is to deconstruct the semantics of field names, providing the smallest analyzable semantic unit for subsequent component type matching and business intent determination.

[0057] In operation S330, each segmented word fragment is matched with the standardized user interface component types in the user interface component model library. The successfully matched segmented word fragments are used as the user interface component type field, and the unmatched segmented word fragments are used as the business intent field.

[0058] In the embodiments of this application, the standardized user interface component type can refer to the predefined, uniformly standardized UI component categories in the user interface component model library, such as Button, Input box, Dropdown menu, Table, etc. Each segmented word is compared one by one with the standardized user interface component types in the UI component model library to determine whether the segment belongs to the identifier of a certain type of component. For example, the identifier for a button in the UI component model library is "Button," and the identifier for an input box is "Input." If a segmented word matches the identifier of a standardized component type, the segment is directly used as the user interface component type field, representing the UI component category corresponding to this variable field. If a segmented word cannot match any standardized component type, these segments constitute a business intent field, reflecting the business purpose of the UI component.

[0059] When operating S340, target abstract syntax tree nodes are formed based on variable fields, user interface component type fields, and business intent fields.

[0060] In the embodiments of this application, variable fields, user interface component type fields, and business intent fields are integrated to generate a target abstract syntax tree node. The integration process is not a simple concatenation; rather, the three types of fields are appended as core attributes to the initial AST node. For example, the integrated target AST node attributes are: variable field = submit Order Btn, user interface component type field = button, business intent field = submit order, etc.

[0061] According to embodiments of this application, by extracting variable fields, interference from irrelevant syntactic fields in the inference process can be avoided. By segmenting variable fields according to preset naming rules, the problem of traditional initial AST nodes only carrying syntactic structure and lacking UI business semantics is solved. By matching segmented fragments with standardized user interface component types in the user interface component model library, the consistency and accuracy of UI component type determination are improved. Unmatched segmented fragments are categorized as business intent fields, accurately extracting the business meaning at the UI level, giving the abstract syntax tree node a clear business meaning. By integrating the above three fields to form the target AST node, the target AST node is endowed with UI-level business meaning beyond its purely syntactic structure, reducing reliance on manual interpretation of business requirements during testing, supporting the decoupling of test code from UI front-end implementation details, and further reducing code maintenance costs.

[0062] Figure 4 The flowchart illustrating another method for obtaining a target abstract syntax tree node according to an embodiment of this application is shown schematically.

[0063] like Figure 4 As shown, another method for obtaining the target abstract syntax tree node in this embodiment includes operations S410 to S450.

[0064] In operation S410, the fields in the initial abstract syntax tree node are extracted and used as variable fields.

[0065] In the embodiments of this application, fields related to UI testing are first extracted from the initial abstract syntax tree nodes and used as variable fields.

[0066] In operation S420, the context information of the field is extracted, which includes the set of methods called by the field and the locator call type corresponding to the field.

[0067] In the embodiments of this application, context information can refer to code association information surrounding a field in UI testing, reflecting the purpose of that field. This includes the set of methods invoked by the field and the locator call type corresponding to the field. The set of methods invoked by the field can refer to the combination of UI testing-related methods subsequently executed for that field in the code to be tested, such as click methods, input methods, etc. Different UI components support different methods. The locator call type can refer to the technical method used in the code to locate the actual UI component corresponding to the field. It is a key identifier connecting code variables and UI entities, such as locating by component ID (By.ID), locating by link text (By.LINK_TEXT), locating by tag name (By.TAG_NAME), etc. Different locator types typically correspond to specific UI components.

[0068] When operating S430, based on the user interface component model library, the method set is analyzed to obtain the candidate user interface component type field; the locator call type is analyzed to determine the user interface component type field from the candidate user interface component type field.

[0069] In the embodiments of this application, the candidate user interface component type field can refer to multiple UI component types that may match the field, initially filtered based on the method set. The mapping relationship between the method set and the component types and supported methods in the UI component model library is compared to filter out all UI component types that support the method set, which are then used as the candidate UI component type field. If the candidate UI component type field contains only one type, it is directly used as the final result; if it contains multiple types, further filtering is performed using the locator call type. That is, the locator call type is compared with the mapping relationship between UI component types and locator types in the UI component model library, eliminating unmatched candidate types and retaining the uniquely matching type as the user interface component type field.

[0070] When operating S440, based on the method set and the business scenario corresponding to the locator call type, the business intent field representing the business meaning at the user interface level is obtained.

[0071] In the embodiments of this application, a business scenario can refer to the specific UI operation purpose jointly pointed to by the method set and the locator call type, such as clicking a method and locating the corresponding order submission scenario by pressing the submit order button ID. Based on the method set, the locator call type, and the corresponding business scenario, the business intent field representing the business meaning at the UI level is derived. By analyzing the purpose of the business scenario, the business intent field can be extracted.

[0072] When operating S450, target abstract syntax tree nodes are formed based on variable fields, user interface component type fields, and business intent fields.

[0073] In the embodiments of this application, variable fields, user interface component type fields, and business intent fields are integrated to generate target abstract syntax tree nodes. The integration process is not a simple concatenation, but rather the three types of fields are attached as core attributes to the initial AST node.

[0074] According to embodiments of this application, by further extracting contextual information of fields, the strong dependence of traditional intent inference on field naming conventions is overcome, improving the scenario adaptability of intent inference. By determining the final UI component type field based on method sets and locator call type analysis, misjudgments of UI component types caused by single-dimensional analysis are avoided, improving the accuracy of UI component type inference. By deriving business intent fields based on the business scenarios corresponding to method sets and locator call types, the extraction of business intent is combined with the actual logic of the code, enabling the business intent fields to accurately represent the core business meaning at the user interface level. By integrating the above three fields to form a target AST node, the target AST node is given UI-level business meaning beyond its purely syntactic structure, reducing the reliance on manual interpretation of business requirements during UI testing, supporting the decoupling of test code from UI front-end implementation details, and reducing the high fragility and inefficient maintenance problems in the user interface testing process.

[0075] Figure 5 The flowchart illustrates yet another method for obtaining a target abstract syntax tree node according to an embodiment of this application.

[0076] In the embodiments of this application, the process of performing intent inference on the initial abstract syntax tree node of the code to be tested to obtain the target abstract syntax tree node further includes: extracting and standardizing the code to be tested corresponding to the initial abstract syntax tree node to obtain standard data; and using a pre-trained large model based on a user interface component model library to perform semantic parsing on the standard data to obtain the target abstract syntax tree node including variable fields, user interface component type fields, and business intent fields.

[0077] like Figure 5 As shown, standard data is obtained by standardizing the test code related to the syntax logic of the initial abstract syntax tree node, either by association or direct filtering. This standard data is then input into a pre-trained large model, which performs semantic parsing to obtain target abstract syntax tree nodes that include variable fields, user interface component type fields, and business intent fields.

[0078] In the embodiments of this application, standardization processing can refer to the preprocessing operation of converting the extracted original code fragments into a unified format that can be efficiently parsed by a pre-trained large model. The purpose is to eliminate format noise and semantic ambiguity in the original code, such as cleaning up redundant comments, standardizing indentation, line breaks, and standardizing capitalization, as well as redundancy removal and context completion. The large model refers to an artificial intelligence model trained on a large amount of data and with a large parameter scale based on deep learning technology. Pre-training can refer to training the large model in advance using a large amount of UI test code, UI component types, and business intent annotation data. The pre-trained large model has the ability to understand code syntax logic, associate UI component attributes, and deduce business meanings, and can autonomously learn the relationship between code and UI business. Semantic parsing can refer to the process of deep analysis of the standardized data by the pre-trained large model. The pre-trained large model can not only identify syntactic elements such as variables and functions in the code, but also understand the relationship between these elements and UI component entities and business goals, ultimately outputting a target AST node including three types of fields.

[0079] According to embodiments of this application, by extracting and standardizing the code to be tested corresponding to the initial abstract syntax tree nodes, the formatting chaos, semantic ambiguity, and interference from invalid information in the original code are eliminated, thereby improving the basic accuracy of semantic parsing. Utilizing the powerful natural language and code semantic understanding generalization capabilities of pre-trained large models, combined with a user interface component model library, the reliance on manual intervention in UI testing can be reduced. This results in more complete and semantically accurate output target AST node information, imbuing the target AST nodes with UI-level business meaning beyond mere syntactic structure. This enhances the scenario adaptability and automation of intent inference, further supports the decoupling of test code from UI front-end implementation details, and strengthens the adaptability of UI testing to complex and dynamic UI scenarios.

[0080] In the embodiments of this application, in the event of test failure, an execution snapshot graph is constructed based on the target abstract syntax tree node and the test data in the test results to obtain a repair solution, including: extracting problem vectors based on the execution snapshot graph, wherein the problem vectors represent the reasons for test failure; retrieving a repair solution from a historical vector library based on vector similarity according to the problem vectors, and storing the problem vectors and repair solutions in the historical vector library, wherein the historical vector library contains N historical problem vectors and at least one corresponding repair solution and repair effect evaluation value, where N is a positive integer greater than or equal to 1.

[0081] In the embodiments of this application, a problem vector can refer to a vector obtained by transforming multi-dimensional features representing test failures in the execution snapshot graph, such as failure type, UI component information, and code features, through an encoder. Its function is to quantify the cause of failure; different failure scenarios correspond to different vectors, and similar failure causes can correspond to vectors with high similarity. A test failure cause can refer to a key feature of the failure scenario corresponding to each dimension of the problem vector. The overall features of the vector can uniquely map a certain type of failure cause. For example, when a failure occurs, multi-dimensional features are extracted from the execution snapshot graph. These multi-dimensional features may include: code-side features, i.e., variable fields, user interface component type fields, and business intent fields in the target AST node; UI-side features, i.e., attributes of the DOM node, such as the type of UI component, the types and text of surrounding related components, and the depth of the DOM tree; state information of the state node, such as whether the component is disabled or hidden; and diagnostic features, i.e., the preliminary diagnostic results given by the deterministic engine. These multi-dimensional features are then transformed into a problem vector through a preset feature encoder.

[0082] In the embodiments of this application, the historical vector library can refer to a structured knowledge base that stores past test failure scenarios, repair solutions, and repair effects. It is the core carrier for accumulating repair experience and contains N historical entries, where N is a positive integer greater than or equal to 1. Each entry consists of three parts: a historical problem vector, representing the vector transformed from a past test failure scenario; at least one historical repair solution, representing a repair solution formulated and applied for that historical failure scenario; and a repair effect evaluation value, representing a quantitative score of the actual application effect of the historical repair solution. The scoring rules can be customized. Vector similarity can refer to a quantitative indicator such as cosine similarity or Euclidean distance that measures the similarity between the current problem vector and historical problem vectors. The higher the similarity, the closer the causes of the two failure scenarios are. Similarity retrieval can refer to using a vector similarity algorithm to quickly find several historical entries in the historical vector library that have the highest similarity to the current problem vector. The purpose is to reuse successful repair experience in similar scenarios. Existing vector similarity algorithms can be used and are not limited here.

[0083] For example, using the problem vector as the core of retrieval, and relying on vector similarity algorithms such as k-nearest neighbors and cosine similarity matching algorithms, the system searches the historical vector database to obtain a repair solution suitable for the current test failure scenario. After the repair solution is applied and its effectiveness is verified, the current problem vector, the finally adopted repair solution, and the corresponding repair effect evaluation value are integrated into a new historical entry and stored in the historical vector database, achieving a closed-loop knowledge accumulation of failure scenarios, repair solutions, and effect feedback.

[0084] According to embodiments of this application, by transforming multi-dimensional features in the execution snapshot graph into problem vectors representing the causes of test failures, the problems of vague descriptions and difficulty in precise matching of traditional fault causes are avoided, providing a calculable and comparable basis for the rapid retrieval of subsequent repair solutions. By searching the historical vector library based on vector similarity, the optimal repair solution for similar failure scenarios can be quickly matched, overcoming the inefficiency of relying on manual derivation and repetitive analysis in the traditional repair process, and improving the speed of repair response and the accuracy of solution adaptation after test failure. By storing the current problem vector, repair solution, and repair effect evaluation value in the historical vector library, the systematic accumulation and dynamic sedimentation of repair experience can be achieved, breaking the "knowledge silo" dilemma of traditional repair knowledge at the individual tester level, enabling UI testing to have self-evolution capabilities, that is, as historical data continues to enrich, the retrieval accuracy and repair success rate of subsequent similar failure scenarios continue to improve.

[0085] In the embodiments of this application, extracting the problem vector based on the execution snapshot graph includes: finding the corresponding target document object model node based on the target abstract syntax tree node corresponding to the test failure; if the target document object model node is not found, the test failure reason is determined to be locator failure; if the target document object model node is found, checking the target state node corresponding to the target document object model node; if the target state node is abnormal, the test failure reason is determined to be state error; if the target state node is normal, checking the parent node of the target abstract syntax tree node and the parent node of the target document object model node; and obtaining the problem vector based on the execution snapshot graph and the test failure reason.

[0086] In the embodiments of this application, the target document object model node can refer to a DOM node that is associated with the target AST node corresponding to the test failure. The locator can refer to the technical identifier connecting the test code and the actual UI component entity in UI testing; essentially, it is an address mapping rule or technique used to uniquely identify and locate the target UI component in the document object model tree. Its function is to enable the test code to accurately find the UI component to be interacted with, providing a basis for component location for subsequent click, input, and read operations. The target state node can refer to the state node associated with the target DOM node, recording the actual usable state of the UI component. The parent node of the target AST node can refer to the upper-level node of the target AST node in the abstract syntax tree. The parent node of the target DOM node can refer to the upper-level node of the target DOM node in the document object model tree.

[0087] Figure 6 A schematic diagram of an execution snapshot map according to an embodiment of this application is shown.

[0088] like Figure 6As shown, the retrieval is based on the target AST node corresponding to the test failure. The corresponding DOM node, i.e., the target DOM node, is quickly located by using the association edges between the target AST node and the DOM node in the execution snapshot graph. If no target DOM node corresponding to the target AST node is found in the execution snapshot graph, it means that the component the code intends to manipulate cannot be matched by the current locator in the actual UI. The test failure can be attributed to locator failure. Examples of locator failure include changes to component IDs, class names, etc., after UI refactoring, or incorrect locator paths. If the target DOM node is successfully found, it means the locator is effective. Further verification is needed to confirm whether the component has an interactive state, i.e., checking the corresponding state node associated with the DOM node, i.e., the target state node associated with the target DOM node. If the component state recorded by the target state node conflicts with the expected state of the code operation, i.e., the state node is abnormal, the test failure can be attributed to a state error. For example, the code wants to click the checkout button, expecting the state to be enabled, but the target state node shows disabled. If the target state node is normal, meaning the component state matches the expected code operation (e.g., button enabled, input box visible), it indicates the component itself is fine. The problem lies in investigating the influence of the related hierarchy, which involves checking the parent node of the target AST node and the parent node of the corresponding DOM node—in other words, the parent node of the target DOM node. After this layer-by-layer investigation, the cause of the test failure has been identified. The multi-dimensional features and failure reasons identified in the snapshot graph will be converted into a high-dimensional problem vector using a feature encoder.

[0089] According to embodiments of this application, the above troubleshooting logic enables precise classification and localization of test failure causes, thoroughly solving the problems of vague and difficult-to-quantify general fault attribution, and improving the accuracy and reliability of finding failure causes. By generating problem vectors based on multi-dimensional features in the failure cause and execution snapshot graph, the accuracy of problem vectors in representing failure scenarios is improved, providing more targeted matching basis for subsequent vector similarity retrieval, thereby enhancing the fault diagnosis capability and robustness of UI testing in complex scenarios.

[0090] In the embodiments of this application, the process of obtaining a repair solution based on a historical vector database by searching the historical vector database based on vector similarity, according to the problem vector, includes: retrieving N historical problem vectors similar to the problem vector from the historical vector database based on similarity; obtaining at least one historical repair solution associated with each historical problem vector and the corresponding historical repair effect evaluation value; calculating the expected success rate of each historical repair solution based on its frequency of occurrence and using the corresponding historical repair effect evaluation value as a weight; and comparing the expected success rates of each historical repair solution to obtain a repair solution.

[0091] For example, using the generated problem vector as the core of the retrieval, and relying on preset vector similarity algorithms such as cosine similarity algorithm and k-nearest neighbor algorithm, matching is performed in the historical vector database. By calculating the similarity between the current problem vector and all historical problem vectors in the database, they are sorted in descending order of similarity, and finally the top N historical problem vectors with the highest similarity are selected. After obtaining N similar historical problem vectors, the complete historical entries corresponding to each historical problem vector are associated one by one, and the frequency of their respective adopted historical repair solutions is counted. The historical repair effect evaluation value of each solution is used as a weight to calculate the expected success rate of each repair solution, that is, the average of the frequency of occurrence of the solution. The expected success rates of all historical repair solutions are compared numerically, and the solution with the highest expected success rate is selected as the final repair solution for the current test failure scenario. If there are multiple historical repair solutions with the same expected success rate (i.e., tied for first place), a preset priority rule can be used, such as prioritizing the solution with higher frequency of occurrence or prioritizing the earliest verified effective solution, for secondary filtering to ensure the uniqueness of the output results. Here, the preset priority rule is not limited. It should be clarified that a historical problem vector may be associated with at least one historical repair solution. This is because when dealing with similar failure scenarios in the past, multiple repair paths may have been tried and their effects recorded. The historical repair effect evaluation value is a quantitative representation of the actual application effect of the solution, such as 1.0 for successful repair, 0.0 for failed repair, and -0.5 for manual intervention repair. The higher the effect evaluation value, the greater the weight and the greater the contribution to the final result.

[0092] Each addition of a repair plan updates and enriches the historical vector database. For example, the entire process can be understood as a more standard reinforcement learning problem. A model is built around the problem vector, available historical repair plans, and corresponding historical repair performance evaluations. The input is the problem vector, and the output is the expected success rate of each historical repair plan. The historical repair plan with the highest expected success rate is then executed.

[0093] According to embodiments of this application, N historical problem vectors are selected based on vector similarity retrieval instead of a single one, avoiding interference from the randomness of individual cases and improving the reliability of historical repair solutions. By calculating the expected success rate by weighting historical repair effect evaluation values ​​and combining them with the frequency of occurrence of historical repair solutions, the one-sidedness of ignoring effects by only looking at frequency and the randomness of ignoring the generalizability by only looking at single effects are avoided, thus improving the objectivity, adaptability, and accuracy of historical repair solutions. By comparing the expected success rates of each historical repair solution to determine the final repair solution, the output repair solution is highly adaptable and has a high success rate, verified by historical data. This improves the reliability and efficiency of UI test failure repair, reduces code maintenance costs caused by dynamic UI changes, and enhances the robustness of UI testing to complex and ever-changing UI scenarios.

[0094] Based on the above-described user interface testing method, this application also provides a user interface testing apparatus. The following will combine... Figure 7 The device is described in detail.

[0095] Figure 7 A schematic block diagram of a user interface testing apparatus according to an embodiment of this application is shown.

[0096] like Figure 7 As shown, the user interface testing apparatus 700 of this embodiment includes an intent inference module 710, a testing module 720, and a repair solution acquisition module 730.

[0097] The intent inference module 710 is used to infer the intent of the initial abstract syntax tree node of the code to be tested based on a pre-built user interface component model library, to obtain a target abstract syntax tree node. The target abstract syntax tree node includes a variable field, a user interface component type field, and a business intent field, where the business intent field represents the business meaning at the user interface level. In one embodiment, the intent inference module 710 can be used to perform the operation S210 described above, which will not be repeated here.

[0098] The testing module 720 is used to test the target user interface based on the target abstract syntax tree nodes and the code to be tested, and output the test results. In one embodiment, the testing module 720 can be used to perform the operation S220 described above, which will not be repeated here.

[0099] The repair solution module 730 is used to construct an execution snapshot graph based on the target abstract syntax tree nodes and test data in the test results to obtain a repair solution in the event of test failure. The execution snapshot graph includes target abstract syntax tree nodes, document object model nodes representing user interface component entities, state nodes representing the available state of user interface component entities, and edges representing the relationships between the nodes. In one embodiment, the repair solution module 730 can be used to perform the operation S230 described above, which will not be repeated here.

[0100] According to an embodiment of this application, the intent inference module 710 is further configured to extract fields from the initial abstract syntax tree node and use the fields as variable fields; perform word segmentation on the fields according to a preset naming rule to obtain multiple word segmentation fragments; match each word segmentation fragment with the standardized user interface component type in the user interface component model library, use the successfully matched word segmentation fragment as the user interface component type field, and use the unmatched word segmentation fragment as the business intent field; and form a target abstract syntax tree node based on the variable fields, user interface component type fields, and business intent fields.

[0101] According to an embodiment of this application, the intent inference module 710 is further configured to extract fields from the initial abstract syntax tree node and use the fields as variable fields; extract the context information of the fields, wherein the context information includes the set of methods called by the field and the locator call type corresponding to the field; analyze the method set based on the user interface component model library to obtain candidate user interface component type fields; analyze the locator call type to determine the user interface component type field from the candidate user interface component type fields; obtain the business intent field representing the business meaning at the user interface level based on the business scenario corresponding to the method set and the locator call type; and form the target abstract syntax tree node based on the variable fields, the user interface component type fields, and the business intent fields.

[0102] According to an embodiment of this application, the intent inference module 710 is further used to extract and standardize the code to be tested corresponding to the initial abstract syntax tree node to obtain standard data; based on the user interface component model library, the standard data is semantically parsed using a pre-trained large model to obtain the target abstract syntax tree node including variable fields, user interface component type fields and business intent fields.

[0103] According to an embodiment of this application, the repair scheme module 730 is further used to extract problem vectors based on the execution snapshot map, the problem vectors representing the reasons for test failure; based on the problem vectors, a repair scheme is obtained by searching the historical vector library based on vector similarity, and the problem vectors and repair schemes are stored in the historical vector library, wherein the historical vector library contains N historical problem vectors and at least one corresponding repair scheme and repair effect evaluation value, where N is a positive integer greater than or equal to 1.

[0104] According to an embodiment of this application, the repair solution module 730 is further configured to search for the corresponding target document object model node based on the target abstract syntax tree node corresponding to the test failure; if the target document object model node is not found, the test failure reason is determined to be locator failure; if the target document object model node is found, the target state node corresponding to the target document object model node is checked; if the target state node is abnormal, the test failure reason is determined to be state error; if the target state node is normal, the parent node of the target abstract syntax tree node and the parent node of the target document object model node are checked; and a problem vector is obtained based on the execution snapshot graph and the test failure reason.

[0105] According to an embodiment of this application, the repair scheme module 730 is further configured to retrieve N historical problem vectors similar to the problem vector from the historical vector library based on similarity retrieval; retrieve at least one historical repair scheme associated with each historical problem vector and the corresponding historical repair effect evaluation value; calculate the expected success rate of each historical repair scheme based on the frequency of occurrence of each historical repair scheme and using the corresponding historical repair effect evaluation value as the weight; and obtain a repair scheme by comparing the expected success rates of each historical repair scheme.

[0106] According to embodiments of this application, any multiple modules among the intent inference module 710, test module 720, and repair solution generation module 730 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the intent inference module 710, test module 720, and repair solution generation module 730 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), programmable logic array (PLA), system-on-a-chip, system-on-a-substrate, system-on-package, application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the intent inference module 710, test module 720, and repair solution generation module 730 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0107] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a user interface testing method according to an embodiment of this application.

[0108] like Figure 8 As shown, an electronic device 800 according to an embodiment of this application includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage portion 809 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0109] RAM 803 stores various programs and data required for the operation of electronic device 800. Processor 801, ROM 802, and RAM 803 are interconnected via bus 804. Processor 801 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0110] According to embodiments of this application, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to a bus 804. The electronic device 800 may also include one or more of the following components connected to the input / output (I / O) interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 809 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 809 as needed.

[0111] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0112] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 802 and / or RAM 803 and / or one or more memories other than ROM 802 and RAM 803 described above.

[0113] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the user interface testing method provided in the embodiments of this application.

[0114] When the computer program is executed by the processor 801, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0115] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 809, and / or installed from a removable medium 811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0116] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, it performs the functions defined in the system of this application embodiment. According to the embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0117] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0118] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. Furthermore, each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0119] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A user interface testing method, characterized in that, include: Based on a pre-built user interface component model library, intent inference is performed on the initial abstract syntax tree node of the code to be tested to obtain the target abstract syntax tree node. The target abstract syntax tree node includes a variable field, a user interface component type field, and a business intent field, wherein the business intent field represents the business meaning at the user interface level. Based on the target abstract syntax tree node and the code to be tested, the target user interface is tested, and the test results are output. In the event of a test failure, an execution snapshot graph is constructed based on the target abstract syntax tree node and the test data in the test results to obtain a repair solution. The execution snapshot graph includes the target abstract syntax tree node, document object model nodes representing the user interface component entity, state nodes representing the available state of the user interface component entity, and edges representing the relationships between the nodes.

2. The method according to claim 1, characterized in that, The intent inference performed on the initial abstract syntax tree node of the code to be tested to obtain the target abstract syntax tree node includes: Extract the fields from the initial abstract syntax tree node and use the fields as the variable fields; The field is segmented according to a preset naming rule to obtain multiple segmented fragments; Each segmented word fragment is matched with a standardized user interface component type in the user interface component model library. The segmented word fragments that match successfully are used as the user interface component type field, and the segmented word fragments that do not match successfully are used as the business intent field. Based on the variable field, the user interface component type field, and the business intent field, the target abstract syntax tree node is formed.

3. The method according to claim 1, characterized in that, The process of inferring intent from the initial abstract syntax tree node of the code to be tested to obtain the target abstract syntax tree node also includes: Extract the fields from the initial abstract syntax tree node and use the fields as the variable fields; Extract the context information of the field, wherein the context information includes the set of methods called by the field and the locator call type corresponding to the field; Based on the user interface component model library, the method set is analyzed to obtain a candidate user interface component type field; the locator call type is analyzed to determine the user interface component type field from the candidate user interface component type field; Based on the method set and the business scenario corresponding to the locator call type, the business intent field representing the business meaning at the user interface level is obtained. Based on the variable field, the user interface component type field, and the business intent field, the target abstract syntax tree node is formed.

4. The method according to claim 1, characterized in that, The process of inferring intent from the initial abstract syntax tree node of the code to be tested to obtain the target abstract syntax tree node also includes: Extract and standardize the test code corresponding to the nodes of the initial abstract syntax tree to obtain standard data; Based on the user interface component model library, the standard data is semantically parsed using a pre-trained large model to obtain the target abstract syntax tree node, which includes the variable field, the user interface component type field, and the business intent field.

5. The method according to claim 1, characterized in that, In the event of test failure, the remediation plan includes constructing an execution snapshot graph based on the target abstract syntax tree node and the test data in the test results, which includes: Problem vectors are extracted based on the execution snapshot graph, wherein the problem vectors represent the reasons for test failure; Based on the problem vector, a repair solution is obtained by searching the historical vector library based on vector similarity. The problem vector and the repair solution are then stored in the historical vector library, which contains N historical problem vectors and their corresponding at least one repair solution and repair effect evaluation value, where N is a positive integer greater than or equal to 1.

6. The method according to claim 5, characterized in that, The extraction of the problem vector based on the execution snapshot map includes: Based on the target abstract syntax tree node corresponding to the test failure, find the corresponding target document object model node; If the target document object model node is not found, the test failure will be determined as a locator failure. If the target document object model node is found, check the target status node corresponding to the target document object model node. If the target status node is abnormal, the reason for the test failure is determined to be a status error. If the target state node is normal, check the parent node of the target abstract syntax tree node and the parent node of the target document object model node; The problem vector is obtained based on the execution snapshot map and the reasons for test failure.

7. The method according to claim 5, characterized in that, The step of retrieving a repair solution from a historical vector database based on vector similarity, according to the problem vector, includes: Based on similarity retrieval, N historical question vectors similar to the question vector are obtained from the historical vector library; Obtain at least one historical remediation scheme and its corresponding historical remediation effect evaluation value associated with each of the historical problem vectors; Based on the frequency of occurrence of each historical repair solution, and using the corresponding historical repair effect evaluation value as a weight, the expected success rate of each historical repair solution is calculated. A repair plan is obtained by comparing the expected success rate of each of the historical repair plans.

8. A user interface testing device, characterized in that, The device includes: The intent inference module is used to infer the intent of the initial abstract syntax tree node of the code to be tested based on a pre-built user interface component model library, and obtain the target abstract syntax tree node. The target abstract syntax tree node includes a variable field, a user interface component type field, and a business intent field, wherein the business intent field represents the business meaning at the user interface level. The testing module is used to test the target user interface based on the target abstract syntax tree nodes and the code to be tested, and output the test results; and The module for obtaining a repair solution is used to construct an execution snapshot graph based on the target abstract syntax tree node and the test data in the test results in the event of test failure, thereby obtaining a repair solution. The execution snapshot graph includes the target abstract syntax tree node, document object model nodes representing the user interface component entity, state nodes representing the available state of the user interface component entity, and edges representing the relationships between the nodes.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images