Interface element positioning method, device and equipment for automated testing, and storage medium
By generating an ordered sequence of interface elements and using structured query objects for element location, the fragility and maintenance challenges of element location in complex front-end interfaces in existing technologies are solved, achieving stable and accurate interface element location and simplified maintenance.
Patent Information
- Application Number
- CN202611629584.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-21
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies for element positioning in complex front-end interfaces are prone to fragility, difficulty in maintenance, and poor reusability, making it difficult to achieve stable and accurate element positioning without relying on volatile page structure details.
By acquiring snapshot information, an ordered sequence of interface elements is generated. Relationship symbols are scanned and identified character by character, parsed into structured query objects, and element searches are performed using logical sequence views. Direct name, anchor relative, and context anchor queries are supported, reducing the coupling between modules and simplifying maintenance.
It achieves stability and accuracy in element positioning when the page layout is adjusted, reduces maintenance complexity, improves the independence and scalability of the positioning process, and adapts to different traversal modes without modifying the original element sequence.
Smart Images

Figure CN123114989A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of software testing, and in particular to a method, apparatus, device, and storage medium for locating interface elements in automated testing. Background Technology
[0002] In existing technologies, front-end automated testing often uses methods such as fixed coordinates, DOM paths, XPath, CSS Selectors, control tree paths, and attribute combination matching to locate elements. These methods can meet basic needs in scenarios with stable interface structures, clear element attributes, and minimal page layout changes. However, in real business systems, front-end interfaces often involve dynamic rendering, repeated tags, frequent layout adjustments, changes in control hierarchy, and a large number of similar elements. This leads to traditional location methods being prone to fragility, difficulty in maintenance, and poor reusability.
[0003] With the development of automated testing technology, element location techniques have evolved from early coordinate positioning and simple attribute matching to location methods based on DOM trees, accessibility trees, control attribute sets, and multi-condition combinations. Current mainstream automated testing frameworks generally support mechanisms such as XPath, CSS Selectors, text matching, attribute filtering, and hierarchical traversal to improve element finding capabilities on complex pages.
[0004] From the perspective of the current technology, there is an urgent need to provide a new interface element positioning technology solution to express the element positioning intent in automated testing more stably and clearly without relying on the volatile page structure details, and to improve the positioning accuracy, execution stability and subsequent maintenance efficiency in complex front-end scenarios. Summary of the Invention
[0005] This application aims to at least solve the technical problems existing in the prior art, and to provide a method, apparatus, device and storage medium for locating interface elements in automated testing.
[0006] In a first aspect, the present invention provides a method for locating interface elements in automated testing, the method comprising: Acquire and parse snapshot information to generate an ordered sequence of interface elements, wherein the interface elements include at least a name field and a type field; The system receives a location statement and performs a character-by-character scan to identify unescaped relational characters. Based on these relational characters, the system splits the location statement into multiple segments, including direction relational characters and sequence number relational characters. The fragment is parsed to generate a structured query object; the structured query object includes a traversal pattern field, a query type field, an anchor selector field, a target type selector field, and a direction field; A logical sequence view is established based on the traversal mode field, and the logical sequence view accesses the sequence of interface elements through the mapping relationship between logical indexes and actual indexes; The target interface element is obtained by performing an element search in the logical sequence view based on the structured query object.
[0007] By adopting the above technical solution, the snapshot information of the front-end interface is parsed into an ordered sequence of interface elements, and the name field and type field are used as the basic descriptive dimensions of the elements. This makes the positioning process independent of the absolute hierarchical path of the elements in the DOM tree structure. When the page layout is adjusted, the container nesting changes, or the class name is changed, the positioning effectiveness can still be maintained as long as the display name and type of the interface element remain unchanged.
[0008] By scanning character by character to identify unescaped relational characters in the location statement and splitting them into segments, this method can more accurately handle scenarios where the element name contains the same character as the relational character compared to the overall regular expression matching method, avoiding parsing errors caused by character ambiguity.
[0009] Parsing the location statement into a structured query object decouples the statement parsing from the element lookup in the data flow. The execution layer only needs to perform the lookup operation based on the fields in the query object without being aware of the original syntax of the location statement. This reduces the coupling between modules and facilitates the independent maintenance and expansion of the parsing logic and the execution logic.
[0010] By establishing a logical sequence view through the traversal mode field, the mapping relationship between the logical index and the real index is used to access the sequence of interface elements. This eliminates the need for physical reversal of the original element sequence in non-sequential access modes such as reverse traversal, avoiding the side effects of modifying the sequence in place. At the same time, it provides a unified access interface for different traversal directions.
[0011] The introduction of directional and ordinal relational operators enables positioning statements to express relative positioning relationships based on business semantics and element order, such as "the Nth element of a specified type after a certain anchor element," overcoming the limitation of existing path-based positioning technologies that cannot concisely express the positioning intent of the combination of direction and ordinal.
[0012] Optionally, during the character-by-character scanning process, escape status markers are maintained. When an escape marker is scanned, the escape marker and its subsequent characters are marked as escaped. In the escaped state, relational characters are retained as literal text and are not recognized as valid relational characters. In the non-escaped state, directional relational characters and ordinal relational characters are recognized as valid relational characters and their positions and orders in the positioning statement are recorded.
[0013] By adopting the above technical solution, when the name field of an interface element contains a character that is the same as a direction relation or sequence relation, testers can mark the character using escape tags. The parser then treats this character as part of the name text rather than a syntax separator, thus correctly parsing the element name containing the special character. Compared to directly splitting characters according to their meaning, the escape state marking mechanism expands the applicability of location statements to complex element names and reduces fragment splitting errors caused by name character ambiguity. Simultaneously, recognizing valid relational characters only in their non-escaped state and recording their position and order in the location statement ensures the accuracy of relational character recognition results, providing a reliable positional information foundation for subsequent semantic parsing of fragments.
[0014] Optionally, before the character-by-character scanning, the positioning statement is preprocessed: leading and trailing spaces are removed, it is determined whether it is empty, and it is determined whether the first character is a reverse traversal identifier. If the first character is a reverse traversal identifier, the traversal mode field is set to reverse traversal mode and the identifier character is removed.
[0015] By adopting the above technical solution, the empty operation can intercept invalid input actions and return an error in advance when the positioning statement is an empty character, thus preventing empty statements from entering the subsequent scanning process and reducing invalid processing. The identification of reverse traversal markers moves the determination of the traversal mode from the query execution stage to the preprocessing stage, so that subsequent character-by-character scanning and fragment parsing can be uniformly processed based on the determined traversal mode, without having to determine the traversal direction again when performing the search, thereby simplifying the logical branches of the execution layer.
[0016] Optionally, before parsing the fragment, the query type is determined based on the number of directional relational characters: if no directional relational character is present, it is determined to be a direct name query; if one directional relational character is present, it is determined to be an anchor relative query; if two directional relational characters are present, it is determined to be a context anchor query; if the number of directional relational characters exceeds a preset upper limit, the positioning statement is determined to be an illegal statement.
[0017] By adopting the above technical solution, under the above rules, there is no need to pre-judge the semantic content of the positioning statement. It only relies on the objective information of the number of identified relational symbols, and the overall judgment process is simple and efficient. When the number of directional relational symbols exceeds the threshold limit, the positioning statement can be identified as an illegal statement and intercepted in the early stage of parsing, thereby preventing invalid statements from entering the subsequent fragment parsing steps, thus reducing unnecessary resource waste and improving overall efficiency.
[0018] Optionally, the step of determining the query type based on the number of directional relational operators specifically includes the following sub-steps: When the query type is a direct name query, the structured query object only contains name matching information, and the anchor selector field, target type selector field, and direction field are empty; When the query type is anchor relative query, the anchor name of the anchor selector field is taken from the segment before the direction relation, the target type value of the target type selector field is taken from the segment after the direction relation, and the direction field is determined according to the type of the direction relation. When the query type is a context anchor query, the structured query object also includes a context static text field. The context static text field is taken from the segment before the first direction relation, the anchor selector field is taken from the segment between the two direction relation, and the target type selector field is taken from the segment after the second direction relation. The interface element type corresponding to the context static text field is limited to static text type and does not support the appearance of serial numbers.
[0019] By adopting the above technical solution, the execution layer can execute corresponding search strategies for fields of different query types without repeatedly parsing the valid fields of the query object during the execution process, thereby improving the overall adaptability.
[0020] Optionally, performing an element lookup in the logical sequence view includes: When the query type is a direct name query, the system scans from the beginning of the logical sequence view to find the interface element that matches the name field and returns the Nth matching element corresponding to the occurrence number. When the query type is anchor relative query, first find the anchor element whose name field matches the anchor name and whose serial number appears, and then use the anchor element as the reference to find the Mth target interface element whose type field matches the serial number appears in the direction indicated by the direction field. When the query type is a context anchor query, first find the context element whose type field is static text and whose name field matches the context static text field, then find the anchor element after the context element, and finally find the target interface element based on the anchor element and in the direction indicated by the direction field.
[0021] By adopting the above technical solutions, the three query types correspond to different search strategies: direct name queries distinguish elements with the same name by their serial numbers; anchor-relative queries implement a complete process of "anchor point positioning + direction search + serial number counting"; and context anchor queries complete the parsing of multiple layers of contextual relationships in a single positioning through a three-level progressive search. All three search methods are executed based on the same logical sequence view, eliminating the need to maintain different data access interfaces for different query types. In reverse traversal mode, the search direction is automatically reversed through the index mapping of the logical sequence view, requiring no additional processing.
[0022] Secondly, the present invention provides an automated testing interface element positioning system, the system comprising: The snapshot parsing module is used to obtain snapshot information of the target front-end interface and parse it into an ordered sequence of interface elements; The statement receiving module is used to receive positioning statements that describe the relationship between a target interface element and at least one anchor element. The parsing module is used to perform a character-by-character scan of the positioning statement to identify unescaped relational characters, and to split the positioning statement into multiple segments based on the relational characters and perform semantic parsing on the segments; The query object generation module is used to generate a structured query object based on the parsing results, including a traversal pattern field, a query type field, an anchor selector field, a target type selector field, and a direction field. A logical view construction module is used to construct a logical sequence view based on the traversal mode field. The logical sequence view accesses the sequence of interface elements through the mapping relationship between logical indexes and actual indexes. The element location module is used to perform element lookup in the logical sequence view based on the structured query object; The results output module is used to output target interface elements or error information.
[0023] By adopting the above technical solution, modular design decouples the parsing of the location statement from element lookup at the module level. The structured query object output by the parsing module serves as a standardized data interface between the query object generation module and the element location module. When the syntax of the location statement needs to be expanded or the structure of the query object needs to be adjusted, only the parsing module and the query object generation module need to be modified; the element location module remains unchanged. The logical view construction module is independent of the element location module, separating the implementation of the index mapping mechanism from the lookup logic, facilitating the reuse of the same set of lookup code for different traversal modes. The result output module uniformly encapsulates the output interface for location results and error messages, so that external callers do not need to concern themselves with the internal lookup process.
[0024] Optionally, the parsing module maintains escape status flags during character-by-character scanning, retains subsequent characters as literal text when an escape flag is detected, and identifies valid relational characters in the non-escaped state; the query object generation module determines the query type based on the number of directional relational characters; the logical view construction module sets the logical index to be equal to the actual index in normal traversal mode, and sets the actual index to the sequence length minus one minus the logical index in reverse traversal mode; the element positioning module performs direct name lookup, anchor relative lookup, or context anchor lookup according to the query type.
[0025] By adopting the above technical solutions, the query object generation module determines the query type based on the number of directional relational operators, enabling automatic query type determination without external intervention. The logical view construction module sets the logical index to equal the actual index in normal traversal mode and sets the actual index to the sequence length minus the logical index in reverse traversal mode. It provides a clear index mapping formula, allowing reverse traversal to be achieved through index calculation rather than sequence reversal, thus avoiding modification of the original element sequence. The element positioning module performs direct name lookup, anchor-relative lookup, or context anchor lookup based on the query type. These three lookup methods share the same logical sequence view interface, enabling the lookup logic for different query types to be implemented at a unified access layer, reducing implementation complexity and maintenance costs.
[0026] Thirdly, the present invention provides an electronic device, the electronic device comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the interface element positioning method for automated testing described above.
[0027] Fourthly, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the interface element positioning method for automated testing described above.
[0028] In summary, this application includes the following beneficial technical effects: The UI element sequence is generated based on snapshot parsing. The location is determined by the logical order and business semantic relationships of the elements within the sequence, rather than their absolute hierarchical paths in the DOM tree. When the page structure is adjusted or the node hierarchy changes, the location statements remain valid as long as the element names and type information remain unchanged.
[0029] Structured location statements can concisely express multi-level relationships such as "the Nth target type after the anchor" and "context → anchor → target". The location semantics are consistent with the operational logic of testers, reducing the complexity of writing and maintaining the code. Direct name location, anchor relative location, context anchor location, ordinal location, and reverse traversal location are all unified into the structured query object framework, avoiding the need to write different code logic for different location requirements.
[0030] Reverse traversal is implemented through the index mapping mechanism of the logical sequence view. The data structure of the original interface element sequence is not modified, avoiding the side effects of in-place reversal on shared data and making it safer in concurrent scenarios. The parsing and execution processes of the positioning statement are independent of each other. The parsing layer is responsible for converting the string into a structured query object, and the execution layer is responsible for searching on the sequence based on the query object. The two can evolve and be reused independently. Attached Figure Description
[0031] Figure 1 A flowchart illustrating a method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the location statement parsing process provided in an embodiment of the present invention; Figure 3 Here is a flowchart illustrating the execution of three query types according to an embodiment of the present invention; Figure 4 This is a system functional block diagram provided according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device for implementing the method according to an embodiment of the present invention.
[0032] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0033] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.
[0034] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.
[0035] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.
[0036] Reference Figure 1 and Figure 2 The diagram shown is a flowchart illustrating an automated testing interface element location method according to an embodiment of the present invention. In this embodiment, the automated testing interface element location method includes: S100: Obtain and parse snapshot information to generate an ordered sequence of interface elements, each of which must include at least a name field and a type field. Specifically, snapshot information of the target front-end interface is obtained through automated testing tools or interface capture interfaces. Snapshot information can originate from one or more of the following data sources: Browser Render Tree: Page rendering information generated by the browser's rendering engine; Control Tree: Information about the hierarchical relationship of controls in desktop or hybrid applications; Accessibility Tree: Information about UI elements provided by the system's accessibility interface; Interface text descriptions collected by the testing system: The text and attribute information of interface elements obtained by the automated testing framework through interface calls.
[0037] It is important to note that this embodiment does not directly traverse the HTML / DOM source code of the webpage, but instead obtains the interface element information through the snapshot parsing method described above. This data acquisition method allows this solution to be independent of the webpage DOM structure and applicable to various front-end forms such as browser interfaces, desktop application interfaces, and hybrid interfaces.
[0038] After parsing the snapshot information into an ordered sequence of UI elements, each UI element object includes at least the following fields: Name field: The text label, display name, or business semantic name of the interface element. For example, the "Approval Amount (Yuan):" label and the "Business Date:" label in a form; Type field: The control type of the interface element, such as static text (StaticText), input box (TextBox), button (Button), drop-down list (ComboBox), check box (CheckBox), etc.
[0039] In addition, each UI element object may include one or more of the following additional fields: control state field (indicating the element's enabled / disabled, visible / hidden, etc.), display order field (indicating the element's display order in the UI), coordinate position field (indicating the element's screen coordinate position), and hierarchy information field (indicating the element's hierarchy in the control tree).
[0040] The order of the interface element sequence can be arranged according to the order in which the elements were captured in the snapshot, or according to preset rules (such as visual reading order from top to bottom or from left to right).
[0041] S200: Receives the location statement and performs a character-by-character scan, identifies the unescaped relational characters, and splits the location statement into multiple segments based on the relational characters. The relational characters include direction relational characters and sequence number relational characters. Wherein: the positioning statement comes from the test script or external input. In this embodiment, the positioning statement is a string used to describe the relationship between the target interface element and at least one anchor element.
[0042] In a preferred embodiment of this example, the location statement supports the following syntax (the symbols in the following syntax are only preferred embodiments and do not constitute a limitation on the scope of protection):
[0043] The above syntax symbols (>, <, +) are only preferred embodiments of this invention, and the scope of protection of this invention is not limited to the use of these specific symbols. In other embodiments, other characters or symbol combinations can be used as direction relation symbols, sequence relation symbols, and reverse traversal identifiers.
[0044] Step S200 also includes the following sub-steps: S210: Perform preprocessing on the location statement.
[0045] Specifically, it removes leading and trailing spaces, checks if the string is empty, and determines if the first character is a reverse traversal identifier. If the first character is a reverse traversal identifier, it sets the traversal mode field to reverse traversal mode and removes the identifier character. After removing the reverse traversal identifier character, it continues with subsequent scanning.
[0046] S220: Character-by-character scan and location statement.
[0047] Specifically, during the character-by-character scanning process, escape status markers are maintained. When an escape marker is scanned, the escape marker and its subsequent characters are marked as escaped. In the escaped state, relational characters are preserved as literal text and are not recognized as valid relational characters. In the non-escaped state, directional relational characters and ordinal relational characters are recognized as valid relational characters and their positions and orders in the positioning statement are recorded.
[0048] For example, each character of the positioning statement is read character by character; when an escape character is detected, the escape character and its subsequent characters are marked as escaped. In the escaped state, subsequent directional relational characters (> or <) and ordinal relational characters (+) are not recognized as valid relational characters, but are retained as literal text content; In the non-escaped state, when the direction relation (> or <) and the sequence relation (+) are recognized, they are recorded as valid relation symbols, and their positions in the positioning statement are recorded. When escaping is not enabled, all relation characters in the positioning statement are treated as unescaped relation characters, meaning that all directional and ordinal relation characters are recognized as valid relation characters.
[0049] After scanning, the order and position information of each valid relation symbol are obtained. The role of the escaping mechanism in this embodiment is to prevent these characters from being misidentified as relation symbols when the name of an interface element itself contains characters that are the same as the syntax symbols (for example, the element name is "Amount>Maximum" or "A+B"). This ensures the correct parsing of the positioning statement.
[0050] Furthermore, a more detailed explanation of the escape handling mechanism is provided: In real-world testing scenarios, the names of UI elements may contain characters that are identical to the syntax symbols in locator statements. For example, an interface might have a label named "Amount > Maximum" and a button named "A+B". If these names are used directly in a locator statement, the > and + characters will be misinterpreted as direction and sequence symbols, leading to parsing errors.
[0051] To address this issue, this invention provides an escaping mechanism, specifically implemented as follows: During character-by-character scanning, an escape status flag `isEscaped` is maintained, initially set to false. When an escape character (in this embodiment, a backslash `\` is used as the escape flag, but it is not limited to this) is encountered, `isEscaped` is set to true; the next character is scanned, and this character (regardless of whether it is a relational character) is retained as literal text content in the current segment; `isEscaped` is then reset to false. When relational characters (>, <, +) are encountered in the non-escaped state, they are correctly identified as valid relational characters. After scanning is complete, the escape flag is removed during the segment parsing phase, restoring the escaped characters to ordinary text characters.
[0052] For example, when the positioning statement is "amount\>upper limit>textbox": the backslash after "amount" is scanned and escaped; the next character > is preserved as a literal in the escaped state and is not recognized as a direction relation character; the > after "upper limit" is recognized as a direction relation character in the uncaught state; the final parsing result is: anchor name="amount>upper limit", direction = backward, target type="textbox".
[0053] S230: Query type determination.
[0054] Specifically, the query type is determined by the number of directional relational operators: If no directional relational operator is included, it is determined to be a direct name query; A directional relation: the query type is anchor-relative query; Two directional relational operators: The query type is a context anchor query; The number of directional relational operators exceeds the preset limit (2 in this embodiment): the positioning statement is an illegal statement, and a parsing error is returned.
[0055] When the query type is a direct name query, the structured query object only contains name matching information, and the anchor selector field, target type selector field, and direction field are empty; When the query type is an anchor relative query, the anchor name in the anchor selector field is taken from the segment before the direction relation, the target type value in the target type selector field is taken from the segment after the direction relation, and the direction field is determined according to the type of the direction relation. When the query type is a context anchor query, the structured query object also includes a context static text field. The context static text field is taken from the segment before the first direction relation, the anchor selector field is taken from the segment between the two direction relations, and the target type selector field is taken from the segment after the second direction relation. The UI element type corresponding to the context static text field is limited to static text type and does not support the appearance of serial numbers.
[0056] S240: Fragment splitting and parsing.
[0057] Specifically, the positioning statement is split into multiple fragments based on the position of the valid relational characters, and the following parsing operations are performed on each fragment: remove spaces in the fragment; extract the ordinal relational characters and ordinal values (if they exist) in the fragment; restore the escaped characters to their literal meanings; and perform validity checks on the fragment (the fragment cannot be empty, the ordinal number must be a positive integer, etc.).
[0058] After parsing, each fragment is converted into a corresponding selector object: the name fragment is converted into a name selector object (containing the name value and the occurrence number, which is 1 by default); the type fragment is converted into a type selector object (containing the type value and the occurrence number, which is 1 by default); and the context first fragment is converted into a static text name selector (the interface element type corresponding to this selector is limited to static text type, and the occurrence number setting is not supported).
[0059] S250: Semantic validation.
[0060] Perform semantic validation on the parsed results: check if all segments are non-empty; check if all occurrence numbers are positive integers; check if numbers are used in illegal positions; check if the first direction relation in the context anchor query meets the preset rules; check if escape characters have been correctly restored to plain text. If the semantic validation fails, return a parsing error message.
[0061] S300: Parse the fragment and generate a structured query object; Specifically, structured query objects include traversal pattern fields, query type fields, anchor selector fields, target type selector fields, and direction fields;
[0062] The traversal mode field is set to normal traversal mode by default. When the preprocessing stage (S210) detects that the first character of the location statement is a reverse traversal identifier, the traversal mode field is set to reverse traversal mode. If the location statement is not preprocessed, the traversal mode field retains its default value, which is normal traversal mode.
[0063] Specifically, the query type determination results (S230) and fragment parsing results (S240) are summarized into a structured query object. The valid field composition of the query object is determined based on the query type field: when the query type is a direct name query, the name fragment is parsed into name matching information, and the remaining fields are left blank; when the query type is an anchor-relative query, the fragment before the direction relation is parsed into an anchor selector field (including the anchor name and anchor occurrence number), and the fragment after the direction relation is parsed into a target type selector field (including the target type value and target occurrence number), with the direction field set according to the type of the direction relation; when the query type is a context anchor query, the anchor selector field and target type selector field are set in the above manner, and a context static text field is added. The traversal mode field is set based on the preprocessing results.
[0064] S400: Create a logical sequence view based on the traversal mode field.
[0065] Specifically, the logical sequence view accesses the sequence of interface elements through the mapping relationship between logical indexes and actual indexes; It is worth noting that a logical sequence view is an access view of a sequence of interface elements. Its core feature is that it does not modify the data structure of the original sequence of interface elements, but accesses the elements in the sequence through the mapping relationship between logical indexes and actual indexes.
[0066] Normal traversal mode: The logical index is equal to the real index, i.e., logicalIndex = realIndex. Starting from logical index 0, which corresponds to real index 0, i.e., the first element of the sequence, we visit the elements sequentially.
[0067] Reverse traversal mode: The real index is calculated using the following mapping formula: realIndex = length - 1 - logicalIndex, where length is the length of the sequence of interface elements. Starting from logical index 0, corresponding to the real index length - 1, which is the last element of the sequence, the actual index is moved forward according to the logical order.
[0068] This design avoids directly performing a reversal operation on the original sequence of interface elements (such as reversing an array in place), thus avoiding modification of the state of the shared data structure and avoiding side effects in multi-threaded or concurrent access scenarios; logical sequence views can be created and destroyed at any time without affecting the original data; the same element sequence can support both ascending and descending order access methods simultaneously, without the need to maintain two copies of the data.
[0069] Reference Figure 1 and Figure 3 S500: Perform an element search in the logical sequence view based on the structured query object to obtain the target interface element.
[0070] Different search logics are executed depending on the query type: Direct Name Query: When the query type is a direct name query, the scan starts from the beginning of the logical sequence view (logical index 0), searching for UI elements whose name field matches the name matching information in the structured query object, and returning the Nth matching element corresponding to the occurrence sequence number. For example, when the locating statement is "Submit Button", the scan starts from the beginning of the sequence and searches for the first UI element with the name "Submit Button".
[0071] Anchor point relative query: When the query type is an anchor-relative query, perform the following steps: In the logical sequence view, find anchor elements whose names match the anchor names in the name field and anchor selector field, and whose occurrence numbers correspond to the anchor occurrence numbers. Using the anchor element as a reference, search for interface elements whose type field matches the target type value in the direction indicated by the direction field (forward or backward); return the Mth matching interface element corresponding to the target occurrence number.
[0072] For example, when the locator is "Approval Amount (Yuan):>textbox": it finds the first element named "Approval Amount (Yuan):" as the anchor point, and then finds the first UI element of type textbox after that anchor point. As another example, when the locator is "Approval Amount (Yuan):+2>textbox+3": it finds the second element named "Approval Amount (Yuan):" as the anchor point, and then finds the third UI element of type textbox after that anchor point.
[0073] Context anchor query: When the query type is a context anchor query, the following steps are executed: Search the logical sequence view for a context element whose type field is StaticText and whose name field matches the context static text field; search after the context element for an anchor element whose name field matches the anchor name of the anchor selector field, and whose occurrence sequence number corresponds to the anchor occurrence sequence number; Using the anchor element as a reference, search in the direction indicated by the direction field for the M-th target interface element whose type field matches the target type value and whose occurrence sequence number corresponds to the target occurrence sequence number.
[0074] For example, when the positioning statement is "Business Date: >Home<StaticText": search for a context element whose type is static text and whose name is "Business Date:", search for an anchor element named "Home" after the context element, and then search for the first interface element of type StaticText before the anchor element.
[0075] S600: Output the search result.
[0076] Specifically, when the search is successful, the information of the target interface element is output, for example: the index, name, type, etc. of the element in the sequence, for subsequent test operations by the automated test executor, such as clicking, inputting, value retrieval, status check, etc. When the search fails, positioning failure information is output.
[0077] In addition, if a syntax or semantic error is found in the positioning statement during the parsing phase, parsing error information is output. Parsing errors and positioning failures are two different types of errors: a parsing error refers to that the positioning statement itself is invalid, such as an empty statement, missing fragments, an invalid number of relational operators, an invalid sequence number, a semantic structure that does not meet the rules, etc.; a positioning failure refers to that the positioning statement is valid, but no element meeting the conditions is found in the current interface element sequence. Distinguishing these two types of errors helps testers quickly locate the source of the problem: a parsing error requires modifying the positioning statement, and a positioning failure requires checking the actual content of the tested interface.
[0078] Refer to Figure 4 , based on the same inventive concept, an embodiment of the present invention provides an interface element positioning system for automated testing, comprising: A snapshot parsing module, configured to acquire snapshot information of a target front-end interface and parse it into an ordered sequence of interface elements; A statement receiving module, configured to receive a positioning statement that describes the relationship of a target interface element relative to at least one anchor element; The parsing module is used to perform a character-by-character scan of the location statement to identify unescaped relational characters, and to split the location statement into multiple fragments based on the relational characters and perform semantic parsing on the fragments; The query object generation module is used to generate a structured query object based on the parsing results, including a traversal pattern field, a query type field, an anchor selector field, a target type selector field, and a direction field. The logical view construction module is used to construct a logical sequence view based on the traversal mode field. The logical sequence view accesses the sequence of interface elements through the mapping relationship between logical index and real index. Specifically, when the traversal mode is normal traversal, the logical index is set to be equal to the real index; when the traversal mode is reverse traversal, the real index is set to the length of the interface element sequence minus one minus the logical index. The element locator module is used to perform element lookup in the logical sequence view based on the structured query object; specifically, it is used to perform direct name lookup, anchor relative lookup, or context anchor lookup based on the query type.
[0079] The results output module is used to output target interface elements or error information.
[0080] Error handling module: The error handling module is used to return parsing error information when the location statement contains an empty segment, the sequence number is not a positive integer, the number of directional relational characters exceeds the preset limit, or the semantic structure does not meet the preset rules, and to return location failure information when the location statement is parsed as valid but no target interface element matching the conditions is found.
[0081] The parsing module maintains escape status flags during character-by-character scanning. When an escape flag is detected, the subsequent characters are retained as literal text, and valid relational characters are identified in the non-escaped state. The query object generation module determines the query type based on the number of directional relational characters. The logical view construction module sets the logical index to be equal to the actual index in normal traversal mode, and sets the actual index to the sequence length minus one minus the logical index in reverse traversal mode. The element location module performs direct name lookup, anchor relative lookup, or context anchor lookup based on the query type.
[0082] The automated testing interface element positioning system described in this invention can be installed in an electronic device. The module described in this invention, also referred to as a unit, refers to a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, and are stored in the memory of the electronic device.
[0083] The various variations and specific examples of the interface element location method for automated testing provided in the above embodiments are also applicable to the interface element location system for automated testing in this embodiment. Through the foregoing detailed description of the interface element location method for automated testing, those skilled in the art can clearly understand the implementation method of the interface element location system for automated testing in this embodiment. For the sake of brevity, it will not be described in detail here.
[0084] The present invention also discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the interface element positioning method for automated testing provided by the present invention. The computer program product should be understood as a software product that mainly implements its solution through a computer program, such as a program product integrated in the cloud or a software library.
[0085] This application also discloses an electronic device, such as Figure 5 The diagram shown is a structural schematic of an electronic device for an automated testing interface element positioning method according to an embodiment of the present invention. The electronic device may include at least one processor, a memory communicatively connected to the at least one processor, a communication bus, and a communication interface, and may also include a computer program stored in the memory and executable on the processor.
[0086] In some embodiments, the processor may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits packaged with the same or different functions. This includes combinations of one or more Central Processing Units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor is the control unit of the electronic device, connecting various components of the device through various interfaces and lines. It executes programs or modules stored in the memory and calls data stored in the memory to perform various functions and process data within the electronic device.
[0087] The memory includes at least one type of readable storage medium, including flash memory, portable hard drives, multimedia cards, card-type memory (e.g., SD or DX memory), magnetic storage, magnetic disks, optical disks, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, FlashCard, etc. Furthermore, the memory can include both internal and external storage units of the electronic device. The memory can be used not only to store application software and various types of data installed on the electronic device, such as the code of an automated testing interface element positioning method program, but also to temporarily store data that has been output or will be output.
[0088] The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory and at least one processor, etc.
[0089] The communication interface is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.
[0090] Figure 5 Only electronic devices with components are shown; those skilled in the art will understand that... Figure 5The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.
[0091] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to at least one processor via a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power sources, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be elaborated further here.
[0092] It should be understood that the embodiments are for illustrative purposes only and are not limited to this structure in the scope of the patent application.
[0093] Furthermore, if the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile.
[0094] This application provides a computer-readable storage medium, including, for example, any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM). The computer-readable storage medium stores a computer program capable of being loaded by a processor and executing the interface element location method for automated testing described in the above embodiments.
[0095] In the description of this specification, the references to terms such as "an embodiment," "some embodiments," "example," "specific example," "a implementation," "a preferred implementation," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0096] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.
Claims
1. A method for locating interface elements in automated testing, characterized in that, The method includes: Acquire and parse snapshot information to generate an ordered sequence of interface elements, wherein the interface elements include at least a name field and a type field; The system receives a location statement and performs a character-by-character scan to identify unescaped relational characters. Based on these relational characters, the system splits the location statement into multiple segments, including direction relational characters and sequence number relational characters. The fragment is parsed to generate a structured query object; the structured query object includes a traversal pattern field, a query type field, an anchor selector field, a target type selector field, and a direction field; A logical sequence view is established based on the traversal mode field, and the logical sequence view accesses the sequence of interface elements through the mapping relationship between logical indexes and actual indexes; The target interface element is obtained by performing an element search in the logical sequence view based on the structured query object.
2. The method for locating interface elements in automated testing as described in claim 1, characterized in that, During the character-by-character scanning process, escape status markers are maintained. When an escape marker is scanned, the escape marker and its subsequent characters are marked as escaped. In the escaped state, relational characters are retained as literal text and are not recognized as valid relational characters. In the non-escaped state, directional relational characters and sequence relational characters are recognized as valid relational characters and their positions and orders in the positioning statement are recorded.
3. The method for locating interface elements in automated testing as described in claim 2, characterized in that, Before the character-by-character scan, the positioning statement is preprocessed: leading and trailing spaces are removed, it is determined whether it is empty, and it is determined whether the first character is a reverse traversal identifier. If the first character is a reverse traversal identifier, the traversal mode field is set to reverse traversal mode and the identifier character is removed.
4. The method for locating interface elements in automated testing as described in claim 1, characterized in that, Before parsing the fragment, the query type is determined based on the number of directional relational characters: if no directional relational character is present, it is determined to be a direct name query; if one directional relational character is present, it is determined to be an anchor relative query; if two directional relational characters are present, it is determined to be a context anchor query; if the number of directional relational characters exceeds a preset upper limit, the positioning statement is determined to be an illegal statement.
5. The method for locating interface elements in automated testing as described in claim 4, characterized in that, The step of determining the query type based on the number of directional relational operators specifically includes the following sub-steps: When the query type is a direct name query, the structured query object only contains name matching information, and the anchor selector field, target type selector field, and direction field are empty; When the query type is anchor relative query, the anchor name of the anchor selector field is taken from the segment before the direction relation, the target type value of the target type selector field is taken from the segment after the direction relation, and the direction field is determined according to the type of the direction relation. When the query type is a context anchor query, the structured query object also includes a context static text field. The context static text field is taken from the segment before the first direction relation, the anchor selector field is taken from the segment between the two direction relation, and the target type selector field is taken from the segment after the second direction relation. The interface element type corresponding to the context static text field is limited to static text type and does not support the appearance of serial numbers.
6. The method for locating interface elements in automated testing as described in claim 5, characterized in that, Performing an element lookup in the logical sequence view includes: When the query type is a direct name query, the system scans from the beginning of the logical sequence view to find the interface element that matches the name field and returns the Nth matching element corresponding to the occurrence number. When the query type is anchor relative query, first find the anchor element whose name field matches the anchor name and whose serial number appears, and then use the anchor element as the reference to find the Mth target interface element whose type field matches the serial number appears in the direction indicated by the direction field. When the query type is a context anchor query, first find the context element whose type field is static text and whose name field matches the context static text field, then find the anchor element after the context element, and finally find the target interface element based on the anchor element and in the direction indicated by the direction field.
7. An automated testing interface element positioning system, used to implement the automated testing interface element positioning method according to any one of claims 1 to 6, characterized in that, include: The snapshot parsing module is used to obtain snapshot information of the target front-end interface and parse it into an ordered sequence of interface elements; The statement receiving module is used to receive positioning statements that describe the relationship between a target interface element and at least one anchor element. The parsing module is used to perform a character-by-character scan of the positioning statement to identify unescaped relational characters, and to split the positioning statement into multiple segments based on the relational characters and perform semantic parsing on the segments; The query object generation module is used to generate a structured query object based on the parsing results, including a traversal pattern field, a query type field, an anchor selector field, a target type selector field, and a direction field. A logical view construction module is used to construct a logical sequence view based on the traversal mode field. The logical sequence view accesses the sequence of interface elements through the mapping relationship between logical indexes and actual indexes. The element location module is used to perform element lookup in the logical sequence view based on the structured query object; The results output module is used to output target interface elements or error information.
8. The interface element positioning system for automated testing as described in claim 7, characterized in that, The parsing module maintains escape status flags during character-by-character scanning. When an escape flag is detected, the subsequent characters are retained as literal text, and valid relational characters are identified in the non-escaped state. The query object generation module determines the query type based on the number of directional relational characters. The logical view construction module sets the logical index to be equal to the actual index in normal traversal mode, and sets the actual index to the sequence length minus one minus the logical index in reverse traversal mode. The element location module performs direct name search, anchor relative search, or context anchor search based on the query type.
9. An electronic device, characterized in that, The electronic device includes: At least one processor (10); and, A memory (11) communicatively connected to the at least one processor (10); The memory (11) stores a computer program that can be executed by the at least one processor (10) to enable the at least one processor (10) to perform the interface element positioning method for automated testing as described in any one of claims 1 to 6.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program; when the computer program is executed by a processor, it implements the interface element positioning method for automated testing as described in any one of claims 1 to 6.
