Automatic script determination method and system, electronic equipment and storage medium

Abstract

The invention provides an automatic script determination method and system, electronic equipment and a storage medium, and relates to the technical field of intelligent cockpit testing, interface information is obtained by analyzing a UI interface of to-be-tested equipment, a structure tree model is constructed based on the interface information, and a target element positioning instruction can be responded, so that the test efficiency is improved. The path information and the positioning information of the target element are determined from the structure tree model, so that the corresponding automatic script is recommended for the determined target element, the UI is combined with the structure tree model, the target element can be quickly positioned, the positioning time of the target element is shortened, and the user experience is improved. The corresponding automated script is automatically obtained from the preset script library according to the positioning information, so that the workload of testers is further reduced, the test process is smoother, and the error rate of manual script writing is also reduced.

Description

Technical Field

[0001] The present application relates to the field of intelligent cockpit testing technology, and in particular to a method, system, electronic device and storage medium for determining an automated script. Background Art

[0002] With the development of smart cockpit technology, more and more automakers are taking smart cockpits as the core function of their products, which has promoted the rise of the concept of "software-defined cars". In order to ensure the stability and reliability of the smart cockpit system, testers need to write a large number of test scripts to test the cockpit system. Therefore, how to improve the efficiency and accuracy of writing automated scripts has become an important issue faced by all automated testing teams.

[0003] At present, the common method for implementing automated script writing in the market is to obtain the element information of the device UI interface through Webitor (used to locate elements and debug code). Automated test engineers usually write scripts in IDE (Integrated Development Environment) or some open source automation frameworks (such as Robot Framework) and encapsulate elements to implement automated testing of operations such as click, long press, and double click.

[0004] However, the existing solutions have some shortcomings. Testers need to switch between Weditor (web page) and in-vehicle devices (Android system) to obtain the XPath path and attributes of the operation control, and manually copy this information to the IDE or open source automation framework. This frequent switching not only reduces the writing efficiency, but also requires testers to perform secondary processing on the obtained element information when relative positioning, node parent-child node or sibling node positioning is required, which places high demands on their skills and experience, and ultimately affects the writing efficiency and accuracy.

[0005] Therefore, improving the efficiency and accuracy of automated test script writing is a key issue that the current testing team urgently needs to solve. Summary of the invention

[0006] In view of the above problems, an embodiment of the present application provides a method for determining an automation script, the method comprising: Get the UI interface of the device to be tested; Parsing the UI interface to obtain a replica image and interface information of the UI interface; Constructing a structure tree model of the UI interface according to the interface information; In response to the triggered instruction to locate the target element, determining the path information and location information of the target element corresponding to the instruction to locate the target element in the structure tree model; According to the path information and the location information, an automation script of the target element is determined from a preset script library.

[0007] Optionally, the target element positioning instruction is an instruction triggered by searching for a target element in the structure tree model, or an instruction triggered by clicking a target element on the structure tree model, and the step of determining the path information and the location information of the target element corresponding to the target element positioning instruction in the structure tree model in response to the triggered target element positioning instruction includes: In response to a target element location instruction triggered by searching for a target element on the structure tree model, path information and location information of a target element corresponding to the target element location instruction in the structure tree model are determined, and the target element is highlighted; or, In response to a target element positioning instruction triggered by clicking a target element on the structure tree model, path information and positioning information of a target element corresponding to the target element positioning instruction in the structure tree model are determined.

[0008] Optionally, the method further comprises: Obtaining a replica image of the UI interface; The target element positioning instruction is an instruction triggered by clicking on the replica image of the UI interface, and the path information and positioning information of the target element corresponding to the target element positioning instruction in the structure tree model are determined in response to the triggered target element positioning instruction, including: In response to a target element positioning instruction triggered by clicking on a replica image of the UI interface, determining the coordinates of an interaction point corresponding to the target element positioning instruction, the interaction point being a trigger point on the replica image when the replica image is clicked; Determining the target element according to the coordinates of the interaction point; Obtaining attribute information of the target element; Element ID information is acquired from the attribute information of the target element, and path information and location information of the target element in the structure tree model are determined according to the element ID information.

[0009] Optionally, determining the target element according to the coordinates of the interaction point includes: According to the coordinates of the interaction point, calculate all elements containing the coordinates of the interaction point; Determine and select the smallest area element containing the coordinates of the interaction point from all elements containing the coordinates of the interaction point; The minimum area element is determined as the target element.

[0010] Optionally, determining the path information and location information of the target element in the structure tree model according to the element ID information includes: According to the element ID information, selecting the target element in the structure tree model; Acquire the attribute set of the target element from the structure tree model; Determine whether there is a unique attribute in the attribute set of the target element, wherein the unique attribute is an attribute with a unique attribute value; In the case that there is a unique attribute in the attribute set of the target element, the path information and the location information of the target element in the structure tree model are determined according to the unique attribute of the target element.

[0011] Optionally, the method further comprises: If there is no unique attribute in the attribute set of the target element, obtaining the attribute set of the upper-level element of the target element; Determine whether there is a unique attribute in the attribute set of the upper-level element; In the case where a unique attribute exists in the attribute set of the upper-level element, determining the path information and location information of the target element in the structure tree model according to the unique attribute of the upper-level element and the category and index of the target element; In the case that there is no unique attribute in the attribute set of the upper-level element, the attribute set of the upper-level element of the upper-level element continues to be acquired until the attribute set of the root element is acquired.

[0012] Optionally, parsing the UI interface to obtain interface information of the UI interface includes: Parsing the UI interface to obtain hierarchical structure information of the UI interface in XML format; Each node in the hierarchical structure information is respectively set with a corresponding key-value pair to obtain the interface information in JSON format, wherein each key-value pair is set with an element ID information; The step of constructing a structure tree model of the UI interface according to the interface information includes: The interface information in the JSON format is rendered according to a hierarchical structure to obtain a structure tree model of the UI interface.

[0013] Optionally, the method further comprises: According to the path information and location information of the target element in the structure tree model, respectively determine the path information and location information of the parent and child elements of the target element in the structure tree model, and the path information and location information of the sibling elements of the target element in the structure tree model; Determining the automation scripts of the parent and child elements from the preset script library according to the path information and location information of the parent and child elements in the structure tree model; According to the path information and location information of the brother element in the structure tree model, the automation script of the brother element is determined from the preset script library.

[0014] The second party of the present application provides a system for determining an automation script, the system comprising: The acquisition module is used to obtain the UI interface of the device to be tested; A parsing module, used to parse the UI interface to obtain a replica image and interface information of the UI interface; A construction module, used to construct a structure tree model of the UI interface according to the interface information; A first determination module, configured to determine, in response to a triggered instruction to locate a target element, path information and location information of a target element corresponding to the instruction to locate the target element in the structure tree model; The second determination module is used to determine the automation script of the target element from a preset script library according to the path information and the location information.

[0015] Optionally, the target element positioning instruction is an instruction triggered by searching for a target element in the structure tree model, or an instruction triggered by clicking a target element on the structure tree model, and in response to the triggered target element positioning instruction, path information and location information of the target element corresponding to the target element positioning instruction in the structure tree model are determined, and the first determination module includes: A first determination submodule is configured to, in response to a target element location instruction triggered by searching for a target element on the structure tree model, determine the path information and location information of the target element corresponding to the target element location instruction in the structure tree model, and highlight the target element; or The second determination submodule is used to determine the path information and location information of the target element corresponding to the target element location instruction in the structure tree model in response to the target element location instruction triggered by clicking the target element on the structure tree model.

[0016] Optionally, the system further comprises: A first acquisition submodule is used to acquire a replica image of the UI interface; The target element positioning instruction is an instruction triggered by clicking on the replica image of the UI interface. In response to the triggered target element positioning instruction, the path information and positioning information of the target element corresponding to the target element positioning instruction in the structure tree model are determined. The first determination submodule includes: A third determination submodule is used to determine the coordinates of an interaction point corresponding to the positioning target element instruction in response to the positioning target element instruction triggered by clicking the replica image of the UI interface, where the interaction point is a trigger point on the replica image when the replica image is clicked; A fourth determination submodule, used to determine the target element according to the coordinates of the interaction point; The second acquisition submodule is used to acquire the attribute information of the target element; The fifth determining submodule is used to obtain element ID information from the attribute information of the target element, and determine the path information and location information of the target element in the structure tree model according to the element ID information.

[0017] Optionally, the determining the target element according to the coordinates of the interaction point, the fourth determining submodule includes: A calculation subunit, configured to calculate all elements including the coordinates of the interaction point according to the coordinates of the interaction point; A first determining subunit is used to determine and select a minimum area element containing the coordinates of the interaction point from all elements containing the coordinates of the interaction point; The second determining subunit is used to determine the minimum area element as the target element.

[0018] Optionally, the determining, according to the element ID information, the path information and the location information of the target element in the structure tree model, the fifth determining submodule includes: A selection subunit is used to select the target element in the structure tree model according to the element ID information; An acquisition subunit, used to acquire the attribute set of the target element from the structure tree model; A first judgment subunit is used to judge whether there is a unique attribute in the attribute set of the target element, wherein the unique attribute is an attribute with a unique attribute value; The third determining subunit is configured to determine the path information and the location information of the target element in the structure tree model according to the unique attribute of the target element when there is a unique attribute in the attribute set of the target element.

[0019] Optionally, the system further comprises: A third acquisition submodule is used to acquire the attribute set of the parent element of the target element when there is no unique attribute in the attribute set of the target element; A judgment submodule, used to judge whether there is a unique attribute in the attribute set of the upper-level element; A sixth determination submodule, configured to determine the path information and location information of the target element in the structure tree model according to the unique attribute of the upper-level element and the category and index of the target element when there is a unique attribute in the attribute set of the upper-level element; The fourth acquisition submodule is used to, if there is no unique attribute in the attribute set of the upper-level element, continue to acquire the attribute set of the upper-level element until the attribute set of the root element is acquired.

[0020] Optionally, the UI interface is parsed to obtain interface information of the UI interface, and the parsing module includes: A parsing submodule, used to parse the UI interface to obtain the hierarchical structure information of the UI interface in XML format; A setting submodule is used to set a corresponding key-value pair for each node in the hierarchical structure information to obtain the interface information in JSON format, wherein each key-value pair is set with an element ID information; The structure tree model of the UI interface is constructed according to the interface information, and the construction module includes: The rendering submodule is used to render the interface information in the JSON format according to a hierarchical structure to obtain a structure tree model of the UI interface.

[0021] Optionally, the system further comprises: a seventh determination submodule, configured to determine the path information and location information of the parent and child elements of the target element in the structure tree model, and the path information and location information of the sibling elements of the target element in the structure tree model, respectively, according to the path information and location information of the target element in the structure tree model; An eighth determination submodule, configured to determine the automation scripts of the parent and child elements from the preset script library according to the path information and location information of the parent and child elements in the structure tree model; The ninth determination submodule is used to determine the automation script of the brother element from the preset script library according to the path information and location information of the brother element in the structure tree model.

[0022] In a third aspect of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method for determining an automation script as described in the first aspect of the present application.

[0023] In a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the method for determining the automation script as described in the first aspect of the present application is implemented.

[0024] Beneficial effects of this application: The present application provides a method for determining an automation script, the method comprising: obtaining a UI interface of a device to be tested; parsing the UI interface to obtain interface information of the UI interface; constructing a structure tree model of the UI interface according to the interface information; responding to a triggered instruction to locate a target element, determining the path information and location information of a target element corresponding to the instruction to locate the target element in the structure tree model; and determining the automation script of the target element from a preset script library according to the path information and the location information. The present application parses the UI interface of the device to be tested to obtain interface information, and constructs a structure tree model based on the interface information, and is able to respond to the instruction to locate the target element, determine the path information and location information of the target element from the structure book model, thereby recommending a corresponding automation script for the determined target element. The present application combines the UI interface with the structure tree model, can quickly locate the target element, shorten the time of locating the target element, and automatically obtain the corresponding automation script from the preset script library according to the location information, further reducing the workload of testers, making the test process smoother, and also reducing the error rate of manually written scripts. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 This is a schematic diagram of the steps of a method for determining an automated script provided in an embodiment of the present application; Figure 2 It is a flowchart of a method for determining an automated script provided in an embodiment of the present application; Figure 3 It is a flowchart of a UI structure tree parsing provided by an embodiment of the present application; Figure 4is a schematic diagram of a structure tree model provided in an embodiment of the present application; Figure 5 It is a schematic diagram of element positioning on a replica image provided by an embodiment of the present application; Figure 6 This is a flowchart of a click event triggering a target element positioning event provided by an embodiment of the present application; Figure 7 It is a flowchart of a method for locating a target element provided by an embodiment of the present application; Figure 8 is a schematic diagram of an automated script determination system provided in an embodiment of the present application; Fig. 9 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The exemplary embodiments of the present application will be described in more detail below in conjunction with the accompanying drawings in the embodiments of the present application. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to enable the scope of the present application to be fully communicated to those skilled in the art.

[0028] In a first aspect of the present application, a method for determining an automation script is provided. Figure 1 The steps of the automation script to determine the method, and Figure 2 The flowchart of the method for determining the automation script is shown, including: Step S101, obtaining the UI interface of the device to be tested.

[0029] In this step, it is necessary to obtain the UI interface of the device to be tested. In this application, a connection can be established with the device to be tested through ADB (Android Debug Bridge). In this application, a wired connection or a wireless connection can be established, and the UI interface of the current device to be tested can be obtained in sequence by sending ADB commands. In actual applications, the UI interface information is the hierarchical structure information of the UI interface of the current device to be tested in XML format.

[0030] Step S102, parsing the UI interface to obtain interface information of the UI interface.

[0031] In this step, the UI interface of the device to be tested obtained in step S101 is parsed, that is, the hierarchical structure information in XML format of the current device to be tested is parsed, and the attribute information of each element therein is extracted, such as: the class name (ClassName) of the element, the resource ID (ResourceID) of the element, the text (Text) displayed by the element, the description information (Description) of the element, and other related attribute information (bounds, enabled), etc.

[0032] Furthermore, a unique id_key-value pair is assigned to each element to obtain interface information in JSON format.

[0033] Step S103: construct a structure tree model of the UI interface according to the interface information.

[0034] In this step, the interface data in JSON format is rendered to obtain a structure tree model of the UI interface.

[0035] Specifically: First, you need to initialize the structure tree model, that is, create an empty tree data structure as the root node of the UI interface. Then you need to traverse the interface information, that is, use the parsed JSON format interface information to traverse each element and extract its attributes (such as id_, ClassName, resourceId, etc.). Then you need to build the node, that is, create a node object for each element, set its attributes, add the node to the corresponding position in the tree, and nest it according to the hierarchical relationship of the elements (such as parent-child relationship). During the traversal process, keep a reference to the parent node of each element. Make sure that each child node is correctly attached to its parent node, and set a unique id_ on each node for subsequent positioning and operation. Finally, after the construction is completed, the entire structure tree model is used as the return value for subsequent operations and display.

[0036] Step S104: in response to the triggered instruction to locate the target element, determining the path information and location information of the target element corresponding to the instruction to locate the target element in the structure tree model.

[0037] In this step, the user's input or instructions are monitored, such as by clicking or entering the id_ or other attributes of the target element. According to the received instructions, the corresponding target element is found in the constructed structure tree model. The tree structure can be traversed using a depth-first or breadth-first search algorithm. Once the target element is found, the path information of the element in the structure tree is recorded. The path information is usually composed of node names or attributes, such as the complete path from the root node to the target element. The location information of the target element is obtained, including: xpath: The XPath path generated according to the attributes and hierarchical relationship of the target element. Finally, the path information and location information are organized into a structured data object and returned to the caller for subsequent operations.

[0038] Step S105: determining the automation script of the target element from a preset script library according to the path information and the location information.

[0039] In this step, a preset automation script library is determined, which contains script templates for different element types and operations. The script library can be stored in a database, file, or other form. According to the path information and location information of the target element, a match is made in the script library to determine the operation type related to the target element, such as click, slide, long press, etc. Then, the necessary information is extracted from the matching script template, and an automation script for the target element is generated. The generated automation script is used as a return value for the user to view, edit, or directly execute.

[0040] The present application parses the UI interface of the device to be tested to obtain interface information, and constructs a structure tree model based on the interface information. The application can respond to instructions for locating target elements, determine the path information and location information of the target element from the structure tree model, and recommend corresponding automation scripts for the determined target elements. The present application combines the UI interface with the structure tree model to quickly locate the target element, shorten the target element locating time, and automatically obtain the corresponding automation script from the preset script library according to the location information, further reducing the workload of testers, making the testing process smoother, and reducing the error rate of manually written scripts.

[0041] In one embodiment, the target element location instruction is an instruction triggered by searching for a target element in the structure tree model, or an instruction triggered by clicking a target element on the structure tree model, and the step of determining the path information and location information of the target element corresponding to the target element location instruction in the structure tree model in response to the triggered target element location instruction includes: In response to a target element location instruction triggered by searching for a target element on the structure tree model, path information and location information of a target element corresponding to the target element location instruction in the structure tree model are determined, and the target element is highlighted; or, In response to a target element positioning instruction triggered by clicking a target element on the structure tree model, path information and positioning information of a target element corresponding to the target element positioning instruction in the structure tree model are determined.

[0042] In this embodiment, the processing methods of locating the target element instruction include two main situations: search and click. The response process of these two methods can be described in detail as follows: In response to the instruction to search for the target element, after the user enters the keyword, the matching node will be found in the structure tree model and its path information will be recorded. After the target element is found, the element will be highlighted for the user to quickly identify. On the other hand, when the user clicks the target element in the structure tree model, the path information and other positioning information of the node will be immediately determined without the need to highlight it again. Through these two response mechanisms, the flexibility and efficiency of the user's interaction with the structure tree model are improved.

[0043] In one embodiment, the present application also provides Figure 3 The flowchart of the UI structure tree parsing is shown in the figure: Start device connection: By calling the (11) ConnectDevice method, the user device establishes a wired or wireless connection with the vehicle device and communicates using ADB.

[0044] Get UI interface information: Use the (12) GetHierarchy method to obtain the XML format hierarchical structure information of the UI interface from the connected device. After parsing the XML data, assign a unique id_key-value pair to each node and convert the processed data into JSON format and return it.

[0045] Rendering structure tree: Use the (13) RefreshTreeView method to render the obtained JSON format interface information as follows Figure 4 At the same time, users can manually refresh the structure tree model through the (14) Refresh method, or set a timed refresh to keep the information updated.

[0046] Element search function: (15) The Serch method is used to search for elements in the structure tree, and fuzzy search is supported. The search results will be highlighted by the (16) HighLighItem method, so that users can clearly identify the target element. In addition, the path information and location information of the element will be calculated by the GetXpathByID (8) method.

[0047] Element selection function: The Select(17) method is used to obtain the id_ value of an element when clicking on it in the structure tree, and to call the GetXpathByID(18) method based on the value to locate the element, so as to ensure that the user can accurately find the location information of the selected element.

[0048] Through the above steps, it is possible to effectively interact with the vehicle equipment, display and operate UI interface elements, and improve user experience and operating efficiency.

[0049] In one embodiment, obtaining a replica image of the UI interface; The target element positioning instruction is an instruction triggered by clicking on the replica image of the UI interface, and the path information and positioning information of the target element corresponding to the target element positioning instruction in the structure tree model are determined in response to the triggered target element positioning instruction, including: In response to a target element positioning instruction triggered by clicking on a replica image of the UI interface, determining the coordinates of an interaction point corresponding to the target element positioning instruction, the interaction point being a trigger point on the replica image when the replica image is clicked; Determining the target element according to the coordinates of the interaction point; Obtaining attribute information of the target element; Element ID information is acquired from the attribute information of the target element, and path information and location information of the target element in the structure tree model are determined according to the element ID information.

[0050] In this embodiment, a replica image of the UI interface is obtained by using an ADB command or a corresponding API interface to obtain a snapshot of the current screen of the device. This command captures the image of the current screen and saves it as a file in PNG format. Process the image: The captured image is loaded into the local system, and image processing can be performed if necessary, such as resizing, cropping, or enhancing the image quality to better match the needs of the user. Display the replica image: The processed replica image is displayed in the user interface so that the user can interact, such as clicking or selecting a specific area. Interaction point recognition: When the user clicks on the replica image, the coordinates of the click are recorded for subsequent processing. Through these steps, the replica image of the UI interface can be effectively obtained and displayed, providing the user with an intuitive interactive interface.

[0051] Furthermore, when the user clicks on the replica image, an instruction to locate the target element is triggered. In response to the instruction, the coordinates of the click event on the replica image are recorded to determine the position of the interaction point. Based on the coordinates of the interaction point, the system analyzes the relationship between the replica image and the elements in the structure tree model to determine the target element corresponding to the interaction point.

[0052] Once the target element is determined, the relevant attribute information of the target element will be obtained, and the element ID information will be extracted from the attribute information of the target element. Using the extracted element ID information, the path information and location information of the target element are found in the structure tree model to provide necessary data support for the subsequent automatic script generation.

[0053] In one embodiment, determining the target element according to the coordinates of the interaction point includes: According to the coordinates of the interaction point, calculate all elements containing the coordinates of the interaction point; Determine and select the smallest area element containing the coordinates of the interaction point from all elements containing the coordinates of the interaction point; The minimum area element is determined as the target element.

[0054] In this embodiment, according to the coordinates of the interaction point clicked by the user, all elements of the current UI interface are traversed to check the bounding rectangle of each element (usually defined by the coordinates of the upper left corner, width and height of the element). If the coordinates of the interaction point are within the bounds of the element, the element is included in the inclusion list.

[0055] Among all the elements containing the interaction point, the smallest area element will be found by comparing the area of ​​each element, and the smallest area element will be selected. The smallest area element refers to the element with the smallest area and containing the interaction point among all the elements containing the interaction point, and the found smallest area element will be determined as the target element. This means that this element is the most relevant element when the user clicks.

[0056] Exemplarily, when clicking on the replica image of the UI interface, the clicked coordinate point (x, y) is obtained. Through the above point (x, y), the list of all elements containing the coordinate all_contain_list is calculated, and its calculation formula is: [rect['rect']['x']<= int(point_x)<= (rect['rect']['x']) + int(rect['rect']['width']) andrect['rect']['y']<= int(point_y)<= (rect['rect']['y']) + int(rect['rect']['height'])], and then the minimum area element containing the coordinate is calculated [min(all_contain_list, key=lambdax: x['rect']['width']* x['rect']['height'])] to obtain the located Element element information, where the obtained Element contains rect(x, y, w, h) information, and the area is drawn on the replica image to prompt the user whether the positioning is accurate, as follows Figure 5 The rectangular box of the example is the rectangular area of ​​the positioned element.

[0057] In one embodiment, the present application provides a Figure 6 The flowchart of the click event triggering the target element positioning event is shown in the figure: (21) ClickEvent handles the click event on the replica image; (22) GetInfoByPoint is used to obtain the element id_value of the click coordinates; (23) SerchItemByID is used to search for the element in the structure tree model according to the id_value; (24) SetCurrentItem is used to locate and jump to the element position in the structure tree model; (25) WriteRectToPic is used to draw the element area in the replica image; (26) The GetXpathByID method is used to determine the path information and positioning information of the element; (27) The GenerateCodeSuggest method is used to generate the recommended automation script.

[0058] In one embodiment, determining the path information and location information of the target element in the structure tree model according to the element ID information includes: According to the element ID information, selecting the target element in the structure tree model; Acquire the attribute set of the target element from the structure tree model; Determine whether there is a unique attribute in the attribute set of the target element, wherein the unique attribute is an attribute with a unique attribute value; In the case that there is a unique attribute in the attribute set of the target element, the path information and the location information of the target element in the structure tree model are determined according to the unique attribute of the target element.

[0059] In this embodiment, the steps of determining the path information and location information of the target element in the structure tree model based on the element ID information are as follows: Select the target element according to the element ID information: Use the obtained element ID information to locate and select the corresponding target element in the structure tree model. Specifically, traverse the structure tree model to find the node that matches the ID.

[0060] Get the attribute set of the target element: Once the target element is selected, extract all the attribute information of the element from the structure tree model to form an attribute set. These attributes include the element's ClassName, resourceId, text, description, etc.

[0061] Determine whether there is a unique attribute: Analyze the extracted attribute set and check whether there is any attribute whose value is unique in the entire structure tree model. A unique attribute means that the value of the attribute is not shared by other elements and can be used to uniquely identify the element.

[0062] Determine path information and location information: If a unique attribute is found in the attribute set of the target element, the information of the attribute will be used to determine the path information and location information of the target element in the structure tree model.

[0063] This embodiment can efficiently utilize the unique attribute information of the element, provide accurate path and positioning information, and enhance the reliability and efficiency of automated testing.

[0064] In one embodiment, when there is no unique attribute in the attribute set of the target element, obtaining the attribute set of the upper-level element of the target element; Determine whether there is a unique attribute in the attribute set of the upper-level element; In the case where a unique attribute exists in the attribute set of the upper-level element, determining the path information and location information of the target element in the structure tree model according to the unique attribute of the upper-level element and the category and index of the target element; In the case that there is no unique attribute in the attribute set of the upper-level element, the attribute set of the upper-level element of the upper-level element continues to be acquired until the attribute set of the root element is acquired.

[0065] In the present embodiment, when there is no unique attribute in the attribute set of the target element, it is necessary to search upwards and obtain the attribute set of the upper level element of the target element. Analyze the attribute set of the upper level element and check whether there is a unique attribute therein. If the value of a certain attribute is unique in the whole structure tree, then this attribute is regarded as a unique attribute. If there is a unique attribute in the attribute set of the upper level element, use this unique attribute, as well as the category (such as ClassName) and index information of the target element, to build the path information and location information of the target element in the structure tree model. And if there is still no unique attribute in the attribute set of the upper level element, then it is necessary to continue to search upwards and obtain the upper level attribute set of the upper level element, and repeat the judgment process until the attribute set of the root element is found or confirmation does not have the unique attribute.

[0066] In this embodiment, it is ensured that even when the target element and its parent element lack unique attributes, the exact path of the target element in the structure tree can still be effectively found.

[0067] In one embodiment, a flowchart of locating a target element is provided, such as Figure 7 As shown: the process includes: (31) CurrentItem is used to obtain the current node (target element); (32) UniqueAttribute is used to calculate whether the current node has a unique attribute; (33) GetTypeIndex returns the ClassName and index information of the current node; (34) PetParent is used to obtain the parent node element of the node; (35) ReturnXpath is used to return the recursively obtained path information and location information.

[0068] In a preferred embodiment, parsing the UI interface to obtain the interface information of the UI interface includes: Parsing the UI interface to obtain hierarchical structure information of the UI interface in XML format; Each node in the hierarchical structure information is respectively set with a corresponding key-value pair to obtain the interface information in JSON format, wherein each key-value pair is set with an element ID information; The step of constructing a structure tree model of the UI interface according to the interface information includes: The interface information in the JSON format is rendered according to a hierarchical structure to obtain a structure tree model of the UI interface.

[0069] In this embodiment, the connected UI interface is first parsed. The goal of this step is to obtain the hierarchical information of the UI interface in XML format, describing the relationship between the various components and elements in the interface. The hierarchical information in XML format obtained by parsing is processed node by node. Each node will be assigned a unique key-value pair. In this way, the entire hierarchical information is converted into interface information in JSON format. This JSON structure is convenient for subsequent processing and display. Each key-value pair contains an element ID information, which is an important attribute for uniquely identifying each element.

[0070] Based on the generated JSON format interface information, the original hierarchical structure is rendered to build a structure tree model of the UI interface. This model intuitively displays the hierarchical relationship of the UI interface, allowing users to easily understand the relationship between various elements.

[0071] In one embodiment, according to the path information and location information of the target element in the structure tree model, the path information and location information of the parent and child elements of the target element in the structure tree model, and the path information and location information of the sibling elements of the target element in the structure tree model are determined respectively; Determining the automation scripts of the parent and child elements from the preset script library according to the path information and location information of the parent and child elements in the structure tree model; According to the path information and location information of the brother element in the structure tree model, the automation script of the brother element is determined from the preset script library.

[0072] In this embodiment, based on the path information and location information of the target element in the structure tree model, the path information and location information of its parent element and child element are first determined. Secondly, the path information and location information of the sibling elements of the target element are determined. These sibling elements are other elements at the same level as the target element and are equally important to the generation of the automation script.

[0073] According to the path information and location information of the parent and child elements determined above, the corresponding automation script is retrieved from the preset script library. These scripts usually contain operation instructions for the parent and child elements, such as click, slide or other interactive operations. The automation script can be obtained by comparing the attributes of the matching element (such as ID, class name, etc.) with the definition in the script library to find the appropriate operation method.

[0074] Similarly, according to the path information and location information of the sibling elements, the corresponding automation scripts are retrieved from the preset script library. These scripts also contain relevant operation instructions for the sibling elements, allowing these elements to interact during the automated testing process.

[0075] Through the above steps, this application can comprehensively generate necessary automation scripts for the target element and its related elements (parent, child and sibling elements), providing support for subsequent automated testing. This structured approach not only improves the efficiency of script generation, but also enhances the flexibility and reliability of testing.

[0076] The present application provides a method for determining an automation script, the method comprising: obtaining a UI interface of a device to be tested; parsing the UI interface to obtain interface information of the UI interface; constructing a structure tree model of the UI interface according to the interface information; responding to a triggered instruction to locate a target element, determining the path information and location information of a target element corresponding to the instruction to locate the target element in the structure tree model; and determining the automation script of the target element from a preset script library according to the path information and the location information. The present application parses the UI interface of the device to be tested to obtain interface information, and constructs a structure tree model based on the interface information, and is able to respond to the instruction to locate the target element, determine the path information and location information of the target element from the structure book model, thereby recommending a corresponding automation script for the determined target element. The present application combines the UI interface with the structure tree model, can quickly locate the target element, shorten the time of locating the target element, and automatically obtain the corresponding automation script from the preset script library according to the location information, further reducing the workload of testers, making the test process smoother, and also reducing the error rate of manually written scripts.

[0077] Based on the same inventive concept, the second party of the present application provides a system for determining an automated script, such as Figure 8 As shown, the system comprises: The acquisition module 201 is used to acquire the UI interface of the device to be tested; The parsing module 202 is used to parse the UI interface to obtain a replica image and interface information of the UI interface; A construction module 203 is used to construct a structure tree model of the UI interface according to the interface information; A first determination module 204 is configured to determine, in response to a triggered instruction to locate a target element, path information and location information of a target element corresponding to the instruction to locate the target element in the structure tree model; The second determination module 205 is used to determine the automation script of the target element from a preset script library according to the path information and the location information.

[0078] Optionally, the target element positioning instruction is an instruction triggered by searching for a target element in the structure tree model, or an instruction triggered by clicking a target element on the structure tree model. In response to the triggered target element positioning instruction, path information and location information of the target element corresponding to the target element positioning instruction in the structure tree model are determined. The first determining module 204 includes: A first determination submodule is configured to, in response to a target element location instruction triggered by searching for a target element on the structure tree model, determine the path information and location information of the target element corresponding to the target element location instruction in the structure tree model, and highlight the target element; or The second determination submodule is used to determine the path information and location information of the target element corresponding to the target element location instruction in the structure tree model in response to the target element location instruction triggered by clicking the target element on the structure tree model.

[0079] Optionally, the system further comprises: A first acquisition submodule is used to acquire a replica image of the UI interface; The target element positioning instruction is an instruction triggered by clicking on the replica image of the UI interface. In response to the triggered target element positioning instruction, the path information and positioning information of the target element corresponding to the target element positioning instruction in the structure tree model are determined. The first determination submodule includes: A third determination submodule is used to determine the coordinates of an interaction point corresponding to the positioning target element instruction in response to the positioning target element instruction triggered by clicking the replica image of the UI interface, where the interaction point is a trigger point on the replica image when the replica image is clicked; A fourth determination submodule, used to determine the target element according to the coordinates of the interaction point; The second acquisition submodule is used to acquire the attribute information of the target element; The fifth determining submodule is used to obtain element ID information from the attribute information of the target element, and determine the path information and location information of the target element in the structure tree model according to the element ID information.

[0080] Optionally, the determining the target element according to the coordinates of the interaction point, the fourth determining submodule includes: A calculation subunit, configured to calculate all elements including the coordinates of the interaction point according to the coordinates of the interaction point; A first determining subunit is used to determine and select a minimum area element containing the coordinates of the interaction point from all elements containing the coordinates of the interaction point; The second determining subunit is used to determine the minimum area element as the target element.

[0081] Optionally, the determining, according to the element ID information, the path information and the location information of the target element in the structure tree model, the fifth determining submodule includes: A selection subunit is used to select the target element in the structure tree model according to the element ID information; An acquisition subunit, used to acquire the attribute set of the target element from the structure tree model; A first judgment subunit is used to judge whether there is a unique attribute in the attribute set of the target element, wherein the unique attribute is an attribute with a unique attribute value; The third determining subunit is configured to determine the path information and the location information of the target element in the structure tree model according to the unique attribute of the target element when there is a unique attribute in the attribute set of the target element.

[0082] Optionally, the system further comprises: A third acquisition submodule is used to acquire the attribute set of the parent element of the target element when there is no unique attribute in the attribute set of the target element; A judgment submodule, used to judge whether there is a unique attribute in the attribute set of the upper-level element; A sixth determination submodule, configured to determine the path information and location information of the target element in the structure tree model according to the unique attribute of the upper-level element and the category and index of the target element when there is a unique attribute in the attribute set of the upper-level element; The fourth acquisition submodule is used to, if there is no unique attribute in the attribute set of the upper-level element, continue to acquire the attribute set of the upper-level element until the attribute set of the root element is acquired.

[0083] Optionally, the UI interface is parsed to obtain interface information of the UI interface, and the parsing module 202 includes: A parsing submodule, used to parse the UI interface to obtain the hierarchical structure information of the UI interface in XML format; A setting submodule is used to set a corresponding key-value pair for each node in the hierarchical structure information to obtain the interface information in JSON format, wherein each key-value pair is set with an element ID information; The structure tree model of the UI interface is constructed according to the interface information, and the construction module includes: The rendering submodule is used to render the interface information in the JSON format according to a hierarchical structure to obtain a structure tree model of the UI interface.

[0084] Optionally, the system further comprises: a seventh determination submodule, configured to determine the path information and location information of the parent and child elements of the target element in the structure tree model, and the path information and location information of the sibling elements of the target element in the structure tree model, respectively, according to the path information and location information of the target element in the structure tree model; An eighth determination submodule, configured to determine the automation scripts of the parent and child elements from the preset script library according to the path information and location information of the parent and child elements in the structure tree model; The ninth determination submodule is used to determine the automation script of the brother element from the preset script library according to the path information and location information of the brother element in the structure tree model.

[0085] Based on the same inventive concept, the third aspect of the present application provides a Fig. 9 The electronic device 100 shown includes a memory 110, a processor 120, and a computer program stored in the memory 110. The processor 120 executes the computer program to implement the method for determining the automation script as described in the first aspect of the present application.

[0086] In a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the method for determining the automation script as described in the first aspect of the present application is implemented.

[0087] Each embodiment in this specification focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referenced to each other.

[0088] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0089] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0090] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0092] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0093] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0094] The above is a detailed introduction to the method, system, electronic device and storage medium for determining an automation script. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for determining an automation script, characterized in that: The method comprises: Get the UI interface of the device to be tested; Parsing the UI interface to obtain a replica image and interface information of the UI interface; Constructing a structure tree model of the UI interface according to the interface information; In response to the triggered instruction to locate the target element, determining the path information and location information of the target element corresponding to the instruction to locate the target element in the structure tree model; According to the path information and the location information, an automation script of the target element is determined from a preset script library.

2. The method for determining an automation script according to claim 1, characterized in that: The target element positioning instruction is an instruction triggered by searching for a target element in the structure tree model, or an instruction triggered by clicking a target element on the structure tree model. In response to the triggered target element positioning instruction, path information and positioning information of the target element corresponding to the target element positioning instruction in the structure tree model are determined, including: In response to a target element location instruction triggered by searching for a target element on the structure tree model, path information and location information of a target element corresponding to the target element location instruction in the structure tree model are determined, and the target element is highlighted; or, In response to a target element positioning instruction triggered by clicking a target element on the structure tree model, path information and positioning information of a target element corresponding to the target element positioning instruction in the structure tree model are determined.

3. The method for determining an automation script according to claim 1, characterized in that: The method further comprises: Obtaining a replica image of the UI interface; The target element positioning instruction is an instruction triggered by clicking on the replica image of the UI interface, and the path information and positioning information of the target element corresponding to the target element positioning instruction in the structure tree model are determined in response to the triggered target element positioning instruction, including: In response to a target element positioning instruction triggered by clicking on a replica image of the UI interface, determining the coordinates of an interaction point corresponding to the target element positioning instruction, the interaction point being a trigger point on the replica image when the replica image is clicked; Determining the target element according to the coordinates of the interaction point; Obtaining attribute information of the target element; Element ID information is acquired from the attribute information of the target element, and path information and location information of the target element in the structure tree model are determined according to the element ID information.

4. The method for determining an automation script according to claim 3, characterized in that: The step of determining the target element according to the coordinates of the interaction point includes: According to the coordinates of the interaction point, calculate all elements containing the coordinates of the interaction point; Determine and select the smallest area element containing the coordinates of the interaction point from all elements containing the coordinates of the interaction point; The minimum area element is determined as the target element.

5. The method for determining an automation script according to claim 3, characterized in that: Determining the path information and location information of the target element in the structure tree model according to the element ID information includes: According to the element ID information, selecting the target element in the structure tree model; Acquire the attribute set of the target element from the structure tree model; Determine whether there is a unique attribute in the attribute set of the target element, wherein the unique attribute is an attribute with a unique attribute value; In the case that there is a unique attribute in the attribute set of the target element, the path information and the location information of the target element in the structure tree model are determined according to the unique attribute of the target element.

6. The method for determining an automation script according to claim 5, characterized in that: The method further comprises: If there is no unique attribute in the attribute set of the target element, obtaining the attribute set of the upper-level element of the target element; Determine whether there is a unique attribute in the attribute set of the upper-level element; In the case where a unique attribute exists in the attribute set of the upper-level element, determining the path information and location information of the target element in the structure tree model according to the unique attribute of the upper-level element and the category and index of the target element; In the case that there is no unique attribute in the attribute set of the upper-level element, the attribute set of the upper-level element of the upper-level element continues to be acquired until the attribute set of the root element is acquired.

7. The method for determining an automation script according to claim 1, characterized in that: Parsing the UI interface to obtain interface information of the UI interface includes: Parsing the UI interface to obtain hierarchical structure information of the UI interface in XML format; Each node in the hierarchical structure information is respectively set with a corresponding key-value pair to obtain the interface information in JSON format, wherein each key-value pair is set with an element ID information; The step of constructing a structure tree model of the UI interface according to the interface information includes: The interface information in the JSON format is rendered according to a hierarchical structure to obtain a structure tree model of the UI interface.

8. The method for determining an automation script according to claim 1, characterized in that: The method further comprises: According to the path information and location information of the target element in the structure tree model, respectively determine the path information and location information of the parent and child elements of the target element in the structure tree model, and the path information and location information of the sibling elements of the target element in the structure tree model; Determining the automation scripts of the parent and child elements from the preset script library according to the path information and location information of the parent and child elements in the structure tree model; According to the path information and location information of the brother element in the structure tree model, the automation script of the brother element is determined from the preset script library.

9. A system for determining an automation script, characterized in that: The system comprises: The acquisition module is used to obtain the UI interface of the device to be tested; A parsing module, used to parse the UI interface to obtain a replica image and interface information of the UI interface; A construction module, used to construct a structure tree model of the UI interface according to the interface information; A first determination module, configured to determine, in response to a triggered instruction to locate a target element, path information and location information of a target element corresponding to the instruction to locate the target element in the structure tree model; The second determination module is used to determine the automation script of the target element from a preset script library according to the path information and the location information.

10. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the method for determining an automation script according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the method for determining an automation script according to any one of claims 1 to 8 is implemented.

Images