Element positioning method and device, equipment and medium

Abstract

The embodiment of the invention discloses an element positioning method and device, equipment and a medium, and the method comprises the steps: determining a target element from a layout file of a current application interface in response to a selection operation of a user on the current application interface, obtaining an element attribute of the target element and a first screenshot picture of the target element in the current application interface; when it is detected that the layout of the current application interface is changed, obtaining a changed layout file; and according to the first screenshot picture and the element attribute, identifying a candidate element corresponding to the target element in the changed layout file, and updating the target element in a set element information tree according to an identification result.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of automated testing, and more particularly, to an element positioning method, device, apparatus and medium. BACKGROUND

[0002] In the field of software automated testing, using automated tools to simulate user operations in software or applications can verify whether the functions of the user interface meet expectations, reduce the workload of manual testing, improve testing efficiency and accuracy, and ensure the quality and stability of software. However, in existing software automated testing, when the layout of the current application interface changes, causing the test script to fail to run normally, the test engineer must manually reposition the elements and update the test script, which is tedious, time-consuming and error-prone. SUMMARY

[0003] The purpose of the embodiments of the present disclosure is to provide a new technical solution for element positioning.

[0004] According to a first aspect of the embodiments of the present disclosure, an element positioning method is provided, which comprises: In response to a selection operation of a user on a current application interface, determining a target element from a layout file of the current application interface, and obtaining element attributes of the target element and a first screenshot picture of the target element in the current application interface; In the case of detecting that the layout of the current application interface has changed, obtaining a changed layout file; According to the first screenshot picture and the element attributes, identifying a candidate element corresponding to the target element in the changed layout file, and updating the target element in the set element information tree according to the identification result.

[0005] Optionally, the layout file comprises element attributes corresponding to each element respectively, and the element attributes comprise element positions, In response to a selection operation of a user on a current application interface, determining a target element from a layout file of the current application interface, comprises: In response to a selection operation of a user on a current application interface, obtaining the selection position of the user; Matching the selection position with element positions corresponding to each element in the layout file, and taking the element corresponding to the matched element position as the target element.

[0006] Optionally, the method further comprises: Highlighting the target element on the current application interface.

[0007] Optionally, the method further comprises: updating an element name in an element attribute of the target element in the layout file according to a set naming rule, to obtain an updated layout file; generating the set element information tree according to an element structure of each element in the updated layout file and the element attribute of each element; wherein the set element information tree reflects a parent-child relationship between each element, each element displays the element attribute of the element in the set element information tree, and the target element is associated with the display of the first screenshot picture in the set element information tree.

[0008] Optionally, the identifying the candidate element corresponding to the target element in the changed layout file according to the first screenshot picture and the element attribute comprises: obtaining a second screenshot picture of the changed current application interface; obtaining a picture similarity between the first screenshot picture and the second screenshot picture; obtaining an attribute similarity between the element attribute and the element attribute corresponding to each element in the changed layout file; determining the candidate element corresponding to the target element in the changed layout file according to the picture similarity and the attribute similarity.

[0009] Optionally, the layout file further comprises descriptive text content of an element, The method further comprises: obtaining a content similarity between the descriptive text content of the target element and the descriptive text content corresponding to each element in the changed layout file; determining the candidate element corresponding to the target element in the changed layout file according to the picture similarity, the attribute similarity, and the content similarity.

[0010] Optionally, the method further comprises: identifying an application type of the current application; obtaining the layout file of the current application interface according to the application type.

[0011] According to a second aspect of the embodiments of the present disclosure, an element positioning device is provided, and the device comprises: a determination module configured to determine a target element from a layout file of a current application interface in response to a selection operation of a user on the current application interface; a first obtaining module configured to obtain an element attribute of the target element and a first screenshot picture of the target element in the current application interface; The second acquisition module is used to acquire the changed layout file when the layout of the current application interface is detected to have changed. The identification module is used to identify candidate elements corresponding to the target element in the changed layout file based on the first screenshot image and the element attributes, and to update the target element in the set element information tree based on the identification result.

[0012] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a memory for storing executable computer instructions; and a processor for executing the method described in accordance with the first aspect above, under the control of the executable computer instructions. According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, perform the method described in the first aspect above. One beneficial effect of this disclosure is that it can acquire the element attributes of the target element and a screenshot of the target element in the current application interface. When a change in the layout of the current application interface is detected, it can identify candidate elements corresponding to the target element in the changed layout file using the screenshot and the element attributes of the target element, and update the target element in the set element information tree based on the identification results. Thus, even when elements change, the accuracy of element location can be improved by simultaneously combining visual and attribute information to locate and update the element, thereby enhancing the efficiency of software automation testing and the stability of test scripts. Other features and advantages of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of this specification.

[0014] Figure 1 This is a schematic diagram of the hardware configuration of the electronic device provided in the embodiments of this disclosure; Figure 2 This is a flowchart illustrating the element positioning method provided in this embodiment of the disclosure; Figure 3 This is a block diagram of the element positioning device provided in the embodiments of this disclosure; Figure 4 This is a block diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0015] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the embodiments of the present disclosure.

[0016] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0017] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0018] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0019] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0020] <Hardware Configuration> Figure 1 This is a block diagram of the hardware configuration of an electronic device 1000 according to an embodiment of the present disclosure.

[0021] In one embodiment, the electronic device 1000 may be a server or a terminal device. The server may be a monolithic server or a distributed server spanning multiple computers or a computer data center. The terminal device may be a portable computer, desktop computer, wearable device, or any other device having a processor or other computing device and a memory or other storage device; this embodiment does not limit the specific device.

[0022] like Figure 1 As shown, the electronic device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, etc.

[0023] Processor 1100 may be a mobile processor. Memory 1200 includes, for example, ROM (Read-Only Memory), RAM (Random Access Memory), and non-volatile memory such as a hard disk. Interface device 1300 includes, for example, a USB interface and a headphone jack. Communication device 1400 may be capable of wired or wireless communication. Communication device 1400 may include short-range communication devices, such as any device that performs short-range wireless communication based on short-range wireless communication protocols such as Hilink, WiFi (IEEE 802.11), Mesh, Bluetooth, ZigBee, Thread, Z-Wave, NFC, UWB, and LiFi. Communication device 1400 may also include long-range communication devices, such as any device that performs WLAN, GPRS, or 2G / 3G / 4G / 5G long-range communication. Display device 1500 is, for example, an LCD screen or a touch screen. Input device 1600 may include, for example, a touch screen or a keyboard. Electronic device 1000 can output audio information through a speaker 1700 and acquire audio information through a microphone 1800.

[0024] Despite Figure 1 The electronic device 1000 shows multiple devices, but this disclosure may only relate to some of them. For example, electronic device 1000 may only relate to memory 1200 and processor 1100.

[0025] In embodiments of this disclosure, the memory 1200 of the electronic device 1000 is used to store instructions for controlling the processor 1100 to execute the element positioning method provided in the embodiments of this disclosure.

[0026] In the above description, those skilled in the art can design instructions based on the scheme disclosed in this disclosure. How the instructions control the processor to operate is well known in the art, and therefore will not be described in detail here.

[0027] <Method Implementation> Figure 2 An element positioning method according to an embodiment of this disclosure is shown, which can be provided by... Figure 1 The illustrated electronic device 1000 performs this operation. For example... Figure 3 As shown, the element positioning method of this embodiment may include the following steps S2100 to S2300: Step S2100: In response to the user's selection operation on the current application interface, determine the target element from the layout file of the current application interface, and obtain the element attributes of the target element and the first screenshot image of the target element in the current application interface.

[0028] The layout file for the current application interface refers to a structured data file generated by the application framework at runtime to describe the structure and state of the current user interface. The layout file may include, for example, the element attributes, element structure, and descriptive text information for each element in the current application interface.

[0029] The aforementioned element attributes may include, for example, element identifiers (such as element ID), element class names, element coordinates, and element states. The aforementioned element structure can represent parent-child relationships between elements. The aforementioned descriptive text information can be semantic strings that indicate to the user the function, content, or identity of the element.

[0030] The user's selection action on the current application interface can be a touch operation, such as input or clicking. The target element can refer to the element actively located by the user through a selection action, such as a touch operation.

[0031] The first screenshot image mentioned above may include a full-screen screenshot of the current application interface, including the target element. Alternatively, the first screenshot image may include an element screenshot obtained by cropping the target element from the bounding box of the target element in the full-screen screenshot.

[0032] In one example, the above-mentioned response to a user's selection operation on the current application interface, determining the target element from the layout file of the current application interface and obtaining the element attributes of the target element, may further include: responding to a user's selection operation on the current application interface, obtaining the user's selection position; matching the selection position with the element positions corresponding to each element in the layout file, and taking the element corresponding to the matched element position as the target element.

[0033] In this example, when the electronic device receives a touch operation from the user on the current application interface, it can respond to the touch operation by obtaining the user's touch location, loading the layout file of the current application interface, and traversing each element in the layout file to detect whether the touch location is located at the element position of one of the elements. If the touch location is located at the element position of one of the elements, that element can be used as the target element, and the element attributes of the target element can be obtained from the layout file. Finally, a screenshot of the target element on the current application interface can be taken to obtain a screenshot image.

[0034] After performing the above step S2100 in response to the user's selection operation on the current application interface, determining the target element from the layout file of the current application interface, and obtaining the element attributes of the target element and the first screenshot image of the target element in the current application interface, proceeds to: Step S2200: If the layout of the current application interface is detected to have changed, obtain the changed layout file.

[0035] The modified layout file may include the element attributes, element structure, and descriptive text information for each element in the current application interface after the layout change.

[0036] In this embodiment, when the electronic device detects a change in the layout of the current application interface, it can load the layout file of the current application interface after the layout change.

[0037] After executing step S2200 above, if a change in the layout of the current application interface is detected, and the changed layout file is obtained, then proceed to: Step S2300: Based on the first screenshot image and the element attributes, identify candidate elements corresponding to the target element in the changed layout file, and update the target element in the set element information tree according to the identification results.

[0038] The element information tree reflects the parent-child and nesting relationships between elements. Each element displays its element attributes within this tree, and the target element is associated with and displayed as shown in the first screenshot within the same tree. The element information tree can be a DOM tree or a UI hierarchy tree. Furthermore, the element name in the target element's element attributes within the element information tree can be a renamed version of the element name.

[0039] In one example, step S2300, which identifies candidate elements corresponding to the target element in the modified layout file based on the first screenshot image and the element attributes, may further include: obtaining a second screenshot image of the modified current application interface; obtaining the image similarity between the first screenshot image and the second screenshot image; obtaining the attribute similarity between the element attributes and the element attributes corresponding to each element in the modified layout file; and determining candidate elements corresponding to the target element in the modified layout file based on the image similarity and the attribute similarity.

[0040] In this example, the element attributes of the target element and the first screenshot image can be obtained from the set element information tree.

[0041] The second screenshot can be a full-screen screenshot of the current application interface after the layout has changed.

[0042] Image similarity and attribute similarity can each have their own weights, and the weights corresponding to image similarity and attribute similarity can be preset according to the actual scenario and experience.

[0043] In this example, the overall similarity between the target element and each element in the modified layout file can be calculated based on image similarity and attribute similarity. Candidate elements can then be identified from the modified layout file based on the overall similarity to obtain the matching results.

[0044] Here, the matching results can include a first matching result, a second matching result, and a third matching result. The first matching result can be an exact match, meaning that a candidate element is found in the changed layout file whose element attributes are completely identical to those of the target element. For example, the overall similarity between the candidate element and the target element can be a first set threshold, such as 100%, indicating that the target element has not been modified after the layout change. In this case, it is not necessary to update the element information of the target element in the set element information tree, and the target element is marked in green to indicate that the target element has not changed.

[0045] The second matching result can be a fuzzy match. This means that in the modified layout file, no candidate elements with attributes exactly matching the target element's attributes are found. However, one or more elements with a comprehensive similarity score greater than a second set threshold and less than a first set threshold are identified as candidate elements. This indicates that the target element may have been modified. In this case, the element information of the target element in the set element information tree can be updated to the element information of the candidate elements, and the target element is marked in yellow to indicate that it has been modified. If there are multiple candidate elements, the one with the highest comprehensive similarity score is usually selected by default, but the user can also manually confirm or select one.

[0046] The third matching result can be a match failure, meaning that in the modified layout file, no candidate element can be found whose element attributes are completely identical to those of the target element, and the overall similarity is less than or equal to the second set threshold. This indicates that the target element may have been deleted. In this case, the target element can be directly deleted from the set element information tree. Alternatively, the user can manually delete the target element from the set element information tree and mark it in red to indicate that the target element has been deleted.

[0047] In this example, it can combine visual matching and attribute matching to locate candidate elements, which can improve the accuracy of element location.

[0048] In one example, step S2300, based on the first screenshot image and the element attributes, may further include: obtaining a second screenshot image of the modified current application interface; obtaining the image similarity between the first screenshot image and the second screenshot image; obtaining the attribute similarity between the element attributes and the element attributes corresponding to each element in the modified layout file; obtaining the content similarity between the descriptive text content of the target element and the descriptive text content corresponding to each element in the modified layout file; and determining the candidate element corresponding to the target element in the modified layout file based on the image similarity, the attribute similarity, and the content similarity.

[0049] In this example, it can combine visual matching, attribute matching, and content matching to locate candidate elements, which can further improve the accuracy of element location.

[0050] This embodiment can acquire the element attributes of a target element and a screenshot of the target element in the current application interface. When a change in the layout of the current application interface is detected, the system can identify candidate elements corresponding to the target element in the changed layout file using the screenshot and the target element's attributes. Based on the identification results, the system updates the target element in the element information tree. Thus, even when elements change, the system can locate and update elements by simultaneously combining visual and attribute information, improving the accuracy of element location and enhancing the efficiency of automated software testing and the stability of test scripts.

[0051] In one embodiment, before performing the above step S2100 in response to the user's selection operation on the current application interface, determining the target element from the layout file of the current application interface, and obtaining the element attributes of the target element and the first screenshot image of the target element in the current application interface, the element positioning method of this embodiment further includes the step of determining the layout file of the current application interface. In specific implementation, the determination of the layout file of the current application interface can be carried out in the following way: identifying the current type of the current application, and determining the layout file of the current application interface according to the application type.

[0052] The application type indicates the application category to which the current application belongs. Application categories may include, for example, web applications, mobile applications, and desktop applications.

[0053] In this example, when the user opens the automated testing tool and executes the test script, the electronic device first identifies the application type of the current application, and then calls the interface or protocol that matches the application type to obtain the layout file of the current application interface. The layout file is then saved locally in a set format, such as HTML or XML. When the user selects an element in the current application interface, the device can determine the target element from the layout file and obtain the element attributes and a screenshot of the target element in the current application interface.

[0054] According to this embodiment, it can automatically identify the application type and call the matching interface to obtain the layout file, ensuring the authenticity and completeness of the obtained layout file.

[0055] In one embodiment, after performing the above step S2100 in response to the user's selection operation on the current application interface, determining the target element from the layout file of the current application interface, and obtaining the element attributes of the target element, the element positioning method of this embodiment further includes: highlighting the target element on the current application interface.

[0056] For example, highlighting can be done by color coding, such as displaying other elements in the current application interface in a first color and the target element in a second color, with the first and second colors being different, in order to provide visual feedback to the user.

[0057] In one embodiment, after performing the above step S2100 in response to the user's selection operation on the current application interface, determining the target element from the layout file of the current application interface, and obtaining the element attributes of the target element and the first screenshot image of the target element in the current application interface, the element positioning method of this embodiment further includes: updating the element name in the element attributes of the target element in the layout file according to a set naming rule to obtain an updated layout file; parsing the element structure in the updated layout file to generate the set element information tree.

[0058] Here, the element name in the element attribute of the target element in the layout file can be renamed according to the set naming rules. The naming rules can include, for example, the element type, function description and index, so that the renamed element name becomes a stable abstract identifier, which can improve the stability of the test script and reduce the maintenance cost of the test script.

[0059] The set element information tree reflects the parent-child relationship between elements. Each element displays its element attributes in the set element information tree, and the target element displays the first screenshot image in association with the set element information tree.

[0060] In one example, step S2300 above, which identifies candidate elements corresponding to the target element in the changed layout file based on the screenshot image and the element attributes, and updates the target element in the set element information tree based on the identification result, may further include: The element attributes and the first screenshot image of the target element are obtained from the set element information tree. Based on the first screenshot image and the element attributes, the candidate elements corresponding to the target element are identified in the changed layout file. That is, the element name in the element attributes of the target element is the renamed element name.

[0061] This embodiment provides a visual interface for users by generating a set element information tree, making it easy to view and manage elements. It also supports users to search for target elements by renaming them, allowing them to quickly locate target elements.

[0062] In one embodiment, the element location method of this disclosure further includes: providing a library file calling interface; wherein the library file calling interface is used to receive a calling instruction consisting of a renamed element name; and according to the library file calling instruction, calling an element operation method corresponding to the current application type to perform automated testing operations.

[0063] <Device Embodiment> Figure 3 This is a schematic diagram of an element positioning device according to one embodiment, with reference to... Figure 3 As shown, the element positioning device 300 may include a determining module 310, a first acquiring module 320, a second acquiring module 330, and an identification module 340.

[0064] The determination module 310 is used to determine the target element from the layout file of the current application interface in response to the user's selection operation on the current application interface. The first acquisition module 320 is used to acquire the element attributes of the target element and the first screenshot image of the target element in the current application interface; The second acquisition module 330 is used to acquire the changed layout file when the layout of the current application interface is detected to have changed. The identification module 340 is used to identify candidate elements corresponding to the target element in the changed layout file based on the first screenshot image and the element attributes, and to update the target element in the set element information tree based on the identification result.

[0065] In one embodiment, the layout file includes element attributes corresponding to each element, and the element attributes include element position.

[0066] The determination module 310 is specifically used to respond to the user's selection operation on the current application interface, obtain the user's selection position, match the selection position with the element positions corresponding to each element in the layout file, and take the element corresponding to the matched element position as the target element.

[0067] In one embodiment, the device 300 further includes a display module (not shown).

[0068] The display module is used to highlight the target element in the current application interface.

[0069] In one embodiment, the device 300 further includes a generation module (not shown in the figure).

[0070] The generation module is used to update the element name in the element attribute of the target element in the layout file according to the set naming rules to obtain the updated layout file; and to generate the set element information tree according to the element structure and element attributes of each element in the updated layout file; wherein the set element information tree reflects the parent-child relationship between each element, each element displays the element attribute in the set element information tree, and the target element displays the first screenshot image in association in the set element information tree.

[0071] In one embodiment, the identification module 340 is specifically used to acquire a second screenshot image of the modified current application interface; acquire the image similarity between the first screenshot image and the second screenshot image; acquire the attribute similarity between the element attributes and the element attributes corresponding to each element in the modified layout file; and determine the candidate element corresponding to the target element in the modified layout file based on the image similarity and the attribute similarity.

[0072] In one embodiment, the layout file further includes descriptive text content of elements. The recognition module 340 is specifically used to obtain the content similarity between the descriptive text content of the target element and the descriptive text content corresponding to each element in the modified layout file; and to determine the candidate element corresponding to the target element in the modified layout file based on the image similarity, the attribute similarity, and the content similarity.

[0073] In one embodiment, the first acquisition module 320 is further configured to identify the application type of the current application; and acquire the layout file of the current application interface according to the application type.

[0074] According to embodiments of this disclosure, it can acquire the element attributes of a target element and a screenshot of the target element in the current application interface. When a change in the layout of the current application interface is detected, it can identify candidate elements corresponding to the target element in the changed layout file using the screenshot and the element attributes of the target element, and update the target element in the element information database based on the identification results. Thus, even when elements change, the accuracy of element location can be improved by simultaneously combining visual and semantic information to locate and update elements, thereby enhancing the efficiency of automated software testing and the stability of test scripts.

[0075] <Equipment Example> Figure 4 This is a schematic diagram of the hardware structure of an electronic device according to one embodiment. For example... Figure 4 As shown, the electronic device 1000 includes a processor 1100 and a memory 1200.

[0076] The memory 1200 can be used to store executable computer instructions.

[0077] The processor 1100 can be used to execute an element positioning method according to an embodiment of the method disclosed herein, under the control of executable computer instructions.

[0078] The electronic device 1000 can be as follows: Figure 1 The electronic device 1000 shown may also be a device with other hardware structures, which are not limited here.

[0079] In another embodiment, the electronic device 1000 may include the above-mentioned element positioning device 300. In one embodiment, each module of the above element positioning device 300 can be implemented by the processor 1100 running computer instructions stored in the memory 1200.

[0080] Computer-readable storage media This disclosure also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, perform the element location method provided in this disclosure.

[0081] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0082] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0083] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0084] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0085] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0086] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0087] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0088] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are equivalent.

[0089] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. An element positioning method, wherein, The method comprises: In response to a selection operation of a user in a current application interface, determining a target element from a layout file of the current application interface, and obtaining an element attribute of the target element and a first screenshot picture of the target element in the current application interface; In a case where it is detected that a layout of the current application interface changes, obtaining a changed layout file; According to the first screenshot picture and the element attribute, identifying a candidate element corresponding to the target element in the changed layout file, and updating the target element in a set element information tree according to an identification result.

2. The method of claim 1, wherein, The layout file comprises element attributes corresponding to respective elements, and the element attributes comprise element positions, In response to a selection operation of a user in a current application interface, determining a target element from a layout file of the current application interface, comprising: In response to a selection operation of a user in a current application interface, obtaining a selection position of the user; Matching the selection position with element positions corresponding to respective elements in the layout file, and taking an element corresponding to a matched element position as the target element.

3. The method of claim 2, wherein, The method further comprises: Highlighting the target element in the current application interface.

4. The method of claim 1, wherein, The method further comprises: According to a set naming rule, updating an element name in an element attribute of the target element in the layout file, to obtain an updated layout file; According to element structures of respective elements in the updated layout file and element attributes of the respective elements, generating the set element information tree; The set element information tree reflects parent-child relationships among the respective elements, each element displays the element attribute of the element in the set element information tree, and the target element is associated with the first screenshot picture in the set element information tree.

5. The method of claim 1, wherein, According to the first screenshot picture and the element attribute, identifying a candidate element corresponding to the target element in the changed layout file, comprising: Obtaining a second screenshot picture of the current application interface after the change; Obtaining a picture similarity between the first screenshot picture and the second screenshot picture; Obtaining an attribute similarity between the element attribute and element attributes corresponding to respective elements in the changed layout file; According to the picture similarity and the attribute similarity, determining a candidate element corresponding to the target element in the changed layout file.

6. The method of claim 5, wherein, The layout file further comprises descriptive text content of elements, The method further comprises: Obtaining a content similarity between the descriptive text content of the target element and descriptive text content corresponding to respective elements in the changed layout file; According to the picture similarity, the attribute similarity, and the content similarity, determining a candidate element corresponding to the target element in the changed layout file.

7. The method of claim 1, wherein, The method further comprises: Identifying an application type of a current application; According to the application type, obtaining the layout file of the current application interface.

8. An element positioning device, wherein, The apparatus comprises: A determination module, configured to, in response to a selection operation of a user in a current application interface, determine a target element from a layout file of the current application interface; The first obtaining module is configured to obtain an element attribute of the target element and a first screenshot picture of the target element in the current application interface. The second obtaining module is configured to obtain a changed layout file when detecting that a layout of the current application interface is changed. The identifying module is configured to identify a candidate element corresponding to the target element in the changed layout file according to the first screenshot picture and the element attribute, and update the target element in the set element information tree according to an identification result.

9. An electronic device, comprising: The computer program product comprises: The memory is configured to store executable computer instructions. The processor is configured to execute the method according to any one of claims 1-7 under the control of the executable computer instructions.

10. A computer readable storage medium, wherein, The computer program product comprises: The computer program product comprises:

Images