Webpage element positioning method and device, equipment, storage medium and program product

Abstract

The invention relates to the technical field of automatic testing, in particular to a webpage element positioning method and device, equipment, a storage medium and a program product, and the method comprises the steps that in response to a selection operation of a target element in a front-end page, a first tree structure corresponding to the target element in a webpage is obtained; according to a current node corresponding to the target element in the first tree structure and an associated node of a link where the current node is located, performing node cutting on the first tree structure to obtain a cut target format text; selecting a target node from the associated nodes according to the semantic information of the associated nodes; according to the semantic information of the target node, the target format text and the element positioning description information, cue words are obtained; and generating a positioning code according to the cue word and the target model. According to the design method for conducting UI element positioning based on the large model, the finally obtained positioning code has semantics and is high in readability, and the professional requirement for research and development workers does not need to be too high.

Description

Technical Field

[0001] The present disclosure relates to the field of automated testing technology, and in particular to a method, device, equipment, storage medium and program product for locating web page elements. Background Art

[0002] Web UI automation testing refers to the process of using tools and scripts to automatically test the web user interface. Its purpose is to ensure that the various functions and interface elements of the web page work properly on different browsers and devices, thereby improving the user experience. UI automation testing can replace traditional manual testing and quickly execute a large number of test cases. It is a very important part of testing. When writing web UI automation testing, it is often necessary to locate the elements to be operated on the web page, and then test the located elements.

[0003] In addition to the above-mentioned manual testing, traditional element locating methods also include: calculating the CSS selector / XPath path selector of the element through a fixed algorithm; and calculating the element locator by configuring different weights for attributes.

[0004] However, the selectors generated by the above-mentioned traditional element positioning methods are not semantic, have poor readability, and require too high professional quality of R&D staff. Summary of the invention

[0005] In view of this, the present disclosure provides a method, apparatus, device, storage medium and program product for locating web page elements. The selector generated by the traditional element locating method does not have semantics, has poor readability, and places too high a requirement on the professional quality of R&D staff.

[0006] In a first aspect, the present disclosure provides a method for locating a web page element, the method comprising:

[0007] In response to a selection operation of a target element in a front-end page, a first tree structure corresponding to the target element in the web page is obtained;

[0008] Execute node pruning on the first tree structure according to the current node corresponding to the target element in the first tree structure and the associated node of the link where the current node is located, to obtain a pruned target format text;

[0009] According to the semantic information of the associated nodes, a target node is selected from the associated nodes;

[0010] Obtain prompt words according to the semantic information of the target node, the target format text and the element location description information;

[0011] A positioning code is generated according to the prompt word and the target model, wherein the positioning code is used to locate the target element in the front-end page.

[0012] In the disclosed embodiment, by responding to the selection operation of the target element in the front-end page, the first tree structure corresponding to the target element in the web page is obtained; according to the current node corresponding to the target element in the first tree structure and the associated node of the link where the current node is located, the first tree structure is subjected to node clipping to obtain the clipped target format text; according to the semantic information of the associated node, the target node is selected from the associated node; according to the semantic information of the target node, the target format text and the element positioning description information, the prompt word is obtained; according to the prompt word and the target model, the positioning code is generated, wherein the positioning code is used to locate the target element in the front-end page. In this way, since the first tree structure corresponding to the target element in the web page is clipped, the complexity of generating the prompt word is reduced, and the design method for positioning UI elements based on the large model makes the final positioning code have semantics and strong readability, and there is no need to place too high professional requirements on R&D staff.

[0013] In an optional implementation, obtaining a first tree structure corresponding to a target element in a webpage includes:

[0014] Get the web page information of the front-end page;

[0015] Generate a corresponding second tree structure according to the web page information;

[0016] Obtaining a current node whose attribute identifier is a target identifier in the second tree structure, wherein the current node corresponds to the target element;

[0017] The first tree structure corresponding to the current node is obtained from the second tree structure.

[0018] In the disclosed embodiment, the target element is found according to the target identifier, and then the current node is found from the target element. The first tree structure is obtained from the second tree structure based on the current node, thereby narrowing the scope of traversing the tree structure to locate the target element.

[0019] In an optional implementation, according to the current node corresponding to the target element in the first tree structure and the associated node of the link where the current node is located, performing node clipping on the first tree structure to obtain the clipped target format text includes:

[0020] According to the first tree structure, obtaining the position relationship between the associated node and the current node;

[0021] According to the position relationship, obtain the sibling nodes of the current node, the ancestor node of the target element, and the sibling nodes of the ancestor node in the associated nodes;

[0022] Determine the clippable nodes based on the sibling nodes of the target element, the ancestor nodes of the target element, and the sibling nodes of the ancestor nodes;

[0023] Get the attributes of the target element, the class name of the target element, and the text information of the target element;

[0024] According to the attributes of the target element, the class name of the target element and the text information of the target element, the node to be retained is determined from the clippable nodes to obtain the target clipping node, wherein the target clipping node is the node in the clippable nodes except the node to be retained;

[0025] According to the target, the nodes are cut to obtain the target tree structure;

[0026] Get the corresponding target format text according to the target tree structure.

[0027] In the disclosed embodiment, the DOM tree is pruned to retain nodes useful for generating prompt words, thereby facilitating the generation of accurate positioning codes.

[0028] In an optional implementation, determining the nodes to be retained from the clippable nodes according to the attributes of the target element, the class name of the target element, and the text information of the target element to obtain the target clipping node includes:

[0029] If there is a reference node in the clippable node that has the same attribute as the target element or the same class name as the target element or contains text information of the target element, the reference node and its ancestor node are retained to obtain the target clipping node.

[0030] In an optional implementation, determining the clippable node based on the sibling nodes of the target element, the ancestor node of the target element, and the sibling nodes of the ancestor node includes:

[0031] Based on the first tree structure, traverse upward from the current node and the nodes at the same level of the current node until reaching the marked node or the root node, then stop traversing and obtain the target range;

[0032] The clippable nodes are determined based on the target range, the sibling nodes of the target element, the ancestor nodes of the target element, and the sibling nodes of the ancestor nodes.

[0033] In the disclosed embodiment, the DOM tree clipping logic is set to prevent the length of a complex page from being too long, which causes the length of the combined prompt words to exceed the limit.

[0034] In an optional implementation, selecting a target node from the associated nodes according to the semantic information of the associated nodes includes:

[0035] Obtaining a candidate node whose semantic information includes an identification attribute according to the semantic information of the associated node, wherein the identification attribute is used to uniquely represent the candidate node;

[0036] A target node is selected from candidate nodes according to preset conditions, wherein the preset conditions are used to narrow the scope of selecting the target node.

[0037] In the disclosed embodiment, the candidate nodes and target nodes obtained by identifying attributes can achieve the effect of fine-tuning the page structure while still taking effect on element positioning. By selecting the target node according to the set preset conditions, the element positioning can have semantic information and improve readability.

[0038] In an optional implementation, selecting a target node from candidate nodes according to a preset condition includes:

[0039] If there is a node among the candidate nodes whose semantic information is different from that of other nodes, whose attribute name representing the target element can uniquely match the target element, and whose distance from the target element is less than a preset distance, then the node is taken as the target node.

[0040] In an optional implementation, after generating the positioning code according to the prompt word and the target model, the method further includes:

[0041] Store target format text and positioning codes;

[0042] Train and optimize the target model based on the target format text and positioning code.

[0043] In a second aspect, the present disclosure provides a device for locating a web page element, the device comprising:

[0044] An acquisition module, configured to respond to a selection operation of a target element in a front-end page and acquire a first tree structure corresponding to the target element in the web page;

[0045] A clipping module, configured to perform node clipping on the first tree structure according to the current node corresponding to the target element in the first tree structure and the associated node of the link where the current node is located, to obtain a clipped target format text;

[0046] A selection module is used to select a target node from the associated nodes according to the semantic information of the associated nodes;

[0047] A obtaining module is used to obtain a prompt word according to the semantic information of the target node, the target format text and the element location description information;

[0048] The generation module is used to generate a positioning code according to the prompt word and the target model, wherein the positioning code is used to locate the target element in the front-end page.

[0049] In a third aspect, the present disclosure provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for locating web page elements of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0050] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for locating a web page element of the first aspect or any corresponding embodiment thereof.

[0051] In a fifth aspect, the present disclosure provides a computer program product, including computer instructions, where the computer instructions are used to enable a computer to execute the method for locating a web page element in the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 is a flow chart of a method for locating a web page element according to an embodiment of the present disclosure;

[0054] Figure 2 is a schematic diagram of an interactive window and function description according to an embodiment of the present disclosure;

[0055] Figure 3 is a schematic diagram of the clipping logic according to the disclosed embodiment of the present invention;

[0056] Figure 4 is a schematic diagram of clipping logic according to another embodiment disclosed in the present invention;

[0057] Figure 5 is a schematic diagram of a clipping logic according to another embodiment disclosed in the present invention;

[0058] Figure 6 is a structural block diagram of a device for locating web page elements according to an embodiment of the present disclosure;

[0059] Figure 7 It is a schematic diagram of the hardware structure of the computer device of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0061] The purpose of web page testing is to make the various functions and interface elements of the web page work properly on different browsers and devices. The existing testing methods include manual testing, but manual testing requires testers to read and check the HTML / CSS code structure of the web page, and then write appropriate code and verify it. Manual writing is inefficient and has high requirements for test writers.

[0062] In order to solve the defects of manual testing, related technologies propose a method to locate the elements to be operated on the web page, and then automatically test the located elements, such as: (1) Calculate the CSS selector / XPath path selector of the element through a fixed algorithm: obtain the element clicked by the user through a script, and traverse the CSS attributes or path attributes of the link where the element is located in turn to calculate the relative or complete CSS selector / XPath path selector. The disadvantage of this solution is that the generated selector is not semantic. When the page is slightly changed, the corresponding element selector is likely to need to be modified. (2) Calculate the element locator by configuring different weights for the attributes: select certain attributes according to different weights, and finally form an element locator. The disadvantage of this solution is that the generated selector does not consider the semantics of the attribute itself, and cannot automatically select the appropriate attribute for positioning based on the meaning contained in the attribute itself.

[0063] In order to solve the above problems, according to an embodiment of the present disclosure, a method embodiment of locating web page elements is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0064] In this embodiment, a method for locating a web page element is provided. Figure 1 is a flow chart of a method for locating a web page element according to an embodiment of the present disclosure. Figure 1 As shown, the method can be applied to the server side, and the process includes the following steps:

[0065] Step S101, in response to a selection operation of a target element in a front-end page, a first tree structure corresponding to the target element in the web page is obtained.

[0066] Optionally, the user activates the function of selecting elements in the front-end page through the UI interaction of the browser plug-in, and selects the target element to be located on the front-end page. Figure 2 As shown, the target element to be located is selected by using the start selection button, and then the corresponding HTML document is displayed in the backend web page, and the corresponding first tree structure is generated based on the HTML document. The first tree structure refers to a DOM (Document Object Model) tree.

[0067] Step S102: prune nodes on the first tree structure according to the current node corresponding to the target element in the first tree structure and the associated nodes of the link where the current node is located, to obtain a pruned target format text.

[0068] Optionally, since the information on the web page is abstracted into the first tree structure, in order to avoid the length of the complex page being too long, resulting in the length of the generated prompt word (ie, Prompt) exceeding the limit, the DOM tree needs to be pruned without affecting the accuracy of positioning.

[0069] Furthermore, in order to indicate the target element to be located in the XML format file representing the DOM tree, a special attribute is added to the target element node, such as (markID="target"). It should be noted that this special attribute is only used to mark the target element and cannot be used for positioning.

[0070] Then, the first tree structure is carefully analyzed to obtain the current node corresponding to the target element in the first tree structure and the associated node of the link where the current node is located. Then, the first tree structure is node-pruned based on the current node and the associated node, and then the pruned target tree structure is obtained. Then, the target tree structure is analyzed again to obtain the target format text. The target format text is actually an XML format text, for example: <div xmlns＝"http： xxxhtml"class＝"part activitybar right"id＝"workbench.parts.activitybar"role＝"none"> <div class＝"content"> <div class＝"icube-sidebar-aiicube-sidebar-active"> <div class＝"icube-sidebar-ai-inner"markid＝"target"> .

[0071] It should be noted that the current node is the node where the target element is located in the first tree structure, and the link where the current node is located refers to the link on one side of the first tree structure that contains the current node, such as Figure 3 As shown in FIG. 1 , the link where the current node is located is the right link of the tree structure. The right link includes the current node and multiple associated nodes associated with the current node. Figure 3 In , the associated node can be a node at the same level as the current node, or a node above the current node.

[0072] Step S103: selecting a target node from the associated nodes according to the semantic information of the associated nodes.

[0073] Optionally, the associated nodes contain semantic information, and the target node is selected from multiple associated nodes based on multiple dimensions of the semantic information, such as clarity (determined by whether the target element can be clearly represented), uniqueness (whether it is unique to a certain associated node), etc. In this case, the target node is usually the parent node or child node of the current node or the node itself.

[0074] Step S104, obtaining a prompt word according to the semantic information of the target node, the target format text and the element location description information.

[0075] Optionally, after obtaining the target node, its corresponding semantic information can also be obtained, and then combined into a prompt word based on the semantic information of the target node, the target format XML text of the target tree structure, and some element positioning description information (i.e., the role information required to generate the large model prompt word, the task information required to generate the large model prompt word, the constraint conditions required to generate the large model prompt word, and the positive and negative examples required to generate the large model prompt word).

[0076] The role information can be: You are a professional test automation developer, focusing on the automation development of the Playwright framework, using the programming language **{language}**, and providing Locator writing suggestions to other developers.

[0077] The task information may be: generate a robust, semantically reasonable, and stable Playwrightlocator code for the target element.

[0078] Since id information is usually immutable, the constraints can be: (1) If the target element or its ancestor element contains meaningful id information, the id information must be used. However, the markID="target" manually defined for the target element cannot be used. (2) The preferred Playwright methods are: `getByRole` (locating by the role attribute of the element), `getByLabel` (locating by the label attribute of the element), `getByText` (locating by the text attribute of the element), `locator` (locating by the locator of the element), `getByTitle` (locating by the title attribute of the element). (3) The locating code can uniquely match an element. (4) If using attribute locating, the writing must comply with the requirements of the Playwright framework and use the text-containing writing method "[attri*=text]". (5) The preferred attributes are: "id" (account), "role" (role), "aria-label" (label), "data-testid" (data id), "data" (data), and "placeholder" (placeholder).

[0079] Positive and negative examples can be: good locator: the locator code obtained using the id `page.locator('#active-frame')`; bad locator: a locator code different from the good locator mentioned above.

[0080] It should be noted that the above process of obtaining the semantic information of the target node is actually the workflow required to generate the large model prompt word, which is combined with the target format text and the element positioning description information to obtain the prompt word.

[0081] Step S105, generating a positioning code according to the prompt word and the target model, wherein the positioning code is used to locate the target element in the front-end page.

[0082] Optionally, the prompt words combined in the previous step are sent to the target model (i.e., the large model), and the large model generates the target element positioning information, which is then post-processed to remove redundant text, retain only the positioning code that can locate the target element, and returned to the front-end page for display.

[0083] If the target element is not found based on the positioning code, the front-end page will also display "No suitable positioning was found, it is recommended to add data-testid positioning" to remind R&D personnel to make improvements.

[0084] like Figure 2As shown in the figure, the interactive window contains a result display area (including the positioning results generated by the large model and the positioning results generated by the existing fixed algorithm). 15 page elements were selected for experimental generation and evaluated from the perspectives of uniqueness (standard: accurate to a certain element on the page), semantics (standard: element positioning has element semantic information), and stability (standard: element positioning is still effective when the page structure is fine-tuned). Among them, 86.7% (13 / 15) of them met the uniqueness, 66.7% (10 / 15) met the semantics, and 73.3% (11 / 15) met the stability.

[0085] The interactive window also includes a copy button. Using Smart Locator, you can directly generate positioning information, which can be copied and used. The time for element positioning can be saved by 90%, and the efficiency of UI automation writing can be improved.

[0086] In addition, the interactive window also includes a verification button, a column area for user input element positioning and verification, and a column area for displaying information of the selected element, all of which can be used to verify the accuracy and efficiency of the method for locating web page elements proposed in the embodiment of the present disclosure.

[0087] In the disclosed embodiment, by responding to the selection operation of the target element in the front-end page, the first tree structure corresponding to the target element in the web page is obtained; according to the current node corresponding to the target element in the first tree structure and the associated node of the link where the current node is located, the first tree structure is subjected to node clipping to obtain the clipped target format text; according to the semantic information of the associated node, the target node is selected from the associated node; according to the semantic information of the target node, the target format text and the element positioning description information, the prompt word is obtained; according to the prompt word and the target model, the positioning code is generated, wherein the positioning code is used to locate the target element in the front-end page. In this way, since the first tree structure corresponding to the target element in the web page is clipped, the complexity of generating the prompt word is reduced, and the design method for positioning UI elements based on the large model makes the final positioning code have semantics and strong readability, and there is no need to place too high professional requirements on R&D staff.

[0088] In some optional implementations, obtaining a first tree structure corresponding to a target element in a webpage includes:

[0089] Get the web page information of the front-end page;

[0090] Generate a corresponding second tree structure according to the web page information;

[0091] Obtaining a current node whose attribute identifier is a target identifier in the second tree structure, wherein the current node corresponds to the target element;

[0092] The first tree structure corresponding to the current node is obtained from the second tree structure.

[0093] Optionally, in the disclosed embodiment, the backend generates corresponding web page information based on the frontend page, and then a tree structure is abstracted based on the web page information, such as the content and behavior of the web page, which is called the second tree structure. The current node with the attribute identifier as the target identifier (such as id="xxx") is obtained from the second tree structure, such as Figure 3 As shown, the markID="target" is located at a node called the current node, and then the first tree structure is selected from the second tree structure based on the current node, wherein the first tree structure is a tree structure that is included in the second tree structure and includes the current node.

[0094] In the disclosed embodiment, the target element is found according to the target identifier, and then the current node is found from the target element. The first tree structure is obtained from the second tree structure based on the current node, thereby narrowing the scope of traversing the tree structure to locate the target element.

[0095] In some optional implementations, performing node pruning on the first tree structure according to the current node corresponding to the target element in the first tree structure and the associated node of the link where the current node is located to obtain the pruned target format text includes:

[0096] According to the first tree structure, obtaining the position relationship between the associated node and the current node;

[0097] According to the position relationship, obtain the sibling nodes of the current node, the ancestor node of the target element, and the sibling nodes of the ancestor node in the associated nodes;

[0098] Determine the clippable nodes based on the sibling nodes of the target element, the ancestor nodes of the target element, and the sibling nodes of the ancestor nodes;

[0099] Get the attributes of the target element, the class name of the target element, and the text information of the target element;

[0100] According to the attributes of the target element, the class name of the target element and the text information of the target element, the node to be retained is determined from the clippable nodes to obtain the target clipping node, wherein the target clipping node is the node in the clippable nodes except the node to be retained;

[0101] According to the target, the nodes are cut to obtain the target tree structure;

[0102] Get the corresponding target format text according to the target tree structure.

[0103] Alternatively, if Figure 3As shown, according to the first tree structure, the positional relationship between the current node corresponding to the target element with markID="target" and its associated nodes can be determined. At this time, it is necessary to obtain the sibling nodes of the current node, the ancestor node of the target element, and the sibling nodes of the ancestor node in the associated nodes, that is, Figure 3 Then move the nodes outside the white square (i.e. Figure 3 The nodes corresponding to the black squares in the figure are used as cuttable nodes.

[0104] Then, the attributes, class name, and text information of the target element are obtained through the background program. If there is a reference node in the clippable node that has the same attributes as the target element, the same class name as the target element, or contains the text information of the target element, such as Figure 4 As shown, the reference node (such as Figure 4 The square block M) and the ancestor node of the link where the reference node is located (such as Figure 4 Block N) is used as a reserved node because the class name of the element in block M is the same as the class name of the target element of the current node.

[0105] Then the nodes in the prunable nodes except the nodes to be retained are called target prunable nodes, corresponding to Figure 4 The 6 black squares in the figure are cut into target nodes to obtain the target tree structure, and then the corresponding target format text is generated.

[0106] In the disclosed embodiment, the DOM tree is pruned to retain nodes useful for generating prompt words, thereby facilitating the generation of accurate positioning codes.

[0107] In some optional implementations, determining the clippable nodes based on the sibling nodes of the target element, the ancestor nodes of the target element, and the sibling nodes of the ancestor nodes includes:

[0108] Based on the first tree structure, traverse upward from the current node and the nodes at the same level of the current node until reaching the marked node or the root node, then stop traversing and obtain the target range;

[0109] The clippable nodes are determined based on the target range, the sibling nodes of the target element, the ancestor nodes of the target element, and the sibling nodes of the ancestor nodes.

[0110] Alternatively, if Figure 5As shown, based on the first tree structure, the clipping is from the bottom to the top, starting from the current node and the nodes at the same level of the current node and traversing upward until reaching the marked node (such as a node containing an id or a test id) or the root node, then the traversal is stopped to obtain the target range, which is actually the range of nodes to be retained. Then, based on the sibling nodes of the target element, the ancestor nodes of the target element, and the sibling nodes of the ancestor nodes within this target range, the nodes to be retained and the nodes to be clipped are determined, wherein the nodes to be clipped are the current clippable nodes.

[0111] Figure 5 When the node with id = 'sidePanel' is reached, the traversal will no longer proceed upwards. The portion within the corresponding dotted box is the reserved portion. Since the current clipping traversal reaches the node containing the id, it is not necessary to reach the root node, and the current traversal ends.

[0112] In the disclosed embodiment, the DOM tree clipping logic is set to prevent the length of a complex page from being too long, which causes the length of the combined prompt words to exceed the limit.

[0113] In some optional implementations, selecting a target node from the associated nodes according to the semantic information of the associated nodes includes:

[0114] Obtaining a candidate node whose semantic information includes an identification attribute according to the semantic information of the associated node, wherein the identification attribute is used to uniquely represent the candidate node;

[0115] A target node is selected from candidate nodes according to preset conditions, wherein the preset conditions are used to narrow the scope of selecting the target node.

[0116] It is possible to analyze the target element, obtain the semantic information of the associated nodes, and extract the nodes containing identification attributes in the semantic information as candidate nodes, wherein the identification attribute refers to an attribute that will not change easily and can uniquely represent the node, wherein the attributes that will not change easily include the following identifiers: "aria-label" (label), "data-testid" (data ID), "data" (data), "placeholder" (placeholder), etc.

[0117] Then, the target node is selected from the candidate nodes according to the preset conditions, wherein the preset conditions are used to narrow the scope of selecting the target node. For example, "the semantic information is different from that of other nodes, and the attribute name representing the target element can uniquely match the target element, and the distance to the target element is close, such as less than the preset distance 0.1" is used as the preset condition to select the target node from multiple candidate nodes.

[0118] Assume that className = 'sideChat' is unique to a candidate node and not found in other nodes. Moreover, the meaning of this attribute name is clear enough and can uniquely match the target element, and the distance from the target element is less than the preset distance. Then the node corresponding to this attribute can be used as the target node.

[0119] In the disclosed embodiment, the candidate nodes and target nodes obtained by identifying attributes can achieve the effect of fine-tuning the page structure while still taking effect on element positioning. By selecting the target node according to the set preset conditions, the element positioning can have semantic information and improve readability.

[0120] In some optional implementations, after generating the positioning code according to the prompt word and the target model, the method further includes:

[0121] Store target format text and positioning codes;

[0122] Train and optimize the target model based on the target format text and positioning code.

[0123] Optionally, the backend service stores the clipped target format text and the positioning code generated by the large model for training and optimization of the large model. The training and optimization are performed offline. This step can improve the accuracy of the large model positioning.

[0124] In this embodiment, a device for locating web page elements is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0125] This embodiment provides a device for locating web page elements, such as Figure 6 As shown, including:

[0126] The acquisition module 601 is used to acquire a first tree structure corresponding to the target element in the webpage in response to a selection operation of the target element in the front-end page;

[0127] A clipping module 602, configured to perform node clipping on the first tree structure according to the current node corresponding to the target element in the first tree structure and the associated nodes of the link where the current node is located, to obtain a clipped target format text;

[0128] A selection module 603 is used to select a target node from the associated nodes according to the semantic information of the associated nodes;

[0129] Obtaining module 604, for obtaining a prompt word according to the semantic information of the target node, the target format text and the element location description information;

[0130] The generating module 605 is used to generate a positioning code according to the prompt word and the target model, wherein the positioning code is used to locate the target element in the front-end page.

[0131] In some optional implementations, the acquisition module 601 includes:

[0132] A first acquisition unit, used to acquire web page information of a front-end page;

[0133] A generating unit, used to generate a corresponding second tree structure according to the web page information;

[0134] A second acquisition unit, configured to acquire a current node whose attribute identifier is a target identifier in a second tree structure, wherein the current node corresponds to a target element;

[0135] The third acquisition unit is used to acquire the first tree structure corresponding to the current node from the second tree structure.

[0136] In some optional implementations, the clipping module 602 includes:

[0137] A fourth acquisition unit, used for acquiring a position relationship between the associated node and the current node according to the first tree structure;

[0138] A fifth acquisition unit, used to acquire, according to the position relationship, the brother nodes of the current node, the ancestor node of the target element, and the brother nodes of the ancestor node in the associated nodes;

[0139] A determination unit, used to determine the clippable nodes based on the sibling nodes of the target element, the ancestor nodes of the target element, and the sibling nodes of the ancestor nodes;

[0140] A sixth acquisition unit, used to acquire the attribute of the target element, the class name of the target element and the text information of the target element;

[0141] A retaining unit, used to determine the nodes to be retained from the clippable nodes according to the attributes of the target element, the class name of the target element and the text information of the target element, and obtain a target clipping node, wherein the target clipping node is a node in the clippable nodes except the nodes to be retained;

[0142] A first obtaining unit is used to obtain a target tree structure according to target clipping nodes;

[0143] The seventh acquisition unit is used to acquire the corresponding target format text according to the target tree structure.

[0144] In some optional embodiments, the retaining unit includes:

[0145] The retaining submodule is used to retain the reference node and the ancestor node of the reference node to obtain the target clipping node if there is a reference node in the clippable node that has the same attribute as the target element or the same class name as the target element or contains the text information of the target element.

[0146] In some optional implementations, the determining unit includes:

[0147] A submodule is obtained, which is used to traverse upward from the current node and the nodes at the same level of the current node based on the first tree structure until a marked node or a root node is reached, and then the traversal is stopped to obtain a target range;

[0148] The determination submodule is used to determine the clippable nodes based on the target range, the sibling nodes of the target element, the ancestor nodes of the target element, and the sibling nodes of the ancestor nodes.

[0149] In some optional implementations, the selection module 603 includes:

[0150] A second obtaining unit is used to obtain a candidate node whose semantic information contains an identification attribute according to the semantic information of the associated node, wherein the identification attribute is used to uniquely represent the candidate node;

[0151] The selection unit is used to select a target node from the candidate nodes according to a preset condition, wherein the preset condition is used to narrow the scope of selecting the target node.

[0152] In some optional implementations, the selection unit includes:

[0153] A submodule is set to use the node as the target node if there is a node in the candidate node whose semantic information is different from that of other nodes, whose attribute name representing the target element can uniquely match the target element, and whose distance from the target element is less than a preset distance.

[0154] In some optional embodiments, the device further comprises:

[0155] A storage module, used for storing target format text and positioning code;

[0156] The optimization module is used to train and optimize the target model based on the target format text and positioning code.

[0157] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0158] The web page element positioning device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0159] The present disclosure also provides a computer device having the above Figure 6 Means for positioning elements of the web page shown.

[0160] See also Figure 7 , Figure 7 is a schematic diagram of a computer device provided by an optional embodiment of the present disclosure, such as Figure 7 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0161] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0162] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0163] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0164] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0165] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0166] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0167] A part of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present disclosure through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0168] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for locating a web page element, characterized in that: The method comprises: In response to a selection operation of a target element in a front-end page, obtaining a first tree structure corresponding to the target element in the web page; Execute node pruning on the first tree structure according to the current node corresponding to the target element in the first tree structure and the associated node of the link where the current node is located, to obtain a pruned target format text; Selecting a target node from the associated nodes according to the semantic information of the associated nodes; Obtaining a prompt word according to the semantic information of the target node, the target format text and the element location description information; A positioning code is generated according to the prompt word and the target model, wherein the positioning code is used to locate the target element in the front-end page.

2. The method according to claim 1, characterized in that The obtaining of a first tree structure corresponding to the target element in the webpage includes: Get the web page information of the front-end page; Generate a corresponding second tree structure according to the webpage information; Acquire the current node whose attribute identifier is the target identifier in the second tree structure, wherein the current node corresponds to the target element; The first tree structure corresponding to the current node is obtained from the second tree structure.

3. The method according to claim 1, characterized in that The performing node pruning on the first tree structure according to the current node corresponding to the target element in the first tree structure and the associated node of the link where the current node is located to obtain the pruned target format text includes: According to the first tree structure, acquiring a positional relationship between the associated node and the current node; According to the positional relationship, obtain the sibling nodes of the current node, the ancestor node of the target element, and the sibling nodes of the ancestor node in the associated nodes; Determine a clippable node based on the sibling nodes of the target element, the ancestor node of the target element, and the sibling nodes of the ancestor node; Obtaining the attributes of the target element, the class name of the target element, and the text information of the target element; According to the attribute of the target element, the class name of the target element and the text information of the target element, determine the node to be retained from the clippable nodes to obtain the target clipping node, wherein the target clipping node is the node in the clippable nodes except the node to be retained; Obtain a target tree structure according to the target clipping nodes; The corresponding target format text is acquired according to the target tree structure.

4. The method according to claim 3, characterized in that The determining the nodes to be retained from the clippable nodes according to the attributes of the target element, the class name of the target element and the text information of the target element to obtain the target clipping node includes: If there is a reference node in the clippable node that has the same attribute as the target element or the same class name as the target element or contains text information of the target element, the reference node and its ancestor node are retained to obtain the target clipping node.

5. The method according to claim 3, characterized in that: The determining the clippable nodes based on the sibling nodes of the target element, the ancestor nodes of the target element, and the sibling nodes of the ancestor nodes includes: Based on the first tree structure, traverse upward from the current node and nodes at the same level of the current node until reaching a marked node or a root node, then stop traversing to obtain a target range; The clippable node is determined based on the target range, the sibling nodes of the target element, the ancestor node of the target element, and the sibling nodes of the ancestor node.

6. The method according to claim 1, characterized in that The selecting a target node from the associated nodes according to the semantic information of the associated nodes includes: Obtaining a candidate node whose semantic information contains an identification attribute according to the semantic information of the associated node, wherein the identification attribute is used to uniquely characterize the candidate node; The target node is selected from the candidate nodes according to a preset condition, wherein the preset condition is used to narrow the range of selecting the target node.

7. The method according to claim 6, characterized in that The selecting the target node from the candidate nodes according to a preset condition comprises: If there is a node among the candidate nodes whose semantic information is different from that of other nodes, whose attribute name representing the target element can uniquely match the target element, and whose distance from the target element is less than a preset distance, then the node is used as the target node.

8. The method according to any one of claims 1 to 7, characterized in that: After generating the positioning code according to the prompt word and the target model, the method further includes: storing the target format text and the positioning code; The target model is trained and optimized based on the target format text and the positioning code.

9. A device for locating web page elements, characterized in that: The device comprises: An acquisition module, configured to, in response to a selection operation of a target element in a front-end page, acquire a first tree structure corresponding to the target element in the web page; A clipping module, configured to perform node clipping on the first tree structure according to a current node corresponding to the target element in the first tree structure and an associated node of a link where the current node is located, to obtain a clipped target format text; A selection module, used for selecting a target node from the associated nodes according to the semantic information of the associated nodes; An obtaining module, used for obtaining a prompt word according to the semantic information of the target node, the target format text and the element location description information; A generation module is used to generate a positioning code according to the prompt word and the target model, wherein the positioning code is used to locate the target element in the front-end page.

10. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for locating a web page element according to any one of claims 1 to 8 by executing the computer instructions.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for locating a web page element according to any one of claims 1 to 8.

12. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to cause a computer to execute the method for locating a web page element according to any one of claims 1 to 8.

Images