Element positioning method, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an element positioning method, electronic equipment and a storage medium, and the method comprises the steps: analyzing a user interface, and extracting semantic information of a target element; semantic features of the semantic information are extracted, and an element positioning model is constructed based on the semantic features; and positioning a target element in a to-be-identified interface by using the element positioning model. According to the embodiment of the invention, the target element in the to-be-detected interface is positioned by utilizing the element positioning model, accurate element positioning can be realized under the condition that the HTML page structure is changed, and the interface testing efficiency is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of interface technology, and in particular to an element positioning method, electronic device, and storage medium. Background Art

[0002] In automated testing of web interfaces, such as web pages and application program interfaces, element positioning is often required to ensure the proper functionality of interface elements. The element positioning methods used in related technologies rely on the specific structure and attributes of the HTML page of the web interface. However, in actual development, the structure of HTML pages may change frequently. For example, changes in the hierarchy, attributes, or identifiers of elements can render the element positioning methods ineffective, leading to test failures and thus affecting test efficiency. Summary of the Invention

[0003] In view of this, it is necessary to provide an element positioning method, electronic device and storage medium to solve the above-mentioned problem that the element positioning method becomes invalid due to the frequent changes in the structure of the HTML page during the development of the Web interface.

[0004] In a first aspect, an embodiment of the present application provides an element positioning method, which is applied to an electronic device. The method includes: parsing a user interface to extract semantic information of a target element; extracting semantic features of the semantic information and constructing an element positioning model based on the semantic features; and using the element positioning model to locate the target element in the interface to be identified.

[0005] In a possible implementation, parsing the user interface and extracting semantic information of the target element includes: parsing the Hypertext Markup Language (HTML) page of the user interface, identifying the type of the target element, and extracting the text and context of the target element.

[0006] In one possible implementation, extracting the semantic features of the semantic information includes: preprocessing the semantic information, including one or more of word segmentation, stop word removal, and part-of-speech tagging; and performing word embedding on the preprocessed semantic information to obtain semantic features.

[0007] In a possible implementation, the extracting of semantic features of the semantic information further includes: performing dimensionality reduction processing on the semantic features of the semantic information to determine key features in the semantic features; and assigning weights to the key features based on a rule engine.

[0008] In one possible implementation, the constructing of the element positioning model based on the semantic features includes: constructing a training data set based on the semantic features of the target element and the position information of the target element in the user interface, the semantic features being input data and the position information being output data; training a preset deep learning model based on the training data set to obtain the predicted position information of the target element corresponding to each set of training data; calculating a loss function value based on the predicted position information of the target element and the position information in the corresponding training data; adjusting the parameters of the preset deep learning model until the loss function value meets preset conditions, thereby obtaining the trained deep learning model as the element positioning model.

[0009] In one possible implementation, the use of the element positioning model to locate the target element in the interface to be identified includes: using the element positioning model to locate the target element in the HTML page of the interface to be identified, and obtaining the position information of the target element in the interface to be identified.

[0010] In a possible implementation, the method further includes: verifying the location information, and determining that the location information is qualified if a function of an element corresponding to the location information is the same as a preset function.

[0011] In a possible implementation, the location information of the target element is an Extensible Markup Language path XPath or a Cascading Style Sheet CSS selector.

[0012] In a second aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory is used to store instructions, and the processor is used to call the instructions in the memory so that the electronic device executes the above-mentioned element positioning method.

[0013] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned element positioning method.

[0014] The element positioning method, electronic device and storage medium provided in the embodiments of the present application parse the existing interface to obtain semantic information of each element, extract semantic features of the elements based on the semantic information, construct an element positioning model based on the semantic features, and use the element positioning model to locate the target elements in the interface to be detected. Accurate element positioning can also be achieved when the HTML page structure changes, effectively improving the efficiency of interface testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0016] Figure 1 This is a flowchart of an element positioning method provided in one embodiment of the present application.

[0017] Figure 2 A flowchart of constructing an element positioning model provided in one embodiment of the present application.

[0018] Figure 3 A flowchart of an element positioning method provided in another embodiment of the present application.

[0019] Figure 4 A schematic structural diagram of an element positioning device provided in one embodiment of the present application.

[0020] Figure 5 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0021] It should be noted that, in this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A alone, A and B together, and B alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," and so on (if any) in the specification, claims, and drawings of this application are used to distinguish similar objects, not to describe a specific order or precedence.

[0022] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete manner. The following embodiments and features in the embodiments may be combined with each other unless there is a conflict.

[0023] In automated testing of web interfaces, such as web pages and application program interfaces, element positioning is often required to ensure the proper functionality of interface elements. The element positioning methods used in related technologies rely on the specific structure and attributes of the HTML page of the web interface. However, in actual development, the structure of HTML pages may change frequently. For example, changes in the hierarchy, attributes, or identifiers of elements can render the element positioning methods ineffective, leading to test failures and thus affecting test efficiency.

[0024] Element positioning methods in related technologies include element positioning based on fixed paths, attribute values, image recognition, etc. Element positioning based on fixed paths such as XPath or CSS selectors is sensitive to changes in HTML page structure and has high maintenance costs. Element positioning based on attribute values such as ID and Class may fail to position because attribute values may be dynamically modified or reused. Element positioning based on image recognition methods such as OCR technology achieves positioning by identifying page elements, but this method has high computational complexity and is greatly affected by page rendering effects.

[0025] To solve the above problems, an embodiment of the present application provides an element positioning method, which obtains the semantic information of each element by parsing the existing interface, extracts the semantic features of the elements based on the semantic information, constructs an element positioning model based on the semantic features, and uses the element positioning model to locate the target elements in the interface to be detected. Even when the HTML page structure changes, accurate element positioning can be achieved, effectively improving the efficiency of interface testing.

[0026] See Figure 1 FIG. 1 is a flow chart of an element positioning method provided by an embodiment of the present application. The element positioning method is applied to an electronic device, and the method includes: S101: Parse the user interface to extract semantic information of target elements.

[0027] In one embodiment of the present application, the user interface can be a browser web page or an application program interface, and multiple user interfaces can be obtained by web crawling, or an existing user interface that has passed web testing can be obtained. The semantic information of the target element includes but is not limited to: type, text, and context.

[0028] In one embodiment of the present application, an HTML parsing library (such as Jsoup) or a parser is used to parse the source code of Hypertext Markup Language (HTML) pages of multiple user interfaces, identify the types of multiple target elements, and extract the text and contextual relationship of each target element to collect the semantic information of each target element. For example, the types of target elements may include controls, input boxes, drop-down menus, etc. The text of the target element may include the text content corresponding to the target element, such as the text "Start", "End", "Submit" on the control, the text corresponding to the link, etc. The contextual relationship of the target element can reflect the relationship between the target element and other interface elements in the same hierarchical structure. For example, the target element can be a parent element, a child element, a sibling element, etc.

[0029] For example, for the following HTML code 1: <form> <button type=""submit”"> submit< / button> < / form> , The extracted semantic information includes the element type as "button control", the text as "submit", and the context relationship as "located within a tag".

[0030] For the following HTML code 2: <form id=""loginForm”"> <input type=""text”" placeholder=""请输入用户名”"> < / form> , The extracted semantic information includes the element type "input box", the text "please enter your username", and the context relationship "located in <form id=""loginForm”">tag".

[0031] For the following HTML code 3: <label for=""city”"> Select City< / label> <select id=""city”"> <option value=""1”"> Beijing< / option> <option value=""2”"> Shanghai< / option> < / select> < / form> , The extracted semantic information includes the element type "drop-down menu", the text "select a city", and the context relationship "located in <form>tag".

[0032] S102, extracting semantic features of the semantic information, and building an element positioning model based on the semantic features.

[0033] In one embodiment of the present application, semantic features are extracted based on a feature modeling engine from a collected semantic information dataset, and a semantic feature model, i.e., an element location model, is constructed based on the semantic features to locate target elements in an interface to be identified or detected. The semantic information is preprocessed using one or more of word segmentation, stop word removal, and part-of-speech tagging. Feature extraction is then performed on the preprocessed semantic information to obtain semantic features.

[0034] In one embodiment of the present application, word embedding processing is performed on the pre-processed semantic information, and the text vector of each semantic information is extracted to obtain semantic features. For example, one-hot encoding is performed on each semantic information to obtain a one-hot encoded vector, and the one-hot encoded vector is compressed using a conversion matrix to obtain a short vector. The target word is predicted based on the short vector of each semantic information, and the short vector is adjusted based on the prediction result to obtain the text vector of the semantic information. For example, if the predicted target word is the same as the actual target word, the short vector is determined as the text vector of the semantic information. If the predicted target word is different from the actual target word, the short vector is adjusted, and the target word is predicted based on the adjusted short vector until the predicted target word is the same as the actual target, and the adjusted short vector is determined as the text vector of the semantic information.

[0035] In one embodiment of the present application, dimensionality reduction processing is performed on the semantic features of semantic information to determine key features in the semantic features. For example, a determination is made as to whether the semantic features of each type of semantic information are preset key features. If the semantic features are preset key features, the semantic features are retained; if the semantic features are not preset key features, the semantic features are deleted. For example, preset key features include element type and element text.

[0036] In one embodiment of the present application, weights are assigned to key features based on a machine learning approach or using a rule engine. For example, the rule engine can preset the weight of each key feature, thereby assigning a preset weight to each key feature. For example, the key features include semantic features corresponding to element types and element texts, and the preset weights corresponding to the semantic features of element types are 0.2, 0.3, 0.4 or other values, and the preset weights corresponding to the semantic features of element texts are 0.6, 0.7, 0.8 or other values. In another embodiment of the present application, the rule engine can also assign weights to each key feature based on the importance of the semantic features. For example, the word frequency of semantic information can reflect the importance of the corresponding semantic features. Therefore, the rule engine assigns weights to key features based on the word frequency of semantic information.

[0037] See Figure 2 FIG. 1 is a flowchart of constructing an element positioning model provided by an embodiment of the present application.

[0038] S1011, constructing a training dataset based on the semantic features of the target element and the position information of the target element in the user interface.

[0039] In one embodiment of the present application, the location information of the target element is an Extensible Markup Language (XPath) path or a Cascading Style Sheets (CSS) selector. The training dataset includes multiple sets of training data, each set of training data including semantic features and location information of each target element. The semantic features serve as input data for a preset deep learning model, and the location information serves as output data for the preset deep learning model.

[0040] S1012: Based on the training data set, a preset deep learning model is trained to obtain predicted position information of the target element corresponding to each set of training data.

[0041] In one embodiment of the present application, a preset deep learning model natural language processing model, such as a recurrent neural network model (RNN), a convolutional neural network model (CNN), a transformer model (transformer), or a large language model. The initial parameters of the preset deep learning model are set, and the input features in a set of training data, that is, the semantic features of the target element, are input into the preset deep learning model, and the predicted position information of the target element in the user interface is output through the preset deep learning model. Taking the transformer model as an example, the parameters of the preset deep learning model may include embedding layer and position encoding parameters, self-attention mechanism parameters, feedforward neural network parameters, layer normalization and residual connection parameters, decoder-specific parameters, output layer parameters, etc.

[0042] S1013, calculating a loss function value based on the predicted position information of the target element and the position information in the corresponding training data.

[0043] In one embodiment of the present application, the loss function of the preset deep learning model can be a cross entropy function, a mean square error function, or a masked language model loss function. For example, the cross entropy function is: .

[0044] Among them, in the cross entropy function, is the one-hot encoding of the actual position information of the target element in the training data, is the probability corresponding to the predicted position information, and N is the vocabulary size. The probability of the predicted position information output by the preset deep learning model and the one-hot encoding of the actual position information of the target element in the training data are input into the cross entropy function to obtain the loss function value.

[0045] S1014, adjusting the parameters of the preset deep learning model until the loss function value meets the preset conditions, and obtaining the trained deep learning model as the element positioning model.

[0046] In one embodiment of the present application, a determination is made as to whether the loss function value satisfies a preset condition. If the loss function value does not satisfy the preset condition, the parameters of the preset deep learning model are adjusted, another set of training data is input into the preset deep learning model, and steps S1012 and S1013 are continued to train the preset deep learning model until the loss function value satisfies the preset condition. If the loss function value satisfies the preset condition, it is determined that the preset deep learning model has converged, i.e., training is completed, and the trained deep learning model is used as the element positioning model.

[0047] In one embodiment of the present application, it is determined whether the loss function value is less than or equal to a preset value. If the loss function value is less than or equal to the preset value, it is determined that the loss function value meets the preset condition. If the loss function value is greater than the preset value, it is determined that the loss function value does not meet the preset condition.

[0048] S103: Utilize the element positioning model to locate the target element in the interface to be recognized.

[0049] In one embodiment of the present application, an element positioning model is used to locate a target element in an HTML page of an interface to be identified, thereby obtaining positional information of the target element in the interface to be identified. The HTML page of the interface to be identified is input into the element positioning model, and the element positioning model is used to perform semantic feature matching on each element in the interface to be identified, identifying the matching target element, and outputting the location result of the target element.

[0050] In one embodiment of the present application, the HTML page code of the interface to be identified is input into the element positioning model, the element positioning model is used to extract the semantic information of each element in the interface to be identified, the semantic features of the semantic information of each element are extracted, the semantic features of each element are identified, the semantic features of each element are matched with the semantic features of the target element, and it is determined whether the element is the target element. If the element is the target element, the position information of the element in the interface to be identified is output; if the element is not the target element, no content is output.

[0051] Based on the above embodiments of the present application, the element positioning model is used to locate the target element in the interface to be detected. Even when the HTML page structure changes, accurate element positioning can be achieved, effectively improving the efficiency of interface testing.

[0052] For example, if the above HTML code 1 is changed to: <button type=""submit”"> submit< / button> , The element positioning model can be used to identify the element as a target element through the semantic information "submit" and "button control", and determine the position information of the target element.

[0053] For example, if the above HTML code 2 is changed to: <div class=""login-container”"> <input type=""text”" placeholder=""请输入用户名”"> , The element positioning model can be used to identify the element as a target element through the semantic information "please enter user name" and "input box", and determine the location information of the target element.

[0054] For example, if the above HTML code 3 is changed to: <label> Select City< / label> <select> <option value=""1”"> Beijing< / option> <option value=""2”"> Shanghai< / option> < / select> , The element positioning model can be used to identify the element as the target element through the semantic information "select a city" and "drop-down menu", and determine the location information of the target element.

[0055] For example, if the location information is an XPath, the location information of the target element can be / / select[@id='dynamicOptions'] / option[1]. If the location information is a CSS selector, the location information of the target element can be select#dynamicOptions>option:first-child.

[0056] See Figure 3 FIG. 1 is a flow chart of an element positioning method provided by an embodiment of the present application. The element positioning method is applied to an electronic device, and the method includes: S201: Parse the user interface to extract semantic information of target elements.

[0057] S202, extracting semantic features of the semantic information, and building an element positioning model based on the semantic features.

[0058] S203: Utilize the element positioning model to locate the target element in the interface to be identified.

[0059] The specific implementation of S201-S203 is the same as that of S101-S103, and will not be repeated here.

[0060] S204 , verifying the location information. If the function of the element corresponding to the location information is the same as the preset function, determining that the location information is qualified.

[0061] In one embodiment of the present application, the positioning result output by the element positioning model is verified by a verification engine to obtain a verification result. The identified element is determined based on the position information output by the element positioning model, and the uniqueness and functionality of the element are verified. It is determined whether the identifier of the element is the same as the preset identifier, and whether the function of the element is the same as the function of the target element. If the identifier of the element is the same as the preset identifier, and the function of the element is the same as the function of the target element, then it is determined that the position information verification is qualified. If the identifier of the element is different from the preset identifier, and / or the function of the element is different from the function of the target element, then it is determined that the position information verification is unqualified. For example, if the button control can be triggered and responded normally, then it is determined that the function of the button control is normal. If the content can be entered normally in the input box, then it is determined that the function of the input box is normal. If the drop-down menu can select options normally, then it is determined that the function of the drop-down menu is normal.

[0062] Based on the above embodiments of the present application, the accuracy of the positioning result can be guaranteed by verifying the element position information output by the element positioning model.

[0063] See Figure 4 , which is a schematic diagram of the structure of an element locating device provided in one embodiment of the present application. In one embodiment of the present application, the element locating device 200 may include multiple functional modules composed of computer program segments. The computer program segments in the element locating device 200 may be stored in a memory of an electronic device and executed by at least one processor to perform the element locating function.

[0064] In one embodiment of the present application, the element locating device 200 can be divided into multiple functional modules according to the functions it performs. The functional modules of the element locating device 200 may include: an extraction module 201, a construction module 202, and a positioning module 203. A module in the embodiment of the present application refers to a series of computer program segments that can be executed by at least one processor and can perform fixed functions, which are stored in a memory.

[0065] The extraction module 201 is used to parse the user interface and extract semantic information of the target element.

[0066] The construction module 202 is used to extract semantic features of semantic information and construct an element positioning model based on the semantic features.

[0067] The positioning module 203 is used to locate the target element in the interface to be identified by using the element positioning model.

[0068] See Figure 5 1 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. The element location method provided in this embodiment of the present application is applied to an electronic device 1, which may be a smartphone, a personal computer, a wearable device, or the like. Electronic device 1 includes, but is not limited to, a processor 110 and a memory 120 connected via a communication bus 130. Figure 5 This is merely an example of the electronic device and does not constitute a corresponding limitation. In other embodiments, the electronic device may include more components than those shown in the figure.

[0069] Memory can include one or more random access memories (RAMs) and one or more non-volatile memories (NVMs). RAM can be directly read and written by the processor and can be used to store executable programs (such as machine instructions) for the operating system or other running programs, as well as user and application data. RAM can include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), and double data rate synchronous dynamic random access memory (DDR SDRAM).

[0070] The non-volatile memory can also store executable programs and user and application data, etc., and can be pre-loaded into the random access memory for direct reading and writing by the processor 110. The non-volatile memory can include disk storage devices and flash memory.

[0071] The memory is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor. The one or more computer programs include multiple instructions. When the multiple instructions are executed by the processor, the element positioning method executed on the electronic device can be implemented.

[0072] In other embodiments, the electronic device further includes an external memory interface for connecting to an external memory to expand the storage capacity of the electronic device.

[0073] The processor may include one or more processing units, for example, an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0074] The processor provides computing and control capabilities. For example, the processor is used to execute a computer program stored in the memory to implement the above-mentioned element positioning method.

[0075] The communication bus is at least used to provide a channel for communication between the memory and the processor in the electronic device.

[0076] It is understood that the structures illustrated in the embodiments of the present application do not constitute specific limitations on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0077] An embodiment of the present application also provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the element positioning method in the above-mentioned embodiment.

[0078] An embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the element positioning method in the above-mentioned embodiment.

[0079] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to execute the element positioning method in the above-mentioned method embodiments.

[0080] Among them, the electronic device, computer storage medium, computer program product or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0081] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0083] Units described as separate components may or may not be physically separate, and components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0084] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0085] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.< / form>

Claims

1. An element positioning method, applied to electronic equipment, characterized in that: The method comprises: Parse the user interface and extract the semantic information of the target elements; Extracting semantic features of the semantic information, and constructing an element positioning model based on the semantic features; The element positioning model is used to locate the target element in the interface to be identified.

2. The element positioning method according to claim 1, wherein: Parsing the user interface to extract semantic information of target elements includes: The hypertext markup language HTML page of the user interface is parsed to identify the type of the target element and extract the text and contextual relationship of the target element.

3. The element positioning method according to claim 1, wherein: The extracting the semantic features of the semantic information includes: Preprocessing the semantic information, including one or more of word segmentation, stop word removal, and part-of-speech tagging; The preprocessed semantic information is subjected to word embedding processing to obtain semantic features.

4. The element positioning method according to claim 1 or 3, characterized in that: The extracting of semantic features of the semantic information further includes: Performing dimensionality reduction processing on the semantic features of the semantic information to determine key features in the semantic features; Weights are assigned to the key features based on a rule engine.

5. The element positioning method according to claim 4, characterized in that: The constructing of the element positioning model based on the semantic features includes: Constructing a training data set based on the semantic features of the target element and the position information of the target element in the user interface, wherein the semantic features are input data and the position information is output data; Based on the training data set, a preset deep learning model is trained to obtain predicted position information of the target element corresponding to each set of training data; Calculating a loss function value based on the predicted position information of the target element and the position information in the corresponding training data; The parameters of the preset deep learning model are adjusted until the loss function value meets the preset conditions, and the trained deep learning model is obtained as the element positioning model.

6. The element positioning method according to claim 1, wherein: The method of locating a target element in the interface to be identified by using the element positioning model includes: The target element in the HTML page of the interface to be identified is located using the element positioning model to obtain position information of the target element in the interface to be identified.

7. The element positioning method according to claim 1 or 6, characterized in that: The method further comprises: The location information is verified, and when the function of the element corresponding to the location information is the same as the preset function, it is determined that the location information is qualified.

8. The element positioning method according to claim 1, wherein: The location information of the target element is an Extensible Markup Language path XPath or a Cascading Style Sheet CSS selector.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, the memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the electronic device executes the element positioning method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The method comprises computer instructions, which, when executed on an electronic device, enable the electronic device to execute the element positioning method according to any one of claims 1 to 8.

Images