Web element matching method and device suitable for RPA

Abstract

The invention discloses a web element matching method and device suitable for RPA, and the method comprises the steps: obtaining a pickup element according to the basic information of a webpage element picked by an RPA tool in an RPA pickup stage; in the RPA playback stage, according to the positioning parameters of the picked elements, obtaining elements which can be directly operated and are used for document structure types on a playback page as candidate elements; respectively calculating the similarity of each positioning parameter by adopting a corresponding similarity calculation mode, calculating to obtain the similarity score of each candidate element and the pickup element according to the similarity of each positioning parameter and the corresponding weight, and selecting the candidate element with the highest similarity score as a matching element; calculating a threshold value according to the occurrence condition of the positioning parameter in the matching element and the corresponding weight; and comparing the similarity score of the matching element and the pickup element with a threshold value, and determining whether to execute the matching element or not according to a comparison result.

Description

Technical Field

[0001] The present application relates to a web element matching method and device applicable to RPA, and belongs to the field of software technology. Background Art

[0002] Robotic Process Automation (RPA) is a technology that automates repetitive tasks by using software robots to imitate human operations on computers. RPA mainly consists of two stages: the pickup stage and the playback stage. The pickup stage is the process of using RPA tools to record user operations, and the playback stage is to automatically execute user operations based on the recorded script.

[0003] However, when using RPA, it is often found that the operations recorded in the picking phase cannot be correctly executed in the playback phase. The main reason is that the picked elements in the picking phase cannot be located in the playback phase. This problem is often caused by multiple reasons, such as: the GUI visualization interface version of the software is updated, the position of the picked element has changed, and the dynamic fields in the element information such as css-selctor, XPath, etc. will change with the version update. This version update in the playback phase will cause the picked element to be unable to be located. This update frequency is related to the web page or software environment. In some web page systems, the dynamic fields are related to the user session. When the user logs in again or changes the user, the element will change dynamically. For example, the id of the web page element = "57a20371-34f5-4b06-b793-a1c667238c42-BILLMENU_TOOLBAR-tbDelete", where "57a20371-34f5-4b06-b793-a1c667238c42" will change with the login.

[0004] During the picking phase, RPA records multiple attributes of elements. During playback, the matching strategy is generally based on a single attribute. For example, web pages often use the cssSelector and XPath attributes, which are matched precisely based on the absolute XPath path or matched regularly based on the XPath regular expression. However, regardless of the matching method, dynamic field changes caused by updates will cause matching errors. When playback fails, many RPA tools can be replayed by re-picking elements, modifying element attributes (modifying cssSelector, XPath), or setting matching strategies (selecting cssSelector or XPath matching). Although this can sometimes solve some problems, it cannot fundamentally solve such problems. In addition, this modification method is not friendly to users who are not familiar with web page operations, and it will also consume a lot of user time and reduce work efficiency. Summary of the invention

[0005] Purpose: In view of at least one of the above technical problems, the present application provides a web element matching method and device suitable for RPA. By applying a similarity matching method, the robustness of RPA element recognition and the efficiency of execution can be improved.

[0006] Technical solution: To solve the above technical problems, the technical solution adopted in this application is:

[0007] In a first aspect, the present application provides a web element matching method applicable to RPA, the method comprising:

[0008] S1. In the RPA picking stage, basic information of web page elements is picked up by the RPA tool, and a picked element is obtained according to the basic information of the web page elements; the picked element includes multiple positioning parameters and values ​​of the positioning parameters;

[0009] S2. In the RPA playback stage, elements of the document structure type that can be directly operated and used on the playback page are obtained as candidate elements according to the positioning parameters of the picked elements;

[0010] S3, for each candidate element and the picked element, respectively calculate the similarity of each positioning parameter using the corresponding similarity calculation method, calculate the similarity score of each candidate element and the picked element according to the similarity of each positioning parameter and the corresponding weight, and select the candidate element with the highest similarity score as the matching element;

[0011] S4. Calculate a threshold value according to the occurrence of the positioning parameter in the matching element and the corresponding weight;

[0012] S5. Compare the similarity scores between the matching element and the picked element with the threshold, and determine whether to execute the matching element according to the comparison result.

[0013] In a second aspect, the present application provides a web element matching device applicable to RPA, the device comprising:

[0014] The picked-up element acquisition module is used to: in the RPA picking stage, pick up the basic information of the web page element through the RPA tool, and acquire the picked-up element according to the basic information of the web page element; the picked-up element includes multiple positioning parameters and the values ​​of the positioning parameters;

[0015] The candidate element acquisition module is used to: in the RPA playback stage, acquire the elements of the document structure type that can be directly operated and used on the playback page as candidate elements according to the positioning parameters of the picked elements;

[0016] The element matching module is used to: for each candidate element and the picked element, respectively calculate the similarity of each positioning parameter using the corresponding similarity calculation method, calculate the similarity score of each candidate element and the picked element according to the similarity of each positioning parameter and the corresponding weight, and select the candidate element with the highest similarity score as the matching element;

[0017] A threshold calculation module, used to calculate the threshold according to the occurrence of the positioning parameter in the matching element and the corresponding weight;

[0018] The execution module is used to compare the similarity score between the matching element and the picked element with the threshold, and determine whether to execute the matching element according to the comparison result.

[0019] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.

[0020] In a fourth aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in the first aspect when executing the computer program.

[0021] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0022] Beneficial effect: The web element matching method and device suitable for RPA provided in the present application have the following advantages: in the picking stage, the basic information of the web page elements is picked up by the RPA tool to obtain the picked elements and positioning parameters, and in the playback stage, a group of candidate elements are obtained in the page, and the similarity between the candidate elements and the picked elements is compared, and the candidate elements with the highest similarity scores are taken as matching elements. The higher the similarity, the more similar the element is to the picked element, and then the matching elements are used for execution in the playback, which solves the problem in the prior art that the operations recorded in the picking stage cannot be correctly executed in the playback stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic diagram of an application scenario of a method according to an embodiment of the present application;

[0024] Figure 2 The following is a flow chart of a web element matching method applicable to RPA according to an embodiment of the present application;

[0025] Figure 3 A schematic diagram of a web element matching device applicable to RPA provided according to an embodiment of the present application;

[0026] Figure 4A schematic diagram of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] The present application is further described below in conjunction with the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present application, and cannot be used to limit the protection scope of the present application.

[0028] In this application, the professional terms are explained as follows:

[0029] Robotic process automation (RPA) is a program that simulates the manual operations of users on the computer desktop through RPA process code to achieve process automation.

[0030] Before describing the embodiments of the present application in detail, a specific scenario example is used to reveal the scenario in which the technical solution of the present application can be used. In a possible scenario example, specifically, Figure 1 As shown, the user performs specific operations on the client 110, and the client 110 is deployed with a data collection device and an RPA, the data collection device is used to collect basic information of web page elements picked up by the RPA tool in the picking stage on the client 110, and obtain the picked elements according to the basic information of the web page elements; the picked elements include multiple positioning parameters and the values ​​of the positioning parameters; and in the playback stage, the elements that can be directly operated and used for the document structure type on the playback page are obtained as candidate elements according to the positioning parameters of the picked elements; the collected picked elements and candidate elements are transmitted to the server through the network, and the server calculates the similarity of each positioning parameter using the corresponding similarity calculation method through a preset program, and obtains the similarity score of each candidate element and the picked element according to the similarity of each positioning parameter and the corresponding weight calculation, and selects the candidate element with the highest similarity score as the matching element; the threshold is calculated according to the occurrence of the positioning parameter in the matching element and the corresponding weight; the similarity score of the matching element and the picked element is compared with the threshold, and whether to execute the matching element is determined according to the comparison result, and whether to execute the matching element is returned to the RPA on the client 110.

[0031] RPA can run on the client terminal side or the server side. The method of the present application can be configured in the RPA on the client terminal / or server side, and the client terminal and the server are connected through network transmission to realize multi-terminal interaction.

[0032] The server may be an electronic device with certain computing and processing capabilities. For example, the server may be a server of a distributed system, or a system with multiple processors, memories, network communication modules, etc. operating in collaboration. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The server may also be a server cluster formed by several servers. Alternatively, with the development of science and technology, the server may also be a new technical means capable of realizing the corresponding functions of the embodiments of the specification. For example, it may be a new form of "server" based on quantum computing.

[0033] The client terminal may be an electronic device with network access capability. Specifically, for example, the terminal may be a desktop computer, a tablet computer, a laptop computer, a smart phone, etc. Alternatively, the terminal may also be software that can run in the electronic device.

[0034] The network may be any type of network that can support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. The one or more networks may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0035] Example 1: Figure 2 As shown, this embodiment provides a web element matching method applicable to RPA, which can be applied to RPA, and the RPA can run in a client terminal. The web element matching method applicable to RPA includes:

[0036] Step S1: In the RPA picking stage, basic information of web page elements is picked up by the RPA tool, and a picked element is obtained according to the basic information of the web page elements; the picked element includes multiple positioning parameters and values ​​of the positioning parameters;

[0037] The positioning parameters include ID, name, type tagName, class, hyperlink href, image description alt, XML path XPath, location Location, area Area, shape Shape, visible text Visible Text and community text Neighbor Texts.

[0038] In some embodiments, step S1 may specifically include:

[0039] S11. Pick up basic information of web page elements through the RPA tool, where the basic information of the web page elements includes: ID id, name name, type tagName, class name class, hyperlink href, image description alt, XML path XPath, bounding rectangle boundingRect, built-in text innerText, input value value, and placeholder placeholder;

[0040] It should be noted that these basic information describe the characteristics of web page elements from different perspectives: (1) element attributes, such as id, name, tagName, class, href, and alt. These information appear specifically in the HTML of the web page and are unique information features set by the web page designer for each element. (2) area positioning information, such as XPath and boundingRect. These information are not directly displayed in the HTML of the web page but are obtained through some specific calculation methods and indicate the structural or visual position information of the element. (3) and text visual information on the web page, such as innerText, value, and placeholder. These are text contents that users can directly capture visually and also appear in the HTML of the web page.

[0041] The basic information of these web page elements obtained in S11 directly or indirectly constitutes the positioning parameters used for calculation in subsequent steps. The positioning parameters and their values ​​constitute the picking elements. The specific definitions of these positioning parameters are as follows:

[0042] Original information attributes: including id, name, tagName, class, href, alt, and XPath. Traditional RPA element positioning also uses these information to provide direct and stable element identification information. These attributes will each be used as a separate positioning parameter for subsequent calculations.

[0043] Coordinate shape attributes: including element location Location, area Area, and shape Shape. These three positioning parameters are calculated through boundingRect information. Basic boundingRect information includes x, y, width, height, and the x coordinate and y coordinate of the upper left corner of the element, as well as the width and height of the element. Location information is the original four items of boundingRect. Area is calculated by width*height, indicating the size of the element area. Shape Shape is calculated by width / height, and the aspect ratio of the element represents the shape of the element. These converted positioning parameters describe the geometric characteristics of the element from different angles.

[0044] Visible text attribute: Visible Text, which combines the visible text content of the three attributes of innerText, value, and placeholder, describes all the text information displayed by the element itself.

[0045] Neighbor Text attributes: Neighbor Texts, that is, text information around web page elements. There are many ways to obtain them, which are not described in detail here; the community text positioning parameters describe the context of web page elements and mark the element community.

[0046] S12. The values ​​of the positioning parameters such as id, name, tagName, class, href, alt, and XPath are directly obtained through the basic information of the web page elements. The values ​​of the three positioning parameters, Location, Area, and Shape, are calculated through the boundingRect element information. The value of the visible text is obtained by integrating the visible text content of the three attributes of innerText, value, and placeholder to describe all the text information displayed by the element itself. The value of the Neighbor Texts is obtained by integrating the text information around the web page elements.

[0047] The reason for setting so many positioning parameters is that the basic information of all elements and the derived positioning parameters may change dynamically. Introducing more positioning parameters can reduce the dynamic change noise generated by a single positioning parameter.

[0048] Step S2: In the RPA playback stage, elements of the document structure type that can be directly operated and used on the playback page are obtained as candidate elements according to the positioning parameters of the picked elements;

[0049] The element types of the candidate elements include: input control input, multi-line text textarea, button button, drop-down list select, anchor a, title tag (h1, h2, h3, h4, h5), list item li, inline container span, block-level container div, paragraph p, table header th, table row tr, cell td, label label, vector label svg.

[0050] It should be noted that, first, all candidate elements on the playback page need to be obtained, and the picked element needs to be matched among the candidate elements.

[0051] Referring to step S1, each candidate element also needs to generate corresponding positioning parameters to facilitate subsequent calculations.

[0052] In order to generate the values ​​of the positioning parameters of all candidate elements, it is necessary to traverse the playback page. However, this traversal contains a large number of redundant elements (such as script, noscript, style and other types of elements that have no practical meaning). Matching these elements is not only meaningless, but also greatly increases the time and consumes system performance when generating positioning parameters for these elements.

[0053] To this end, this method narrows the scope of candidate elements and only obtains elements that can be directly operated (such as input controls, buttons) and used for document structure (such as title h1, inline container span) in the page as candidate elements. This method only obtains the following element types: input controls, multi-line text textarea, buttons, drop-down lists, anchors, title tags (h1, h2, h3, h4, h5), list items, inline containers, span, block-level containers, paragraphs, th, table rows, tr, cells, td, labels, and vector labels.

[0054] These types of elements basically cover most of the interactive elements, as well as elements used to organize document structure and content display. Through this method, the present application can effectively reduce the interference of redundant elements and improve the efficiency and accuracy of subsequent step calculations.

[0055] Step S3: for each candidate element and the picked element, respectively, the similarity of each positioning parameter is calculated by using the corresponding similarity calculation method, and the similarity score of each candidate element and the picked element is calculated according to the similarity of each positioning parameter and the corresponding weight, and the candidate element with the highest similarity score is selected as the matching element;

[0056] The similarity score between each candidate element and the picked element is calculated based on the similarity of each positioning parameter and the corresponding weight. The formula is:

[0057]

[0058] Among them, n represents the nth positioning parameter, k represents the total number of positioning parameters, Similarity n Indicates the similarity of the nth positioning parameter, Weight n Represents the weight of the nth positioning parameter. The similarity score is actually the sum of the similarity and weight product of each positioning parameter.

[0059] It should be noted that for different positioning parameters, different similarity calculation methods are selected and different weights are set, which are explained in detail as follows:

[0060] S3.1 Choice of similarity calculation method

[0061] Different positioning parameters correspond to different similarity calculation methods (Table 1), which include string comparison, edit distance, Euclidean distance, numerical distance and cosine distance. The values ​​of these calculation methods are between 0 and 1.

[0062] String comparison is used for the three positioning parameters id, name, and tagName. String comparison indicates whether two strings are exactly the same; for example, when two element ids are exactly the same, the similarity of the positioning parameter id is expressed as 1, otherwise it is 0. String comparison is a more stringent calculation method because id, name, and tagName are used to identify and select elements and rarely change.

[0063] Edit distance is used for class, alt, XPath and visible text, and neighbor texts. Edit distance can be used to measure the difference in character order between two strings to represent similarity, and these positioning information often change in order, but the overall character style changes are small. For example, XPath changes sequentially and only changes a certain node. For example, / html / body / div / div[2] / p is changed to / html / body / div / div[3] / p in the new version, and only the second div has changed. Directly using XPath positioning in the traditional way may fail to locate the element, or locate the wrong element. A higher similarity calculated using the edit distance means that they should be very similar in structure; otherwise, the difference is large.

[0064] Euclidean distance is used for location, and the relative position difference is represented by calculating the Euclidean distance of the x and y coordinates of two elements.

[0065] This application designs a numerical distance value_dist to compare area and shape, indicating the difference in area or shape. The formula is as follows:

[0066]

[0067] Where a and b represent the positioning parameter of the picked element (such as area) and the value of the positioning parameter corresponding to the candidate element, respectively.

[0068] Cosine distance is used to compare href. href is a regular URL representation, such as "https: / / example.com / article?category=books&id=214". First, the URL representation is split with symbols such as " / ", "?", "=", etc. to form a field vector, such as "['https', 'example.com', 'article', 'category', 'books', 'id', '214']"; then the field vector is digitized. The picked element is the matched vector, so the value of each field is set to 1 by default, such as [1,1,1,1,1,1,1]. If the candidate element corresponds to a different href, such as "https: / / example.com / article?category=knowledge&id=215", it is segmented into field vectors, such as "['https', 'example.com', 'article', 'category', 'knowledge', 'id', '215']"; calculate the edit distance between the candidate element field vector and the elements at the corresponding positions in the picked element field vector. If the first element is "https", the edit distance is 1. The edit distance of the elements at each position constitutes the numerical vector of the candidate element, such as [1,1,1,1,0,1,0.6]. Substitute the two numerical vectors into the cosine distance formula to obtain the final similarity of href.

[0069] S3.2 Choice of weights

[0070] Set a fixed high weight of 1.5 for id, name, tagName, Visual Text, and Neighbor Texts. In web design, id, name, and tagName are usually used to identify and select elements and rarely change, so they are relatively important. Visual Text and Neighbor Texts are the most direct web page visualization information and are also extremely important matching parameters.

[0071] Href and alt are relatively less important. Not all elements contain this parameter, so their weight is low and is set to 0.5.

[0072] Class and XPath are relatively important information, but they change frequently. Class is used to define the style of an element, but it changes frequently. For example, when the class changes from 'btn-noboard' to 'btn-board btn-gray', the visual style of the element changes. Because the frequency of change is high and the magnitude of change is variable, the weight of these parameters is low and is set to 0.5.

[0073] Area,location is easily affected by window size. Different window sizes will cause changes in element arrangement. In addition, web page scaling will also cause changes in element size. Therefore, a smaller weight of 0.5 is used.

[0074] Shape is not affected by window size or web page scaling, so its weight is relatively high and is set to 0.8.

[0075] In summary, the positioning parameters and their similarity calculation methods and weights are shown in Table 1:

[0076] Table 1

[0077]

[0078]

[0079] According to this method, the similarity scores of all candidate elements and the picked element are calculated in turn, and the candidate element with the highest score is taken out as the matching element.

[0080] Because the algorithm in step S3 introduces enough positioning parameters and assigns reasonable weights to each parameter, the algorithm is highly resistant to noise and can match picked elements in a variety of page scenarios. It is not prone to mismatching like traditional RPA, and does not require users to repeatedly re-pick elements, manually modify element attributes, or set matching modes to perform playback.

[0081] In some extreme cases, the playback page has been significantly modified, and the picked-up element does not exist in the new page. Through similarity matching, a matching element with the highest score will still be obtained. If this matching element is still executed, an incorrect result may be obtained. Therefore, it is necessary to observe the correctness and rationality of the matching element.

[0082] Therefore, this application sets a threshold to determine whether the element with the highest score can be adopted. Element web pages are changeable, and the positioning parameters used to calculate similarity are not fixed. Some positioning parameters may only appear in certain types of elements, and the positioning parameters listed in Table 1 may not appear in every element. For example: the positioning parameter alt is only used to describe image type elements, and href is used for link tags to define hyperlinks. Therefore, the threshold is a dynamic quantity and needs to be changed according to actual conditions.

[0083] Step S4, calculating a threshold value according to the occurrence of the positioning parameter in the matching element and the corresponding weight;

[0084] If the positional parameter P n Exists in the matching element, single threshold is Weight n *0.7; if the positioning parameter P nIf it does not exist in the matching element, the single threshold is 0, and the threshold is the sum of all single thresholds. The calculation formula of the threshold threshhold is:

[0085]

[0086] Among them, the positioning parameter P n Indicates the nth positioning parameter; Weight n is the weight of the nth positioning parameter.

[0087] Step S5: Compare the similarity scores between the matching element and the picked element with the threshold, and determine whether to execute the matching element according to the comparison result.

[0088] S51, if the similarity score between the matching element and the picked element is greater than or equal to the threshold, execute matching element;

[0089] S52: If the similarity score between the matching element and the picked element is lower than the threshold, the matching element is not executed.

[0090] In summary, the web element matching method applicable to RPA provided in the above embodiment is to obtain the picked-up elements and positioning parameters by picking up the basic information of the web page elements through the RPA tool in the picking stage, and to obtain a group of candidate elements in the page in the playback stage, compare the similarity between the candidate elements and the picked-up elements, and take the candidate element with the highest similarity score as the matching element. The higher the similarity, the more similar the element is to the picked-up element, and then use the matching element to execute in the playback. This solves the problem that the operations recorded in the picking stage in the prior art cannot be correctly executed in the playback stage.

[0091] Example 2: Based on Example 1, Figure 3 As shown, the present application provides a web element matching device applicable to RPA, the device comprising:

[0092] The picked-up element acquisition module is used to: in the RPA picking stage, pick up the basic information of the web page element through the RPA tool, and acquire the picked-up element according to the basic information of the web page element; the picked-up element includes multiple positioning parameters and the values ​​of the positioning parameters;

[0093] The candidate element acquisition module is used to: in the RPA playback stage, acquire the elements of the document structure type that can be directly operated and used on the playback page as candidate elements according to the positioning parameters of the picked elements;

[0094] The element matching module is used to: for each candidate element and the picked element, respectively calculate the similarity of each positioning parameter using the corresponding similarity calculation method, calculate the similarity score of each candidate element and the picked element according to the similarity of each positioning parameter and the corresponding weight, and select the candidate element with the highest similarity score as the matching element;

[0095] A threshold calculation module, used to calculate the threshold according to the occurrence of the positioning parameter in the matching element and the corresponding weight;

[0096] The execution module is used to compare the similarity score between the matching element and the picked element with the threshold, and determine whether to execute the matching element according to the comparison result.

[0097] The specific functional implementation of each of the above modules can be found in the relevant contents of the method in Example 1 and will not be elaborated here.

[0098] Embodiment 3: Based on Embodiment 1, this embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in Embodiment 1 are implemented.

[0099] Embodiment 4: Based on embodiment 1, as Figure 4 As shown, this embodiment provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in Embodiment 1 when executing the computer program.

[0100] Embodiment 5: Based on Embodiment 1, this embodiment proposes a hardware system that applies a web element matching method applicable to RPA in Embodiment 1. The method is applied on the server side.

[0101] The terminal may be an electronic device with network access capability, for example, a desktop computer, a tablet computer, a laptop computer, a smart phone, a digital assistant, a shopping guide terminal, a television, etc.

[0102] The server can be an electronic device with certain computing and processing capabilities. For example, the server can be a server of a distributed system, or a system with multiple processors, memories, network communication modules, etc. operating in collaboration. The server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The server can also be a server cluster formed by several servers. Alternatively, with the development of science and technology, the server can also be a new technical means that can realize the corresponding functions of the implementation method of the specification. For example, it can be a new form of "server" based on quantum computing.

[0103] Embodiment 6: Based on Embodiment 1, this embodiment further provides a computer program product comprising instructions, which, when executed by a computer, enables the computer to execute a web element matching method applicable to RPA in Embodiment 1.

[0104] It should be understood that the specific examples in this article are only intended to help those skilled in the art better understand the implementation methods of this specification, rather than limiting the scope of this application.

[0105] It can be understood that in the various implementations of this specification, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation methods of this specification.

[0106] It can be understood that the various embodiments described in this specification can be implemented individually or in combination, and the embodiments of this specification are not limited to this.

[0107] Unless otherwise specified, all technical and scientific terms used in the embodiments of this specification have the same meaning as those generally understood by those skilled in the art of the technical field of this specification. The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the scope of this specification. The term "and / or" used in this specification includes any and all combinations of one or more related listed items. The singular forms of "a", "above", and "the" used in the embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0108] It can be understood that the processor of the embodiment of this specification can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method implementation can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiment of this specification can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of this specification can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0109] It is understood that the memory in the embodiments of this specification may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasablePROM, EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0110] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this specification.

[0111] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method implementation methods and will not be repeated here.

[0112] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0113] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0114] In addition, each functional unit in each embodiment of the present specification may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0115] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of this specification. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0116] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A web element matching method applicable to RPA, characterized in that: The method comprises: S1. In the RPA picking stage, basic information of web page elements is picked up by the RPA tool, and a picked element is obtained according to the basic information of the web page elements; the picked element includes multiple positioning parameters and values ​​of the positioning parameters; S2. In the RPA playback stage, elements of the document structure type that can be directly operated and used on the playback page are obtained as candidate elements according to the positioning parameters of the picked elements; S3, for each candidate element and the picked element, respectively calculate the similarity of each positioning parameter using the corresponding similarity calculation method, calculate the similarity score of each candidate element and the picked element according to the similarity of each positioning parameter and the corresponding weight, and select the candidate element with the highest similarity score as the matching element; S4. Calculate a threshold value according to the occurrence of the positioning parameter in the matching element and the corresponding weight; S5. Compare the similarity scores between the matching element and the picked element with the threshold, and determine whether to execute the matching element according to the comparison result.

2. The web element matching method applicable to RPA according to claim 1, characterized in that: The positioning parameters include ID, name, type tagName, class name, hyperlink href, image description alt, XML path XPath, location Location, area Area, shape Shape, visible text Visible Text and community text Neighbor Texts.

3. The web element matching method applicable to RPA according to claim 2, characterized in that: S1. In the RPA picking stage, the basic information of the web page elements is picked up by the RPA tool, and the picked elements are obtained according to the basic information of the web page elements, including: S11. Pick up basic information of web page elements through the RPA tool, where the basic information of the web page elements includes: ID id, name name, type tagName, class name class, hyperlink href, image description alt, XML path XPath, bounding rectangle boundingRect, built-in text innerText, input value value, and placeholder placeholder; The values ​​of these positioning parameters S12, including id, name, tagName, class, href, alt, and XPath, are directly obtained through the basic information of the web page elements. The values ​​of the three positioning parameters, including Location, Area, and Shape, are calculated through the boundingRect element information. The value of the visible text VisibleText is obtained by integrating the visible text content of the three attributes of innerText, value, and placeholder to describe all the text information displayed by the element itself. The value of the Neighbor Texts is obtained by integrating the text information around the web page elements.

4. The web element matching method applicable to RPA according to claim 1, characterized in that: In S2, the element types of the candidate elements include: input control input, multi-line text textarea, button button, drop-down list select, anchor point a, title tag (h1, h2, h3, h4, h5), list item li, inline container span, block-level container div, paragraph p, table header th, table row tr, cell td, label label, vector label svg.

5. The web element matching method applicable to RPA according to claim 2, characterized in that: In S3, the similarity of each positioning parameter is calculated using corresponding similarity calculation methods, including: The similarity calculation method used for positioning parameters id, name, and tagName is string comparison; The similarity calculation method used for positioning parameters class, alt, XPath, Visible text, and Neighbor texts is edit distance; The similarity calculation method used by the location parameter location is Euclidean distance; The similarity calculation method used by the positioning parameter area and shape is the numerical distance value_dist, and the calculation formula is as follows: Where a and b represent the values ​​of the positioning parameter of the picked element and the positioning parameter corresponding to the candidate element, respectively; The similarity calculation method used by the positioning parameter href is cosine distance.

6. The web element matching method applicable to RPA according to claim 2, characterized in that: In S3, the weights of the location parameters id, name, tagName, Visual Text, and Neighbor Texts are set to 1.5; The weights of the positioning parameters href, alt, class, XPath, area, and location are set to 0.5; The weight of the positioning parameter shape is set to 0.

8.

7. The web element matching method applicable to RPA according to claim 1, characterized in that: In S3, the similarity scores of each candidate element and the picked element are calculated according to the similarity of each positioning parameter and the corresponding weight, including; Among them, n represents the nth positioning parameter, k represents the total number of positioning parameters, Similarity n Indicates the similarity of the nth positioning parameter, Weight n Represents the weight of the nth positioning parameter.

8. The web element matching method applicable to RPA according to claim 1, characterized in that: S4. Calculate the threshold threshhold according to the occurrence of the positioning parameter in the matching element and the corresponding weight, including: Among them, the positioning parameter P n Indicates the nth positioning parameter; Weight n is the weight of the nth positioning parameter.

9. The web element matching method applicable to RPA according to claim 1, characterized in that: S5, comparing the similarity score between the matching element and the picked element with the threshold, and determining whether to execute the matching element according to the comparison result, including: If the similarity score between the matching element and the picked element is greater than or equal to the threshold, the matching element is executed; If the similarity score between the matching element and the picked element is lower than the threshold, the matching element is not performed.

10. A web element matching device suitable for RPA, characterized in that: The device comprises: The picked-up element acquisition module is used to: in the RPA picking stage, pick up the basic information of the web page element through the RPA tool, and acquire the picked-up element according to the basic information of the web page element; the picked-up element includes multiple positioning parameters and the values ​​of the positioning parameters; The candidate element acquisition module is used to: in the RPA playback stage, acquire the elements of the document structure type that can be directly operated and used on the playback page as candidate elements according to the positioning parameters of the picked elements; The element matching module is used to: for each candidate element and the picked element, respectively calculate the similarity of each positioning parameter using the corresponding similarity calculation method, calculate the similarity score of each candidate element and the picked element according to the similarity of each positioning parameter and the corresponding weight, and select the candidate element with the highest similarity score as the matching element; A threshold calculation module, used to calculate the threshold according to the occurrence of the positioning parameter in the matching element and the corresponding weight; The execution module is used to compare the similarity score between the matching element and the picked element with the threshold, and determine whether to execute the matching element according to the comparison result.

Images