Template-based web page element positioning method, device and equipment and storage medium

Abstract

The application relates to the development of auxiliary technology, and discloses a template-based WEB page element positioning method, device, equipment and medium. The method comprises the following steps: using a marking tool to select a region of interest in a screenshot of a WEB page currently displayed by a browser, and generating a template definition file of the region of interest; obtaining a target screenshot of a target WEB page in the browser, and converting corresponding elements in the target screenshot into a conversion matrix list by using elements with an anchor type identified by a type field in the template definition file; calculating the positions of elements with the same element name and a value type identified by the type field in the target screenshot according to the conversion matrix list, obtaining a text box of the value type element corresponding to the positions, and obtaining an xpath of the value type element; and outputting corresponding elements in the target page screenshot according to the xpath of the value type element by using an RPA system. The application can improve the accuracy and efficiency of information acquisition in a WEB page of a hospital system.

Description

Technical Field

[0001] This invention relates to development assistance technology, and more particularly to a template-based method, apparatus, device, and medium for locating web page elements. Background Technology

[0002] With the rapid development of web technology, web applications are increasingly used to provide services in daily life and business operations. For example, hospitals use web pages to display doctor profiles and registration information. While web applications bring convenience to people's lives and work, the continuous expansion of web applications and the increasing number of web application platforms have made interactions between different web applications and data collection within web application systems increasingly complex. For instance, traditional methods for obtaining information from web pages may require manual data copying or taking screenshots and using text recognition technology to identify the content in the screenshots. However, manual methods are time-consuming, labor-intensive, and prone to errors, and text recognition of screenshots cannot specifically extract designated content from the screenshots. Therefore, existing methods for obtaining web page information suffer from low accuracy and efficiency. Summary of the Invention

[0003] This invention provides a template-based method, apparatus, electronic device, and storage medium for locating web page elements, with the main objective of improving the accuracy and efficiency of web page information retrieval.

[0004] To achieve the above objectives, this invention proposes a template-based method for locating web page elements, the method comprising:

[0005] Capture the currently displayed web page in the preset browser and generate a screenshot of the current page;

[0006] Using a preset annotation tool, select a region of interest in the current page screenshot, generate a tag for the region of interest, and generate a template definition file based on the region of interest and the corresponding tag. The tag includes the name, location coordinates, and type field identifier of the region of interest.

[0007] In the browser, a page capture extension plugin is used to obtain a screenshot of the target web page, and the element in the target page screenshot is converted into a transformation matrix list using the element whose type field is identified as anchor in the template definition file.

[0008] Based on the list of transformation matrices corresponding to the anchor type elements, calculate the position of the value type element with the same element name in the target page screenshot, and then use the document object model to obtain the text box of the value type element corresponding to the position, and obtain the xpath of the value type element.

[0009] By using a pre-built RPA system to call the browser augmented API interface, the corresponding element in the target page screenshot is output based on the XPath of the value type element.

[0010] Optionally, generating the labels for the region of interest includes:

[0011] Record the width and height of the region of interest;

[0012] By combining the name, position coordinates, and type field identifier of each element, a list of element field labels is obtained;

[0013] Based on the width and height of the region of interest, and the element field list annotations, generate labels for the region of interest.

[0014] Optionally, obtaining a screenshot of the target webpage using a page capture extension in the browser includes:

[0015] The target web page URL is received via the API interface and the page capture extension plugin.

[0016] Based on the target web page URL, locate the tab page containing the target web page;

[0017] Obtain a screenshot of the target page corresponding to the tab.

[0018] Optionally, the step of converting elements in the target page screenshot whose type field is identified as "anchor" into a list of transformation matrices using the template definition file includes:

[0019] Select an element in the template definition file that is identified as type "anchor". Search for text boxes that match the selected anchor type element in the target page screenshot and calculate the transformation matrix of the text boxes.

[0020] Determine whether all anchor-type elements in the template definition file have been selected;

[0021] If no anchor type element is selected in the template definition file, return to the step described above of selecting an element in the template definition file whose type field is identified as anchor type;

[0022] If all anchor-type elements in the template definition file have been selected, a list of transformation matrices is obtained based on the transformation matrix of the text box corresponding to each anchor-type element.

[0023] Optionally, the step of obtaining a list of transformation matrices based on the transformation matrix of the text box corresponding to each anchor type element includes:

[0024] When an anchor type element has multiple matching text boxes, calculate the error between each transformation matrix in the transformation matrix of all text boxes and the position coordinates of the anchor type element in the template definition file, and select the text box with the smallest error as the text box that matches the anchor type element.

[0025] When no matching text box is found for an element of type anchor, the transformation matrix that is closest to the element of type anchor is selected as the matching text box for the element of type anchor.

[0026] To address the above problems, the present invention also provides a template-based web page element positioning device, the device comprising:

[0027] The template file generation module is used to capture the web page currently displayed in the preset browser, generate a screenshot of the current page, select the area of ​​interest in the current page screenshot using a preset annotation tool, generate a tag for the area of ​​interest, and generate a template definition file based on the area of ​​interest and the corresponding tag. The tag includes the name, location coordinates, and type field identifier of the area of ​​interest.

[0028] The WEB page element positioning module is used to obtain a screenshot of the target WEB page in the browser using a page capture extension plugin, and to convert the corresponding elements in the target page screenshot into a transformation matrix list using elements with the type field identified as "anchor" in the template definition file. Based on the transformation matrix list corresponding to the "anchor" type elements, the module calculates the position of elements with the same element name but identified as "value" type in the target page screenshot. Then, it uses the Document Object Model to obtain the text box of the value type element corresponding to the position, obtains the XPath of the value type element, and uses a pre-built RPA system to call the browser's augmented API interface to output the corresponding element in the target page screenshot based on the XPath of the value type element.

[0029] Optionally, the template file generation module generates the labels for the region of interest using the following method:

[0030] Record the width and height of the region of interest;

[0031] By combining the name, position coordinates, and type field identifier of each element, a list of element field labels is obtained;

[0032] Based on the width and height of the region of interest, and the element field list annotations, generate labels for the region of interest.

[0033] Optionally, the web page element positioning module uses the template definition file to convert elements in the target page screenshot whose type field is marked as "anchor" into a list of transformation matrices using the following method:

[0034] Select an element in the template definition file that is identified as type "anchor". Search for text boxes that match the selected anchor type element in the target page screenshot and calculate the transformation matrix of the text boxes.

[0035] Determine whether all anchor-type elements in the template definition file have been selected;

[0036] If no anchor type element is selected in the template definition file, return to the step described above of selecting an element in the template definition file whose type field is identified as anchor type;

[0037] If all anchor-type elements in the template definition file have been selected, a list of transformation matrices is obtained based on the transformation matrix of the text box corresponding to each anchor-type element.

[0038] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0039] Memory, storing at least one computer program; and

[0040] The processor executes the computer program stored in the memory to implement the template-based web page element positioning method described above.

[0041] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the template-based web page element positioning method described above.

[0042] This invention encapsulates the element matching process within a browser augmentation plugin. Users only need to define the positions of the page elements to be extracted and the corresponding reference page elements on the template in a WYSIWYG manner to obtain the unique identifiers of the page elements. This simplifies the complexity of page element location and improves the accuracy and efficiency of obtaining web page information. Attached Figure Description

[0043] Figure 1 A flowchart illustrating a template-based web page element location method according to an embodiment of the present invention;

[0044] Figure 2 This is a flowchart illustrating the process of obtaining plugin detection results in a template-based web page element location method according to an embodiment of the present invention.

[0045] Figure 3 A schematic diagram of a template-based web page element positioning device provided in an embodiment of the present invention;

[0046] Figure 4 This is a schematic diagram of the internal structure of an electronic device that implements a template-based web page element positioning method according to an embodiment of the present invention.

[0047] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0049] This invention provides a template-based web page element location method. The execution subject of the template-based web page element location method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application embodiment: a server, a terminal, etc. In other words, the template-based web page element location method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0050] Reference Figure 1 The flowchart shown is a template-based web page element location method according to an embodiment of the present invention. In this embodiment, the template-based web page element location method includes:

[0051] S1. Capture the currently displayed web page in the preset browser and generate a screenshot of the current page.

[0052] In this embodiment of the invention, any screenshot software can be used to take a screenshot of the currently displayed web page, generating a screenshot of the current page. The currently displayed web page can be the registration page of one department in a hospital's registration system, such as the pediatrics registration page.

[0053] S2. Using a preset annotation tool, select the area of ​​interest in the current page screenshot, generate a label for the area of ​​interest, and generate a template definition file based on the area of ​​interest and the corresponding label. The label includes the name, location coordinates, and type field identifier of the area of ​​interest.

[0054] In this embodiment of the invention, the preset annotation tool can be any deep learning image annotation tool, such as labelme, labelimg, etc. The region of interest can be the doctor's profile information, registration details, etc.

[0055] Specifically, generating the label for the region of interest includes:

[0056] Record the width and height of the region of interest;

[0057] Combine the name, position coordinates, and type field identifier of each element to obtain a list of element fields;

[0058] Based on the width and height of the region of interest, and the element field list annotations, generate labels for the region of interest.

[0059] In one embodiment of the present invention, the region of interest is represented by the following format label:

[0060] {

[0061] "width": 531,

[0062] "height": 1024,

[0063] “elements”: [{

[0064] "name": "Dr. Wang Hong"

[0065] "pos": [[x0, y0], [x0, y1]],

[0066] “label”: “anchor:Daily registration quantity:class-fee "

[0067] },

[0068] {

[0069] “name”: “Dr. Li Xia”

[0070] "pos": [[x0, y0], [x0, y1]],

[0071] “label”: “value:10”

[0072] }

[0073] … ]

[0075] }

[0076] Wherein, width and height represent the width and height of the region of interest. The element field list contains the label of the region of interest, which includes two types of field identifiers: anchor and value, identified by the label field. The format of the label field is as follows:

[0077] label := anchor:text matching expression[:other element attributes]

[0078] | value[:other attributes of the element]

[0079] When the label is of type value, it indicates that the current element in the region of interest is a target element, and a unique XPath identifier needs to be output. When the label is of type anchor, it indicates that the current element in the region of interest is an auxiliary element for positioning, and a text matching expression for that element needs to be provided. Value-type elements cannot have duplicate names; that is, the name field must be unique. Anchor-type elements can have duplicate names.

[0080] In this embodiment of the invention, the template definition file includes the region of interest and its corresponding tag.

[0081] S3. In the browser, use the page capture extension plugin to obtain a screenshot of the target web page, and use the element with the type field identified as anchor type in the template definition file to convert the corresponding element in the target page screenshot into a list of transformation matrices;

[0082] Specifically, the step of obtaining a screenshot of the target web page using a page capture extension plugin in the browser includes:

[0083] The target web page URL is received via the API interface and the page capture extension plugin.

[0084] Based on the target web page URL, locate the tab page containing the target web page;

[0085] Obtain a screenshot of the target page corresponding to the tab.

[0086] Further, see Figure 2 As shown, the step of converting elements in the target page screenshot with the type field identified as "anchor" into a list of transformation matrices using the template definition file includes:

[0087] S30. Select an element in the template definition file whose type field is identified as anchor type, search for a text box matching the selected anchor type element in the target page screenshot, and calculate the transformation matrix of the text box.

[0088] In this embodiment of the invention, the transformation matrix transforms the position coordinates [x0, y0], [x0, y1] of the anchor type element in the template definition file to the left border [x0, y0] of the matched text box. y0 ], [x0 y1 ];

[0089] S31. Determine whether all anchor type elements in the template definition file have been selected.

[0090] If no anchor type element in the template definition file is selected, then return to S30 above;

[0091] If all anchor-type elements in the template definition file have been selected, then S32, based on the transformation matrix of the text box corresponding to each anchor-type element, obtain a list of transformation matrices {T}. ij， i=1,…m; j=1,…J i}, complete the initialization settings of the page capture extension plugin. Where T ij It is a 2x3 affine transformation matrix, T ij Let J represent the j-th transformation matrix with the i-th element. i =0 indicates that no matching text box was found; when J = 0, it means no matching text box was found. i A value greater than 1 indicates that multiple matching text boxes were found; when J... i =1 indicates that a unique matching text box was found.

[0092] In this embodiment of the invention, when an anchor-type element has multiple matching text boxes, the error between each transformation matrix in the transformation matrix of all text boxes and the position coordinates of the anchor-type element in the template definition file is calculated; and the text box with the smallest error is selected as the text box matching the anchor-type element; when no matching text box is found for the anchor-type element, the transformation matrix closest to the anchor-type element is selected as the text box matching the anchor-type element.

[0093] S4. Based on the list of transformation matrices corresponding to the anchor type elements, calculate the position of the value type element with the same element name in the target page screenshot, and then use the document object model to obtain the text box of the value type element corresponding to the position, and obtain the xpath of the value type element.

[0094] In this embodiment of the invention, the output format of the XPath is:

[0095] {"element1_name": xpath1,

[0096] “element2_name”: xpath2,

[0097] …}

[0098] S5. Using a pre-built RPA system, by calling the browser augmented API interface, output the element corresponding to the target page screenshot based on the XPath of the value type element.

[0099] In this embodiment of the invention, the RPA (Robotic Process Automation) system is an application whose main function is to automatically execute work information and business interactions through a robot according to a pre-designed process. In this embodiment, the RPA system automatically outputs the corresponding element in the target page screenshot based on the XPath of the value type element.

[0100] This invention encapsulates the element matching process within a browser augmentation plugin. Users only need to define the location of the page element to be extracted and the corresponding reference page element on the template in a WYSIWYG manner to obtain the unique identifier of the page element, thus simplifying the complexity of page element location.

[0101] like Figure 3 The diagram shown is a functional block diagram of the template-based WEB page element positioning device of the present invention.

[0102] The template-based web page element positioning device 100 of the present invention can be installed in an electronic device. Depending on the functions implemented, the template-based web page element positioning device 100 may include a template file generation module 101 and a web page element positioning module 102. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.

[0103] In this embodiment, the functions of each module / unit are as follows:

[0104] The template file generation module 101 is used to capture the web page currently displayed in the preset browser, generate a screenshot of the current page, select the area of ​​interest in the current page screenshot using a preset annotation tool, generate a tag for the area of ​​interest, and generate a template definition file based on the area of ​​interest and the corresponding tag. The tag includes the name, location coordinates, and type field identifier of the area of ​​interest.

[0105] The WEB page element positioning module 102 is used to obtain a target page screenshot of the target WEB page in the browser using a page capture extension plugin, and to convert the corresponding element in the target page screenshot into a transformation matrix list using the element with the type field identified as anchor type in the template definition file. Based on the transformation matrix list corresponding to the anchor type element, the position of the element with the same element name and the type field identified as value type in the target page screenshot is calculated. Then, the text box of the value type element corresponding to the position is obtained using the Document Object Model, and the XPath of the value type element is obtained. Using a pre-built RPA system, the corresponding element in the target page screenshot is output based on the XPath of the value type element by calling the browser augmented API interface.

[0106] In detail, the modules in the template-based WEB page element positioning device 100 described in this embodiment of the invention employ the same methods as described above. Figure 1 The technique used is the same as the template-based web page element positioning method described above, and it can produce the same technical effect, so it will not be repeated here.

[0107] like Figure 4 The diagram shown is a structural schematic of an electronic device that implements the template-based web page element positioning method of the present invention.

[0108] The electronic device may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a template-based web page element locator.

[0109] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as the code of a template-based web page element locator, but also to temporarily store data that has been output or will be output.

[0110] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., template-based web page element locating programs) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0111] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The communication bus 12 is configured to enable communication between the memory 11 and at least one processor 10, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus.

[0112] Figure 4 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 4The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0113] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault classification circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0114] Optionally, the communication interface 13 may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device and other electronic devices.

[0115] Optionally, the communication interface 13 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0116] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0117] The template-based web page element locator stored in the memory 11 of the electronic device is a combination of multiple computer programs that, when run in the processor 10, can achieve the following:

[0118] Capture the currently displayed web page in the preset browser and generate a screenshot of the current page;

[0119] Using a preset annotation tool, select a region of interest in the current page screenshot, generate tags for the region of interest, and generate a template definition file based on the region of interest and the corresponding tags. The tags include a name, location coordinates, and a type field identifier.

[0120] In the browser, a page capture extension plugin is used to obtain a screenshot of the target web page, and the element in the target page screenshot is converted into a transformation matrix list using the element whose type field is identified as anchor in the template definition file.

[0121] Based on the list of transformation matrices corresponding to the anchor type elements, calculate the position of the value type element with the same element name in the target page screenshot, and then use the document object model to obtain the text box of the value type element corresponding to the position, and obtain the xpath of the value type element.

[0122] By using a pre-built RPA system to call the browser augmented API interface, the corresponding element in the target page screenshot is output based on the XPath of the value type element.

[0123] Specifically, the processor 10's implementation method of the above-mentioned computer program can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0124] Furthermore, if the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable medium can be non-volatile or volatile. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0125] Embodiments of the present invention may also provide a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0126] Capture the currently displayed web page in the preset browser and generate a screenshot of the current page;

[0127] Using a preset annotation tool, select a region of interest in the current page screenshot, generate tags for the region of interest, and generate a template definition file based on the region of interest and the corresponding tags. The tags include a name, location coordinates, and a type field identifier.

[0128] In the browser, a page capture extension plugin is used to obtain a screenshot of the target web page, and the element in the target page screenshot is converted into a transformation matrix list using the element whose type field is identified as anchor in the template definition file.

[0129] Based on the list of transformation matrices corresponding to the anchor type elements, calculate the position of the value type element with the same element name in the target page screenshot, and then use the document object model to obtain the text box of the value type element corresponding to the position, and obtain the xpath of the value type element.

[0130] By using a pre-built RPA system to call the browser augmented API interface, the corresponding element in the target page screenshot is output based on the XPath of the value type element.

[0131] Furthermore, the computer's usable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store the operating system, applications required for at least one function, etc.; and the data storage area may store data created based on the use of blockchain nodes, etc.

[0132] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0133] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0134] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0135] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0136] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0137] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0138] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0139] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A template-based WEB page element positioning method, characterized in that, The method comprises: page capturing is performed on a currently displayed WEB page in a preset browser to generate a current page screenshot; a page region of interest is selected in the current page screenshot using a preset labeling tool, a label of the region of interest is generated, and a template definition file is generated according to the region of interest and the corresponding label, wherein the label comprises a name, position coordinates and a type field identifier of the region of interest; a target page screenshot of a target WEB page is obtained in the browser using a page capturing extension plug-in, and elements corresponding to the target page screenshot are converted into a conversion matrix list using elements of the type field identifier of the anchor type in the template definition file; the position of a type field identifier of the same element name in the value type element in the target page screenshot is calculated according to the conversion matrix list corresponding to the elements of the anchor type, and a text box of the value type element corresponding to the position is obtained using a document object model to obtain an xpath of the value type element; the corresponding elements in the target page screenshot are output according to the xpath of the value type element by calling an api interface of the browser extension using a pre-built RPA system; the conversion of the elements of the type field identifier of the anchor type in the target page screenshot into the conversion matrix list using the template definition file comprises: one of the elements of the type field identifier of the anchor type in the template definition file is selected, a text box matching the selected anchor type element is searched in the target page screenshot, and a conversion matrix of the text box is calculated; it is judged whether all the anchor type elements in the template definition file have been selected; if any anchor type element in the template definition file has not been selected, the step of selecting one of the elements of the type field identifier of the anchor type in the template definition file is returned; if all the anchor type elements in the template definition file have been selected, a conversion matrix list is obtained according to the conversion matrix of the text box corresponding to each anchor type element.

2. The template-based WEB page element positioning method of claim 1, wherein, the generation of the label of the region of interest comprises: the width and height of the region of interest are recorded; the name, position coordinates and type field identifier of each element are combined to obtain element field list labeling; the label of the region of interest is generated according to the width and height of the region of interest and the element field list labeling.

3. The template-based WEB page element positioning method of claim 1, wherein, the obtaining of the target page screenshot of the target WEB page in the browser using the page capturing extension plug-in comprises: a target WEB page url is received using the page capturing extension plug-in through an api interface; the tab page where the target WEB page is located is located according to the target WEB page url; the target page screenshot corresponding to the tab page is obtained.

4. The template-based WEB page element positioning method of claim 1, wherein, the obtaining of the conversion matrix list according to the conversion matrix of the text box corresponding to each anchor type element comprises: When there are multiple matched text boxes for an anchor type element, errors between each of the transformation matrices of all text boxes and position coordinates of the anchor type element in the template definition file are calculated, and a text box with the minimum error is selected as a text box matched with the anchor type element; When no matched text box is found for an anchor type element, a transformation matrix closest to the anchor type element is selected as a text box matched with the anchor type element.

5. A template-based WEB page element positioning apparatus, characterized by, The device comprises: A template file generation module is configured to perform page capturing on a WEB page currently displayed in a preset browser, generate a current page screenshot, select a page region of interest in the current page screenshot using a preset labeling tool, generate a label of the region of interest, and generate a template definition file according to the region of interest and the corresponding label, wherein the label comprises a name, position coordinates and a type field identifier of the region of interest; A WEB page element positioning module is configured to acquire a target page screenshot of a target WEB page in the browser using a page capturing extension plug-in, convert elements in the target page screenshot into a transformation matrix list using an element with a type field identifier as anchor type in the template definition file, calculate a position of an element with a same element name and a type field identifier as value type in the target page screenshot according to the transformation matrix list corresponding to the anchor type element, acquire a text box of the value type element at the position using a document object model, obtain an xpath of the value type element, and output the corresponding element in the target page screenshot according to the xpath of the value type element using a pre-constructed RPA system by calling a browser extension api interface. The conversion of the element with the type field identifier as anchor type in the target page screenshot into the transformation matrix list using the template definition file comprises: selecting one of the elements with the type field identifier as anchor type in the template definition file, searching for a text box matched with the selected anchor type element in the target page screenshot, and calculating a transformation matrix of the text box; judging whether all anchor type elements in the template definition file have been selected; if any anchor type element in the template definition file is not selected, returning to the step of selecting one of the elements with the type field identifier as anchor type in the template definition file; if all anchor type elements in the template definition file have been selected, obtaining a transformation matrix list according to the transformation matrix of each anchor type element corresponding to a text box.

6. The template-based WEB page element positioning apparatus of claim 5, wherein, The template file generation module generates the label of the region of interest using the following method: recording a width and a height of the region of interest; combining the name of each element, the position coordinates and the type field identification to obtain an element field list label; generating a label of the region of interest according to the width and height of the region of interest and the element field list label.

7. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the template-based WEB page element positioning method according to any one of claims 1 to 4.

8. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the template-based WEB page element positioning method according to any one of claims 1 to 4.

Images