A method and device for structured web page crawling suitable for RPA

Abstract

The present application discloses a method and device for structured webpage crawling suitable for RPA, the method comprising: obtaining list samples using a multimodal large language model MLLM according to webpage screenshots and user needs; extracting all element field texts for each list sample, matching the HTML information of the parent node element containing these element field texts on the webpage HTML as the HTML information of the list sample; generating a code for obtaining list information using a multimodal large language model MLLM according to all list samples and HTML information; running the list information code to obtain structured data of the webpage. It can automatically crawl the elements to be extracted according to user needs, and the user no longer needs to provide samples; in addition, it will automatically generate crawling logic according to different webpage environments, so that structured crawling has stronger adaptability.

Description

Technical Field

[0001] The present application relates to a method and device for structured web page crawling suitable for RPA, and belongs to the field of software technology. Background Art

[0002] In the process automation design of RPA (Robotic Process Automation), users can plan continuous web page operations according to their own needs. Each operation is an interactive action between the user and the web page UI (User Interface), such as clicking and inputting. After the web page automation process design is completed, the RPA program will automatically execute the page operation according to the user's design steps. Through the process design and automatic execution of RPA, users can greatly reduce repetitive and tedious manual page operations.

[0003] In addition to conventional clicks, inputs, hovers, or text acquisition, web page operations in the RPA process also involve some more complex operations, such as obtaining web page table data or product list data. In order to obtain data in list form, RPA software developers often add some special functional components. For obvious table elements on the web page, RPA software will add a "table acquisition" tool, that is, to capture table elements based on the obvious table type in the HTML of the web page. However, for some non-table list data, such as JD's product list and forum discussion posts, RPA software will add a tool called "structured crawling", that is, to obtain list data through certain fixed structural relationships in HTML, such as the price, name, and evaluation of all products.

[0004] Structured crawling in RPA software often requires users to provide element crawling samples. For example, users need to manually crawl the relevant information of the first product to tell the structured crawler what elements to obtain. Structured crawling will identify the HTML relationship between these elements and obtain the product information of the entire web page. User-provided examples make the structured crawler behave unnaturally. In addition, the tool's ability to identify element HTML relationships also varies in different web page environments. Summary of the invention

[0005] Multimodal large language models (MLLM), such as GPT4o and Claude3.5, have strong context comprehension and image recognition capabilities. Agents developed based on MLLM can plan page operations and automatically operate the user interface according to user needs. MLLM agents have great potential in web automation, and RPA vendors are integrating agents into their products.

[0006] Purpose: In view of at least one of the above technical problems, the present application provides a method and device for structured web page crawling suitable for RPA.

[0007] Technical solution: To solve the above technical problems, the technical solution adopted in this application is:

[0008] In a first aspect, the present application provides a method for crawling structured web pages applicable to RPA, the method comprising:

[0009] S1. According to the webpage screenshot and user needs, a list sample is obtained using a multimodal large language model MLLM; wherein the list sample includes a plurality of list samples with the same element fields;

[0010] S2. For each list sample, extract all element field texts, and match the HTML information of the parent node element containing these element field texts on the web page HTML as the HTML information of the list sample;

[0011] S3, based on all list samples and HTML information, use the multimodal large language model MLLM to generate the code for obtaining list information;

[0012] S4. Run the list information code to obtain web page structured data.

[0013] In a second aspect, the present application provides a webpage structured crawling device applicable to RPA, the device comprising:

[0014] A list sample acquisition module is used to: acquire list samples using a multimodal large language model MLLM according to web page screenshots and user needs; wherein the list samples include list samples with the same multiple element fields;

[0015] The HTML information acquisition module is used to: extract all element field texts for each list sample, and match the HTML information of the parent node element containing these element field texts on the web page HTML as the HTML information of the list sample;

[0016] A code generation module is used to: generate code for obtaining list information using a multimodal large language model MLLM based on all list samples and HTML information;

[0017] The structured data acquisition module is used to: run the list information code to obtain the structured data of the web page.

[0018] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.

[0019] In a fourth aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in the first aspect when executing the computer program.

[0020] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0021] Beneficial effects: The webpage structured crawling method and device for RPA provided in this application have the following advantages: a structured crawling tool developed based on MLLM is proposed, which can automatically crawl the elements to be extracted according to the needs of the user, and no longer requires the user to provide samples. In addition, the tool will automatically generate crawling logic according to different webpage environments, making structured crawling more adaptable. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of an application scenario of a method according to an embodiment of the present application;

[0023] Figure 2 The following is a flow chart of a method for crawling structured web pages applicable to RPA according to an embodiment of the present application;

[0024] Figure 3 A schematic diagram of a web page structured crawling device applicable to RPA according to an embodiment of the present application;

[0025] Figure 4 A schematic diagram of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] The present application is further described below in conjunction with the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present application, and cannot be used to limit the protection scope of the present application.

[0027] In this application, the professional terms are explained as follows:

[0028] Multimodal Large Language Model (MLLM): Using a powerful large language model (LLM) as the "brain", various multimodal tasks can be performed. The large language model LLM is a deep learning model trained with a large amount of text data, with billions of parameters, capable of understanding and processing natural language. It is a general natural language processing (NLP) model.

[0029] GPT-4o is a language model released by OpenAI for the chatbot ChatGPT, which "can reason about audio, vision, and text in real time.

[0030] Prompt words: text content used to prompt MLLM to provide targeted answers, usually including system prompts used to prompt MLLM in what role, in what scenario, and in what format to generate answers in the current context window; user prompts generally refer to the conversation content entered by the user, used to prompt MLLM to complete tasks, including but not limited to daily conversations, answering questions, translation, writing, and coding.

[0031] Robotic process automation (RPA) is a program that simulates the manual operations of users on the computer desktop through RPA process code to achieve process automation.

[0032] Before describing the embodiments of the present application in detail, a specific scenario example is used to reveal the scenario in which the technical solution of the present application can be used. In a possible scenario example, specifically, Figure 1 As shown, the user performs specific operations on the client 110, and the client 110 is deployed with a data collection device and a multimodal large language model MLLM. The data collection device is used to collect web page screenshots and user needs on the client 110, and the collected web page screenshots and user needs are transmitted to the server through the network. The server obtains list samples using the multimodal large language model MLLM through a preset program; for each list sample, all element field texts are extracted, and the HTML information of the parent node element containing these element field texts is matched on the web page HTML as the HTML information of the list sample; according to all list samples and HTML information, the multimodal large language model MLLM is used to generate a code for obtaining list information; the list information code is run to obtain web page structured data.

[0033] The multimodal large language model MLLM can run on the client terminal side or the server side. The method of the present application can be configured in the multimodal large language model MLLM on the client terminal / or server side, and the client terminal and the server are connected through network transmission to realize multi-terminal interaction.

[0034] The server may be an electronic device with certain computing and processing capabilities. For example, the server may be a server of a distributed system, or a system with multiple processors, memories, network communication modules, etc. operating in collaboration. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The server may also be a server cluster formed by several servers. Alternatively, with the development of science and technology, the server may also be a new technical means capable of realizing the corresponding functions of the embodiments of the specification. For example, it may be a new form of "server" based on quantum computing.

[0035] The client terminal may be an electronic device with network access capability. Specifically, for example, the terminal may be a desktop computer, a tablet computer, a laptop computer, a smart phone, etc. Alternatively, the terminal may also be software that can run in the electronic device.

[0036] The network may be any type of network that can support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. The one or more networks may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0037] Example 1: Figure 2 As shown, this embodiment provides a method for crawling a structured web page for RPA, which can be applied to a multimodal large language model MLLM, which can run in a client terminal. The method for crawling a structured web page for RPA includes:

[0038] Step S1: According to the webpage screenshot and user demand, a list sample is obtained using a multimodal large language model MLLM; wherein the list sample includes a plurality of list samples with the same element fields.

[0039] It should be noted that this application no longer requires users to provide a list sample themselves, that is, users no longer need to manually capture specific web page elements on the page, such as the name, price, address and other field specific elements of a certain product. Traditional RPA structured picking tools then build picking logic based on the HTML characteristics of these elements. Now, users only need to enter their own picking requirements, which can be specific, such as "I want to get the price, evaluation and store address of all products on the current web page"; it can also be a more general description, such as "get all product information." User needs will be used as part of the MLLM prompt words, allowing MLLM to organize the required element fields and obtain multiple list samples.

[0040] The multimodal large language model MLLM is selected from at least one of GPT-4o, Claude3.5, and Gork2. The MLLM used in this embodiment is GPT-4o, which supports data input in multiple modes such as text, pictures, and audio.

[0041] In this step, you need to input the web page screenshot and the text prompt word into GPT-4o to obtain a list sample.

[0042] Web page screenshots can be obtained through open source web automation tools such as playwright and selenium. Web page screenshots need to be scaled and 64-bit transcoded before being input to GPT-4o. The resolution of general office monitors is relatively high, such as 1920*1080. High-resolution images will slow down the response rate of GPT-4o, so this application scales high-resolution web page screenshots to around 960*960 pixels by default.

[0043] The text prompts in this step are relatively simple, consisting of four parts: task description, user requirements, image description, and output format. The task description requires obtaining structured data, and the output format requires output in the form of a dictionary list.

[0044] GPT-4o will directly return multiple list samples according to the output format. The element fields of each sample are the same, such as "{Product Name: Apple iPhone 15th Generation White, Price: 5999, Number of Positive Reviews: 100,000+}". Because the information of some products in the web page screenshot may be blocked by the window, for example, the price of the product is blocked and not displayed in the picture, the output dictionary information will be missing at this time, and this application will automatically discard the samples with missing information. In addition, when the demand does not clearly specify the element fields that need to be obtained, for example, the demand is "to obtain all product information", GPT-4o will summarize these important and obvious element fields according to the web page screenshot, saving the user the trouble of providing specific element fields.

[0045] With GPT-4o, not only can we select list samples by ourselves, but we can also summarize list fields by ourselves.

[0046] Step S2: for each list sample, extract all element field texts, and match the HTML information of the parent node element containing these element field texts on the web page HTML as the HTML information of the list sample.

[0047] Step S21, obtaining matching anchor elements using text similarity;

[0048] In some embodiments, step S21 may specifically include:

[0049] S211, extracting all element field texts of a list sample;

[0050] S212. Select the longest text in all element field texts as the matching anchor text; for example, select "Apple iPhone 15th Generation White" as the anchor in "{Product Name: Apple iPhone 15th Generation White, Price: 5999, Number of Positive Reviews: 100,000+}".

[0051] S213: Using a text similarity algorithm based on Jaccard distance, search the HTML of the web page for a web page element whose element text is closest to the anchor text as the anchor element.

[0052] The algorithm for calculating text similarity cannot use the conventional edit distance, because the anchor text is obtained by GPT-4o from the image, which is quite different from the actual text of the HTML element on the web page. When the actual element text is long, GPT-4o may only obtain part of the text on the image. For example, the text displayed in the HTML or on the web page is "iPhone 15th Generation - 128G White Original Stock, Shipping Immediately", while the text obtained by GPT-4o is "Apple iPhone 15th Generation White". GPT-4o cannot accurately obtain all the text on the image. In addition, GPT-4o may mix the meaning of the pictures around the text into the obtained text, and the obtained text is likely to be missing characters or add non-existent characters.

[0053] Because the content and length of the text obtained by GPT-4o are uncontrollable, ordinary text similarity algorithms will involve some invalid characters, which will degrade the calculation results.

[0054] Furthermore, in some embodiments, the text similarity algorithm based on Jaccard distance includes:

[0055] First, calculate the coexisting character set InterSection of the web page element text and the anchor text wa :

[0056]

[0057] Among them, C w is the character set of the web page element text, C a is the character set of the anchor text; through the intersection, the character set InterSection that coexists in the two texts is obtained wa ;

[0058] Then, calculate the coexisting character set InterSection wa Character set C with anchor text a The intersection and union ratio of Jaccard similarity is obtained. inter-a :

[0059]

[0060] Among them, len represents the length of the character set, Jaccard similarity Jaccard_Similarity inter-aIt represents the similarity between the web page element text and all valid characters in the anchor text; for example, "iPhone 15th generation - 128G white original stock, shipped immediately" or "ship immediately" will not enter the similarity algorithm.

[0061] Because the Jaccard similarity algorithm ignores the impact of the length of the web page element text on similarity. Therefore, based on the Jaccard similarity, a penalty term length is calculated for the text length difference. decay :

[0062]

[0063] When the length difference between the two text characters is too large, the penalty term length decay It will slightly reduce the similarity in this case.

[0064] When the length of the web page element text is greater than the length of the anchor text, the text similarity is expressed as:

[0065] Similarity=Jaccard_Similarity inter-a -length decay

[0066] When the length of the web page element text is less than or equal to the length of the anchor text, the text similarity is expressed as:

[0067]

[0068] That is, the string intersection and union ratio of the original two texts.

[0069] Using the above similarity algorithm, this application matches the web page element whose element text is closest to the anchor text from all web page elements. This web page element is called an anchor element, for example, the element with the text "iPhone 15th generation - 128G white original stock, shipped immediately".

[0070] Step S22: Use the anchor element to locate and obtain the HTML information of the parent node element.

[0071] The web page elements corresponding to the anchor element and other element fields must be under the same web page node. For example, "{Product Name: Apple iPhone 15th Generation White, Price: 5999, Number of Positive Reviews: 100,000+}", there must be a node that contains all the text of these three fields.

[0072] In some embodiments, step S22 may specifically include:

[0073] S221. Get the xpath of the anchor element, for example, " / html / body / div[1] / div[4] / div / div / ul / li[2]". The xpath describes the hierarchical location of a web page element in the entire HTML of the web page. The slash in the middle indicates the node name.

[0074] S222. According to the xpath of the anchor element, locate the parent node element layer by layer; if the parent node element contains other field texts, for example, the parent node element of " / html / body / div[1] / div[4] / div / div / ul" contains two texts "5999" and "100,000+";

[0075] S223, obtaining the parent node element HTML information that contains the field texts of these elements at the same time as the list sample HTML information.

[0076] The HTML information of all list samples can be obtained by using the method in step S2.

[0077] Step S3: Based on all list samples and HTML information, a multimodal large language model MLLM is used to generate a code for obtaining list information.

[0078] According to all the list samples and HTML information obtained in steps S1 and S2, GPT-4o is used again to generate Python code for obtaining global list information. At this time, the text prompt words are more complex, consisting of task description, user requirements, list sample information, Python function, task rules, and output format.

[0079] Table 1 The form of single list sample information in the prompt word

[0080]

[0081]

[0082] The task description requires generating Python code based on the list sample information to obtain all the list data of the entire web page. Each sample in the list sample information is arranged in the form of Table 1, which provides the list samples and HTML information of S1 and S2. The Python function gives some code styles. The task rules stipulate the specific logic of the coding task, including: (1) summarizing the common features of the list samples to write code; (2) summarizing the list information, using third-party Python libraries such as pandas to build and display data tables; (3) discarding data with missing element fields.

[0083] GPT-4o can generate structured picking codes that meet the current web page environment based on list samples, and generate element extraction codes with extremely high fault tolerance based on the HTML structure of the observed samples, with a higher code level than normal human coding. Each time the RPA program enters the same web page, these codes can be guaranteed to be continuously effective, that is, the structured loss codes generated in the RPA design phase can ensure continuous use in the running phase. In addition, for example, on shopping websites such as Taobao, the product list needs to be displayed on multiple pages, and the code generated based on the first page can be used to obtain the list information of subsequent pages. This method generates a set of special codes for different web page environments. For example, Taobao and Baidu Tieba correspond to different code logics, which ensures that structured picking can adapt to different web page environments and improves the robustness of the tool itself.

[0084] Step S4: Run the list information code to obtain web page structured data.

[0085] By running the code generated by GPT-4o in step S3, you can obtain the global list information as shown in Table 2, that is, the structured data of the web page.

[0086] Table 2 Example of structured data obtained by code

[0087]

[0088] In summary, the above-mentioned implementation method provides a structured webpage crawling method suitable for RPA. According to webpage screenshots and user needs, a multimodal large language model MLLM is used to obtain list samples; for each list sample, all element field texts are extracted, and the HTML information of the parent node element containing these element field texts is matched on the webpage HTML as the HTML information of the list sample; according to all list samples and HTML information, a multimodal large language model MLLM is used to generate a code for obtaining list information; the list information code is run to obtain structured data of the webpage. It can automatically crawl the elements that need to be extracted according to user needs, and the user no longer needs to provide samples. In addition, the crawling logic will be automatically generated according to different webpage environments, so that structured crawling has stronger adaptability.

[0089] Example 2: Based on Example 1, Figure 3 As shown, the present application provides a web page structured crawling device applicable to RPA, the device comprising:

[0090] A list sample acquisition module is used to: acquire list samples using a multimodal large language model MLLM according to web page screenshots and user needs; wherein the list samples include list samples with the same multiple element fields;

[0091] The HTML information acquisition module is used to: extract all element field texts for each list sample, and match the HTML information of the parent node element containing these element field texts on the web page HTML as the HTML information of the list sample;

[0092] A code generation module is used to: generate code for obtaining list information using a multimodal large language model MLLM based on all list samples and HTML information;

[0093] The structured data acquisition module is used to: run the list information code to obtain the structured data of the web page.

[0094] The specific functional implementation of each of the above modules can be found in the relevant contents of the method in Example 1 and will not be elaborated here.

[0095] Embodiment 3: Based on Embodiment 1, this embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in Embodiment 1 are implemented.

[0096] Embodiment 4: Based on embodiment 1, as Figure 4 As shown, this embodiment provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in Embodiment 1 when executing the computer program.

[0097] Embodiment 5: Based on Embodiment 1, this embodiment proposes a hardware system that applies a method for crawling a structured web page suitable for RPA in Embodiment 1. The method is applied on the server side.

[0098] The terminal may be an electronic device with network access capability, for example, a desktop computer, a tablet computer, a laptop computer, a smart phone, a digital assistant, a shopping guide terminal, a television, etc.

[0099] The server can be an electronic device with certain computing and processing capabilities. For example, the server can be a server of a distributed system, or a system with multiple processors, memories, network communication modules, etc. operating in collaboration. The server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The server can also be a server cluster formed by several servers. Alternatively, with the development of science and technology, the server can also be a new technical means that can realize the corresponding functions of the implementation method of the specification. For example, it can be a new form of "server" based on quantum computing.

[0100] Example 6: Based on Example 1, this example further provides a computer program product comprising instructions, which, when executed by a computer, enables the computer to execute a method for structured web page crawling suitable for RPA in Example 1.

[0101] It should be understood that the specific examples in this article are only intended to help those skilled in the art better understand the implementation methods of this specification, rather than limiting the scope of this application.

[0102] It can be understood that in the various implementations of this specification, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation methods of this specification.

[0103] It can be understood that the various embodiments described in this specification can be implemented individually or in combination, and the embodiments of this specification are not limited to this.

[0104] Unless otherwise specified, all technical and scientific terms used in the embodiments of this specification have the same meaning as those generally understood by those skilled in the art of the technical field of this specification. The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the scope of this specification. The term "and / or" used in this specification includes any and all combinations of one or more related listed items. The singular forms of "a", "above", and "the" used in the embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0105] It can be understood that the processor of the embodiment of this specification can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method implementation can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiment of this specification can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of this specification can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0106] It is understood that the memory in the embodiments of this specification may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasablePROM, EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0107] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this specification.

[0108] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method implementation methods and will not be repeated here.

[0109] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0110] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0111] In addition, each functional unit in each embodiment of the present specification may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0112] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of this specification. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0113] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for crawling structured web pages suitable for RPA, characterized in that: The method comprises: S1. According to the webpage screenshot and user needs, a list sample is obtained using a multimodal large language model MLLM; wherein the list sample includes a plurality of list samples with the same element fields; S2. For each list sample, extract all element field texts, and match the HTML information of the parent node element containing these element field texts on the web page HTML as the HTML information of the list sample; S3, based on all list samples and HTML information, use the multimodal large language model MLLM to generate the code for obtaining list information; S4. Run the list information code to obtain web page structured data.

2. The method for crawling structured web pages applicable to RPA according to claim 1, characterized in that: Step S2 specifically includes: S21, using text similarity to obtain matching anchor elements; S22. Use the anchor element to locate and obtain the parent node element HTML information as the list sample HTML information.

3. The method for crawling structured web pages applicable to RPA according to claim 2, characterized in that: S21. Obtain matching anchor elements using text similarity, including: S211, extracting all element field texts of a list sample; S212, selecting the longest text among all element field texts as the matching anchor text; S213: Using a text similarity algorithm based on Jaccard distance, search the HTML of the web page for a web page element whose element text is closest to the anchor text as the anchor element.

4. The method for crawling structured web pages for RPA according to claim 2, characterized in that: S22. Using the anchor element to locate, obtain the parent node element HTML information as the list sample HTML information, including: S221, obtaining the xpath of the anchor element, where the xpath describes the hierarchical location of a web page element in the entire web page HTML; S222, according to the xpath of the anchor element, locate the parent node element layer by layer; S223. Obtain HTML information of the parent node element that contains the field texts of these elements at the same time as list sample HTML information.

5. The method for crawling structured web pages applicable to RPA according to claim 3, characterized in that: Text similarity algorithms based on Jaccard distance include: First, calculate the coexisting character set InterSection of the web page element text and the anchor text wa : InterSection wa =C w ∩C a , Among them, C w is the character set of the web page element text, C a is the character set of the anchor text; through the intersection, the character set InterSection that coexists in the two texts is obtained wa ; Then, calculate the coexisting character set InterSection wa Character set C with anchor text a The intersection and union ratio of Jaccard similarity is obtained. inter-a : Among them, len represents the length of the character set, Jaccard similarity Jaccard_Similarity inter-a It represents the similarity between the web page element text and all valid characters in the anchor text; Based on Jaccard similarity, calculate a penalty term length for text length difference decay : When the length of the web page element text is greater than the length of the anchor text, the text similarity is expressed as: Similarity=Jaccard_Similarity inter-a -length decay , When the length of the web page element text is less than or equal to the length of the anchor text, the text similarity is expressed as:

6. The method for crawling structured web pages suitable for RPA according to claim 1, characterized in that: The multimodal large language model MLLM is selected from at least one of GPT-4o and Claude3.

5.

7. The method for crawling structured web pages for RPA according to claim 1, characterized in that: In step S1, the output format of the list sample is output in the form of a dictionary list.

8. A webpage structured crawling device suitable for RPA, characterized in that: The device comprises: A list sample acquisition module is used to: acquire list samples using a multimodal large language model MLLM according to web page screenshots and user needs; wherein the list samples include list samples with the same multiple element fields; The HTML information acquisition module is used to: extract all element field texts for each list sample, and match the HTML information of the parent node element containing these element field texts on the web page HTML as the HTML information of the list sample; A code generation module is used to: generate code for obtaining list information using a multimodal large language model MLLM based on all list samples and HTML information; The structured data acquisition module is used to: run the list information code to obtain the structured data of the web page.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images