Webpage element acquisition method and device, electronic equipment and storage medium

Abstract

The invention provides a webpage element acquisition method and device, electronic equipment and a storage medium, and relates to the technical field of data processing, in particular to the fields of webpage content capture, equipment scheduling and task arrangement. According to the specific implementation scheme, the method comprises the steps of obtaining element information of a to-be-obtained webpage element in a webpage, and obtaining a target task flow according to the element information; and executing the target task flow to obtain the target webpage element from the webpage.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, specifically to the field of web page content capture, device scheduling and task scheduling, and in particular to a method, device, electronic device and storage medium for acquiring web page elements. Background Art

[0002] When automatically crawling and processing network data, it is necessary to quickly obtain and operate web page elements. Existing technologies consume a lot of computing resources and do not support highly customized operations, resulting in poor user experience. Summary of the invention

[0003] The present disclosure provides a method, device, electronic device and storage medium for acquiring web page elements.

[0004] According to one aspect of the present disclosure, a method for acquiring a web page element is provided, comprising:

[0005] Obtaining element information of a web page element to be obtained in a web page;

[0006] Acquire a target task flow according to the element information;

[0007] The target web page element is obtained from the web page by executing the target task flow.

[0008] According to another aspect of the present disclosure, a device for acquiring web page elements is provided, comprising:

[0009] A first acquisition module is used to acquire element information of a web page element to be acquired in a web page;

[0010] A second acquisition module, used to acquire a target task flow according to the element information;

[0011] The third acquisition module is used to acquire a target web page element from the web page by executing the target task flow.

[0012] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the execution is performed by the at least one processor so that the at least one processor can execute the method described in the first aspect embodiment.

[0013] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the embodiment of the first aspect.

[0014] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the steps of the method according to the embodiment of the first aspect are implemented.

[0015] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0017] Figure 1 It is a flowchart of a method for obtaining web page elements provided by an embodiment of the present disclosure;

[0018] Figure 2 is a flow chart of another method for obtaining web page elements provided by an embodiment of the present disclosure;

[0019] Figure 3 is a flow chart of another method for obtaining web page elements provided by an embodiment of the present disclosure;

[0020] Figure 4 It is a logic schematic diagram of another method for obtaining web page elements provided by an embodiment of the present disclosure;

[0021] Figure 5 It is a structural schematic diagram of a device for acquiring web page elements provided by an embodiment of the present disclosure;

[0022] Figure 6 A schematic block diagram of an electronic device for implementing an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0023] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0024] Data processing is the collection, storage, retrieval, processing, transformation and transmission of data. Its basic purpose is to extract and derive valuable and meaningful data for certain specific people from a large amount of data that may be disorganized and difficult to understand.

[0025] Web scraping, also known as web crawling, refers to the automatic acquisition of information on the Internet through programs; web scraping is the process of automatically extracting data from websites by writing programs to simulate human browsing behavior.

[0026] Equipment Scheduling refers to the allocation, scheduling and control of various equipment, while making the equipment operate in a coordinated manner under the premise of meeting the conditions to achieve the company's production goals; using manual or automated systems to allocate, manage and schedule equipment to optimize production efficiency and reduce costs.

[0027] Task Orchestration refers to the process of organizing and managing multiple tasks according to specific dependencies and execution order to ensure that each task can be executed efficiently in a predetermined logical order.

[0028] Figure 1 is a flow chart of a method for obtaining web page elements provided by an embodiment of the present disclosure; Figure 1 As shown, the method includes:

[0029] S101, obtaining element information of a web page element to be obtained in a web page.

[0030] Optionally, the web page element to be obtained is a component element existing in the web page, such as text, image, animation, sound, table, hyperlink or navigation bar, etc. The element to be obtained can be used as the target web page element.

[0031] In some implementations, the target web page element can be a target selected or pre-configured by the user, such as an element selected by the user through a click or other operation, or the user inputs the coordinates or path information of the element's location in advance to determine the location of the target web page element to be obtained, that is, the location of the web page element to be obtained.

[0032] In some implementations, the element information of the web page element to be obtained may include information such as element type, element position, and element attributes.

[0033] S102: Acquire a target task flow according to the element information.

[0034] Optionally, task flows can be arranged based on element information to obtain a target task flow. For example, task flows can be arranged based on element positions and element attributes in the element information to define the capture of web page elements to be acquired, such as how to capture and the steps for capturing, thereby generating a target task flow. The acquisition of the web page elements to be acquired is implemented based on the target task flow. The target task flow may include execution steps and execution conditions for acquiring the web page elements to be acquired.

[0035] S103, acquiring a target web page element from the web page by executing the target task flow.

[0036] Optionally, the corresponding execution steps may be executed based on the execution conditions in the target task flow, thereby realizing the process of obtaining the target web page elements from the web page, thereby improving the efficiency and flexibility of automated network data capture.

[0037] In this embodiment, the target web page elements that need to be obtained are determined, and the task flow is arranged based on the element information of the target web page elements that need to be obtained to obtain a target task flow. The target task flow can extract the target web page elements in a more targeted manner, thereby improving the efficiency and flexibility of automated data capture. The target web page elements are acquired by executing the target task flow, thereby reducing data processing time and resource costs, and improving element capture efficiency.

[0038] Figure 2 is a flow chart of another method for obtaining web page elements provided by an embodiment of the present disclosure; Figure 2 As shown, the method includes:

[0039] S201, obtaining element information of a web page element to be obtained in a web page.

[0040] In the embodiments of the present disclosure, the implementation method of step S201 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0041] S202: Acquire a target task flow according to the element information.

[0042] In some implementations, attribute information of the web page element to be obtained may be determined based on element information of the web page element to be obtained, such as a unique identifier of the element, an element type, or other description information, which is not specifically limited.

[0043] According to the attribute information of the web page element to be obtained, the acquisition steps and execution conditions corresponding to the web page element to be obtained are determined; for example, the acquisition step is to locate the web page element to be obtained according to the unique identifier of the web page element to be obtained, and then determine the web page element to be obtained according to the location, or the acquisition step is to determine one or more web page elements according to the element type, and further determine the web page elements to be obtained that meet the requirements according to other description information, and the execution conditions may include information such as the execution requirements of each acquisition step.

[0044] According to the element information of the web page elements to be obtained, the association relationship between the web page elements to be obtained is determined; optionally, the association relationship may include a parent-child relationship, a hierarchical relationship, and a sibling relationship, etc., and the acquisition steps are logically analyzed according to the association relationship to obtain a logical analysis result, such as obtaining the execution order logic of the acquisition steps, and according to the logical analysis result, the acquisition steps and execution conditions are configured to generate a more detailed and clear target task flow.

[0045] S203, sending a remote call request to the scheduling server of the server cluster through the application programming interface API.

[0046] It can be understood that the Application Programming Interface (API) is a communication protocol that defines a set of methods, rules and standards, allowing different applications to exchange data and share functions with each other. It is a bridge for communication and interaction between applications. Therefore, remote call requests can be sent to the scheduling server of the server cluster through the API.

[0047] In some implementations, the remote call request is used to request to call a server in a server cluster. After receiving the remote call request, the scheduling server can select one or more servers from the server cluster for call feedback according to the remote call request.

[0048] S204, after passing the identity authentication, sending the target task flow to the target server in the server cluster.

[0049] Optionally, identity authentication can be verified by means of an account password or whitelist record information, so as to determine that the device sending the remote call request is a verified secure device.

[0050] After the identity authentication is passed, the scheduling server sends the target task flow to the target server in the server cluster. The target server executes the target task flow to obtain the target web page elements from the web page. The target server is the server determined by the scheduling server from the server cluster based on the scheduling algorithm to execute the target task flow.

[0051] In some implementations, the scheduling algorithm is used to determine the best scheduled server from a server cluster, for example, obtaining the working status of each server in the server cluster, preliminarily screening out available servers from the server cluster based on the working status, and determining from the available servers the server whose execution capability meets the current target task flow and achieves the least resource waste as the target server.

[0052] In this embodiment, attribute information, acquisition steps, execution conditions and associations between elements are acquired according to element information of the web page element to be acquired, so as to construct a corresponding target task flow, which can accurately acquire the target web page element; a remote call request is sent to the scheduling server of the server cluster through the API interface, and the target task flow is sent to the target server in the server cluster after the identity authentication is passed, thereby improving data security, so that the target server executes the target task flow to obtain the target web page element, and the target server is the best server determined from the server cluster based on the scheduling algorithm, thereby avoiding unsuccessful execution of the target task flow or waste of resources, and acquiring the target web page element by executing the acquisition of the target web page element by the remote server, thereby reducing the burden of the user device to execute the capture of the target web page element, and improving the stability and efficiency of element acquisition.

[0053] Figure 3 is a flow chart of another method for obtaining web page elements provided by an embodiment of the present disclosure; Figure 3 As shown, the method includes:

[0054] S301, obtaining element information of a web page element to be obtained in a web page.

[0055] Optionally, the element selection operation information performed on the visual interface can be obtained; the path information can be determined based on the element selection operation information; in some implementations, the element selection operation performed on the visual interface can be recorded by a recording engine, that is, the user's operation behavior is recorded, and the user's operation behavior is the operation of selecting elements on the visual interface, and the recording information is obtained, and then the front-end tracking is performed based on the recording information to determine the path information of the selected element, and the path information may include the location of the target web page element.

[0056] Furthermore, the source code of the web page is parsed to obtain a parsing result; the web page element to be obtained is determined from the parsing result according to the path information; it can be understood that the source code of the web page can be used to describe the result and content of the web page, including various tags and elements in the web page, so the parsing result includes all element information of the web page, and matching and positioning are performed from the parsing result according to the path information of the web page element to be obtained, and the location of the web page element to be obtained is determined, thereby determining the web page element to be obtained that needs to be obtained from the web page, thereby improving the positioning accuracy of the web page element to be obtained, and the determination efficiency of the web page element to be obtained is higher.

[0057] S302: Acquire the target task flow according to the element information.

[0058] In some implementations, an entry control of a task flow orchestration tutorial can be displayed on a visualization interface, where the orchestration tutorial includes at least one of a text tutorial and a video tutorial; that is, the entry control can be seen on the visualization interface, and the entry control can be used to obtain the task flow orchestration tutorial, thereby learning the task flow orchestration operations.

[0059] It is understandable that the click operation of the entry control can be monitored, and in response to detecting the selection operation on the entry control, at least one of the following items can be output on the visual interface: a text tutorial and a video tutorial; that is, when the user clicks to select the entry control, it indicates that the user needs to view the orchestration tutorial at this time, so the text tutorial is displayed or the video tutorial is played on the visual interface to enhance the user experience and task flow orchestration efficiency.

[0060] In the embodiments of the present disclosure, the implementation method of step S302 may be implemented in any manner in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0061] S303, sending a remote call request to the scheduling server of the server cluster through the application programming interface API.

[0062] In the embodiments of the present disclosure, the implementation method of step S303 may be implemented in any manner in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0063] S304, after passing the identity authentication, sending the target task flow to the target server in the server cluster.

[0064] In the embodiments of the present disclosure, the implementation method of step S304 may be implemented in any manner in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0065] S305, acquiring a target web page element from the web page by executing the target task flow.

[0066] Optionally, the target task flow can be updated and deleted based on the element information on the visualization interface, that is, the target task flow can be maintained on the visualization interface, such as creating, viewing, updating or deleting the target task flow on the visualization interface, so as to improve the operational flexibility of editing the target task flow.

[0067] In some implementations, the target task flow can also be executed locally, that is, the target task flow is directly clicked and executed based on the current client device to obtain the target web page element in the web page.

[0068] In response to the target task flow being executed locally, a first request message can be sent to the execution device of the target task flow to obtain the real-time status of the target task flow corresponding to the first request message; in response to the target task flow being executed on the target server, a first request message can be sent to the target server to obtain the real-time status of the target task flow corresponding to the first request message. That is, when the target task flow is executed locally or on the target server, a first request message can be sent to obtain the real-time status. For example, the real-time status can be a status reflecting the completion status of the task, such as active, completed or stopped. The current status of the target task flow is determined in real time based on the real-time status.

[0069] In some implementations, in response to the target task flow being executed locally, a second request message can also be sent to the execution device of the target task flow to obtain the execution data of the target task flow corresponding to the second request message; in response to the target task flow being executed on the target server, a second request message can be sent to the target server to obtain the execution data of the target task flow corresponding to the second request message; that is, when the target task flow is executed locally or on the target server, a second request message can be sent to obtain the execution data of the target task flow, and the execution data can include task execution-related data information such as progress data of the task flow, detailed information of the task flow, and memory usage, so as to obtain the status of the execution process of the target task flow.

[0070] S306, obtaining the configured callback uniform resource locator URL.

[0071] In some implementations, a Uniform Resource Locator (URL) is a coding method for identifying resources on the Internet and is the address of a standard resource on the Internet. The callback URL can be used to store the acquired target web page element, so the pre-configured callback URL is acquired and the callback URL is used as the storage location of the target web page element.

[0072] S307: In response to the target task being executed locally, sending the target web page element to the business system associated with the callback URL or the storage location associated with the callback URL.

[0073] Optionally, the target structured format of the target web page element can be determined according to the business system or storage location associated with the callback URL; the target structured format is a format supported for storage and processing by the business system or storage location associated with the callback URL, and the target web page element is formatted according to the target structured format to obtain target structured information, and the target structured information after the format conversion is further sent to the business system associated with the callback URL or the storage location associated with the callback URL, and the associated system or storage location stores the target structured information, so that data storage is more compatible and the data transmission speed is improved.

[0074] S308: In response to the target task being executed on the target server, a callback URL is sent to the target server.

[0075] It is understandable that the callback URL is sent to the target server. After the target server executes the task flow and obtains the target web page element, it sends the target web page element according to the callback URL. The business system or storage location associated with the callback URL is used to receive the target web page element sent by the target server.

[0076] Optionally, the target structured format of the target web page element can also be determined based on the business system associated with the callback URL or the storage location associated with the callback URL; the target web page element is formatted according to the target structured format to obtain target structured information, and the target structured information is stored in the business system and storage location associated with the callback URL, so that data storage is more compatible and the data transmission speed is improved.

[0077] In this embodiment, the recording engine obtains the selection operation information of the element on the visualization interface, and the path information of the element is determined based on the selection operation information to obtain the web page element to be obtained, thereby improving the flexibility of obtaining the web page element to be obtained, and arranging the target task flow according to the path information of the web page element to be obtained. In addition, the entry control of the task flow arrangement tutorial can be displayed on the visualization interface, so that the user can view the text tutorial or video tutorial in time when the user is not familiar with the task flow arrangement, thereby realizing the arrangement of the task flow and improving the user experience. In addition, the task flow can also be updated or deleted on the visualization interface, and the arrangement of the task flow is more flexible. After determining the target task flow, the target task flow is executed by local execution or remote call to the target server to obtain the target web page element, and the first request information and the second request information can be sent to the execution device to obtain the real-time status and execution data of the target task flow, and the execution status of the target task flow is obtained in real time. The configured callback URL can also be obtained, and based on the local execution or server execution status, the target web page element is stored in the business system or storage location associated with the callback URL, thereby improving the flexibility and efficiency of element data processing.

[0078] Figure 4It is a logical schematic diagram of another method for obtaining web page elements provided by an embodiment of the present disclosure; a user (user) performs task flow orchestration through a recording engine and an orchestration engine to obtain a target task flow corresponding to a target web page element, and executes the target task flow locally through a personal machine or remotely calls a target server through an API interface to execute the target task flow, thereby obtaining the target web page element; during the execution of the target task flow by the personal machine, the target web page element can be displayed through a visual interface; during the execution of the target task flow by the API interface calling the target server, the scheduling server provides a task scheduling service, schedules the target server from an execution machine group (server cluster), and the target server determines the target web page element, and can also perform format conversion processing based on business service requirements, and store according to a callback URL, thereby improving the flexibility and efficiency of crawling the target web page elements and improving the user experience.

[0079] Figure 5 is a schematic diagram of a structure of a device for acquiring web page elements provided by an embodiment of the present disclosure; Figure 5 As shown, the web page element acquisition device 500 includes:

[0080] The first acquisition module 501 is used to acquire element information of a web page element to be acquired in a web page;

[0081] The second acquisition module 502 is used to acquire the target task flow according to the element information;

[0082] The third acquisition module 503 is used to acquire a target web page element from the web page by executing the target task flow.

[0083] In some implementations, the third acquisition module 503 includes:

[0084] Send a remote call request to the scheduling server of the server cluster through the application program interface API;

[0085] After passing the identity authentication, the target task flow is sent to the target server in the server cluster. The target server is a server for executing the target task flow determined by the scheduling server from the server cluster based on the scheduling algorithm.

[0086] In some implementations, the third acquisition module 503 further includes:

[0087] Get the configured callback uniform resource locator URL;

[0088] In response to the target task being executed locally, sending the target web page element to a business system associated with the callback URL or a storage location associated with the callback URL;

[0089] In response to the target task being executed on the target server, a callback URL is sent to the target server, and the business system or the storage location is used to receive the target web page element sent by the target server.

[0090] In some implementations, the apparatus 500 further includes:

[0091] Determine the target structured format of the target web page elements based on the business system or storage location;

[0092] According to the target structured format, the target web page elements are formatted and the target structured information is obtained.

[0093] In some implementations, the second acquisition module 502 includes:

[0094] Determine attribute information based on element information;

[0095] According to the attribute information, determine the acquisition steps and execution conditions;

[0096] Determine the element association relationship based on element information;

[0097] Performing logic analysis on the acquisition steps according to the association relationship to obtain the logic analysis results;

[0098] According to the results of logical analysis, the configuration acquisition steps and execution conditions are arranged to obtain the target task flow.

[0099] In some implementations, the first acquisition module 501 includes:

[0100] Get the element selection operation information performed on the visual interface;

[0101] Select operation information based on the element and determine the path information;

[0102] Parse the source code of the web page to obtain the parsing result;

[0103] Determine the web page element to be obtained from the parsed result according to the path information.

[0104] In some implementations, the apparatus 500 further includes:

[0105] In response to the target task flow being executed locally, sending first request information to an execution device of the target task flow to obtain a real-time state of the target task flow corresponding to the first request information;

[0106] In response to the target task flow being executed on the target server, first request information is sent to the target server to obtain a real-time status of the target task flow corresponding to the first request information.

[0107] In some implementations, the apparatus 500 further includes:

[0108] In response to the target task flow being executed locally, sending second request information to an execution device of the target task flow to obtain execution data of the target task flow corresponding to the second request information;

[0109] In response to the target task flow being executed on the target server, second request information is sent to the target server to obtain execution data of the target task flow corresponding to the second request information.

[0110] In some implementations, the apparatus 500 further includes:

[0111] Displaying an entry control of an orchestration tutorial of a task flow on a visual interface, wherein the orchestration tutorial includes at least one of a text tutorial and a video tutorial;

[0112] In response to detecting a selection operation on the entry control, at least one of the following is output on the visual interface:

[0113] Text and video tutorials.

[0114] In some implementations, after obtaining the target web page element, the apparatus 500 further includes:

[0115] Update the target task flow according to the element information;

[0116] Delete the target task flow.

[0117] In this embodiment, the target web page elements to be acquired are determined, and the task flow is arranged based on the element information of the web page elements to be acquired to obtain a target task flow. The target task flow can extract the target web page elements in a more targeted manner, thereby improving the efficiency and flexibility of automated data capture. The acquisition of the target web page elements is achieved by executing the target task flow, thereby reducing data processing time and resource costs, and improving element capture efficiency.

[0118] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0119] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0120] Figure 6A schematic block diagram of an electronic device for implementing an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0121] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0122] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0123] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the acquisition method of web page elements. For example, in some embodiments, the acquisition method of web page elements may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the acquisition method of the web page elements described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the acquisition method of the web page elements in any other appropriate manner (e.g., by means of firmware).

[0124] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0126] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0128] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0129] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0130] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0131] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for obtaining web page elements, wherein: The method comprises: Obtaining element information of a web page element to be obtained in a web page; Acquire a target task flow according to the element information; The target web page element is obtained from the web page by executing the target task flow.

2. The method according to claim 1, wherein: The step of acquiring a target webpage element from the webpage by executing the target task flow includes: Send a remote call request to the scheduling server of the server cluster through the application program interface API; After passing the identity authentication, the target task flow is sent to the target server in the server cluster, where the target server is a server in the server cluster determined by the scheduling server based on a scheduling algorithm to execute the target task flow.

3. The method according to claim 2, wherein: After acquiring the target web page element from the web page by executing the target task flow, the method further includes: Get the configured callback uniform resource locator URL; In response to the target task being executed locally, sending the target web page element to a business system associated with the callback URL or a storage location associated with the callback URL; In response to the target task being executed on the target server, the callback URL is sent to the target server, and the business system or the storage location is used to receive the target web page element sent by the target server.

4. The method according to claim 3, wherein: The method further comprises: Determining a target structured format of the target web page element according to the business system or the storage location; According to the target structured format, the target web page elements are formatted and converted to obtain target structured information.

5. The method according to any one of claims 1 to 4, wherein: The acquiring the target task flow according to the element information includes: Determine attribute information according to the element information; Determining acquisition steps and execution conditions according to the attribute information; Determine the element association relationship according to the element information; Performing a logical analysis on the acquisition step according to the association relationship to obtain a logical analysis result; According to the logic analysis result, the acquisition step and the execution condition are configured to obtain the target task flow.

6. The method according to claim 5, wherein: The step of obtaining element information of a webpage element to be obtained in a webpage includes: Get the element selection operation information performed on the visual interface; Selecting operation information according to the element and determining path information; Parse the source code of the web page to obtain the parsing result; The web page element to be obtained is determined from the parsing result according to the path information.

7. The method according to any one of claims 2 to 4, wherein: The method further comprises: In response to the target task flow being executed locally, sending first request information to an execution device of the target task flow to obtain a real-time state of the target task flow corresponding to the first request information; In response to the target task flow being executed on the target server, first request information is sent to the target server to obtain a real-time status of the target task flow corresponding to the first request information.

8. The method according to any one of claims 2 to 4, wherein: The method further comprises: In response to the target task flow being executed locally, sending second request information to an execution device of the target task flow to obtain execution data of the target task flow corresponding to the second request information; In response to the target task flow being executed on the target server, second request information is sent to the target server to obtain execution data of the target task flow corresponding to the second request information.

9. The method according to any one of claims 1 to 4, wherein: The method further comprises: Displaying an entry control of a task flow orchestration tutorial on a visual interface, wherein the orchestration tutorial includes at least one of a text tutorial and a video tutorial; In response to detecting a selection operation on the portal control, at least one of the following is output on the visual interface: The text tutorial and the video tutorial.

10. The method according to any one of claims 1 to 4, wherein: After obtaining the target webpage element, the method further includes at least one of the following: Update the target task flow according to the element information; Delete the target task flow.

11. A device for acquiring web page elements, comprising: A first acquisition module is used to acquire element information of a web page element to be acquired in a web page; A second acquisition module, used to acquire a target task flow according to the element information; The third acquisition module is used to acquire a target web page element from the web page by executing the target task flow.

12. The device according to claim 11, wherein: The third acquisition module includes: Send a remote call request to the scheduling server of the server cluster through the application program interface API; After passing the identity authentication, the target task flow is sent to the target server in the server cluster, where the target server is a server in the server cluster determined by the scheduling server based on a scheduling algorithm to execute the target task flow.

13. The device according to claim 12, wherein: The third acquisition module further includes: Get the configured callback uniform resource locator URL; In response to the target task being executed locally, sending the target web page element to a business system associated with the callback URL or a storage location associated with the callback URL; In response to the target task being executed on the target server, the callback URL is sent to the target server, and the business system or the storage location is used to receive the target web page element sent by the target server.

14. The device according to claim 13, wherein: The device also includes: Determining a target structured format of the target web page element according to the business system or the storage location; According to the target structured format, the target web page elements are formatted and converted to obtain target structured information.

15. The device according to any one of claims 11 to 14, wherein: The second acquisition module includes: Determine attribute information according to the element information; Determining acquisition steps and execution conditions according to the attribute information; Determine the element association relationship according to the element information; Performing a logical analysis on the acquisition step according to the association relationship to obtain a logical analysis result; According to the logic analysis result, the acquisition step and the execution condition are configured to obtain the target task flow.

16. The device according to claim 15, wherein: The first acquisition module includes: Get the element selection operation information performed on the visual interface; Selecting operation information according to the element and determining path information; Parse the source code of the web page to obtain the parsing result; The web page element to be obtained is determined from the parsing result according to the path information.

17. The device according to any one of claims 12 to 14, wherein: The device also includes: In response to the target task flow being executed locally, sending first request information to an execution device of the target task flow to obtain a real-time state of the target task flow corresponding to the first request information; In response to the target task flow being executed on the target server, first request information is sent to the target server to obtain a real-time status of the target task flow corresponding to the first request information.

18. The device according to any one of claims 12 to 14, wherein: The device also includes: In response to the target task flow being executed locally, sending second request information to an execution device of the target task flow to obtain execution data of the target task flow corresponding to the second request information; In response to the target task flow being executed on the target server, second request information is sent to the target server to obtain execution data of the target task flow corresponding to the second request information.

19. The device according to any one of claims 11 to 14, wherein: The device also includes: Displaying an entry control of a task flow orchestration tutorial on a visual interface, wherein the orchestration tutorial includes at least one of a text tutorial and a video tutorial; In response to detecting a selection operation on the portal control, at least one of the following is output on the visual interface: The text tutorial and the video tutorial.

20. The device according to any one of claims 11 to 14, wherein: After acquiring the target webpage element, the device further includes: Update the target task flow according to the element information; Delete the target task flow.

21. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

23. A computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.

Images