Content extraction method, device, electronic device, storage medium and program product

Abstract

The present disclosure discloses a content extraction method, apparatus, electronic device, storage medium, and program product. The method comprises: obtaining a content extraction flowchart; the content extraction flowchart comprises a plurality of sequentially connected process components, the plurality of process components being used to configure flowchart configuration information; parsing the process components and their sequential relationships in the content extraction flowchart to obtain flowchart configuration information; the flowchart configuration information comprises a content extraction address and a content parsing expression; obtaining a webpage based on the content extraction address; extracting a target variable from the webpage based on the content parsing expression; and storing the target variable. This method can reduce labor and time costs and improve content extraction efficiency.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and more particularly to a content extraction method, device, electronic device, storage medium, and program product. Background Art

[0002] Currently, a lot of business information is usually carried in the form of HTML web pages, and related businesses need to extract corresponding content from these web pages, and process the extracted content according to business needs before displaying or storing it.

[0003] Typically, automated content extraction from web pages requires developers to implement content scraping scripts and execute them to extract the required content. However, business personnel are often familiar with business requirements but lack programming skills. Extracting the required content from web pages requires the assistance of additional programmers, requiring collaboration between business personnel and programmers, resulting in high labor and time costs.

[0004] Therefore, it is necessary to propose a solution that can be completed by business personnel to extract business-required content from web pages, so as to reduce labor costs and time costs and improve content extraction efficiency. Summary of the Invention

[0005] Embodiments of the present disclosure provide a content extraction method, apparatus, electronic device, storage medium, and program product.

[0006] In a first aspect, an embodiment of the present disclosure provides a content extraction method, comprising:

[0007] Obtaining a content extraction flowchart; the content extraction flowchart includes a plurality of flow components connected in sequence, and the plurality of flow components are used to configure flow chart configuration information;

[0008] Parsing each process component and sequence relationship in the content extraction flow chart to obtain flow chart configuration information; the flow chart configuration information includes a content extraction address and a content parsing expression;

[0009] Acquire a web page based on the content extraction address;

[0010] Extracting a target variable from the web page based on the content parsing expression;

[0011] The target variable is stored.

[0012] Furthermore, the flowchart configuration information further includes a content processing expression; and the method further includes:

[0013] The target variable is processed based on the content processing expression, and the processing result of the target variable is output.

[0014] Furthermore, the flowchart configuration information further includes a content loop expression, wherein the content loop expression is configured with at least an initial value of a webpage index and an upper limit value of a webpage index; before storing the target variable, the method further includes:

[0015] Increasing the initial value of the webpage index in the content loop expression by a set value to obtain the current value of the webpage index;

[0016] Obtaining a new content extraction address based on the current value of the webpage index and the content extraction address;

[0017] Return to the step of obtaining a web page based on the content extraction address and execute again until the current value of the web page index reaches the web page index upper limit value.

[0018] Furthermore, extracting a target variable from the webpage based on the content parsing expression includes:

[0019] Parsing the variable extraction method and variable identification in the content parsing expression;

[0020] The target variable is obtained from the webpage based on the variable extraction method and the variable identifier.

[0021] Furthermore, obtaining the target variable from the webpage based on the variable extraction method and the variable identifier includes:

[0022] Calling a corresponding expression parsing plug-in based on the variable extraction method, and transmitting the variable identifier to the expression parsing plug-in;

[0023] The expression parsing plug-in obtains the target variable from the web page based on the variable identifier.

[0024] Furthermore, the method further comprises:

[0025] A corresponding expression parsing plug-in is pre-set for the content parsing expression; the expression parsing plug-in is used to extract the target variable configured in the content parsing expression from the corresponding web page.

[0026] Furthermore, the method further comprises:

[0027] A corresponding expression processing plug-in is pre-set for the content processing expression; the expression processing plug-in is used to perform corresponding processing on the target variable according to the parameters configured by the content processing expression.

[0028] Furthermore, the method further comprises:

[0029] Display candidate process components on the flowchart creation page;

[0030] After detecting a drag operation on the candidate process component, generating and displaying an instance of the dragged process component in the main editing area of the flowchart creation page;

[0031] In response to an edit operation on the instance of the process component displayed in the main editing area, obtaining edit content corresponding to the edit operation;

[0032] Configuration content is generated based on the edited content, and the configuration content is associated with the instance of the process component and stored in the flowchart configuration information corresponding to the content extraction flowchart.

[0033] Furthermore, the method further comprises:

[0034] In response to an association operation for a front-to-back execution order of the two instances of the process component displayed in the main editing area, a front-to-back execution order between the two instances of the process component is established based on an association direction specified by the association operation;

[0035] The sequential execution association between the two instances of the process components is stored in the flow chart configuration information corresponding to the content extraction flow chart.

[0036] In a second aspect, an embodiment of the present disclosure provides a content extraction device, including:

[0037] A first acquisition module is configured to acquire a content extraction flowchart; the content extraction flowchart includes a plurality of flow components connected in sequence, and the plurality of flow components are used to configure the flow chart configuration information;

[0038] A first parsing module is configured to parse the various process components and sequence relationships in the content extraction flow chart to obtain flow chart configuration information; the flow chart configuration information includes a content extraction address and a content parsing expression;

[0039] An extraction module configured to obtain a web page based on the content extraction address;

[0040] a second parsing module, configured to extract a target variable from the webpage based on the content parsing expression;

[0041] The storage module is configured to store the target variable.

[0042] Furthermore, the flowchart configuration information further includes a content processing expression; and the apparatus further includes:

[0043] The first processing module is configured to process the target variable based on the content processing expression and output a processing result of the target variable.

[0044] Furthermore, the flowchart configuration information further includes a content loop expression, wherein the content loop expression is configured with at least an initial value of a webpage index and an upper limit value of a webpage index; before the storage module, the device further includes:

[0045] An adding module configured to increase the initial value of the webpage index in the content loop expression by a current value of the webpage index of a set value;

[0046] A second acquisition module is configured to obtain a new content extraction address based on the current value of the webpage index and the content extraction address;

[0047] The return module is configured to return to the extraction module for re-execution until the current value of the webpage index reaches the upper limit value of the webpage index.

[0048] Furthermore, the extraction module includes:

[0049] a parsing submodule configured to parse the variable extraction method and variable identification in the content parsing expression;

[0050] The first acquisition submodule is configured to acquire the target variable from the webpage based on the variable extraction method and the variable identifier.

[0051] Furthermore, the first acquisition submodule includes:

[0052] a calling submodule configured to call a corresponding expression parsing plug-in based on the variable extraction method and transmit the variable identifier to the expression parsing plug-in;

[0053] The second acquisition submodule is configured to acquire the target variable from the web page based on the variable identifier by the expression parsing plug-in.

[0054] Furthermore, the device further comprises:

[0055] The first setting module is configured to pre-set a corresponding expression parsing plug-in for a content parsing expression; the expression parsing plug-in is used to extract a target variable configured in the content parsing expression from a corresponding web page.

[0056] Furthermore, the device further comprises:

[0057] The second setting module is configured to pre-set a corresponding expression processing plug-in for the content processing expression; the expression processing plug-in is used to perform corresponding processing on the target variable according to the parameters configured by the content processing expression.

[0058] Furthermore, the device further comprises:

[0059] A display module is configured to display candidate process components on a flowchart making page;

[0060] a detection module configured to, upon detecting a drag operation on the candidate process component, generate and display an instance of the dragged process component in the main editing area of the flowchart creation page;

[0061] a response module configured to, in response to an edit operation on the instance of the process component displayed in the main editing area, obtain edit content corresponding to the edit operation;

[0062] The generating module is configured to generate configuration content based on the edited content, and associate the configuration content with the instance of the process component and store it in the flowchart configuration information corresponding to the content extraction flowchart.

[0063] Furthermore, the device further comprises:

[0064] an establishing module configured to, in response to an association operation of a front-to-back execution order being performed on the two instances of the process component displayed in the main editing area, establish a front-to-back execution order between the two instances of the process component based on an association direction specified by the association operation;

[0065] The association module is configured to associate the execution order between the two instances of the process components and store it in the flowchart configuration information corresponding to the content extraction flowchart.

[0066] The functions can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions.

[0067] In one possible design, the apparatus includes a memory and a processor. The memory is configured to store one or more computer instructions that enable the apparatus to perform the corresponding method, and the processor is configured to execute the computer instructions stored in the memory. The apparatus may also include a communication interface for communicating with other devices or a communication network.

[0068] In a third aspect, embodiments of the present disclosure provide an electronic device, comprising a memory and a processor, wherein the memory is configured to store one or more computer instructions that support any of the aforementioned devices in executing the corresponding methods described above, and the processor is configured to execute the computer instructions stored in the memory. Any of the aforementioned devices may also include a communication interface for communicating with other devices or a communication network.

[0069] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium for storing computer instructions used by any of the above-mentioned devices, which includes computer instructions involved in executing any of the above-mentioned methods.

[0070] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, which includes computer instructions, and when the computer instructions are executed by a processor, they are used to implement the steps of the method described in any of the above aspects.

[0071] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0072] The disclosed embodiment proposes a content extraction method, in which relevant personnel can pre-configure a content extraction flowchart, configure multiple process components for extracting content in the content extraction flowchart, and connect the multiple process components in sequence. The configuration content of each process component in the content extraction flowchart and the sequential relationship between the process components are parsed to obtain flowchart configuration information, and the flowchart configuration information includes at least a content extraction address and a content parsing expression; the web page of the content to be extracted can be obtained through the content extraction address, and then the target variable in the web page can be obtained through the content parsing expression. The target variable is stored in a database as the content to be extracted. Finally, after the above process is performed on all web pages that need to extract content, the required content can be obtained from the database. In this approach, personnel don't need to program to extract content from web pages. Instead, they configure a content extraction flowchart, assigning the content extraction address of the web page to be extracted, the content parsing expression of the target element to be extracted from the web page, and other information to the various process components of the content extraction flowchart. These components are then connected in the corresponding order to form a content extraction flowchart. A pre-implemented general-purpose parsing engine in the backend then parses the content extraction flowchart, automatically extracting the target variable from the web page and storing it in the database. This approach reduces labor and time costs and improves content extraction efficiency.

[0073] It should be understood that the foregoing general description and the following detailed description are merely exemplary and explanatory and are not restrictive of the embodiments of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Other features, objectives and advantages of the embodiments of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0075] Figure 1 A flowchart of a content extraction method according to an embodiment of the present disclosure is shown.

[0076] Figure 2 A schematic diagram showing the effect of a content extraction flow chart of a process component configured with a content loop expression according to an embodiment of the present disclosure is shown.

[0077] Figure 3 A schematic diagram showing the effect of calling an engine plug-in according to an embodiment of the present disclosure is shown.

[0078] Figure 4 A structural block diagram of a content extraction device according to an embodiment of the present disclosure is shown.

[0079] Figure 5 An electronic device provided by an embodiment of the present disclosure is shown.

[0080] Figure 6 It is a structural diagram of a computer system suitable for implementing the content extraction method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0081] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0082] In the embodiments of the present disclosure, it should be understood that terms such as "including" or "having" are intended to indicate the existence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in this specification, and are not intended to exclude the possibility of one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof existing or being added.

[0083] It should also be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present disclosure can be combined with each other. The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0084] Figure 1 A flow chart of a content extraction method according to an embodiment of the present disclosure is shown as follows: Figure 1 As shown, the content extraction method includes the following steps:

[0085] In step S101, a content extraction flowchart is obtained; the content extraction flowchart includes a plurality of flow components connected in sequence, and the plurality of flow components are used to configure flow chart configuration information;

[0086] In step S102, each process component and sequence relationship in the content extraction flow chart is parsed to obtain flow chart configuration information; the flow chart configuration information includes a content extraction address and a content parsing expression;

[0087] In step S103, a web page is obtained based on the content extraction address;

[0088] In step S104, a target variable is extracted from the web page based on the content parsing expression;

[0089] In step S105, the target variable is stored.

[0090] As mentioned above, currently, a lot of business information is usually carried in the form of HTML web pages, and related businesses need to extract corresponding content from these web pages, and process the extracted content according to business needs before displaying or storing it.

[0091] Typically, automated content extraction from web pages requires developers to implement content scraping scripts and execute them to extract the required content. However, business personnel are often familiar with business requirements but lack programming skills. Extracting the required content from web pages requires the assistance of additional programmers, requiring collaboration between business personnel and programmers, resulting in high labor and time costs.

[0092] Therefore, an embodiment of the present disclosure proposes a content extraction method, in which relevant personnel can pre-configure a content extraction flowchart, in which multiple process components for extracting content are configured, the multiple process components are connected in sequence, and the configuration content in each process component forms configuration flowchart configuration information; the configuration content of each process component in the content extraction flowchart and the sequential relationship between each process component are parsed to obtain flowchart configuration information, and the flowchart configuration information at least includes a content extraction address and a content parsing expression; the web page of the content to be extracted can be obtained through the content extraction address, and then the target variable in the web page can be obtained through the content parsing expression, and the target variable is stored in the database as the content to be extracted. Finally, after the above process is performed on all web pages that need to extract content, the required content can be obtained from the database. In this approach, personnel don't need to program to extract content from web pages. Instead, they configure a content extraction flowchart, assigning the content extraction address of the web page to be extracted, the content parsing expression of the target element to be extracted from the web page, and other information to the various process components of the content extraction flowchart. These components are then connected in the corresponding order to form a content extraction flowchart. A pre-implemented general-purpose parsing engine in the backend then parses the content extraction flowchart, automatically extracting the target variable from the web page and storing it in the database. This approach reduces labor and time costs and improves content extraction efficiency.

[0093] In one embodiment of the present disclosure, the flowchart configuration information further includes a content processing expression, and the method further includes the following steps:

[0094] The target variable is processed based on the content processing expression, and the processing result of the target variable is output.

[0095] In this optional implementation, the content extraction flow chart may also include a process component for configuring a content processing expression. After configuring the content parsing expression and parsing the corresponding target variable from the web page based on the content parsing expression, the process component for configuring the content processing expression may process the target variable based on the configuration content in the content processing expression to obtain a processing result. The configuration content of the content processing expression may configure how to process the target variable. For example, if the target variable is a string, it may be configured to intercept the nth to mth characters in the target variable, and the processing result obtained is the nmth character string in the target variable. For example, the content processing expression may be configured as "${substring(var,0,10)}", where var represents the target variable and the expression represents intercepting the 0th to 10th character string from var.

[0096] The processing result of the target variable can be displayed through the terminal or output through other required channels. Of course, in some embodiments, the target variable and the processing result of the target variable can be associated and stored in a database for subsequent use according to business needs.

[0097] In one embodiment of the present disclosure, the flowchart configuration information further includes a content loop expression, wherein the content loop expression is configured with at least an initial value of a webpage index and an upper limit value of a webpage index; before step S105, i.e., the step of storing the target variable, the method further includes the following steps:

[0098] Increasing the initial value of the webpage index in the content loop expression by a set value to obtain the current value of the webpage index;

[0099] Obtaining a new content extraction address based on the current value of the webpage index and the content extraction address;

[0100] Return to the step of obtaining a web page based on the content extraction address and execute again until the current value of the web page index reaches the web page index upper limit value.

[0101] In this optional implementation, a content loop extraction of multiple web pages can also be implemented. A process component of a content loop expression can be configured in the content extraction flow chart. The content loop expression can be configured with at least a web index initial value and a web index upper limit value. After obtaining the first web page using the content extraction address and extracting the target variable from it, the current web index value can be obtained based on the web index initial value and the set value in the content loop expression. That is, the set value, such as 1, is added to the initial value of the web index value to obtain the current web index value. Based on the current web index value and the content extraction address of the previous web page, a new content extraction address can be obtained. Based on the new content extraction address, a new web page can be obtained. For the new web page, the target variable can be extracted from it by continuing to use the content parsing expression. In this way, the target variable on multiple web pages can be obtained. The web index upper limit value configured in the content loop expression can be used to limit the maximum number of web pages. When the web index current value reaches the web index upper limit value, the loop can be exited, thereby storing the target variables extracted from all pages and / or storing or outputting the target variables after performing corresponding processing on the target variables.

[0102] Figure 2 Schematic diagram showing the effect of a content extraction flow chart of a process component configured with a content loop expression according to an embodiment of the present disclosure. Figure 2As shown, the first process component in the content extraction flow chart is the web crawling component, which can be used to configure the content extraction address, such as "http: / / www.url.com?page=${index}", which can be the address of the first web page, where ${index} is an expression. After passing through the page loop process component, it can be dynamically increased to achieve content extraction of multiple dynamic web pages. The next process component of the content extraction component is the variable parsing component, which is used to configure the content parsing expression, such as "${page.selector('.main')}" or

[0103] "${page.xpath('#mainId')}"; Based on these two content parsing expressions, it can be determined that the target variable to be extracted is the target variable with the selector value "main" or the target variable with the xpath value "mainId". The next process component after the variable parsing component is the page loop component, which is used to configure the content loop expression, including the initial and upper limits of the webpage index.

[0104] The initial web page index value can be the same as the address of the first web page. The next process components after the page loop component include the web page extraction component and the variable output component. While the loop is still in progress, execution can proceed to the web page extraction component, and after the loop is complete, execution can proceed to the variable output component. The variable output component is used to configure content processing expressions. The next component after the variable output component is the variable storage component, which stores the target variable and / or the processing results of the target variable.

[0105] In one embodiment of the present disclosure, step S104, i.e., the step of extracting the target variable from the webpage based on the content parsing expression, further includes the following steps:

[0106] Parsing the variable extraction method and variable identification in the content parsing expression;

[0107] The target variable is obtained from the webpage based on the variable extraction method and the variable identifier.

[0108] In this optional implementation, the variable extraction method and variable identification can be configured in the content parsing expression. The variable extraction method is used to configure the method for obtaining the target variable to be extracted from the web page, and the variable identification is used to configure the reference information of the target variable to be extracted from the web page.

[0109] In some embodiments, the variable extraction method may be based on the language in which the webpage is written, and this is not specifically limited herein. The variable identifier may be based on the variable extraction method and the language in which the webpage is written, and any method may be applicable as long as the configured variable extraction method can extract the target variable indicated by the variable identifier from the webpage.

[0110] The following uses the content of an HTML web page as an example to illustrate how to configure content parsing expressions:

[0111] If the selector content extraction method is used, the target variable is parsed and extracted according to the CSS of the HTML page tag. If the CSS style is main, the content parsing expression to extract the target variable can be configured as: ${page.selector('.main')}. If the xpath content extraction method is used, if the id corresponding to the target variable tag is mainId, the content parsing expression to extract the target variable can be configured as: ${page.xpath('#mainId')}.

[0112] In one embodiment of the present disclosure, the step of obtaining the target variable from the webpage based on the variable extraction method and the variable identifier further includes the following steps:

[0113] Calling a corresponding expression parsing plug-in based on the variable extraction method, and transmitting the variable identifier to the expression parsing plug-in;

[0114] The expression parsing plug-in obtains the target variable from the web page based on the variable identifier.

[0115] In this optional implementation, an expression parsing plug-in can be pre-programmed for all available content parsing expressions. After the variable extraction method and variable identifier are extracted based on the content parsing expression, the corresponding expression parsing plug-in can be determined and called based on the variable extraction method, and the extracted variable identifier is transmitted to the expression parsing plug-in, so that the expression parsing plug-in uses the variable identifier as an input parameter to execute the corresponding content extraction process. During this content extraction process, the expression parsing plug-in extracts the target variable corresponding to the variable identifier from the web page.

[0116] In one embodiment of the present disclosure, the method further comprises the following steps:

[0117] A corresponding expression parsing plug-in is pre-set for the content parsing expression; the expression parsing plug-in is used to extract the target variable configured in the content parsing expression from the corresponding web page.

[0118] In this optional implementation, an expression parsing plug-in can be pre-set for the content parsing expression. The expression parsing plug-in can realize the function of extracting the target variable from the web page and is a general engine plug-in. After the content parsing expression is configured in the content extraction flow chart, the expression type and expression parameters are extracted from the content parsing expression. The expression type is used to indicate what type of expression the content parsing expression is configured for, and the expression parameters are used to indicate the variable identifier of the target variable. Based on the expression type, the corresponding expression parsing plug-in can be determined. By calling and executing the corresponding expression parsing plug-in, the corresponding content can be extracted from the web page. When calling the corresponding expression parsing plug-in, the extracted expression parameters can be transmitted to the expression parsing plug-in. After the expression parsing plug-in receives the expression parameters, it can determine the target variable to be extracted from the web page.

[0119] In one embodiment of the present disclosure, the method further comprises the following steps:

[0120] A corresponding expression processing plug-in is pre-set for the content processing expression; the expression processing plug-in is used to perform corresponding processing on the target variable according to the parameters configured by the content processing expression.

[0121] In this optional implementation, an expression processing plug-in can be set for the content processing expression. The expression processing plug-in can realize the function of performing corresponding processing on the target variable extracted from the web page, and is a universal engine plug-in. Different processing methods can correspond to different expression processing plug-ins. After the content processing expression is configured in the content extraction flow chart, the expression type and expression parameters are extracted from the content processing expression. The expression type is used to indicate what type of processing method the content processing expression is configured with, and the expression parameters are used to indicate the parameters to be used when processing the target variable. For example, in the content processing expression of intercepting characters, the expression type can be the intercepted character type, and the expression parameters are the number of digits of the intercepted characters. Based on the expression type, the corresponding expression processing plug-in can be determined. By calling and executing the corresponding expression processing plug-in, the corresponding processing of the target variable is realized. When calling the corresponding expression processing plug-in, the extracted expression parameters can be transmitted to the expression processing plug-in. After the expression processing plug-in receives the expression parameters, it can determine how to perform specific processing on the target variable.

[0122] Figure 3 Schematic diagram showing the effect of calling the engine plug-in according to an embodiment of the present disclosure. Figure 3As shown, the expression language parser may include a plug-in loader and a plug-in executor. The plug-in loader can load multiple plug-ins, and the plug-in executor can extract expressions and expression parameters from the configuration content of the process component. After determining the plug-in to be called based on the extracted expression, the plug-in loader loads the corresponding plug-in and transmits the extracted expression parameters to the plug-in. The plug-in executes the corresponding processing flow based on the expression parameters, thereby realizing the processing function corresponding to the expression.

[0123] In one embodiment of the present disclosure, the method further comprises the following steps:

[0124] Display candidate process components on the flowchart creation page;

[0125] After detecting a drag operation on the candidate process component, generating and displaying an instance of the dragged process component in the main editing area of the flowchart creation page;

[0126] In response to an edit operation on the instance of the process component displayed in the main editing area, obtaining edit content corresponding to the edit operation;

[0127] Configuration content is generated based on the edited content, and the configuration content is associated with the instance of the process component and stored in the flowchart configuration information corresponding to the content extraction flowchart.

[0128] In this optional implementation, a human-computer interaction interface can be provided, through which relevant personnel can open a flowchart production page, and all or some types of candidate process components can be displayed on the flowchart production page. Different types of process components can be used to configure different types of content, such as content extraction addresses, content parsing expressions, content processing expressions, etc. In some embodiments, different processing methods can correspond to the same type of process components, or to different types of process components. When corresponding to the same type of process components, different processing methods can be obtained by parsing the content processing expression. When corresponding to different types of process components, different processing methods can be determined by the type of process component.

[0129] When creating a content extraction flowchart, relevant personnel can drag candidate process components into the main editing area of the flowchart creation page. After the human-computer interaction interface detects the drag action, an instance of the corresponding candidate process component is generated and displayed at the corresponding position in the main editing area.

[0130] When relevant personnel edit the process components generated and displayed in the main editing area, the editing content of the relevant personnel can be obtained, and the configuration content of the process component can be generated based on the editing content. The configuration content is associated with the instance of the process component and stored in the flowchart configuration information corresponding to the content extraction flowchart.

[0131] After completing the editing operation of the entire content extraction flowchart, the final flowchart configuration information can be obtained. Then, the corresponding content, such as the target variable, can be obtained from the corresponding web page based on the flowchart configuration information.

[0132] In one embodiment of the present disclosure, the method further comprises the following steps:

[0133] In response to an association operation for a front-to-back execution order of the two instances of the process component displayed in the main editing area, a front-to-back execution order between the two instances of the process component is established based on an association direction specified by the association operation;

[0134] The sequential execution association between the two instances of the process components is stored in the flow chart configuration information corresponding to the content extraction flow chart.

[0135] In this optional implementation, when creating a content extraction flowchart, the relevant personnel can also specify the forward and backward execution order between any two process components. The relevant personnel can specify the forward and backward execution order between the two process components by connecting the two process components and specifying the direction of the connection line, or they can specify the forward and backward execution order between the two process components by other means. After the relevant personnel detects an association operation on the human-computer interaction interface in which they specify the forward and backward execution order between the two process components, the forward and backward execution order between the two process components can be established based on the association direction specified in the association operation. This forward and backward execution order can also be stored in the flowchart configuration information corresponding to the content extraction flowchart. After the content extraction flowchart is finally obtained, when the flowchart configuration information corresponding to the content extraction flowchart is processed, the forward and backward execution order relationship can be used to determine which process component to parse first and which process component to parse later. In some embodiments, this forward and backward execution order is not only the forward and backward execution order of the parsed process components, but also the forward and backward execution order of the processing flows corresponding to each process component. The processing flows corresponding to each process component may include processes such as the web page acquisition process and the call of corresponding plug-ins to implement corresponding functions based on the parsed configuration content. For details, please refer to the description above and will not be repeated here.

[0136] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein.

[0137] Figure 4FIG1 shows a structural block diagram of a content extraction device according to an embodiment of the present disclosure. The device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 4 As shown, the content extraction device includes:

[0138] A first acquisition module 401 is configured to acquire a content extraction flowchart; the content extraction flowchart includes a plurality of process components connected in sequence, and the plurality of process components are used to configure flowchart configuration information;

[0139] A first parsing module 402 is configured to parse the various process components and sequence relationships in the content extraction flow chart to obtain flow chart configuration information; the flow chart configuration information includes a content extraction address and a content parsing expression;

[0140] Extraction module 403, configured to obtain a web page based on the content extraction address;

[0141] A second parsing module 404 is configured to extract a target variable from the webpage based on the content parsing expression;

[0142] The storage module 405 is configured to store the target variable.

[0143] As mentioned above, currently, a lot of business information is usually carried in the form of HTML web pages, and related businesses need to extract corresponding content from these web pages, and process the extracted content according to business needs before displaying or storing it.

[0144] Typically, automated content extraction from web pages requires developers to implement content scraping scripts and execute them to extract the required content. However, business personnel are often familiar with business requirements but lack programming skills. Extracting the required content from web pages requires the assistance of additional programmers, requiring collaboration between business personnel and programmers, resulting in high labor and time costs.

[0145] Therefore, an embodiment of the present disclosure proposes a content extraction device, in which relevant personnel can pre-configure a content extraction flowchart, in which multiple process components for extracting content are configured, and the multiple process components are connected in sequence, and the configuration content in each process component forms configuration flowchart configuration information; the configuration content of each process component in the content extraction flowchart and the sequential relationship between each process component are parsed to obtain flowchart configuration information, and the flowchart configuration information at least includes a content extraction address and a content parsing expression; the web page of the content to be extracted can be obtained through the content extraction address, and then the target variable in the web page can be obtained through the content parsing expression, and the target variable is stored in the database as the content to be extracted. Finally, after the above process is performed on all web pages that need to extract content, the required content can be obtained from the database. In this approach, personnel don't need to program to extract content from web pages. Instead, they configure a content extraction flowchart, assigning the content extraction address of the web page to be extracted, the content parsing expression of the target element to be extracted from the web page, and other information to the various process components of the content extraction flowchart. These components are then connected in the corresponding order to form a content extraction flowchart. A pre-implemented general-purpose parsing engine in the backend then parses the content extraction flowchart, automatically extracting the target variable from the web page and storing it in the database. This approach reduces labor and time costs and improves content extraction efficiency.

[0146] In one embodiment of the present disclosure, the flowchart configuration information further includes a content processing expression; and the apparatus further includes:

[0147] The first processing module is configured to process the target variable based on the content processing expression and output a processing result of the target variable.

[0148] In this optional implementation, the content extraction flow chart may also include a process component for configuring a content processing expression. After configuring the content parsing expression and parsing the corresponding target variable from the web page based on the content parsing expression, the process component for configuring the content processing expression may process the target variable based on the configuration content in the content processing expression to obtain a processing result. The configuration content of the content processing expression may configure how to process the target variable. For example, if the target variable is a string, it may be configured to intercept the nth to mth characters in the target variable, and the processing result obtained is the nmth character string in the target variable. For example, the content processing expression may be configured as "${substring(var,0,10)}", where var represents the target variable and the expression represents intercepting the 0th to 10th character string from var.

[0149] The processing result of the target variable can be displayed through the terminal or output through other required channels. Of course, in some embodiments, the target variable and the processing result of the target variable can be associated and stored in a database for subsequent use according to business needs.

[0150] In one embodiment of the present disclosure, the flowchart configuration information further includes a content loop expression, wherein the content loop expression is configured with at least an initial value of a webpage index and an upper limit value of a webpage index; before the storage module, the device further includes:

[0151] An adding module configured to increase the initial value of the webpage index in the content loop expression by a current value of the webpage index of a set value;

[0152] A second acquisition module is configured to obtain a new content extraction address based on the current value of the webpage index and the content extraction address;

[0153] The return module is configured to return to the extraction module for re-execution until the current value of the webpage index reaches the upper limit value of the webpage index.

[0154] In this optional implementation, a content loop extraction of multiple web pages can also be implemented. A process component of a content loop expression can be configured in the content extraction flow chart. The content loop expression can be configured with at least a web index initial value and a web index upper limit value. After obtaining the first web page using the content extraction address and extracting the target variable from it, the current web index value can be obtained based on the web index initial value and the set value in the content loop expression. That is, the set value, such as 1, is added to the initial value of the web index value to obtain the current web index value. Based on the current web index value and the content extraction address of the previous web page, a new content extraction address can be obtained. Based on the new content extraction address, a new web page can be obtained. For the new web page, the target variable can be extracted from it by continuing to use the content parsing expression. In this way, the target variable on multiple web pages can be obtained. The web index upper limit value configured in the content loop expression can be used to limit the maximum number of web pages. When the web index current value reaches the web index upper limit value, the loop can be exited, thereby storing the target variables extracted from all pages and / or storing or outputting the target variables after performing corresponding processing on the target variables.

[0155] Figure 2 Schematic diagram showing the effect of a content extraction flow chart of a process component configured with a content loop expression according to an embodiment of the present disclosure. Figure 2As shown, the first process component in the content extraction flow chart is the web crawling component, which can be used to configure the content extraction address, such as "http: / / www.url.com?page=${index}", which can be the address of the first web page, where ${index} is an expression. After passing through the page loop process component, it can be dynamically increased to achieve content extraction of multiple dynamic web pages. The next process component of the content extraction component is the variable parsing component, which is used to configure the content parsing expression, such as "${page.selector('.main')}" or

[0156] "${page.xpath('#mainId')}"; Based on these two content parsing expressions, it can be determined that the target variable to be extracted is the target variable with the selector value "main" or the target variable with the xpath value "mainId". The next process component after the variable parsing component is the page loop component, which is used to configure the content loop expression, including the initial and upper limits of the webpage index.

[0157] The initial web page index value can be the same as the address of the first web page. The next process components after the page loop component include the web page extraction component and the variable output component. While the loop is still in progress, execution can proceed to the web page extraction component, and after the loop is complete, execution can proceed to the variable output component. The variable output component is used to configure content processing expressions. The next component after the variable output component is the variable storage component, which stores the target variable and / or the processing results of the target variable.

[0158] In one embodiment of the present disclosure, the extraction module includes:

[0159] a parsing submodule configured to parse the variable extraction method and variable identification in the content parsing expression;

[0160] The first acquisition submodule is configured to acquire the target variable from the webpage based on the variable extraction method and the variable identifier.

[0161] In this optional implementation, the variable extraction method and variable identification can be configured in the content parsing expression. The variable extraction method is used to configure the method for obtaining the target variable to be extracted from the web page, and the variable identification is used to configure the reference information of the target variable to be extracted from the web page.

[0162] In some embodiments, the variable extraction method may be based on the language in which the webpage is written, and this is not specifically limited herein. The variable identifier may be based on the variable extraction method and the language in which the webpage is written, and any method may be applicable as long as the configured variable extraction method can extract the target variable indicated by the variable identifier from the webpage.

[0163] The following uses the content of an HTML web page as an example to illustrate how to configure content parsing expressions:

[0164] If the selector content extraction method is used, the target variable is parsed and extracted according to the CSS of the HTML page tag. If the CSS style is main, the content parsing expression to extract the target variable can be configured as: ${page.selector('.main')}. If the xpath content extraction method is used, if the id corresponding to the target variable tag is mainId, the content parsing expression to extract the target variable can be configured as: ${page.xpath('#mainId')}.

[0165] In one embodiment of the present disclosure, the first acquisition submodule includes:

[0166] a calling submodule configured to call a corresponding expression parsing plug-in based on the variable extraction method and transmit the variable identifier to the expression parsing plug-in;

[0167] The second acquisition submodule is configured to acquire the target variable from the web page based on the variable identifier by the expression parsing plug-in.

[0168] In this optional implementation, an expression parsing plug-in can be pre-programmed for all available content parsing expressions. After the variable extraction method and variable identifier are extracted based on the content parsing expression, the corresponding expression parsing plug-in can be determined and called based on the variable extraction method, and the extracted variable identifier is transmitted to the expression parsing plug-in, so that the expression parsing plug-in uses the variable identifier as an input parameter to execute the corresponding content extraction process. During this content extraction process, the expression parsing plug-in extracts the target variable corresponding to the variable identifier from the web page.

[0169] In one embodiment of the present disclosure, the device further comprises:

[0170] The first setting module is configured to pre-set a corresponding expression parsing plug-in for a content parsing expression; the expression parsing plug-in is used to extract a target variable configured in the content parsing expression from a corresponding web page.

[0171] In this optional implementation, an expression parsing plug-in can be pre-set for the content parsing expression. The expression parsing plug-in can realize the function of extracting the target variable from the web page and is a general engine plug-in. After the content parsing expression is configured in the content extraction flow chart, the expression type and expression parameters are extracted from the content parsing expression. The expression type is used to indicate what type of expression the content parsing expression is configured for, and the expression parameters are used to indicate the variable identifier of the target variable. Based on the expression type, the corresponding expression parsing plug-in can be determined. By calling and executing the corresponding expression parsing plug-in, the corresponding content can be extracted from the web page. When calling the corresponding expression parsing plug-in, the extracted expression parameters can be transmitted to the expression parsing plug-in. After the expression parsing plug-in receives the expression parameters, it can determine the target variable to be extracted from the web page.

[0172] In one embodiment of the present disclosure, the device further comprises:

[0173] The second setting module is configured to pre-set a corresponding expression processing plug-in for the content processing expression; the expression processing plug-in is used to perform corresponding processing on the target variable according to the parameters configured by the content processing expression.

[0174] In this optional implementation, an expression processing plug-in can be set for the content processing expression. The expression processing plug-in can realize the function of performing corresponding processing on the target variable extracted from the web page, and is a universal engine plug-in. Different processing methods can correspond to different expression processing plug-ins. After the content processing expression is configured in the content extraction flow chart, the expression type and expression parameters are extracted from the content processing expression. The expression type is used to indicate what type of processing method the content processing expression is configured with, and the expression parameters are used to indicate the parameters to be used when processing the target variable. For example, in the content processing expression of intercepting characters, the expression type can be the intercepted character type, and the expression parameters are the number of digits of the intercepted characters. Based on the expression type, the corresponding expression processing plug-in can be determined. By calling and executing the corresponding expression processing plug-in, the corresponding processing of the target variable is realized. When calling the corresponding expression processing plug-in, the extracted expression parameters can be transmitted to the expression processing plug-in. After the expression processing plug-in receives the expression parameters, it can determine how to perform specific processing on the target variable.

[0175] Figure 3 A schematic diagram showing the effect of calling an engine plug-in according to an embodiment of the present disclosure is shown as follows: Figure 3As shown, the expression language parser may include a plug-in loader and a plug-in executor. The plug-in loader can load multiple plug-ins, and the plug-in executor can extract expressions and expression parameters from the configuration content of the process component. After determining the plug-in to be called based on the extracted expression, the plug-in loader loads the corresponding plug-in and transmits the extracted expression parameters to the plug-in. The plug-in executes the corresponding processing flow based on the expression parameters, thereby realizing the processing function corresponding to the expression.

[0176] In one embodiment of the present disclosure, the device further comprises:

[0177] A display module is configured to display candidate process components on a flowchart making page;

[0178] a detection module configured to, upon detecting a drag operation on the candidate process component, generate and display an instance of the dragged process component in the main editing area of the flowchart creation page;

[0179] a response module configured to, in response to an edit operation on the instance of the process component displayed in the main editing area, obtain edit content corresponding to the edit operation;

[0180] The generating module is configured to generate configuration content based on the edited content, and associate the configuration content with the instance of the process component and store it in the flowchart configuration information corresponding to the content extraction flowchart.

[0181] In this optional implementation, a human-computer interaction interface can be provided, through which relevant personnel can open a flowchart production page, and all or some types of candidate process components can be displayed on the flowchart production page. Different types of process components can be used to configure different types of content, such as content extraction addresses, content parsing expressions, content processing expressions, etc. In some embodiments, different processing methods can correspond to the same type of process components, or to different types of process components. When corresponding to the same type of process components, different processing methods can be obtained by parsing the content processing expression. When corresponding to different types of process components, different processing methods can be determined by the type of process component.

[0182] When creating a content extraction flowchart, relevant personnel can drag candidate process components into the main editing area of the flowchart creation page. After the human-computer interaction interface detects the drag action, an instance of the corresponding candidate process component is generated and displayed at the corresponding position in the main editing area.

[0183] When relevant personnel edit the process components generated and displayed in the main editing area, the editing content of the relevant personnel can be obtained, and the configuration content of the process component can be generated based on the editing content. The configuration content is associated with the instance of the process component and stored in the flowchart configuration information corresponding to the content extraction flowchart.

[0184] After completing the editing operation of the entire content extraction flowchart, the final flowchart configuration information can be obtained. Then, the corresponding content, such as the target variable, can be obtained from the corresponding web page based on the flowchart configuration information.

[0185] In one embodiment of the present disclosure, the device further comprises:

[0186] an establishing module configured to, in response to an association operation of a front-to-back execution order being performed on the two instances of the process component displayed in the main editing area, establish a front-to-back execution order between the two instances of the process component based on an association direction specified by the association operation;

[0187] The association module is configured to associate the execution order between the two instances of the process components and store it in the flowchart configuration information corresponding to the content extraction flowchart.

[0188] In this optional implementation, when creating a content extraction flowchart, the relevant personnel can also specify the forward and backward execution order between any two process components. The relevant personnel can specify the forward and backward execution order between the two process components by connecting the two process components and specifying the direction of the connection line, or they can specify the forward and backward execution order between the two process components by other means. After the relevant personnel detects an association operation on the human-computer interaction interface in which they specify the forward and backward execution order between the two process components, the forward and backward execution order between the two process components can be established based on the association direction specified in the association operation. This forward and backward execution order can also be stored in the flowchart configuration information corresponding to the content extraction flowchart. After the content extraction flowchart is finally obtained, when the flowchart configuration information corresponding to the content extraction flowchart is processed, the forward and backward execution order relationship can be used to determine which process component to parse first and which process component to parse later. In some embodiments, this forward and backward execution order is not only the forward and backward execution order of the parsed process components, but also the forward and backward execution order of the processing flows corresponding to each process component. The processing flows corresponding to each process component may include processes such as the web page acquisition process and the call of corresponding plug-ins to implement corresponding functions based on the parsed configuration content. For details, please refer to the description above and will not be repeated here.

[0189] Figure 5 An electronic device provided by an embodiment of the present disclosure is shown. Figure 5As shown, the electronic device 500 includes a memory 501 and a processor 502; wherein,

[0190] The memory 501 is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 502 to implement any of the above method steps.

[0191] Figure 6 It is a structural diagram of a computer system suitable for implementing the content extraction method according to an embodiment of the present disclosure.

[0192] like Figure 6 As shown, the computer system 600 includes a processing unit 601, which can execute various processes in the above-mentioned embodiments according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the computer system 600 are also stored in the RAM 603. The processing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0193] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed so that a computer program read therefrom can be installed into the storage section 608 as needed. Among them, the processing unit 601 can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, or an NPU.

[0194] In particular, according to embodiments of the present disclosure, the methods described above can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a computer-readable medium, the computer program comprising program code for executing the data transmission method. In such embodiments, the computer program can be downloaded and installed from a network via the communication portion 609 and / or installed from the removable medium 611.

[0195] The embodiments of the present disclosure further disclose a computer program product, which includes a computer program / instruction, and implements any of the above method steps when the computer program / instruction is executed by a processor.

[0196] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the diagram or block diagram can represent a module, program segment or part of the code, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, as well as the combination of boxes in the block diagram and / or flow chart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0197] The units or modules described in the embodiments of the present disclosure may be implemented in software or hardware. The units or modules described may also be provided in a processor, and the names of these units or modules do not, in certain circumstances, limit the units or modules themselves.

[0198] As another aspect, embodiments of the present disclosure further provide a computer-readable storage medium. This computer-readable storage medium may be included in the apparatus described in the above embodiments, or may be a standalone computer-readable storage medium not incorporated into the apparatus. The computer-readable storage medium stores one or more programs, which are used by one or more processors to execute the methods described in the embodiments of the present disclosure.

[0199] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also encompass other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A content extraction method, comprising: Get the content extraction flow chart; The content extraction flow chart includes a plurality of flow components connected in sequence, and the plurality of flow components are used to configure flow chart configuration information; Parsing each process component and sequence relationship in the content extraction flow chart to obtain flow chart configuration information; the flow chart configuration information includes a content extraction address, a content parsing expression, and a content processing expression; Acquire a web page based on the content extraction address; Parsing the variable extraction method and variable identification in the content parsing expression; Calling a corresponding expression parsing plug-in based on the variable extraction method, and transmitting the variable identifier to the expression parsing plug-in, so that the expression parsing plug-in uses the variable identifier as an input parameter to perform a corresponding content extraction process, during which the expression parsing plug-in extracts a target variable corresponding to the variable identifier from the web page; storing the target variable; Extracting an expression type and expression parameters from a content processing expression; the content processing expression includes an expression type and expression parameters, the expression type being used to indicate what type of processing mode the content processing expression configures, and the expression parameters being used to indicate parameters to be used when processing a target variable; The corresponding expression processing plug-in is determined based on the expression type, and the expression processing plug-in is called and executed. After the expression processing plug-in determines the corresponding processing method for the target variable based on the expression parameters, the corresponding processing of the target variable is implemented.

2. The method according to claim 1, wherein The flow chart configuration information further includes a content loop expression, wherein the content loop expression is configured with at least an initial value of a webpage index and an upper limit value of a webpage index; Before storing the target variable, the method further includes: Increasing the initial value of the webpage index in the content loop expression by a set value to obtain the current value of the webpage index; Obtaining a new content extraction address based on the current value of the webpage index and the content extraction address; Return to the step of obtaining a web page based on the content extraction address and re-execute until the current value of the web page index reaches the web page index upper limit.

3. The method according to any one of claims 1 to 2, wherein: The method further comprises: A corresponding expression parsing plug-in is pre-set for the content parsing expression; the expression parsing plug-in is used to extract the target variable configured in the content parsing expression from the corresponding web page.

4. The method according to any one of claims 1 to 2, wherein: The method further comprises: A corresponding expression processing plug-in is pre-set for the content processing expression; the expression processing plug-in is used to perform corresponding processing on the target variable according to the parameters configured by the content processing expression.

5. The method according to any one of claims 1 to 2, wherein: The method further comprises: Display candidate process components on the flowchart creation page; After detecting a drag operation on the candidate process component, generating and displaying an instance of the dragged process component in the main editing area of the flowchart creation page; In response to an edit operation on the instance of the process component displayed in the main editing area, obtaining edit content corresponding to the edit operation; Configuration content is generated based on the edited content, and the configuration content is associated with the instance of the process component and stored in the flowchart configuration information corresponding to the content extraction flowchart.

6. The method according to claim 5, wherein: The method further comprises: In response to an association operation for a front-to-back execution order of the two instances of the process component displayed in the main editing area, a front-to-back execution order between the two instances of the process component is established based on an association direction specified by the association operation; The sequential execution association between the two instances of the process components is stored in the flow chart configuration information corresponding to the content extraction flow chart.

7. A content extraction device comprising: A first acquisition module is configured to acquire a content extraction flow chart; The content extraction flow chart includes a plurality of flow components connected in sequence, and the plurality of flow components are used to configure flow chart configuration information; A first parsing module is configured to parse the various process components and sequence relationships in the content extraction flow chart to obtain flow chart configuration information; the flow chart configuration information includes a content extraction address, a content parsing expression, and a content processing expression; An extraction module configured to obtain a web page based on the content extraction address; a second parsing module configured to parse a variable extraction method and a variable identifier in the content parsing expression, call a corresponding expression parsing plug-in based on the variable extraction method, and transmit the variable identifier to the expression parsing plug-in so that the expression parsing plug-in uses the variable identifier as an input parameter to perform a corresponding content extraction process, wherein the expression parsing plug-in extracts a target variable corresponding to the variable identifier from the web page during the content extraction process; a storage module, configured to store the target variable; The variable processing module is configured to extract an expression type and expression parameters from a content processing expression; the content processing expression includes an expression type and expression parameters, the expression type is used to indicate what type of processing method the content processing expression is configured for, and the expression parameters are used to indicate the parameters to be used when processing the target variable, and to determine the corresponding expression processing plug-in based on the expression type, call and execute the expression processing plug-in, and after the expression processing plug-in determines the corresponding processing method for the target variable based on the expression parameters, implement the corresponding processing of the target variable.

8. The device according to claim 7, wherein The flow chart configuration information further includes a content loop expression, wherein the content loop expression is configured with at least an initial value of a webpage index and an upper limit value of a webpage index; Before the storage module, the device further includes: An adding module configured to increase the initial value of the webpage index in the content loop expression by a current value of the webpage index of a set value; A second acquisition module is configured to obtain a new content extraction address based on the current value of the webpage index and the content extraction address; The return module is configured to return to the extraction module for re-execution until the current value of the webpage index reaches the upper limit value of the webpage index.

9. The device according to any one of claims 7 to 8, wherein: The device further comprises: The first setting module is configured to pre-set a corresponding expression parsing plug-in for a content parsing expression; the expression parsing plug-in is used to extract a target variable configured in the content parsing expression from a corresponding web page.

10. The device according to any one of claims 7 to 8, wherein: The device further comprises: The second setting module is configured to pre-set a corresponding expression processing plug-in for the content processing expression; the expression processing plug-in is used to perform corresponding processing on the target variable according to the parameters configured by the content processing expression.

11. The device according to any one of claims 7 to 8, wherein: The device further comprises: A display module is configured to display candidate process components on a flowchart making page; a detection module configured to, upon detecting a drag operation on the candidate process component, generate and display an instance of the dragged process component in the main editing area of the flowchart creation page; a response module configured to, in response to an edit operation on the instance of the process component displayed in the main editing area, obtain edit content corresponding to the edit operation; The generating module is configured to generate configuration content based on the edited content, and associate the configuration content with the instance of the process component and store it in the flowchart configuration information corresponding to the content extraction flowchart.

12. The device according to claim 11, wherein The device further comprises: an establishing module configured to, in response to an association operation of a front-to-back execution order being performed on the two instances of the process component displayed in the main editing area, establish a front-to-back execution order between the two instances of the process component based on an association direction specified by the association operation; The association module is configured to associate the execution order between the two instances of the process components and store it in the flowchart configuration information corresponding to the content extraction flowchart.

13. An electronic device comprising a memory and a processor; wherein: The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the steps of the method according to any one of claims 1 to 6.

14. A computer-readable storage medium having computer instructions stored thereon, wherein: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

15. A computer program product comprising a computer program / instruction, which implements the steps of the method according to any one of claims 1 to 6 when executed by a processor.

Images