A semantic-based open domain webpage knowledge extraction method and system

Abstract

The application provides a semantic-based open domain webpage knowledge extraction method, which comprises the following steps: obtaining a skeleton tree of an open domain webpage, splitting a skeleton node of the skeleton tree to obtain skeleton sub-nodes of the skeleton node, and generating a skeleton sub-node sequence; labeling a classification label for the skeleton sub-nodes and the skeleton node, performing relation extraction on the skeleton tree according to the classification label, obtaining a relation sub-node sequence of an extraction task, and generating a relation fragment; performing object extraction on the skeleton tree based on the relation fragment, taking the extracted skeleton sub-node sequence as an object fragment; and taking the relation fragment and the corresponding object fragment as an extraction result of the extraction task. The application also provides a semantic-based open domain webpage knowledge extraction system and a data processing device for open domain webpage knowledge extraction.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of network information technology, and particularly relates to an open-domain webpage knowledge extraction method and system. BACKGROUND

[0002] Knowledge Extraction (KE) is mainly to extract facts from natural language texts, and the facts are represented in the form of triples of <subject, relation, object>. Knowledge extraction is an indispensable part of knowledge graph construction, and is widely used in question-answering systems and automatic abstracts. Taking knowledge graph construction as an example, most of the current entity and relation extraction methods are designed for free texts, and cannot well handle semi-structured web pages. There are a large amount of knowledge in semi-structured web pages that do not exist in free texts, especially the knowledge of long-tail entities. Extracting such knowledge can enrich and supplement the knowledge graph of long-tail entities. Web Knowledge Extraction (WKE) is mainly to extract triples of <subject, relation, object> from detail pages. According to whether the relation type appears in the corresponding knowledge graph, web knowledge extraction is divided into closed-domain web knowledge extraction (Closed IE) and open-domain web knowledge extraction (Open IE). The relation type of open-domain web knowledge extraction is not defined, and the input is a detail page of a given subject, and the output is all <relation, object> pairs corresponding to the subject.

[0003] Web pages have three forms of representation: serialized HTML source code, networked DOM tree, and image form. These three forms of representation provide three different features: text features, structural features, and visual features. Existing methods are based on or combine these three features for web knowledge extraction.

[0004] Methods based on text features often annotate the HTML original sequence or the pure text after deleting the mark symbols, and use natural language processing models for free texts for extraction. However, the loss of layout information is a fatal shortcoming of such methods: in the DOM tree form, the web page has semantic meaning about the organization of the content, and most natural language processing models only process one-dimensional input, thus losing the layout information of the web page. At the same time, in the HTML source code form of the web page, the page contains markup language, and the sentences of the web page are usually longer than the sentences of natural language, and the sentences are truncated by the markup, resulting in incoherent semantics, which hinders the use of pre-trained language models. Thus, the knowledge extraction methods designed for free texts cannot be well utilized in semi-structured web pages.

[0005] The method based on structural features performs feature statistics and classification on nodes on the DOM tree, or generates an extraction template for extraction by inducing structural features of similar web pages. However, this method needs to manually label a large number of features, and is limited to websites with similar structural types. In addition, this method also has the problem of coarse extraction granularity, and cannot distinguish different semantics in a single node.

[0006] The method based on visual features loses semantic information during extraction. Such methods often need to convert the web page into an image and introduce an additional OCR step, which introduces errors. And the web page usually exceeds the maximum input length of the model, and cutting the web page snapshot may cause the <relationship, object> pair to be separated into multiple images. In addition, such methods assume that the web page writer will concentrate similar semantic content in the same area, and thus block according to the separation degree between elements in the web page, rather than according to the semantic closeness.

[0007] Based on a single feature representation, it is difficult to perform good extraction, so it is necessary to combine the above three feature representations for extraction. Such methods generally perform text representation, layout representation and visual representation on DOM nodes, concatenate the three representations for node classification, and still cannot perform fine-grained representation and fine-grained extraction on node semantics.

[0008] Based on a single feature representation, it is difficult to perform good extraction, so it is necessary to combine the above three feature representations for extraction. Such methods generally perform text representation, layout representation and visual representation on DOM nodes, concatenate the three representations for node classification, and still cannot perform fine-grained representation and fine-grained extraction on node semantics. SUMMARY

[0009] To solve the above problems, the application provides an open domain webpage knowledge extraction method, which comprises the following steps: obtaining a skeleton tree of an open domain webpage, splitting a skeleton node of the skeleton tree to obtain skeleton sub-nodes of the skeleton node, and generating a skeleton sub-node sequence; labeling a classification label for the skeleton sub-nodes and the skeleton node, performing relation extraction on the skeleton tree according to the classification label, obtaining a relation sub-node sequence of an extraction task, and generating a relation fragment; performing object extraction on the skeleton tree based on the relation fragment, taking the extracted skeleton sub-node sequence as an object fragment; and taking the relation fragment and the corresponding object fragment as an extraction result of the extraction task.

[0010] In the open domain webpage knowledge extraction method, the step of labeling the classification label for the skeleton sub-nodes comprises labeling the skeleton sub-nodes as a start word of a relation fragment, a non-start word of the relation fragment, or a non-relation fragment word by using a labeling model; and the step of labeling the classification label for the skeleton node comprises using a voting method to aggregate classification results of the skeleton sub-nodes of each skeleton node, and selecting a classification label with the largest number as the classification label of the skeleton node.

[0011] In the open domain webpage knowledge extraction method, the labeling model uses a long text pre-training model as an encoding layer and a feedforward neural network as an output layer.

[0012] In the open domain webpage knowledge extraction method, the step of performing the relation extraction further comprises the following steps: for any layer of skeleton node, finding a nearest common parent node of all skeleton nodes in the layer, selecting a skeleton node with the same relation fragment path type as the skeleton node as an expansion node of the skeleton node from a subtree with the nearest common parent node as the root, and taking a relation fragment of the expansion node as a relation fragment of the skeleton node.

[0013] In the open domain webpage knowledge extraction method, the step of performing the object extraction comprises the following steps: using an extraction model to perform question and answer type object extraction by taking the relation fragment as a question and the object fragment as an answer, and predicting a start position and an end position of the object fragment in the skeleton sub-node sequence.

[0014] In the open domain webpage knowledge extraction method, the extraction model uses a long text pre-training model as an encoding layer and a linear layer as an output layer; when the extraction model is pre-trained, a cross-entropy loss function is used to calculate a loss of a start position vector and a loss of an end position vector, and an average value of the loss of the start position vector and the loss of the end position vector is taken as a loss of the extraction model; and when the object extraction is performed, a position with the highest score in the start position vector is taken as the start position, and a position with the highest score in the end position vector is taken as the end position.

[0015] The open-domain webpage knowledge extraction method of the present invention includes obtaining the open-domain webpage W. i The DOM tree, through preprocessing, makes all the skeleton nodes of the DOM tree... in All text is generated, W i skeleton tree S i ; for s in Perform word-level splitting to obtain S i skeleton child nodes W i The nth skeleton node s in The m-th skeleton child node, m∈[1,M] i ], n∈[1, N i ], M i s in The number of skeleton child nodes, N i S represents i The number of skeleton nodes.

[0016] This invention also proposes a semantic-based open-domain webpage knowledge extraction system, comprising: a skeleton tree generation module, used to obtain the skeleton tree of an open-domain webpage, split the skeleton nodes of the skeleton tree to obtain the skeleton child nodes of the skeleton nodes, and generate a sequence of skeleton child nodes; a relation extraction module, used to label the skeleton child nodes and the skeleton nodes with classification tags, extract relations from the skeleton tree according to the classification tags, obtain the sequence of relation child nodes for the extraction task, and generate relation fragments; an object extraction module, used to extract objects from the skeleton tree based on the relation fragments, and use the extracted sequence of skeleton child nodes as object fragments; and a result generation module, used to use the relation fragments and their corresponding object fragments as the extraction result of the extraction task.

[0017] The present invention also proposes a computer-readable storage medium storing computer-executable instructions, characterized in that, when the computer-executable instructions are executed, semantic-based open-domain web page knowledge extraction as described above is realized.

[0018] The present invention also proposes a data processing apparatus, including the computer-readable storage medium as described above, wherein when the processor of the data processing apparatus retrieves and executes the computer-executable instructions in the computer-readable storage medium, it realizes semantic-based open domain web page knowledge extraction. Attached Figure Description

[0019] Figure 1A This is a diagram of an open-domain encyclopedia webpage.

[0020] Figure 1B yes Figure 1A A diagram showing the skeleton tree segment corresponding to the encyclopedia page.

[0021] Figure 2 This is a schematic diagram of the overall structure of the OpenSCIEnt model of this invention.

[0022] Figure 3 This is a schematic diagram of a webpage representation based on a skeleton tree according to the present invention.

[0023] Figure 4 This is a schematic diagram of the self-attention-based relation extraction of the present invention.

[0024] Figure 5 This is a schematic diagram illustrating the expanded position of the relational nodes in this invention.

[0025] Figure 6 This is a schematic diagram of the relation-driven question-and-answer object extraction method of the present invention.

[0026] Figure 7 This is a flowchart of the semantic-based open-domain webpage knowledge extraction method of the present invention.

[0027] Figure 8 This is a schematic diagram of the data processing device of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0029] The purpose of this invention is to solve the problems of high computational overhead and low extraction accuracy in the process of obtaining location information from web page rendering in the existing technology, and to propose a semantic-based open domain web page knowledge extraction method and system.

[0030] While conducting research on knowledge extraction from open-domain web pages, the inventors discovered that semi-structured web pages contain rich, multi-dimensional information, including not only text and images but also information related to style, layout, and structure. For example... Figure 1A As shown, at Figure 1AIn the encyclopedia webpages shown, the parts marked with red boxes represent the relational fragments of the webpage's main body, including "Chinese name" and "foreign name." The parts marked with blue boxes represent the object fragments corresponding to these relational predicates, including "Zhang San" and "ZhangSan." It can be seen that there are clear layout and positional relationships between relational predicates, between objects, and between relations and objects. Existing technologies utilize these positional relationships by directly taking screenshots of the webpage to obtain a snapshot, thus visually obtaining the coordinates of these elements. This method aligns well with human visual perception during modeling, but it has several problems: firstly, rendering images, locating text fragments, and obtaining coordinates is time-consuming; secondly, text fragment positioning errors result in inaccurate positional relationship features initially input into the model, leading to further accumulation of computational errors.

[0031] Through research, the inventors discovered that web pages exist not only in image form but also in serialized HTML source code form, networked DOM tree form, and skeleton tree representation. Analysis revealed that these positional relationships cannot be represented in one-dimensional HTML source code but can be shown in the parsed DOM tree. Furthermore, the skeleton tree preserves this positional relationship information while simplifying the DOM tree and representing the semantics of the web page. Figure 1B Showing Figure 1A The skeleton tree fragment corresponding to the encyclopedia page, such as Figure 1B As shown, the paths of the nodes containing relational predicates exhibit high similarity, and there is also a contextual order relationship between the relational predicates and their corresponding object nodes. Due to the similarity between the paths of relational predicates, when some relational predicates are known, the remaining relational predicates can be searched based on the known relational predicate paths. Because of the contextual order relationship between relational predicates and objects, when relational predicates are known, they can be used as clues for object extraction, assisting in object location and extraction. Furthermore, while existing research on open-domain knowledge extraction methods for natural language text is relatively mature, these methods cannot be directly transferred to semi-structured web pages. However, skeleton trees can extract web page text, forming unstructured text representations, thus allowing the application of unstructured text processing techniques on skeleton trees.

[0032] Considering timeliness and accuracy, this method does not render webpage images and obtain location features. Instead, it extracts relevant location structure information from the webpage's skeleton tree representation without parsing or processing the images. Based on the path similarity of relational predicates, it searches for the remaining relational predicates through known paths. Given the contextual order relationship between relational predicates and objects, a two-stage extraction method is employed. The first extracted relational predicates serve as clues for object extraction, assisting in object localization and extraction.

[0033] The knowledge extraction method of this invention first employs a skeleton tree to represent open-domain web pages at the lexical level to improve extraction accuracy. During the extraction process, it performs relation extraction based on self-attention and object extraction driven by relations in a question-and-answer format. In this way, by modeling and representing the semantics of the web page text, the accuracy of relation extraction is improved. At the same time, relation fragments are abstracted into questions, and the start and end positions of the corresponding objects in the sequence of child nodes of the web page skeleton are predicted using relation fragments as the driving force, thereby improving the object extraction accuracy. During the relation extraction process, the extracted relations are expanded through position expansion to improve the recall rate of relation extraction and recall more relation predicates.

[0034] Based on the above, this invention proposes a semantic-based open-domain webpage knowledge extraction model, OpenSCIEnt. The OpenSCIEnt model uses a skeleton tree to represent webpages and, based on this representation, designs a two-stage model for webpage knowledge extraction. The first stage model extracts relations based on a self-attention mechanism and expands relations based on the positional information of known relations. The second stage model uses a question-and-answer approach, extracting corresponding objects by using input relation predicates as prompts. The method proposed in this invention has high accuracy and can identify most relational object fragments covering webpages.

[0035] Figure 2 This is a schematic diagram of the overall structure of the OpenSCIEnt model of this invention. (See diagram below.) Figure 2 As shown, the OpenSCIEnt model of this invention consists of three parts: 1) web page representation based on skeleton tree, 2) relation extraction based on self-attention and position expansion, and 3) relation-driven question-and-answer object extraction.

[0036] (1) Web page representation based on skeleton tree

[0037] like Figure 3 As shown, after pruning the DOM tree so that all leaf nodes are text, a webpage skeleton tree is constructed, and the skeleton nodes are finely split by word. The resulting webpage skeleton representation is as follows. in, This indicates that the webpage W i The m-th skeleton child node of the n-th skeleton node, where n∈[1, N] i The granularity of the skeleton child nodes is at the word level, containing text sequences. and the corresponding path sequence

[0038] (2) Relation extraction based on self-attention and position expansion

[0039] Tag Meaning R-B Beginning word of a relation fragment R-I Non-beginning word of a relation fragment O Non-relation fragment word

[0040] Table 1

[0041] Relation extraction is abstracted into a sequence labeling task. The input is a sequence of child nodes representing the skeleton of a webpage, and each child node is labeled with a word-level classification. Each child node has three label types, and the meanings of the labels are shown in Table 1.

[0042] like Figure 4 As shown, in the relation extraction stage, based on the self-attention mechanism, the long text pre-trained model BigBird is used as the encoding layer to encode the skeleton child node sequence, followed by a feedforward neural network to classify and label each skeleton child node.

[0043] The relation extraction granularity is at the skeleton node level, therefore it is necessary to map the classification results of the skeleton child nodes back to the skeleton nodes, such as... Figure 4 As shown in the child node classification result mapping section, a voting method is used. For each skeleton node, the classification results of its skeleton child nodes are summarized, and the category label with the most votes is selected as the label for that skeleton node. Finally, relationship fragments are extracted based on the label category.

[0044] Considering the possibility of limited training data, if the test set contains many relation fragments not present in the training set, the extraction results may not cover these fragments. Observation reveals a high degree of similarity in the paths among the relation nodes within the main body of the webpage. Therefore, the extracted relation fragments are then positionally expanded to enrich the relation network. For example... Figure 5 As shown, summarize all the relation nodes obtained in the previous stage, find the nearest common parent node of these relation nodes, and select the skeleton nodes with the same path type as the relation fragment from the subtree rooted at that parent node as the expanded relation fragment.

[0045] (3) Relationship-driven question-and-answer object extraction

[0046] Since there is a contextual order relationship between relational predicates and their corresponding object nodes, object extraction is driven by relations. Object extraction is abstracted into a relation-driven question-answering task. After obtaining relational fragments, the relational fragments are abstracted into questions, the webpage skeleton is abstracted into articles, and the object fragments are abstracted into answers. Using relational fragments as the driver, the start and end positions of the corresponding objects in the sequence of child nodes of the webpage skeleton are predicted.

[0047] like Figure 6 As shown, the sequence of relation nodes and the sequence of skeleton child nodes are concatenated as the model input, denoted as [S]. ir ,S i The model uses the pre-trained language model BigBird as the encoding layer to encode the input sequence, which is then fed into a linear layer to predict whether each skeleton child node has a start or end position.

[0048] During model training, the cross-entropy loss function is used to calculate the loss of the start position vector and the loss of the end position vector, respectively, and the average of the two is taken as the final loss. During model prediction, the position with the highest score in the start position vector is selected as the start position of the object fragment, and the same applies to the end position. Furthermore, since the object extraction granularity is at the skeleton node level, similar to relation extraction, the predicted start and end positions are mapped back to skeleton nodes, and the sequence of skeleton nodes between the start and end positions is extracted as the final object fragment.

[0049] After completing the above work, the relation fragments and their corresponding object fragments are taken as the final result of the knowledge extraction task from open domain web pages.

[0050] Figure 7 This is a flowchart of the semantic-based open-domain webpage knowledge extraction method of the present invention. Figure 7 As shown, the open-domain webpage knowledge extraction method of the present invention includes:

[0051] Step S1: Obtain the DOM tree of the open domain webpage. Preprocess the DOM tree so that all skeleton nodes are text. The preprocessing can be pruning or other methods to generate the skeleton tree of the open domain webpage from the preprocessed DOM tree. Split the skeleton nodes of the skeleton tree at the word level to obtain the skeleton child nodes of the skeleton tree, and further generate the skeleton child node sequence.

[0052] Specifically, this includes: obtaining open domain web pages W i The DOM tree, through pruning, makes all skeleton nodes of the DOM tree... in All text is generated to create an open domain webpage W. i skeleton tree S i For skeleton node s in Perform word-level splitting to obtain skeleton child nodes. The skeleton tree can then be represented as:

[0053]

[0054] This indicates an open domain webpage W i The nth skeleton node s in The m-th skeleton child node, m∈[1,M] i ], n∈[1, N i ], M i Represents the skeleton node s in The number of skeleton child nodes, N i Represents the skeleton tree S i The number of skeleton nodes;

[0055] Step S2, for the skeleton tree S iAll skeleton child nodes and skeleton nodes are labeled with classification tags, and the skeleton tree is subjected to relation extraction based on the classification tags of skeleton child nodes and skeleton nodes to obtain the relation child node sequence of the extraction task and generate relation fragments.

[0056] In this process, the skeleton child nodes are labeled as the start words of a relation segment, or the non-start words of a relation segment, or the words of a non-relation segment using a labeling model. After labeling all skeleton child nodes, the classification results of each skeleton node are summarized by voting, and the category label with the most votes is selected as the classification label of the skeleton node.

[0057] The annotation model is based on the self-attention mechanism. It uses the long text pre-trained model BigBird as the encoding layer to encode the skeleton sub-node sequence, followed by a feedforward neural network to classify and label each skeleton sub-node.

[0058] Considering the possibility of limited training data, if the test set contains numerous relation fragments not present in the training set, the extraction results may not cover these fragments. Observation reveals a high degree of path similarity among relation nodes within the main body of the webpage. This invention expands the relationships by positionally extending the extracted relation fragments. Specifically, it summarizes all relation nodes obtained in the previous stage, finds their nearest common parent node, and selects skeleton nodes with the same path type as the relation fragments from the subtree rooted at the nearest common parent node, using these as the expanded relation fragments.

[0059] Step S3: Extract objects from the skeleton tree in a question-and-answer manner based on relation fragments, and use the extracted skeleton child node sequence as object fragments.

[0060] Because there is a contextual order relationship between relational predicates and their corresponding object nodes, object extraction is driven by relations. Object extraction is abstracted as a relation-driven question-answering task. After obtaining relational fragments, these fragments are abstracted into questions, the webpage skeleton into articles, and object fragments into answers. Using relational fragments as the driver, the start and end positions of the corresponding objects in the webpage skeleton's child node sequence are predicted. Specifically, the relational node sequence and the skeleton child node sequence are concatenated as model input. The extraction model uses a pre-trained language model, BigBird, as the encoding layer to encode the input sequence, which is then fed into a linear layer to predict the start and end positions of each skeleton child node.

[0061] During model training, the cross-entropy loss function is used to calculate the loss of the start position vector and the loss of the end position vector, respectively, and the average of the two is taken as the final loss. During model prediction, the position with the highest score in the start position vector is selected as the start position of the object fragment, and the same applies to the end position. Furthermore, since the granularity of object extraction is at the skeleton node level, similar to relation extraction, the predicted start and end positions are mapped back to skeleton nodes, and the sequence of skeleton nodes between the start and end positions is extracted as the final object fragment.

[0062] Step S4: The object fragments of the relation fragments obtained in step S2 and the corresponding relation fragments obtained in step S3 are used as the extraction results of the extraction task.

[0063] Figure 8 This is a schematic diagram of the data processing apparatus of the present invention. Figure 8 As shown, embodiments of the present invention also provide a computer-readable storage medium and a data processing apparatus. The computer-readable storage medium of the present invention stores computer-executable instructions. When these computer-executable instructions are executed by the processor of the data processing apparatus, the aforementioned knowledge extraction from open domain web pages is achieved. Those skilled in the art will understand that all or part of the steps in the above methods can be implemented by a program instructing related hardware (e.g., processor, FPGA, ASIC, etc.), and the program can be stored in a readable storage medium, such as a read-only memory, a disk, or an optical disk. All or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module in the above embodiments can be implemented in hardware, for example, by implementing its corresponding function through an integrated circuit, or it can be implemented in the form of a software functional module, for example, by a processor executing a program / instruction stored in memory to implement its corresponding function. Embodiments of the present invention are not limited to any particular combination of hardware and software.

[0064] Compared with existing knowledge extraction methods, this method has higher extraction accuracy, more stable extraction results, and can better adapt to changes in webpage structure.

[0065] The above embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all equivalent technical solutions also fall within the scope of the present invention, and the patent protection scope of the present invention should be defined by the claims.

Claims

1. A semantic-based open domain web page knowledge extraction method, characterized in that, The method comprises the following steps: obtaining a skeleton tree of an open domain webpage, splitting a skeleton node of the skeleton tree to obtain skeleton sub-nodes of the skeleton node, and generating a skeleton sub-node sequence; annotating a classification label for the skeleton sub-nodes and the skeleton node, and performing relation extraction on the skeleton tree according to the classification label, including: for any layer of skeleton node, finding the nearest common parent node of all skeleton nodes in the layer, selecting a skeleton node with the same relationship fragment path type as the skeleton node as an expansion node of the skeleton node from the subtree with the nearest common parent node as the root, and taking the relationship fragment of the expansion node as the relationship fragment of the skeleton node; performing object extraction on the skeleton tree based on the relationship fragment, taking the extracted skeleton sub-node sequence as an object fragment; taking the relationship fragment and the corresponding object fragment as the extraction result of the extraction task.

2. The open domain web page knowledge extraction method of claim 1, wherein, The step of annotating a classification label for the skeleton sub-nodes comprises: using an annotation model to annotate the skeleton sub-nodes as a start word of a relationship fragment, a non-start word of a relationship fragment, or a non-relationship fragment word; The step of annotating a classification label for the skeleton node comprises: using a voting method to aggregate the classification results of the skeleton sub-nodes of each skeleton node, and selecting the classification label with the largest number as the classification label of the skeleton node.

3. The open domain web page knowledge extraction method of claim 2, wherein, The annotation model uses a long text pre-training model as an encoding layer and a feedforward neural network as an output layer.

4. The open domain web page knowledge extraction method of claim 1, wherein, The step of performing the object extraction comprises: using an extraction model to perform question and answer type object extraction by taking the relationship fragment as a question and the object fragment as an answer, and predicting the start position and end position of the object fragment in the skeleton sub-node sequence.

5. The open domain web page knowledge extraction method of claim 4, wherein, The extraction model uses a long text pre-training model as an encoding layer and a linear layer as an output layer; When pre-training the extraction model, a cross-entropy loss function is used to calculate the loss of the start position vector and the loss of the end position vector, and the average of the loss of the start position vector and the loss of the end position vector is taken as the loss of the extraction model; When performing the object extraction, the position with the highest score in the start position vector is taken as the start position, and the position with the highest score in the end position vector is taken as the end position.

6. The open domain web page knowledge extraction method of claim 1, wherein, Retrieve the open domain webpage W i The DOM tree, through preprocessing, makes all the skeleton nodes of the DOM tree... All text is generated, W i Skeleton tree S i ; right Perform word-level splitting to obtain S i skeleton child nodes , , W i The nth skeleton node The m-th skeleton child node, m∈[1,M] i ], n∈[1, N i ], M i express The number of skeleton child nodes, N i S represents i The number of skeleton nodes.

7. A semantic-based open domain web page knowledge extraction system, which employs the open domain web page knowledge extraction method according to any one of claims 1-6 to realize semantic-based open domain web page knowledge extraction, characterized in that, The open domain webpage knowledge extraction system comprises: a skeleton tree generation module, configured to obtain a skeleton tree of an open domain webpage, split a skeleton node of the skeleton tree to obtain skeleton sub-nodes of the skeleton node, and generate a skeleton sub-node sequence; a relation extraction module, configured to annotate a classification label for the skeleton sub-nodes and the skeleton node, and perform relation extraction on the skeleton tree according to the classification label, including: for any layer of skeleton node, finding the nearest common parent node of all skeleton nodes in the layer, selecting a skeleton node with the same relationship fragment path type as the skeleton node as an expansion node of the skeleton node from the subtree with the nearest common parent node as the root, and taking the relationship fragment of the expansion node as the relationship fragment of the skeleton node; an object extraction module, configured to perform object extraction on the skeleton tree based on the relationship fragment, taking the extracted skeleton sub-node sequence as an object fragment; and taking the relationship fragment and the corresponding object fragment as the extraction result of the extraction task. A result generation module is configured to generate the extraction result of the extraction task by using the relationship fragment and the corresponding object fragment.

8. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, cause the processor to perform operations comprising: When the computer executable instructions are executed, the computer executable instructions realize the semantic-based open domain webpage knowledge extraction as claimed in any one of claims 1-6.

9. A data processing apparatus comprising the computer readable storage medium as claimed in claim 8, when a processor of the data processing apparatus invokes and executes the computer executable instructions in the computer readable storage medium, the semantic-based open domain webpage knowledge extraction is realized.

Images