Internet online common sense extraction method and device, electronic equipment and storage medium

Abstract

The invention provides an internet online common sense extraction method and device, electronic equipment and a storage medium, multiple co-reference sets and entities existing in each co-reference set are obtained by carrying out co-reference resolution and entity recognition on sub-documents of an internet document, and a co-reference sparse matrix corresponding to each co-reference set of each sub-document is constructed; the method comprises the following steps: acquiring a plurality of sub-documents, calculating a total co-reference score of two entity mentions in the sub-documents, calculating a total global co-reference score of any two entity mentions in the internet document based on the total co-reference score of the two entity mentions in each sub-document, and determining a global co-reference set; the entity mentions in the corresponding global co-reference sets are replaced on the basis of the entity names of the entities in the global co-reference sets, updated documents are obtained, entity relation extraction is conducted on the updated documents, triples of the internet documents are obtained, the knowledge base is updated on the basis of the triples of the internet documents, and the knowledge base is updated. The entity relationship extraction capability of the long text is improved, and the knowledge extraction precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method, device, electronic device and storage medium for extracting online common sense from the Internet. Background Art

[0002] Automatically extracting common sense knowledge from the Internet and building a knowledge base is one of the important tasks in the field of artificial intelligence. The core of this process involves identifying key entities and their relationships from unstructured text, thereby generating a triple consisting of a head entity, an entity relationship, and a tail entity. Among them, in the initial stage of knowledge extraction, entity recognition is used to locate and classify key terms in the text, such as names of people, places, or organizations. This task requires the model to understand the context and distinguish which phrases are real entities. Next, relationship extraction technology is used to identify the associations between these entities, such as "someone founded a certain company" or "something happened in a certain place." These structured data provide an important foundation for question-answering systems, information retrieval, and decision support.

[0003] However, there are a lot of long texts in Internet documents, and long texts bring many challenges to current natural language processing tasks. In short texts, it is relatively easy to identify entities and relations because the information is relatively concentrated and the connection between sentences is relatively simple. However, for long texts, the problem becomes much more complicated. The content of long texts often consists of multiple topics, and its description often uses pronouns or synonymous phrases to repeatedly mention the same entity, resulting in some key information that may span multiple paragraphs and be relatively scattered. The current model lacks good long-distance dependency modeling capabilities, which makes it impossible to associate semantic information that is far apart, affecting the accuracy and completeness of knowledge extraction. Summary of the invention

[0004] The present invention provides an Internet online common sense extraction method, device, electronic device and storage medium, which are used to solve the defect of poor accuracy in knowledge extraction of long texts in the prior art.

[0005] The present invention provides an Internet online common sense extraction method, comprising: Segment the Internet document to obtain multiple sub-documents; Perform coreference resolution and entity recognition on any sub-document to obtain multiple coreference sets and entities in each coreference set, and construct a coreference sparse matrix corresponding to each coreference set of any sub-document; wherein the elements in the coreference sparse matrix corresponding to any coreference set represent the coreference scores of two entity mentions in any sub-document relative to any coreference set; Based on the coreference sparse matrix corresponding to each coreference set of any sub-document, calculating the coreference total score of two entity mentions in any sub-document, calculating the global coreference total score of any two entity mentions in the Internet document based on the coreference total score of the two entity mentions in each sub-document, and determining the global coreference set based on the global coreference total score of any two entity mentions in the Internet document; Based on the entity names of the entities in each global coreference set, entity mentions in the corresponding global coreference set are replaced to obtain an updated document, and entity relations are extracted from the updated document to obtain triples of the Internet document, and a knowledge base is updated based on the triples of the Internet document.

[0006] According to a method for extracting online common sense from the Internet provided by the present invention, the total coreference score of any two entity mentions in the Internet document is calculated based on the total coreference score of the two entity mentions in each sub-document, including: Constructing a non-coreference sparse matrix corresponding to each coreference set of each sub-document, and calculating the total non-coreference score of the two entity mentions in each sub-document based on the non-coreference sparse matrix corresponding to each coreference set of each sub-document; wherein the elements in the non-coreference sparse matrix corresponding to any coreference set of any sub-document represent the non-coreference score of the two entity mentions in any sub-document relative to the any coreference set; Based on the total coreference scores and the total non-coreference scores of the two entity mentions in each sub-document, a global coreference score of any two entity mentions in the Internet document is calculated.

[0007] According to a method for extracting online common sense from the Internet provided by the present invention, the total coreference score and the total non-coreference score of two entity mentions in each sub-document are calculated to calculate the total global coreference score of any two entity mentions in the Internet document, including: Obtain any two entity mentions belonging to the same sub-document, calculate the global coreference score of the any two entity mentions based on the coreference score and the non-coreference score of the any two entity mentions in each sub-document, and add the any two entity mentions to the mention pair set in the form of a mention pair; For any first entity mention and second entity mention that do not belong to the same sub-document, the intermediate mention is determined based on the global coreference score of the mention pairs in the mention pair set that includes the first entity mention, and the global coreference score of the mention pairs in the mention pair set that includes the second entity mention, and the global coreference score of the first entity mention and the second entity mention is determined based on the global coreference score of the first entity mention and the intermediate mention and the global coreference score of the intermediate mention and the second entity mention; wherein the intermediate mention satisfies the condition that the product of the global coreference score of the first entity mention and the intermediate mention and the global coreference score of the intermediate mention and the second entity mention is the largest.

[0008] According to a method for extracting online common sense from the Internet provided by the present invention, determining a global coreference set based on the total global coreference score of any two entity mentions in the Internet document comprises: Determining a judgment threshold based on the total global coreference score of any two entity mentions in the Internet document; Based on the total global coreference score of any two entity mentions in the Internet document and the judgment threshold, constructing an entity mention graph; wherein a node in the entity mention graph corresponds to any entity mention, and if the total global coreference score of any two entity mentions is higher than the judgment threshold, then there is a connecting edge between the nodes corresponding to the any two entity mentions; A node vector representation of each node in the entity mention graph is obtained based on a graph convolutional neural network, and the global coreference set is determined based on the similarity between the node vector representations of each node in the entity mention graph.

[0009] According to a method for extracting online common sense from the Internet provided by the present invention, extracting entity relationships from the update document includes: splicing the relationship names of the preset relationships to the end of the update document to obtain a network input text; Encoding the network input text based on a coding network to obtain entity representations of each entity in the network input text and relationship representations of each preset relationship; the coding network is constructed based on a Bert model; Constructing a relational representation matrix based on the relational representations of each preset relation; For any head entity and any tail entity, determine the attention score of any head entity and the attention score of any tail entity calculated by each self-attention module of the last Transformer layer in the encoding network; Determine an entity attention score based on the attention score of any head entity and the attention score of any tail entity calculated by each self-attention module; Determine the product of the entity attention score and the relationship representation matrix as the relationship attention representation matrix corresponding to any head entity and any tail entity; Based on the relationship attention representation matrix corresponding to any head entity and any tail entity and the entity representation of any head entity and any tail entity, a classifier is used to determine the relationship type between any head entity and any tail entity.

[0010] According to a method for extracting online common sense from the Internet provided by the present invention, the method uses a classifier to determine the relationship type between any head entity and any tail entity based on the relationship attention representation matrix corresponding to any head entity and any tail entity and the entity representation of any head entity and any tail entity, including: The relation attention representation matrices corresponding to any head entity and any tail entity are respectively fused with the entity representation of any head entity and the entity representation of any tail entity to obtain the relation fused entity representation of any head entity and the relation fused entity representation of any tail entity; Based on the classifier, relationship classification is performed on the relationship fusion entity representation of any head entity and the relationship fusion entity representation of any tail entity to obtain the relationship type between any head entity and any tail entity.

[0011] According to an Internet online common sense extraction method provided by the present invention, based on the attention score of any head entity and the attention score of any tail entity calculated by each self-attention module, determining the entity attention score includes: Multiply the attention score of any head entity and the attention score of any tail entity calculated by any self-attention module to obtain a single attention score corresponding to any self-attention module; The single attention scores corresponding to each self-attention module are added together to obtain the entity attention score.

[0012] The present invention also provides an Internet online common sense extraction device, comprising: A segmentation unit, used for segmenting an Internet document to obtain a plurality of sub-documents; A local coreference resolution unit, used for performing coreference resolution and entity recognition on any sub-document, obtaining multiple coreference sets and entities existing in each coreference set, and constructing a coreference sparse matrix corresponding to each coreference set of any sub-document; wherein the elements in the coreference sparse matrix corresponding to any coreference set represent the coreference scores of two entity mentions in any sub-document relative to any coreference set; A global coreference resolution unit, configured to calculate a total coreference score of two entity mentions in any sub-document based on a coreference sparse matrix corresponding to each coreference set of any sub-document, calculate a total global coreference score of any two entity mentions in the Internet document based on the total coreference scores of the two entity mentions in each sub-document, and determine a global coreference set based on the total global coreference scores of any two entity mentions in the Internet document; A knowledge extraction unit is used to replace entity mentions in corresponding global co-reference sets based on entity names of entities in each global co-reference set to obtain updated documents, extract entity relationships from the updated documents to obtain triples of the Internet documents, and update the knowledge base based on the triples of the Internet documents.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the above-mentioned methods for extracting online common sense from the Internet is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for extracting online common sense from the Internet as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for extracting online common sense from the Internet.

[0016] The Internet online common sense extraction method, device, electronic device and storage medium provided by the present invention perform coreference resolution and entity recognition on sub-documents of Internet documents to obtain multiple coreference sets and entities existing in each coreference set, and construct coreference sparse matrices corresponding to each coreference set of each sub-document, and then calculate the coreference total score of two entity mentions in the sub-document based on the coreference sparse matrix corresponding to each coreference set of any sub-document, calculate the global coreference total score of any two entity mentions in the Internet document based on the coreference total score of the two entity mentions in each sub-document, and determine the global coreference set based on the global coreference total score of any two entity mentions in the Internet document, so as to replace the entity mentions in the corresponding global coreference set based on the entity names of the entities in each global coreference set to obtain an updated document, and perform entity relationship extraction on the updated document to obtain a triple of the Internet document, and update the knowledge base based on the triple of the Internet document, thereby overcoming the long-distance dependency problem existing in long texts, improving the entity relationship extraction capability of long texts, and improving the knowledge extraction accuracy for Internet documents. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the method for extracting online common sense from the Internet provided by the present invention; Figure 2 It is a flowchart of a method for calculating a global coreference score provided by the present invention; Figure 3 It is a flow chart of the method for constructing a global coreference set provided by the present invention; Figure 4 It is a structural schematic diagram of the Internet online common sense extraction device provided by the present invention; Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Figure 1 is a flow chart of the method for extracting online common sense from the Internet provided by the present invention, such as Figure 1 As shown, the method includes: Step 110, segmenting the Internet document to obtain a plurality of sub-documents; Step 120, performing coreference resolution and entity recognition on any sub-document to obtain multiple coreference sets and entities existing in each coreference set, and constructing a coreference sparse matrix corresponding to each coreference set of any sub-document; wherein the elements in the coreference sparse matrix corresponding to any coreference set represent the coreference scores of two entity mentions in any sub-document relative to any coreference set; Step 130, based on the coreference sparse matrix corresponding to each coreference set of any sub-document, calculating the coreference total score of two entity mentions in any sub-document, calculating the global coreference total score of any two entity mentions in the Internet document based on the coreference total score of the two entity mentions in each sub-document, and determining the global coreference set based on the global coreference total score of any two entity mentions in the Internet document; Step 140, replacing the entity mentions in the corresponding global coreference set based on the entity names of the entities in each global coreference set to obtain an updated document, extracting entity relationships from the updated document to obtain triples of the Internet document, and updating the knowledge base based on the triples of the Internet document.

[0021] Specifically, for a longer Internet document, it can be divided into multiple sub-documents of shorter length. For each sub-document, co-reference resolution can be performed on each sub-document based on a co-reference resolution model (such as the Maverick model) to obtain multiple co-reference sets corresponding to each sub-document. Among them, any co-reference set corresponding to any sub-document contains entity mentions (entity mentions) in the sub-document that refer to the same entity object. In addition, entity recognition can be performed on each sub-document based on an entity recognition model to identify the entities in the co-reference set of each sub-document. For any sub-document, a co-reference sparse matrix corresponding to each co-reference set of the sub-document can be constructed. Here, any element in the co-reference sparse matrix corresponding to any co-reference set of any sub-document represents the co-reference score of two entity mentions in the sub-document relative to the co-reference set. In some embodiments, if the two entity mentions are both in the co-reference set, the co-reference score of the two entity mentions relative to the co-reference set is 1, otherwise it is 0.

[0022] Based on the co-reference sparse matrix corresponding to each co-reference set of any sub-document, the total co-reference score of the two entity mentions in the sub-document can be calculated. The total co-reference score of any two entity mentions in the sub-document indicates the probability that the two entity mentions refer to the same entity within the sub-document. In some embodiments, the co-reference scores of the two entity mentions relative to each co-reference set (which can be obtained from the co-reference sparse matrix corresponding to each co-reference set) can be accumulated to obtain the total co-reference score of the two entity mentions.

[0023] Due to the long length of Internet documents, key information related to the same entity may span multiple paragraphs, and there may be multiple different pronouns or synonymous phrases used to describe the same entity. In order to improve the accuracy of entity relationship extraction in long texts and solve the long-distance dependency problem in long texts, further coreference resolution can be performed on the global scope of the long text. Specifically, based on the total coreference score of the two entity mentions in each sub-document, the global coreference score of any two entity mentions in the Internet document can be calculated, and based on the global coreference score of any two entity mentions in the Internet document, the global coreference set can be determined. Among them, the global coreference score of any two entity mentions in the Internet document indicates the probability that the two entity mentions refer to the same entity in the entire Internet document.

[0024] In some embodiments, in order to further improve the accuracy of coreference resolution, a non-coreference sparse matrix corresponding to each coreference set of each sub-document can be constructed. Among them, the elements in the non-coreference sparse matrix corresponding to any coreference set of any sub-document represent the non-coreference scores of the two entity mentions in the sub-document relative to the coreference set. In some embodiments, if at least one of the two entity mentions is not in the coreference set, the non-coreference scores of the two entity mentions relative to the coreference set are 1, otherwise they are 0. Based on the non-coreference sparse matrix corresponding to each coreference set of each sub-document, the total non-coreference score of the two entity mentions in each sub-document can be calculated. Among them, the total non-coreference score of any two entity mentions in the sub-document indicates the probability that the two entity mentions refer to different entities within the scope of the sub-document. In some embodiments, the non-coreference scores of the two entity mentions relative to each coreference set (which can be obtained from the non-coreference sparse matrix corresponding to each coreference set) can be accumulated to obtain the total non-coreference score of the two entity mentions. Subsequently, based on the coreference scores and non-coreference scores of the two entity mentions in each sub-document, the coreference scores and non-coreference scores are used to complement each other, so as to accurately calculate the global coreference score of any two entity mentions in the Internet document.

[0025] In other embodiments, Figure 2 As shown, the total global coreference score of any two entity mentions in the Internet document can be calculated as follows: Step 210, obtaining any two entity mentions belonging to the same sub-document, calculating the global coreference score of the any two entity mentions based on the coreference scores and non-coreference scores of the any two entity mentions in each sub-document, and adding the any two entity mentions to the mention pair set in the form of a mention pair; Step 220, for any first entity mention and second entity mention that do not belong to the same sub-document, determine the intermediate mention based on the global coreference score of the mention pair in the mention pair set that includes the first entity mention, and the global coreference score of the mention pair in the mention pair set that includes the second entity mention, and determine the global coreference score of the first entity mention and the second entity mention based on the global coreference score of the first entity mention and the intermediate mention and the global coreference score of the intermediate mention and the second entity mention; wherein the intermediate mention satisfies the condition that the product of the global coreference score of the first entity mention and the intermediate mention and the global coreference score of the intermediate mention and the second entity mention is the largest.

[0026] Specifically, we can first filter out entity mentions that belong to the same sub-document (which can be any one or more sub-documents).<mentioni,mentionj> , for any entity mention pair belonging to the same sub-document, the total co-reference score (denoted as co1, co2, ..., con, n is the number of sub-documents) and the total non-co-reference score (denoted as nco1, nco2, ..., ncon) of the two entity mentions in each sub-document can be determined. It should be noted that for any sub-document, if the two entity mentions do not appear in the sub-document at the same time, the total co-reference score of the two entity mentions in the sub-document is 0, and the total non-co-reference score is equal to the number of co-reference sets of the sub-document. Subsequently, based on the total co-reference score and non-co-reference score of the two entity mentions in each sub-document, the global co-reference score of the two entity mentions can be calculated, and the two entity mentions can be grouped as mention pairs (i.e.<mentioni,mentionj> ) is added to the mention pair set in the form of . Among them, the sum of the total co-reference scores of the two entity mentions in each sub-document can be determined as the first parameter, the sum of the total co-reference scores of the two entity mentions in each sub-document and the sum of the total non-co-reference scores of the two entity mentions in each sub-document can be determined as the second parameter, and the ratio of the first parameter to the second parameter is determined to be the global co-reference score of the two entity mentions.

[0027] After calculating the global co-reference scores of all entity mention pairs belonging to the same sub-document in the above manner, for any first entity mention and second entity mention that do not belong to the same sub-document, the global co-reference scores of the first entity mention and the second entity mention can be indirectly determined based on the global co-reference scores of the entity mention pairs belonging to the same sub-document calculated in the previous step. Specifically, an intermediate mention can be determined based on the global co-reference scores of the mention pairs that include the first entity mention in the above-mentioned mention pair set and the global co-reference scores of the mention pairs that include the second entity mention in the above-mentioned mention pair set. Among them, the intermediate mention satisfies the condition that the product of the global co-reference scores of the first entity mention and the intermediate mention and the global co-reference scores of the intermediate mention and the second entity mention is the largest. It can be seen that the mention pair consisting of the first entity mention and the intermediate mention and the mention pair consisting of the second entity mention and the intermediate mention are both in the above-mentioned mention pair set. That is, the intermediate mention satisfies the condition: argmax mentionp(wscore(mention1,mentionp)×wscore(mentionp,mention2)), where mention1 and mention2 are the first entity mention and the second entity mention, mentionp is another entity mention, and wscore() is the global coreference score. Subsequently, the product of the global coreference score of the first entity mention and the middle mention and the global coreference score of the middle mention and the second entity mention is determined as the global coreference score of the first entity mention and the second entity mention.

[0028] After obtaining the global co-reference score of any two entity mentions in an Internet document, a global co-reference set can be determined based on it. The entity mentions in any global co-reference set refer to the same entity in the entire Internet document. In some embodiments, two entity mentions whose global co-reference score exceeds a certain threshold can be placed in the same global co-reference set. Considering that the calculation of the global co-reference score of any two entity mentions in an Internet document is based on the understanding of each sub-document (i.e., short text) by the existing co-reference resolution model, for some complex long texts (especially long texts with implicit reasoning), its global understanding ability of the complex long text may still be slightly lacking. Therefore, in order to further improve the analysis ability of long texts, in other embodiments, graph theory can be used for co-reference resolution. Specifically, if Figure 3 As shown, the global coreference set can be determined as follows: Step 310, determining a judgment threshold based on the total global coreference score of any two entity mentions in the Internet document; Step 320, constructing an entity mention graph based on the total global coreference score of any two entity mentions in the Internet document and the judgment threshold; wherein a node in the entity mention graph corresponds to any entity mention, and if the total global coreference score of any two entity mentions is higher than the judgment threshold, then there is a connecting edge between the nodes corresponding to the any two entity mentions; Step 330: Obtain a node vector representation of each node in the entity mention graph based on a graph convolutional neural network, and determine the global coreference set based on the similarity between the node vector representations of each node in the entity mention graph.

[0029] Here, the average value of the global co-reference score of any two entity mentions in the Internet document can be determined as the judgment threshold. When constructing the entity mention graph, all entity mentions are taken as nodes in the entity mention graph. Further, according to the global co-reference score of any two entity mentions in the Internet document and the above judgment threshold, a connection is established for each entity mention. Specifically, if the global co-reference score of any two entity mentions is higher than the above judgment threshold, a connection edge is established between the nodes corresponding to the two entity mentions. Subsequently, the characteristics of each node in the entity mention graph and the structural relationship between nodes are automatically learned based on the graph convolutional neural network, so as to obtain the node vector representation of each node in the entity mention graph. In some embodiments, for any node corresponding to an entity mention, the encoding features of the entity mention and its context can be obtained based on a pre-trained language model (such as Bert), and the encoding features of the entity mention and its context can be fused to obtain the node features of the node to improve the accuracy of node semantic extraction. The node features of each node and the adjacency matrix corresponding to each node are input into the graph convolutional neural network together, and the node vector representation of each node output by the graph convolutional neural network can be obtained. Based on the similarity between the node vector representations of each node in the entity mention graph, the entity mentions corresponding to the nodes with similarity higher than a preset similarity threshold can be placed in the same global coreference set, thereby determining the global coreference set.

[0030] After obtaining the global coreference set, the relationship between entities can be further extracted. In order to overcome the problem of long-distance dependency during relationship extraction, other entity mentions belonging to the global coreference set can be replaced based on the entity name of the entity in any global coreference set in the Internet document to obtain an updated document, and the updated document is used as the basis for entity relationship extraction. Among them, the entities in the global coreference set can be determined based on the previous entity recognition results. When extracting entity relationships from the updated document, the relationship names of each preset relationship can be spliced ​​to the end of the updated document to obtain the network input text. Subsequently, the network input text is encoded based on the pre-trained encoding network to obtain the entity representation of each entity in the network input text and the relationship representation of each preset relationship. Among them, the encoding network is built based on the Bert model. After the network input text is input into the encoding network, the encoding network will output the vector representation corresponding to each token in the network input text. For any entity, according to the position of the entity in the network input text (since the entity mentions pointing to the same entity were replaced in the previous text, there are multiple places), all the tokens corresponding to the entity in the network input text can be obtained, and the vector representations of the corresponding tokens can be fused to obtain the entity representation of the entity. For any preset relationship, the vector representation of the token corresponding to the preset relationship in the network input text can be directly obtained to fuse the relationship representation of the preset relationship.

[0031] Based on the relational representation of each preset relationship, a relational representation matrix can be constructed. For any head entity and any tail entity, the attention score of the head entity and the attention score of the tail entity calculated by each self-attention module of the last Transformer layer in the encoding network are determined. Among them, since the text input into the encoding network includes an update document and each preset relationship, the attention scores of the head entity and the tail entity calculated by each self-attention module can indicate the degree of correlation between the head entity and the tail entity and each preset relationship. Based on the attention score of the head entity and the attention score of the tail entity calculated by each self-attention module, the entity attention score is determined. In some embodiments, the attention score of the head entity and the attention score of the tail entity calculated by any self-attention module can be multiplied to obtain a single attention score corresponding to the self-attention module, and then the single attention scores corresponding to each self-attention module are added to obtain the entity attention score. Subsequently, the product of the entity attention score and the relational representation matrix is ​​determined as the relational attention representation matrix corresponding to the head entity and the tail entity, so as to introduce the influence of the head entity and the tail entity in the relational representation.

[0032] Based on the relationship attention representation matrix corresponding to the head entity and the tail entity and the entity representation of the head entity and the tail entity, a classifier can be used to determine the relationship type between the head entity and the tail entity by integrating the semantic information of the corresponding entities contained in the entity representation of the head entity and the tail entity and the semantic information of the relationship related to the head and tail entities contained in the relationship attention representation matrix. In some embodiments, the relationship attention representation matrix corresponding to the head entity and the tail entity can be fused with the entity representation of the head entity and the entity representation of the tail entity, respectively, to obtain the relationship fusion entity representation of the head entity and the relationship fusion entity representation of the tail entity, and then the relationship fusion entity representation of the head entity and the relationship fusion entity representation of the tail entity are classified based on the classifier to obtain the relationship type between the head entity and the tail entity. Among them, the following formula can be used to achieve the fusion of the above-mentioned relationship attention representation matrix with the entity representation of the head entity and the entity representation of the tail entity: f s = tanh(W s h s +C s,o W r1 ) f o = tanh(W o h o +C s,o W r2 ) Among them, f s and fo are the relational fusion entity representation of the head entity and the relational fusion entity representation of the tail entity, respectively. s and h o are the entity representation of the head entity and the entity representation of the tail entity, respectively, C s,o is the above relational attention representation matrix, W s , W r1 , W o and W r2 is the learnable weight matrix.

[0033] Based on the head entity, the tail entity and the relationship between the head entity and the tail entity, a triple can be constructed. Based on the triples in the Internet document, the knowledge base can be updated to realize the common sense extraction of the Internet document.

[0034] In summary, the method provided by the embodiment of the present invention performs coreference resolution and entity recognition on the sub-documents of the Internet document to obtain multiple coreference sets and entities existing in each coreference set, and constructs a coreference sparse matrix corresponding to each coreference set of each sub-document, and then calculates the coreference total score of two entity mentions in the sub-document based on the coreference sparse matrix corresponding to each coreference set of any sub-document, calculates the global coreference total score of any two entity mentions in the Internet document based on the coreference total score of the two entity mentions in each sub-document, and determines the global coreference set based on the global coreference total score of any two entity mentions in the Internet document, thereby replacing the entity mentions in the corresponding global coreference set based on the entity names of the entities in each global coreference set to obtain an updated document, and extracts entity relationships from the updated document to obtain triples of the Internet document, and updates the knowledge base based on the triples of the Internet document, thereby overcoming the long-distance dependency problem existing in long texts, improving the entity relationship extraction capability of long texts, and improving the knowledge extraction accuracy for Internet documents.

[0035] The Internet online common sense extraction device provided by the present invention is described below. The Internet online common sense extraction device described below and the Internet online common sense extraction method described above can be referred to each other.

[0036] Based on any of the above embodiments, Figure 4 is a schematic diagram of the structure of the Internet online common sense extraction device provided by the present invention, such as Figure 4 As shown, the device comprises: The segmentation unit 410 is used to segment the Internet document to obtain multiple sub-documents; A local coreference resolution unit 420 is used to perform coreference resolution and entity recognition on any sub-document, obtain multiple coreference sets and entities existing in each coreference set, and construct a coreference sparse matrix corresponding to each coreference set of any sub-document; wherein the elements in the coreference sparse matrix corresponding to any coreference set represent the coreference scores of two entity mentions in any sub-document relative to any coreference set; A global coreference resolution unit 430 is configured to calculate a coreference total score of two entity mentions in any sub-document based on a coreference sparse matrix corresponding to each coreference set of any sub-document, calculate a global coreference total score of any two entity mentions in the Internet document based on the coreference total score of the two entity mentions in each sub-document, and determine a global coreference set based on the global coreference total score of any two entity mentions in the Internet document; The knowledge extraction unit 440 is used to replace the entity mentions in the corresponding global co-reference sets based on the entity names of the entities in each global co-reference set to obtain an updated document, and to extract entity relationships from the updated document to obtain triples of the Internet document, and to update the knowledge base based on the triples of the Internet document.

[0037] The device provided by the embodiment of the present invention performs coreference resolution and entity recognition on sub-documents of Internet documents to obtain multiple coreference sets and entities existing in each coreference set, and constructs a coreference sparse matrix corresponding to each coreference set of each sub-document, and then calculates the coreference total score of two entity mentions in the sub-document based on the coreference sparse matrix corresponding to each coreference set of any sub-document, calculates the global coreference total score of any two entity mentions in the Internet document based on the coreference total score of the two entity mentions in each sub-document, and determines the global coreference set based on the global coreference total score of any two entity mentions in the Internet document, thereby replacing the entity mentions in the corresponding global coreference set based on the entity names of the entities in each global coreference set to obtain an updated document, and extracts entity relationships from the updated document to obtain triples of the Internet document, and updates the knowledge base based on the triples of the Internet document, thereby overcoming the long-distance dependency problem existing in long texts, improving the entity relationship extraction capability of long texts, and improving the knowledge extraction accuracy for Internet documents.

[0038] Based on any of the above embodiments, the step of calculating the global coreference score of any two entity mentions in the Internet document based on the coreference score of the two entity mentions in each sub-document includes: Constructing a non-coreference sparse matrix corresponding to each coreference set of each sub-document, and calculating the total non-coreference score of the two entity mentions in each sub-document based on the non-coreference sparse matrix corresponding to each coreference set of each sub-document; wherein the elements in the non-coreference sparse matrix corresponding to any coreference set of any sub-document represent the non-coreference score of the two entity mentions in any sub-document relative to the any coreference set; Based on the total coreference scores and the total non-coreference scores of the two entity mentions in each sub-document, a global coreference score of any two entity mentions in the Internet document is calculated.

[0039] Based on any of the above embodiments, the calculating the global coreference score of any two entity mentions in the Internet document based on the coreference score and the non-coreference score of the two entity mentions in each sub-document includes: Obtain any two entity mentions belonging to the same sub-document, calculate the global coreference score of the any two entity mentions based on the coreference score and the non-coreference score of the any two entity mentions in each sub-document, and add the any two entity mentions to the mention pair set in the form of a mention pair; For any first entity mention and second entity mention that do not belong to the same sub-document, the intermediate mention is determined based on the global coreference score of the mention pairs in the mention pair set that includes the first entity mention, and the global coreference score of the mention pairs in the mention pair set that includes the second entity mention, and the global coreference score of the first entity mention and the second entity mention is determined based on the global coreference score of the first entity mention and the intermediate mention and the global coreference score of the intermediate mention and the second entity mention; wherein the intermediate mention satisfies the condition that the product of the global coreference score of the first entity mention and the intermediate mention and the global coreference score of the intermediate mention and the second entity mention is the largest.

[0040] Based on any of the above embodiments, determining a global coreference set based on a total global coreference score of any two entity mentions in the Internet document includes: Determining a judgment threshold based on the total global coreference score of any two entity mentions in the Internet document; Based on the total global coreference score of any two entity mentions in the Internet document and the judgment threshold, constructing an entity mention graph; wherein a node in the entity mention graph corresponds to any entity mention, and if the total global coreference score of any two entity mentions is higher than the judgment threshold, then there is a connecting edge between the nodes corresponding to the any two entity mentions; A node vector representation of each node in the entity mention graph is obtained based on a graph convolutional neural network, and the global coreference set is determined based on the similarity between the node vector representations of each node in the entity mention graph.

[0041] Based on any of the above embodiments, extracting entity relationships from the update document includes: splicing the relationship names of the preset relationships to the end of the update document to obtain a network input text; Encoding the network input text based on a coding network to obtain entity representations of each entity in the network input text and relationship representations of each preset relationship; the coding network is constructed based on a Bert model; Constructing a relational representation matrix based on the relational representations of each preset relation; For any head entity and any tail entity, determine the attention score of any head entity and the attention score of any tail entity calculated by each self-attention module of the last Transformer layer in the encoding network; Determine an entity attention score based on the attention score of any head entity and the attention score of any tail entity calculated by each self-attention module; Determine the product of the entity attention score and the relationship representation matrix as the relationship attention representation matrix corresponding to any head entity and any tail entity; Based on the relationship attention representation matrix corresponding to any head entity and any tail entity and the entity representation of any head entity and any tail entity, a classifier is used to determine the relationship type between any head entity and any tail entity.

[0042] Based on any of the foregoing embodiments, the method of determining the relationship type between any head entity and any tail entity using a classifier based on the relationship attention representation matrix corresponding to any head entity and any tail entity and the entity representation of any head entity and any tail entity includes: The relation attention representation matrices corresponding to any head entity and any tail entity are respectively fused with the entity representation of any head entity and the entity representation of any tail entity to obtain the relation fused entity representation of any head entity and the relation fused entity representation of any tail entity; Based on the classifier, relationship classification is performed on the relationship fusion entity representation of any head entity and the relationship fusion entity representation of any tail entity to obtain the relationship type between any head entity and any tail entity.

[0043] Based on any of the foregoing embodiments, determining an entity attention score based on the attention score of any head entity and the attention score of any tail entity calculated by each self-attention module includes: Multiply the attention score of any head entity and the attention score of any tail entity calculated by any self-attention module to obtain a single attention score corresponding to any self-attention module; The single attention scores corresponding to each self-attention module are added together to obtain the entity attention score.

[0044] Figure 5 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 5 As shown, the electronic device may include: a processor 510, a memory 520, a communications interface 530 and a communication bus 540, wherein the processor 510, the memory 520 and the communications interface 530 communicate with each other via the communications bus 540. The processor 510 may call the logic instructions in the memory 520 to execute the Internet online common sense extraction method, which includes: segmenting an Internet document to obtain a plurality of sub-documents; performing co-reference resolution and entity recognition on any sub-document to obtain a plurality of co-reference sets and entities existing in each co-reference set, and constructing a co-reference sparse matrix corresponding to each co-reference set of any sub-document; wherein the elements in the co-reference sparse matrix corresponding to any co-reference set represent the co-reference scores of two entity mentions in any sub-document relative to any co-reference set; based on the co-reference corresponding to each co-reference set of any sub-document The invention relates to a sparse matrix, calculating the total co-reference score of the two entity mentions in any sub-document, calculating the total global co-reference score of any two entity mentions in the Internet document based on the total co-reference score of the two entity mentions in each sub-document, and determining the global co-reference set based on the total global co-reference score of any two entity mentions in the Internet document; replacing the entity mentions in the corresponding global co-reference set based on the entity name of the entity in each global co-reference set to obtain an updated document, extracting entity relations from the updated document to obtain a triple of the Internet document, and updating the knowledge base based on the triple of the Internet document.

[0045] In addition, the logic instructions in the above-mentioned memory 520 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0046] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the Internet online common sense extraction method provided by the above methods, and the method includes: segmenting an Internet document to obtain multiple sub-documents; performing co-reference resolution and entity recognition on any sub-document to obtain multiple co-reference sets and entities existing in each co-reference set, and constructing a co-reference sparse matrix corresponding to each co-reference set of any sub-document; wherein the elements in the co-reference sparse matrix corresponding to any co-reference set represent two entities in any sub-document. Mention the coreference score relative to any coreference set; calculate the total coreference score of two entity mentions in any sub-document based on the coreference sparse matrix corresponding to each coreference set of any sub-document, calculate the global coreference score of any two entity mentions in the Internet document based on the total coreference score of the two entity mentions in each sub-document, and determine the global coreference set based on the global coreference score of any two entity mentions in the Internet document; replace the entity mentions in the corresponding global coreference set based on the entity name of the entity in each global coreference set to obtain an updated document, extract entity relations on the updated document to obtain a triple of the Internet document, and update the knowledge base based on the triple of the Internet document.

[0047] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the processor executes the above-mentioned Internet online common sense extraction method, the method comprising: segmenting an Internet document to obtain a plurality of sub-documents; performing co-reference resolution and entity recognition on any sub-document to obtain a plurality of co-reference sets and entities existing in each co-reference set, and constructing a co-reference sparse matrix corresponding to each co-reference set of any sub-document; wherein the elements in the co-reference sparse matrix corresponding to any co-reference set represent the co-reference scores of two entity mentions in any sub-document relative to any co-reference set; based on The invention relates to a co-reference sparse matrix corresponding to each co-reference set of any sub-document, calculating the total co-reference score of two entity mentions in any sub-document, calculating the total global co-reference score of any two entity mentions in the Internet document based on the total co-reference score of the two entity mentions in each sub-document, and determining the global co-reference set based on the total global co-reference score of any two entity mentions in the Internet document; replacing the entity mentions in the corresponding global co-reference set based on the entity name of the entity in each global co-reference set to obtain an updated document, extracting entity relations from the updated document to obtain a triple of the Internet document, and updating the knowledge base based on the triple of the Internet document.

[0048] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0049] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0050] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting online common sense from the Internet, characterized in that: include: Segment the Internet document to obtain multiple sub-documents; Perform coreference resolution and entity recognition on any sub-document to obtain multiple coreference sets and entities in each coreference set, and construct a coreference sparse matrix corresponding to each coreference set of any sub-document; wherein the elements in the coreference sparse matrix corresponding to any coreference set represent the coreference scores of two entity mentions in any sub-document relative to any coreference set; Based on the coreference sparse matrix corresponding to each coreference set of any sub-document, calculating the coreference total score of two entity mentions in any sub-document, calculating the global coreference total score of any two entity mentions in the Internet document based on the coreference total score of the two entity mentions in each sub-document, and determining the global coreference set based on the global coreference total score of any two entity mentions in the Internet document; Based on the entity names of the entities in each global coreference set, entity mentions in the corresponding global coreference set are replaced to obtain an updated document, and entity relations are extracted from the updated document to obtain triples of the Internet document, and a knowledge base is updated based on the triples of the Internet document.

2. The method for extracting online common sense from the Internet according to claim 1, characterized in that: The calculating the global coreference score of any two entity mentions in the Internet document based on the coreference score of the two entity mentions in each sub-document includes: Constructing a non-coreference sparse matrix corresponding to each coreference set of each sub-document, and calculating the total non-coreference score of the two entity mentions in each sub-document based on the non-coreference sparse matrix corresponding to each coreference set of each sub-document; wherein the elements in the non-coreference sparse matrix corresponding to any coreference set of any sub-document represent the non-coreference score of the two entity mentions in any sub-document relative to the any coreference set; Based on the total coreference scores and the total non-coreference scores of the two entity mentions in each sub-document, a global coreference score of any two entity mentions in the Internet document is calculated.

3. The method for extracting online common sense from the Internet according to claim 2, characterized in that: The calculating of the global coreference score of any two entity mentions in the Internet document based on the coreference score and the non-coreference score of the two entity mentions in each sub-document comprises: Obtain any two entity mentions belonging to the same sub-document, calculate the global coreference score of the any two entity mentions based on the coreference score and the non-coreference score of the any two entity mentions in each sub-document, and add the any two entity mentions to the mention pair set in the form of a mention pair; For any first entity mention and second entity mention that do not belong to the same sub-document, the intermediate mention is determined based on the global coreference score of the mention pairs in the mention pair set that includes the first entity mention, and the global coreference score of the mention pairs in the mention pair set that includes the second entity mention, and the global coreference score of the first entity mention and the second entity mention is determined based on the global coreference score of the first entity mention and the intermediate mention and the global coreference score of the intermediate mention and the second entity mention; wherein the intermediate mention satisfies the condition that the product of the global coreference score of the first entity mention and the intermediate mention and the global coreference score of the intermediate mention and the second entity mention is the largest.

4. The method for extracting online common sense from the Internet according to claim 1, characterized in that: The determining of a global coreference set based on the total global coreference score of any two entity mentions in the Internet document comprises: Determining a judgment threshold based on the total global coreference score of any two entity mentions in the Internet document; Based on the total global coreference score of any two entity mentions in the Internet document and the judgment threshold, constructing an entity mention graph; wherein a node in the entity mention graph corresponds to any entity mention, and if the total global coreference score of any two entity mentions is higher than the judgment threshold, then there is a connecting edge between the nodes corresponding to the any two entity mentions; A node vector representation of each node in the entity mention graph is obtained based on a graph convolutional neural network, and the global coreference set is determined based on the similarity between the node vector representations of each node in the entity mention graph.

5. The method for extracting online common sense from the Internet according to any one of claims 1 to 4, characterized in that: The extracting entity relationships from the update document includes: splicing the relationship names of the preset relationships to the end of the update document to obtain a network input text; Encoding the network input text based on a coding network to obtain entity representations of each entity in the network input text and relationship representations of each preset relationship; the coding network is constructed based on a Bert model; Constructing a relational representation matrix based on the relational representations of each preset relation; For any head entity and any tail entity, determine the attention score of any head entity and the attention score of any tail entity calculated by each self-attention module of the last Transformer layer in the encoding network; Determine an entity attention score based on the attention score of any head entity and the attention score of any tail entity calculated by each self-attention module; Determine the product of the entity attention score and the relationship representation matrix as the relationship attention representation matrix corresponding to any head entity and any tail entity; Based on the relationship attention representation matrix corresponding to any head entity and any tail entity and the entity representation of any head entity and any tail entity, a classifier is used to determine the relationship type between any head entity and any tail entity.

6. The method for extracting online common sense from the Internet according to claim 5, characterized in that: The determining the relationship type between any head entity and any tail entity using a classifier based on the relationship attention representation matrix corresponding to any head entity and any tail entity and the entity representation of any head entity and any tail entity includes: The relation attention representation matrices corresponding to any head entity and any tail entity are respectively fused with the entity representation of any head entity and the entity representation of any tail entity to obtain the relation fused entity representation of any head entity and the relation fused entity representation of any tail entity; Based on the classifier, relationship classification is performed on the relationship fusion entity representation of any head entity and the relationship fusion entity representation of any tail entity to obtain the relationship type between any head entity and any tail entity.

7. The method for extracting online common sense from the Internet according to claim 5, characterized in that: Determining an entity attention score based on the attention score of any head entity and the attention score of any tail entity calculated by each self-attention module includes: Multiply the attention score of any head entity and the attention score of any tail entity calculated by any self-attention module to obtain a single attention score corresponding to any self-attention module; The single attention scores corresponding to each self-attention module are added together to obtain the entity attention score.

8. An Internet online common sense extraction device, characterized in that: include: A segmentation unit, used for segmenting an Internet document to obtain a plurality of sub-documents; A local coreference resolution unit, used for performing coreference resolution and entity recognition on any sub-document, obtaining multiple coreference sets and entities existing in each coreference set, and constructing a coreference sparse matrix corresponding to each coreference set of any sub-document; wherein the elements in the coreference sparse matrix corresponding to any coreference set represent the coreference scores of two entity mentions in any sub-document relative to any coreference set; A global coreference resolution unit, configured to calculate a total coreference score of two entity mentions in any sub-document based on a coreference sparse matrix corresponding to each coreference set of any sub-document, calculate a total global coreference score of any two entity mentions in the Internet document based on the total coreference scores of the two entity mentions in each sub-document, and determine a global coreference set based on the total global coreference scores of any two entity mentions in the Internet document; A knowledge extraction unit is used to replace entity mentions in corresponding global co-reference sets based on entity names of entities in each global co-reference set to obtain updated documents, extract entity relationships from the updated documents to obtain triples of the Internet documents, and update the knowledge base based on the triples of the Internet documents.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for extracting online common sense from the Internet as claimed in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for extracting online common sense from the Internet as claimed in any one of claims 1 to 7 is implemented.

Images