Knowledge graph construction method, knowledge graph construction system and computing device

Abstract

The present invention discloses a knowledge graph construction method, a knowledge graph construction system and a computing device, the method comprising: obtaining text as raw data, performing chapter-level knowledge extraction and sentence-level event extraction on the text to obtain knowledge extraction results and event extraction results; and performing knowledge fusion on the knowledge extraction results and the event extraction results to obtain a knowledge graph. The present invention changes the concept of knowledge graph from traditional static knowledge triples to composite knowledge including static knowledge quintuples and dynamic event knowledge, and uses chapter-level element extraction and element association to greatly improve the extraction efficiency and recall rate of quintuples, making the knowledge structured attribute stronger.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more specifically, to a knowledge graph construction method, a knowledge graph construction system and a computing device. Background Art

[0002] At present, with the continuous development of intelligent information service applications, knowledge graphs have been widely used in intelligent search, intelligent question and answer, personalized recommendation, intelligence analysis, anti-fraud and other fields. Knowledge graphs effectively process, process and integrate the data of complex documents, transforming them into simple and clear "entity, relationship, entity" triples, and finally aggregate a large amount of knowledge, thereby achieving rapid response and reasoning of knowledge.

[0003] The existing knowledge graph construction method is implemented through sentence-level relationship extraction. However, the implicit triples in some data are not within the scope of a sentence, and the existing sentence-level relationship extraction method often loses a lot of structured data information. In addition, in traditional knowledge graphs, knowledge is generally static knowledge, while human society is generally dynamic. Therefore, static triple information does not take into account dynamically changing knowledge and may contain errors. In addition, traditional methods only construct knowledge maps through extraction, without a more complete graph cleaning and automatic supplementation mechanism. The constructed knowledge graph is not accurate and is prone to the loss of some attributes or relationships.

[0004] Therefore, a new knowledge graph construction method, knowledge graph construction system and computing device are needed to solve the above problems. Summary of the invention

[0005] A series of simplified concepts are introduced in the Summary of the Invention, which will be further described in detail in the Detailed Description of the Invention. The Summary of the Invention does not mean to attempt to define the key features and essential technical features of the claimed technical solution, nor does it mean to attempt to determine the scope of protection of the claimed technical solution.

[0006] According to one aspect of the present invention, a knowledge graph construction method is provided, the method comprising: obtaining text as raw data, performing paragraph-level knowledge extraction and sentence-level event extraction on the text to obtain knowledge extraction results and event extraction results; and performing knowledge fusion on the knowledge extraction results and the event extraction results to obtain a knowledge graph.

[0007] In one embodiment, extracting knowledge from the text at the chapter level includes: based on preset category labels, using a sliding window to extract knowledge elements from the text corresponding to the category labels; and associating the extracted knowledge elements to obtain the knowledge extraction result.

[0008] In one embodiment, using a sliding window to extract knowledge from the text includes: using the sliding window to slide on the text, and for each category label, calculating the prediction probability of each word in the text within the sliding window; calculating the average of each prediction probability of the same word at each position in the sliding window as the final prediction probability of the word; based on the final prediction probability of each word, selecting the knowledge element from the text according to preset knowledge element selection conditions.

[0009] In one embodiment, associating the extracted knowledge elements includes: processing each character of the knowledge element to obtain a character vector of each character; processing the character vectors of all characters corresponding to the knowledge element to obtain an element vector of the knowledge element; performing association judgment on the element vectors of each knowledge element to obtain an association result for each knowledge element; and combining the knowledge elements according to the association result to obtain the knowledge extraction result.

[0010] In one embodiment, sentence-level event extraction of the text includes: predicting whether each word in each sentence of the text is a trigger word and its trigger word type based on a preset event trigger word label; using the predicted trigger word, trigger word type and the position of the trigger word, predicting the event parameters and parameter types corresponding to the trigger word in the sentence based on a preset event parameter label; and combining the trigger word and the event parameters to obtain the event extraction result.

[0011] In one embodiment, knowledge fusion of the knowledge extraction result and the event extraction result includes: performing entity linking processing on the knowledge extraction result and the event extraction result to obtain an entity linking result, wherein the entity linking result indicates whether each entity in the knowledge extraction result and the event extraction result is the same entity, and whether each entity is the same entity as an existing entity in the knowledge graph; and recombining the knowledge extraction result and the event extraction result based on the entity linking result to obtain graph knowledge for the knowledge graph.

[0012] In one embodiment, the knowledge fusion of the knowledge extraction result and the event extraction result also includes: inferring new graph knowledge based on the graph knowledge and / or existing knowledge in the knowledge graph; and updating the knowledge graph with the new graph knowledge.

[0013] In one embodiment, the method further includes: performing intra-text reference chain identification on the event extraction results to determine whether the various event extraction results of the text point to the same event based on a preset intra-text event co-reference feature type; performing cross-text event co-reference resolution processing on the event extraction results to determine whether the event extraction results of the text and the existing events in the knowledge graph point to the same event based on a preset cross-text event co-reference feature type; and removing duplicate events based on the determination.

[0014] In one embodiment, the method also includes a knowledge cleaning step for correcting or removing redundant or erroneous information, wherein the knowledge cleaning step includes merging redundant expressions, correcting and removing obvious errors, inferring similar entities and supplementing attribute names, and merging synonymous attribute names.

[0015] According to another aspect of the present invention, a knowledge graph construction system is provided, the system comprising: a composite knowledge extraction module for performing paragraph-level knowledge extraction and sentence-level event extraction on a text acquired as raw data to obtain knowledge extraction results and event extraction results; and a composite knowledge fusion module for performing knowledge fusion on the knowledge extraction results and the event extraction results to obtain a knowledge graph.

[0016] According to another embodiment of the present invention, a computing device is provided. The computing device includes a memory and a processor. A computer program is stored in the memory. When the computer program is executed by the processor, the processor executes the method described above.

[0017] According to yet another embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed, the method described above is executed.

[0018] According to the knowledge graph construction method, knowledge graph construction system and computing device of the embodiments of the present invention, the concept of knowledge graph is changed from traditional static knowledge triples to composite knowledge including static knowledge quintuples and dynamic event knowledge. The use of chapter-level element extraction and element association can greatly improve the extraction efficiency and recall rate of quintuples, making the knowledge more structured. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The following drawings of the present invention are used to understand the present invention as part of the present invention. The embodiments of the present invention are shown in the drawings and the description thereof is used to explain the principle of the present invention.

[0020] In the attached figure:

[0021] Figure 1A schematic structural block diagram of an electronic device that implements a knowledge graph construction method, a knowledge graph construction system, and a computing device according to an embodiment of the present invention.

[0022] Figure 2 The present invention is a flowchart of exemplary steps of a method for constructing a knowledge graph according to an embodiment of the present invention.

[0023] Figure 3 A schematic diagram of using a sliding window to extract knowledge from text according to an embodiment of the present invention is shown.

[0024] Figure 4 A schematic structural block diagram of a knowledge graph construction system according to an embodiment of the present invention is shown.

[0025] Figure 5 A schematic structural block diagram of a computing device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the present invention more obvious, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the protection scope of the present invention.

[0027] As mentioned above, existing knowledge graph construction methods will lose a lot of structured data information, do not consider dynamically changing knowledge, and may contain errors.

[0028] Therefore, the present invention provides a knowledge graph construction method, which includes: obtaining text as original data, performing paragraph-level knowledge extraction and sentence-level event extraction on the text to obtain knowledge extraction results and event extraction results; and performing knowledge fusion on the knowledge extraction results and the event extraction results to obtain a knowledge graph.

[0029] According to the knowledge graph construction method of the present invention, the concept of knowledge graph is changed from traditional static knowledge triples to composite knowledge including static knowledge quintuples and dynamic event knowledge. The use of chapter-level element extraction and element association can greatly improve the extraction efficiency and recall rate of quintuples, making the knowledge more structured.

[0030] The knowledge graph construction method, knowledge graph construction system and computing device according to the present invention are described in detail below in conjunction with specific embodiments.

[0031] First, refer to Figure 1 To describe an electronic device 100 for implementing a knowledge graph construction method, a knowledge graph construction system, and a computing device according to an embodiment of the present invention.

[0032] In one embodiment, the electronic device 100 can be, for example, a laptop computer, a desktop computer, a tablet computer, a learning machine, a mobile device (such as a smart phone, a phone watch, etc.), an embedded computer, a tower server, a rack server, a blade server, or any other suitable electronic device.

[0033] In one embodiment, the electronic device 100 may include at least one processor 102 and at least one memory 104 .

[0034] The memory 104 may be a volatile memory, such as a random access memory (RAM), a cache memory (cache), a dynamic random access memory (DRAM) (including stacked DRAM) or a high bandwidth memory (HBM), etc., or a non-volatile memory, such as a read-only memory (ROM), a flash memory, 3D Xpoint, etc. In one embodiment, some parts of the memory 104 may be volatile memory, while another part may be non-volatile memory (for example, using a two-level memory hierarchy). The memory 104 is used to store a computer program, which, when executed, can implement the client functions (implemented by the processor) and / or other desired functions in the embodiments of the present invention described below.

[0035] Processor 102 may be a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, or other processing unit having data processing capabilities and / or instruction execution capabilities. Processor 102 may be communicatively coupled to any suitable number or type of components, peripherals, modules, or devices via a communication bus. In one embodiment, the communication bus may be implemented using any suitable protocol, such as Peripheral Component Interconnect (PCI), Peripheral Component Interconnect Express (PCIe), Accelerated Graphics Port (AGP), HyperTransport, or any other bus or one or more point-to-point communication protocols.

[0036] The electronic device 100 may further include an input device 106 and an output device 108. The input device 106 is a device for receiving user input, which may include a keyboard, a mouse, a touch pad, a microphone, etc. In addition, the input device 106 may also be any interface for receiving information. The output device 108 may output various information (such as images or sounds) to the outside (such as a user), which may include one or more of a display, a speaker, etc. In addition, the output device 108 may also be any other device with an output function, such as a printer, etc.

[0037] Reference below Figure 2 The following is a flowchart of exemplary steps of a knowledge graph construction method 200 according to an embodiment of the present invention. Figure 2 As shown, the knowledge graph construction method 200 may include the following steps:

[0038] In step S210, a text as raw data is obtained, and chapter-level knowledge extraction and sentence-level event extraction are performed on the text to obtain knowledge extraction results and event extraction results.

[0039] In one embodiment, the original data may include structured data, semi-structured data, and unstructured data. This step is mainly used to process the unstructured data.

[0040] Among them, structured data refers to data that can be directly stored using a relational database, that is, the traditional triple form, such as (Mao Dun, original name, Shen Dehong).

[0041] Semi-structured data is a form of structured data, but it does not conform to the data model structure of a relational database or other data tables, but contains relevant tags to separate semantic elements and hierarchical records and fields. Therefore, it is also called a self-describing structure. Common semi-structured data formats include XML, JSON, etc.

[0042] Unstructured data refers to ordinary text data without structured information.

[0043] In one embodiment, any pre-trained language model known in the art (such as BERT (Bidirectional Encoder Representation Based on Transformer) model, RoBERTa model, ALBERT model, NEZHA model, XLNET model, GPT model, UniLM model, etc.) can be used to extract chapter-level knowledge from text, and the present invention does not limit this. However, the above-mentioned pre-trained language models known in the art cannot solve the problem of chapter-level knowledge extraction. Therefore, on the basis of using the pre-trained language model, the present invention performs some processing on the pre-trained language model in the training stage and adopts a sliding window strategy, thereby solving the problem of chapter-level knowledge extraction.

[0044] Specifically, the processing of the pre-trained language model during the training phase includes:

[0045] (1) When the text is too long, use the sliding window strategy to split the long text (for example, more than 512 characters) into multiple text segments, and each text segment is combined with the question to form a training data;

[0046] (2) Refer to the SQuAD (reading comprehension) task and predict the start and end positions of the answer in the text. This is divided into the following four cases:

[0047] -The sliding window contains the complete answer, and the start and end positions are specific Tok positions;

[0048] - There is no answer fragment in the sliding window, and the start and end positions are both [CLS] positions;

[0049] -The first part of the answer in the sliding window, starting at the specific Tok position and ending at the [CLS] position;

[0050] -The sliding window contains the latter part of the answer, starting at the [CLS] position and ending at the specific Tok position.

[0051] In one embodiment, extracting chapter-level knowledge from a text may include the following steps:

[0052] In step a1, based on the preset category labels, a sliding window is used to perform knowledge extraction on the text to extract knowledge elements corresponding to the category labels in the text;

[0053] In step a2, the extracted knowledge elements are associated to obtain a knowledge extraction result.

[0054] In one embodiment, the category labels may be arranged hierarchically, such as primary labels, secondary labels, tertiary labels, etc. The following table 1 shows exemplary category labels:

[0055] Table 1

[0056]

[0057]

[0058] Among them, secondary tags such as network location, WeChat, Sina, QQ, Taobao, occupation, and alma mater can be grouped.

[0059] Reference now Figure 3 , Figure 3 A schematic diagram of using a sliding window to extract knowledge from text according to an embodiment of the present invention is shown.

[0060] like Figure 3 As shown, for the long text chapter scenario, a sliding window can be preset (such as Figure 3 The size of the scroll bar (shown as the solid rectangle in the figure) and the sliding step length are set to slide over the entire text of the chapter. Figure 3Taking a 1000-word text (including construction marks such as CLS and SEP) as an example, the preset sliding window size is 512 words, and the sliding step is 128 words, then a total of 8 windows are obtained by sliding (not all are shown in the figure), and the starting positions are the 1st, 129th, 257th, 385th, 513th, 641st, 769th, and 897th words respectively.

[0061] When the sliding window is used to slide on the long text as the original data, the prediction probability of each word in the text within the sliding window is calculated for each preset category label, and then the average value of each prediction probability of the same word at each position in the sliding window is calculated (such as Figure 3 ), as the final prediction probability of the word, and then based on the final prediction probability of each word, knowledge elements are selected from the text according to preset knowledge element selection conditions.

[0062] In one embodiment, after obtaining the final prediction probability of each word, knowledge elements may be selected from the original data according to the following exemplary knowledge element selection conditions:

[0063] —Output the 10 groups of start and end position probabilities with the highest final prediction probability;

[0064] —Compare with the CLS position probability, and remove the start and end position probabilities whose final predicted probabilities are lower than the CLS position probability among the 10 groups of start and end position probabilities;

[0065] —Among the remaining start and end position probabilities, remove the start and end position probabilities whose final predicted probabilities are lower than one tenth of the highest predicted probabilities.

[0066] In addition, text segments composed of different groups of start and end positions can remove repeated parts of some repeated text segments. For example, two groups of start and end position combinations of <5,10> and <8,12> can be corrected to <5,7> and <8,12>.

[0067] After the final groups of start and end positions are obtained according to the above-mentioned knowledge element selection conditions, the corresponding text fragments are the selected knowledge elements.

[0068] In one embodiment, any language representation model known in the art (eg, BERT model) may be used to associate the extracted knowledge elements, and the present invention is not limited to this.

[0069] In one embodiment, associating the extracted knowledge elements may include the following steps:

[0070] In step b1, each word of the extracted knowledge element is processed to obtain a word vector representation of each word.

[0071] Exemplarily, a pre-trained model or other models (eg, a BERT model) is used to obtain a word vector for each word.

[0072] In step b2, the word vectors of all words corresponding to each knowledge element are processed to obtain the element vector of the knowledge element.

[0073] Exemplarily, the word vectors of all words can be processed using a neural network algorithm + pooling method to obtain the element vector of the knowledge element.

[0074] In step b3, association judgment is performed on the element vectors of each knowledge element to obtain an association result for each knowledge element.

[0075] Exemplarily, any classification network known in the art (such as a sigmoid network, etc.) can be used to perform association judgment on the element vectors of two knowledge elements, and the present invention is not limited to this.

[0076] For example, one or more knowledge elements can be used as the center, and other knowledge elements can be judged in binary classification with them, so as to obtain the association results for each knowledge element. For example, the name of a person is used as the center, and all other knowledge elements are judged in binary classification with them. If the extracted knowledge element has no relevant name, the association judgment of the knowledge element is no longer performed, and the content of the knowledge element is invalid.

[0077] For example, for secondary tags that need to be grouped, such as network location, WeChat, Sina, QQ, Taobao, occupation, and alma mater, the content of the first line of the tertiary tag is prioritized to determine whether other knowledge elements are associated with it.

[0078] In step b4, the knowledge elements are combined according to the association results to obtain the knowledge extraction results.

[0079] Exemplarily, the knowledge extraction result may include triplets and / or quintuples of knowledge elements.

[0080] Table 2 below shows an exemplary knowledge extraction result.

[0081]

[0082]

[0083] In one embodiment, after obtaining the knowledge extraction result, the knowledge extraction result may be post-processed. Exemplarily, the post-processing may include:

[0084] (1) If a certain knowledge element is associated with multiple knowledge elements, based on their association probabilities, if all the association probabilities are greater than a preset threshold, the corresponding group or groups of associated content are retained;

[0085] (2) If two knowledge elements to be predicted appear in multiple sliding windows, and multiple sliding windows give prediction results, the set of results with the highest association probability will be taken as the final prediction probability.

[0086] In one embodiment, extracting sentence-level events from text may include the following steps:

[0087] In step c1, whether each word in each sentence of the text is a trigger word and its trigger word type are predicted based on the preset event trigger word label.

[0088] In one embodiment, the event trigger word label can be customized as needed, which is not limited in the present invention.

[0089] Table 3 below shows example event trigger word tags according to one embodiment.

[0090] Table 3

[0091]

[0092] In one embodiment, a trained trigger word model can be used to predict trigger words and their trigger word types. The trigger word model can be obtained by training any suitable model known in the art (such as the BERT semantic representation model, etc.), and the present invention is not limited to this.

[0093] In one embodiment, the semantic representation model in the trigger word model (e.g., the trained BERT semantic representation model) can be used to obtain the semantic representation of each word in the sentence, and then each word is classified (the category is the sequence annotation label system, and BIOES is combined with the trigger word category name), and the trigger word sequence with an output probability exceeding a predetermined threshold is used as the prediction result of the trigger word. It should be understood that there may be multiple trigger words in a sentence.

[0094] In one embodiment, an example process of the trigger word model obtaining the semantic representation of each word in a sentence is as follows: [CLS] and the semantic representation of each word in the sentence are obtained, and then the sequence annotation output result of each word (the combination of BIOES and the trigger word category name) is output for the semantic representation of each word in the sentence, and the trigger word sequence whose output probability exceeds a predetermined threshold is used as the trigger word prediction result.

[0095] In step c2, the predicted trigger word, trigger word type and trigger word position are used to predict the event parameter and its parameter type corresponding to the trigger word in the sentence based on the preset event parameter label.

[0096] In one embodiment, the event parameter label can be customized as needed, which is not limited in the present invention.

[0097] Table 4 below shows example event parameter tags according to one embodiment.

[0098] Table 4

[0099]

[0100] In one embodiment, a trained parameter model can be used to predict event parameters and parameter types corresponding to the predicted trigger words. The parameter model can be obtained by training any suitable model known in the art (such as the BERT semantic representation model, etc.), and the present invention is not limited to this.

[0101] In one embodiment, the parameter model is the same type as the trigger word model, except that it has one more marker bit at the input end. The marker bit of the trigger word position currently being processed in the sentence is 1, and the marker bits of other trigger word positions are 0, to indicate which trigger word is currently being processed. After the semantic representation model in the parameter model, the subsequent steps are similar to the prediction process of the trigger word model.

[0102] In one embodiment, an example process of the parameter model predicting event parameters and their parameter types corresponding to the predicted trigger words is as follows: the Segment of the position corresponding to the predicted trigger word is set to 1, and the other positions are set to 0, and the semantic representation of each word in the sentence is obtained using, for example, the BERT semantic representation model. For the semantic representation of each word in the sentence, the sequence annotation output result of each word is output (BIOES combined with parameter category), and the trigger word sequence with an output probability exceeding a predetermined threshold is used as the parameter prediction result.

[0103] In step c3, the predicted trigger words and event parameters are combined to obtain event extraction results.

[0104] In one embodiment, the event extraction result may include a triple and / or a quintuple of trigger words and event parameters.

[0105] In one embodiment, after obtaining the event extraction result, the event coreference resolution process can be performed on the event extraction result. The goal of event coreference resolution process is to effectively understand whether the references in the text refer to the same event, and to remove duplicate events when it is determined that they refer to the same event.

[0106] In one embodiment, event co-reference resolution processing may include two steps: one is intra-text reference chain identification, which is used to determine whether the event extraction results of the text point to the same event based on the preset intra-text event co-reference feature type; the other is cross-text event co-reference resolution based on the reference chain, which is used to determine whether the event extraction results of the text and the existing events in the knowledge graph point to the same event based on the preset cross-text event co-reference feature type.

[0107] In one embodiment, first, feature types for event coreference resolution processing can be pre-set, including intra-chapter event coreference feature types and cross-chapter event coreference feature types, and then, based on the intra-chapter event coreference feature types, it is determined according to predetermined judgment rules whether the event extraction results of the text point to the same event, and based on the preset cross-chapter event coreference feature types, it is determined according to predetermined judgment rules whether the event extraction results of the text and the existing events in the knowledge graph point to the same event.

[0108] In one embodiment, the judgment rules can be preset as needed. For example, if 2, 3, 4, etc. features of an event are the same, two events can be determined to be the same event. The present invention is not limited to this.

[0109] Most of the known studies on event coreference resolution within a text rely on the selection of feature space. In the past decade or so, many scholars have proposed different features for the particularity of event coreference resolution tasks. Compared with traditional natural language processing, since reference is composed of attributes, arguments, etc., feature engineering also presents more discrete characteristics.

[0110] Tables 5 and 6 below give exemplary designs of intra-text event co-reference feature types and cross-text event co-reference feature types, respectively.

[0111] Table 5. Co-reference characteristics of events within a text

[0112]

[0113] Table 6. Features of cross-text event co-reference

[0114]

[0115]

[0116] For ease of understanding, the characteristics of Table 6 are explained as follows:

[0117] 1) Characteristics of referential chain attributes

[0118] Each denotation chain corresponds to a type and subtype. When extracting the type features of the denotation chains, by judging whether the types and subtypes of the two denotation chains are consistent, the impact of inconsistent type denotation chains on the performance can be eliminated. When extracting the generic features of the denotation chains, since there may be multiple different generics in each denotation chain, we enumerate and encode these possible generics, and use the encoded generics as generic category features. At the same time, in order to judge whether the generics between the two denotation chains are consistent, we select the most frequently appearing generics in each denotation chain as the main generics of the chain, and judge the generic consistency between the denotation chains based on the main generics in the two denotation chains.

[0119] 2) Features of referential chain trigger words

[0120] As the core part of the event composition, trigger words also play a vital role in the event coreference resolution task, and many works are also carried out around trigger words. Based on traditional feature engineering, we took into account the fact that the reference chain may be composed of multiple references, and extracted the trigger word set from the reference chain. By comparing whether there are the same trigger words, we can get the number of identical and different trigger words; by using the interface in NLTK to obtain the root and part of speech of each trigger word, and judge the consistency of their roots and parts of speech respectively; in order to avoid differences between different article narratives, we calculated the similarity of trigger words as the trigger word similarity feature.

[0121] 3) Argument Features of Reference Chain

[0122] As an important component of an event, arguments play a very important role in understanding event information. Through argument information, we can know the time, place, people, and other information of the event. Since the references in the reference chain all point to the same event, the argument information contained in each reference also points to the argument corresponding to the same event. By obtaining and merging different argument roles and their central words of different references in the same reference chain, we can get the arguments of a reference chain. By merging different arguments and central words in different references, the information contained in the arguments of the reference chain is greatly enriched. In the argument feature extraction part of the reference chain, we compare the number of the same argument roles between reference chains as features, and compare whether the corresponding central words in the same argument roles overlap as further features.

[0123] 4) Distribution characteristics of referential chains

[0124] The distribution of reference chains in the text can also help us understand the resolution of cross-chapter event co-reference. The more references there are in a reference chain, the more important the event the chain points to is in the chapter. At the same time, the event is mentioned many times in other articles on the same topic, which is consistent with the characteristics of news texts chasing hot events. According to the frequency of reference chain mentions, we propose the reference chain length ratio feature and the reference chain text importance feature. The text importance feature is obtained by calculating the proportion of the number of references contained in the reference chain to the number of references in the article. In addition, the breadth distribution of the reference chain in the text also helps us understand the importance of the reference chain in the chapter. Based on this, we obtain the relative position of each reference in the chapter and the relative position of the sentence in which the reference is located in the chapter. Based on the distribution of the above positions in the reference chain, the reference relative position head-to-end difference ratio feature and the sentence relative position head-to-end difference ratio feature are proposed.

[0125] In step S220, knowledge fusion is performed on the knowledge extraction results and the event extraction results to obtain a knowledge graph.

[0126] The knowledge fusion may include the following steps: performing entity linking processing on the knowledge extraction results and the event extraction results to obtain entity linking results, wherein the entity linking results indicate whether each entity in the knowledge extraction results and the event extraction results is the same entity, and whether each entity is the same entity as an entity already in the knowledge graph. Based on the entity linking results, the knowledge extraction results and the event extraction results are recombined to obtain graph knowledge for the knowledge graph.

[0127] In one embodiment, entity linking processing includes: comparing entities and attributes in the knowledge extraction results with entities and attributes in the existing knowledge graph to determine whether they are the same entity, and comparing entities and attributes in the event extraction results with entities and attributes in the existing knowledge graph to determine whether they are the same entity. In other words, the entity linking of the present invention is not only for the triples or quintuples extracted from knowledge, but also needs to determine whether the entity type parameters in the event extraction results are the same entity as the entities in the existing triples or quintuples.

[0128] In one embodiment, the entity linking process uses an unsupervised entity alignment algorithm, which uses relationship and attribute triples for entity linking. The algorithm first uses attribute triples to perform iterative entity alignment, and the generated alignment results can be used as training data for the embedding model. The embedding model uses relationship triples to obtain another part of the alignment results. Finally, the two parts of the alignment results are merged through the regression model as the final entity linking result.

[0129] In one embodiment, entity linking processing of knowledge extraction results and event extraction results may include the following steps:

[0130] In step a, the entity data in the knowledge base is preprocessed. According to the type of the object, the triples in the knowledge graph are divided into attribute triples and relationship triples. For the attribute triples, rules are formulated to normalize the attribute values.

[0131] In step b, attribute triples are used to perform entity alignment.

[0132] In step c, entity alignment is performed using relation triples.

[0133] In step d, the regression model is used to merge the two parts of the results: the results of the iterative model are used as training data, and the regression model is used to learn a certain weight, and the two parts of the results are merged to obtain the final entity alignment result, rather than directly manually specifying the weight and directly merging the two result sets. This is done to better utilize the distribution characteristics of the relationships and attributes of the entities in each data set. Intuitively, since the number and quality of relationships and attributes owned by entities in different data sets are different, it is unreasonable to set fixed weights.

[0134] In one embodiment, entity alignment using attribute triples may include: calculating the similarity of attribute values ​​of the same attribute between two entities (for example, edit distance, cosine distance, etc. may be used), adding and averaging all attribute values ​​under the same attribute, and using the obtained average attribute value similarity to measure the similarity between the two entities, thereby obtaining an entity alignment result.

[0135] Since different knowledge graphs have too few identical attributes, many attribute pairs with inconsistent expressions but the same meaning can be found based on the aligned entity pairs. These new attribute pairs can be used to find more entity pairs. Therefore, in one embodiment, when performing entity alignment, attribute alignment is also performed to find more entity pairs. In other words, the two tasks of entity alignment and attribute alignment are allowed to interact with each other, and the two tasks are bundled together and executed iteratively to reduce the problems caused by the diversity of attribute names to entity alignment.

[0136] In one embodiment, entity alignment using relation triples may include: setting a relation threshold, screening out high-quality entity pairs from the results of entity alignment using attribute triples as training data for a model (e.g., an embedding model) that uses relation triples to align entities, thereby obtaining other entity alignment results.

[0137] In one embodiment, a regression model can be used to merge the entity alignment results of the two parts to obtain an entity linking result. Using a regression model for merging can better utilize the distribution characteristics of the relationship and attributes of the entities in each data set, thereby making the entity linking result more reasonable and accurate.

[0138] In one embodiment, knowledge fusion may also include: inferring new graph knowledge based on the graph knowledge obtained above and / or the existing knowledge in the knowledge graph, and updating the knowledge graph with the new graph knowledge. The process of using the existing knowledge (triplets) in the graph to obtain some new relationships between entities or attributes of entities (triplets) is called knowledge reasoning. For example, there are two triplets in the knowledge graph: <Zhang San, wife, Li Si>, and <Zhang San, mother, Wang Wu>. Through knowledge reasoning, <Li Si, mother-in-law, Wang Wu> can be obtained.

[0139] Exemplarily, any reasoning method known in the art may be used to infer new graph knowledge, such as methods based on Tableaux operations, methods based on logic programming rewriting, methods based on first-order query rewriting, methods based on production rules, etc., and the present invention is not limited to this.

[0140] In one embodiment, the knowledge graph construction method 200 may further include a knowledge cleaning step for correcting or removing redundant or erroneous information. The knowledge cleaning step may include merging redundant expressions, correcting and removing obvious errors, inferring similar entities and supplementing attribute names, merging synonymous attribute names, etc., which are not limited in the present invention.

[0141] In one embodiment, any model or neural network known in the art may be used for knowledge cleaning, and the present invention is not limited to this.

[0142] In one embodiment, the merging of redundant expressions can be achieved by using an attribute value information parsing model. For example, taking the place of household registration as an example, the attribute value information parsing model can be used to first parse out information such as province, city, district (county), township, village, community, residential building, household number, etc., and then merge expressions with high repetition. Among them, the attribute value information parsing model usually includes two parts: entity boundary recognition and entity classification, wherein entity boundary recognition is used to determine whether a string constitutes a complete entity, and entity classification is used to divide the recognized entities into different preset categories.

[0143] Among them, obvious error correction and removal can set verification rules for some attribute value data with clear restrictions such as format requirements and value ranges to remove attribute value data that do not meet the verification rules. For example, a domestic mobile phone number has 11 digits. If the mobile phone number is 1336836, which is obviously erroneous data, the data can be corrected or removed.

[0144] In one embodiment, a graph neural network with inductive reasoning capabilities, such as a Graph Inductive Learning (GraIL) model, may be used to infer similar entities and supplement attribute names, but the present invention is not limited to this.

[0145] The following briefly describes an example process of using the GraIL model to infer similar entities and supplement attribute names.

[0146] First, the GraIL model extracts an envelope subgraph around the fact to be judged, that is, extracts k-hop neighborhood subgraphs for the first and last nodes in the fact respectively, and takes the intersection of the obtained first and last neighborhood subgraphs to obtain an envelope graph around the fact.

[0147] Secondly, the extracted envelope subgraph structure is used to encode the features of the nodes in the subgraph. Here, the feature representation is obtained by measuring the distance between each point in the subgraph and the target node. For example, for a point i in the subgraph with target nodes u and v, its feature is represented by a tuple (d(i,u),d(i,v)), where d(.,.) represents the shortest distance between the two points. In particular, points u and v are represented by (0,1) and (1,0) respectively.

[0148] Finally, the facts to be judged are evaluated, and the GraIL model reasoning on the envelope subgraph and scores the rationality of the facts.

[0149] Since the whole process does not require pre-training of the features of the nodes in the graph, GraIL has the characteristics of inductive learning.

[0150] When training the network parameters of the GraIL model, the example loss function is designed as follows:

[0151]

[0152] Among them, ni and pi represent negative samples and positive samples respectively. Positive samples come from the training data set, and negative samples are generated by randomly replacing the head and tail entities of positive samples. The higher the score of the negative sample and the lower the score of the positive sample, the greater the loss. The ultimate goal of optimization is to make all reasonable facts score high and unreasonable facts score low, even if the GraIL model has the ability to judge the correct facts.

[0153] When using the trained GraIL model to implement reasoning about the target attribute, a candidate set can be generated based on the target attribute to be predicted, and the probability of each fact in the candidate set is scored. The attribute in the fact with the highest score is the predicted attribute.

[0154] The essence of merging synonymous attribute names is to find attribute names with the same meaning, that is, text similarity test. In one embodiment, the Sentence-BERT (SBERT) model can be used to merge synonymous attribute names. The model structure uses twin networks and triplet network structures to generate sentence embedding vectors with semantic meaning. For sentences with similar semantics, the distance between their embedding vectors is relatively close, so that they can be used to calculate similarity (cosine similarity, Manhattan distance, Euclidean distance). When searching for the most similar sentence pairs, the model structure has a computational overhead of as low as 5 seconds (calculating cosine similarity is about 0.01s), and the accuracy can remain unchanged. In this way, the SBERT model can complete certain new specific tasks, such as similarity comparison, clustering, and semantic-based information retrieval.

[0155] For fine-tuning, the sentence BERT model can use twin networks and triplet networks to update weight parameters so that the generated sentence vectors have semantic meaning.

[0156] The sentence BERT model depends on the specific training data. In one embodiment, the following objective functions may be adopted in this article: classification objective function, regression objective function or triple objective function.

[0157] Among them, when using the regression objective function or the triple objective function, given a main sentence a, a positive sentence p and a negative sentence n, the triple loss adjusts the network so that the distance between a and p is less than the distance between a and n. Mathematically, we minimize the following loss function:

[0158] max(||s a -s p ||-||s a -s n ||+∈,0)

[0159] s represents the sentence embedding vector of a, p, and n, ||·|| represents the distance, and the edge parameter ε represents s p With s a The distance is at least greater than s n close.

[0160] In another embodiment, the present invention provides a knowledge graph construction system. Figure 4 , Figure 4 FIG. 4 shows a schematic structural block diagram of a knowledge graph construction system 400 according to an embodiment of the present invention. Figure 4As shown, the knowledge graph construction system 400 may include a composite knowledge extraction module 410 and a composite knowledge fusion module 420. The composite knowledge extraction module 410 is used to perform chapter-level knowledge extraction and sentence-level event extraction on the text obtained as raw data to obtain knowledge extraction results and event extraction results. The composite knowledge fusion module 420 is used to perform knowledge fusion on the knowledge extraction results and the event extraction results to obtain a knowledge graph.

[0161] Those skilled in the art can understand the specific implementation method of the knowledge graph construction system 400 according to an embodiment of the present invention in combination with the contents described above. For the sake of brevity, the specific details are not repeated here.

[0162] In yet another embodiment, the present invention provides a computing device. Figure 5 , Figure 5 A schematic structural block diagram of a computing device 500 according to an embodiment of the present invention is shown. As shown in FIG5 , the computing device 500 may include a memory 510 and a processor 520, wherein the memory 510 stores a computer program, and when the computer program is executed by the processor 520, the processor 520 executes the knowledge graph construction method 200 as described above.

[0163] Those skilled in the art can understand the specific operation of the computing device 500 according to the embodiment of the present invention in combination with the above contents. For the sake of brevity, the specific details are not repeated here, and only some main operations of the processor 520 are described as follows:

[0164] Acquire a text as raw data, perform paragraph-level knowledge extraction and sentence-level event extraction on the text to obtain a knowledge extraction result and an event extraction result; and

[0165] The knowledge extraction result and the event extraction result are fused to obtain a knowledge graph.

[0166] In another embodiment, the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program executes the knowledge graph construction method 200 as described in the above embodiment when running. Any tangible, non-temporary computer-readable medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROMs, DVDs, Blu-ray disks, etc.), flash memory, and / or the like. These computer program instructions may be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing device to form a machine, so that these instructions executed on a computer or other programmable data processing device may generate a device that implements a specified function. These computer program instructions may also be stored in a computer-readable memory, which may instruct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory may form a manufactured product, including an implementation device that implements a specified function. The computer program instructions may also be loaded onto a computer or other programmable data processing device, so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device may provide steps for implementing a specified function.

[0167] The beneficial effects of the present invention are as follows:

[0168] (1) The present invention changes the concept of knowledge graph from traditional static knowledge triples to composite knowledge including static knowledge quintuples and dynamic event knowledge. By using chapter-level element extraction and element association, the extraction efficiency and recall rate of quintuples can be greatly improved, making the knowledge more structured.

[0169] (2) Through the entity parameters in the event extraction results, entity linking and cross-text event co-reference resolution are performed to associate events with entity information to form a composite knowledge graph.

[0170] (3) The present invention has a complete set of automated knowledge cleaning and updating processes, which can effectively improve the quality of the final knowledge graph.

[0171] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. Various changes and modifications may be made therein by one of ordinary skill in the art without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as required by the appended claims.

[0172] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0173] Similarly, it should be understood that in order to streamline the present invention and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present invention should not be interpreted as reflecting the following intention: the claimed invention requires more features than the features explicitly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present invention.

[0174] It will be understood by those skilled in the art that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstract and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0175] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0176] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0177] The above is only a specific embodiment of the present invention or an explanation of a specific embodiment. The protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A knowledge graph construction method, It is characterized in that The method comprises: Acquire a text as raw data, perform paragraph-level knowledge extraction and sentence-level event extraction on the text to obtain a knowledge extraction result and an event extraction result; and Performing knowledge fusion on the knowledge extraction result and the event extraction result to obtain a knowledge graph, The chapter-level knowledge extraction of the text includes: Based on a preset category label, using a sliding window to perform knowledge extraction on the text to extract knowledge elements in the text corresponding to the category label; The extracted knowledge elements are associated to obtain the knowledge extraction result.

2. The method according to claim 1, It is characterized in that The use of a sliding window to extract knowledge from the text includes: Slide the sliding window over the text, and calculate the predicted probability of each word in the text within the sliding window for each category label; Calculate the average of the prediction probabilities of the same word at each position in the sliding window as the final prediction probability of the word; Based on the final prediction probability of each word, the knowledge element is selected from the text according to a preset knowledge element selection condition.

3. The method according to claim 1, It is characterized in that The association of the extracted knowledge elements includes: Processing each word of the knowledge element to obtain a word vector of each word; Processing the word vectors of all words corresponding to the knowledge element to obtain the element vector of the knowledge element; Perform association judgment on the element vectors of each knowledge element to obtain the association results for each knowledge element; The knowledge elements are combined according to the association results to obtain the knowledge extraction result.

4. The method according to claim 1, It is characterized in that Sentence-level event extraction of the text includes: Predicting whether each word in each sentence of the text is a trigger word and its trigger word type based on a preset event trigger word label; Using the predicted trigger word, trigger word type and the position of the trigger word, predict the event parameter and its parameter type corresponding to the trigger word in the sentence based on the preset event parameter label; The trigger word and the event parameter are combined to obtain the event extraction result.

5. The method according to claim 1, It is characterized in that The knowledge fusion of the knowledge extraction result and the event extraction result includes: Performing entity linking processing on the knowledge extraction result and the event extraction result to obtain an entity linking result, wherein the entity linking result indicates whether each entity in the knowledge extraction result and the event extraction result is the same entity, and whether each entity is the same entity as an existing entity in the knowledge graph; The knowledge extraction result and the event extraction result are recombined based on the entity linking result to obtain graph knowledge for the knowledge graph.

6. The method according to claim 5, It is characterized in that The step of fusing the knowledge extraction result and the event extraction result further includes: Inferring new graph knowledge based on the graph knowledge and / or existing knowledge in the knowledge graph; Update the knowledge graph with the new graph knowledge.

7. The method according to claim 4, It is characterized in that The method further comprises: Performing in-text reference chain recognition on the event extraction results to determine whether the event extraction results of the text point to the same event according to a preset in-text event co-reference feature type; Performing cross-chapter event coreference resolution processing on the event extraction result to determine whether the event extraction result of the text and the existing event in the knowledge graph point to the same event according to a preset cross-chapter event coreference feature type; Based on the determination, duplicate events are removed.

8. The method according to claim 1, It is characterized in that The method also includes a knowledge cleaning step for correcting or removing redundant or erroneous information, wherein the knowledge cleaning step includes merging redundant expressions, correcting and removing obvious errors, inferring similar entities and supplementing attribute names, and merging synonymous attribute names.

9. A knowledge graph construction system, It is characterized in that The system comprises: A composite knowledge extraction module is used to extract chapter-level knowledge and sentence-level events from the text obtained as raw data to obtain knowledge extraction results and event extraction results; A composite knowledge fusion module is used to fuse the knowledge extraction results and the event extraction results to obtain a knowledge graph. The chapter-level knowledge extraction of the text includes: Based on a preset category label, using a sliding window to perform knowledge extraction on the text to extract knowledge elements in the text corresponding to the category label; The extracted knowledge elements are associated to obtain the knowledge extraction result.

10. A computing device, It is characterized in that The computing device comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the method according to any one of claims 1 to 8.

11. A computer readable medium, It is characterized in that The computer readable medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 8 is executed.

Images