Efficient natural language knowledge storage method based on knowledge decoupling and semantic representation

Abstract

The invention discloses an efficient natural language knowledge storage method based on knowledge decoupling and semantic representation, and belongs to the field of natural language processing. According to the method, firstly, complex knowledge sentences are decoupled into simple knowledge sentences, then, the simple knowledge sentences are decoupled into knowledge tuples, and the knowledge tuples are subjected to hierarchical knowledge tuple storage according to the appearance sequence of the knowledge tuples in original sentences. By adopting the method, the knowledge in the large-scale natural language text can be efficiently and quickly stored, and tight coupling does not exist among the knowledge. Compared with DrKIT, the remote supervision time is saved by 100%, and compared with knowledge storage in the form of questions and answers, the storage space is saved by 147%.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing and provides a natural language knowledge decoupled storage technology. Background Art

[0002] Natural language text corpora (such as sentences from Wikipedia) contain a large amount of coupled complex factual knowledge. Compared with knowledge graphs in triple form, natural language text corpora do not need to be manually constructed and are more suitable for combination with existing natural language processing models. In order to utilize the knowledge in natural language text, a series of existing studies focus on how to represent and store the factual knowledge in natural language text. The prior art DrKIT uses an existing knowledge graph, annotates the valid entities and their contexts therein in a remote supervision manner, and encodes and stores them; while OPQL stores the knowledge in natural language by encoding and indexing a set of relation mentions, which can perform reasoning naturally and can be trained without any structured supervision; Chen et al. represent natural language text knowledge in the form of simple question and answer pairs, and the stored content is the questions and answers. However, the prior art has low efficiency in representing and storing knowledge when storing natural language-form knowledge, resulting in large storage overhead. Summary of the Invention

[0003] The present invention aims to solve the problem of low efficiency in storing natural language knowledge and provides an efficient storage method for natural language knowledge based on knowledge decoupling and semantic representation.

[0004] The method of the present invention includes the following steps:

[0005] A. Decouple complex knowledge sentences into simple knowledge sentences, and the implementation method is as follows:

[0006] A1. Use a semantic role labeling model to perform semantic role labeling on each complex sentence;

[0007] A2. Take each predicate-centered component obtained by semantic role labeling as a simple knowledge sentence;

[0008] A3. Use a heuristic method to delete the parts with common expressions in the simple knowledge sentences;

[0009] B. Decouple the simple knowledge sentences obtained in step A into knowledge tuples, and arrange these knowledge tuples in the order they appear in the original sentence;

[0010] C. Hierarchical knowledge tuple storage, and the implementation method is as follows:

[0011] C1. Identify the named entities in each simple knowledge sentence in step A, and separately split the named entities as a knowledge tuple, and replace each named entity with a special symbol [MASK];

[0012] C2. Index each named entity in the knowledge tuple to all the simple knowledge sentences in which it appears. The indexed simple knowledge sentences are represented by the arrangement of the knowledge tuples, and each simple knowledge sentence is further indexed to the number of the original complex knowledge sentence.

[0013] C3. Encode each knowledge tuple with a pre-trained model and store each simple knowledge sentence in the knowledge base in the form of "entity - knowledge tuple encoding combination".

[0014] As a preferred solution, the semantic role labeling model in step A1 can use the semantic role labeling model V2.10.0 of AllenNLP.

[0015] As a preferred solution, in step B, for clauses containing special words such as who, that, where, when, or which, if the word before the special word is a noun or a pronoun, then delete the special word and its related clause part.

[0016] As a preferred solution, the named entity recognition in step C1 can be implemented using the NER model in the SpaCy library.

[0017] As a preferred solution, the pre-trained model in step C3 can adopt SimCSE to obtain the overall vector representation for each knowledge tuple.

[0018] Advantages of the present invention: By using the storage method of natural language knowledge provided by the present invention, the efficiency of representing and storing natural language knowledge is improved. Compared with DrKIT, the present invention saves 100% of the remote supervision time, and compared with storing knowledge in the form of questions and answers, the present invention saves 147% of the storage space. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of a specific embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] Define terms:

[0021] 1) Predicate: A term used to replace or show the nature, characteristics of its object, or the relationship between objects.

[0022] 2) Named entity: An entity with a specific meaning in the text, mainly including personal names, place names, organization names, proper nouns, etc., as well as words such as time, quantity, currency, and proportional values. It refers to things that can be identified by proper nouns (names). A named entity generally represents a single specific thing individual, including personal names, place names, etc.

[0023] 3) Simple knowledge sentence: A sentence whose all factual knowledge can be represented by a single <head entity, relation, tail entity> triple.

[0024] 4) Complex knowledge sentence: A sentence whose all factual knowledge requires more than one <head entity, relation, tail entity> triple to represent.

[0025] 5) Knowledge tuple: The smallest unit that can express independent semantics in a simple knowledge sentence.

[0026] Such as Figure 1 shown, for the instance "Ernest Miller Hemingway, born on July 21, 1899, was an American novelist who won the Nobel Prize in Literature in 1954.",

[0027] A. First, decouple the complex knowledge sentence into simple knowledge sentences. The semantic role labeling model can be directly used to obtain Figure 1 the three clauses on the left. The corresponding central predicates are "born", "was", and "won" respectively. Using a heuristic method to delete the parts commonly expressed by multiple simple sentences, that is, the component "who won the Nobel Prize in Literature in 1954" in the second simple knowledge sentence can be deleted, and three non-redundant and non-missing simple knowledge sentences can be obtained, namely: (1) Ernest Miller Hemingway born on July 21, 1899; (2) Ernest Miller Hemingway was an American novelist; (3) Ernest Miller Hemingway won the Nobel Prize in Literature in 1954.

[0028] B. Decouple the above simple knowledge sentences into a set of knowledge tuples, that is, for each predicate "born", "was", and "won", each related sentence component is regarded as a knowledge tuple, and these tuples are arranged in the order they appear in the original sentence; three sets of simple knowledge tuples can be obtained: (1) <Ernest Miller Hemingway|born on|July 21, 1899>; (2) <Ernest Miller Hemingway|was|an|American|novelist>; (3) <Ernest Miller Hemingway|won|the Nobel Prize|in Literature|in 1954>.

[0029] C. Input each simple knowledge sentence into the NER model in SpaCy, record the named entities belonging to ['PERSON', 'EVENT', 'FAC', 'LOC', 'WORK_OF_ART', 'GPE', 'NORP', 'LANGUAGE', 'LAW'], and replace the entities with the special symbol [MASK]. According to the named entities in the knowledge tuples of the simple knowledge sentences, first index each named entity to all the simple knowledge sentences in which it appears. The indexed simple knowledge sentences are represented by the arrangement of knowledge tuples, and each simple knowledge sentence is then indexed to the number of the original complex knowledge sentence. The pre-trained model encodes each knowledge tuple, and stores each simple knowledge sentence in the knowledge base in the form of "entity-knowledge tuple encoding combination", where each knowledge tuple is represented as a vector form by SimCSE. In this way, the knowledge in large-scale natural language texts can be stored efficiently and quickly, and there is no tight coupling between the knowledge.

[0030] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to the embodiments without departing from the principles and purposes of the present invention. The scope of the present invention is defined by the claims and their equivalents.

Claims

1. An efficient storage method for natural language knowledge based on knowledge decoupling and semantic representation, characterized in that, It includes the following steps: A. Decouple complex knowledge sentences into simple knowledge sentences. The implementation method is as follows: A1. Use a semantic role labeling model to perform semantic role labeling on each complex sentence; A2. Take each predicate-centered component obtained from semantic role labeling as a simple knowledge sentence; A3. Use a heuristic method to delete the parts that are commonly expressed in the simple knowledge sentences; B. Decouple the simple knowledge sentences obtained in step A into knowledge tuples, and arrange these knowledge tuples in the order they appear in the original sentence; C. Hierarchical storage of knowledge tuples. The implementation method is as follows: C1. Identify the named entities in each simple knowledge sentence in step A, and separately split each named entity as a knowledge tuple, and replace each named entity with a special symbol [MASK]; C2. According to the named entities in the knowledge tuples, index each named entity to all the simple knowledge sentences in which it appears. The indexed simple knowledge sentences are represented by the arrangement of knowledge tuples, and each simple knowledge sentence is further indexed to the number of the original complex knowledge sentence; C3. Encode each knowledge tuple with a pre-trained model, and store each simple knowledge sentence in the knowledge base in the form of "entity-knowledge tuple encoding combination".

2. The storage method of efficient natural language knowledge based on knowledge decoupling and semantic representation according to claim 1, characterized in that The semantic role labeling model in step A1 uses the semantic role labeling model V2.10.0 of AllenNLP.

3. The storage method of efficient natural language knowledge based on knowledge decoupling and semantic representation according to claim 1, characterized in that, In step B, for clauses containing special words such as who, that, where, when or which, if the word before the special word is a noun or a pronoun, then delete the special word and its related clause part.

4. The storage method of efficient natural language knowledge based on knowledge decoupling and semantic representation according to claim 1, characterized in that The named entity recognition in step C1 is implemented using the NER model in the SpaCy library.

5. The storage method of efficient natural language knowledge based on knowledge decoupling and semantic representation according to claim 1, characterized in that, The pre-trained model in step C3 uses SimCSE to obtain an overall vector representation for each knowledge tuple.

Images