A method, system, device and medium for automatically constructing a knowledge graph
The method enhances data sets and uses a BERT-BiLSTM-CRF model for efficient and accurate construction of vertical domain knowledge graphs, overcoming data scarcity and annotation challenges through text enhancement and pre-trained model updates.
Patent Information
- Application Number
- CN202310571923.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-19
- Publication Date
- 2024-12-20
- Estimated Expiration
- 2043-05-19
AI Technical Summary
Existing methods for constructing vertical domain knowledge graphs face challenges due to insufficient data sets, labor-intensive manual labeling, and the need for tailored models, making it difficult to achieve efficient and accurate entity extraction.
A method involving knowledge modeling, data set division, text enhancement, and training a BERT-BiLSTM-CRF model for named entity recognition, followed by triple mapping to construct knowledge graphs, using pre-trained models for dynamic updates.
Enhances data sets through text enhancement, improves model training efficiency and accuracy, and allows for dynamic updates to adapt to changing vertical field data, addressing data resource scarcity and annotation insufficiencies.
Smart Images
Figure CN116541535B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of knowledge graph information extraction, and in particular to a method, system, device and medium for automatically constructing a knowledge graph. Background Art
[0002] There are two main methods for constructing knowledge graphs in existing vertical fields: one is manual construction and the other is automatic construction. The automatic construction method mainly includes two steps: knowledge modeling and information extraction. Information extraction is the key to automatic construction of knowledge graphs. Information extraction includes named entity recognition (i.e. entity extraction) and entity relationship extraction (i.e. relationship extraction).
[0003] Named entity recognition technology mainly includes rule-based and dictionary-based methods, statistical-based methods, deep learning-based methods, etc. Rule-based and dictionary-based methods are early methods of named entity recognition. They rely on manually customized rules and a complete entity library, which are prone to rule conflicts, and it is time-consuming and labor-intensive to formulate rules and dictionaries. Statistical-based methods include hidden Markov models, maximum entropy models, support vector machine models, conditional random fields, etc. Statistical-based methods rely heavily on corpora, so the construction of corpora in vertical fields that can be used to build and evaluate named entity recognition models is a major factor restricting this method. With the continuous improvement of computer performance, machine learning methods have been widely used in the field of natural language processing, including recurrent neural networks (RNN), long short-term memory networks (LSTM), gated recurrent neural networks (GRU), etc. Named entity recognition technology based on deep learning has become the mainstream method for entity extraction. In addition, pre-trained models represented by the BERT model have become the latest paradigm of deep learning in recent years. Pre-trained models are formed by pre-training a large amount of text data, and then high-quality data is used for model fine-tuning to improve model training efficiency and accuracy. This method is widely used in the field of natural language processing, including information extraction from general domain knowledge graphs.
[0004] Different from general domain knowledge graphs, the construction of vertical domain knowledge graphs has higher requirements on data sources and data quality. However, there is a certain gap in data volume and data sets compared with general fields. In order to meet different field problems, different disciplines, and different task requirements, it is often necessary to train new entity extraction models. Therefore, it is difficult to ensure sufficient data sets to train the models. The data used to train the models often rely on manual labeling, which is time-consuming and labor-intensive. Summary of the invention
[0005] The purpose of the present invention is to provide a method, system, device and medium for automatically constructing a knowledge graph to realize the automatic construction of a vertical field knowledge graph.
[0006] To achieve the above object, the present invention provides the following solutions:
[0007] A method for automatically constructing a knowledge graph, comprising:
[0008] Performing knowledge modeling on the target vertical field to obtain a knowledge model of the target vertical field; the knowledge model includes entity types and entity relationships; the entity relationships represent entities and relationships between entities;
[0009] Acquire an initial data set, and divide the initial data set into large sample data and small sample data; the large sample data is text data in the initial data set whose number of characters is greater than a first set value; the small sample data is text data in the initial data set whose number of characters is greater than a third set value and less than or equal to a second set value; the first set value is greater than the second set value, and the second set value is greater than the third set value;
[0010] Performing data labeling on the small sample data to obtain labeled small sample data, and determining an entity dictionary based on the labeled small sample data;
[0011] According to the entity dictionary, text enhancement is performed on the large sample data to obtain text-enhanced large sample data;
[0012] Constructing a text dataset based on the annotated small sample data and the text-enhanced large sample data;
[0013] Training a named entity recognition model according to the text data set; the named entity recognition model is a BERT-BiLSTM-CRF model;
[0014] Acquire target text data, use the trained named entity recognition model to perform entity extraction on the target text data, and perform triple mapping on the extraction results according to the knowledge model to obtain triple instances;
[0015] Construct a knowledge graph for the target vertical field based on the triple instances.
[0016] Optionally, data labeling is performed on the small sample data to obtain labeled small sample data, and an entity dictionary is determined according to the labeled small sample data, specifically including:
[0017] The small sample data is annotated using the BIO annotation method to obtain annotated small sample data; wherein B represents the starting character of the entity, I represents the middle and ending characters of the entity, and O represents a non-entity character;
[0018] Entities are extracted from the annotated small sample data to obtain an entity dictionary; the entity dictionary includes entity names and corresponding entity types in the small sample data.
[0019] Optionally, performing text enhancement on the large sample data according to the entity dictionary to obtain text-enhanced large sample data specifically includes:
[0020] Performing word segmentation and stop word removal processing on the large sample data to obtain pre-processed large sample data;
[0021] Calculating semantic similarity between the preprocessed large sample data and each entity name in the entity dictionary to obtain a similarity value corresponding to the preprocessed large sample data;
[0022] Data annotation is performed on the preprocessed large sample data whose similarity value is greater than a set similarity threshold to obtain text-enhanced large sample data.
[0023] Optionally, training a named entity recognition model according to the text dataset specifically includes:
[0024] Dividing the text data set into a number of sample subsets of equal size;
[0025] The K-fold cross-validation method is adopted to train and verify the named entity recognition model according to the sample subset to obtain a trained named entity recognition model.
[0026] Optionally, using a trained named entity recognition model to perform entity extraction on the target text data, and performing triple mapping on the extraction results according to the knowledge model to obtain triple instances, specifically including:
[0027] Using the trained named entity recognition model to extract entities from the target text data sentence by sentence to obtain extraction results; the extraction results include entity names and corresponding entity types in the target text data;
[0028] The entity names in the extraction results are triple-mapped according to the corresponding entity types and the entity relationships in the knowledge model to obtain triple instances; the triple instances include the starting entity name, the ending entity name and the relationship between the starting entity and the ending entity.
[0029] Optionally, it also includes:
[0030] Saving the trained named entity recognition model as a pre-trained model;
[0031] Using the target text data and the corresponding extraction results as a first expanded data set;
[0032] When the data volume of the first expanded data set is greater than a set amount, the pre-trained model is trained according to the first expanded data set to update the trained named entity recognition model.
[0033] Optionally, it also includes:
[0034] When there is a new manually annotated data set, using the new manually annotated data set as a second expanded data set;
[0035] When the data volume of the second expanded data set is greater than a set amount, the pre-trained model is trained according to the second expanded data set to update the trained named entity recognition model.
[0036] A knowledge graph automatic construction system, comprising:
[0037] A knowledge modeling module is used to perform knowledge modeling on a target vertical field to obtain a knowledge model of the target vertical field; the knowledge model includes entity types and entity relationships; the entity relationships represent entities and relationships between entities;
[0038] A data set division module is used to obtain an initial data set and divide the initial data set into large sample data and small sample data; the large sample data is text data in the initial data set whose number of characters is greater than a first set value; the small sample data is text data in the initial data set whose number of characters is greater than a third set value and less than or equal to a second set value; the first set value is greater than the second set value, and the second set value is greater than the third set value;
[0039] A small sample data processing module is used to perform data labeling on the small sample data to obtain the labeled small sample data, and determine an entity dictionary based on the labeled small sample data;
[0040] A large sample data processing module, used for performing text enhancement on the large sample data according to the entity dictionary to obtain text enhanced large sample data;
[0041] A text data set construction module, used to construct a text data set based on the annotated small sample data and the text enhanced large sample data;
[0042] A model training module, used for training a named entity recognition model according to the text data set; the named entity recognition model is a BERT-BiLSTM-CRF model;
[0043] A triple instance generation module is used to obtain target text data, perform entity extraction on the target text data using a trained named entity recognition model, and perform triple mapping on the extraction results according to the knowledge model to obtain a triple instance;
[0044] A knowledge graph construction module is used to construct a knowledge graph of the target vertical field based on the triple instances.
[0045] An electronic device includes a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned method for automatically constructing a knowledge graph.
[0046] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned method for automatically constructing a knowledge graph.
[0047] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:
[0048] The automatic knowledge graph construction method provided by the present invention divides the initial data set into small sample data and large sample data, and performs text enhancement on the large sample data based on the manually annotated small sample data to obtain text-enhanced large sample data. A sufficient amount of data can be obtained to train a named entity recognition model, thereby overcoming difficulties such as complex vertical field knowledge, difficulty in obtaining data resources, and insufficient data set annotations, thereby improving the efficiency of model training set preparation and the accuracy of entity extraction. By using the above data to train a named entity recognition model, using the trained named entity recognition model to extract entities from the target text data, and performing triple mapping on the extraction results according to a pre-established knowledge model, the automatic construction of a knowledge graph for a vertical field can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0050] Figure 1 A flowchart of the method for automatically constructing a knowledge graph provided by the present invention;
[0051] Figure 2 A specific flow chart of the method for automatically constructing a knowledge graph provided by the present invention;
[0052] Figure 3 An example diagram of knowledge modeling provided by the present invention;
[0053] Figure 4 An example diagram of the small sample data annotation provided by the present invention;
[0054] Figure 5A structural diagram of the named entity recognition model provided by the present invention;
[0055] Figure 6 An example diagram of triple mapping provided by the present invention;
[0056] Figure 7 This is a module diagram of the knowledge graph automatic construction system provided by the present invention. DETAILED DESCRIPTION
[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0058] The purpose of the present invention is to provide a method, system, device and medium for automatically constructing a knowledge graph to realize the automatic construction of a vertical field knowledge graph.
[0059] Specifically, in order to solve the problem of insufficient training data sets for named entity recognition models for vertical domain knowledge graphs, the present invention provides a method, system, device and medium for automatically constructing knowledge graphs based on text enhancement and pre-trained models, which mainly involves named entity recognition technology, and the relationship between entities is obtained by mapping the pattern layer established in knowledge modeling. On the one hand, the present invention expands the data set through text enhancement and improves the extraction accuracy of the pre-trained model; on the other hand, the present invention also uses the trained model as a pre-trained model, constructs a new training data set after annotating the new data, and trains the new model on this basis to realize dynamic update of the named entity recognition model to ensure that the model can be updated with the update of the data.
[0060] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0061] Embodiment 1
[0062] The embodiment of the present invention provides a method for automatically constructing a knowledge graph. Figure 1 As shown, the method includes:
[0063] Step 101: Perform knowledge modeling on the target vertical field to obtain a knowledge model of the target vertical field; the knowledge model includes entity types and entity relationships; the entity relationships represent entities and relationships between entities.
[0064] Step 102: Obtain an initial data set, and divide the initial data set into large sample data and small sample data; the large sample data is text data in the initial data set whose number of characters is greater than a first set value; the small sample data is text data in the initial data set whose number of characters is greater than a third set value and less than or equal to a second set value; the first set value is greater than the second set value, and the second set value is greater than the third set value.
[0065] Step 103: annotate the small sample data to obtain annotated small sample data, and determine an entity dictionary based on the annotated small sample data.
[0066] Preferably, the BIO annotation method is used to annotate the small sample data to obtain annotated small sample data; wherein B represents the starting character of the entity, I represents the middle and ending characters of the entity, and O represents non-entity characters; entities are extracted from the annotated small sample data to obtain an entity dictionary; the entity dictionary includes the entity names and corresponding entity types in the small sample data.
[0067] Step 104: Perform text enhancement on the large sample data according to the entity dictionary to obtain text enhanced large sample data.
[0068] Preferably, the large sample data is segmented and stop words are removed to obtain preprocessed large sample data; semantic similarity is calculated between the preprocessed large sample data and each entity name in the entity dictionary to obtain a similarity value corresponding to the preprocessed large sample data; data annotation is performed on the preprocessed large sample data whose similarity value is greater than a set similarity threshold to obtain text-enhanced large sample data.
[0069] Step 105: construct a text dataset based on the annotated small sample data and the text enhanced large sample data.
[0070] Step 106: training a named entity recognition model according to the text data set; the named entity recognition model is a BERT-BiLSTM-CRF model.
[0071] Preferably, step 106 specifically includes: dividing the text data set into a number of sample subsets of equal size; using a K-fold cross-validation method to train and verify the named entity recognition model according to the sample subsets to obtain a trained named entity recognition model.
[0072] Step 107: Acquire target text data, use the trained named entity recognition model to perform entity extraction on the target text data, and perform triple mapping on the extraction results according to the knowledge model to obtain triple instances.
[0073] Preferably, step 107 specifically includes: acquiring target text data; performing entity extraction on the target text data sentence by sentence using a trained named entity recognition model to obtain extraction results; the extraction results include entity names and corresponding entity types in the target text data; performing triple mapping on the entity names in the extraction results according to the corresponding entity types and entity relationships in the knowledge model to obtain triple instances; the triple instance includes a starting entity name, an ending entity name, and a relationship between the starting entity and the ending entity.
[0074] Step 108: Construct a knowledge graph of the target vertical field based on the triple instances.
[0075] Furthermore, the above method also includes: saving the trained named entity recognition model as a pre-trained model; using the target text data and the corresponding extraction results as a first extended data set; when the data amount of the first extended data set is greater than a set amount, training the pre-trained model according to the first extended data set to update the trained named entity recognition model.
[0076] Furthermore, the above method also includes: when there is a new manually annotated data set, using the new manually annotated data set as a second extended data set; when the data amount of the second extended data set is greater than a set amount, training the pre-trained model according to the second extended data set to update the trained named entity recognition model.
[0077] The following takes the construction of process knowledge graph as an example to explain the above steps in detail. Figure 2 .
[0078] Step S1: Knowledge modeling. Construct the domain and knowledge requirements based on the knowledge graph, sort out and classify the vertical domain knowledge, and define the types and attributes of entities and relationships.
[0079] like Figure 3 As shown in the figure, the specific method of knowledge modeling is as follows:
[0080] (1) According to the knowledge graph, the domain and knowledge requirements are constructed, the vertical domain knowledge is sorted and classified, and the classified entity types are saved in the form of labels, that is, L = {Label 1 ,Label 2 ,…,Label n}, where Label 1 ~Label n Indicates the tag name, which contains n tags in total.
[0081] (2) Define the types and attributes of entities and relationships, determine the relationships between different tags and the attributes contained in each tag, and save them in the form of triples, that is,<S,R,E> , where S represents the starting entity, S∈L, and contains M attributes, denoted as P s ={p 1 ,p 2 ,…,p M}; R represents a relationship, R = {r 1 ,r 2 ,…,r T}, where r 1 ~r T Represents the relationship name, which contains T relationships in total; E represents the end entity, E∈L, and contains N attributes P E ={p 1 ,p 2 ,…,p N In this embodiment, only the entity name Name is considered as the attribute of the entity, then P s ={Name 1 ,Name 2 ,…,Name M}, P E ={Name 1 ,Name 2 ,…,Name N The process of instantiating an entity type is the process of assigning values to entity attributes, that is, assigning actual entity names to labels, thus forming nodes of the knowledge graph, instantiating attributes, thus forming properties of nodes, and defining relationships between nodes, thus forming a complete knowledge graph.
[0082] Step S2: Data acquisition and division: The acquired vertical field text dataset (ie, initial dataset) is divided into small sample data and large sample data.
[0083] The specific methods of data acquisition and division are as follows:
[0084] (1) The data with more than 100,000 characters in the text data is defined as large sample data, and the data with more than 8,000 characters and less than or equal to 10,000 characters is defined as small sample data.
[0085] (2) The obtained text data is shuffled in sentence units, randomly arranged, and then sentences are randomly extracted from them so that the number of characters contained in the extracted sentences meets the definition of small sample data, and the remaining part is the large sample data.
[0086] Step S3: Label the small sample data and obtain an entity dictionary. Label the entity types of the divided small sample data set, extract entities from the labeled small sample data, and obtain an entity dictionary.
[0087] As a specific implementation method, the specific method of annotating small sample data and obtaining an entity dictionary is as follows:
[0088] (1) The BIO annotation method is used to annotate text data. That is, B represents the starting character of the entity, I represents the middle and ending characters of the entity, and O represents non-entity characters. The type of entity is reflected after the B and I tags, such as Figure 4 As shown, for example, the entity types are defined to include products, components, and parts, which are represented by the letters "P, W, and L" respectively. Then "BP" represents the starting characters of the product entity, and "IP" represents the middle and ending characters of the product entity.
[0089] (2) Extract entities from the labeled small sample data, obtain an entity dictionary, store the name and type of the entity, and set the entity dictionary to EntityData = {[Name 1 :Label 1 ],[Name 2 :Label 1 ],…,[Name m :Label n ]}, where Name m Indicates the name of the mth entity, Label n Indicates the nth label, [Name m :Label n ] indicates that the entity type corresponding to the mth entity name is the nth label. For example, EntityData = {[diesel engine: P], [reducer: W], [crankshaft: L], [cylinder head: L], [engine body: L], [piston: L] ...}.
[0090] Step S4: Text enhancement of large sample data. To perform text enhancement on the divided large sample data set, first preprocess the large sample data, including word segmentation and stop word removal, calculate the semantic similarity between the processed data and each entity in the entity dictionary, set a similarity threshold, and mark the data that exceeds the similarity threshold.
[0091] As a specific implementation method, the specific method of text enhancement of large sample data is as follows:
[0092] (1) Perform data preprocessing on the large sample data, including word segmentation and removal of stop words. Taking sentences as units, let the large sample data be RawData = {S 1 ,S 2 ,…,Si ,…,S m}, where S i S is a sentence in a large sample data, after word segmentation and removal of stop words i ={w 1 ,w 2 ,…,w j ,…,w n}, where w j Indicates key words in a sentence.
[0093] (2) Compute the semantic similarity between the preprocessed large sample data and each entity in the entity dictionary, that is, compare w j (w j ∈S i , S i ∈RawData) and Name i (Name i ∈EntityData). Semantic similarity is calculated using the Jaccard correlation coefficient. The Jaccard correlation coefficient is used to compare the similarity between finite sample sets. Given two sets A and B, the Jaccard correlation coefficient is defined as the ratio of the size of the intersection of the two sets to the size of the union: J(A,B) = |A∩B| / |A∪B| = |A∩B / (|A|+|B|-|A∩B|). Here, A = RawData, B = EntityData. The larger the ratio obtained using this formula, the higher the similarity, and vice versa, the lower the similarity.
[0094] (3) Set a similarity threshold and mark the data that exceeds the similarity threshold.
[0095] Step S5: text data set construction and named entity recognition model training set division: that is, the annotated small sample data and the text-enhanced large sample data are merged to obtain a text data set, and the training set and validation set are divided by the K-fold cross-validation method.
[0096] As a specific implementation method, the specific method of constructing a text dataset and dividing a named entity recognition model training set is as follows:
[0097] The small sample data that has been labeled and the large sample data that has been enhanced with text are merged, and the training set and validation set are divided by the K-fold cross-validation method. First, all data samples are divided into k sample subsets of equal size; these k subsets are traversed in turn, each time the current subset is used as the validation set, and all other samples are used as the training set, for model training and evaluation; finally, the average value of the k evaluation indicators is used as the final evaluation indicator.
[0098] Step S6: Named Entity Recognition Model Construction. The BERT model provides a new deep learning paradigm based on pre-training, which greatly improves the model training efficiency. The bidirectional long short-term memory network (BiLSTM) is an improvement of the long short-term memory network, which can handle the problem of long-distance dependency in vertical fields. The conditional random field (CRF) can further constrain the model annotation ability. The BERT-BiLSTM-CRF model is constructed as a named entity recognition model for vertical fields. The model structure is as follows: Figure 5 As shown, the divided training set and validation set are used for model training and validation, and a general pre-trained model is used for initial training.
[0099] Step S7: Information extraction and knowledge graph construction: Use the trained and verified named entity recognition model to extract the vertical field data to be extracted, perform triple mapping based on the results of knowledge modeling, and form a knowledge graph.
[0100] like Figure 6 As shown in the figure, the specific methods of information extraction and knowledge graph construction are as follows:
[0101] (1) Use the trained and verified named entity recognition model to extract entities from the vertical domain data to be extracted (i.e. named entity recognition). Suppose the text data to be extracted is FinalData = {S 1 ,S 2 ,…,S k ,…,S M},S 1 ~S M Represents the sentences in the text data. There are M sentences in total. Perform named entity recognition on each sentence to obtain the entities contained in them. 1 :Label 1 ],[Name 2 :Label 1 ],[Name 3 :Label 2 ],…,[Name i :Label j ],…},Name i Indicates the name of the i-th entity, Label j represents the jth entity label, i.e., entity type, and the entity label L defined in step S1 = {Label 1 ,Label 2 ,…,Label n} consistent.
[0102] (2) Perform triple mapping based on the results of knowledge modeling to form a knowledge graph. In the previous step, named entity recognition is performed sentence by sentence on the extracted text data. Let the entity set extracted from the hth sentence be S h _ExtractEntity={[Name 1 :Label 1 ],[Name 2 :Label 1 ],[Name 3 :Label 2 ],…,[Name i :Label j ],…}, traverse the entire S h _ExtractEntity, compare whether the current Label has a relationship r∈R with other Labels. If there is a relationship r∈R, extract the entity according to the entity label and the entity relationship defined by the knowledge modeling.<S,R,E> Mapping, set Label x With Label Y There is a relationship z , then Label x The corresponding entity name A With Label Y The corresponding entity name B There is a relationship z , get the instantiated triple <Name A ,r z ,Name B >, that is, a triple instance. Then, a knowledge graph of the target vertical field can be constructed based on several triple instances.
[0103] Step S8: The extracted data is added to the text dataset. After the entity extraction of the vertical field data is completed, the extracted data is used as a new dataset. When the number of datasets reaches the minimum requirement for model training, the dataset is divided into a training set and a validation set again.
[0104] Step S9: Save the pre-trained model. The named entity recognition model trained in step S6 is saved as a pre-trained model, and the data set generated in step S8 is used for training and verification to obtain a new named entity recognition model. After completion, return to step S7.
[0105] Step S10: Newly obtained manually annotated data supplement. When there is a new manually annotated data set, similar to step S8, the data set is saved. When the number of data sets reaches the minimum requirement for model training, the data set is divided into a training set and a validation set, and the process returns to step S9.
[0106] Embodiment 2
[0107] In order to execute the method corresponding to the above embodiment 1 to achieve the corresponding functions and technical effects, a knowledge graph automatic construction system is provided below. Figure 7 As shown, the system includes:
[0108] The knowledge modeling module 701 is used to perform knowledge modeling on the target vertical field to obtain a knowledge model of the target vertical field; the knowledge model includes entity types and entity relationships; the entity relationships represent entities and relationships between entities.
[0109] The data set division module 702 is used to obtain an initial data set and divide the initial data set into large sample data and small sample data; the large sample data is text data in the initial data set whose number of characters is greater than a first set value; the small sample data is text data in the initial data set whose number of characters is greater than a third set value and less than or equal to a second set value; the first set value is greater than the second set value, and the second set value is greater than the third set value.
[0110] The small sample data processing module 703 is used to perform data annotation on the small sample data to obtain annotated small sample data, and determine an entity dictionary according to the annotated small sample data.
[0111] The large sample data processing module 704 is used to perform text enhancement on the large sample data according to the entity dictionary to obtain text enhanced large sample data.
[0112] The text data set construction module 705 is used to construct a text data set according to the annotated small sample data and the text enhanced large sample data.
[0113] The model training module 706 is used to train a named entity recognition model according to the text data set; the named entity recognition model is a BERT-BiLSTM-CRF model.
[0114] The triple instance generation module 707 is used to obtain target text data, perform entity extraction on the target text data using the trained named entity recognition model, and perform triple mapping on the extraction results according to the knowledge model to obtain triple instances.
[0115] The knowledge graph construction module 708 is used to construct the knowledge graph of the target vertical field based on the triple instance.
[0116] Embodiment 3
[0117] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to run the computer program so that the electronic device executes the method for automatically constructing a knowledge graph in Embodiment 1. The electronic device may be a server.
[0118] In addition, the present invention also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the method for automatically constructing a knowledge graph in Example 1.
[0119] Aiming at the problems of insufficient annotated data in vertical fields, difficulty in training models for automatic extraction of unstructured information in knowledge graphs, and difficulty in updating models as data changes, the present invention provides a method for automatically constructing knowledge graphs based on text enhancement and transfer learning. Compared with the prior art, the method has the following advantages:
[0120] 1. The present invention expands the data set based on the text enhancement method, and obtains large sample data after text enhancement based on manually annotated small sample data, so as to obtain a sufficient amount of data to train the named entity recognition model, improve the efficiency of model training set preparation, and improve the model accuracy and robustness.
[0121] 2. The present invention constructs a named entity recognition model BERT-BiLSTM-CRF based on a pre-trained model, and the trained model can be used as a pre-trained model to train a new model, thereby realizing dynamic updating of the named entity recognition model.
[0122] In summary, the present invention proposes a method for automatically constructing a knowledge graph based on text enhancement and pre-training models. It is suitable for the construction of knowledge graphs in vertical fields, targeting the difficulties of complex vertical field knowledge, difficult acquisition of data resources, and insufficient annotation of data sets. The method can dynamically update the named entity recognition model as the vertical field data set changes to adapt to the ever-changing vertical field data resources and knowledge needs.
[0123] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0124] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of the present invention. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A method for automatically constructing a knowledge graph, characterized in that: include: Conducting knowledge modeling on the target vertical field to obtain the knowledge model of the target vertical field, specifically including: sorting and classifying the knowledge of the target vertical field, and saving the classified entity types in the form of labels; defining the types and attributes of entities and relationships, determining the relationship between different labels and the attributes contained in each label, and saving them in the form of triples; wherein the attribute of the entity is the entity name; the knowledge model includes entity types and entity relationships; the entity relationships represent the relationships between entities; Acquire an initial data set, and divide the initial data set into large sample data and small sample data; the large sample data is text data in the initial data set whose number of characters is greater than a first set value; the small sample data is text data in the initial data set whose number of characters is greater than a third set value and less than or equal to a second set value; the first set value is greater than the second set value, and the second set value is greater than the third set value; The small sample data is annotated to obtain annotated small sample data, and an entity dictionary is determined based on the annotated small sample data, specifically including: using the BIO annotation method to annotate the small sample data to obtain annotated small sample data; wherein B represents the starting character of the entity, I represents the middle and ending characters of the entity, and O represents a non-entity character; entities are extracted from the annotated small sample data to obtain an entity dictionary; the entity dictionary includes entity names and corresponding entity types in the small sample data; According to the entity dictionary, text enhancement is performed on the large sample data to obtain text-enhanced large sample data, specifically including: performing word segmentation and stop word removal on the large sample data to obtain pre-processed large sample data; performing semantic similarity calculation on the pre-processed large sample data and each entity name in the entity dictionary to obtain a similarity value corresponding to the pre-processed large sample data; performing data annotation on the pre-processed large sample data whose similarity value is greater than a set similarity threshold to obtain text-enhanced large sample data; wherein the semantic similarity is calculated by the Jaccard correlation coefficient; Constructing a text dataset based on the annotated small sample data and the text-enhanced large sample data; Training a named entity recognition model according to the text data set; the named entity recognition model is a BERT-BiLSTM-CRF model; Obtain target text data, perform entity extraction on the target text data using a trained named entity recognition model, and perform triple mapping on the extraction results according to the knowledge model to obtain a triple instance, specifically comprising: performing entity extraction on the target text data sentence by sentence using the trained named entity recognition model to obtain an extraction result; the extraction result includes entity names and corresponding entity types in the target text data; performing triple mapping on the entity names in the extraction results according to corresponding entity types and entity relationships in the knowledge model to obtain a triple instance; the triple instance includes a starting entity name, an ending entity name, and a relationship between the starting entity and the ending entity; Construct a knowledge graph for the target vertical field based on the triple instances.
2. The method for automatically constructing a knowledge graph according to claim 1, characterized in that: Training a named entity recognition model according to the text dataset specifically includes: Dividing the text data set into a number of sample subsets of equal size; The K-fold cross-validation method is adopted to train and verify the named entity recognition model according to the sample subset to obtain a trained named entity recognition model.
3. The method for automatically constructing a knowledge graph according to claim 1, characterized in that: Also includes: Saving the trained named entity recognition model as a pre-trained model; Using the target text data and the corresponding extraction results as a first expanded data set; When the data volume of the first expanded data set is greater than a set amount, the pre-trained model is trained according to the first expanded data set to update the trained named entity recognition model.
4. The method for automatically constructing a knowledge graph according to claim 3, characterized in that: Also includes: When there is a new manually annotated data set, using the new manually annotated data set as a second expanded data set; When the data volume of the second expanded data set is greater than a set amount, the pre-trained model is trained according to the second expanded data set to update the trained named entity recognition model.
5. A knowledge graph automatic construction system, characterized in that: include: The knowledge modeling module is used to perform knowledge modeling on the target vertical field and obtain the knowledge model of the target vertical field, which specifically includes: sorting and classifying the knowledge of the target vertical field, and saving the classified entity types in the form of labels; defining the types and attributes of entities and relationships, determining the relationship between different labels and the attributes contained in each label, and saving them in the form of triples; wherein the attribute of the entity is the entity name; the knowledge model includes entity types and entity relationships; the entity relationships represent the relationships between entities; A data set division module is used to obtain an initial data set and divide the initial data set into large sample data and small sample data; the large sample data is text data in the initial data set whose number of characters is greater than a first set value; the small sample data is text data in the initial data set whose number of characters is greater than a third set value and less than or equal to a second set value; the first set value is greater than the second set value, and the second set value is greater than the third set value; The small sample data processing module is used to perform data annotation on the small sample data to obtain the annotated small sample data, and determine the entity dictionary based on the annotated small sample data, specifically including: using the BIO annotation method to perform data annotation on the small sample data to obtain the annotated small sample data; wherein B represents the starting character of the entity, I represents the middle and ending characters of the entity, and O represents the non-entity character; extracting entities from the annotated small sample data to obtain an entity dictionary; the entity dictionary includes the entity name and the corresponding entity type in the small sample data; The large sample data processing module is used to perform text enhancement on the large sample data according to the entity dictionary to obtain text-enhanced large sample data, specifically including: performing word segmentation and stop word removal on the large sample data to obtain pre-processed large sample data; performing semantic similarity calculation on the pre-processed large sample data and each entity name in the entity dictionary to obtain a similarity value corresponding to the pre-processed large sample data; performing data annotation on the pre-processed large sample data whose similarity value is greater than a set similarity threshold to obtain text-enhanced large sample data; wherein the semantic similarity is calculated by the Jaccard correlation coefficient; A text data set construction module, used to construct a text data set based on the annotated small sample data and the text enhanced large sample data; A model training module, used for training a named entity recognition model according to the text data set; the named entity recognition model is a BERT-BiLSTM-CRF model; A triple instance generation module is used to obtain target text data, perform entity extraction on the target text data using a trained named entity recognition model, and perform triple mapping on the extraction results according to the knowledge model to obtain a triple instance, specifically comprising: performing entity extraction on the target text data sentence by sentence using the trained named entity recognition model to obtain an extraction result; the extraction result includes the entity name and the corresponding entity type in the target text data; performing triple mapping on the entity name in the extraction result according to the corresponding entity type and the entity relationship in the knowledge model to obtain a triple instance; the triple instance includes a starting entity name, an ending entity name, and a relationship between the starting entity and the ending entity; A knowledge graph construction module is used to construct a knowledge graph of the target vertical field based on the triple instances.
6. An electronic device, characterized in that: It includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the method for automatically constructing a knowledge graph as described in any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the method for automatically constructing a knowledge graph as described in any one of claims 1 to 4.