A method for constructing NLP knowledge graphs combining information volume and BERT-BiLSTM-CRF

By combining the amount of information and BERT-BiLSTM-CRF, the pattern layer in the field of natural language processing is defined, and the new word discovery and feature fusion algorithm is used to solve the problems of high labor costs and low accuracy in knowledge graph construction, and the problem of high labor costs and low accuracy is achieved. The construction and accuracy of automated knowledge graphs in the field of natural language processing.

CN114817454BInactive Publication Date: 2024-11-08BEIJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210150583.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-18
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing technology has problems such as long manual construction time, high cost, low automatic construction accuracy and complex ontology layer definition in the construction of knowledge graphs, especially in the field of natural language processing, lack of reusable ontology layer and fine-grained research task definitions.

Method used

Combining the amount of information and the NLP knowledge graph construction method of BERT-BiLSTM-CRF, by defining the pattern layer in the field of natural language processing, using new word discovery algorithm and XGBoost-based feature fusion text multi-classification algorithm to obtain key terms and fine-grained research tasks , build a natural language processing knowledge graph.

Benefits of technology

It realizes the accuracy of automatically building knowledge graphs, simplifies pattern layer definition, improves the accuracy of new word recognition and text multi-classification, reduces labor costs, and provides a reusable natural language processing knowledge graph.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114817454B_ABST
    Figure CN114817454B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for constructing an NLP knowledge graph that combines information volume and BERT-BiLSTM-CRF. The method proposes a pattern layer structure of a knowledge graph in the field of natural language processing by analyzing the structure of CNKI journal paper data and combining it with the research tasks of natural language processing. Then, a new word discovery algorithm is proposed to obtain the key term entity class in the paper data, and a feature fusion multi-classification algorithm is proposed to obtain the fine-grained NLP research task entity class of the paper. Then, the knowledge extraction module acquires triples, and finally constructs the natural language processing knowledge graph. The new word recognition algorithm of the present invention makes up for the defect of traditional technology that only uses information volume to obtain new words, and greatly improves the effect of new word discovery. The classification of fine-grained research tasks of the paper using the XGBoost model has a higher accuracy rate than other machine learning models. In addition, after feature fusion, the accuracy of the classification model is improved by about five percentage points compared with the classification accuracy of the model without feature fusion.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for constructing an NLP knowledge graph combining information volume and BERT-BiLSTM-CRF. Background Art

[0002] In recent decades, natural language processing has been in a rapid development stage, and the amount of academic research papers related to natural language processing has increased dramatically. Whether in academia or industrial production, the demand for consulting papers related to natural language processing has become increasingly strong. However, the diverse research content and complex conceptual relationships in the field of natural language processing have brought considerable challenges to people's reading of papers.

[0003] When constructing a vertical domain knowledge graph, the two most important subtasks are the construction of the knowledge graph model layer and knowledge extraction. The knowledge graph can be logically divided into two levels: the model layer and the data layer. The data layer is mainly composed of a series of facts, and knowledge is stored in units of facts. The model layer is built on the data layer, defining the concepts of the knowledge graph and the connections between concepts. The definition of the model layer can standardize the data structured expression of the entire knowledge graph; the basic component unit of the knowledge graph is the "entity-relationship-entity" triple, and the main purpose of knowledge extraction is to obtain triples to form a network knowledge graph.

[0004] Technical solution of prior art 1

[0005] One of the prior arts related to the technical solution of the present invention is the construction of a knowledge graph. There are currently only two methods for knowledge graph construction technology: manual construction and automatic construction. Manual construction is the use of human resources to construct a knowledge graph. Compared with manual construction, automatic construction is more widely used. In various fields currently known, the way to construct knowledge graphs almost all uses automatic construction technology. The reason is that automatic construction technology has its own advantages: using some algorithms to identify entities and entity relationships, the construction technology can be universally developed, and the automatic construction technology can be applied to various fields. In fact, the main technologies include: entity recognition and relationship extraction. After the two parts are completed, some knowledge fusion and processing are used to form a large number of triples, and then to form a huge network knowledge graph. The flowchart of automatically constructing a knowledge graph is as follows. Figure 1 As shown:

[0006] Disadvantages of the prior art 1

[0007] The manual construction of knowledge graphs often has the problems of long construction time and high labor costs. In addition, if you want to build a larger knowledge graph using manual construction technology, it requires more manpower, resulting in high labor costs, and it is difficult to build a huge knowledge graph. Therefore, only the manual construction method was used in early research. After the emergence of automatic construction technology, manual construction technology was abandoned by researchers.

[0008] However, the automatic construction of knowledge graphs often results in complex definitions of the ontology layer, especially for knowledge graphs in specific fields, where there is no reusable ontology layer. Compared with purely manual ontology construction methods, semi-automatic and automated ontology construction methods save efficiency, but do not meet the standards of ontology methodology. In addition, the automatic construction of knowledge graphs has the problem of lack of algorithm accuracy. Through research, the scheme of the present invention defines the ontology layer in the field of natural language processing for the first time. At the same time, by combining the amount of information with deep learning algorithms, and using BERT to propose a new word discovery algorithm, and proposing a feature fusion text multi-classification algorithm based on XGBoost, the accuracy of the algorithm is improved.

[0009] Prior art related to the technical solution of the present invention

[0010] Technical solution of prior art 2

[0011] One of the prior arts related to the technical solution of the present invention is the definition of the knowledge graph pattern layer. The knowledge graph can be logically divided into two levels: the pattern layer and the data layer. The data layer is mainly composed of a series of facts, and the knowledge will be stored in units of facts. The pattern layer is built on the data layer, defining the concepts of the knowledge graph and the connections between concepts. The ontology library is mainly used to standardize a series of fact expressions in the data layer. Ontology is a conceptual template of a structured knowledge base. The knowledge base formed by the ontology library not only has a strong hierarchical structure, but also has a low degree of redundancy. Vertical domain knowledge graphs usually require experts to define the ontology and complete the definition of the pattern layer so that the constructed domain knowledge graph can include all important knowledge in the field. There are three main methods for constructing the pattern layer: manual construction, semi-automatic construction, and automatic construction. At present, the more mature construction methods are IDEF-5, TOVE method, skeleton method, KACTUS method, seven-step method, and cyclic acquisition method.

[0012] Disadvantages of the second prior art

[0013] Compared with manual ontology construction methods, semi-automatic and automated ontology construction methods save efficiency, but do not meet the standards of ontology methodology. Since there is no reusable natural language processing ontology, this paper will use the seven-step method and combine the characteristics of the natural language processing field to build the natural language processing model layer for the first time.

[0014] Prior art related to the technical solution of the present invention

[0015] 1.5.1 Technical solution of existing technology 3

[0016] The third prior art related to the technical solution of the present invention is to find out the key terms in the abstract by using a new word discovery algorithm. The key terms are a very important type of entity in the defined natural language processing model layer. Knowledge extraction from knowledge graphs generally requires the use of named entity recognition and relationship extraction methods. The entities therein are obtained by the method of named entity recognition, and the relationships between entities are obtained by relationship extraction. However, the existing named entity recognition algorithms recognize names of people, places, and institutions, and most of them recognize entities in general fields. The recognition accuracy of professional terms in specific fields is low. In the present invention, it is hoped that a new word discovery algorithm can be used to find the corresponding professional terms.

[0017] At present, the more common algorithm is the domain new word discovery algorithm that combines information volume and deep learning [Huang Wenming, Yang Liuqingqing, Ren Chong. Domain new word discovery combining information volume and deep learning [J]. Computer Engineering and Design, 2019, 40(07): 1903-1907+1914.]. This algorithm proposes a method to calculate the information volume of the word segmentation results and use the deep learning model BiLSTM-CRF for new word discovery. The calculated information volume is used to represent the internal cohesion and separation of words, and artificial rules are added for filtering. The BiLSTM-CRF model has high accuracy, low dependence on word vectors, and takes context information into account. The combination of information volume and deep learning model solves a large number of manual sequence labeling problems. The recognition rate of low-frequency new words is improved.

[0018] Disadvantages of the prior art 3

[0019] The new word discovery model proposed in the technical solution of the invention combines the technology of the traditional information quantity statistical method with the deep learning BiLSTM-CRF sequence labeling algorithm, and identifies low-frequency new words based on the context information of the text, greatly improving the efficiency and accuracy of new word recognition. However, since the BiLSTM model is not strong enough in extracting the features of words, the accuracy of new word recognition needs to be improved.

[0020] Prior art related to the technical solution of the present invention

[0021] Technical solution of prior art 4

[0022] The fourth prior art related to the technical solution of the present invention is to find the fine-grained research task classification of each journal article through text multi-classification. Regarding the research tasks of natural language processing, there is currently no fine-grained research task definition, which is simply divided into four categories: sequence labeling tasks, classification tasks, sentence relationship judgment, and generative tasks. However, for the specific field of natural language processing, readers often expect to obtain more fine-grained research tasks and quickly filter papers through more fine-grained research task categories.

[0023] There are many achievements in text classification made by current scholars. There are sentiment polarity classification based on short text comment data: Wang Xianwei [Wang Xianwei. Research on the application of text sentiment analysis in product reviews - taking Jingdong smart refrigerator reviews as an example [D]. Zhejiang Gongshang University, 2018.] uses Jingdong smart refrigerator review data to classify text sentiment into positive and negative polarities; there are also scholars who classify based on text topics: Huo Tingting [Huo Tingting. Research on news text classification based on fastText and its application in agricultural news [D]. Jilin University, 2019.] classifies news text based on three improved algorithms of FastText model. Category, mainly divided into agricultural and non-agricultural news; Jing Yongxia [Jing Yongxia, Wang Zhihe, Gou Heping. Research on Text Classification Algorithm Based on Matrix Singular Value Decomposition [J]. Journal of Northwest Normal University, 2018, 54 (3): 51-56.] et al. used the matrix SVD method to perform multi-classification of 10 categories of text data, which was improved compared with the previous KNN algorithm based on information gain; some scholars did not classify documents by topic but by the overall sentiment of the document [Pang B., L. Lee, S. Vaithyanathan. Thumbs up? Sentiment Classification using Machine Learning Techniques [J]. EMNLP'02: Proceedings of the ACL-02 conference on Empirical Methods in Natural Language Processing, 2002, 10: 79-86.]. This paper uses XGBoost to perform single-label text multi-classification on CNKI journal paper data containing 48 categories, and analyzes the performance of the classification model.

[0024] Disadvantages of the prior art 4

[0025] At present, common machine learning algorithms such as logistic regression, naive Bayes, random forest, linearSVC and other algorithms are not perfect for the processing of missing values ​​in the data, and currently only the abstract features in the paper data are used for text multi-classification, without fully using the acquired data information, and the final classification effect is poor. The present invention selects five machine learning algorithms and LSTM deep learning algorithm to carry out text multi-classification experiments, and compares three different feature extraction algorithms of TF-IDF, Word2vec and Doc2Vec. Theoretical and experimental results prove that using TF-IDF for feature extraction and fusing multiple features and performing text multi-classification based on XGBoost algorithm have the best effect. Summary of the invention

[0026] Based on the above research shortcomings, the present invention takes the two subtasks mentioned in the background technology as the foothold, analyzes the structure of CNKI journal paper data and combines it with the research task of natural language processing to propose the model layer structure of the knowledge graph in the field of natural language processing; then, by proposing a new word discovery algorithm to obtain the key term entity class in the paper data, a feature fusion multi-classification algorithm is proposed to obtain the fine-grained NLP research task entity class of the paper. Then, the knowledge extraction module completes the acquisition of triples, and finally constructs the natural language processing knowledge graph.

[0027] To achieve the purpose of the invention, the technical solution provided by the present invention is: a method for constructing an NLP knowledge graph combining information volume and BERT-BiLSTM-CRF, comprising the following steps:

[0028] Step 1), define the NLP knowledge graph model layer;

[0029] Define the schema layer to describe the entities, relationships, and attributes in the graph;

[0030] Step 2), data set construction:

[0031] NPL paper data acquisition, using crawlers to obtain data in the field of natural language processing and perform data annotation;

[0032] Step 3), new word discovery:

[0033] Find out the key terms in the abstract by using new word discovery algorithms;

[0034] The new word algorithm includes the following steps: obtaining candidate new words by counting the amount of information, identifying low-frequency new words through the BiLSTM-CRF model, obtaining the vector corresponding to the input text by adding BERT in front of BILSTM, obtaining more low-frequency new words by using vectors to represent context information, and completing new word discovery after model testing and evaluation;

[0035] Step 4), text multi-classification;

[0036] Define the research tasks of fine-grained NLP and derive the research tasks of fine-grained NLP through a text multi-classification model based on feature fusion.

[0037] Step 5), knowledge graph construction;

[0038] The triple data is stored and visualized through the Neo4j graph database, and the construction of the NLP knowledge graph is finally completed.

[0039] In the step 1), the model layer definition is completed by the following steps:

[0040] Step 1.1), determine the domain and scope of ontology construction as the field of natural language processing;

[0041] Step 1.2), analyze whether there is an ontology that can be reused in this field;

[0042] Step 1.3), sort out and determine the important terms in the domain ontology;

[0043] Step 1.4), determine the structural hierarchy of the classes in the ontology;

[0044] Step 1.5), determine the attribute information of the class;

[0045] Step 1.6), determine the constraints of class attributes;

[0046] Step 1.7), instantiate the ontology.

[0047] In the step 1), the entities and attributes are defined by constructing a natural language processing knowledge graph. The entities in the final design graph include: papers, research tasks, key terms, and attributes and relationships are defined based on these entities.

[0048] In the step 2), the crawled paper content includes title, author, publishing unit, publishing time, publishing journal, keywords, and abstract, and the crawled data is used to perform knowledge extraction to obtain knowledge data in the form of triples.

[0049] In the step 4), the text multi-classification model based on feature fusion includes a feature fusion module and a text classification module. The feature fusion module is: the pre-processed data is converted into a vector for representation by using the TF-IDF word vector algorithm, and then the features of the title, keywords, author, publication time, publication agency, and publication unit are fused and converted into a vector as the input of the text classification module; the text classification module uses the XGBoost algorithm for classification processing.

[0050] In the step 5), the neo4j graph database is used as a tool for storing and visualizing the natural language processing knowledge graph.

[0051] The neo4j interactive interface allows users to explore the graph or search the graph database using the Cypher query language.

[0052] The beneficial effects of the present invention are:

[0053] (1) The definition of the model layer of the automatic construction of the knowledge graph is relatively complex, especially for the specific field of natural language processing, where there is no reusable ontology layer. The present invention defines the knowledge graph model layer, combines industry database standards and manual definitions, and uses a top-down approach to define the entities, relationships, and concepts within the graph, thereby completing the definition of the natural language processing knowledge graph model layer.

[0054] (2) By using a new word discovery algorithm to find out the key terms in the abstract, the key terms are a very important type of entity in the defined natural language processing model layer. The existing new word recognition algorithms are not strong enough in the feature extraction capability of the word context information. The existing new word discovery technology combines the amount of information and the deep learning BiLSTM-CRF algorithm to identify domain terms, which makes up for the defect of traditional technology that only uses the amount of information to obtain new words. By using a deep learning algorithm, low-frequency new words can be more accurately identified. However, the feature extraction capability of the BiLSTM-CRF algorithm alone for the context information of Chinese words is not strong enough, resulting in the effect of new word recognition to be improved. In the present invention, by adding the BERT model, the vector obtained using BERT has a more powerful ability to extract text context information, which greatly improves the effect of new word discovery.

[0055] (3) Find out the fine-grained NLP research task classification for each journal article by using a text multi-classification algorithm. The current common text classification machine learning algorithms are not perfect in handling missing values ​​in the data, and currently only use the abstract features in the paper data for text multi-classification, without fully using the acquired data information, the feature extraction is not perfect, and the final classification effect is poor. In the present invention, five machine learning algorithms and one deep learning algorithm are compared, and three feature extraction algorithms are compared. After experimental comparison, it is found that the classification accuracy of the fine-grained research tasks of the paper using the XGBoost model is higher than that of other machine learning models. In addition, after feature fusion, the accuracy of the classification model is improved by about five percentage points compared to the classification accuracy of the model without feature fusion. Experiments have proved that the feature fusion text classification algorithm proposed in the present invention has a higher accuracy rate for the multi-classification task of journal articles.

[0056] Definitions of abbreviations and key terms used in this document:

[0057] Knowledge Graph

[0058] Knowledge graph is a structured semantic knowledge base used to describe concepts and their relationships in the physical world in symbolic form. Its basic component is the "entity-relationship-entity" triple or the "entity-attribute-attribute value" form. Entities are connected to each other through relationships to form a network of knowledge structures. Knowledge graphs are divided into two categories. General knowledge graphs are the most common and most used type of knowledge graphs because of their wide knowledge coverage and many fields. Domain knowledge graphs are mainly targeted at specific fields and emphasize the depth of knowledge.

[0059] The knowledge graph stores knowledge in the form of structured triples through a network of structural relationships, which can more appropriately display, store and manage complex knowledge systems and meet the needs of the present invention.

[0060] New word discovery

[0061] New word discovery can also be called unregistered word recognition. Strictly speaking, new words refer to words that have emerged with the development of the times or old words that have been used in new ways, such as: awesome, cottage, etc.; and unregistered words are words that do not exist in the dictionary, but in actual use, there is no strict distinction between the two. The present invention hopes to find out the key terms in the paper abstract. Since it is unsupervised data, the effect of identifying domain terms through named entity recognition is not very good, and named entity recognition is essentially a kind of new word discovery. At present, the methods of new word discovery are mainly divided into four categories: rule-based, statistical-based, rule-based and statistical-based methods, and deep learning-based methods. The more common method is the combination of rule and statistics.

[0062] Text multi-classification

[0063] Text classification refers to determining a category for each document in a document collection according to predefined subject categories. Text classification tasks are generally divided into two categories. The classification in which the input text corresponds to one output category is called single-label classification; if a text is input and the output category is greater than or equal to two, it is called multi-label classification. Among them, single-label classification is further divided into binary text classification and multi-category text classification: binary text classification means that the data set has only two categories, and the data set of multi-category text classification has more than two categories. The present invention performs multi-classification of text under a single label based on XGBoost for paper data containing 48 categories through feature fusion, analyzes the performance of the classification model, and determines a fine-grained natural language processing research task category for each journal paper. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 Construct a flow chart for the knowledge graph of prior art 1;

[0065] Figure 2 The algorithm flow of new word discovery for the prior art three;

[0066] Figure 3 It is the overall flow chart of the technical solution of the present invention;

[0067] Figure 4 This is a framework diagram of the knowledge graph model layer in the field of natural language processing of the present invention;

[0068] Figure 5 This is a crawler workflow diagram in the present invention;

[0069] Figure 6 A flowchart of the new word discovery algorithm in the present invention;

[0070] Figure 7 This is a text multi-classification model diagram based on feature fusion in the present invention;

[0071] Figure 8 It is a schematic diagram of the overall nodes of the research task in the present invention;

[0072] Fig. 9 It is a node graph related to the topic model in the present invention. DETAILED DESCRIPTION

[0073] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings.

[0074] The implementation process of the technical solution of the present invention includes the following five steps: defining the NLP model layer, obtaining NLP paper data, new word discovery, text multi-classification, and completing the construction of the knowledge graph. The overall structure diagram is as follows Figure 3 As shown:

[0075] Step 1), define the NLP knowledge graph model layer

[0076] The pattern layer describes the entities, relationships and attributes in the graph and is the framework of the knowledge graph. In the domain knowledge graph, it is usually necessary to have a deep understanding of the domain knowledge and define the pattern layer in combination with the domain data model. This paper defines the pattern layer of the NLP knowledge graph through a seven-step method.

[0077] (1) First, the domain and scope of ontology construction are determined to be the field of natural language processing. The data source of this embodiment is journal articles published in the field of natural language processing in HowNet between 2011 and 2020;

[0078] (2) Then analyze whether there is a reusable ontology in this field: there is currently no reusable ontology;

[0079] (3) Organize and determine the important terms in the domain ontology;

[0080] (4) Determine the structural hierarchy of classes in the ontology;

[0081] (5) Determine the attribute information of the class;

[0082] (6) Determine the constraints of class attributes;

[0083] (7) Instantiate the ontology;

[0084] Among them, regarding the entity and attribute definitions,

[0085] This paper constructs a natural language processing knowledge graph for the first time. The entities in the final design graph include: papers, research tasks, key terms, etc., and define attributes and relationships based on these entities. The model layer framework diagram of the NLP knowledge graph finally constructed is as follows: Figure 4 As shown:

[0086] Specifically, the basic attributes of each main entity are defined as follows:

[0087] (1) Papers. Paper data mainly comes from journal papers in the decade from 2011 to 2020 in CNKI. The attributes of paper entities include: paper title, author, publishing unit, publication time, publication journal, keywords, and abstract. As shown in Table 1:

[0088] Table 1 Paper entity attributes and constraints

[0089]

[0090] (2) Research tasks. The main data of research tasks are statistical data from Baidu and CNKI papers. The research task attributes of natural language processing include: task name. As shown in Table 2:

[0091] Table 2 Research task attributes and constraints

[0092] property Data Types Sample Value Task Name string Knowledge Graph

[0093] (3) Key terms. The main data of key terms come from the abstracts of journal articles in HowNet. The attributes of key terms include: term name, as shown in Table 3:

[0094] Table 3 Key term attributes and constraints

[0095] property Data Types Sample Value Term Name string Word2vec, electronic medical records, information extraction

[0096] Second, regarding the relationship definition:

[0097] According to the relationship between the ontology and the attributes in the real world, the present invention defines their relationship and the attributes of the relationship in the graph. As shown in Table 4:

[0098] Table 4 Some relationship types in the graph

[0099] Relationship Name Subject Entity Object Entity termsName Key Terms Term Name TaskName Research Mission Task Name publisherName Publisher Unit Name publicationName Publications Publication Name hasArticleTitle paper Paper Title hasArticleAuthor paper author hasArticlePublisher paper Publisher hasArticlePublishTime paper Published hasArticlePublishPublication paper Publications hasArticleKeyWords paper Keywords hasArticleSummary paper summary

[0100] Step 2), dataset construction

[0101] After the knowledge graph model layer is defined, it is necessary to find suitable data instances. Unlike general knowledge graphs, which are often built on knowledge encyclopedias and open linked data, domain knowledge graphs are used within the industry, with a wide range of professional knowledge and deeper knowledge. Open encyclopedia data cannot meet data needs. Therefore, for the construction of industry knowledge graphs, the industry's internal standard databases, open knowledge bases, and vertical websites are the main data sources.

[0102] The data in this embodiment mainly comes from journal articles in the field of natural language processing in HowNet from 2011 to 2020, and the relevant data is obtained by crawling. Figure 5 As shown:

[0103] After the above crawler process, 18,345 journal articles were finally obtained. The crawled articles included titles, authors, publishing units, publishing time, publishing journals, keywords, and abstracts. In addition, through research, a total of 48 natural language processing research tasks were defined. In subsequent processing, these data will be used to extract knowledge in different forms to obtain knowledge data in the form of triples.

[0104] Step 3), new word discovery

[0105] The purpose of this stage is to find out the key terms in the abstract by using new word discovery algorithms. Key terms are a very important type of entity in the defined natural language processing model layer.

[0106] In the new word discovery algorithm for domains that combines information volume and deep learning proposed by Huang Wenming [Huang Wenming, Yang Liuqingqing, Ren Chong. Combining information volume and deep learning [J]. Computer Engineering and Design, 2019, 40(07): 1903-1907+1914.] and others, more low-frequency new words were identified through the BiLSTM-CRF model. However, this method is not strong enough to obtain contextual information. In the present invention, BERT is added in front of BILSTM to obtain the vector corresponding to the input text, thereby improving the ability of the vector to represent news information, thereby improving the accuracy of new word recognition. The flowchart of the new new word recognition algorithm proposed in the present invention is as follows: Figure 6 As shown:

[0107] On the one hand, we obtain candidate new words by counting the amount of statistical information. On the other hand, we use deep learning models and vectors to represent contextual information to obtain more low-frequency new words. In addition, we input the text into the BERT model to improve the vector's ability to represent contextual information, thereby greatly improving the accuracy of new word recognition.

[0108] Step 4), text multi-classification;

[0109] The purpose of this phase is to define fine-grained NLP research tasks, a total of 48 categories, based on the XGBoost algorithm, and optimize the text multi-classification algorithm through feature fusion to achieve the goal of multi-classification of paper and journal research tasks. Fine-grained research tasks are also a very important type of entity in the defined natural language processing model layer.

[0110] The overall experimental model of this module is shown in the figure Figure 7 As shown in the figure. Through data preprocessing and feature extraction algorithms, the text is converted into vectors and input into the XGBoost algorithm. The output result is the fine-grained research task classification for each journal article. The overall model is shown in the figure Figure 7 As shown:

[0111] The entire model consists of two parts: feature fusion and text classification. The feature fusion module uses the TF-IDF word vector algorithm to convert the text into a vector for representation, and then fuses the features such as title, keywords, author, publication time, publishing organization, and publishing unit into a vector as the input of the classification model. The text classification module compares five machine learning algorithms and the deep learning algorithm LSTM. The experimental results show that the use of the XGBoost algorithm is better. The reason may be that the XGBoost classifier can automatically learn and process the missing feature values. In order to avoid falling into the local optimum, reverse pruning is performed. Hundreds of models with low classification accuracy can be combined into a model with higher classification accuracy for classification processing. The experimental results show that the model graph proposed in this patent has a high accuracy rate.

[0112] Step 5), knowledge graph construction

[0113] Through the above steps, the definition of the natural language processing model layer is completed, and the new word discovery algorithm is used to complete the extraction of key term entity classes. The feature fusion text multi-classification algorithm is used to complete the extraction of fine-grained research task classes. The main function of this module is to store and visualize the triple data through the Neo4j graph database, and finally complete the construction of the NLP knowledge graph. The neo4j graph database is used as a tool for storage and visualization of natural language processing knowledge graphs. Through the neo4j interactive interface, users can easily explore the graph, and can also search the database through the Cypher query language. Figure 8 It uses Cypher language to query all research task categories. Fig. 9 It is the paper data related to the "topic model" research task being queried.

[0114] Through the specific description of the above embodiments, it can be seen that the present invention is aimed at the construction of the prior art - knowledge graph. The present invention constructs a knowledge graph in the field of natural language processing for the first time; the construction of this knowledge graph can, on the one hand, provide a quick and effective means to understand natural language processing in scientific research and industrial production, and on the other hand, can provide a basic knowledge base for NLP applications such as recommendation systems, question-answering systems, etc.

[0115] In response to the definition of the second model layer in the prior art, the present invention defines the model layer in the field of natural language processing for the first time: through the seven-step method, the entity and attribute definitions, as well as the entity relationship are defined, and the model layer of the natural language processing knowledge graph is defined for the first time.

[0116] In view of the three new word discovery algorithms in the prior art, the present invention improves the new word discovery algorithm and proposes a new word recognition model that combines information volume and the BERT-BiLSTM-CRF algorithm to extract key term entity classes.

[0117] In view of the four text multi-classification algorithms in the prior art, the present invention improves the text multi-classification algorithm and proposes a feature fusion multi-classification algorithm based on XGBoost, which realizes the classification of 48 fine-grained research tasks and extracts the research task entity class through the algorithm.

[0118] The described embodiments are only a part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

Claims

1. A method for constructing an NLP knowledge graph combining information volume and BERT-BiLSTM-CRF, characterized in that: The following steps are involved: Step 1), define the NLP knowledge graph model layer; Define the schema layer to describe the entities, relationships, and attributes in the graph; Step 2), data set construction: NPL paper data acquisition, using crawlers to obtain data in the field of natural language processing and perform data annotation; Step 3), new word discovery: Find out the key terms in the abstract by using new word discovery algorithms; The new word algorithm includes the following steps: obtaining candidate new words by counting the amount of information, identifying low-frequency new words through the BiLSTM-CRF model, obtaining the vector corresponding to the input text by adding BERT in front of BILSTM, obtaining more low-frequency new words by using vectors to represent context information, and completing new word discovery after model testing and evaluation; Step 4), text multi-classification; Define the research tasks of fine-grained NLP and derive them through a text multi-classification model based on feature fusion; Step 5), knowledge graph construction; The triple data is stored and visualized through the Neo4j graph database, and the construction of the NLP knowledge graph is finally completed.

2. The method for constructing an NLP knowledge graph combining information content and BERT-BiLSTM-CRF according to claim 1, characterized in that: In the step 1), the model layer definition is completed by the following steps: Step 1.1), determine the domain and scope of ontology construction as the field of natural language processing; Step 1.2), analyze whether there is an ontology that can be reused in this field; Step 1.3), sort out and determine the important terms in the domain ontology; Step 1.4), determine the structural hierarchy of the classes in the ontology; Step 1.5), determine the attribute information of the class; Step 1.6), determine the constraints of class attributes; Step 1.7), instantiate the ontology.

3. The method for constructing an NLP knowledge graph combining information volume and BERT-BiLSTM-CRF according to claim 1, characterized in that: In the step 1), the entities and attributes are defined by constructing a natural language processing knowledge graph. The entities in the final design graph include: papers, research tasks, key terms, and attributes and relationships are defined based on these entities.

4. The method for constructing an NLP knowledge graph combining information content and BERT-BiLSTM-CRF according to claim 1, characterized in that: In the step 2), the crawled paper content includes title, author, publishing unit, publishing time, publishing journal, keywords, and abstract, and the crawled data is used to perform knowledge extraction to obtain knowledge data in the form of triples.

5. The method for constructing an NLP knowledge graph combining information content and BERT-BiLSTM-CRF according to claim 1, characterized in that: In the step 4), the text multi-classification model based on feature fusion includes a feature fusion module and a text classification module. The feature fusion module is: the pre-processed data is converted into a vector for representation by using the TF-IDF word vector algorithm, and then the features of the title, keywords, author, publication time, publication agency, and publication unit are fused and converted into a vector as the input of the text classification module; the text classification module uses the XGBoost algorithm for classification processing.

6. The method for constructing an NLP knowledge graph combining information content and BERT-BiLSTM-CRF according to claim 1, characterized in that: In the step 5), the neo4j graph database is used as a tool for storing and visualizing the natural language processing knowledge graph.

7. The method for constructing an NLP knowledge graph combining information content and BERT-BiLSTM-CRF according to claim 6, characterized in that: The neo4j interactive interface allows users to explore the graph or search the graph database using the Cypher query language.