Entity classification model training and predicting method based on digital humanity

Abstract

The invention discloses an entity classification model training and predicting method based on digital humanity, which comprises the following steps of: retrieving at least six types of entities from a part of reference books, extracting texts with part of speech nr from texts with special description prefixes and suffixes in character entities by utilizing an HMM (Hidden Markov Model) model, and taking the texts as cleaned samples; dividing the retrieved entities into a positive type and a negative type, inputting the entities into BertTokenizer to be converted into token vectors, and expanding the vectors into 16-dimensional vectors according to text length features of the entities; training the token text vector to obtain a BIRNN classification model structure; recognizing and calibrating the result of the BIRNN classification model, and generating dictionaries of different categories; and exporting the result into a database, and auditing, duplicate removal and information complementation are carried out.

Description

technical field [0001] The present invention relates to the fields of natural language processing and computer information processing technology, in particular to a digital humanities-based entity classification model training and prediction method. Background technique [0002] The existing digital humanities is an innovation in the field of big data. In recent years, it has emerged in various fields of humanities and has attracted the attention of many scholars at home and abroad. Due to the update and iteration of digital technology, personal computers are widely used, and even liberal arts scholars can use digital technology to carry out research work. In order to further improve the research efficiency of scholars, there is an urgent need for a visual method to display the humanities information of the past dynasties. Since the knowledge map has the function of structuring human knowledge, the information of the Humanities Encyclopedia is usually expressed in this way. ...

AI Technical Summary

Problems solved by technology

[0004] General text classification models are mainly divided into four types: one is to classify data based on the method of compiling rules, but the method of manually labeling entity categories on a large scale requires a high price

The second is a dictionary-based method, such as directly using a dictionary to match entities, but this requires a very comprehensive and powerful thesaurus, which is obviously not applicable in the classification of ancient Chinese entities

The third is the classification method based on traditional machine learning, such as SVM, etc., but it is necessary to manually define the rules of some features in the text. Judging from family surnames, when encountering irregular names (such as posthumous names of ancient emperors), place names, historical events, etc., and there is no more information to extract, this method will fail. The fourth is the classification based on the deep learning model method, currently widely used in text classification projects in the industry, but for entities, especially ancient Chinese entities, there is no special deep learning method for modeling and classification

Images