Tourism named entity identification method based on BBLC model

Abstract

The invention discloses a tourism named entity recognition method based on a BBLC model, and the method comprises the steps: carrying out the BIO marking of statements in a corpus, and obtaining a BIOmarking set; inputting the BIO annotation set into a BERT pre-training language model, and outputting vector representation of each word in the statement, namely a word embedding sequence in each statement; 3, taking the word embedding sequence as the input of each time step of the bidirectional LSTM, and carrying out the further semantic coding to obtain a statement feature matrix; taking the statement feature matrix as input of a CRF model, labeling and decoding the statement x to obtain a word label sequence of the statement x, outputting a probability value that a label of the statement xis equal to y, solving an optimal path by using a dynamically planned Viterbi algorithm, and outputting a label sequence with the maximum probability. According to the method, local context information can be obtained by adding the BERT pre-training language model, the accuracy, recall rate and F value are higher, the generalization ability and robustness are stronger, and the defects of a traditional model can be overcome.

Description

technical field [0001] The invention belongs to the technical field of semantic recognition, and relates to a method for recognizing tourism named entities based on a BBLC model. Background technique [0002] With the rise of the tourism industry, the volume of tourism data has become larger and larger. While enriching the field, the complexity of information acquisition caused by massive data greatly reduces the efficiency of people's information acquisition. Obtaining more useful travel information in a short time has become an important demand for tourism in the era of big data. A large amount of structured information on existing travel websites provides people with great convenience, but there are more information that can better reflect the user's tendency in texts such as travel notes, strategies, and comments, so from unstructured Extracting useful information from the text is the focus of research, and its essence is to improve the efficiency of named entity recog...

AI Technical Summary

Problems solved by technology

[0004](1) Entities are constantly being upgraded, unregistered words are constantly appearing, and it is difficult to enumerate them using dictionaries

For example, it is unrealistic to put all the names of cities and all people in the dictionary at present. With the development of the times, the continuous generation of new words will bring greater difficulties to named entity recognition;

[0005](2) There is no obvious characteristic mark between Chinese words, unlike English, which has spaces and upper and lower case as distinctions;

[0007](4) There are a large number of polysemy words in Chinese words, such as apple in different contexts can be the name of fruit, movie name or company name, in the recognition process Caused a large number of recognition errors caused by ambiguity;

[0009] In the current named entity recognition, most researchers use Word2Vec to obtain the word embedding of each word (Word Embedding), but such word embedding When encoding a word, it cannot distinguish the ambiguity of the same word due to the context, resulting in a large number of recognition errors

Images