Social media named entity identification method based on affix perception

Abstract

The invention discloses a social media named entity identification method based on affix perception. The method comprises the following steps: collecting a social media data set marked with named entities; capturing embedded representation, character level representation and affix feature representation of the words, and fusing the embedded representation, the character level representation and the affix feature representation of the words to serve as final representation of the words; inputting the obtained final representation of the word into a bidirectional convolutional neural network anda conditional random field, predicting a label sequence and calculating a loss value; training the model by adopting a random gradient descent algorithm according to the obtained loss value; and inputting the text into the trained model, and identifying the named entity in the text. According to the method, semantic representation of the words is enriched, the problem of unregistered words in thesocial media data is relieved, and the named entity recognition effect is improved.

Description

technical field [0001] The present invention relates to the technical field of natural language processing, in particular to a social media named entity recognition method based on affix perception. Background technique [0002] In today's world, with the vigorous development of the mobile Internet, people publish information on social media all the time, which constitutes a huge amount of social media data. Compared with traditional newswire manuscript data, data on social media is more time-sensitive. These data contain rich information and have gradually become a potential source of information for many applications, such as news hotspot tracking, user public opinion analysis, and potential violent incidents. Early warning, etc. Therefore, how to mine potential information from social media data has become an important task. Entity extraction is a fundamental task of information extraction, and building a powerful entity extraction system for these applications is essen...

AI Technical Summary

Problems solved by technology

Although these methods have achieved good performance on news texts, when faced with social media data, due to the inherent characteristics of social media, such as informal expressions, irregular noun abbreviations, ungrammatical expressions, With more unregistered words, etc., the performance of such methods will drop sharply

Images