An automatic labeling method for massive texts based on exception handling

Abstract

The invention discloses an automatic massive-text labeling method based on exception handling. The automatic massive-text labeling method based on the exception handling comprises the following steps of S1, extracting named entities and related named entity pairs from a knowledge base and respectively storing the named entities and the related named entity pairs; S2, finding the named entities of all sentences in a massive amount of texts through string matching according to the stored named entities, finding co-occurring sentences of all named entity pairs in the texts according to the stored named entity pairs and performing overstriking labeling; S3, judging the named entities in the sentences subjected to overstriking labeling, deleting abnormal word pairs when the abnormal word pairs exist, using a filter algorithm to filter out all abnormal named entities in the named entities when the abnormal named entities exist, and finally obtaining final text labeling results. The automatic massive-text labeling method based on exception handling can effectively solve the mistaken labeling problem in the labeling process and improves the text labeling accuracy rate.

Description

technical field [0001] The invention relates to an automatic labeling method for massive texts, in particular to an automatic labeling method for massive texts based on exception processing, which belongs to the field of natural language processing. Background technique [0002] With the rapid development of today's science and technology, all kinds of information emerge in an endless stream, even far beyond the ability of human reading. How to effectively use massive data and find out the required information has attracted more and more attention. Information Extraction (Information Extraction) is a technology to help people use massive data. The main purpose of information extraction is to extract specific events, facts and other information from unstructured natural language texts, and then convert them into structured or semi-structured The information is then stored in the database for query and further analysis, providing an important basis for application systems such...

AI Technical Summary

Problems solved by technology

[0004] However, there are many limitations in the remote supervised entity relationship extraction method. According to the training situation, the named entity parser can find limited types of named entities. For example, Stanford's named entity parser can only identify people, organizations The three named entity types of and location are less than one percent of the number of named entity types in Freebase, and the recall rate cannot well meet the needs of users

Secondly, the named entity parser cannot correctly mark all the entities of people, organizations and places, and there will be errors and omissions, which cannot effectively meet the needs of text labeling

Images