Automatic massive-text labeling method based on exception handling

Abstract

The invention discloses an automatic massive-text labeling method based on exception handling. The automatic massive-text labeling method based on the exception handling comprises the following steps of S1, extracting named entities and related named entity pairs from a knowledge base and respectively storing the named entities and the related named entity pairs; S2, finding the named entities of all sentences in a massive amount of texts through string matching according to the stored named entities, finding co-occurring sentences of all named entity pairs in the texts according to the stored named entity pairs and performing overstriking labeling; S3, judging the named entities in the sentences subjected to overstriking labeling, deleting abnormal word pairs when the abnormal word pairs exist, using a filter algorithm to filter out all abnormal named entities in the named entities when the abnormal named entities exist, and finally obtaining final text labeling results. The automatic massive-text labeling method based on exception handling can effectively solve the mistaken labeling problem in the labeling process and improves the text labeling accuracy rate.

Description

technical field [0001] The invention relates to a text labeling method, in particular to a massive text automatic labeling method based on exception processing, and belongs to the technical field of natural language processing. Background technique [0002] With the rapid development of science and technology, all kinds of information emerge in endlessly, even far beyond the ability of human beings to read. How to effectively use massive data and find out the needed information has been paid more and more attention by people. Information Extraction is a technique to help people use massive data. Its main purpose is to extract specific events, facts and other information from unstructured natural language texts, and then convert them into structured or semi-structured information, and then store them in the database for query and further analysis and utilization. , question answering system, text mining and other application systems provide an important foundation. Entity ...

AI Technical Summary

Problems solved by technology

[0004] However, there are many limitations in the remote supervised entity relationship extraction method. According to the training situation, the named entity parser can find limited types of named entities. For example, Stanford's named entity parser can only identify people, organizations The three named entity types of and location are less than one percent of the number of named entity types in Freebase, and the recall rate cannot well meet the needs of users

Secondly, the named entity parser cannot correctly mark all the entities of people, organizations and places, and there will be errors and omissions, which cannot effectively meet the needs of text labeling

Images