Noise reduction device and method for deducing classification data set based on natural language

Abstract

The invention discloses a noise reduction device and method for deducing a classification data set based on a natural language. A data format conversion module constructs templates for various features in a relation classification data set, converts triples in the relation classification data set into assumptions in natural language inference, and converts corresponding text corpora into prerequisites in the natural language inference; if high-quality annotation data can be divided in the original data set, the data set is directly used as a training set, and supervised learning is used for training a natural language inference model; the noise reduction effect of the current model on the verification set serves as feedback, and parameters of a natural language inference model are trained through a reinforcement learning method; the data set noise reduction template evaluates a relation classification data set obtained by remote supervision through a trained natural language inference model, and selects a data set with high confidence as a denoised data set according to scores.

Description

technical field [0001] The invention belongs to the technical field of data processing methods, and in particular relates to a noise reduction device and method for a remote supervision relationship classification data set based on natural language inference. Background technique [0002] The task of relation classification is to predict the semantic relationship between two entities from a given text. Understanding the relationship of entities is essential for many downstream applications, such as tasks such as knowledge graph completion and question answering. Relation classification tasks usually rely on large-scale human-annotated data, which is expensive and time-consuming. In order to solve this problem, remote supervision is often used to automatically label a large amount of corpus. Distant supervision is based on the assumption that if a sentence contains an entity pair in the knowledge base, then it can be considered that the entity pair in the sentence also has ...

AI Technical Summary

Problems solved by technology

However, they are computationally expensive, and the performance of these methods needs to be improved

Images