Corpus intention prediction method, corpus labeling method and electronic equipment

Abstract

The invention relates to a natural language processing technology, and provides a corpus intention prediction method, which comprises the following steps: training to obtain N prediction models on thebasis of a preprocessed sample; predicting a corpus to be predicted based on each prediction model to obtain N prediction results; and matching a preset rule based on the N prediction results, and determining intention information corresponding to the to-be-predicted corpus, wherein N is an odd number greater than or equal to 3, and the preset rule comprises the step of determining the same prediction result as intention information corresponding to the to-be-predicted corpus if the same prediction result exists in the N prediction results and the number of the same prediction results is greater than N / 2. Based on the corpus intention prediction method provided by the embodiment of the invention, the intention prediction of the corpus is realized, and the prediction accuracy is improved,so that repeated manual processing work can be greatly reduced. In addition, the invention further provides a corpus labeling method and electronic equipment.

Description

technical field [0001] The invention relates to natural language processing technology, in particular to a corpus intent prediction method, a corpus labeling method and electronic equipment. Background technique [0002] Corpus is the basic resource of corpus linguistics research and the main resource of empirical language research methods. Traditional corpora are mainly used in dictionary compilation, language teaching, traditional language research, statistical or case-based research in natural language processing, etc. With the development of Internet big data and artificial intelligence technology, corpus has also been widely used. [0003] The corpus has three characteristics. The corpus stores the language materials that have actually appeared in the actual use of the language, such as user messages and customer service dialogues obtained directly from the web page. The corpus is the basic resource for carrying language knowledge, but it does not mean Language knowle...

AI Technical Summary

Problems solved by technology

Most of the labeling work is based on manual labeling. In most cases, the corpus has not been processed in advance, and there will be a large amount of duplicate data. If these duplicate data are not filtered, one will affect the efficiency of labeling, and the other will be a waste of manpower.

Images