Text labeling method and device based on teacher supervision

Abstract

The embodiment of the invention provides a text labeling method and device based on teacher supervision, and belongs to the technical field of natural language processing. The method comprises: usinga character labeling model to label a to-be-labeled text, and generating a character labeling result containing labeling words; carrying out word segmentation processing on the to-be-labeled text through a preset word segmentation model to generate a word segmentation result containing word segmentation words; and according to the similarity between each annotated word and each segmented word, carrying out character annotation on the character annotation result again based on the segmented words to obtain a fusion annotation result, and outputting the fusion annotation result. By adopting themethod and the device, the text labeling accuracy and recall rate can be improved.

Description

technical field [0001] The invention relates to the technical field of natural language processing, in particular to a teacher-supervised text labeling method and equipment. Background technique [0002] Natural language processing (Natural Language Processing, NLP) technology can efficiently systematically analyze, understand and extract text data, so that computers can understand natural language and generate natural language, and then realize effective communication between humans and computers using natural language. Interaction (e.g. use of applications such as message auto-reply, voice assistant, etc.). Among them, text annotation technology provides a basis for the industrial application of natural language processing. [0003] Traditional machine learning (Machine Learning, ML) can learn a certain amount of text data, combined with keywords (Seed Words) to mine the associated features between texts, obtain a traditional machine learning model, and use the traditiona...

AI Technical Summary

Problems solved by technology

Due to the continuous development of natural language processing technology, the existing deep learning model based on character granularity is not enough to meet the increasing accuracy requirements of natural language processing technology for text annotation

Moreover, when a well-trained deep learning model is applied to a new field, the recall rate of the deep learning model is insufficient or even zero, resulting in poor generalization of the deep learning model and easy curing of word edge labels

Images