Method and system for generating ancient Chinese annotation model

Abstract

The invention provides a method and system for generating an ancient Chinese annotation model. The method comprises the following steps: S1, training and forming a language model capable of representing context semantics; s2, constructing a multi-task joint learning model; and S3, simultaneously training the language model and the multi-task joint learning model, wherein the language model and the multi-task joint learning model form an ancient Chinese annotation model. According to the invention, punctuations, quotations, book names and entities can be marked at the same time, the F1 index on tasks such as automatic punctuation, book name and entity identification reaches more than 90%, and an ancient Chinese information marking effect similar to that of manual work is realized.

Description

technical field [0001] The invention relates to the technical field of natural language processing, in particular to a method and system for generating an ancient Chinese annotation model. Background technique [0002] Chinese classics are an important carrier of Chinese traditional culture, but an important feature of ancient Chinese expression is the lack of punctuation, and the grammar and semantic expressions are quite different from modern vernacular Chinese. Therefore, in the collation and publication of ancient books, it is often necessary to rely on experts to manually punctuate sentences, mark proper names, annotate, translate, etc., to assist modern people in understanding ancient texts. It takes a lot of time and energy to sort out ancient books manually, and it has high requirements for the sorters' knowledge of ancient Chinese, so that a large number of ancient books are in an unsorted state due to lack of manpower. Since the sentence punctuation and proper nam...

AI Technical Summary

Problems solved by technology

[0009] (1) The processing of ancient texts is only for individual tasks (such as punctuation, entities) or for specialized fields (such as Chinese medicine), lack of general ancient text information annotation technology

[0010] (2) The existing technology does not take into account the knowledge relevance of tasks such as ancient book punctuation and entities, and only models for a single task, and it is difficult for the model to capture the dependencies between different tasks

[0013] (3) The training of deep learning models often relies on large-scale labeled data, while ancient Chinese annotation resources are often scarce and insufficient, especially the named entity data of ancient books

Existing methods often directly use the deep learning model to train the existing data, without considering the problem of insufficient learning and difficulty in convergence of the model on small samples

[0014] Due to the above deficiencies in the existing technology, the annotation effect of the text information of ancient books is not good

Images