Automatic identification ancient book introduction system and method based on deep learning model

Abstract

The invention discloses a system and a method for automatically identifying ancient book introduction based on a deep learning model. The system comprises a corpus preprocessing module; a language model pre-training module; a book introduction entry identification test module; and a book introduction item identification effect evaluation module. The method comprises the following steps: S1, selecting a target research corpus, manually processing and proofreading the target research corpus, and determining a model category; S2, compiling related model codes, modifying and adjusting model parameters according to corpora and research purposes, and performing iterative training on the model; S3, selecting a model evaluation method, scoring the model evaluation method according to a result, and selecting an optimal model for storage; S4, selecting the optimal model as a tool for the user to directly carry out the task in the future, and realizing automatic identification of ancient book introduction. The method has the advantages that a large amount of manual labor is avoided, recognition and extraction of ancient book introduction can be automatically realized directly by means of a computer technology, and the problem that large-scale ancient book text introduction corpora are difficult to construct is effectively solved.

Description

technical field [0001] The present invention relates to the technical field of computer software, in particular to an ancient book quotation automatic recognition system and method developed based on deep learning models SIKU-BERT and SIKU-RoBERTa. Background technique [0002] Different from references in modern Chinese texts, whose citations have a unified description format and summary at the end of the text, citations in ancient texts are scattered throughout the text in various forms, with neither obvious citation signs nor standardized formats. Therefore, the identification of citations in ancient texts is an extremely challenging task. In previous studies, obtaining citation entries mostly relied on manually reading volume by volume. However, in the face of the vast amount of ancient books, pure manual operation is already weak. For large-scale or even super-large-scale ancient text corpora, if you want to dig deep into the mutual reference relationship, it is obviou...

AI Technical Summary

Problems solved by technology

However, in the face of the vast amount of ancient books, pure manual operation is already weak. For large-scale or even super-large-scale ancient text corpora, if you want to dig deep into the mutual reference relationship, it is obviously difficult to manually mark them one by one as the predecessors did. Unrealistic