Chinese text key information extraction method based on pre-trained language model

Abstract

The invention discloses a Chinese text key information extraction method based on a pre-trained language model, which comprises the following specific steps of: (1) classifying key information to be extracted, easily concluding information categories forming rules, and extracting by using a regular matching method; and (2) extracting the named entities by using a sequence labeling model. (3) constructing the sequence labeling model by adopting a method of finely adjusting a pre-training language model, wherein firstly, a large-scale unlabeled text corpus is used for learning to obtain the pre-training language model, and word boundary features are introduced in a pre-training stage; (4) replacing the data content matched by using the rule with the corresponding rule template label so as tocomplete fusion of rule matching and the deep network; and (5) performing fine adjustment on the pre-trained language model according to the marked training data, and migrating the pre-trained language model to the sequence marking task of the named entity. According to the method, text context semantic features can be effectively extracted, and each information type can be effectively identifiedin a complex information type scene.

Description

technical field [0001] The invention relates to a method for extracting key information of Chinese text based on a pre-trained language model, and belongs to the technical field of natural language processing and recognition. Background technique [0002] Text key information extraction refers to the identification and extraction of key data types specified in the text according to specific business needs. It mainly includes the recognition of named entities (Named Entity) and the recognition of some specific types of digital strings and character strings. The recognition of named entities can be better solved by using the sequence annotation model based on deep learning, but it cannot solve the recognition requirements of other numeric strings and character strings at the same time. Because the number strings cannot carry effective semantic information, and various number strings will interfere with each other. [0003] Most of the existing Chinese named entity recognitio...

AI Technical Summary

Problems solved by technology

[0006] Purpose of the invention: Aiming at problems such as polysemy of a word and lack of word boundary information that cannot be solved in traditional methods, the present invention proposes a key information extraction method based on a pre-trained language model

Images