Deep learning machine reading understanding training method based on course learning

Abstract

The invention provides a deep learning machine reading understanding training method based on curriculum learning, and the method employs a BERT pre-training language model to build (articles, questions and options) triples into a sequence, and does not need to carry out the independent operation of each tuple. Four option sequences forming a question are input into a network, a fine adjustment process the same as BERT is carried out, a maximum probability option is selected as a prediction answer through a full connection layer and a softmax classification layer, and parameters of a model arereversely updated by maximizing the logarithm probability of a correct answer, so that the model learns text information. A three-stage training framework is firstly finely adjusted in a simple dataset; text knowledge can be learned in sequence from shallow to deep by conducting fine adjustment on a common data set, and finally the test effect after training on a difficult data set is 2.5% higher than the accuracy of fusion learning (fine adjustment is conducted on a set formed by mixing a simple data set and the common data set, and then training is conducted on the difficult data set).

Description

technical field [0001] The present invention relates to the technical field of natural language processing, and more specifically, to a deep learning machine reading comprehension training method based on course learning. Background technique [0002] In today's era of big data, the Internet generates massive amounts of text information every day. How to efficiently and accurately extract the required information from these complicated information is particularly important. Machine Reading Comprehension (MRC) with natural language processing technology as its core can help people complete these tasks—teaching machines to read and understand, and answer questions raised. Machine reading comprehension is widely used in business applications, such as answering general questions raised by people based on Wikipedia documents, answering some queries raised by employees based on company financial reports, or solving troubles encountered by users in the process of using products bas...

Images