Machine reading understanding method for complex data

Abstract

The invention discloses a machine reading understanding method for complex data, which comprises the following steps: S1, preprocessing: carrying out word segmentation on original data information, counting word frequencies in corpora, selecting vocabularies with higher word frequencies to form a vocabulary, and replacing words which are not in the vocabulary with special marks; S2, establishmentof a convolutional neural network-based paragraph ranking model, wherein the model is used for ranking a plurality of paragraphs in an article, and selecting the paragraph with the highest ranking foranswer extraction; and S3, an answer prediction method based on Gaussian distribution: improving an extraction method in a reading understanding task, and converting answer positioning using a classification method into a probability learning method based on Gaussian distribution. The invention provides a paragraph ranking model and Gaussian distribution answer prediction method for the noise problem and the long answer problem of a machine reading understanding task in a complex data set, and the practicability of the machine reading understanding task in a complex scene is effectively improved.

Description

technical field [0001] The invention relates to a machine reading comprehension method for complex data, which belongs to the technical field of natural language processing. Background technique [0002] In recent years, deep learning technology has developed rapidly, and has surpassed human level in areas such as image recognition, speech recognition and Go. Natural language processing is the core technology for realizing artificial intelligence. In the field of natural language processing, deep learning technology has also made important breakthroughs in machine translation and human-computer dialogue. As the core technology in the field of natural language processing, machine reading comprehension has achieved rapid development with the help of deep learning technology. [0003] Machine reading comprehension, as the name suggests, is to let the machine learn to read and understand articles, that is, to find answers from related articles for a given question. Machine rea...

AI Technical Summary

Problems solved by technology

[0009] The purpose of the present invention is to propose a solution to the problem that the length of the answer in the real data mentioned in the background technology is relatively long, and there is inconsistency between the training target and the evaluation index in the current machine reading comprehension prediction method for processing this type of data. A machine reading comprehension method for complex data, specifically an answer prediction method based on Gaussian distribution, to greatly improve the consistency between the two, and solve the noise problem and long answer problem in real application scenarios

Images