Paper fragmentation information extraction method based on machine learning

Abstract

The invention discloses a paper fragmentation information extraction method based on machine learning. The method comprises the steps of extracting PDF text contents, pictures and tables by using XPDF, and saving in an xml form; performing data analysis on paragraph texts in the xml, calculating and extracting feature vectors of each paragraph block para, converting the feature vectors of each para into feature vectors of a machine learning model, analyzing according to the selection and accuracy of the machine model to select reasonable feature vectors, and using the obtained feature vectorsof paragraphs to train a support vector machine model and a random forest model; predicting a title and structure information of a target PDF article according to the feature vectors of the machine learning model, and saving in a database in an xml format. The method makes full use of the advantages of machine learning in information classification, selects sample features to construct a trainingset and selects the RF (random forest) model to complete the information extraction based on the machine learning.

Description

technical field [0001] The invention relates to the technical field of information extraction, in particular to a machine learning-based method for extracting paper fragmented information. Background technique [0002] With the development of the Internet and information technology, big data has become the most popular term in various fields. In the face of massive information and data resources, quickly obtaining potential and useful knowledge is an important direction of data mining today. Academic papers have strong professionalism and accuracy. The information and data in the papers can play a great role in many professional fields, and can provide underlying data support for many application technologies. Therefore, it is very meaningful to extract information and data from academic papers. [0003] At present, academic papers at home and abroad are mostly stored in PDF format. There are two main ways to extract the content of PDF documents. One is to directly extract...

AI Technical Summary

Problems solved by technology

Although great achievements have been made in metadata classification and extraction, the effect of the above individual methods is not ideal in some cases due to the complexity and variety of academic papers.

And most of the traditional studies only focus on the extraction of metadata, and do not give a good description of the content structure of the paper and the information and data in the content.

From previous studies, it can be found that the single method is sometimes particularly effective in the process of extracting metadata, and sometimes the effect is very poor.

Images