Method for establishing Vietnamese dependency tree bank based on improved Nivre algorithm

Abstract

The invention relates to a method for establishing a Vietnamese dependency tree bank based on an improved Nivre algorithm, and belongs to the technical field of natural language processing. The method comprises the steps of firstly, establishing an initial training corpus, an expansion corpus and a test corpus; secondly training two dependency parsing weak learners S1 and S2 based on the improved Nivre algorithm by utilizing the established initial training corpus to serve as two fully redundant views; thirdly, performing dependency parsing on the expansion corpus by utilizing the two trained weak learners S1 and S2 and building a Vietnamese dependency tree bank model; and finally, performing dependency parsing testing on the test corpus and finally establishing the Vietnamese dependency tree bank. According to the method, the powerful support can be provided for upper applications of syntactic analysis, machine translation, information acquisition and the like of a Vietnamese language; the process of manually marking a dependency relation of Vietnamese sentences can be effectively avoided, so that the time of manpower and material resources is saved; and a large amount of unmarked Vietnamese sentence level corpora can be effectively utilized for improving the accuracy of dependency parsing.

Description

technical field [0001] The invention relates to a method for constructing a Vietnamese language dependency tree bank based on an improved Nivre algorithm, and belongs to the technical field of natural language processing. Background technique [0002] In the field of Vietnamese information processing research, some achievements have been made in lexical and bilingual alignment methods, but there is still little work in dependency syntax analysis and dependency treebank construction. With the rapid development of statistical learning, the use of statistical learning to study language information processing has become the mainstream. Among them, Lai et al. solved the problem of Chinese dependency analysis through the method of statistical learning based on the idea of ​​span in 2001; Yamada et al. completely converted the English sentences in Penn Treebank into a dependency structure in 2003, and then used the method of statistical learning Modeling and analysis of sentences ...

AI Technical Summary

Problems solved by technology

[0003] The present invention provides a method for constructing a Vietnamese language dependency tree bank based on the improved Nivre algorithm, to solve the problem of difficult labeling of Vietnamese sentence dependencies, and to effectively utilize a large number of unmarked Vietnamese sentence sub-level corpus for tree bank construction , which can solve the difficulties brought by the treebank construction due to the small size of the initial training corpus; it can be used to effectively avoid the cumbersome process of manually labeling the dependencies of Vietnamese sentences, which fully saves the time of manpower and material resources; it can be used to effectively improve the Vietnamese language The accuracy of dependency analysis; to provide strong support for upper-level applications such as syntactic analysis, machine translation, and information acquisition of Vietnamese

Images