A method and device for interactively extracting comparable corpus and bilingual dictionaries

Abstract

The invention relates to a method for interactively extracting a comparable corpus and a bilingual dictionary and a device thereof, and aims to overcome the defects of difficulty in identifying the comparable corpus under the condition of insufficient domain seed bilingual dictionary scale and difficulty in extracting inter-translation vocabulary under the condition of different comparable degrees. The method comprises the following steps: performing word characteristic reduction, word segmentation and stop word removing on a document to obtain a preprocessed document set and a vocabulary set; constructing relations between a source language document and a target language document, between source language vocabulary and target language vocabulary and between a bilingual vocabulary pair and a bilingual document pair respectively; iterating, enhancing and calculating the weights of the bilingual document pair and the bilingual vocabulary pair; selecting a bilingual document pair of which the weight is the largest for constructing the comparable corpus, and selecting a bilingual vocabulary pair of which the weight is the largest for constructing the bilingual dictionary. The judgment that similarity among different language vocabularies is facilitated through the similarity among different language documents is performed, the similarity among different language documents is increased through the similarity among different language vocabularies, and synchronous extraction of the comparable corpus and the bilingual dictionary is realized through interactive iteration and enhancement.

Description

technical field [0001] The present invention relates to the technical field of cross-language information processing, in particular to a method and device for interactively extracting comparable corpus and bilingual dictionaries. Background technique [0002] Bilingual comparable corpus and bilingual dictionaries are two kinds of cross-lingual resources with different granularities, which are of great value to cross-lingual information processing such as statistical machine translation and cross-lingual information retrieval. The comparable corpus is composed of document pairs with different languages ​​and similar content but not inter-translatable, from which excavating translation equivalence pairs with different granularities such as bilingual inter-translation vocabulary, bilingual named entities, and parallel sentence pairs can effectively solve the new problems faced by bilingual dictionary compilation. More fine-grained bilingual knowledge such as bilingual dictionar...

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to solve the defect that it is difficult to identify comparable corpus under the condition of insufficient scale of bilingual dictionaries in the field seed and it is difficult to extract inter-translation vocabulary under the condition of different degrees of comparability, and to provide a method and device for interactively extracting comparable corpus and bilingual dictionaries solve the above problems

Images