Unlock instant, AI-driven research and patent intelligence for your innovation.

Method and system for filtering bilingualism corpora

A technology of bilingual corpus and filtering method, applied in the field of bilingual corpus filtering method and system, can solve the problems of decreasing recall rate, decreasing accuracy rate, unable to cover the distribution of corpus, etc., and achieving the effect of improving accuracy rate and recall rate

Inactive Publication Date: 2010-06-02
BEIJING KINGSOFT SOFTWARE +2
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The feature threshold is set empirically, and the feature threshold may often be determined by the setter based on only a few corpus resources, which cannot cover the distribution of most corpora
Moreover, when the feature threshold set by experience is too low, the accuracy rate will drop, and if it is too high, the recall rate will drop.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for filtering bilingualism corpora
  • Method and system for filtering bilingualism corpora
  • Method and system for filtering bilingualism corpora

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] The invention provides a filtering method of a bilingual corpus, which is used to improve the universality, accuracy rate and recall rate of the corpus.

[0042] see reference figure 1 and figure 2 , figure 1 It is the flow chart of the first embodiment of the bilingual corpus filtering method of the present invention, figure 2 yes figure 1 Flow chart of building a classification model in .

[0043] The bilingual corpus filtering method described in the first embodiment of the present invention comprises the following steps:

[0044] S100. Determine the sentence length ratio characteristic value of the bilingual sentence pair.

[0045] Determine the number of words or characters used in a bilingual sentence pair. The number of words or characters in one of the two sentences is compared with the number of words or characters in the other of the above two sentences, and the obtained value is the characteristic value of the sentence length ratio.

[0046] When the...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a filtering method of a bilingual corpus and the method comprises the following steps: A. ratio flag value of sentence length of English-Chinese bilingual sentence pair is confirmed; B. the number of different parts of speech in the English-Chinese bilingual sentence pair is respectively counted, the matching number of the corresponding words in a bilingual intertranslatingdictionary and words of the part of speech are calculated and the interpretation eigenvalue is confirmed according to the number of different parts of speech and the matching number; C. the filtration and classification are carried out by the ratio eigenvalue of the sentence length and the interpretation eigenvalue according to a classification model established by using a training set in advance. The invention discloses a bilingual corpus system; the invention also provides a filtering method of the bilingual corpus and a system thereof, which are used for improving universality, accuracy and recalling rate of the corpus.

Description

technical field [0001] The invention relates to a corpus filtering method, in particular to a bilingual corpus filtering method and system. Background technique [0002] The great value of corpus resources for natural language processing research has been increasingly recognized. Especially the parallel bilingual corpus, which is a special corpus that contains information about mutual translation between two languages. Parallel bilingual corpora can provide rich matching information between two languages, and have important application value in the acquisition of translation knowledge, the establishment of bilingual dictionaries, machine translation based on statistics or examples, word sense disambiguation, etc., especially high-quality The role of corpus is more prominent. [0003] There are two main ways to build a corpus, one is the traditional method of manual collection; the other is to obtain the text-level aligned corpus through computer automatic sentence alignmen...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/27
Inventor 王刚高立琦刘挺王海洲
Owner BEIJING KINGSOFT SOFTWARE