Fault-tolerant romanized input method for non-roman characters

Abstract

Fault-tolerant systems and methods to process and correct input spelling errors for non-Roman based languages such as Chinese, Japanese, and Korean (CJK) are disclosed. The method may be applied to a Chinese input method using pinyin. For example, the method may generally include receiving a pinyin input representing characters in Chinese, the input having at least one original pinyin, identifying potentially incorrect pinyins in the input, expanding each potentially incorrect pinyin to at least one additional alternative pinyin, each pair of potentially incorrect and corresponding alternative pinyin having a proximity measurement, converting each pinyin in the input and each alternative pinyin to Chinese characters, computing likelihoods of possible conversions of the pinyin input to Chinese characters, each possible Chinese conversion being a combination of the converted original and / or alternative pinyins of the input, the probabilities being based on the proximity measurement and optionally on a context of the possible Chinese conversion, and determining a most likely Chinese conversion from the possible conversions.

Description

BACKGROUND OF THE INVENTION [0001] 1. Field of the Invention [0002] The present invention relates generally to processing non-Roman based languages. More specifically, fault-tolerant systems and methods to process and correct input spelling errors for non-Roman based languages such as Chinese, Japanese, and Korean (CJK) are disclosed. [0003] 2. Description of Related Art [0004] Spell correction generally includes detecting erroneous words and determining appropriate replacements for the erroneous words. Most spelling errors in alphabetical, i.e., Roman-based, languages such as English are either out of vocabulary words, e.g., “thna” rather than “than,” or valid words improperly used in its context, e.g., “stranger then” rather than “stranger than.” Spell checkers that detect and correct out of vocabulary spelling errors in Roman-based languages are well known. [0005] Users of non-Roman based languages such as Chinese, Japanese, and Korean (CJK) often utilize Roman-based (alphabetica...

AI Technical Summary

Benefits of technology

[0019] These and other features and advantages of the present invention will be presented in more detail in the follo

Problems solved by technology

However, Chinese language users may not know the correct pronunciations (pinyins) of some Chinese characters due to, for example, their dialect and / or accent, and therefore may enter incorrect pinyin inputs.

However, the user's intended character set may not be included in the candidate list as most pinyin input methods have a low or no fault tolerance.

Spell correction for non-Roman languages such as CJK languages is also complex and challenging in that there are no standard dictionaries in such languages because the definition of CJK words are not clean.

In contrast, the English dictionary / wordlist lookup is a key feature in English spell correction and thus English spell correction methods cannot be easily adapted for use in CJK languages.

Furthermore, the Chinese language has a high concentration of homographs and homophones as well as invisible (or hidden) word boundaries that create ambiguities that also make efficient and effective Chinese spell correction complex and difficult to implement.

As is evident with such differences between Chinese and English, many efficient techniques available for English spell correction are not suitable for Chinese spell correction.

Images