Transliteration Probability Calculation for Unknown Source Languages
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing transliteration methods cannot accurately determine whether a string is a transliteration of another string when the original language of the string to be transliterated is unknown, as they fail to switch models or calculate probabilities effectively in such scenarios.
Innovation Solution
A transliteration device and method that generate and update rewriting and transliteration tables based on training sets, using the EM algorithm to maximize the likelihood function, allowing for the calculation of transliteration probabilities and origin probabilities to determine the accuracy of transliteration relationships across different languages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the original language of the string is unknown, then the existing transliteration methods cannot switch models or calculate probabilities, but the patent enables accurate determination of transliteration relationships by generating and updating rewriting tables for multiple languages simultaneously
Solution Approach 1:
The patent segments the transliteration problem by creating separate rewriting tables for each target language (English, French, German, Spanish, Italian) with language-specific rewriting probabilities. This segmentation allows the system to handle unknown original languages by evaluating multiple language-specific models simultaneously and selecting the best match, thereby resolving the contradiction between measurement precision and adaptability to unknown languages.
Solution Approach 2:
The patent creates a universal transliteration determination system that can handle multiple original languages (Chinese, Japanese, English, and others) through a single framework. The system generates rewriting tables for multiple target languages and uses EM algorithm to update probabilities across all languages, making the system versatile for unknown original languages while maintaining high accuracy through multi-language probability calculations.
2Measurement precision
If multiple rewriting tables for different languages are generated and updated using EM algorithm, then the transliteration probability calculation becomes more accurate, but the computational complexity increases
Solution Approach 1:
The patent employs the EM (Expectation-Maximization) algorithm to dynamically update rewriting probabilities in the rewriting tables based on training data. This dynamic update mechanism allows the system to iteratively improve transliteration probability calculations by adjusting probabilities according to observed data patterns, resolving the contradiction between measurement precision and device complexity through adaptive learning rather than static rules.
Solution Approach 2:
The EM algorithm implements a feedback mechanism where the system calculates transliteration probabilities, compares them with actual training data, and uses this feedback to update the rewriting probabilities in subsequent iterations. This feedback loop enables the system to progressively improve accuracy while managing complexity through automated learning from data rather than manual rule creation for each language pair.
Data Source
AI summary
A transliteration device (100) comprises a generation part (105) generating rewriting tables corresponding to K different languages and including multiple rewriting probabilities that an original segment can be rewritten as a transliterated segment for transliteration and transliteration tables corresponding to the K languages. The transliteration device (100) further comprises an update part (107) saving the transliteration probability that an original spelling string originating from a language corresponding to the used rewritten language is transliterated to a target spelling string in the transliteration able corresponding to the language, and so updating the K rewriting tables as to maximize the expected value of a likelihood function calculating the likelihood of the K transliteration tables.