Probabilistic Spelling Model for Geographic Entity Names
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional spelling checkers are ineffective in domains with large vocabularies and varied word combinations, such as geographic entity names, due to the immense number of possible valid combinations and language-specific ordering conventions.
Innovation Solution
A computer-implemented automated spelling system generates a language model by calculating probabilities for query entity types and their orderings, and computes conditional probabilities to suggest alternative spellings for user queries, reducing the number of possible orderings to a manageable set.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional spelling checkers store all combinations of validly-spelled words with probabilities, then spelling correction can be achieved in domains with restrictive grammatical rules, but the system becomes infeasible in domains with large vocabularies and varied word combinations such as geographic entity names
Solution Approach 1:
The patent segments the language model into two distinct components: a template distribution model that captures ordering patterns of entity types, and a location distribution model that captures entity-specific probabilities. This segmentation allows the system to handle large vocabularies by separating structural patterns from content-specific information, making the overall system feasible for domains with extensive vocabularies like geographic entity names.
2Adaptability or versatility
If the language model includes all possible orderings of entity names, then complete coverage of valid combinations is achieved, but the number of orderings becomes extremely large and computationally infeasible
Solution Approach 1:
The patent extracts and removes the ordering structure from the vocabulary combinations by using templates that represent only the sequence of entity types (e.g., STREET, CITY, STATE). This extraction reduces the language model from needing to store all possible named entity combinations to storing only the structural patterns, dramatically reducing the quantity of data while maintaining adaptability to various ordering conventions.
3Adaptability or versatility
If the system accounts for multiple language conventions and naming variations, then adaptability to different user backgrounds is improved, but the complexity of handling varied ordering conventions increases
Solution Approach 1:
The template distribution model serves as a universal framework that can represent multiple language conventions and ordering patterns through a single probabilistic structure. Rather than creating separate models for different languages or conventions, the system uses a unified template-based approach that can accommodate various ordering patterns (e.g., STREET-CITY-STATE vs. CITY-DISTRICT-STREET) within the same model structure, reducing complexity while maintaining multi-language support.
Data Source
AI summary
A spelling system derives a language model for a particular domain of structured data, the language model enabling determinations of alternative spellings of queries or other strings of text from that domain. More specifically, the spelling system calculates (a) probabilities that the various query entity types—such as STREET, CITY, or STATE for queries in the geographical domain—are arranged in each of the various possible orders, and (b) probabilities that an arbitrary query references given particular ones of the entities, such as the street “El Camino Real.” Based on the calculated probabilities, the spelling system generates a language model that has associated scores (e.g., probabilities) for each of a set of probable entity name orderings, where the total number of entity name orderings is substantially less than the number of all possible orderings. The language model can be applied to determine probabilities of arbitrary queries, and thus to suggest alternative queries more likely to represent what a user intended.