Custom language models

Abstract

Systems, methods, and apparatuses including computer program products for generating a custom language model. In one implementation, a method is provided. The method includes receiving a collection of documents; clustering the documents into one or more clusters; generating a cluster vector for each cluster of the one or more clusters; generating a target vector associated with a target profile; comparing the target vector with each of the cluster vectors; selecting one or more of the one or more clusters based on the comparison; and generating a language model using documents from the one or more selected clusters.

Description

technical field

[0001] This specification deals with language models. Background technique

[0002] Language models are used to model the probability that strings of tokens (eg, words or characters) in a given vocabulary will occur in a language. For example, language models are used in input methods such as, but not limited to, input method editors (IME), automatic speech recognition (ASR), machine translation, handwriting recognition, and optical character recognition (OCR) applications. Modeling the probability of strings of symbols in the vocabulary is typically performed using the chain rule and computing the probability p(w|context) of a given symbol w in the context of a given string, where the context is the The symbol in the string preceding the given symbol w.

[0003] In an n-gram (n-gram) language model, n consecutive symbols in the text form an n-gram, and the probability of the current word z depends, for example, on the probabilities of n-1 previous words, e...

Examples

Embodiment Construction

[0029] review

[0030] figure 1 is a diagram illustrating an example system 100 for generating a custom language model 124 . System 100 includes a first client 112 (eg, a desktop computer), a second client 114 (eg, a cell phone), and a server 120 . Server 120 may include a target profile 122, a custom language model 124, and training data 126 (eg, an unannotated corpus).

[0031] As an overview for generating custom language models, training data 126 may be classified into sets of training data 126 (eg, one or more sets of documents). Server 120 may receive user input identifying a user (eg, a first user on first client 112 , a second user on first client 112 , a first user on second client 114 ). For example, server 120 may use a user login or a cookie to identify a user. The server 120 may generate a target profile 122 (eg, a user profile) corresponding to each user.

[0032]In some implementations, target profile 122 may be predetermined. For example, server 120 may i...

technical field

[0001] This specification deals with language models. Background technique

[0002] Language models are used to model the probability that strings of tokens (eg, words or characters) in a given vocabulary will occur in a language. For example, language models are used in input methods such as, but not limited to, input method editors (IME), automatic speech recognition (ASR), machine translation, handwriting recognition, and optical character recognition (OCR) applications. Modeling the probability of strings of symbols in the vocabulary is typically performed using the chain rule and computing the probability p(w|context) of a given symbol w in the context of a given string, where the context is the The symbol in the string preceding the given symbol w.

[0003] In an n-gram (n-gram) language model, n consecutive symbols in the text form an n-gram, and the probability of the current word z depends, for example, on the probabilities of n-1 previous words, e...

Examples

Embodiment Construction

[0029] review

[0030] figure 1 is a diagram illustrating an example system 100 for generating a custom language model 124 . System 100 includes a first client 112 (eg, a desktop computer), a second client 114 (eg, a cell phone), and a server 120 . Server 120 may include a target profile 122, a custom language model 124, and training data 126 (eg, an unannotated corpus).

[0031] As an overview for generating custom language models, training data 126 may be classified into sets of training data 126 (eg, one or more sets of documents). Server 120 may receive user input identifying a user (eg, a first user on first client 112 , a second user on first client 112 , a first user on second client 114 ). For example, server 120 may use a user login or a cookie to identify a user. The server 120 may generate a target profile 122 (eg, a user profile) corresponding to each user.

[0032]In some implementations, target profile 122 may be predetermined. For example, server 120 may i...

Images