35 results about "Bigram" patented technology

A method for identifying data that is meaningless and generating a natural language statistical model which can reject meaningless input. The method can include identifying unigrams that are individually meaningless from a set of training data. At least a portion of the unigrams identified as being meaningless can be assigned to a first n-gram class. The method also can include identifying bigrams that are entirely composed of meaningless unigrams and determining whether the identified bigrams are individually meaningless. At least a portion of the bigrams identified as being individually meaningless can be assigned to the first n-gram class.

The present invention leverages iterative transformations of search query strings along with statistics extracted from search query logs and / or web data to provide possible alternative spellings for the search query strings. This provides a spell checking means that can be influenced to provide individualized suggestions for each user. By utilizing search query logs, the present invention can account for substrings not found in a lexicon but still acceptable as a search query of interest. This allows a means to provide a higher quality proposal for alternative spellings, beyond the content of the lexicon. One instance of the present invention operates at a substring level by utilizing word unigram and / or bigram statistics extracted from query logs combined with an iterative search. This provides substantially better spelling alternatives for a given query than employing only substring matching. Other instances can receive input data from sources other than a search query input.

The present invention relates to an error-tolerant language understanding, system and method. The system and the method is using example sentences to provide the clues for detecting and recovering errors. The procedure of detection and recovery is guided by a probabilistic scoring function which integrated the scores from the speech recognizer, concept parser, the scores of concept-bigram and edit operations, such as deleting, inserting and substituting concepts. Meanwhile, the score of edit operations refers the confidence measure achieving more precise detection and recovery of the speech recognition errors. That said, a concept with lower confidence measure tends to be deleted or substituted, while a concept with higher one tends to be retained.

A method and a system for making merging decisions for a translation are disclosed which are suited to use where the target language is a productive compounding one. The method includes outputting decisions on merging of pairs of words in a translated text string with a merging system. The merging system can include a set of stored heuristics and / or a merging model. In the case of heuristics, these can include a heuristic by which two consecutive words in the string are considered for merging if the first word of the two consecutive words is recognized as a compound modifier and their observed frequency f1 as a closed compound word is larger than an observed frequency f2 of the two consecutive words as a bigram. In the case of a merging model, it can be one that is trained on features associated with pairs of consecutive tokens of text strings in a training set and predetermined merging decisions for the pairs. A translation in the target language is output, based on the merging decisions for the translated text string.

The present invention discloses a method for semantically processing speech for speech recognition purposes. The method can reduce an amount of memory required for a Viterbi search of an N-gram language model having a value of N greater than two and also having at least one embedded grammar that appears in a multiple contexts to a memory size of approximately a bigram model search space with respect to the embedded grammar. The method also reduces needed CPU requirements. Achieved reductions can be accomplished by representing the embedded grammar as a recursive transition network (RTN), where only one instance of the recursive transition network is used for the contexts. Other than the embedded grammars, a Hidden Markov Model (HMM) strategy can be used for the search space.

Systems and methods are described that facilitate phrase-based statistical machine translation (SMT) incorporating bigram (or higher n-gram) language models by modeling bi-phrases as nodes in a graph. Additionally, construction of a translation is modeled as a “tour” amongst the nodes of the graph, such that a translation solution is generated by treating the graph as a generalized traveling salesman problem (GTSP) and solving for an optimal tour. The overall cost of a tour is computed by adding the costs associated with the edges traversed during the tour. Thus, the described systems and methods map the SMT problem directly into a GTSP problem, which itself can be directly converted into a TSP problem.

Systems and associated methodology are presented for Arabic handwriting synthesis including partitioning a dataset of sentences associated with the alphabet into a legative partition including isolated bigram representation and classified words that contain ligature representations of the collected dataset, an unlegative partion including single character shape representation of the collected data set, an isolated characters partition, and a passages and repeated phrases partition, generating a pangram, the pangram including the occurrence of every character shape in the collected dataset and further including a special lipogram condition set based on a desired digital output of the collected dataset, and outputting a digital representation of the pangram including synthesized text.

Techniques are provided for performing sentiment analysis on words in a first data set. An example embodiment includes generating a word embedding model including a first plurality of features. A value indicating sentiment for the words in the first data set can be determined using a convolutional neural network (CNN). A second plurality of features are generated based on bigrams identified in the data set. The bigrams can be generated using a co-occurrence graph. The model is updated to include the second plurality of features, and sentiment analysis can be performed on a second data set using the updated model.

Systems and associated methodology are presented for Arabic handwriting synthesis including partitioning a dataset of sentences associated with the alphabet into a legative partition including isolated bigram representation and classified words that contain ligature representations of the collected dataset, an unlegative partition including single character shape representation of the collected data set, an isolated characters partition, and a passages and repeated phrases partition, generating a pangram, the pangram including the occurrence of every character shape in the collected dataset and further including a special lipogram condition set based on a desired digital output of the collected dataset, and outputting a digital representation of the pangram including synthesized text.

The invention discloses a text positive and negative emotion classification method. The method comprises the steps that all texts in a text set are preprocessed to form a noiseless positive and negative text set; unigram word segmentation and bigram word segmentation are performed on positive and negative texts; after stop words are removed, a non-repeat multidimensional feature vector space is formed; inverse document frequency calculation is performed on variant word frequency of all-dimensional feature vectors in the multidimensional feature vector space; and finally after training is performed with a formed lexical item-document matrix being a supervised classifier support vector machine and an input factor of logic regression in combination with marked positive and negative emotion category tags, a final text linear classifier prediction model is obtained, that is, emotion classification can be performed on a new unknown text. Through the method, the characteristic that emotional words in a marked corpus have innate classification capability is effectively utilized, a new calculation method is proposed to maximize category discrimination of the emotional words, and therefore the precision of text emotion classification through a computer is improved.

The present invention claims to protect a feature-aligned Chinese word segmentation method, which includes: 101 extracting features of bigrams from marked data and unmarked data; 102 combining marked data with unmarked data by the Earth Mover's Distance (hereinafter referred to as EMD) method 103 Use the classifier xgboost to train the features of the labeled data after feature alignment, so as to predict the probability of big words in unlabeled data becoming words; 104 Extract some big words and steps from the results of the classifier 101 The bigram integration of labeled data is used as the feature of the conditional random field and trained; 105 The unlabeled data is sequence-labeled and segmented through the established model. The invention mainly performs feature alignment on marked data and unmarked data through EMD, predicts the word formation probability of binary words through classifier learning, and then integrates conditional random fields in a stacking manner to form a new tokenizer.

A method for user authentication based on keystroke dynamics is provided. The user authentication method includes receiving a keystroke input implemented by a user; separating a sequence of pressed keys into a sequence of bigrams having bigram names simultaneously with the user typing free text; collecting a timing information for each bigram of the sequence of bigrams; extracting a feature vector for each bigram based on the timing information; separating feature vectors into subsets according to the bigram names; estimating a GMM user model using subsets of feature vectors for each bigram; providing real time user authentication using the estimated GMM user model for each bigram and bigram features from current real time user keystroke input. The corresponding system is also provided. The GMM based analysis of the keystroke data separated by bigrams provides strong authentication using free text input, while user additional actions (to be verified) are kept at a minimum. The present invention allows to drastically improve accuracy of user authentication with low performance requirements that allows to implement authentication software for low-power mobile devices.