42 results about "Word-sense disambiguation" patented technology

The present invention provides a method of processing at least one natural language text using a graph. The method includes determining a plurality of text units based upon the natural language text, associating the plurality of text units with a plurality of graph nodes, and determining at least one connecting relation between at least two of the plurality of text units. The method also includes associating the at least one connecting relation with at least one graph edge connecting at least two of the plurality of graph nodes and determining a plurality of rankings associated with the plurality of graph nodes based upon the at least one graph edge. The method can also include a graphical visualization of at least one important text unit in a natural language text or collection of texts. Methods for word sense disambiguation, keyword extraction, and sentence extraction are also provided.

The invention discloses a word multi-prototype vector representation and word sense disambiguation method based on CRP clustering, which comprises the following steps: the text in the massive text corpus is purified and pretreated to obtain plain text, CRP algorithm is used to cluster the context window representation of target polysemous word in the text corpus set. The target polysemous words inthe text corpus set are marked according to the clustering classification, and the polysemous words are trained on the marked text corpus set to obtain the multi-prototype vector representation of the polysemous words; 2, the target short text is preprocessed to obtain a short text word sequence, a target polysemous word in a word sequence is identifued, the contextual window of the target polysemous words is used to represent the similarity between the centroids of clusters corresponding to the words in the text corpus, and the word vector corresponding to the maximum similarity clusters isused as the word vector representation of the specific meaning of the polysemous words in the context to disambiguate the meanings of the polysemous words. The invention solves the problem of polysemyexpression in word expression and the problem of ambiguity identification in word meaning expression.

The invention relates to a word sense disambiguation method and device based on a word vector. The method comprises the following steps that: data preprocessing: carrying out processing, including punctuation removal, word segmentation and the like, on a document and a semantic item; training the word vector: using a word vector training tool to train the word vector; carrying out context vector representation: obtaining the word vector, and adopting a local weighting method to calculate the context vector; carrying out semantic item vector representation: obtaining the word vector of each word of the semantic item, and carrying out calculation to obtain a semantic item vector; carrying out similarity calculation: calculating a cosine similarity between the context vector and each semanticitem vector; carrying out semantic item distribution frequency calculation: carrying out statistics on the distribution frequency of each semantic item of an ambiguous term in a dataset; and carryingout final score statistics: calculating the cosine similarity between the context and each semantic item and the comprehensive score of each piece of semantic item frequency, wherein the semantic item with a highest score is an optimal word meaning.

The invention discloses a multi-document subject discovery method based on two-layer clustering. The multi-document subject discovery method comprises the following steps: S1 using a plurality of documents as input, pretreating each document, i.e. the documents are broken up into clauses, and the clauses are broken up into words, so as to obtain a noun group and a verb group in a multi-document group, and performing emantic disambiguation processing on polysemes in the noun group and the verb group; S2 respectively performing word clustering analysis on the noun group and the verb group which are output in the step S1 according to word similarity by adopting an improved OPTICS algorithm, extracting semantic concepts, and establishing vector space models on the clauses according to the semantic concepts; S3 performing clustering analysis on the clauses by using an improved K-medoid algorithm, so as to obtain a subject. Inner semantic relations between words are extracted by the multi-document subject discovery method, and the problem of non-orthogonality among feature items when feature vectors of the clauses are established is solved.

The invention discloses a word sense disambiguation method and system based on a graph model, and belongs to the field of natural language processing technology. The technical problem to be solved bythe present invention is how to combine multiple Chinese and English resources, complement each other's advantages, realize full exploitation of disambiguation knowledge in resources, and improve wordsense disambiguation performance.The technical scheme adopted is as follows: 1, a word sense disambiguation method based on graph model, comprising the following steps: S1, extracting contextual knowledge: carrying out part-of-speech tagging on ambiguous sentences, extracting substantive words as contextual knowledge, wherein the substantive words refer to nouns, verbs, adjectives and adverbs; S2, similarity calculation: performing similarity calculation based on English, similarity calculation based on word vector and similarity calculation based on HowNet; 3, constructing a disambiguation graph; S4, performing the correct choice of word meaning. 2, A word sense disambiguation system based on graph model, which comprises a context knowledge extraction unit, a similarity calculation unit,a disambiguation graph construction unit and a word sense correct selection unit.

The invention discloses an ambiguity elimination method and apparatus for e-commerce product comment vocabularies. The method comprises the steps of obtaining a historical comment text, and extracting historical feature words and historical sentiment words matched with the historical feature words from the historical comment text; according to a co-occurrence relationship between the historical feature words and the historical sentiment words corresponding to the historical feature words, screening out a combination of the historical feature words and the historical sentiment words, which occur most frequently; according to the combination of the historical feature words and the historical sentiment words, which occur most frequently, generating meaning item annotations of the historical feature words; obtaining a new comment text, and extracting a combination of new feature words and corresponding new sentiment words from the new comment text; and according to the combination of the new feature words and the new sentiment words, querying the combination of the historical feature words and the historical sentiment words, matched with the combination of the new feature words and the new sentiment words, and taking the meaning item annotations of the matched historical feature words as word meanings of the new feature words. According to the technical scheme, word meaning analysis and meaning item determination of the product comment vocabularies are realized, so that accurate paraphrases of product feature words occurring in contexts are accurately judged in different comment contexts.

The invention discloses a word sense disambiguation method and device based on combination of a graph model and a word vector, belongs to the field of natural language processing, and is used for solving the problem of low accuracy of an existing word sense disambiguation method. The method is characterized by comprising the steps of preprocessing a data set to obtain ambiguous words; constructinga graph model, and obtaining context background knowledge according to the graph model; training a word vector model, and performing word vector representation on the obtained ambiguous words and context background knowledge according to the word vector model; and performing cross-weighted similarity calculation on the ambiguous words expressed by the word vectors and the context background knowledge, taking a mean value, and determining the ambiguous word with the highest similarity mean value as a correct meaning item of the ambiguous words. According to the invention, the graph model and the word vector are combined, so that the correct rate of word sense disambiguation is improved, and a good disambiguation effect is achieved. The method is superior to a traditional word sense disambiguation method and can well meet the requirements of practical application.

The invention relates to a word meaning disambiguating system for disambiguating polysemous words. The word meaning disambiguating system comprises an input device and a word meaning disambiguating device, wherein the input device is used for inputting texts including the polysemous words; and the word meaning disambiguating device is used for iteratively determining the meaning of each word on the basis of the meaning obvious degree of the word, wherein the meaning obvious degree is obtained according to the meaning reliability of the word. Besides, the invention also relates to a word meaning disambiguating method. The word meaning disambiguating system and method can improve the consistency of word meaning disambiguating results and shorten calculating time.

The invention relates to a word sense disambiguation method which comprises the following steps of acquiring a to-be-processed text, determining a to-be-disambiguated word in the to-be-processed text, a preceding text of the to-be-disambiguated word and a following text of the to-be-disambiguated word, searching a plurality of definition interpretations corresponding to the to-be-disambiguated word from a preset semantic knowledge base, vectorizing the plurality of definition interpretations, obtaining a definition matrix, constructing a corresponding classifier based on the definition matrix, inputting the preceding text, the following text and the to-be-disambiguated word into the classifier, obtaining the predicted candidate words of the to-be-disambiguated word, replacing the to-be-disambiguated word with the predicted candidate words, and obtaining a disambiguated standard text. In addition, the invention also relates to a word sense disambiguation method and device, an apparatus and a storage medium. According to the present invention, the problem that the accuracy of word sense disambiguation is not high enough can be solved.

The invention particularly relates to a new bayesian word sense disambiguation method based on mass pseudo-data. The problems that a current word sense disambiguation method is poor in disambiguation effect and capable of wasting time and labor when disambiguation knowledge is obtained are solved. The new bayesian word sense disambiguation method includes the steps that through a dependency grammar analyzer, training examples containing ambiguous words in a training corpus base are subjected to syntactic analysis, and tuples with the dependence relationship with the ambiguous words are collected; then through a machine translation system, example sentences containing the tuples in a machine translation corpus base are searched. The steps are repeatedly carried out in a mode, the searched example sentences are added into a pseudo-training corpus base, and then through the training corpus base and the pseudo-training corpus base, a bayesian disambiguation model is trained; word meanings of the ambiguous words are decided through the disambiguation model, and on the basis of a small amount of manually-annotated corpuses, the data sparsity problem of word sense disambiguation can be effectively solved, the accuracy of word sense disambiguation is increased, and the new bayesian word sense disambiguation method has broad development prospects.

The invention discloses a method and a device for noun word sense disambiguation based on dependency constraint and knowledge. The method comprises: performing dependency syntactic analysis on large-scale corpus, collecting obtained dependency tuples and counting frequency number of the dependency tuples, establishing a dependency knowledge base; performing dependency syntactic analysis on a sentence where an ambiguity noun is in, extracting 16 kinds of dependency tuples which meet a set condition, and using the dependency tuples as a dependency constraint set of the ambiguity noun; according to a semantic dictionary, for each meaning of the ambiguity noun, extracting a synonym set, an antonym set, and a hypernym set in sequence, and using the synonym set, the antonym set, and the hypernym set as word meaning representative word sets of the corresponding word meanings; according to the dependency knowledge base and the word meaning representative word sets, calculating posterior probability of each meaning of the ambiguity noun in the dependency constraint set; and according to the posterior probability, selecting the correct meaning of the ambiguity noun. Using the method can fully develop effect of the dependency syntactic analysis, so as to accurately and effectively determine the meaning of the ambiguity noun.

The invention relates to a subject term embedding disambiguation method and system based on LDA, and belongs to the technical field of semantic analysis. The method comprises the following steps: a topic model training step: training a topic model based on a Wiki corpus in combination with an LDA algorithm; a subject term vector generation step: training a subject term vector by utilizing Word2Vecaccording to the Wiki corpus and the subject model; a context vector generation step: calculating the vector representation of the context where the ambiguous words are located by using the topic model and the topic word vectors; and a supervised word sense disambiguation step: combining the context vector with other traditional semantic features, and performing word sense disambiguation by utilizing an SVM.

The invention relates to the technical field of natural language processing, and discloses a natural language information processing method. The method comprises the following steps: storing natural language information by utilizing a Huffman tree, and converting the natural language information into a natural language vector; carrying out word segmentation processing on the natural language vector by utilizing a pre-trained LC-CRF model; extracting vocabulary unit vectors of ambiguous words, and forming an ambiguous word feature matrix; inputting the ambiguous word feature matrix into a pre-constructed word sense disambiguation model, and identifying semantics of ambiguous words by utilizing the word sense disambiguation model; iteratively updating the weight of each word based on the weighted node initial value of the word comprehensive weight and the node probability transfer matrix, and selecting the first N words with relatively large weights as keywords; and taking a keyword vector in the natural language information as an input vector, and identifying emotion in the natural language information by utilizing the neural network based on the segmented pooling layer. The naturallanguage information is processed.

In order to determine the accurate Chinese word sense of words in English sentences, the invention provides an English translation Chinese word sense disambiguation method based on neural network, which comprises the following steps of: firstly, generating a Chinese and English mixed sequence corresponding to each English sentence based on an English-Chinese dictionary and Chinese and English sentences of Chinese and English corpora; secondly, taking an English sentence and Chinese and English mixed sequence as parallel corpora, and performing training through a neural network method to obtain a translation model; and finally, sequentially performing restrictive decoding in the Chinese meaning of each English word of the to-be-translated English sentence by using the translation model to find the accurate meaning of the word in the English sentence, thereby effectively solving the problem of word meaning ambiguity elimination in English translation.

The invention relates to the technical field of natural language processing, and discloses a word sense disambiguation method and device based on context semantics, computer equipment and a storage medium. The method comprises the following steps: acquiring text data, and preprocessing the text data to obtain to-be-processed data; inputting the to-be-processed data into a pre-trained language model for training to obtain previous context information, ambiguous word information and next context information; according to the previous context information, the ambiguous word information and the next context information, obtaining text segment semantic information; inputting the semantic information of the text segment into the pre-trained classifier for training, determining the definition of the ambiguous word information in the current context, so that the accuracy of word sense disambiguation is improved.

The invention discloses a mobile device machine translation system based on a hybrid strategy. The system comprises a translation information acquisition module, a translation keyword grouping module,a machine translation engine, a neural machine translation engine, an interactive machine translation module, a word sense disambiguation module and a self-updating learning module. According to theinvention, the machine translation system is integrated into the mobile device, so that the mobile device can obtain translation information more quickly and conveniently; meanwhile, result translations translated by a machine translation system based on rules, statistics and a neural network are fused through an interactive machine translation module; the translation result of the source statement is more accurate; the machine translation system based on rules, statistics and the neural network is arranged in the mobile device, and translation does not need to be carried out by means of the cloud in the actual translation operation process; meanwhile, network connection is not needed in the whole translation process in the mobile device, and the defect that a traditional translation system needs to be networked to carry out translation by means of cloud resources is overcome.

The invention discloses a Chinese commercial text preprocessing method based on machine learning. The input Chinese commercial text is processed through the following steps that (1) performing sentence segmentation and word segmentation on the Chinese commercial text; (2) carrying out part-of-speech tagging on the segmented words by utilizing a decision tree; (3) performing word sense disambiguation by utilizing conditional probability based on a Bayesian classifier; (4) representing a word vector by using a hybrid model combining One-Hot coding and a Skip-Gram model; (5) adjusting word weights by utilizing TF-IDF, and determining corresponding word meanings of the polysemy under the current context; and (6) outputting the Chinese commercial text preprocessed based on machine learning. TheChinese commercial text preprocessing method can effectively solve the problems that a Chinese commercial question-answering system does not answer questions and is limited in response scene due to insufficient text preprocessing, can improve the text understanding accuracy of a computer, and enables extension work such as machine translation and intelligent question-answering to have implementability.