43 results about "Word-sense disambiguation" patented technology

Systems and methods for word sense disambiguation, including discerning one or more senses or occurrences, distinguishing between senses or occurrences, and determining a meaning for a sense or occurrence of a subject term. In a collection of documents containing terms and a reference collection containing at least one meaning associated with a term, the method includes forming a vector space representation of terms and documents. In some embodiments, the vector space is a latent semantic index vector space. In some embodiments, occurrences are clustered to discern or distinguish a sense of a term. In preferred embodiments, meaning of a sense or occurrence is assigned based on either correlation with an external reference source, or proximity to a reference source that has been indexed into the space.

The present invention provides a method of processing at least one natural language text using a graph. The method includes determining a plurality of text units based upon the natural language text, associating the plurality of text units with a plurality of graph nodes, and determining at least one connecting relation between at least two of the plurality of text units. The method also includes associating the at least one connecting relation with at least one graph edge connecting at least two of the plurality of graph nodes and determining a plurality of rankings associated with the plurality of graph nodes based upon the at least one graph edge. The method can also include a graphical visualization of at least one important text unit in a natural language text or collection of texts. Methods for word sense disambiguation, keyword extraction, and sentence extraction are also provided.

The present invention provides a method of processing at least one natural language text using a graph. The method includes determining a plurality of text units based upon the natural language text, associating the plurality of text units with a plurality of graph nodes, and determining at least one connecting relation between at least two of the plurality of text units. The method also includes associating the at least one connecting relation with at least one graph edge connecting at least two of the plurality of graph nodes and determining a plurality of rankings associated with the plurality of graph nodes based upon the at least one graph edge. The method can also include a graphical visualization of at least one important text unit in a natural language text or collection of texts. Methods for word sense disambiguation, keyword extraction, and sentence extraction are also provided.

A device and related methods for word-sense disambiguation during a text-to-speech conversion are provided. The device, for use with a computer-based system capable of converting text data to synthesized speech, includes an identification module for identifying a homograph contained in the text data. The device also includes an assignment module for assigning a pronunciation to the homograph using a statistical test constructed from a recursive partitioning of training samples, each training sample being a word string containing the homograph. The recursive partitioning is based on determining for each training sample an order and a distance of each word indicator relative to the homograph in the training sample. An absence of one of the word indicators in a training sample is treated as equivalent to the absent word indicator being more than a predefined distance from the homograph.

The invention discloses a word multi-prototype vector representation and word sense disambiguation method based on CRP clustering, which comprises the following steps: the text in the massive text corpus is purified and pretreated to obtain plain text, CRP algorithm is used to cluster the context window representation of target polysemous word in the text corpus set. The target polysemous words inthe text corpus set are marked according to the clustering classification, and the polysemous words are trained on the marked text corpus set to obtain the multi-prototype vector representation of the polysemous words; 2, the target short text is preprocessed to obtain a short text word sequence, a target polysemous word in a word sequence is identifued, the contextual window of the target polysemous words is used to represent the similarity between the centroids of clusters corresponding to the words in the text corpus, and the word vector corresponding to the maximum similarity clusters isused as the word vector representation of the specific meaning of the polysemous words in the context to disambiguate the meanings of the polysemous words. The invention solves the problem of polysemyexpression in word expression and the problem of ambiguity identification in word meaning expression.

The invention relates to a word sense disambiguation method and device based on a word vector. The method comprises the following steps that: data preprocessing: carrying out processing, including punctuation removal, word segmentation and the like, on a document and a semantic item; training the word vector: using a word vector training tool to train the word vector; carrying out context vector representation: obtaining the word vector, and adopting a local weighting method to calculate the context vector; carrying out semantic item vector representation: obtaining the word vector of each word of the semantic item, and carrying out calculation to obtain a semantic item vector; carrying out similarity calculation: calculating a cosine similarity between the context vector and each semanticitem vector; carrying out semantic item distribution frequency calculation: carrying out statistics on the distribution frequency of each semantic item of an ambiguous term in a dataset; and carryingout final score statistics: calculating the cosine similarity between the context and each semantic item and the comprehensive score of each piece of semantic item frequency, wherein the semantic item with a highest score is an optimal word meaning.

The invention discloses a multi-document subject discovery method based on two-layer clustering. The multi-document subject discovery method comprises the following steps: S1 using a plurality of documents as input, pretreating each document, i.e. the documents are broken up into clauses, and the clauses are broken up into words, so as to obtain a noun group and a verb group in a multi-document group, and performing emantic disambiguation processing on polysemes in the noun group and the verb group; S2 respectively performing word clustering analysis on the noun group and the verb group which are output in the step S1 according to word similarity by adopting an improved OPTICS algorithm, extracting semantic concepts, and establishing vector space models on the clauses according to the semantic concepts; S3 performing clustering analysis on the clauses by using an improved K-medoid algorithm, so as to obtain a subject. Inner semantic relations between words are extracted by the multi-document subject discovery method, and the problem of non-orthogonality among feature items when feature vectors of the clauses are established is solved.

The embodiment of the invention provides a training method and device for a word sense disambiguation model, and the method comprises the steps: obtaining a word co-occurrence graph and a semantic association graph; and selecting a first word from the training text; obtaining a positive example sample and a negative example sample corresponding to the first word; calculating the similarity betweeneach word in the training text and the word represented by each node in each semantic association graph, and selecting a target association graph based on the similarity;based on the target association graph, determining a semantic vector of the first word, and based on the word co-occurrence graph, determining word vectors of other words; based on the determined semantic vector and word vector,encoding by utilizing an encoder; based on the word co-occurrence graph, determining a word vector of each word in the two samples; carrying out encoding by using an encoder according to the determined word vecto; based on the coding result, calculating a first text distance between the training text and the positive example sample, and calculating a second text distance between the training textand the negative example sample; and training an encoder by taking the condition that the first text distance is smaller than the second text distance as a target.

The invention discloses a word sense disambiguation method and system based on a graph model, and belongs to the field of natural language processing technology. The technical problem to be solved bythe present invention is how to combine multiple Chinese and English resources, complement each other's advantages, realize full exploitation of disambiguation knowledge in resources, and improve wordsense disambiguation performance.The technical scheme adopted is as follows: 1, a word sense disambiguation method based on graph model, comprising the following steps: S1, extracting contextual knowledge: carrying out part-of-speech tagging on ambiguous sentences, extracting substantive words as contextual knowledge, wherein the substantive words refer to nouns, verbs, adjectives and adverbs; S2, similarity calculation: performing similarity calculation based on English, similarity calculation based on word vector and similarity calculation based on HowNet; 3, constructing a disambiguation graph; S4, performing the correct choice of word meaning. 2, A word sense disambiguation system based on graph model, which comprises a context knowledge extraction unit, a similarity calculation unit,a disambiguation graph construction unit and a word sense correct selection unit.

The invention discloses an ambiguity elimination method and apparatus for e-commerce product comment vocabularies. The method comprises the steps of obtaining a historical comment text, and extracting historical feature words and historical sentiment words matched with the historical feature words from the historical comment text; according to a co-occurrence relationship between the historical feature words and the historical sentiment words corresponding to the historical feature words, screening out a combination of the historical feature words and the historical sentiment words, which occur most frequently; according to the combination of the historical feature words and the historical sentiment words, which occur most frequently, generating meaning item annotations of the historical feature words; obtaining a new comment text, and extracting a combination of new feature words and corresponding new sentiment words from the new comment text; and according to the combination of the new feature words and the new sentiment words, querying the combination of the historical feature words and the historical sentiment words, matched with the combination of the new feature words and the new sentiment words, and taking the meaning item annotations of the matched historical feature words as word meanings of the new feature words. According to the technical scheme, word meaning analysis and meaning item determination of the product comment vocabularies are realized, so that accurate paraphrases of product feature words occurring in contexts are accurately judged in different comment contexts.

The invention relates to a word sense disambiguation method and equipment based on superordinate words, and a word sense expansion method and equipment using the word sense disambiguation method. Theword sense disambiguation method comprises the following steps: receiving an input statement; Determining a disambiguation target word in the input statement based on a predetermined disambiguation word bank; Determining related words of the target word based on syntactic analysis and context information analysis of the input statement; Determining one or more hypernyms of the related words; And determining the word meaning of the target word in the input statement based on the related word, the word forms and the part-of-speech of the one or more hypernyms and the syntactic relationship withthe target word.

The invention discloses a word sense disambiguation method and device based on combination of a graph model and a word vector, belongs to the field of natural language processing, and is used for solving the problem of low accuracy of an existing word sense disambiguation method. The method is characterized by comprising the steps of preprocessing a data set to obtain ambiguous words; constructinga graph model, and obtaining context background knowledge according to the graph model; training a word vector model, and performing word vector representation on the obtained ambiguous words and context background knowledge according to the word vector model; and performing cross-weighted similarity calculation on the ambiguous words expressed by the word vectors and the context background knowledge, taking a mean value, and determining the ambiguous word with the highest similarity mean value as a correct meaning item of the ambiguous words. According to the invention, the graph model and the word vector are combined, so that the correct rate of word sense disambiguation is improved, and a good disambiguation effect is achieved. The method is superior to a traditional word sense disambiguation method and can well meet the requirements of practical application.

The invention relates to a word meaning disambiguating system for disambiguating polysemous words. The word meaning disambiguating system comprises an input device and a word meaning disambiguating device, wherein the input device is used for inputting texts including the polysemous words; and the word meaning disambiguating device is used for iteratively determining the meaning of each word on the basis of the meaning obvious degree of the word, wherein the meaning obvious degree is obtained according to the meaning reliability of the word. Besides, the invention also relates to a word meaning disambiguating method. The word meaning disambiguating system and method can improve the consistency of word meaning disambiguating results and shorten calculating time.

The invention relates to a word sense disambiguation method which comprises the following steps of acquiring a to-be-processed text, determining a to-be-disambiguated word in the to-be-processed text, a preceding text of the to-be-disambiguated word and a following text of the to-be-disambiguated word, searching a plurality of definition interpretations corresponding to the to-be-disambiguated word from a preset semantic knowledge base, vectorizing the plurality of definition interpretations, obtaining a definition matrix, constructing a corresponding classifier based on the definition matrix, inputting the preceding text, the following text and the to-be-disambiguated word into the classifier, obtaining the predicted candidate words of the to-be-disambiguated word, replacing the to-be-disambiguated word with the predicted candidate words, and obtaining a disambiguated standard text. In addition, the invention also relates to a word sense disambiguation method and device, an apparatus and a storage medium. According to the present invention, the problem that the accuracy of word sense disambiguation is not high enough can be solved.

The invention particularly relates to a new bayesian word sense disambiguation method based on mass pseudo-data. The problems that a current word sense disambiguation method is poor in disambiguation effect and capable of wasting time and labor when disambiguation knowledge is obtained are solved. The new bayesian word sense disambiguation method includes the steps that through a dependency grammar analyzer, training examples containing ambiguous words in a training corpus base are subjected to syntactic analysis, and tuples with the dependence relationship with the ambiguous words are collected; then through a machine translation system, example sentences containing the tuples in a machine translation corpus base are searched. The steps are repeatedly carried out in a mode, the searched example sentences are added into a pseudo-training corpus base, and then through the training corpus base and the pseudo-training corpus base, a bayesian disambiguation model is trained; word meanings of the ambiguous words are decided through the disambiguation model, and on the basis of a small amount of manually-annotated corpuses, the data sparsity problem of word sense disambiguation can be effectively solved, the accuracy of word sense disambiguation is increased, and the new bayesian word sense disambiguation method has broad development prospects.

The invention discloses a method and a device for noun word sense disambiguation based on dependency constraint and knowledge. The method comprises: performing dependency syntactic analysis on large-scale corpus, collecting obtained dependency tuples and counting frequency number of the dependency tuples, establishing a dependency knowledge base; performing dependency syntactic analysis on a sentence where an ambiguity noun is in, extracting 16 kinds of dependency tuples which meet a set condition, and using the dependency tuples as a dependency constraint set of the ambiguity noun; according to a semantic dictionary, for each meaning of the ambiguity noun, extracting a synonym set, an antonym set, and a hypernym set in sequence, and using the synonym set, the antonym set, and the hypernym set as word meaning representative word sets of the corresponding word meanings; according to the dependency knowledge base and the word meaning representative word sets, calculating posterior probability of each meaning of the ambiguity noun in the dependency constraint set; and according to the posterior probability, selecting the correct meaning of the ambiguity noun. Using the method can fully develop effect of the dependency syntactic analysis, so as to accurately and effectively determine the meaning of the ambiguity noun.

The invention relates to a Chinese word sense disambiguation method based on a Recurrent Neural Network (RNN). The method comprises the following steps: firstly, preprocessing corpora; in the step, performing word segmentation and feature labeling processing on sentences containing ambiguous vocabularies in the training expectation and the test expectation; the processed training expectation beingused for training the RNN model, and the purpose being to optimize the model; and disambiguating the test expectation by using the optimized RNN model to obtain a probability distribution vector of the ambiguous vocabulary under each semantic category. In the vector, the semantic class corresponding to the maximum value is judged as the semantic class of the ambiguous vocabulary. The method has agood word sense disambiguation effect, and the meaning of ambiguous vocabularies can be better judged.

The invention relates to a subject term embedding disambiguation method and system based on LDA, and belongs to the technical field of semantic analysis. The method comprises the following steps: a topic model training step: training a topic model based on a Wiki corpus in combination with an LDA algorithm; a subject term vector generation step: training a subject term vector by utilizing Word2Vecaccording to the Wiki corpus and the subject model; a context vector generation step: calculating the vector representation of the context where the ambiguous words are located by using the topic model and the topic word vectors; and a supervised word sense disambiguation step: combining the context vector with other traditional semantic features, and performing word sense disambiguation by utilizing an SVM.

The invention relates to the technical field of natural language processing, and discloses a natural language information processing method. The method comprises the following steps: storing natural language information by utilizing a Huffman tree, and converting the natural language information into a natural language vector; carrying out word segmentation processing on the natural language vector by utilizing a pre-trained LC-CRF model; extracting vocabulary unit vectors of ambiguous words, and forming an ambiguous word feature matrix; inputting the ambiguous word feature matrix into a pre-constructed word sense disambiguation model, and identifying semantics of ambiguous words by utilizing the word sense disambiguation model; iteratively updating the weight of each word based on the weighted node initial value of the word comprehensive weight and the node probability transfer matrix, and selecting the first N words with relatively large weights as keywords; and taking a keyword vector in the natural language information as an input vector, and identifying emotion in the natural language information by utilizing the neural network based on the segmented pooling layer. The naturallanguage information is processed.

In order to determine the accurate Chinese word sense of words in English sentences, the invention provides an English translation Chinese word sense disambiguation method based on neural network, which comprises the following steps of: firstly, generating a Chinese and English mixed sequence corresponding to each English sentence based on an English-Chinese dictionary and Chinese and English sentences of Chinese and English corpora; secondly, taking an English sentence and Chinese and English mixed sequence as parallel corpora, and performing training through a neural network method to obtain a translation model; and finally, sequentially performing restrictive decoding in the Chinese meaning of each English word of the to-be-translated English sentence by using the translation model to find the accurate meaning of the word in the English sentence, thereby effectively solving the problem of word meaning ambiguity elimination in English translation.

The invention relates to a Chinese word sense disambiguation method based on a graph convolutional neural network (GCN). According to the invention, firstly, Chinese corpora are preprocessed; word segmentation, part-of-speech tagging and semantic tagging processing are performed on statements, containing ambiguous words, of the training and testing corpora; a word sense disambiguation feature graph is constructed by taking sentences where ambiguous words are located and word forms, part-of-speech and semantics contained in the sentences as disambiguation features and nodes, and weights are embedded into the nodes and edges by using Word2Vec and Doc2Vec tools and point mutual information (PMI) and TF-IDF methods; and the GCN model is trained by using the training corpus, and thus optimizing the model; word sense disambiguation is performed on the test corpus by using the optimized GCN model, so that probability distribution of ambiguous vocabularies under each semantic category can be obtained; and the semantic class corresponding to the maximum probability value is judged as the semantic class of the ambiguous vocabulary. The invention has a good word sense disambiguation effect, and the real meaning of the ambiguous vocabulary is more accurately judged.

The invention relates to the technical field of natural language processing, and discloses a word sense disambiguation method and device based on context semantics, computer equipment and a storage medium. The method comprises the following steps: acquiring text data, and preprocessing the text data to obtain to-be-processed data; inputting the to-be-processed data into a pre-trained language model for training to obtain previous context information, ambiguous word information and next context information; according to the previous context information, the ambiguous word information and the next context information, obtaining text segment semantic information; inputting the semantic information of the text segment into the pre-trained classifier for training, determining the definition of the ambiguous word information in the current context, so that the accuracy of word sense disambiguation is improved.

The invention relates to a Chinese word sense disambiguation method based on a graph convolutional neural network (GCN) fused with a support vector machine (SVM), in particular to a Chinese word sense disambiguation method based on the graph convolutional neural network (GCN) fused with the support vector machine (SVM) and a Chinese word sense disambiguation method based on the graph convolutional neural network (GCN) fused with the Chinese word sense disambiguation method based on the graph convolutional neural network (GCN) fused with the support vector machine (SVM). The method comprises the steps of firstly preprocessing corpora; and performing word segmentation, part-of-speech tagging and semantic tagging processing on statements of the training and testing corpora. A word sense disambiguation graph is constructed by taking sentences where ambiguous words are located and word forms, part-of-speech and semantic classes of vocabulary units on two sides of the ambiguous words as disambiguation features and taking the disambiguation features as nodes. Weights of nodes and edges in the graph are calculated by using Word2Vec, a Doc2Vec tool, point-by-point mutual information (PMI) and a TF-IDF algorithm. And training the GCN model by the training corpus, and optimizing the model. And calculating disambiguation features of training and testing corpora by using the optimized GCN model, inputting the calculated disambiguation features of the training corpora into an SVM classifier, optimizing the SVM classifier, and classifying the testing corpora to obtain classification conditions of ambiguous vocabularies under semantic categories. The method has a good word sense disambiguation effect, and the real meaning of the ambiguous vocabulary is accurately judged.

The invention discloses a mobile device machine translation system based on a hybrid strategy. The system comprises a translation information acquisition module, a translation keyword grouping module,a machine translation engine, a neural machine translation engine, an interactive machine translation module, a word sense disambiguation module and a self-updating learning module. According to theinvention, the machine translation system is integrated into the mobile device, so that the mobile device can obtain translation information more quickly and conveniently; meanwhile, result translations translated by a machine translation system based on rules, statistics and a neural network are fused through an interactive machine translation module; the translation result of the source statement is more accurate; the machine translation system based on rules, statistics and the neural network is arranged in the mobile device, and translation does not need to be carried out by means of the cloud in the actual translation operation process; meanwhile, network connection is not needed in the whole translation process in the mobile device, and the defect that a traditional translation system needs to be networked to carry out translation by means of cloud resources is overcome.

The invention discloses a Chinese commercial text preprocessing method based on machine learning. The input Chinese commercial text is processed through the following steps that (1) performing sentence segmentation and word segmentation on the Chinese commercial text; (2) carrying out part-of-speech tagging on the segmented words by utilizing a decision tree; (3) performing word sense disambiguation by utilizing conditional probability based on a Bayesian classifier; (4) representing a word vector by using a hybrid model combining One-Hot coding and a Skip-Gram model; (5) adjusting word weights by utilizing TF-IDF, and determining corresponding word meanings of the polysemy under the current context; and (6) outputting the Chinese commercial text preprocessed based on machine learning. TheChinese commercial text preprocessing method can effectively solve the problems that a Chinese commercial question-answering system does not answer questions and is limited in response scene due to insufficient text preprocessing, can improve the text understanding accuracy of a computer, and enables extension work such as machine translation and intelligent question-answering to have implementability.

The present invention relates to a method and device for word sense disambiguation based on word vectors. The method of the invention includes a data preprocessing step of performing punctuation and word segmentation on documents and meaning items; a step of training word vectors, using a word vector training tool to train words Vector; the context vector representation step is to obtain the word vector and use the local weighting method to calculate the context vector; the semantic item vector representation step is to obtain the word vector of each word of the semantic item, and calculate the semantic item vector; the similarity calculation step is to calculate the relationship between the context vector and each semantic item vector The cosine similarity between the meaning items; the distribution frequency calculation step of meaning items, the distribution frequency of each meaning item of ambiguous words in the statistical data set; the final score statistics step, the calculation of the cosine similarity between the context and each meaning item and the comprehensive score of each meaning item frequency, the meaning item with the highest score for the best vocabulary.

The invention discloses a dependency constraint and knowledge-based adverb meaning disambiguation method and apparatus. The method comprises the steps of performing dependency syntax analysis on large-scale corpora, collecting obtained dependency tuples and performing statistics on frequency numbers of the dependency tuples, and establishing a dependency knowledge library; performing dependency syntax analysis on a sentence in which ambiguous adverbs are located, and extracting the two dependency tuples meeting a set condition as a dependency constraint set of the ambiguous adverbs; sequentially extracting a synonym set and an antonym set as a word meaning representative word set of corresponding word meanings for the word meanings of the ambiguous adverbs according to a semantic dictionary; sequentially calculating posterior probabilities of the word meanings of the ambiguous adverbs in the dependency constraint set according to the dependency knowledge library and the word meaning representative word set; and selecting correct word meanings of the ambiguous adverbs according to the posterior probabilities. By utilizing the method and the apparatus, the effect of the dependency syntax analysis can be brought into full play and the word meanings of the ambiguous adverbs can be determined more accurately and effectively.

The present application relates to a word sense disambiguation method, the method comprising: obtaining the text to be processed, determining the word to be disambiguated in the text to be processed, and the upper text of the word to be disambiguated and the following text of the word to be disambiguated, from Search for multiple definition explanations corresponding to the word to be disambiguated in the preset semantic knowledge base, vectorize the multiple definition explanations, obtain the definition matrix, build a corresponding classifier based on the definition matrix, and classify the above, below and to be disambiguated The words are input to the classifier to obtain the predicted candidate words of the word to be disambiguated, and the predicted candidate words are used to replace the word to be disambiguated to obtain the standard text after disambiguation. In addition, the present application also relates to a word sense disambiguation device, equipment and storage medium. The application can solve the problem that the accuracy of word sense disambiguation is not high enough.

The invention discloses a concept-based semantic identification method and device. The method comprises the following steps: step S1, carrying out word segmentation on a text to be segmented, and obtaining a plurality of segmented strings; step S2, matching the segmented character string with all nodes of the semantic web; step S3, performing word sense disambiguation on the string successfully matched with the node of the semantic web to obtain a concept path of the disambiguated string, and storing the concept path in a word concept path storage library; step S4, outputting the concept pathof the related words in the concept path storage library of the text to be segmented. The invention can effectively improve the accuracy rate and calculation speed of the text semantic concept recognition.