A patent text modeling method based on word 2vec and semantic similarity

Abstract

The invention relates to the field of text modeling, and provides a patent text modeling method based on word2vec and semantic similarity. The method includes: crawling the patent text set and performing preprocessing; calculating the TF-IDF value of each word in the patent text set sorting and selecting to obtain feature word set; The text set is imported into word2vec model and the word vector is obtained by training. Cosine similarity is calculated to get the similar word set wordC_1; Word2vec similarity was calculated to get the similar word set textC_1. The similarity of word set textC_1and textC_1 was calculated to get the similar word set textC_1. The text set is imported into the text processing system for training, and the semantic similarity is obtained. The similar word set wordC_2 is selected. The semantic similarity is calculated to get the similar word set textC_2. The semantic similarity is calculated to get the similar word set textC_2. The extended word set textC_f isobtained by calculating the mixed similarity. Weights are calculated to form new text identifiers to complete the modeling. The invention adds part of the information between words to the traditionalvector space model from the statistical angle of the word2vec and the semantic angle of the semantic similarity, reduces the sparsity of the text matrix to a certain extent, and the clustering effectis more remarkable and stable, and has stronger text identification ability.

Description

technical field [0001] The present invention relates to the field of text modeling, more specifically, relate to a kind of patent text modeling method of word2vec and semantic similarity. Background technique [0002] In terms of text modeling of patent texts, scholars have tried many different methods to improve traditional text modeling methods, such as expressing patent texts as text vectors with patent semantic weight information and word frequency weight information, and proposing a method based on conditional randomness. Airport (CRF) patented term extraction scheme, proposed Latent Semantic Index (LSI) model realized multilingual vector space, etc. In addition to improving the vector space model of the traditional text modeling method, many scholars have also constructed a text modeling method different from the vector space to improve the text expression ability of patent texts. [0003] However, the problems of sparse feature dimensions and lack of semantic informa...

AI Technical Summary

Problems solved by technology

[0003] However, the problems of sparse feature dimensions and lack of semantic information in the traditional vector space model have not been well resolved. The existing technology for patent text analysis methods still lacks consideration of the full text of the patent text, and cannot deeply mine patents in the same field. The Problem of the Internal Law of the Text

Images