76 results about "Topic mining" patented technology

The disclosed embodiments provide a method, system and apparatus for processing data. During operation, the system obtains a set of content items containing unstructured data. Next, the system obtains a set of part-of-speech (POS) tags for lexical items in the set of content items. The system then uses a computer to match the POS tags to one or more POS tagging patterns to obtain a set of candidate topics for the set of content items and extract a set of topics for the set of content items from the set of candidate topics.

A short text topic model mining method based on a word network to extend characteristics comprises a weighted word network construction step, a short text characteristics extending step, and a topic mining step. The weighted word network construction step comprises preprocessing a text, performing Chinese words segmentation on the text in a short text corpus, and deleting stop words; establishing a weighted word network from a document after the Chinese words segmentation is performed, wherein nodes in the weighted word network are words, each edge between the nodes is cooccurrence relation of two words in the same document, and the weight of the edge is the cooccurrence time of the two words in the whole corpus; and ending. The short text characteristics extending step comprises using the word nodes included by each short text after the Chinese words segmentation is performed as a community of the established weighted word network. According to the short text characteristics sparsity solution method based on word network community module degree, the problem that the effect of applying an LDA topic model to the short text is poor is solved. Accuracy of a short text topic model is increased.

The invention provides a subtopic mining method. The method comprises the steps that (1) a subject value of each term of each document in a corpus is initialized; (2) based on the current subject values of all the terms of all the documents, the probability of each term in each article coming from all subtopics and the probability of each term coming from a background module are calculated, and then a subject value is redistributed for each term in each article through a Gibbs sampling algorithm based on the calculated probabilities, wherein the probability of each term coming from the background module is calculated according to term distribution vectors, subjected to statics in advance, in the background module, and the term distribution vectors in the background module are constant from beginning to end in the iteration process; and (3) if iteration stop conditions are met, LDA subtopics are obtained according to current subject value information, and if not, the step (2) is returned to. Through the method, the topic mining effect targeting a feature article set can be remarkably improved.

The invention discloses a low-rank decomposition based delicate topic mining method. The delicate topic mining method comprises the following steps: conducting word dividing and stopword removal processing on an original corpus text; generating a topic matrix on the basis of a word frequency matrix obtained through pre-processing; decomposing the original corpus text into topic background and keywords by the topic matrix. According to the delicate topic mining method, a delicate model for expressing text contents without introducing a new implicit variable is brought forward; the model adopts an LDA (Latent Dirichlet Allocation) model as the basis to extract topic distribution of a text collection, and introduces in an improvement method of principal component analysis, namely the robustness principal component analysis method, in combination with the characteristics of text topics constituted by different aspects, in order to decompose each topic into a low-rank part and a rarefaction part; the low-rank part represents common words under the topic, and the rarefaction part is the delicate descriptions in different angles under the topic, so that the purpose of delicately expressing a text is realized, and the problems that the conventional topic model can only mine the topic background of the text, and cannot delicately describe emphasis points of the text are effectively solved.

The invention belongs to sentiment analysis and theme mining tasks in the field of natural language processing, and particularly relates to a theme mining sentiment analysis method based on user characteristic optimization. The method comprises: S1, establishing a multi-dimensional theme and emotion joint model MTSM based on an LDA theme model, and text information, time, user characteristics andemotion tags are fused in the model; S2, utilizing the training corpus to train a model, and solving model parameters; and S3, utilizing the trained model to carry out topic mining and emotion prediction on the test corpus. The invention aims at the characteristics of a network social text. The method has the advantages that text information, time, user characteristics, emotion tags and other four-dimensional information are effectively integrated, a network social text generation mode is redefined, a multi-dimensional topic emotion score combination type is established, topic information is observed and compared from multiple perspectives, and the emotion prediction accuracy of the network social text is improved.

The invention discloses a cross-domain knowledge discovery-oriented topic mining method, comprising: constructing a source domain text set and a target domain set; extracting potential class feature information and potential semantic information from the source domain text set; extracting potential feature information and potential semantic information from the target domain set; automatically aggregating a text in the target domain set into a potential style component; modeling the semantic information of the target domain set in a potential topic component; and modeling the potential topic component of the semantic information of the target domain set. The method has the following advantages: features of texts in a source domain are automatically mined for identifying and classifying texts in a target domain; text feature information of the source domain is accurately transferred in a text cluster of the target domain; and text content, different from the source domain, in the target domain is automatically found out.

The invention provides a text theme mining method based on an intra-sentence association graph and relates to the technical field of data mining. The technical problems that an existing mining method is low in quality and poor in universality can be solved by the text theme mining method. The method includes the steps that a target text is firstly divided according to sentences, a sentence sequence table of the text is acquired, then, a sentence association matrix of the target text is established, the weight of each element in the sentence sequence table is calculated, theme sentences are selected according to the calculated weights, the weights of all the non-theme sentences are adjusted each time the theme sentences are selected, theme sentences are selected again according to the adjusted weights, the operation is conducted repeatedly until the sum of character sizes of all the theme sentences reaches a preset character number threshold value, and finally, all the theme sentences serve as the theme content mined from the target text. The method is suitable for text documents of various forms of literature, styles and types.

The present application proposes a document topic mining method and apparatus. The method comprises: according to a preset topic mining number, performing loop iteration processing on information in at least one received document based on a probabilistic latent semantic analysis model, and acquiring a posteriori estimate of each topic implied by each sentence in each document; according to the posteriori estimate of each topic, acquiring a membership weight of each word in each topic in each sentence; and generating a topic set corresponding to the topic mining number, wherein each topic set comprises a word related to each topic and screened out according to the membership weight of each word in each topic in the sentence. According to the document topic mining method and apparatus provided by the present application, the document topic is more comprehensively and accurately mined based on a PLSA (Probabilistic Latent Semantic Analysis) algorithm, and the correlation of document topic content is improved, thereby enabling a result of a search engine to be closer to semantic information of the document.

The invention designs a microblog cell division method based on user comprehensive similarities. According to the specific process of the method, 1, microblog data is acquired, LDA topic model training is performed on a blog article set, and a user topic similarity matrix is obtained through topic mining based on feature extension; 2, a network topological graph with users being nodes and user relations being edges is constructed, and a user comprehensive similarity matrix is obtained according to node link relevancy and topic similarities; and 3, a unique tag is allocated for each node first,the potential influence of each node is evaluated, then the descending order of the potential influences serves as a node selection order, the descending order of node comprehensive similarities serves as a tag update order of the nodes, and finally iterative update of the tags is performed. In this way, cell division can be performed on the microblog users through an improved tag propagation algorithm on the basis of considering the user comprehensive similarities, and the method has high application value for online public opinion monitoring, commercial user mining and the like.

The invention discloses a real-time news content-oriented stream topic evolution tracking method. The method comprises the steps of firstly, batching news contents collected in real time according totime periods, and mining a preliminary topic result for each batch of the news contents by adopting an LDA method; secondly, performing named entity identification in the batch of the news contents, and calculating correlation between topics and entities, thereby updating entity link relationships in an entity library; thirdly, through topic inner lexical item clustering, obtaining a topic-topic inner class cluster corresponding relationship, and storing topic results in a topic library; and finally, calculating popularity information of the topics and topic inner class clusters, and accordingto the popularity information, performing dynamic updating on LDA topic mining parameters for topic evolution tracking of the next batch of the news contents. According to the method, topic featuresand class cluster features of topic inner lexical items in the real-time news contents can be mined; difference among the topics and among different class clusters in the topics is fully utilized; andthe LDA topic mining parameters are dynamically updated.

The embodiments of the invention disclose a method and device for generating information. In one specific embodiment, the method includes: obtaining an article to be mined; using at least two topic mining methods, mining at least two types of topics of the article to be mined, and determining a relevance of the mined topic and the article to be mined; and based on the mined topic and the determined relevance, determining the topic of the article to be mined and the relevance of the article to be mined and the topic. According to the method and device for generating information, the topic of the article to be mined is mined from different dimensions to obtain more comprehensive and more accurate topics.

The invention discloses a short text topic mining method based on a semantic word network. The short text topic mining method comprises the following steps: 1) a model initialization stage: collectingexternal corpora in related fields, preprocessing the corpora, setting parameters and the like; 2) a theme unit construction stage: constructing a semantic word network, searching a specific word triangular structure, calculating model prior parameters and the like; 3) a model training stage: sampling model variables by using a Gibbs sampling method, and judging whether the model reaches a convergence condition or not; and 4) a result output stage: obtaining topic distribution of each word triangle according to a sampling result of each variable after model training is finished, and calculating topic distribution of the original document. According to the method, semantic information learned by an external corpus is combined with the word triangular theme structure, the method is appliedto short text theme mining, compared with a traditional word pair theme model, the method provides a solution for integrating external priori knowledge into the traditional theme model, and the quality of the mined theme is remarkably improved.

The invention discloses an abstract generation method based on a TMPP (Topic Model based on Phrase Parameter). The abstract generation method is characterized in that a parameter [Theta] which shows document-topic in standard LDA (Latent Dirichlet Allocation) is expanded into an (aspect, rating) set, the TMPP is used for simultaneously modeling of the aspect and the rating, and a potential clustering variable c is introduced to show domain priori knowledge to guide the model to generate an aspect with better quality. The TMPP is adopted to generate an (aspect, rating) abstract, topic mining quality is guaranteed, an unsupervised learning way of the LDA is effectively overcome, and the phenomenon of the generation of meaningless topics is avoided.

The invention provides a Biterm topic model (BTM) sampling acceleration method. The method includes: establishing an alias table for each term, and selecting one Biterm topic model; sampling one new topic for the Biterm from a corpus proposal and calculating probability of acceptance; judging whether the probability of acceptance is greater than r or not, if yes, updating the Biterm, or otherwise, performing no updating; sampling another new topic for the Biterm topic model from a word proposal and calculating probability of acceptance; judging whether the probability of acceptance is greater than r or not, if yes, updating the Biterm topic model, or otherwise, performing no updating. With the method, complexity of sampling time of BTM can be optimized, convergence rate of the BTM can be greatly increased, quality of final topic clustering is unaffected, time for essay topic mining can be optimized, and meanwhile, time for text topic mining can be optimized as well.

The invention provides a user text information analysis method. The method includes: processing to-be-analyzed text information; carrying out potential topic mining on the preprocessed to-be-analyzedtext information, and obtaining topic probability distribution of the text; calculating the similarity of the text according to the topic probability distribution, and performing user characteristic value clustering according to the similarity; performing digital marking on the clustered to-be-analyzed text information to obtain to-be-analyzed sample data; and inputting the to-be-analyzed sample data into a pre-established user preference analysis model to obtain a user preference analysis result. According to the scheme, the text similarity between the users is calculated by deeply mining thetext features of the users, and clustering analysis is performed according to the similarity distance, so that the structure of a hidden layer of the deep neural network is simplified, and the learning efficiency of the deep neural network is improved.

The invention provides a topic mining method and apparatus. When an iterative process is performed each time, a target message vector is determined from a message vector according to a residual error of the message vector; a current document-topic matrix and a current word-topic matrix are updated only according to the target message vector; and a target element in a word-document matrix corresponding to the target message vector is only calculated according to the current document-topic matrix and the current word-topic matrix, so that the calculation of all non-zero elements in the word-document matrix in each iterative process is avoided, the update of the current document-topic matrix and the current word-topic matrix according to all message vectors is avoided, the calculation amount is greatly reduced, the topic mining speed is increased, and the topic mining efficiency is improved.

The present invention discloses a topic mining based event cluster acquisition method. The method comprises the following steps: S1, collecting a text data set C; S2, preprocessing the text data set C, and removing meaningless words from the text data set; S3: setting the number n of topics and a parameter, running CTM to obtain a CTM model; S4, in a covariance matrix sigma indicating an association degree between topics of the CTM model, searching for all maximum clusters by using a backtracking algorithm, wherein the maximum clusters are topic clusters; and S5, for each topic comprised in each topic cluster, selecting a most corresponding article from the text data set C, and clustering events corresponding to the most corresponding article to form an event cluster. The method provided by the present invention has the following advantages: association degree information at the topic level is used during association degree analysis, and compared with the traditional technology in which association degree information is mined and used at the word level, the method provided by the present invention can better improve rationality in calculating an event association degree.

The invention discloses a veterinary drug residue knowledge graph construction method based on weighted LDA (Latent Dirichlet Allocation). The method comprises the following steps: firstly, constructing a veterinary drug knowledge framework, and performing deep search and downloading literature by using a web crawler in combination with the knowledge framework; and aiming at topic noise existing in the LDA topic model and a feature word bias problem, performing topic mining by using a weighted LDA method, and downloading veterinary drug related literatures again; completing named entity identification and relationship extraction by using a dictionary-based model; and finally, utilizing the Neo4j graph database to construct a veterinary drug knowledge graph. The veterinary drug residue knowledge graph can be constructed, veterinary drug residue characteristic rules and causes of damage of veterinary drug residues to human bodies can be found out, the quality safety of meat, eggs and milk is guaranteed, and therefore the body health and life safety of people are protected.

The invention discloses a scenic spot evaluation knowledge base construction method based on metaphor topic mining. The method comprises the steps of using a scenic spot recessive topic mining algorithm to construct a scenic spot recessive multi-topic knowledge base; S2, constructing a metaphor multi-topic knowledge base of the scenic spot by adopting a scenic spot metaphor topic feature mining algorithm; S3, constructing a scenic spot evaluation knowledge base based on the semantic matching calculation model of the scenic spot corpus, and identifying the theme to which the tourist comment data belongs and the emotional tendency corresponding to the theme based on the scenic spot evaluation knowledge base. According to the invention, the scenic spot evaluation knowledge base considering metaphor information is constructed; according to the technical scheme, the fine-grained theme of each comment and the emotional tendency information of the corresponding theme in the internet tourism website can be accurately judged, data support is provided for tourists, the tourists are assisted in making decisions conforming to the preferences of the tourists, scenic spot managers can be assisted in improving scenic spot services, and the network public praise of the scenic spots is improved.

The invention relates to the technical field of artificial intelligence, and provides a text topic mining method and device, computer equipment and a storage medium. The method comprises the steps: employing a Gaussian kernel function to calculate and obtain a similar matrix based on a plurality of texts; performing spectral clustering on the plurality of texts based on the similar matrix to obtain a plurality of text clusters; extracting theme keywords of each text cluster; calculating the reading frequency of each text in each text cluster, and calculating the reading frequency of the theme keyword of the corresponding text cluster based on the reading frequency of each text; and mining according to the reading times of the topic keyword of each text cluster to obtain a text topic. According to the method, the problem of theme dispersion can be solved, and the mined themes better meet the actual requirements of users.

The invention discloses a microblog topic mining method based on heterogeneous social media user dynamic behaviors, which comprises the following steps: constructing an attribute multivariate heterogeneous dialogue network, and mining heterogeneous social context for topic detection; introducing a neighbor-level attention mechanism and an interaction-level attention mechanism to model different influences of different neighbors and different types of interaction modes on topic inference, and learning embedding of a specific view; the representations of the plurality of views are used as inputsof multi-view neural variational reasoning, and complex associations among topic semantics carried by different views are captured, so that topics with better consistency are mined.

The invention discloses an urban functional area identification process based on space-time semantic mining, which comprises documents, words, basic functional units, space-time data, a topic model, document topic distribution and unit function distribution, and is characterized in that firstly, hidden functions of an area are tried to discover through the topic model, compared with a text theme mining, the basic function units are equivalent to the documents in a corpus, space-time data in the basic function unit is similar to words in the document, unit function distribution obtained after passing through the topic model is equivalent to document topic distribution, and the used city space-time data is representative Sina microblog position sign-in data. Each piece of sign-in data comprises user information, space coordinates of sign-in positions, publishing time, publishing texts and the like. Dynamic activity modes of people can be reflected from different angles, meanwhile, POIs in a research area are obtained from a Baidu map, and function recognition of the area is achieved.