Keyword identification method based on hidden markov model, keyword identification terminal device based on hidden markov model and storage medium

Abstract

The invention relates to a keyword identification method based on a hidden markov model, and the method comprises the following steps: S1, constructing the hidden markov model, wherein the hidden markov model comprises five elements including a hidden state S, an observable state O, an initial state probability matrix pi, a hidden state transition probability matrix A and an observation state matrix B; S2, after separating a target article into a word + word class format through a word segmentation algorithm, inputting the article into the built hidden markov model, acquiring an observable state sequence O, then, inputting the observable state sequence O into the built hidden markov model, and thereby obtaining a model mu; S3, based on the built hidden markov model mu and the obtained observation state sequence O = {O1, O2, ..., OT}, calculating a maximum possible value of the hidden state through a viterbi algorithm, thereby identifying whether each word is the keyword. With the method, the device and the storage medium provided by the invention, better universality is realized, the keywords can be executed simultaneously for a relatively long article or a relatively short article, and identification accuracy is high.

Description

technical field [0001] The invention relates to a keyword recognition method based on a hidden Markov model, a terminal device and a storage medium. Background technique [0002] At present, the main technologies for identifying sentence opinions in China are divided into three categories: [0003] 1. Recognition method based on dictionary and rule matching: mainly use emotion word ontology or emotion dictionary as the basis of recognition; [0004] 2. Statistics-based identification method: Mainly use Support Vector Machine (SVM), Naive Bayes (NaiveBayes) to train on the marked corpus, and then use the classifier obtained from the training to classify; [0005] 3. Recognition method based on LDA topic model: mainly using LDA topic model, regardless of the order of words in the document, each article is regarded as a mixed distribution of all topics, and the topic is regarded as the distribution of all words in the vocabulary Mixed distributions are used for identification...

AI Technical Summary

Problems solved by technology

[0007] The hidden Markov model is mainly used to solve the probability problem of continuous data. Currently, it is mainly used in word segmentation, speech recognition or data trend prediction (such as stock trend prediction). In terms of keyword extraction, the hidden Markov model is mainly used to cooperate with The textRank algorithm weights and extracts keywords from word frequency, part of speech, etc., so it is difficult to use for keyword judgment of short articles such as Weibo

Images