Hot word analysis and statistic system and method

Abstract

The invention provides a hot word analysis and statistic system and method. The hot word analysis and statistic system comprises a subject analysis module, a focus vocabulary module, a word segmenting service module, an index service module, a lexical unit statistic module, and a hot word analysis module, wherein the subject analysis module is used for determining an analysis data source, defining an analysis subject, and defining the data type of each field; the focus vocabulary module is used for forming a focus vocabulary sequence; the word segmenting service module is used for extracting data information from a corresponding field, and segmenting words in the data information according to the focus vocabulary sequence, so as to generate a lexical unit sequence; the index service module is used for recording the index position, in a corresponding field, of each lexical unit, and recording data information corresponding to the lexical unit, so as to generate a lexical unit index file; the lexical unit statistic module is used for counting the number of each lexical unit; the hot word analysis module is used for feeding back the generated hot word related information and the word frequency of each hot word to a user. The hot word analysis statistic system and method have the advantages that accurate statistics of hot words and index storage related to the hot words are realized; the type analysis requirement of the hot words can be met; the hot word statistic accuracy is improved.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to a hot word analysis and statistics system and method. Background technique [0002] With the continuous improvement of the level of informatization, people's demand for data is no longer limited to simple data acquisition and data retrieval, but more attention is paid to using the collected information to discover and solve hidden problems. For example, in the field of public security technology, public security decision-makers pay more attention to generating hot words from the collected information through technical means such as text analysis and mining technology, and then solve business problems through statistical analysis results of hot words. [0003] Hot words are hot words. Simply put, when the word frequency of a word reaches a certain threshold, it is called a hot word. As a lexical phenomenon, the popularity of hot words reflects the issues and thing...

AI Technical Summary

Problems solved by technology

[0006] However, Lucene's word frequency statistics are used for the calculation of hit result scores, not as overall statistics of data resources

Solr's word frequency statistics are used to realize the automatic completion function. Although it is used as an overall statistics of data resources, for each information unit, if there are multiple target words in multiple fields (ie "domain"), it is only calculated once. It belongs to coarse-grained statistics, and the accuracy of statistics is poor

Images