A short text information retrieval system based on big data application

Abstract

The invention relates to the field of information searching, in particular to a big data application based short text information searching system. The short text information searching system includes a classification acquisition system and a short text pretreatment module; an output end of the classification acquisition system is in signal connection with an input end of a word segment pretreatment module; a word segment output by the word segment pretreatment module is processed by a word segment splitting module and a word segment diversity module, and data in a split word association set is transmitted to a sharing database. The big data application based short text information searching system is reasonable in function design, can count and process searching word segments on the basis of big data, can reasonably split and combine short texts while short text information is searched, can acquire the word frequency through backward recurrence of word segments, can perform information output according to the word frequency, can simplify the load of a searching system to the greatest extent, can achieve rapid comparison, can improve the timeliness of information searching in real-time update of big data, and can avoid data searching leakage.

Description

technical field [0001] The invention relates to the field of information retrieval, and relates to a short text information retrieval system based on big data applications. Background technique [0002] With the continuous advancement of social technology and life, short text information is growing explosively. Big data is a huge collection of data. In the process of information retrieval, it is necessary to use short text for information retrieval. The design of this information retrieval method is relatively complicated, and with the explosive growth of information, the fast and efficient retrieval process of information cannot be well realized, and the phenomenon of information retrieval will be largely missed, resulting in information retrieval. Mistakes have a greater impact. With the advent of the era of big data, it is necessary to design a convenient and efficient information retrieval system based on big data. Contents of the invention [0003] The purpose of the...

AI Technical Summary

Problems solved by technology

[0002] With the continuous advancement of social technology and life, short text information is growing explosively. Big data is a huge collection of data. In the process of information retrieval, it is necessary to use short text for information retrieval. The design of this information retrieval method is relatively complicated, and with the explosive growth of information, the fast and efficient retrieval process of information cannot be well realized, and the phenomenon of information retrieval will be largely missed, resulting in information retrieval. Mistakes have a greater impact. With the advent of the era of big data, it is necessary to design a convenient and efficient information retrieval system based on big data

Images