A Personalized Parallel Word Segmentation Processing System and Processing Method

Abstract

The invention relates to a personalized concurrent word segmentation processing system and a processing method of the processing system. The personalized concurrent word segmentation processing system and the processing method of the processing system comprises a word segmentation requesting module, a word segmentation module based on a personalized word segmentation dictionary, a word segmentation module based on a general word segmentation dictionary, a control module and a high speed word segmentation processing module. Word segmentation requests of a user are simultaneously sent to the word segmentation module based on the personalized word segmentation dictionary and the word segmentation module based on the general word segmentation dictionary. When the word segmentation module based on the personalized word segmentation dictionary is destined, word segmentation processing result is sent back to the word segmentation requesting module through the control module, and meanwhile word segmentation requests of the word segmentation requesting module to the word segmentation module based on the general word segmentation dictionary is interrupted; otherwise, dynamic update of the personalized word segmentation dictionary is proceeded according to an earliest and least using principle and the word segmentation processing result of the word segmentation module based on the personalized word segmentation dictionary by the control module. The personalized concurrent word segmentation processing system and the processing method of the processing system is capable of satisfying accuracy rate of the word segmentation, meanwhile improving word segmentation efficiency of the system greatly and satisfying efficient referring requirements of a mobile user.

Description

technical field [0001] The invention belongs to the fields of mobile search and Chinese information processing, and in particular relates to a personalized parallel word segmentation processing system and a processing method thereof. Background technique [0002] A word is the smallest unit with certain semantics. The so-called word segmentation is to segment a sentence according to the meaning of the words in it. Since the understanding and processing of natural language is generally based on vocabulary, and when Chinese text is expressed in writing or inside a computer, the basic unit of writing is the word, and there is no clear boundary between words. Therefore, Chinese word segmentation is an important aspect of Chinese information processing. It is also the key technology and difficulty in Chinese information processing such as text classification, information retrieval, information filtering, automatic document indexing, and automatic abstract generation. [0003] To...

AI Technical Summary

Problems solved by technology

However, in mobile search, the client has high requirements for real-time query. If the above-mentioned dictionary-based string matching word segmentation processing method is used, there will be a large query delay and the user query experience is poor.

Secondly, the phenomenon that users access the network through mobile terminals is usually concentrated in a few specific time periods. When a large number of users conduct mobile searches at the same time, they all need to rely on the dictionary for word segmentation processing, which will inevitably greatly increase the load of the word segmentation processing module in specific time periods, thereby further Increase query delay and reduce user experience