Chinese word segmentation method and device and search lexicon reading method

Abstract

In order to overcome the defects in the prior art, the invention provides a Chinese word segmentation method and device and a search lexicon reading method. The method comprises the steps of performing word segmentation on sentences to be subjected to word segmentation according to the input maximum word length, and obtaining a first-time word segmentation result; gradually reducing the length of the maximum word length, and performing word segmentation on the sentence to be subjected to word segmentation when the maximum word length changes each time to obtain an Nth word segmentation result; and comparing the first word segmentation result to the Nth word segmentation result with a word bank to obtain an output list. According to the invention, the to-be-segmented sentences can be accurately segmented, and particularly, the recognition rate of fixed words in the middle of the to-be-segmented sentences can be improved. The method has the advantages of being high in word segmentation efficiency and accurate in word segmentation result.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a Chinese word segmentation method and device, a system, an electronic device, a storage medium and a method for reading a search lexicon. Background technique [0002] Word segmentation technology belongs to the category of natural language understanding technology and is the first link in semantic understanding. It is a technology that can correctly segment words in a sentence. It is the foundation of text classification, information retrieval, machine translation, automatic indexing, speech input and output of text and so on. However, due to the complexity of Chinese itself and its writing habits, Chinese word segmentation technology has become a difficult point in word segmentation technology. [0003] Basic Algorithm of Chinese Word Segmentation In recent years, people have done some research on Chinese word segmentation technology, and put forward a variety of Chinese wo...

AI Technical Summary

Problems solved by technology

[0006] It can be seen that for the forward maximum matching algorithm idea and the reverse maximum matching algorithm idea, it is necessary to increase or decrease the process of adding or subtracting one word until the remaining word is left. In this process, if a fixed word is encountered in the middle When , there may be mis-segmentation, which affects the accuracy of word segmentation results

Images