Word segmentation recognition lexicon construction method and Chinese word segmentation method and device

Abstract

The invention discloses a word segmentation recognition lexicon construction method and a Chinese word segmentation method and device, and relates to the technical field of computers. One specific embodiment of the method comprises the steps that for short sentences in a training text set, duplication elimination is conducted on the short sentences, corresponding neurons are constructed for all words in the short sentences obtained after duplication elimination, and the signal types indicated by the neurons are matched with the words corresponding to the neurons; according to the relative position and the occurrence frequency between every two characters in the short sentence, a link relationship is constructed between the two neurons corresponding to every two characters to form a short sentence neural network corresponding to the short sentence, and the link relationship indicates a link coefficient and a signal transmission direction; and the short sentence neural networks are fusedto form a word segmentation recognition word bank. According to the embodiment, the word quantity of the word bank and the word segmentation accuracy can be effectively improved.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a method for constructing a word segmentation recognition lexicon, a Chinese word segmentation method and a device. Background technique [0002] Chinese word segmentation based on thesaurus is one of the more commonly used word segmentation methods at present. Therefore, building and maintaining thesaurus is the basis for word segmentation. [0003] The existing lexicon is mainly built and maintained manually, that is, artificially collect some existing words such as words in the "Modern Chinese Standard Dictionary" and some new words that appear on the Internet, and store the collected words to thesaurus. [0004] In the course of realizing the present invention, the inventor finds that there are at least the following problems in the prior art: [0005] Compared with the massive text data in the network, the words collected manually are very limited, resulting in a very l...

AI Technical Summary

Problems solved by technology

[0005] Compared with the massive text data in the network, the words collected manually are very limited, resulting in a very limited amount of words stored in the thesaurus

Then, when performing word segmentation based on a manually constructed or maintained thesaurus, the amount of words stored in the thesaurus often cannot meet the word segmentation requirements

Images