A Chinese word segmentation method and computing device based on professional vocabulary

Abstract

The invention discloses a Chinese word segmentation method based on professional vocabulary, which is suitable for execution in computing equipment, including: constructing a dictionary with a predetermined structure by reading entries one by one, wherein the entries with the same first word in the dictionary Arrange in ascending order according to the Unicode code, and establish multiple first arrays for storing entries with the same first character, and establish at least one second array in each first array for storing entry content and identification bits, identification bits It is used to identify whether the entry belongs to a professional vocabulary; use the binary search method to search one or more character strings in the sentence to be segmented in the dictionary, and obtain multiple to-be-determined word segments after the initial segmentation; according to the corresponding The identification bit sets the word segmentation weight for the word to be determined; and constructs a segmentation path according to a plurality of word segments to be determined and their weights, and selects the shortest path as the word segmentation result. The invention also discloses a computing device for executing the method.

Description

technical field [0001] The invention relates to the technical field of information processing, in particular to a Chinese word segmentation method and computing equipment based on professional vocabulary. Background technique [0002] Chinese information processing technology has been widely used in computer networks, database technology, software engineering and other computer fields, and Chinese automatic word segmentation is an important basic work for Chinese information processing, and word segmentation is involved in many Chinese information processing projects Problems, such as machine translation, automatic summarization, automatic classification, full-text retrieval of Chinese literature databases, search engines, etc. Since the Chinese text is consecutive and there is no space between words, the first problem encountered in Chinese text processing is the problem of word segmentation. The correct segmentation of words is a necessary condition for Chinese text proces...

AI Technical Summary

Problems solved by technology

However, the dictionaries used in the existing Chinese word segmentation technology are relatively general, and there is no dictionary specifically for professional vocabulary, which may lead to inaccurate word segmentation results.

Images