Text word segmentation method and device

Abstract

The embodiment of the invention provides a text word segmentation method and device, and relates to the technical field of natural language processing. The method comprises the steps of obtaining a coarse-grained word segmentation result and a fine-grained word segmentation result of a text to be subjected to word segmentation; traversing coarse-grained segmented words in the coarse-grained word segmentation result to determine the character length of the coarse-grained segmented words; according to the character length of the coarse-grained word segmentation, correcting the coarse-grained word segmentation result to obtain a corrected coarse-grained word segmentation result; and determining a final word segmentation result of the text according to the fine-grained word segmentation result and the corrected coarse-grained word segmentation result. According to the scheme, the moderate word segmentation granularity of the text can be effectively ensured, so that the word segmentation accuracy of the text is effectively improved.

Description

technical field [0001] The embodiments of the present application relate to the technical field of natural language processing, and in particular to a text word segmentation method, device, electronic equipment, and computer storage medium. Background technique [0002] In the information age with the rapid development of the Internet, search engines are one of the powerful means for people to obtain effective information. The focus of Chinese search engines is on the extraction of Chinese key information, and the difficulty is automatic Chinese word segmentation. A good Chinese word segmentation method can effectively help search engines increase the accuracy and timeliness of information retrieval. [0003] At present, the Chinese word segmentation method used in search engines is word segmentation through the word segmenter, but the word segmenter relies too much on the dictionary. If the dictionary is not fully covered, the word segmentation result will be a single word...

AI Technical Summary

Problems solved by technology

[0003] At present, the Chinese word segmentation method used in search engines is word segmentation through the word segmenter, but the word segmenter relies too much on the dictionary. If the dictionary is not fully covered, the word segmentation result will be a single word or longer, that is, the word segmentation granularity is too large or too small , thus affecting the retrieval hit rate

Images