Standard word library word segmentation method, device and equipment and computer readable storage medium

Abstract

The invention discloses a standard word library word segmentation method, device and equipment and a computer readable storage medium, and the method comprises the steps of splitting standard words ina standard word library to be subjected to word segmentation into single Chinese characters, forming a Chinese character library, and generating the adjacent frequency between every two Chinese characters in the Chinese character library; carrying out merging operation on the Chinese characters in the Chinese character library according to the adjacent frequencies to generate a Chinese charactergroup, and carrying out updating operation on the adjacent frequencies among the Chinese characters in the Chinese character library after the merging operation; judging whether the maximum frequencyvalue of the adjacent frequencies among the Chinese characters in the updated Chinese character library is smaller than a preset threshold value or not; if not, executing the step of carrying out merging operation on each Chinese character in the Chinese character library according to the adjacent frequency; and if yes, forming standard segmented words of the standard word library to be segmentedby the Chinese character groups. According to the scheme, the standard words in the standard word library to be segmented are segmented through the adjacent frequencies among the Chinese characters, and the word segmentation accuracy of the standard word library to be segmented can be effectively improved.

Description

technical field [0001] The present invention mainly relates to the technical field of data processing, and in particular, relates to a standard thesaurus word segmentation method, device, equipment and computer-readable storage medium. Background technique [0002] NLP (Natural Language Processing, Natural Language Processing) is a sub-field of artificial intelligence. Currently, NLP is mainly calculated through the existing word segmentation database, which can be performed through dictionaries or artificially assisted annotation. [0003] When segmenting the standard thesaurus through a dictionary or manual assisted annotation, due to the limited dictionary or manual assisted annotation, it is impossible to accurately segment the standard thesaurus, so that the word segmentation of the standard thesaurus does not meet the requirements of NLP in specific fields, such as the medical field. The application needs to re-segment the existing standard thesaurus. Therefore, how t...

AI Technical Summary

Problems solved by technology

[0004] The main purpose of the present invention is to provide a standard lexicon word segmentation method, device, equipment and computer-readable storage medium, aiming at solving how to improve the standard in the prior art The problem of word segmentation accuracy in thesaurus

Images