Spelling error correction method, device and electronic device based on similarity of Chinese character sound and shape

Abstract

The embodiment of the present application proposes a spelling error correction method, device, and electronic device based on the similarity of Chinese character sound and shape, including constructing a sample Chinese character set containing Chinese character sound and shape information according to a standard Chinese character database; calculating any Chinese character conversion cost based on the sample Chinese character set The similarity between the two Chinese characters corresponds to the phonetic and shape information of the Chinese character, and according to the obtained similarity results, a similar Chinese character set corresponding to each Chinese character in the sample Chinese character set is constructed; the candidate word associated with the target Chinese character is obtained, and the target Chinese character and the candidate word are combined The numerical relationship between the similarity and the threshold is screened to obtain the replacement Chinese character after error correction of the target Chinese character. Since the calculation of the conversion cost of Chinese characters only occurs in the initial process of phonetic and shape editing, the specific process does not involve the calculation between levels, it only needs to be taken from the pre-loaded mapping dictionary between Chinese characters and dictionaries; it can effectively reduce the BK tree Chinese error correction algorithm The huge amount of calculations brought about by continuous comparison from the top of the tree improves the efficiency of error correction.

Description

technical field [0001] The present application relates to the field of text error correction, in particular to a spelling error correction method, device and electronic equipment based on the similarity of phonetic and shape of Chinese characters. Background technique [0002] In scenarios such as Chinese Optical Character Recognition (OCR) and Automatic Speech Recognition (ASR), recognition errors may occur due to the similarity of Chinese characters in sound and shape. Generally, a large number of errors can be resolved by adding post-processing modules (Chinese spelling error correction) output by models such as ASR and OCR. [0003] In order to reduce the number of search traversals, a BK (Burkhard-Keller) tree structure is introduced. The BK tree constructs a tree structure based on the conversion cost of Chinese character strings between the correct lexicons, and then quickly searches for similar (Chinese character conversion cost) Chinese character strings based on t...

AI Technical Summary

Problems solved by technology

[0004] (1) The BK tree structure mainly supports natural word segmentation scenarios (word segmentation is easy to obtain in general search scenarios, and it is almost impossible to obtain correct word segmentation results in other scenarios), and the scope of use is limited;

[0005] (2) BK tree error correction requires real-time calculation of the Chinese character conversion cost between two Chinese character strings, and the performance is poor when the tree depth is deep;

[0006] (3) As the domain dictionary becomes larger and the BK tree becomes deeper, the performance will drop sharply;

[0007] (4) The retrieval range of BK tree increases sharply with the increase of the threshold of the number of wrong Chinese characters, and the performance will also drop sharply

Although BK tree similarity retrieval is a reasonable intervention method for Chinese misspellings, it has problems of insufficient performance and limited scope of use.

Images