A method of constructing typo word knowledge base based on fuzzy matching and statistics

Abstract

The invention discloses a method for constructing a typo word knowledge base based on fuzzy matching and statistics, which performs word segmentation on corpus sentences to obtain scattered word strings, combines the scattered strings of words according to the scattered string combination rules to obtain merged word strings, and utilizes according to Chinese dictionaries The fuzzy matching algorithm obtains the similar word candidate set of the merged word string; obtains the adjacent element set of the merged word string and the adjacent element set of all similar words in its similar word candidate set; according to the co-occurrence of each adjacent element set element of the merged word string in the corpus Frequency judges whether a merged word string is a typo word string, if the merged word string is a typo word string, then establishes the wrong word pair of the merged word string according to the frequency of occurrence of the adjacent element set elements of similar words in the corpus . The method for constructing the wrong word knowledge base of the present invention solves the problem of low correction accuracy caused by sparse data and only judging wrong words based on the Chinese dictionary in the prior art. The system responds quickly and the accuracy meets the actual application requirements, effectively High performance and accuracy.

Description

technical field [0001] The invention relates to natural language processing in the field of artificial intelligence computers, in particular to the field of automatic proofreading of Chinese texts, in particular to a method for constructing a typo word knowledge base based on fuzzy matching and statistics. Background technique [0002] With the rapid development of information processing technology and the Internet, traditional text work is almost completely replaced by computers. Electronic texts such as e-books, e-newspapers, e-mails, and office documents, blogs, and microblogs have all become part of people's daily lives. However, there are more and more errors in the text, which brings great challenges to the proofreading work. Traditional manual proofreading has low efficiency, high intensity, and long cycle obviously cannot meet the needs of text proofreading. [0003] Automatic text proofreading is one of the main applications of natural language processing, and it i...

AI Technical Summary

Problems solved by technology

[0002] With the rapid development of information processing technology and the Internet, traditional text work is almost completely replaced by computers. Electronic texts such as e-books, e-newspapers, emails, and office documents, blogs, and microblogs have all become part of people's daily lives. However, there are more and more errors in the text, which brings great challenges to the proofreading work

Traditional manual proofreading has low efficiency, high intensity, and long cycle obviously cannot meet the needs of text proofreading

Images