Method, device and equipment for realizing standard word mapping

Abstract

The embodiment of the invention discloses a method, device and equipment for realizing standard word mapping, and the method comprises the steps: firstly obtaining first segmented words included in ato-be-recognized word, and respectively determining an information category label and an information content label corresponding to each first segmented word; taking the information category label andthe information content label corresponding to the first segmented word as an information category label and an information content label included in the to-be-identified word; respectively obtaininga first co-occurrence probability and a second co-occurrence probability corresponding to the information category label and the information content label included in the to-be-identified word and the information category label and the information content label included in the target standard word; and finally, determining a probability value of the target standard word according to the first co-occurrence probability and the second co-occurrence probability, and determining the target standard word with the maximum probability value as a standard word corresponding to the to-be-identified word. The probability that a to-be-identified word corresponds to a target standard word is determined from two aspects of the information category and the information content of segmented words, so that the accuracy and the efficiency of determining the standard word are improved.

Description

technical field [0001] The present application relates to the technical field of data processing, in particular to a method, device and equipment for realizing standard word mapping. Background technique [0002] In the process of daily text writing, some more common non-standard words may be used instead of the standard words to be used. A non-standard word refers to a vocabulary that has the same meaning as a standard word, but whose characters are different from those in the standard word. For example, abbreviated words corresponding to standard words, abbreviated words, and other alternative words used in daily use. [0003] Although non-standard words can replace standard words, non-standard words may make the semantics of the text to be represented inaccurate. Moreover, the meanings expressed by some non-standard words are unclear, and may not have a more accurate reference meaning, and it is not convenient to systematically extract relevant information from texts wi...

AI Technical Summary

Problems solved by technology

On the one hand, the efficiency of manual sorting is low; on the other hand, the corresponding relationship between the determined non-standard words and standard words is not accurate enough

Images