Time window and semantic meaning-based word variant normalization method and system

Abstract

The invention discloses a time window and semantic meaning-based word variant normalization method and system. The method comprises the following steps of: 1) selecting corpuses in a set time period before an appearance time of a given variant in a social network according to the appearance time so as to form a candidate corpus set D1; 2) adding corpuses similar to the corpus where the word variant is located in the aspect of semantic meaning in the candidate corpus set D1 into a candidate corpus set D2; 3) extracting candidate words from the set D2 to obtain a candidate word set; and 4) calculating a score of each pair of candidate word and variant according to a literal similarity and a context feature similarity between each candidate word and the word variant, determining a candidate word corresponding to the word variant according to the calculation result, and taking the determined candidate word as a normalized word of the word variant. The system comprises an acquisition module, a filter module, and obtaining module and a normalized word obtaining module. According to the method and system, the texts of social networks become more normalized, and the public opinion analysis and hotspot time tracing are convenient.

Description

technical field [0001] The invention relates to the field of social network data analysis, and is a method and system for standardizing variant words based on time windows and semantics to realize more targeted and accurate normalization of variant words in social networks. Background technique [0002] With the rapid development of social networks, hundreds of millions of information are posted on social networking platforms every day, bringing about an explosive growth of information. Information comes in various forms, including text, pictures, audio, video, etc. Among them, the text in the social network has the characteristics of randomness and informality. Variant words are a distinctive feature of Internet language as an irregular language. People often replace relatively serious, standardized, and sensitive words with relatively irregular and insensitive words in order to avoid censorship, express emotion, satire, and entertainment. A new word used to replace the o...

AI Technical Summary

Problems solved by technology

An important problem in the real network test set is that the known community structure is obtained based on people's observation and experience, while community discovery algorithms generally start from the topological structure, and it is impossible to predict how much the relationship between the two is.

Images