Accidental sensitive word discovery method based on word network

Abstract

The invention discloses an accidental sensitive word discovery method based on a word network, and the method comprises the steps: obtaining Internet text information through the public text data of the Internet or other channels, and setting a corresponding text language and the specific time of message sending; extracting texts, carrying out time division by taking time granularity (generally set to be one day) as a unit, screening sensitive texts of a specific language by combining an existing common sensitive word dictionary, cutting according to the positions of punctuation marks in the texts to obtain a plurality of short texts, and carrying out word segmentation processing on the short texts; constructing a word network on the basis of the short text, and calculating a maximum K-core value of the word network, and a K-core value and a core coefficient of each word in the network; and for the selected core word, extracting the number of times that the core word is located at thecore position in a specified historical period (generally set to be 30 days) and the average core coefficient of the time period that the core word is not located at the core position in the specifiedhistorical period, and finally discovering accidental sensitive words in the word network by utilizing a detection formula.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a word network-based discovery method for occasional sensitive words. Background technique [0002] Internet sensitive words refer to politically sensitive tendencies, violent tendencies, unhealthy color words or uncivilized terms that are blocked or tracked by Internet technology in real time. In today's network environment, sensitive words that have been at the core of discussions for a long time can often be detected and blocked by network technology, and these high-frequency sensitive words form a fixed dictionary of sensitive words. However, in the management of sensitive words, there are relatively few studies on occasional sensitive words. These sporadic sensitive words have never been in the fixed sensitive word dictionary, but with the evolution of some emergencies or popular topics, they will often be closely related to high-frequency sensitive words within a certain...

AI Technical Summary

Problems solved by technology

However, there is currently a lack of a complete and fast discovery method for occasional sensitive words

Images