Junk text filtering method and device, electronic device and storage medium

Abstract

The embodiment of the invention provides a junk text filtering method and device, an electronic device and a storage medium. Characteristic information of a word segmentation result is extracted by using a pre-trained deep learning network model, different weights are given to different components in the junk text through the attention mechanism model, feature information is combined through the attention weights, local key information of the text is captured, the to-be-filtered text is classified, the junk text is filtered, and the junk text filtering accuracy is improved.

Description

technical field [0001] The present application relates to the technical field of computer communication networks, in particular to a spam text filtering method, device, electronic equipment and storage medium. Background technique [0002] With the rapid development of Internet technology, the text system provides users with a place to exchange information, and users have a more free space to express their opinions. With the increasing application of text systems, various types of junk texts such as advertisements, pornography, politics, and terrorism have emerged. Spam text disrupts the normal online order, affects the normal experience of users, and brings major hidden dangers to the healthy development of the website. For the healthy and sustainable development of the website and to improve the user experience in the network environment, it is necessary to find and filter junk text. [0003] The existing filtering method is based on the keyword matching method to filter...

AI Technical Summary

Problems solved by technology

This method only relies on the matching of simple words to filter, and cannot understand the semantic information of the text, and cannot deeply grasp the connotation of the text

In addition, spam publishers will convert the text by adding punctuation marks, replacing traditional characters, or using pinyin. Such a word segmentation effect makes the traditional method based on keyword matching unable to obtain semantic association information and capture key words.

In short, the existing methods cannot solve the problem of Chinese ambiguity, and there are many missed judgments and misjudgments, resulting in unsatisfactory filtering effects.

Images