Multi-level natural language anti-junk text method and system

A natural language, multi-level technology, applied in the field of information processing, can solve problems such as poor recognition of junk text, and achieve the effect of avoiding adverse effects, high robustness, and efficient recognition

Pending Publication Date: 2019-07-05
SUN YAT SEN UNIV +1
View PDF6 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In practical applications, spam text words will be replaced in various deformation ways, resulting in t

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-level natural language anti-junk text method and system
  • Multi-level natural language anti-junk text method and system
  • Multi-level natural language anti-junk text method and system

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0064] Example 1

[0065] see figure 1 , a multi-level natural language anti-spam text method, including the following steps:

[0066] S101, receiving the text to be recognized;

[0067] S102, based on the original sensitive word database, match the original sensitive words on the text to be recognized, identify the original sensitive words in the to-be-recognized text, and output a sensitive word recognition result; wherein, the original sensitive word database includes original sensitive words sensitive words;

[0068] S103, based on the sensitive word variant library, perform matching of sensitive word variants on the to-be-recognized text, and perform semantic analysis on the matched suspected words to verify whether the suspected words belong to sensitive words, and output the identification of sensitive word variants The result; wherein, the sensitive word variant library is established according to the original sensitive word library, and the sensitive word variant...

Example Embodiment

[0083] Example 2

[0084] Embodiment 2 is an improvement on the basis of Embodiment 1, mainly for how to establish the sensitive word variant library, please refer to figure 2 , the establishment of the sensitive word variant library includes the following steps:

[0085] S201, obtaining keywords that form the original sensitive words from the original sensitive word database;

[0086] S202, compare existing Chinese characters and the keyword in terms of phonetic, and obtain the phonetic similarity of the existing Chinese character and the keyword;

[0087] S203, compare existing Chinese characters and the keywords on the glyph, and obtain the glyph similarity between the existing Chinese characters and the keywords;

[0088] S204, filter out the similar words of the keyword according to the phonetic-shape similarity and the glyph similarity;

[0089] S205, according to the mapping relationship corresponding to the split word, obtain the split word of the keyword;

[00...

Example Embodiment

[0113] Example 3

[0114] Embodiment 3 is an improvement on the basis of Embodiment 1 or 2. It mainly focuses on how to classify the text to be recognized, and obtain the pre-judgment probability that the text to be recognized is garbage text. Please refer to Figure 4 , including the following steps:

[0115] S301, performing word segmentation and quantization on the text to be identified, to form vectorized information to be identified;

[0116] S302, using a deep neural network classification model combined with a convolutional neural network and a long short-term memory network and trained on a corpus data set to process the vectorized information to be identified, and obtain a pre-judgment probability that the text to be identified is junk text .

[0117] Through the above steps, the continuous text is segmented and vectorized, which is easy for subsequent analysis by means of mathematical models; the deep neural network classification model that combines convolutiona...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a multi-level natural language anti-junk text method and system. The method comprises the steps of obtaining a sensitive word recognition result and a sensitive word deformation recognition result of a to-be-recognized text; and performing text classification on the to-be-identified text to obtain a pre-judgment probability that the to-be-identified text is a junk text, and performing comprehensive judgment based on the sensitive word identification result, the sensitive word deformation identification result and the pre-judgment probability to obtain a final probability that the to-be-identified text is the junk text. According to the method, the junk text can be efficiently recognized, the adverse effect of the junk text on the Internet health communication environment can be avoided, the robustness is higher, and the method can be widely applied to the social contact, comments and other Internet products.

Description

technical field [0001] The invention relates to the technical field of information processing, in particular to a multi-level natural language anti-spam text method and system. Background technique [0002] With the rapid development of the Internet, users use websites and applications more and more frequently, and the text content generated on the Internet is also emerging at an explosive speed, such as live barrage, post bars, comments, social platforms and other Internet content types. Products that drive a lot of text as the number of active users grows. However, there are still many junk texts in these texts, including advertisements, pornography, insults, violence, drugs, or other bad information. These spam texts contain sensitive words in various forms and are characterized by fast update and high degree of freedom. They are widely spread on the Internet, seriously affecting the healthy development of the Internet. In order to create a harmonious and pure Internet ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/27G06F16/33G06F16/35G06F16/903
CPCG06F40/284G06F40/30Y02D10/00
Inventor 叶志豪刘冶桂进军李宏浩印鉴
Owner SUN YAT SEN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products