High-accuracy website sensitive word detection method based on machine learning

Abstract

The invention relates to a high-accuracy website sensitive word detection method based on machine learning. The method comprises the following steps of performing rule matching on a to-be-detected file and the sensitive word database to obtain a document set containing sensitive words, processing and learning the training data to output a machine learning model, and then inputting the document setinto the model to obtain a website sensitive word detection result. Model training is carried out by combining a machine learning algorithm; sensitive word rule matching is firstly carried out on thecrawled website page, then machine learning automatic analysis is carried out on the output website after rule matching again, the data volume predicted by a machine learning model is reduced, the detection speed and accuracy are improved, and finally the possibility that the page contains sensitive words is obtained through statistical calculation. Due to machine learning, semantic analysis andcombined judgment with word segmentation meanings, the sensitive vocabulary recognition rate can be effectively improved, the monitoring accuracy can be ensured, and the labor cost of a supervision institution can be greatly reduced.

Description

technical field [0001] The present invention relates to the technical field of digital computing equipment or data processing equipment or data processing methods that are especially suitable for specific functions, and in particular to a method for detecting sensitive words on websites with high accuracy based on machine learning. Background technique [0002] For a country and even the world, a healthy network environment is very important, and it is related to the healthy development of this society. However, with the rapid development of the Internet, a large number of sensitive words are flooded on the Internet, such as words related to pornography, politics, people's livelihood, gambling, drugs, etc. This is a very serious challenge for a healthy Internet environment. Therefore, more and more institutions have begun to use specialized software to monitor sensitive words. [0003] In the prior art, many traditional monitoring software are based on rule matching, most o...

AI Technical Summary

Problems solved by technology

[0005] The invention solves the problem in the prior art that the monitoring software is mainly based on rule matching, which increases the false positive rate of sensitive vocabulary detection, and a large number of false positives appear, which brings huge labor costs to the website supervision organization, and provides a Optimized machine learning-based high-accuracy website sensitive word detection method

Images