Method for automatically classifying webpage content visited by Internet users

Abstract

The invention discloses a method for automatically classifying webpage content visited by Internet users. The method comprises the following steps: carrying out machine learning on webpage content samples accessed by finite Internet users based on a text classification technique according to a support vector machine and a decision forest technology, then building a set of decision system by a plurality of classifiers, and finally obtaining new webpage samples by self adaption and automatically classifying the new webpage samples. By virtue of the method, the actual problems of small samples, nonlinearity to linearity, data sparseness, high dimension of data, long classifier training time and local minimum points can be well solved; by virtue of the decision system, the problem of the inaccurate classification caused by a single classifier is solved; a plurality of operations can use a parallel MapReduce structure, so that the classifier training time is greatly shortened; the classification process is also capable of analyzing the webpage content of mobile Internet in milliseconds and finally classifying the webpage content into a predefined class.

Description

technical field [0001] The invention belongs to the field of computer technology, relates to network technology, and is a method for automatically classifying web page content accessed by Internet users. Background technique [0002] With the rapid development of mobile Internet information, in the face of hundreds of millions of massive information, people can no longer simply rely on manual processing of all information, and need auxiliary tools to help people better discover, filter and manage these information resources. Information mining has become a bottleneck in the development of science and technology and the further improvement of human life quality. Automatic text classification as the basis of mining has also become a major research hotspot in modern information processing research. [0003] The automatic text classification system has gone through three milestone stages: [0004] Stage 1: Knowledge engineering method. At the beginning, most of them used the m...

AI Technical Summary

Problems solved by technology

[0008] The technical problem to be solved by the present invention is: with the development of Internet technology, the existing automatic text classification system has problems such as small classification scale, sparse text data, and high data dimension that cannot be solved in the process of large-scale web page text classification. The defect that the classification effect is poor after running for a period of time

Images