Text classification technology-based information processing method

Abstract

The invention discloses a text classification technology-based information processing method. A text is preprocessed by adopting an HTML text mark weighting scheme; before an HTML document is subjected to scan processing, an HTML mark needs to be correctly identified and processed first and texts of different parts of a webpage are subjected to weighting processing according to the HTML mark; descriptive information such as titles, page descriptions, keywords, hyperlinks and the like can be reserved, so that the classification effect is improved; a symbol dictionary is established for filtering non Chinese characters, so that the dimension of an initial text vector is reduced and characteristic information content in the text vector is increased; and stop words are removed, so that the subsequent text filtering accuracy can be improved and the subsequent text filtering rate can be increased. The information processing method is simple in operation and high in practicality, and is capable of improving subsequent information filtering accuracy and efficiency.

Description

technical field [0001] The invention belongs to the field of network methods, and more specifically, the invention relates to an information processing method based on text classification technology. Background technique [0002] At present, network information presents the characteristics of huge quantity, extensive content, and various forms. As far as my country is concerned, relevant survey reports show that by the end of 2005, the total number of national web pages was 187 million, including information on science and technology, news, education, business, entertainment, etc. content. While such a large amount of information has brought great convenience to people's lives, there are also some increasingly prominent problems: First, a large number of information resources bring difficulties to retrieval, and the retrieval results based on keywords will have many irrelevant results. documents; the second is the problem of network security; how to quickly and effectively d...

AI Technical Summary

Problems solved by technology

While such a large amount of information has brought great convenience to people's lives, there are also some increasingly prominent problems: First, a large number of information resources bring difficulties to retrieval, and the retrieval results based on keywords will have many irrelevant results. documents; the second is the problem of network security; how to quickly and effectively discover useful knowledge and information with great potential value in these massive and heterogeneous massive information resources; how to reasonably classify and accurately locate the required information, and simultaneously process a large number Useless or irrelevant content has become the bottleneck of knowledge acquisition and information filtering and a hot topic in today's network security technology