Method and system for recognizing concept type web pages

Abstract

The invention discloses a method for identifying a conceptual web page and a system thereof. The method comprises the following steps: a plurality of conceptual web pages are acquired from a web page database; the URI amount of the conceptual web pages under all levels of directories of each website domain name is compared with a first threshold, the directory under which the URI amount of the conceptual web pages is greater than the first threshold is determined as a conceptual directory; and the URI of the web page to be identified is matched with each conceptual directory, if matched, the web page to be identified is determined as the conceptual web page. The method can quickly and comprehensively distinguish whether the web page is a conceptual web page and the class thereof. The method increases the identification rate and obviously improves the coverage rate in regard to identifying a conceptual document from mass web page data.

Description

technical field [0001] The present invention relates to the field of network information processing, and more specifically, to a method and system for identifying conceptual webpages. Background technique [0002] With the rapid increase of text and multimedia content used in the Internet and other data networks and systems, the data volume of network information has increased dramatically. Therefore, how to help users obtain required information from massive network information as quickly and accurately as possible has become a hot issue in the field of network information processing. [0003] "Concept" usually refers to a knowledge unit (or a general semantic unit) formed by a unique combination of features. Conceptual documents usually take the explanation of the concept as the subject of the document, and describe the connotation and extension of the same concept. [0004] The prior art proposes a technical solution for analyzing and processing various network informat...

AI Technical Summary

Problems solved by technology

[0005] Invention 1 provides an independent, automatic and efficient means of identifying conceptual documents, but it has the following problems in practical application: (1) The method of Invention 1 has a certain error rate

Theoretically speaking, the ability to distinguish between the two cannot be achieved only by adjusting the system itself of Invention 1; and (2) the method of Invention 1 will have a certain degree of omission

That is, although Invention 1 guarantees relatively accurate and high-efficiency recognition, it does not guarantee to cover and recognize all conceptual documents

Especially on the Internet, for example, the description of some conceptual document pages does not appear the concept word itself, but the concept word is displayed separately as the title of the page; there are also some concept document content that is very short, which is not enough to use the method of Invention 1 adequate basis for judging

The method of Invention 1 has some inherent defects in the accuracy and recall of conceptual document recognition, either due to the limitation of the method, or due to the diversity of documents

In addition, even for accurately identified conceptual documents, it can be found in user behavior analysis that users' understanding of the authority of the site will affect the selection of search results

For the results of websites that can provide a large number of conceptual documents, users will be more inclined to trust and choose; while websites with only a small amount of conceptual documents, the user awareness is not high, and the results are more difficult to trust

Therefore, although the method of Invention 1 technically provides a means for quickly and accurately identifying conceptual documents, it still cannot fully meet the needs of users for searching

Images