System and method for extracting object identifiers from web pages

Abstract

This invention discloses a system and a method for extracting object identifiers from webpages. The system includes: an identifier identification module for identifying identifier blocks in a webpage, an identifier fragment extraction module connected to the identifier identification module and an identifier unit labeling module connected to the identifier fragment extraction module. The said webpage contains object identifier related information showing the information of the object identifiers, and an identifier block is a text containing the object identifier related information. The identifier fragment extraction module is used to remove useless information from the identifier block to obtain identifier fragments according to at least one of position information and content information of each wordin the identifier block recognized by the identifier identification module. The identifier unit labeling module is used for labeling the identifier fragments extracted by the identifier fragment extraction module as object identifiers suitable for constructing an object database.

Description

technical field [0001] The present invention relates generally to information processing and information extraction techniques, and more particularly, to systems and methods for identifying and extracting object identifiers from web pages. Background technique [0002] In the current field of information processing technology, it is often necessary to build an object database, which involves providing object identifiers with a hierarchical structure for object generation and object mapping, representing objects and establishing indexes. [0003] Here, the objects to be processed generally involve web pages on the Internet. Objects in the real world have their unique object identifiers (that is, names). Of course, other aliases or conventional abbreviations can also be used to represent object identifiers. For example, it is common for the same object to have different names in different web pages. . In the same web page, the representation of the same object is usually con...

AI Technical Summary

Problems solved by technology

[0008] However, the prior art mentioned above mainly has the following disadvantages: firstly, in the methods for extracting web page titles or product names disclosed in the above documents, only the features of the DOM tree and visual information may not be able to satisfy the requirements for extracting object identifier related information. the accuracy of

Moreover, the object identifier-related information provided by a single webpage is not comprehensive enough, and it is necessary to integrate the object identifier-related information from multiple webpages to obtain the object identifier

Images