Character search system and method based on search engines

Abstract

The invention relates to a character search system and method based on search engines. The system comprises a data collection module, a data preprocessing module, a feature extraction module and a clustering module which are connected in sequence. The data collection module crawls webpage information returned by the search engines through search of names. The data preprocessing module filters webpages irrelevant to the names, carries out block processing and filters vision blocks irrelevant to searched names in the webpages. The feature extraction module extracts attributes and entities related to searched characters, carries out statistics on word frequencies in the vision blocks, constructs a vector expression form of each webpage, and properly increases values of corresponding dimensions of feature values in a vector space. The clustering module takes the vector expression form of each webpage as input, clusters webpage texts and outputs a list composed of webpage class labels. According to the system and method, the problem that in the returned webpages, the names are ambiguous and the information is disordered when the characters are searched is effectively solved, character digests are constructed through extraction of the character attributes and character relationships, and the convenience for a user to search the names is provided.

Description

technical field [0001] The invention relates to a search engine-based character retrieval system and method, and belongs to the technical field of Internet and search. Background technique [0002] At present, the main difficulty of person retrieval is that there are problems of name ambiguity and information clutter in the web pages returned by the person name search. Name disambiguation refers to distinguishing multiple individuals with the same name. The ubiquity of ambiguity in personal names has caused a lot of inconvenience to information dissemination and resource acquisition. The search results for personal names provided by mainstream search engines are often a mixture of all web pages of the same name and irrelevant web pages. These web pages are sorted according to certain rules and are followed. Person information with high degree of accuracy is more likely to be ranked in the front position. For example, for "Li Na" in the Baidu search engine, among the search...

AI Technical Summary

Problems solved by technology

However, this patent directly extracts, word-segments, and part-of-speech tagging from the text of the webpage to form a document. Currently, the types of webpages returned by search engines are complex and have various structures. The sidebars and multi-level titles in the webpage often contain most of the information about the searched names

The method of this patent cannot extract the name-related information in the non-text text of the webpage, which seriously affects the effect of clustering; the clustering algorithm of this patent needs to extract the information of the person field in the text, and the amount of extracted information has a great impact on the effect of clustering The impact is very large, and the threshold of clustering needs to be manually specified, and there is an impact of manual intervention on the clustering effect

Images