Method and system for extracting complex named entities from Web video p ages

Abstract

The invention relates to a method and a system for extracting complex named entities from Web video pages. The method comprises the following steps: step 1: extracting effective text information from each Web video page in a Web video page set, wherein the effective text information forms video texts, and all video texts form a training set; step 2: classifying the Web video pages, selecting classifications, setting guide words for each selected classification, and selecting words which are related to the guide words and are uniformly and intensively distributed among the classifications from the training set as characteristic words; and step 3: extracting the words which are related to the characteristic words from the training set as candidate complex named entities, and selecting corresponding complex named entities for each selected classification from the candidate named entities according to the correlation degree of the classification with the characteristic words related to the candidate named entities. The method and the system of the invention can be used for extracting complex named entities from Web video pages without carrying out model training for a long time.

Description

technical field [0001] The invention relates to the field of information extraction, in particular to a method for extracting complex named entities of Web video pages. Background technique [0002] With the growth of network bandwidth and the application of Web2.0 technology, YouTube, Youku, Tudou and other video sharing websites have developed rapidly at home and abroad, and the number of Internet videos and the scale of users have increased on a large scale. At present, there are more than 300 video sites on the domestic Internet, among which the number of videos on Youku, Tudou and other websites has exceeded 10 million. How to accurately and effectively extract text information from Web video pages has become an important issue in the field of information extraction. Extracting text from web pages is essentially a process of extracting information from semi-structured text. [0003] Web video pages contain a large amount of text information, such as movie titles, TV dr...

AI Technical Summary

Problems solved by technology

However, in Web video pages, named entities lack rich context information; at the same time, the algorithms in the prior art require large-scale data sets and long-term calculations, which are not suitable for timely discovery of emerging complex named entities

Ordinary named entities and complex named entities are very different in concept and form of expression, so that the algorithms in the prior art cannot be directly applied to the recognition and extraction of complex named entities; Classification of

Images