A method for generating web document summaries based on text structure analysis

Abstract

The invention discloses a text structure analysis-based Web document abstract generation method. The method comprises the steps of using a URL (uniform resource locator) as input, integrating the webpage main bodies of visual features and text features for extraction, partitioning the main bodies into a plurality of semantic paragraphs, and abstracting each semantic paragraph, so the generated abstract has higher coverage rate. The text structure analysis-based Web document summary generation method realizes the generation of the text abstract with better quality from a Webpage aiming at the conditions that the Webpage structure is complex, the main body is hard to identify and the Chinese automatic abstract is still positioned in the probe stage.

Description

technical field [0001] The invention relates to the technical fields of web page text extraction, natural language processing, and Chinese automatic summarization, in particular to a generation method of web document summaries based on text structure analysis. Background technique [0002] At present, the Internet has become the main source for people to obtain information. Especially with the rapid development of User Generated Content (UGC) in recent years, the information on the Internet is growing explosively. Although search engines can return search results according to user requirements. But users still need to find the most suitable webpage for their needs from the search list, especially because there are a large number of search engine optimization and reposting phenomena on the Internet, which brings great difficulties to users to quickly and accurately find information. [0003] The automatic summarization system uses computers to quickly process web documents,...

AI Technical Summary

Problems solved by technology

Since there is still no major breakthrough in natural language processing technology, the method based on comprehension cannot really realize automatic summarization

[0006] The research history of automatic summarization technology for Web documents is even shorter. "Compared with traditional texts, the text structure of web pages is loose, the title naming is relatively less rigorous, and there may be no terminator at the end of a sentence, and there are a large number of inconsistencies with the text." related content, which brings some difficulties to the generation of summaries

Images