Automated Document Summarization via Keyword Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manual extraction of information from multiple documents on emerging technologies is time-consuming and prone to delays, as users must sift through numerous web resources, making it challenging for organizations to stay updated and competitive.
Innovation Solution
An AI-based system that automatically summarizes documents by using a web crawler to find relevant documents, determining keywords, clustering documents using Deep Convolution Neural Networks, and generating summaries based on keyword relevance, eliminating the need for manual intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If manual extraction of information from multiple documents is performed, then users can obtain relevant information, but it consumes a lot of time and is prone to delays
Solution Approach 1:
The system enables self-service information extraction by automatically crawling, clustering, and summarizing documents without requiring human intervention. The automated pipeline processes documents through multiple modules including web crawler, feature extraction, clustering algorithms, and summary generation, allowing the system to serve itself in extracting and presenting relevant information.
Solution Approach 2:
The patent replaces the mechanical manual process of reading and extracting information with an automated computational system. Instead of human users manually browsing and extracting information from thousands of websites, the system uses algorithms including web crawlers, text ranking, deep convolutional neural networks for clustering, and automated summary generation to perform the extraction task.
2Loss of information
If users manually research multiple web resources to stay updated, then they can find relevant information, but the process is time-consuming and affects competitiveness
Solution Approach 1:
The system performs preliminary action by proactively crawling and collecting documents before users need the information. The automated pipeline continuously gathers, processes, and prepares summary information in advance, so that when users need updated information, it is already available without requiring them to initiate the research process.
Solution Approach 2:
The system enables self-service information extraction by automatically crawling, clustering, and summarizing documents without requiring human intervention. The automated pipeline processes documents through multiple modules including web crawler, feature extraction, clustering algorithms, and summary generation, allowing the system to serve itself in extracting and presenting relevant information.
3Extent of automation
If automated summarization system is implemented, then information extraction is automated and timely, but system complexity increases
Solution Approach 1:
The patent divides the complex automated summarization task into multiple modular components: document finder module with web crawler, keyword determination module using text ranking algorithms, clustering module with deep convolutional neural networks, score computation module, and summary generation module. Each module handles a specific aspect of the processing pipeline, making the overall complex system manageable through functional segmentation.
Data Source
AI summary
Disclosed is a system for automatically summarizing documents pertaining to a predefined domain. A document finder module enables a web crawler to crawl web resources in order to find a plurality of documents. A keyword determination module determines a set of keywords from the plurality of documents and a rank associated to each keyword of the set of keywords. A clustering module clusters the plurality of documents into one or more clusters. A score computation module identifies a subset of the set of keywords for each cluster upon computing a similarity score, corresponding to each keyword, for each cluster. A summary generation module generates a summary for each cluster based on presence of one or more keywords, of the subset, in each document classified in the cluster.