Lexicon-Based Data Analysis System for Merging Structured and Unstructured Information
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional lexicon analysis methods are inefficient in merging various types of data and presenting them to users for analysis, as they separate lexicons from applications, making it difficult to adjust or update lexicons to meet user needs, and fail to efficiently search through large documents.
Innovation Solution
A system that uses lexical analysis and filtering processes, involving an electronic display, processor, and data storage unit to extract data from items, create tuple-structures, and map them with lexicon terms, allowing for efficient analysis and presentation of data through structured and unstructured data integration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional lexicon analysis methods are used to search through large documents, then search speed is improved, but the ability to handle various types of data and adapt to user needs deteriorates
Solution Approach 1:
The patent merges structured data and unstructured data into a unified lexicon analysis framework. The system integrates multiple data types (databases, spreadsheets, text documents, images, audio, video) and processes them together through a common lexicon mapping mechanism, enabling versatile data handling while maintaining search efficiency
Solution Approach 2:
The lexicon is made dynamic and adaptable through user feedback mechanisms. The system allows users to add, remove, and modify lexicon terms and relationships based on their specific needs, enabling the lexicon to evolve and adapt to different domains and user requirements while preserving fast search capabilities
2Ease of manufacture
If lexicons are separated from applications, then lexicon creation is simplified, but the ability to adjust and update lexicons deteriorates
Solution Approach 1:
The system implements feedback loops where users can interact with the lexicon through the application interface. Users can provide feedback on lexicon accuracy, add new terms, modify existing relationships, and the system updates the lexicon accordingly, maintaining both ease of creation and adaptability to changing needs
Solution Approach 2:
The system performs preliminary lexicon creation through automated processing of source data, but maintains the capability for subsequent updates and modifications. The initial lexicon is generated from available data, but the system is designed to accommodate future adjustments without requiring complete recreation
3Ease of operation
If simple key word searches are used, then ease of operation is improved, but information retrieval accuracy deteriorates
Solution Approach 1:
The system introduces lexicon terms as an intermediary layer between simple keyword input and information retrieval. Users can perform simple searches using common terms, which are then mapped through the lexicon to precise domain-specific concepts, maintaining operational simplicity while improving retrieval accuracy through structured semantic relationships
Data Source
AI summary
The present invention is a method or system of analyzing items to create a data structure using lexicon analysis and filtering process. The present invention drives the analysis without a user being required to provide search terms and Boolean queries. Further, the present invention merges the use of unstructured data and structured data using a lexicon as a focus. Structured data is mapped to lexicon terms through one analysis process, while unstructured data is mapped to lexicon terms through a separate analysis process. In some disclosed embodiments, the mapping process uses a tuple structure of the source data to match with lexicon terms. The result is an analysis that merges the two different sets of data, which enables a user to navigate through the items using lexicons. Analysis results are displayed simultaneously in a tabular view, geospatial view, and lexicon histogram.