Semantic Search System Using Entity Pair Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current search technologies face challenges in providing accurate and contextually relevant results from large datasets, often producing false positives and missing relevant documents due to their reliance on keyword-based similarity scores without considering context.
Innovation Solution
The development of systems and methods that parse document texts into sentences, identify named entities, pair entities, and assess similarity based on semantic pairs, enabling the visualization of document relationships and distinguishing original from derived content, using techniques like Term Frequency-Inverse Document Frequency (TF-IDF) and graphical displays to indicate content overlap and borrowing directions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If keyword-based similarity scores are used for document retrieval, then search speed is improved, but search accuracy deteriorates due to false positives and missed relevant documents
Solution Approach 1:
The patent segments documents into sentences and sentences into semantic pairs of named entities, enabling fine-grained analysis of meaning relationships. This segmentation allows the system to retrieve documents based on semantic pair matches rather than keyword frequency, improving accuracy while maintaining speed through efficient indexing of segmented units.
Solution Approach 2:
The patent changes the retrieval parameter from keyword-based similarity scores to semantic pair-based similarity scores. By transforming the basis of similarity measurement from surface-level keywords to contextual semantic pairs, the system achieves both speed and accuracy improvements through more meaningful document matching.
2Quantity of substance
If general document-similarity scores are used to fetch more documents like a query, then quantity of results is improved, but relevance quality deteriorates due to false positives
Solution Approach 1:
The patent applies local quality by matching specific semantic pairs (local meaning units) rather than relying on general document similarity. Each semantic pair represents a localized concept relationship, allowing the system to retrieve documents with high relevance quality by matching specific local meanings while maintaining adequate quantity through comprehensive semantic indexing.
3Measurement precision
If semantic pair-based similarity scoring is implemented, then search accuracy is improved, but processing complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-processing documents into sentences and semantic pairs before retrieval operations. This preliminary segmentation and indexing of semantic pairs simplifies the actual retrieval process, as the system can directly query semantic pair matches rather than performing complex analysis during search, thus reducing processing complexity while maintaining high accuracy.
4Measurement precision
If content overlap metrics are calculated between documents, then plagiarism detection accuracy is improved, but computation time increases
Solution Approach 1:
The patent segments document content into semantic pairs, enabling plagiarism detection through comparison of these discrete semantic units. This segmentation allows efficient computation of content overlap metrics by counting and comparing semantic pair frequencies rather than processing entire documents, improving detection accuracy while reducing computation time through the modular nature of semantic pair comparisons.
Data Source
AI summary
Methods and systems for searching over large (i.e., Internet scale) data to discover relevant information artifacts based on similar content and/or relationships are disclosed. Improvements over simple keyword and phrase based searching over internet scale data are shown. Search engines providing accurate and contextually relevant search results are disclosed. Users are enabled to identify related documents and information artifacts and quickly, ascertain, via visualization, which of these documents are original, which are derived (or copied) from a source document or information artifact, and which subset is independently generated (i.e., an original document or information artifact).