Document Indexing via Local-Global Weighted Term Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current document-to-document search methods face challenges in determining relevance due to limited input text and reliance on meta information, which is time-consuming and incompatible across different databases, especially when dealing with large datasets and varied document structures.
Innovation Solution
A method and system for indexing documents by parsing text portions into local and global indexes, using weighted text terms to analyze and retrieve relevant documents, which adapts to user interactions and external data for improved relevance scoring.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional keyword search methods are used with limited input text, then search speed is improved, but relevance determination becomes impossible
Solution Approach 1:
The system performs preliminary processing by parsing input text into text terms, assigning local weights, and querying a pre-built global index before actual search occurs. This preliminary action prepares the search query in advance, enabling fast search while maintaining relevance through the pre-computed global term weights that capture document structure and content relationships.
Solution Approach 2:
The patent introduces a global index as an intermediary structure that mediates between the limited input text and the large database. The global index contains pre-computed term weights and document relationships, serving as a mediator that enables relevance determination without requiring extensive input text, thus resolving the contradiction between search speed and relevance precision.
2Productivity
If meta information is used for document searching, then search efficiency is improved, but compatibility across databases deteriorates
Solution Approach 1:
The patent replaces the mechanical/meta-based search system with a text-term-based semantic indexing system. Instead of relying on database-specific meta fields and tags, the system parses text into terms, assigns weights based on document structure and content, and uses a universal global index structure that can accommodate any document type, thereby achieving both efficiency and compatibility.
Solution Approach 2:
The global index structure serves multiple functions: it stores term weights, maintains document relationships, supports various search queries, and works across different database types. This universal structure eliminates the need for database-specific meta schemas, enabling efficient search while maintaining compatibility across diverse document formats and database systems.
3Measurement precision
If more input text is used in document-to-document searching, then relevance accuracy is improved, but search results decrease
Solution Approach 1:
The patent applies local quality by assigning different weights to different text terms based on their local context within the input document. Terms appearing in specific sections, headings, or repeated passages receive higher local weights, allowing the system to maintain relevance accuracy with fewer terms by focusing on the most significant local occurrences rather than treating all text equally.
Solution Approach 2:
The system changes the parameter of term weighting by computing global weights that reflect term importance across the entire database and local weights that reflect term importance within the specific input document. By combining these parameters (global weight × local weight), the system achieves high relevance accuracy with limited input text, as the parameter transformation extracts maximum relevance information from the available text.
Data Source
AI summary
Methods and systems for indexing a plurality of documents, each document comprising a text portion, the method parses the text portion of each of the plurality of documents to form a plurality of respective local document indexes each associated with a respective document, and stores the local document index in a database, Each local document index has a plurality of local text terms and a local weighting associated with each text term From the plurality of local document indexes, forming a global document index associated with each global text term. The global weighting is determined with respect to a parameter associated with a reference global text term. Also, methods and systems for analyzing a text portion, retrieving documents from a database relevant to the text portion and for refining the results of a search are disclosed.