Structured Document Search Using Structure Model Tree and Frequency Indexing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Structured document databases face challenges in implementing high-speed scoring with high precision, as existing methods do not adequately consider the structural elements of documents, leading to inefficiencies in search processing and scoring accuracy.
Innovation Solution
A structured document processing apparatus and search apparatus that utilize a structure model tree to parse and update structured documents, calculate frequency-of-occurrence information, and generate query graphs to select similar structures, thereby optimizing scoring and processing costs, allowing for high-speed and precise search results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If scoring is implemented in structured document database considering structural elements, then scoring precision is improved, but processing speed deteriorates
Solution Approach 1:
The patent segments the structured document into multiple levels (document level, element level, attribute level) and calculates scoring at each level independently. The scoring function is divided into: document frequency component, element frequency component, and attribute frequency component. This segmentation allows parallel processing and reduces computational complexity while maintaining comprehensive scoring precision.
Solution Approach 2:
The patent performs preliminary indexing by pre-calculating and storing frequency information for documents, elements, and attributes before actual search queries are executed. The indexing unit creates inverted indexes that map terms to document IDs, element IDs, and attribute values, enabling rapid retrieval during search operations without re-processing the entire document structure.
2Adaptability or versatility
If full-text search functions (scoring, wild card, neighboring search) are implemented in structured document database, then search capability is improved, but device complexity deteriorates
Solution Approach 1:
The patent implements a universal scoring mechanism that handles multiple search functions through a single integrated framework. The same scoring function serves: exact match search, wildcard search, neighboring search, and fuzzy search. The search unit can invoke the scoring function with different parameters to achieve various search modes, eliminating the need for separate scoring implementations for each function.
Solution Approach 2:
The patent introduces an intermediary indexing structure that mediates between the structured document storage and search operations. The inverted index acts as an intermediary layer that translates diverse search queries into efficient database operations. This intermediary structure simplifies the search processing logic while enabling complex search functions.
3Measurement precision
If tf-idf scoring scheme is applied to structured documents, then scoring precision is improved, but processing time deteriorates
Solution Approach 1:
The patent performs preliminary calculation and storage of tf-idf components during the indexing phase. The indexing unit pre-computes document frequencies, element frequencies, and attribute frequencies, then stores them in the inverted index. During search, the retrieval unit simply retrieves pre-computed values from the index rather than calculating tf-idf scores in real-time, dramatically reducing processing time while maintaining scoring precision.
Solution Approach 2:
The patent creates a copy of the frequency information in the inverted index structure. Instead of storing the complete document structures and recalculating frequencies during search, the system stores compressed frequency statistics (document frequency, element frequency, attribute frequency) as copies that can be rapidly retrieved and used for scoring without re-processing the original documents.
Data Source
AI summary
A structured document processing apparatus includes an acquisition unit configured to acquire a structured document, a storage unit configured to store a structure model tree which indicates a typical structure of the acquired structured document, a parsing unit configured to parse the acquired structured document, an updating unit configured to update the structure model tree to match a structure of the parsed structured document therewith, a division unit configured to divide the acquired structured document into a plurality of lexical items, and a calculation unit configured to calculate frequency-of-occurrence information indicating locations of each of the lexical items in the acquired structured document.