Search Indexing Using Discourse Trees for Text Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing search solutions index entire bodies of text without distinguishing informative parts, leading to erroneous results and lower precision, as they fail to accurately identify relevant information for question-answer pairs, causing autonomous agents to provide incorrect answers.
Innovation Solution
The use of discourse trees to identify informative fragments of text for indexing, employing rhetorical structure theory to differentiate between nucleus and satellite text segments, and training machine learning models to predict which parts of the text should be indexed, thereby improving search engine precision and recall.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If entire bodies of text are indexed by keyword-based search engines, then search coverage is maximized, but search precision is lowered due to inclusion of irrelevant text
Solution Approach 1:
The patent segments text into rhetorical units (nuclei and satellites) based on discourse tree analysis. Nuclei represent central informative content while satellites represent peripheral or less important information. By segmenting text this way, the system can selectively index only nuclei (or prioritize them), thereby improving search precision while maintaining coverage through the hierarchical structure that preserves relationships between segments.
Solution Approach 2:
The patent applies different indexing qualities to different parts of the text based on their rhetorical importance. Nuclei receive full indexing weight while satellites receive reduced or conditional indexing. This local differentiation allows the search engine to treat informative content differently from peripheral content, resolving the contradiction between comprehensive coverage and precision by making quality selective rather than uniform.
2Adaptability or versatility
If irrelevant text is included in search indexing, then more content is available for retrieval, but autonomous agents provide incorrect answers reducing user confidence
Solution Approach 1:
By segmenting text into rhetorical nuclei and satellites, the system can provide comprehensive content availability through the complete discourse tree while ensuring answer accuracy by training autonomous agents to prioritize nuclei when generating responses. The hierarchical segmentation allows agents to access all content (maintaining versatility) while filtering to essential information (maintaining reliability).
Solution Approach 2:
The discourse tree structure acts as an intermediary between raw text and autonomous agent responses. It mediates the relationship by organizing text into informative (nuclei) and peripheral (satellites) components, allowing agents to navigate the full content space while generating accurate answers based on the structured rhetorical relationships that guide meaningful information extraction.
3Ease of manufacture
If keyword-based search indexing is used, then implementation is simple, but search results include erroneous matches from non-responsive or neutral text
Solution Approach 1:
The patent performs preliminary rhetorical analysis to build discourse trees before the actual search indexing process. This preliminary structuring of text into nuclei and satellites with their rhetorical relationships prepares the data in advance, allowing the search system to operate on pre-organized information that filters out erroneous matches while maintaining implementation feasibility through automated discourse parsing techniques.
Solution Approach 2:
The patent changes the indexing parameters from simple keyword frequency to a multi-dimensional schema incorporating rhetorical relationships, nuclei-satellite hierarchies, and discourse structure. This parameter transformation enables the system to distinguish between central informative content and peripheral or erroneous text, improving search result accuracy while the automated nature of the transformation maintains implementation feasibility.
Data Source
AI summary
Systems, devices, and methods of the present invention create a searchable index that includes informative portions of text. In an example, a computer-implemented method creates a discourse tree from a body of text. For each non-terminal node in the discourse tree, the method identifies a rhetorical relationship associated with the non-terminal node. The method labels each terminal node associated with the non-terminal node as either a nucleus or a satellite. The method further accesses a rule associated with the rhetorical relationship, and selects, based on the rule, selects the fragment associated with the nucleus. The method creates a searchable index including the selected fragments.