Hierarchical Structure Parsing Framework for Requirement Engineering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems face challenges in processing unstructured textual data, particularly in requirement engineering, where the logical structure of documents is lost during storage, leading to processing errors and delays due to the lack of internal structure in databases.
Innovation Solution
A novel computerized framework employing a two-phase deep learning process to rebuild document structure, involving text snippet re-segmentation, adaptive embedding, and hierarchy structure prediction, with a requirement text selection strategy to alleviate annotation burdens and handle domain-specific terms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional supervised learning frameworks are used for document structure parsing, then processing accuracy can be improved with sufficient annotation data, but manual annotation effort and time requirements increase significantly
Solution Approach 1:
The system performs preliminary segmentation of raw text into text snippets before the main structure prediction task. This preprocessing step organizes the unstructured text into manageable units with basic linguistic features extracted, reducing the complexity of subsequent annotation and processing tasks while maintaining parsing accuracy.
Solution Approach 2:
The document structure parsing is divided into two distinct phases: (1) text snippet segmentation that breaks raw text into structured units, and (2) hierarchy structure prediction that builds the logical structure from segmented snippets. This segmentation allows each phase to focus on specific tasks, reducing overall annotation burden while improving accuracy.
2Reliability
If deep learning models are trained with large amounts of annotation data, then parsing reliability improves, but the complexity of the system increases due to requirements for data collection and management
Solution Approach 1:
The system automatically extracts linguistic features and generates text snippet representations from raw text without requiring manual feature engineering. The deep learning model self-adapts to domain-specific terminology through training on annotated data, reducing the need for complex manual data preparation pipelines while maintaining high reliability.
Solution Approach 2:
The system employs adaptive embedding techniques that dynamically adjust the representation of text snippets based on their content and context. This allows the model to handle domain-specific terms effectively without requiring separate manual configuration for each domain, simplifying the system while improving reliability across different requirement engineering contexts.
3Ease of manufacture
If conventional systems store text in pure textual surface form, then storage simplicity is maintained, but processing capability deteriorates due to loss of logical structure
Solution Approach 1:
The system transforms one-dimensional raw text into a two-dimensional structured representation by creating text snippets with multiple attributes (content, position, linguistic features) and organizing them into hierarchical relationships. This dimensional transformation enables computers to process the text efficiently while maintaining a simplified storage format that can be converted back to original text when needed.
Solution Approach 2:
The system introduces text snippets as an intermediary representation between raw text and logical structure. These snippets serve as mediators that preserve the original text content while adding structural information, allowing simple storage of the snippet collection that can be easily processed to reconstruct documents or extract information without losing logical structure.
Data Source
AI summary
Disclosed are systems and methods for a computerized framework that provides a document structure parsing system for requirement engineering documents, where the logical structure of the text is not available, and is to be rebuilt based on the raw textual content. The framework approaches the build of the logical structure according to two phases. The first phase involves creating a list of list of text snippets from the raw text, where sequence labeling is adopted to re-segment and merge initially segmented text snippets. The second phase involves the framework executing computerized techniques including embedding adaptation approach, a hierarchy structure rebuilt algorithm, and a requirement text selection strategy to rebuild the hierarchy structure.