Multi-modal Document Feature Extraction via Hierarchical Tree
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models face challenges in efficiently extracting and categorizing information from diverse and inconsistent documents due to variability in file types, quality, and formats, particularly for rare document types, which requires large training sets and struggles with multi-modal data like text and images.
Innovation Solution
A method for generating a machine learning model through multi-modal feature extraction, involving text and image extraction functions, feature tree generation, and input vector creation for a machine learning model, which refines features based on context-specific configurations and normalizes them into a tree structure, allowing for efficient classification and validation of documents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a machine learning model is trained to identify and extract information from documents, then the model can validate documents, but a large training set is required which makes the process inefficient
Solution Approach 1:
The patent segments the document processing into distinct modalities (text extraction, image extraction, layout analysis) and represents features as a hierarchical tree structure where different node types correspond to different feature categories. This segmentation allows the model to process each modality independently and combine results, reducing the complexity and data requirements for training.
Solution Approach 2:
The patent transforms document features into a multi-dimensional feature tree structure with different node types (text, image, layout, relationship nodes) rather than using flat feature vectors. This dimensional transformation enables the model to capture complex document structures and relationships more efficiently, reducing the amount of training data needed.
2Adaptability or versatility
If machine learning models process varying documents with different formats and qualities, then more document types can be handled, but the variability makes consistent detection and categorization challenging
Solution Approach 1:
The patent creates a universal feature tree structure that can represent multiple document types and modalities through a common framework. The hierarchical tree with standardized node types (text, image, layout, relationship) serves as a multi-functional representation that adapts to different document formats while maintaining consistent feature extraction and processing.
Solution Approach 2:
The patent changes the representation parameters from flat feature vectors to a hierarchical tree structure with typed nodes. This parameter transformation enables the system to handle variable document formats by mapping different document types into the same structured framework, ensuring consistent processing regardless of input variability.
3Adaptability or versatility
If rare document types are processed, then comprehensive document validation is achieved, but the lack of availability of these documents makes efficient detection and validation challenging
Solution Approach 1:
The patent performs preliminary structure analysis by creating a feature tree that captures the hierarchical organization and relationships of document elements before detailed classification. This preliminary structuring of features into typed nodes enables the model to recognize document patterns with fewer examples, improving efficiency for rare document types.
Data Source
AI summary
Systems and methods are described for generating a machine learning model for multi-modal feature extraction. The method may include receiving a document in a digital format, where the digital format comprises text information and image information, performing a text extraction function on a first portion of the document to produce a set of text features, performing an image extraction function on a second portion of the document to produce a set of image features, generating a feature tree, wherein a plurality of nodes of the feature tree correspond to the set of text features and the set of image features, and generating an input vector for a machine learning model based on the feature tree. In some cases, the feature tree may be generated synthetically, or modified by a user prior to being converted into the input vector.