Low-Resource Language Information Extraction via Transfer Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing open information extraction methods are limited in extracting structured data from unstructured text in low-resource languages due to the lack of training data and linguistic expertise, as they predominantly focus on high-resource languages like English.
Innovation Solution
The method employs transfer learning by obtaining a corpus of high-resource language data structures, filtering and augmenting them through type substitution and code-switching, and training a multi-head self-attention transformer model with dynamic iterative tagging and gradient descent algorithms to extract machine-readable data structures from low-resource language text.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing open information extraction methods are used, then extraction can be performed on high-resource languages like English, but extraction from low-resource languages fails due to lack of training data
Solution Approach 1:
The patent uses high-resource language data structures as an intermediary to transfer knowledge to low-resource languages. By obtaining corpus data structures from high-resource languages and performing type substitution and code-switching, the system creates augmented training data that enables the model to extract information from low-resource languages without requiring native training data
Solution Approach 2:
The patent changes the linguistic parameters of the training data by performing type substitution (replacing entities with entities of the same type) and code-switching (replacing entities with equivalent low-resource language entities). This transforms high-resource language data into augmented data structures that maintain structural integrity while adapting to low-resource language contexts
2Adaptability or versatility
If transfer learning from high-resource languages is applied, then extraction capability for low-resource languages is enabled, but model complexity increases due to data augmentation and code-switching operations
Solution Approach 1:
The patent performs preliminary data augmentation operations including type substitution and code-switching on high-resource language corpus data structures before training the model. This pre-processing creates augmented data structures that encode linguistic patterns and relationships, allowing the model to learn transferable knowledge without requiring complex architecture modifications
Solution Approach 2:
The patent creates copies of high-resource language data structures and transforms them through type substitution and code-switching to generate augmented training data. These copied and transformed data structures serve as synthetic training examples that enable the model to learn low-resource language patterns without direct exposure to native low-resource language training data
Data Source
AI summary
A method is provided for extracting machine readable data structures from unstructured, low-resource language input text. The method includes obtaining a corpus of high-resource language data structures, filtering the corpus of high-resource language data structures to obtain a filtered corpus of high-resource language data structures, obtaining entity types for each entity of each filtered high-resource language data structure, performing type substitution for each obtained entity by replacing each entity with an entity of the same type to generate type substituted data structures, and replacing each entity with an equivalent a corresponding low-resource language data structure entity to generate code switched sentences. The method further includes generating an augmented data structure corpus, training a multi-head self-attention transformer model, and providing the unstructured low-resource language input text to the trained model to extract the machine readable data structures.