Sequence Labeling Model Training via Multi-Granularity Data Enhancement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The limited performance of sequence labeling models in extracting key information from contract documents due to the small number of labeled contract documents and limited training corpora.
Innovation Solution
A model training method that involves constructing initial training corpora, performing data enhancement using a target algorithm set determined from multiple algorithm sets, and training a language model to obtain a sequence labeling model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If sequence labeling model is trained using limited labeled contract documents, then training cost and time are reduced, but model performance deteriorates
Solution Approach 1:
The patent applies data enhancement techniques to generate synthetic training samples by copying and transforming existing labeled contract documents. This includes techniques such as back-translation, random masking, and template-based generation to create new training pairs from limited original data, thereby increasing the effective training sample quantity without requiring additional manual labeling
Solution Approach 2:
The patent changes the parameters of the training data by applying various transformation algorithms to the initial training corpora. Different algorithm sets are used to perform data enhancement on corpora with different granularity levels, effectively transforming the existing data into diverse training samples that expand the training corpus while maintaining label accuracy
2Quantity of substance
If data enhancement is performed on initial training corpora to increase training data quantity, then model performance is improved, but data processing complexity increases
Solution Approach 1:
The patent segments the data enhancement process into multiple algorithm sets, each handling different granularity levels of the training corpora. This segmentation allows systematic application of enhancement techniques at different levels (e.g., sentence-level, paragraph-level transformations), making the complex data processing task more manageable and organized
Solution Approach 2:
The patent creates a universal data enhancement framework that can apply multiple algorithms to different granularity levels of the same corpora. This multi-functional approach allows a single processing pipeline to handle various transformation types (back-translation, masking, template generation) across different data granularities, reducing overall system complexity through consolidation
Data Source
AI summary
The present disclosure provides a model training method and apparatus, an electronic device, and a storage medium, and relates to the field of artificial intelligence, in particular, to the field of natural language processing and deep learning. A specific implementation solution includes: constructing initial training corpora; performing data enhancement on the initial training corpora based on an algorithm contained in a target algorithm set to obtain target training corpora, wherein the target algorithm set is determined from multiple algorithm sets, and different algorithm sets are used for performing data enhancement on corpora with different granularity in the initial training corpora; and performing training on a language model based on the target training corpora to obtain a sequence labeling model, herein the language model is pre-trained based on text corpora.