Text Generation Model Phrase Granularity Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Word-based text generation models in deep learning are prone to local semantic shift, resulting in low accuracy and poor quality of generated text due to their reliance on word dependencies.
Innovation Solution
A method for training a text generation model by extracting structured information from sample data, generating predicted structured information, and calculating a loss value based on differences with target structured information to improve the phrase generation ability of the model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If word-based text generation model is used, then text can be generated based on word dependencies, but local semantic shift occurs resulting in low accuracy and poor quality
Solution Approach 1:
The patent segments text into phrases instead of processing word-by-word. The text generation model processes phrases as complete units, preserving the semantic relationships within each phrase while reducing the accumulation of local semantic shifts that occur with word-level processing.
Solution Approach 2:
The patent introduces structured information extraction as an intermediary step between input text and model processing. By extracting structured information from phrases and using it to guide generation, the model maintains better semantic fidelity and reduces information loss during the transformation process.
2Ease of manufacture
If word-based generation approach is used, then processing is straightforward, but the generated text quality deteriorates due to semantic shift
Solution Approach 1:
The patent divides text processing into phrase-level segments rather than word-level segments. This segmentation maintains relative processing simplicity while significantly improving generation quality by preserving phrase-level semantic coherence and reducing the compounding effect of semantic drift.
Solution Approach 2:
The patent changes the fundamental parameter of text processing from word-level to phrase-level. This parameter change transforms the granularity of processing units, enabling the model to capture broader semantic contexts while maintaining computational feasibility.
3Measurement precision
If phrase generation ability is trained, then text generation accuracy improves, but model training complexity increases
Solution Approach 1:
The patent uses structured information extraction as an intermediary training mechanism. By extracting structured information from phrases and using it as supervision signals, the training process becomes more targeted and efficient, improving accuracy without proportionally increasing complexity.
Solution Approach 2:
The patent performs preliminary structured information extraction from training data before model training. This preliminary action prepares the training data in a structured format that guides the phrase generation process, making the subsequent training more efficient and directed.
Data Source
AI summary
The disclosure may provide a method for obtaining a document layout, an electronic device, and a storage medium. The method may include: obtaining a plurality of pieces of first sample data; extracting structured information from each of the plurality of pieces of first sample data as target structured information corresponding to each of the plurality of pieces of first sample data; inputting the plurality of pieces of first sample data into an initial text generation model to generate predicted structured information corresponding to each of the plurality of pieces of first sample data; generating a first loss value based on a difference between the predicted structured information corresponding to each of the plurality of pieces of first sample data and the corresponding target structured information; and training a phrase generation ability of the initial text generation model based on the first loss value to generate the text generation model.