Two-Level Semantic Model for Text Vector Representation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing natural language processing models, such as BERT, face limitations in representing long texts and fail to capture information across different levels like paragraphs and sentences, leading to information loss and separation of text semantics.
Innovation Solution
A method involving a two-level semantic model approach, where text is divided into segments, with a first-level model generating segment vectors indicating text segment semantics and a second-level model recognizing these vectors to produce a text vector representing the topic, thereby capturing paragraph-level and text-level information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single-level semantic model is used to represent text, then the model complexity is low, but the text information completeness deteriorates and semantic separation occurs
Solution Approach 1:
The patent divides text representation into two hierarchical levels: segment-level vectors representing local text segments (sentences or paragraphs) and document-level vectors representing the overall text. This segmentation allows different levels of semantic information to be captured separately and then integrated, preventing information loss and semantic separation while maintaining manageable model complexity through modular architecture.
2Ease of operation
If text is processed as a whole by a single model, then the processing simplicity is high, but the representation accuracy for different text levels deteriorates
Solution Approach 1:
The patent introduces a hierarchical dimension to text representation, organizing vectors at two levels: segment-level vectors for local semantics and document-level vectors for global semantics. This dimensional organization allows the model to simultaneously maintain processing simplicity through automated hierarchical generation while achieving high representation accuracy by capturing both local and global semantic features that a single-level model would miss.
Data Source
AI summary
A method for generating a vector representation of a text includes dividing the text into text segments. Each text segment is represented as a segment vector corresponding to the respective text segment by employing a first-level semantic model. The segment vector is configured to indicate a semantics of the text segment. Text semantics recognition is performed on the segment vector of each text segment by employing a second-level semantic model to obtain a text vector for indicating a topic of the text.