A text chunking method, apparatus, device and storage medium
By constructing a semantic tree of Markdown documents and applying multi-strategy adaptive chunking rules, semantic anchors are extracted and the strength of semantic associations between chunks is quantified. Cross-chunk semantic links and chunk relationship graphs are established, solving the problem of lost semantic associations between chunks in existing technologies. This improves the information integration capabilities of the RAG system and generates more coherent and accurate answers.
Patent Information
- Application Number
- CN202610674578.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-15
- Publication Date
- 2026-07-17
AI Technical Summary
Existing technologies ignore the semantic structure information of Markdown documents during text segmentation, resulting in the truncation of important semantic units, the merging of irrelevant content, a mismatch between the segmentation granularity and downstream vector retrieval tasks, and the complete severing of semantic connections between blocks. This leads to fragmented content retrieved by the RAG system and a lack of coherence in answer generation.
By constructing a semantic tree of a Markdown document, applying multi-strategy adaptive block division rules, extracting semantic anchors and calculating the semantic association strength between blocks, establishing cross-block semantic links and block relationship graphs, and forming an interconnected semantic relationship network.
It improves the coherence and completeness of the generated content, enabling the RAG system to effectively recall context related to the semantics of the matching block and generate more accurate and logical answers.
Smart Images
Figure CN122412552A_ABST