Semi-autoregressive hierarchical text generation method based on discrete flow matching diffusion algorithm

By decomposing the text sequence into multiple large blocks for parallel generation, combining the hierarchical block structure and discrete stream matching method, the efficiency and quality trade-off problem of language models in long text generation is solved, and efficient and flexible text generation is achieved.

CN120706403APending Publication Date: 2025-09-26ZHEJIANG UNIV
0 Cites 0 Cited by

Patent Information

Application Number
CN202510745664.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-06-05
Publication Date
2025-09-26

AI Technical Summary

Technical Problem

Existing language models face a trade-off between efficiency and quality in long text generation. Autoregressive models have slow generation speed, diffusion models face challenges in fixed-length generation limitations and perplexity, and block diffusion methods cannot fully utilize semantic hierarchical relationships.

Method used

A semi-autoregressive hierarchical text generation method based on the discrete stream matching diffusion algorithm is adopted to decompose the text sequence into multiple large blocks, with autoregressive generation between blocks and parallel generation within blocks. Combined with the hierarchical block structure, the distribution between and within blocks is learned through the discrete stream matching method, a multi-level block tree structure is constructed, and the parallelism is adaptively adjusted.

Benefits of technology

It significantly improves the efficiency of long text generation and maintains low perplexity. It is suitable for various application scenarios that require efficient text generation. It supports flexible pre-generation strategies, can provide fast preview when computing resources are limited, and maintain high quality while generating in parallel.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention discloses a semi-autoregressive hierarchical text generation method based on a discrete flow matching diffusion algorithm, which comprises the following steps of: (1) constructing a hierarchical block diffusion language model, firstly dividing a text sequence into large blocks by the model, generating in an autoregressive manner among the blocks, and generating in parallel in the blocks in a discrete flow matching diffusion manner; (2) training the hierarchical block diffusion language model to enable the hierarchical block diffusion language model to learn conditional distribution between blocks and joint distribution in the blocks; (3) recursively decomposing each large block into smaller sub-blocks by adopting a hierarchical block structure to form a multi-layer block tree structure; and (4) in a reasoning stage, adopting a top-down generation mode, firstly generating an upper-layer large block, and then generating a lower-layer sub-block based on a generated content condition, thereby realizing efficient text generation. By means of the method and device, the text generation efficiency can be remarkably improved while the language modeling quality is kept.
Need to check novelty before this filing date? Find Prior Art