Autoregressive Poetry Generation with Attention-Based Rule Adherence

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current methods for generating poems or ci-poems lack the ability to consistently adhere to traditional poetic rules such as rhymes, tonal patterns, and antithesis forms, resulting in inconsistent and less artistic outputs.

Innovation Solution

An autoregressive language model is trained on a poetry corpus to learn and generate poems or ci-poems that follow specific poetic rules, using a combination of self-attention and neural network modules to predict characters or words based on attention levels, ensuring consistency and adherence to traditional forms.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional poetry generation methods are used, then poetic rules such as rhymes and tonal patterns can be followed, but the consistency and artistic quality of generated poems deteriorate

Engineering Contradiction:
Improveadherence to poetic rulesVSAvoidconsistency of generated poetry
Core Design Contradiction:
ReliabilityVSManufacturing precision

Solution Approach 1:

The patent transforms discrete poetic rules (rhymes, tonal patterns, antithesis forms) into continuous parameter spaces that can be optimized by neural networks. The language model learns to generate poetry by adjusting parameters such as character probabilities, word sequences, and structural constraints, enabling consistent adherence to poetic rules while maintaining artistic quality across multiple generated poems.

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The patent replaces traditional mechanical approaches to poetry generation (rule-based systems, template matching) with a neural network-based language model. This substitution enables the system to learn complex poetic patterns from training data and generate consistent, high-quality poetry that adheres to traditional rules without requiring explicit programming of each rule.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

2Productivity

If simple generation methods are used, then the process is fast and simple, but the quality and coherence of generated poems deteriorate

Engineering Contradiction:
Improvegeneration speedVSAvoidcoherence of generated poetry
Core Design Contradiction:
ProductivityVSManufacturing precision

Solution Approach 1:

The patent segments the poetry generation process into multiple sequential steps performed by different components of the language model: token prediction, sequence generation, and poetic rule validation. This segmentation allows each component to specialize in specific aspects of poetry generation, maintaining high coherence while enabling efficient parallel processing and fast generation speeds.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent implements continuous refinement of generated poetry through iterative prediction and validation cycles. The language model continuously generates and refines poem sequences, maintaining coherence throughout the generation process by constantly evaluating and adjusting predictions against learned poetic patterns, thereby achieving both high quality and efficient generation.

Inventive Principle:
Principle #20Continuity of useful action

Data Source

PatentUS20230267282A1Poetry generation
Publication Date: 2023.08.24 BEIJING SOGOU TECHNOLOGY DEVELOPMENT CO LTD
  • US20230267282A1 patent drawing
  • US20230267282A1 patent drawing
  • US20230267282A1 patent drawing

AI summary

A method for poetry generation includes receiving generation information indicative of a theme for the poetry generation, and determining at least a candidate piece of poetry corresponding to the generation information according to an autoregressive language model that is configured to generate elements in the candidate piece of poetry in an autoregressive manner with a plurality of regression rounds. An element in the elements is a character or a word. The autoregressive language model is configured for generating poetry in a plurality of formats, the autoregressive language model includes a plurality of processing layers connected sequentially, a processing layer in the plurality of processing layers is configured to determine attention levels for potential elements in a potential element list according to generated elements prior to a current regression round, and predict one or more additional element for the current regression round using a neural network according to the attention levels.