Interactive Composition Generation with Explainable Attribute Controls
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for automated composition generation lack interactivity and provide limited control and insight into the composition process.
Innovation Solution
A computer-implemented method that generates a new composition from a natural language description by determining cluster summaries similar to the description, using a large language model (LLM) to convert the description into composition attribute controls, and interleaving control events with generated events to produce the composition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If automated composition generation systems use traditional algorithms, then composition generation can be achieved, but user creative control and interactivity are limited
Solution Approach 1:
The system segments the composition generation process into distinct controllable components: cluster summary selection, attribute control adjustment, and composition generation. Users can interact with and adjust each segment independently, providing fine-grained creative control without overwhelming system complexity.
Solution Approach 2:
The system implements dynamic interactivity where users can provide feedback during the composition generation process and adjust parameters in real-time. The system adapts to user inputs dynamically, allowing continuous refinement of composition attributes while maintaining manageable complexity through iterative refinement.
2Loss of information
If the system provides detailed control over composition attributes, then user insight and control improve, but the complexity of the generation process increases
Solution Approach 1:
The system provides feedback mechanisms that display cluster summaries and composition attributes to users during the generation process. This feedback loop gives users insight into what the system is doing and why, reducing information loss while managing complexity through transparent, interpretable intermediate representations.
Solution Approach 2:
The system introduces intermediary elements such as cluster summaries and attribute controls that serve as mediators between user intent and the complex generation process. These intermediaries provide users with actionable levers to control the process without requiring them to understand or manage the full complexity of the underlying generation algorithms.
3Ease of operation
If the system generates compositions with high interactivity, then user control improves, but processing time and computational resources increase
Solution Approach 1:
The system performs preliminary actions by pre-computing and storing cluster summaries from the training data before the actual composition generation. This pre-processing creates ready-to-use intermediaries that can be quickly selected and combined during interactive generation, reducing real-time computational requirements while maintaining high interactivity.
Data Source
AI summary
A computer-implemented method includes determining a set of s cluster summaries that are most similar to a natural language description of a new composition and generating the new composition based on s tokens corresponding to the set of s cluster summaries. Each cluster summary of the s cluster summaries corresponds to a cluster of similar compositions. The method may also include converting the natural language description to a set of composition attribute controls using an LLM and generating the new composition based on the set of composition attribute controls. The LLM may specify the set of composition attribute controls according to a JSON interface specification. Generating the new composition may include interleaving control events with generated events. The new composition may include one or more of music, art, graphical art, imagery, photography, video, prose, poetry, writing and literature. A corresponding system and computer program product are also disclosed herein.