LLM Report Generation With Iterative Prompt Quality Feedback

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing technologies fail to effectively facilitate interagency coordination and information sharing among government entities and non-governmental organizations for public safety programming, lacking the ability to generate coherent and contextually complete reports using AI systems.

Innovation Solution

A method and system utilizing large language models to automate the generation of textual reports by prompting and refining user inputs, ensuring contextual completeness and quality through iterative feedback loops and fine-tuned models, enabling generation of reports tailored to specific intents and audiences.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If AI systems are used to generate reports, then productivity and report generation speed are improved, but the quality and contextual completeness of reports deteriorates

Engineering Contradiction:

Improvereport generation speedVSAvoidreport quality and contextual completeness

Core Design Contradiction:

ProductivityVSManufacturing precision

Solution Approach 1:

The system implements a feedback mechanism where the quality rating model evaluates generated reports and provides feedback to the language model. This feedback loop allows the system to iteratively improve report quality while maintaining high generation speed, resolving the contradiction between productivity and precision.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system performs preliminary actions by using a quality rating model to evaluate and rate report descriptions before final report generation. This preliminary evaluation ensures contextual completeness is achieved early in the process, preventing quality deterioration while maintaining efficient generation.

Inventive Principle:

Principle #10Preliminary action

2Stability of the object's composition

If existing management structures are used, then organizational stability is maintained, but the ability to engage in interagency coordination and information sharing deteriorates

Engineering Contradiction:

Improveorganizational stabilityVSAvoidinteragency coordination capability

Core Design Contradiction:

Stability of the object's compositionVSAdaptability or versatility

Solution Approach 1:

The AI system acts as an intermediary between different government agencies and NGOs. It facilitates interagency coordination and information sharing while maintaining the stability of existing management structures, enabling new collaborative capabilities without disrupting established organizational frameworks.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system provides universal functionality across multiple agencies and organizations, enabling a single platform to support diverse interagency coordination needs. This multi-functional approach enhances adaptability while maintaining organizational stability through a unified system architecture.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS20250315909A1Methods for automating the prompting and post-processing of ai systems for interpretation and reporting of public safety data, statistics, and contextual knowledge

Publication Date: 2025.10.09 SIMSI INC

AI summary

A method of textual report generation, comprising: receiving a proposed report description and a report intent; providing the proposed report description and the report intent to a first large language model; prompting the first large language model to output a quality rating of the proposed report description with respect to the report intent; prompting the user to revise the proposed report description until the quality rating exceeds a predetermined threshold; providing the proposed report description and the report intent to a second large language model; and prompting the second large language model to generate a report according to the proposed report description and the report intent.