Prompt Structuring Model for Large Language Model Input Limits

Solution Overview

Problem

Large language models have input size limitations that restrict the amount of data that can be processed, leading to incomplete summaries and increased computational costs, and existing techniques fail to ensure crucial details are included in the summaries.

Innovation Solution

Implement a prompt structuring model using a token classification model and classification algorithm to update user inputs, reducing their size while preserving important context, and identify suitable large language models for analysis.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If large language models are used to review and summarize user sessions, then meaningful insights can be generated, but the input size limit restricts the amount of information that can be processed

Engineering Contradiction:

Improveinformation completenessVSAvoidinput size

Core Design Contradiction:

Loss of informationVSLength of moving object

Solution Approach 1:

The patent segments the user session data into multiple portions and processes them through multiple iterations. In each iteration, the model summarizes portions of the data and uses those summaries as input for the next iteration, progressively reducing the data size while maintaining information completeness through cumulative summarization.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies preliminary processing by using a classification algorithm to identify the nature or structure of the input data before summarization. This preliminary classification guides the summarization process to focus on contextually important information, ensuring that critical details are preserved while reducing overall input size.

Inventive Principle:

Principle #10Preliminary action

2Loss of information

If large amounts of information are input into large language models, then more comprehensive analysis can be performed, but computational costs increase significantly

Engineering Contradiction:

Improvedata coverageVSAvoidcomputational cost

Core Design Contradiction:

Loss of informationVSUse of energy by moving object

Solution Approach 1:

The patent applies preliminary processing by using a classification algorithm to identify the nature or structure of the input data before summarization. This preliminary classification guides the summarization process to focus on contextually important information, ensuring that critical details are preserved while reducing overall input size.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent extracts only the contextually important information from the user session data through iterative summarization. By repeatedly summarizing portions of the data and using those summaries as input, the system extracts and retains only the essential information needed for analysis, eliminating redundant data that would increase computational costs.

Inventive Principle:

Principle #2Taking out (Extraction)

3Length of moving object

If summaries are generated by asking large language models to summarize portions of data, then input size is reduced, but crucial parts of the data may be omitted

Engineering Contradiction:

Improveinput sizeVSAvoiddetail completeness

Core Design Contradiction:

Length of moving objectVSLoss of information

Solution Approach 1:

The patent applies preliminary processing by using a classification algorithm to identify the nature or structure of the input data before summarization. This preliminary classification guides the summarization process to focus on contextually important information, ensuring that critical details are preserved while reducing overall input size.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent implements feedback mechanisms where the model uses its own generated summaries as input for subsequent iterations. This feedback loop allows the model to refine its summarization process, progressively capturing more nuanced and contextually important details while maintaining reduced input size across multiple processing stages.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS20250307546A1Systems and methods for large language model optimization using prompt structuring

Publication Date: 2025.10.02 CYBER ARK SOFTWARE LTD

AI summary

Disclosed embodiments relate to updating an input for a large language model. Techniques include receiving the input from a user, applying a token classification model to the input to generate a replacement dictionary, applying a classification algorithm to the input to classify at least one of a nature or a structure of the input, updating, by a trained machine learning model, the input based on the replacement dictionary and the classified nature or structure of the input and transmitting the updated input to the at least one large language model.