Dynamic Prompt construction and structured output control method based on task type

Abstract

The invention provides a dynamic Prompt construction and structured output control method based on task types, and relates to the technical field of natural language processing and large model application. According to the method, a task type queried by a user is identified through a task classifier, a corresponding template is matched from a Prompt template library, an output format specification is determined through a structured Schema mapper, context and user problems are injected through a Prompt dynamic assembly engine to generate final Prompt, and after a large model is called, it is ensured that output meets the Schema requirement through an output verification and rollback mechanism. The problems that an existing RAG system is disordered in output format, uncontrollable, high in manual intervention cost and poor in expansibility are solved, the technical effects of output consistency improvement, post-processing cost reduction, interpretability enhancement and rapid expansion are achieved, and the method is suitable for multi-scene intelligent question and answer scenes such as recipes, device manuals and FAQ.

Description

Technical Field

[0001] This invention relates to the field of natural language processing and large model application technology, specifically to a method and system for dynamic prompt construction and structured output control based on task type. Background Technology

[0002] Current mainstream RAG (Retrieval Augmentation) systems typically use fixed templates to append the retrieved context to the user's question and then directly input it into a large model to generate the answer. For example, if the user's question is "How do I replace a cooking knife?" and the search result is a product instruction manual excerpt, the Prompt structure would be "Answer the question based on the following information:\n{Search Content}\n\nQuestion: {User Question}".

[0003] However, existing technologies have the following drawbacks: First, they lack task awareness and cannot distinguish between different task types such as "operation steps," "parameter queries," and "troubleshooting," resulting in chaotic output formats. Second, the output is uncontrollable, with large models freely generating text that is difficult to use directly in downstream systems (such as front-end cards, database entries, and API returns). Third, manual intervention is costly, requiring extensive post-processing code to parse unstructured answers, which reduces system stability. Fourth, the user experience is poor, with the same question potentially returning inconsistent formats from different document sources, and the system lacks scalability, requiring significant modifications to the core logic when adding new task types. Summary of the Invention

[0004] This invention aims to solve the technical problems of existing RAG systems, such as chaotic and uncontrollable output formats, high costs of manual intervention, and poor scalability, and provides a method and system that can automatically identify task types, dynamically construct prompts, and control structured output.

[0005] This invention provides a method for dynamic prompt construction and structured output control based on task type, the specific steps of which are as follows: Step 1: Task Type Identification: After receiving the user's original question, the task type is determined by a task classifier. The task classifier can use keyword rule matching (e.g., identifying keywords such as "how to solve" or "fault" as troubleshooting tasks), a classification model fine-tuned based on small models such as BERT, or utilize the discriminative capabilities of LLM to output task type labels. The task type labels include, but are not limited to, "troubleshooting", "spec_query", and "recipe".

[0006] Step 2: Prompt Template Retrieval: The Prompt template library predefines Prompt templates that correspond one-to-one with each task type tag. These templates include prompts adapted to the task scenario and JSON Schema placeholders. Based on the task type tag identified in Step 1, the corresponding Prompt template is retrieved from the template library.

[0007] Step 3: JSON Schema Matching: The structured schema mapper stores the binding relationships between task type tags and JSON schemas. These JSON schemas conform to the JSON Schema specification and explicitly define the field types, attributes, and required fields of the output data. For example, the JSON schema corresponding to a troubleshooting task is: json {"type": "object","properties": {"steps": { "type": "array", "items":{ "type": "string"}},"possible_causes": { "type": "array", "items": { "type": "string"}},"safety_warning": { "type": "string"}},"required": ["steps"]} Based on the task type label, retrieve the corresponding JSON Schema from the structured schema mapper.

[0008] Step 4: Prompt Dynamic Assembly and Large Model Invocation: The Prompt dynamic assembly engine receives the retrieved context, the user's original query, and the JSON Schema obtained in Step 3. It injects these three into the corresponding placeholders in the Prompt template called in Step 2 to generate a final Prompt with a standardized format and complete information. This final Prompt is then input into the target large model (such as Llama3, Qwen, etc.) for text generation.

[0009] Step 5: Output Validation and Rollback Processing: The output validation and rollback module validates the generated results of the large model to determine whether they strictly conform to the field structure requirements defined in the JSON Schema. If they conform, the structured result is output directly; if they do not, a retry operation is triggered (the number of retries can be pre-configured to a maximum value N), and the large model is called again to generate the result; if the results still do not conform after N retries, the system switches to fault-tolerant mode, extracting structured data from the generated results using regular expressions and other methods to ensure that the final output can be directly parsed by downstream programs.

[0010] Correspondingly, the present invention also provides a dynamic Prompt construction and structured output control system based on task type, including a task classifier, a Prompt template library, a structured schema mapper, a dynamic Prompt assembly engine, and an output verification and rollback module, the functions of each module corresponding one-to-one with the above method steps. Beneficial effects

[0011] This invention achieves the following beneficial effects through a task-type-driven dynamic Prompt construction and structured output control mechanism: Improve output consistency: Always return structured data with the same field structure for the same task type to avoid format differences caused by different document sources; Reduce post-processing costs: The output data conforms to the preset JSON Schema, and can be directly connected to downstream scenarios such as front-end display, database insertion and API response without complex NLP parsing; Enhanced interpretability: JSON Schema clearly defines the meaning and format requirements of each output field, improving the understandability and credibility of the results; Supports rapid expansion: When adding a new task type, simply add the corresponding template to the Prompt template library and bind the relevant JSON Schema in the structured Schema mapper. No need to modify the core system logic, adapting to multiple scenario expansion needs. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the steps of the present invention. Detailed Implementation

[0013] The present invention will be further described in detail below with reference to specific embodiments.

[0014] Example 1: Troubleshooting Tasks The user's original question was "My cooking machine can't print, how do I fix it?", and the retrieved context was a relevant excerpt from the cooking machine's product manual regarding cooking machine malfunctions.

[0015] The task classifier identifies keywords such as "cannot cook" and "how to solve" and outputs the task type label "troubleshooting". Calling a troubleshooting template from the Prompt template library: "You are a professional repair assistant. Please follow the steps in the following document to explain how to resolve user issues. Document content: {{context}} User issue: {{query}} Please strictly adhere to the following JSON format for output: {{schema}}"; The structured schema mapper returns the JSON schema corresponding to the troubleshooting class; json {"type": "object","properties": {"steps": { "type": "array", "items":{ "type": "string"}},"possible_causes": { "type": "array", "items": { "type": "string"}},"safety_warning": { "type": "string"}},"required": ["steps"]} The Prompt dynamic assembly engine injects the retrieved instruction manual fragments, user questions, and the aforementioned JSON Schema into the template, generates the final Prompt, and calls the Llama3 model; The output validation and rollback module validates the model output. If the output is JSON data that conforms to the schema (containing the steps, possible_causes, and safety_warning fields), it is output directly. If the output does not conform, it triggers two retries. If it still does not conform, it extracts the required fields such as steps using regular expressions and generates a structured result.

[0016] Example 2: Recipe-based task processing The user's original question was "How to make scrambled eggs with tomatoes?", and the retrieved context was documents related to the ingredients and steps for making scrambled eggs with tomatoes.

[0017] The task classifier identifies the keyword "how to make" and the corresponding recipe scenario for "scrambled eggs with tomatoes", and outputs the task type label "recipe". The recipe class Prompt template is invoked: "You are a culinary expert. Please describe in detail how to prepare the dish based on the following document. Document content: {{context}} User question: {{query}} Please strictly follow the following JSON format for output: {{schema}}"; The structured schema mapper returns a recipe class JSON schema: { "$schema": "http: / / json-schema.org / draft-07 / schema#", "type": "object", "properties": { "id": { "type": "string"}, "nameList": { "type": "array", "items": { "type": "object", "properties": { "name": { "type": "string"}, "language": { "type": "string"}}, "required": ["name", "language"]}}, "serviceSize": { "type": "integer"}, "recipeIntroduceList":{ "type": "array", "items": { "type": "object", "properties": { "recipeIntroduce": { "type": "string"}, "language": { "type": "string"}}, "required": ["recipeIntroduce", "language"]}}, "customFoodList": { "type": "array", "items": { "type": "object", "properties": { "name": { "type": "string"}, "code": { "type": "string"}}, "required": ["name", "code"]}},"recipeStepVoList": { "type": "array", "items": { "type": "object", "properties": { "serialNumb": { "type": "integer"}, "stepDescList": { "type":"array", "items": { "type": "object", "properties": { "stepDesc": { "type": "string"}, "language": { "type": "string"}},"required": ["stepDesc", "language"]}}}, "required": ["serialNumb", "stepDescList"]}}, "kitRecipeNutrientsVoList": { "type": "array", "items": { "type": "object", "properties": { "nutrients": { "type": "string"}, "name": { "type": "string"}, "num": { "type": "number"}, "unit": { "type": "string"}, "selected": { "type": "integer"}}, "required": ["nutrients", "name", "num", "unit", "selected"]}}, "isCustomFood": { "type": "integer"}, "deviceModelIds": { "type": "string"}, "productType": { "type": "integer"}}, "required": [ "id", "nameList", "serviceSize", "customFoodList", "recipeStepVoList", "kitRecipeNutrientsVoList" ]}, Assemble the Prompt and call the Qwen model to generate the results; Once the verification results conform to the schema, they are output to ensure that downstream systems can directly extract information such as ingredients and steps for display.

[0018] This invention is not limited to the above embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the protection scope of this invention.

Claims

1. A method for dynamic prompt construction and structured output control based on task type, characterized in that... This includes the following steps: S1. Receive the user's original question and identify the task type label corresponding to the user's original question through a task classifier. The task classifier is implemented using keyword rules, a fine-tuned small model, or an LLM self-discrimination method. S2. Based on the task type tag, call the corresponding Prompt template from the predefined Prompt template library. The Prompt template library stores templates corresponding to various task types. S3. Based on the task type label, obtain the bound JSONS schema through the structured schema mapper; S4. Using the Prompt dynamic assembly engine, inject the retrieved context, the user's original question, and the JSON Schema into the called Prompt template to generate the final Prompt and call the target large model; S5. Verify the output results of the large model through output validation and rollback mechanisms. If the output does not conform to the JSONSchema, trigger a retry operation or switch to fault-tolerant mode until a structured output that meets the requirements is obtained.

2. A method for dynamic prompt construction and structured output control based on task type according to claim 1, characterized in that... The task type tags include at least the troubleshooting category "troubleshooting", the parameter query category "spec_query", and the recipe category "recipe", and support adding and extending them.

3. A method for dynamic prompt construction and structured output control based on task type according to claim 1, characterized in that... The Prompt template includes pre-defined prompts and JSON Schema placeholders for task adaptation, with the prompts matching the application scenarios of the corresponding task types.

4. A method for dynamic prompt construction and structured output control based on task type according to claim 1, characterized in that... The maximum number of retry operations is configurable, and the fault tolerance mode uses regular expression extraction to extract structured data that meets the requirements from the output of the large model.

5. A dynamic prompt construction and structured output control system based on task type, characterized in that, include: Task classifier: Used to receive original user questions and identify corresponding task type labels, implemented using keyword rules, fine-tuned small models, or LLM self-discrimination methods; Prompt Template Library: Used to store predefined Prompt templates corresponding to various task types, and supports template calling based on task type tags; Structured Schema Mapper: Used to store the binding relationship between task type tags and JSON Schema; The Prompt dynamic assembly engine is used to inject the retrieved context, the user's original question, and the JSON Schema into the Prompt template, generate the final Prompt, and call the target large model. Output validation and rollback module: Used to verify whether the output results of large models conform to the JSON Schema. If they do not conform, a retry or switching to fault-tolerant mode will be triggered.

6. A task-type-based dynamic prompt construction and structured output control system according to claim 5, characterized in that, The JSON Schema in the structured Schema mapper includes field types, required fields, and field meaning definitions. Different task types have different field structures in their corresponding JSON Schemas.

7. A task-type-based dynamic prompt construction and structured output control system according to claim 5, characterized in that, The Prompt template library supports adding new task templates.

Images