Large-model intelligent workflow interaction system and method based on graph process cues

Abstract

The invention relates to a large-model intelligent workflow interaction system and method based on graph process cues. The method comprises the steps of obtaining input information of a current conversation of a user; inputting a preset graph flow cue word template and the input information into a large model, wherein the graph flow cue word template at least comprises a visual workflow execution path graph composed of a plurality of nodes and attribute information of each node; the large model is executed from an initial node according to the visual workflow execution path diagram; in the execution process, an intermediate result is generated according to attribute information corresponding to the node; generating intermediate input information according to the intermediate result and inputting the intermediate input information into a subsequent node; and when the large model is executed to a termination node, generating a processing result. The large model is guided to execute and the answer search space is reduced, so that the output effect of the large model can be improved, and the engineering application and later maintenance cost can be reduced.

Description

Technical Field

[0001] The present application relates to the field of computer information processing, and in particular to a large-scale intelligent workflow interaction system and method based on graph process prompt words. Background Art

[0002] With the development of big model technology, enterprise-level big model applications are becoming increasingly popular. These applications generally involve complex process tasks, requiring big models to understand multi-step logic and complete tasks through a combination of planning, reasoning, and tool invocation. In data query scenarios, big models must determine user intent, identify query metadata, verify permissions, and invoke knowledge base tools.

[0003] Currently, the mainstream approach in the industry is to use Chain-Of-Thought (COT) prompt word engineering to address the above needs. Its core logic is to reduce the effective prompt word space (promptspace) of the large model through manually written prompt word instructions, and then guide the large model to follow the instructions and gradually complete the task with autoregressive conditional probability.

[0004] In existing COT technology, COT prompts for complex processes rely primarily on manual coding, lacking universal templates. Writing varies from person to person, leading to inconsistent guidance logic for large models and poor cross-task reusability. During the actual execution of large models, guidance solely through natural language text fails to accurately define the execution path for multiple branches and multiple tool calls, making it easy for large models to deviate from the process and ensuring the quality of task completion. The relationship between output results and prompts is fuzzy, making it impossible to locate the source of problems. Tuning relies on subjective trial and error, resulting in low efficiency and unstable results. Therefore, the current COT approach can only achieve basic guidance for simple process tasks. In complex enterprise-level process scenarios, this approach has significant technical flaws.

[0005] Therefore, a new large-model intelligent workflow interaction system and method based on graph process prompt words is needed.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the application and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0007] In view of this, the present application provides a large-model intelligent workflow interaction system and method based on graph process prompt words, which can accurately guide the execution of the large model and narrow the answer search space, improve the output effect of the large model, and reduce engineering application and subsequent maintenance costs.

[0008] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by practice of the present application.

[0009] According to one aspect of the present application, a large-model intelligent workflow interaction method based on graph flow prompt words is proposed, the method comprising: obtaining input information of the user's current conversation, the input information including user input content and context information; inputting a preset graph flow prompt word template and the input information into the large model, the graph flow prompt word template including at least a visual workflow execution path diagram consisting of multiple nodes and attribute information of each node; the large model starts execution from the initial node according to the visual workflow execution path diagram: during the execution process, an intermediate result is generated according to the attribute information corresponding to the node; intermediate input information is generated according to the intermediate result and input into subsequent nodes; when the large model executes to the termination node, a processing result is generated; the processing result is output and post-processed and fed back to the user.

[0010] According to one aspect of the present application, a large-model intelligent workflow interaction system based on graph flow prompt words is proposed, which includes: an information module for obtaining input information of the user's current conversation, wherein the input information includes user input content and context information; a graph flow calling module for inputting a preset graph flow prompt word template and the input information into the large model, wherein the graph flow prompt word template includes at least a visual workflow execution path diagram composed of multiple nodes and attribute information of each node; a workflow execution module for enabling the large model to start execution from the initial node according to the visual workflow execution path diagram: during the execution process, an intermediate result is generated according to the attribute information corresponding to the node; intermediate input information is generated according to the intermediate result and input into subsequent nodes; an output module for generating a processing result to feedback to the user when the large model executes to the termination node.

[0011] Optionally, it also includes: an initial template module, used to generate an initial graph flow prompt word template based on the target task; a test question module, used to generate a test question based on the target task; a recording module, used to input the test question into the large model, and record the intermediate results and the actual visual workflow execution path diagram during the execution process; a template iteration module, used to iterate the initial graph flow prompt word template according to the intermediate results and the actual visual workflow execution path diagram to generate the graph flow prompt word template.

[0012] Optionally, the initial template module includes: a background data unit for generating task background data based on the target task; an information variable unit for generating macro auxiliary information and global variables based on the target task; an attribute unit for generating an initial visual workflow execution path diagram and attribute information of each node in the diagram; and a format unit for generating an output format definition.

[0013] Optionally, the attribute unit is further configured to define a connection relationship between nodes through pseudocode to generate the initial visual workflow execution path diagram, wherein the connection relationship between nodes includes conditional branches and loop jump logic.

[0014] Optionally, the template iteration module includes: a result comparison unit for comparing the intermediate result with the expected intermediate result; a path comparison unit for comparing the actual visual workflow execution path diagram with the initial visual workflow execution path diagram in the initial graph flow prompt word template; and an iteration unit for iterating the initial graph flow prompt word template according to the comparison result to generate the graph flow prompt word template.

[0015] Optionally, the information module includes: a content unit for obtaining the input content of the user's current conversation; a context unit for obtaining context information of the user's current conversation by a context manager; and an information unit for generating the input information through the input content and the context information.

[0016] Optionally, the workflow execution module includes: a judgment unit, which is used to jump to the corresponding next node according to the judgment result when the large model executes to a node containing a judgment function according to the visual workflow execution path diagram; and / or a tool unit, which is used to call and execute the corresponding external tool to generate the intermediate result when the large model executes to a node containing a tool call according to the visual workflow execution path diagram.

[0017] Optionally, the judgment unit is also used to obtain the judgment rules preset in the attribute information of the node when the large model executes to the node containing the judgment function according to the visual workflow execution path diagram; generate a judgment result based on the input information and the judgment rule; and jump to the corresponding next node based on the judgment result.

[0018] Optionally, the workflow execution module further includes: a management unit, configured to input the intermediate result into a context manager to generate the intermediate input information.

[0019] Optionally, the output module includes: a post-processing unit, configured to perform output post-processing on the processing result according to the output format definition in the graph flow prompt word template and feed back the output to the user.

[0020] According to one aspect of the present application, an electronic device is proposed, which includes: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0021] According to one aspect of the present application, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described above is implemented.

[0022] According to one aspect of the present application, a computer program product is provided, comprising: a computer program / instruction, wherein the computer program / instruction implements the method described above when executed by a processor.

[0023] According to the large-model intelligent workflow interaction system and method based on graph flow prompt words of the present application, by obtaining the input information of the user's current conversation, the input information includes user input content and context information; the preset graph flow prompt word template and the input information are input into the large model, and the graph flow prompt word template at least includes a visual workflow execution path diagram composed of multiple nodes and attribute information of each node; the large model starts execution from the initial node according to the visual workflow execution path diagram: during the execution process, an intermediate result is generated according to the attribute information corresponding to the node; intermediate input information is generated according to the intermediate result and input into the subsequent node; when the large model executes to the termination node, a processing result is generated; the processing result is output and post-processed and fed back to the user, which can accurately guide the execution of the large model and narrow the answer search space, improve the output effect of the large model, and reduce the engineering application and subsequent maintenance costs.

[0024] It should be understood that the foregoing general description and the following detailed description are merely illustrative and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and other objects, features, and advantages of the present application will become more apparent by describing in detail exemplary embodiments thereof with reference to the accompanying drawings. The drawings described below are merely some embodiments of the present application, and it is apparent to those skilled in the art that other drawings can be derived from these drawings without inventive effort.

[0026] Figure 1 The present invention is a flowchart of a large-model intelligent workflow interaction method based on graph process prompt words according to an exemplary embodiment.

[0027] Figure 2 It is a schematic diagram of a large-model intelligent workflow interaction method based on graph process prompt words according to an exemplary embodiment.

[0028] Figure 3 It is a flowchart of a large-model intelligent workflow interaction method based on graph process prompt words according to another exemplary embodiment.

[0029] Figure 4It is a schematic diagram of a large-model intelligent workflow interaction method based on graph process prompt words according to another exemplary embodiment.

[0030] Figure 5 It is a block diagram of a large-model intelligent workflow interaction system based on graph process prompt words according to an exemplary embodiment.

[0031] Figure 6 It is a block diagram of a large-model intelligent workflow interaction system based on graph process prompt words according to another exemplary embodiment.

[0032] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0033] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. Like reference numerals in the drawings represent like or similar parts, and thus repetitive description thereof will be omitted.

[0034] Figure 1 The flowchart of a large-scale intelligent workflow interaction method based on graph process prompt words according to an exemplary embodiment is shown. The large-scale intelligent workflow interaction method 10 based on graph process prompt words includes at least steps S102 to S110.

[0035] like Figure 1 As shown, in S102, input information of the user's current conversation is obtained, and the input information includes user input content and context information. For example, the input content of the user's current conversation is obtained; the context information of the user's current conversation is obtained by the context manager; and the input information is generated using the input content and the context information. In this application, the context manager is used to maintain historical information during the conversation process.

[0036] like Figure 2 As shown, in specific application scenarios, user input can be obtained from a variety of channels, such as text input from the user in the interactive interface, voice input, structured command input, etc. The input can also be preprocessed, including format cleaning, removing invalid characters from the input, correcting grammatical errors, and converting voice input into text. The user input is then fed into the context manager to generate input information.

[0037] In this application, the session management module maintains contextual information about user interactions with the large model. Using distributed session storage, the module isolates the conversation histories of different users, ensuring that only the latest fragment of the current session is loaded into the model context window for each message. When a user initiates a new request, the system only loads the relevant history, preventing cross-contamination of context.

[0038] In actual application scenarios, user input is first entered into the context manager and combined with the context information corresponding to the current conversation to form the input information. The context manager is responsible for rewriting, summarizing, and summarizing the user's current input and historical information as necessary. It also determines whether the user's current input topic is consistent with the historical conversation, thereby ensuring the consistency of the output topic.

[0039] In S104, the preset graph flow prompt word template and the input information are input into the large model, wherein the graph flow prompt word template at least includes a visual workflow execution path diagram consisting of multiple nodes and attribute information of each node. The generation process of the graph flow prompt word template will be in Figure 3 The corresponding embodiments are described in detail.

[0040] Different diagram flow prompt word templates may be selected based on the user's input information, and different diagram flow prompt word templates may also be selected based on the user's usage scenario or template task, but this application is not limited thereto.

[0041] In actual application, graphical symbols can be combined with branching logic to display nodes and the relationships between nodes. For example, in a data query intent recognition scenario, the path diagram contains multiple nodes such as A (the user's current question and multiple rounds of historical conversations), B (whether to end the conversation?), and D (is it a data query and data analysis question?). The nodes are connected by "conditional branch lines." The attribute information of each node includes the node name, node description, node input, node output, available tools, instructions to be followed, etc. For example: Node N (Entity Identification SQL Metadata Information): The node description is "Identify SQL metadata in user queries." The node input is "current user input + historical conversations." The node output is "time interval, metric name, dimension name." No tools are required. The instructions to follow are "Do not arbitrarily rewrite the metric name and dimension name proposed by the user. Verify the existence of the metadata later using KB_INFO." Node O (calling rag_search to obtain KB_INFO): The node description is "calling a tool to obtain SQL metadata information in the knowledge base." The node input is "metadata identified by node N." The node output is "KB_INFO (format: List[Tuple[candidate metadata name, belonging dataset name, type]])." The available tool is rag_search, and the instructions to be followed are "calling the rag_search tool according to the preset parameter format."

[0042] like Figure 2 As shown, system developers combine global variables, tool definitions, etc. to generate a diagram flow prompt word template. The preset diagram flow prompt word template can be used as the system prompt of the large model, and the user's current conversation input information can be used as the user prompt and input into the large model together.

[0043] In S106, the large model starts to execute from the initial node according to the visual workflow execution path diagram, and generates intermediate results according to the attribute information corresponding to the node during the execution process. Figure 2 As shown, during the execution of the large model, a tool call node or a judgment node may be encountered. When encountering a tool call node, the execution tool is called, and when encountering a judgment node, a judgment is made directly.

[0044] In one embodiment, for example, when the large model executes to a node containing a judgment function, it jumps to the corresponding next node according to the judgment result; in another embodiment, for example, when the large model executes to a node containing a tool call, it calls and executes the corresponding external tool to generate the intermediate result.

[0045] The following uses an online shopping scenario as an example to illustrate the execution process of this application's large model. It's worth noting that the technology in this application can also be applied in other fields. The technical solution of the graph process prompt word template introduced in this application is a universal solution that can replace most workflow implementation mechanisms based on manual decomposition.

[0046] More specifically, specific judgment rules can be preset in the attribute information of the judgment node, for example: Node B (Do you want to end the conversation?): the judgment rule is "Based on the information of multiple rounds of conversations, if the user input contains keywords such as 'end', 'exit', 'goodbye', then it is judged as 'yes', otherwise it is 'no'"; Node D (Is it a data query and data analysis problem?): the judgment rule is "Based on the information of multiple rounds of conversations, if the user input involves specific indicator queries, data statistics, or exploring data trends, comparisons and other analysis needs, then it is judged as 'yes'; if it is a string with no query / analysis meaning (such as 'hello', 'today's weather'), then it is judged as 'no'".

[0047] The large model combines input information with judgment rules to generate results. For example, when node B is executed, if the user's current input in the input information is "end the conversation", a judgment result of "yes" is generated and jumps to node C [politely end the conversation]; if the user's current input is "check shopping data", a judgment result of "no" is generated and jumps to node D.

[0048] More specifically, when the large model executes to a node containing a judgment function, it obtains the judgment rules preset in the attribute information of the node; generates a judgment result based on the input information and the judgment rule; and jumps to the corresponding next node based on the judgment result.

[0049] Taking node O (calling rag_search to obtain KB_INFO) as an example, when the large model executes to this node: the rag_search tool can be called according to the "tool call rules" in the node attribute information (for example, "the call parameters are the indicator name, dimension name, and dataset name identified by node N, and the parameter format is {query: metadata name, dataset: dataset name}"); You can also receive the KB_INFO returned by the tool (such as "List[('Single purchase amount', 'Commodity transaction flow table', 'Indicator'), ('City', 'Commodity transaction flow table', 'Dimension')]") and integrate the "tool name (rag_search), calling parameters (query: single purchase amount, dataset: Commodity transaction flow table), tool return results (KB_INFO specific content), and current execution node (node ​​O)" into the intermediate result.

[0050] In S108, intermediate input information is generated based on the intermediate result and input into the subsequent node. The intermediate result can be input into the context manager to generate the intermediate input information. The generated intermediate result can be input into the context manager and integrated with the original context information.

[0051] In S110, when the large model reaches the termination point, a processing result is generated and fed back to the user. For example, during the execution of the large model, the user's intention is identified based on the intermediate results. If the user's intention is to terminate the conversation, the current conversation is terminated and the processing result of the large model execution to the current step is fed back to the user.

[0052] For example, when the large model is executed to the termination node according to the visual workflow execution path diagram, the user's intention is identified based on the intermediate result of the termination node and the corresponding processing result is generated.

[0053] For example, the processing result can be output and then fed back to the user according to the output format definition in the diagram flow prompt word template. Figure 2As shown, the processing results can be post-processed such as format conversion and then replied to the user.

[0054] According to the large-model intelligent workflow interaction method based on graph flow prompt words of the present application, by obtaining the input information of the user's current conversation, the input information includes user input content and context information; the preset graph flow prompt word template and the input information are input into the large model, and the graph flow prompt word template at least includes a visual workflow execution path diagram composed of multiple nodes and attribute information of each node; the large model starts execution from the initial node according to the visual workflow execution path diagram: during the execution process, an intermediate result is generated according to the attribute information corresponding to the node; intermediate input information is generated according to the intermediate result and input into the subsequent node; when the large model executes to the termination node, a processing result is generated; the processing result is output and post-processed and fed back to the user, which can accurately guide the execution of the large model and narrow the answer search space, improve the output effect of the large model, and reduce the engineering application and subsequent maintenance costs.

[0055] It should be clearly understood that this application describes how to form and use specific examples, but the principles of this application are not limited to any details of these examples. On the contrary, based on the teaching of the content disclosed in this application, these principles can be applied to many other embodiments.

[0056] Figure 3 This is a flowchart of a large model intelligent workflow interaction method based on a diagram flow prompt word according to another exemplary embodiment. The generation process of the diagram flow prompt template corresponds to the generation process 30 of the diagram flow prompt template. Figure 1 Before running a large model, it is usually necessary to generate corresponding graph process prompt templates for different target tasks.

[0057] like Figure 3 As shown, in S302, an initial graph process prompt word template is generated based on the target task. For example, task background data is generated based on the target task; macro auxiliary information and global variables are generated based on the target task; an initial visual workflow execution path diagram and attribute information of each node in the diagram are generated; and an output format definition is generated.

[0058] More specifically, the connection relationship between nodes is defined by pseudo code to generate the initial visual workflow execution path diagram, wherein the connection relationship between nodes includes: conditional branching and loop jump logic.

[0059] The structure of the diagram flow prompt word template can be as follows: 1. Mission Background 2. Macro auxiliary information and global variable definition 3. Execution path diagram definition - including fine control of processes such as conditions, loops, jumps, etc. 4. For the nodes in the previous step, if necessary, accurately define the function of each node: (1) Node name (2) Node description (3) Node input (4) Node output (5) The node includes available tools (6) Instructions and sub-steps to follow within a node 5. Macro system output format, etc. Different initial diagram process prompt word templates can be set according to different task objectives. The following uses the "Internet shopping business data query intention recognition" target task as an example to illustrate.

[0060] Among them, the task background can be explained as follows: You are a data query intent recognition assistant designed specifically for the internet shopping business. Your overall task is to determine the user's query intent based on multiple rounds of conversation information, knowledge base data (KB_INFO), user permissions (USER_DATASET_PERMISSION), and current time (CURRENT_TIME). You then determine whether the query intent can be converted into executable query SQL and output the recognition results in the specified JSON format according to the design requirements.

[0061] Among them, the auxiliary information explanation can be: **CURRENT_TIME**: The current system date is '{CURRENT_TIME}'. All data queries based on time ranges must be automatically limited to the valid period before and including this date.

[0062] **USER_DATASET_PERMISSION**: The list of dataset names that the user has permission to query is {USER_DATASET_PERMISSION}. The user has permission to query only the indicator names and dimension names (including dimension enumeration values) contained in each dataset in this list.

[0063] **KB_INFO**: Search results for SQL statistical dimension and business indicator metadata in the knowledge base data. The search results are obtained by dynamically calling the rag_search tool. The return data format for each search is List[Tuple[Candidate indicator name or dimension name (including dimension enumeration value) recalled from the knowledge base, dataset name, type (indicator / dimension / dimension enumeration)]]. Note: Dimension name and indicator name metadata not found in KB_INFO do not exist.

[0064] The relationship and attribute information between nodes can be: Execute the workflow (to achieve this task, you need to strictly follow the steps below) High-Level Execution Steps The execution steps are given in the form of a diagram as follows: A[User's current question and multiple rounds of historical conversations]-->B{{End the conversation?}} B-->|Yes|C[Politely end the conversation] B-->|No|D{{Is this a data query or data analysis issue?}} D-->|No|E[Reply to the user and guide them to enter the number query question and end the conversation] D-->|Yes|F{{Is this a data query question?}} F-->|No|G[Inform the user that the system does not currently support analysis and end the conversation] F-->|Yes|H{{Do you have any comments or complaints about the number lookup function?}} H-->|Yes|I[The user calms down and ends the conversation] H-->|No|J{{Can you identify the dataset name the user is asking about?}} J-->|No|K[Guide the user to select the query dataset from USER_DATASET_PERMISSION and end the dialog] J-->|Yes|L{{Does USER_DATASET_PERMISSION contain the recognized dataset name?}} L-->|No|M[Inform the user that they do not have permission, guide them to select the query dataset from USER_DATASET_PERMISSION and end the conversation] L-->|Yes|N[Entity Recognition SQL Metadata Information] N-->W{{Did you identify at least 1 SQL metadata?}} W-->|No|R W-->|is|O N-->O[Call rag_search to get KB_INFO] O-->P{{Based on KB_INFO, determine whether user clarification is required?}} P-->|Yes|R[ask user for clarification] R-->A P-->|No|S{{Is the identified SQL metadata complete and without contradiction?}} S-->|Problem|R S-->|No problem|T[Summarize information and confirm with user] T-->U{{Check the user input and determine whether the user confirms it?}} U-->|Yes|V[Intent recognition completed] U-->|No|R Refer to the detailed sections below for more information on each step. ##Node B: Determine whether the user wants to end the count and dialogue with the current input Based on the multi-round dialogue information, determine whether the user's current input wants to end the count and dialogue.

[0065] ##Node C: Politely end the conversation The conversation ends with "Thank you for your use. I hope today's data query is helpful to you. I look forward to your use next time."

[0066] ##Node D: Determine whether it is a data query and data analysis problem Based on the information from multiple rounds of conversations, determine whether the user's current input is relevant to data query and data analysis. If the current conversation is a string of characters that has no practical significance for querying and data analysis, it is not relevant.

[0067] ##Node E: Respond to the user, guide them to enter the number query question and end the conversation The way to end the conversation is "The question you are looking for is not within the scope of my services. I can provide you with data query services. Do you have any related questions that I can help you with?"

[0068] ##Node F: Determine whether it is a data query problem Query questions refer to the user's intention to query metadata such as specific indicators.

[0069] Analytical questions refer to the user's intention to explore trends, comparisons, differences, changes, causes, judgments, decisions and other issues.

[0070] Judgment basis for non-data query issues: -Pure analytical questions - The user's intent includes both counting and analysis, and is also considered a non-data query question ##Node G: Reply to the user. The system does not currently support analyzing the problem and ending the conversation. The reply to the user is "Your question is an analytical one. I'm not capable enough and am still working on it. Please stay tuned."

[0071] ##Node I: Respond to the user to appease their emotions and end the conversation -If the user is complaining, the way to end the conversation is "I'm sorry for the poor service, I'm slowly getting better, please be patient with me."

[0072] -If the user is making a suggestion, the conversation ends with "Your opinion is important, thank you for your feedback."

[0073] ##Node N: Entity Recognition SQL Metadata Information SQL metadata consists of the following three categories, which need to be identified separately: -**Time interval**: Identifies the time interval mentioned in the user's question. This entity is required and cannot be empty. Multiple entries are allowed.

[0074] -**Indicator Name**: Identify the indicator name in the user's question. Be careful not to arbitrarily rewrite the indicator name proposed by the user. Subsequent processes will further verify the existence of the name. This entity is required and cannot be empty. Multiple entries are allowed.

[0075] -**Dimension Name**: Identify the dimension name mentioned in the user's question. Be careful not to arbitrarily rewrite the dimension name proposed by the user. Subsequent processes will further verify the existence of the name. This entity is optional and can be empty or contain multiple items.

[0076] ##Node R: Ask the user User interaction enhancement rules: - Expect cordiality and sincerity -When you need clarification from the user, use the format of **"clarification question + example question"**.

[0077] -Interactive sentences should not exceed 50 words, and emojis can be added as appropriate to increase liveliness.

[0078] ##Node P: Determines whether user clarification is required based on KB_INFO -If there is no information in KB_INFO that closely matches the metadata of a certain dimension or indicator, execute the R node. The principle of the dialogue is to inform the user and allow the user to continue to supplement.

[0079] -If there is multiple pieces of information in KB_INFO that closely match a statistical dimension or business indicator metadata, and they belong to the data set to which the user has permission, the R node is executed. The principle of the dialogue is to list multiple fuzzy-matched indicators or dimensions (including enumerated values) that the user has permission to access, and let the user select or supplement them.

[0080] ##Node S: Determine whether the identified SQL metadata is complete and without contradiction Based on all information including multiple rounds of conversation information, the user's current input, various query SQL metadata extracted, CURRENT_TIME, KB_INFO, and USER_DATASET_PERMISSION, we determine whether there are any inconsistencies or missing information. If there are any inconsistencies or missing information, the final judgment standard is to use this information to generate a runnable query SQL that meets the user's needs and for which the user has query permission: - If the information is contradictory, please tell the user where the contradiction lies and execute node R. The principle of speech is to combine all the information to give some options and suggestions. -If information is missing, please tell the user what type of information is missing and execute node R. The principle of the conversation is to recommend supplementary information based on the knowledge base information. ##Node T: Generate final summary information statement Generate a final summary statement based on multiple rounds of conversation information, the user's current input, various extracted SQL query metadata, CURRENT_TIME, KB_INFO, and USER_DATASET_PERMISSION information. This statement must contain all necessary SQL elements and can directly generate executable query SQL that meets the user's needs.

[0081] -If the metadata (metric / dimension / dimension enumeration) that the user wants to query is not available in KB_INFO, but can be inferred based on the information available in KB_INFO, the original query metadata name must be clearly stated in the final summary information statement.

[0082] -According to the information from multiple rounds of conversations, obtain the output format required by the user, such as various charts, etc. If the user does not specify the output format, do not deliberately ask. Just write "No clear output format requirements" in the summary and confirm with the user -Send to the user and wait for the user to confirm before continuing ##Node V: Intent recognition completed This node marks the completion of the intent recognition task: -Only when reaching the node, is_completed=true. At the same time, set statement_sent_to_user=final summary information statement, and the final summary information statement uses a declarative sentence.

[0083] - If metrics / dimensions is not empty, the recorded elements must be metadata that can be found in KB_INFO; if there are unavailable elements, they are recorded in ext_info -Record the confirmed <dimension name:enumeration value> in the dimensions list - For any other node, is_completed=false #Output format JSON field details: -time_span(List[str]): the recognized time interval string -absolute_time_range(List[List[str]]): Standardized time range -dimensions(List[str]): All statistical dimensions identified -metrics(List[str]): All business indicators identified -scope(List[str]): Identified dataset -ext_info(List[str]): additional metadata information of other build executables that can be checked SQL -statement_sent_to_user(str): statement sent to the user -is_completed(bool): Whether intent recognition is completed -node_name(str): The current execution node name of the workflow, in the format of <letter identifier> Think carefully step by step as per instructed above In S304, test questions are generated based on the target task. Different test questions can be pre-set, and the test questions mainly test the rationality of the node-component relationship settings in the initial graph process prompt word template.

[0084] In S306, the test problem is input into the large model, and the intermediate results and the actual visual workflow execution path diagram during the execution process are recorded. Figure 4 As shown, the initial diagram process prompt word template generated above can be used as system prompt, and the test question can be used as user prompt and input into the large model together.

[0085] For example: "system prompt: [complete initial template, including task background, macro auxiliary information, initial path diagram, node attributes, output format]; User prompt: Check the single purchase amount in the commodity transaction flow table for March 2024, divided by city.

[0086] Record the intermediate results generated by each node in the order of node execution, and record the node flow trajectory actually executed by the large model.

[0087] In S308, the initial graph flow prompt word template is iterated based on the intermediate result and the actual visual workflow execution path diagram to generate the graph flow prompt word template. Figure 4 As shown, the intermediate results output by the large model are post-processed to obtain the answer to the question and the execution order of the actual nodes. The answer to the question and the actual node execution order are compared and adjusted with the initial content in the graph flow prompt word template.

[0088] In one embodiment, for example, the intermediate result and the expected intermediate result can be compared; the actual visual workflow execution path diagram can be compared with the initial visual workflow execution path diagram in the initial graph flow prompt word template; and the initial graph flow prompt word template can be iterated according to the comparison result to generate the graph flow prompt word template.

[0089] Compare the recorded actual intermediate results with the expected intermediate results (based on the task requirements), for example: For the test question "Query the single purchase amount in the commodity transaction flow table in March 2024, divided by city", the expected intermediate result (node ​​N) is "time interval [2024-03-01 to 2024-03-31], indicator name [single purchase amount], dimension name [city], dataset [commodity transaction flow table]"; If the actual intermediate result (node ​​N) is "time interval [March 2024], indicator name [shopping amount], dimension name [city], data set [commodity transaction flow table]", there are differences such as "indicator name rewriting ('single purchase amount' → 'shopping amount') and unstandardized time interval".

[0090] You can also compare the actual visual workflow execution path diagram with the initial visual workflow execution path diagram, for example: The initial path graph is node (identification of data set) → node L (authority verification) → node N (metadata identification); If the actual path is J→N (skipping L), there is a path difference of "missing permission check node".

[0091] According to the comparison results, the corresponding module can be adjusted according to the template defects.

[0092] This application addresses the three major pain points of traditional Chain-of-Thought (COT) prompt word engineering in complex process scenarios: "no fixed template, weak instruction compliance, low tuning efficiency, and strong subjectivity." It proposes a technical solution centered on "structured prompt word templates + graph-based pseudocode process definition." The core design of this technical solution is to construct a structured prompt word template that includes "task background, macro-auxiliary information and global variables, execution path diagram, detailed node definition, and output format." The process definition is implemented through graph-based pseudocode. Pseudocode first clarifies the connection relationships between nodes, such as conditional branches and loop jumps, to generate a visual workflow execution path diagram. Each node in the diagram is then refined to define its function, input and output requirements, tool call rules, and instructions to be followed, forming a complete process guidance framework.

[0093] This application's solution retains the advantage of traditional COT in shrinking the prompt space. It also precisely defines the execution path of the large model (i.e., the exact trajectory of the large model in finding the answer) through pseudocode, further shrinking the large model's answer search space and achieving three key improvements: First, a general structured template is used to fix the prompt word writing method, which is adapted to any workflow-based intelligent agent task, solving the problems of poor standardization of prompt words and difficulty in cross-task reuse; second, relying on the execution path diagram defined by pseudocode, the large model's ability to understand complex processes and the effect of following instructions are greatly improved, avoiding process deviation; third, an explicit association is established between the large model output and the execution path diagram nodes, and the output results can be traced back to specific nodes, replacing subjective trial and error with objective feedback, significantly improving the efficiency of prompt word tuning.

[0094] In summary, this application, through systematic process definition and template design, has greatly improved the output quality of large models in complex process tasks (the entity recognition F score in the data intelligence agent intent recognition scenario has increased by 3 percentage points), and can replace the traditional large model application mode with fixed workflows, significantly reducing engineering development, fine-tuning and maintenance costs, and has strong versatility and practical value.

[0095] Those skilled in the art will appreciate that all or part of the steps implementing the above embodiments can be implemented as a computer program executed by a CPU. When executed by the CPU, this computer program performs the functions defined in the method provided herein. The program can be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk, or an optical disk.

[0096] Furthermore, it should be noted that the aforementioned figures are merely illustrative of the processes included in the methods according to exemplary embodiments of the present application and are not intended to be limiting. It is readily understood that the processes illustrated in the aforementioned figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0097] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0098] Figure 5 This is a block diagram of a large-scale intelligent workflow interaction system based on graph process prompt words according to an exemplary embodiment. Figure 5 As shown, the large model intelligent workflow interaction system 50 based on graph process prompt words includes: an information module 502, a graph process calling module 504, a workflow execution module 506, and an output module 508.

[0099] The information module 502 is used to obtain the input information of the user's current conversation, wherein the input information includes the user input content and context information; The information module 502 may include: a content unit for obtaining the input content of the user's current conversation; a context unit for obtaining context information of the user's current conversation by a context manager; and an information unit for generating the input information using the input content and the context information.

[0100] The graph flow calling module 504 is used to input a preset graph flow prompt word template and the input information into the large model, wherein the graph flow prompt word template at least includes a visual workflow execution path diagram consisting of multiple nodes and attribute information of each node; The workflow execution module 506 is used to make the large model start execution from the initial node according to the visual workflow execution path diagram: during the execution process, generate intermediate results according to the attribute information corresponding to the node; generate intermediate input information according to the intermediate results and input it into the subsequent nodes; The workflow execution module 506 may include: a judgment unit, which is used to jump to the corresponding next node according to the judgment result when the large model executes according to the visual workflow execution path diagram to a node containing a judgment function; the judgment unit is also used to obtain the judgment rules preset in the attribute information of the node when the large model executes according to the visual workflow execution path diagram to a node containing a judgment function; generate a judgment result according to the input information and the judgment rule; and jump to the corresponding next node according to the judgment result. A tool unit is used to call and execute the corresponding external tool to generate the intermediate result when the large model executes according to the visual workflow execution path diagram to a node containing a tool call. The workflow execution module 506 may also include: a management unit, which is used to input the intermediate result into the context manager to generate the input information.

[0101] The output module 508 is used to generate processing results and feedback to the user when the large model executes to the termination node. The output module 508 includes: a post-processing unit, which is used to output the processing results according to the output format definition in the graph flow prompt word template and feedback them to the user.

[0102] Figure 6 This is a block diagram of a large model intelligent workflow interaction system based on graph process prompt words according to another exemplary embodiment. Figure 6 As shown, other modules may also be included in the large-model intelligent workflow interaction system based on the graph process prompt words, such as the generation system 60 of the graph process prompt template, which may include: an initial template module 602, a test question module 604, a recording module 606, and a template iteration module 608.

[0103] The initial template module 602 is used to generate an initial graph process prompt word template based on the target task; The initial template module 602 includes: a background data unit for generating task background data based on the target task; an information variable unit for generating macro-auxiliary information and global variables based on the target task; an attribute unit for generating the initial visual workflow execution path diagram and attribute information for each node in the diagram; and a format unit for generating an output format definition. The attribute unit is also used to define the connection relationship between nodes using pseudocode to generate the initial visual workflow execution path diagram, where the connection relationship between nodes includes conditional branches and loop jump logic.

[0104] The test question module 604 is used to generate test questions based on the target task; The recording module 606 is used to input the test questions into the large model and record the intermediate results and the actual visual workflow execution path diagram during the execution process; The template iteration module 608 is configured to iterate the initial graph flow prompt word template according to the intermediate result and the actual visual workflow execution path diagram to generate the graph flow prompt word template.

[0105] Among them, the template iteration module 608 includes: a result comparison unit, used to compare the intermediate result with the expected intermediate result; a path comparison unit, used to compare the actual visual workflow execution path diagram with the initial visual workflow execution path diagram in the initial graph flow prompt word template; an iteration unit, used to iterate the initial graph flow prompt word template according to the comparison result to generate the graph flow prompt word template.

[0106] According to the large-model intelligent workflow interaction system based on graph flow prompt words of the present application, by obtaining the input information of the user's current conversation, the input information includes the user input content and context information; the preset graph flow prompt word template and the input information are input into the large model, and the graph flow prompt word template at least includes a visual workflow execution path diagram composed of multiple nodes and the attribute information of each node; the large model starts to execute from the initial node according to the visual workflow execution path diagram: during the execution process, an intermediate result is generated according to the attribute information corresponding to the node; intermediate input information is generated according to the intermediate result and input into the subsequent node; when the large model executes to the termination node, a processing result is generated; the processing result is output and post-processed and fed back to the user, which can accurately guide the execution of the large model and narrow the answer search space, improve the output effect of the large model, and reduce the engineering application and subsequent maintenance costs.

[0107] like Figure 7 As shown, an embodiment of the present application provides an electronic device, including a processor 710, a memory 730 and a bus 740, wherein the processor 710 and the memory 730 communicate with each other through the bus 740; Memory 730, for storing computer programs; The processor 710 is configured to implement the large-model intelligent workflow interaction method based on graph process prompt words of any of the above embodiments when executing the program stored in the memory 730.

[0108] The communication interface 720 is used for communication between the electronic device and other devices.

[0109] The memory 730 may include a random access memory (RAM) or a non-volatile memory (non-volatile memory), such as at least one disk storage. Alternatively, the memory 730 may be at least one storage device located away from the processor 710.

[0110] If the above method in this application is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, it can implement the steps of the above method embodiments. Among them, the computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form.

[0111] The embodiment of the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the large-model intelligent workflow interaction method based on graph flow prompt words of any of the above embodiments. For example, the input information of the user's current conversation is obtained, and the input information includes user input content and context information; a preset graph flow prompt word template and the input information are input into the large model, and the graph flow prompt word template at least includes a visual workflow execution path diagram composed of multiple nodes and attribute information of each node; the large model starts execution from the initial node according to the visual workflow execution path diagram: during the execution process, an intermediate result is generated according to the attribute information corresponding to the node; intermediate input information is generated according to the intermediate result and input into the subsequent node; when the large model executes to the termination node, a processing result is generated; the processing result is output and processed and fed back to the user.

[0112] While the exemplary embodiments of the present application have been specifically illustrated and described above, it should be understood that the present application is not limited to the detailed structures, configurations, or implementations described herein; rather, the present application is intended to encompass various modifications and equivalent configurations within the spirit and scope of the appended claims.

Claims

1. A large-scale intelligent workflow interaction system based on graph process prompt words, characterized by: include: An information module is used to obtain input information of the current conversation, wherein the input information includes user input content and context information; A graph flow calling module is used to input a preset graph flow prompt word template and the input information into the large model, wherein the graph flow prompt word template at least includes a visual workflow execution path diagram consisting of multiple nodes and attribute information of each node; The workflow execution module is used to make the large model start execution from the initial node according to the visual workflow execution path diagram; during the execution process, generate intermediate results according to the attribute information corresponding to the node; generate intermediate input information according to the intermediate results and input it into the subsequent nodes; The output module is used to generate processing results to feedback to the user when the large model is executed to the termination node.

2. The system according to claim 1, wherein Also includes: The initial template module is used to generate the initial graph process prompt word template based on the target task; Test question module, used to generate test questions based on target tasks; A recording module is used to input the test questions into the large model and record the intermediate results and actual visual workflow execution path diagram during the execution process; A template iteration module is used to iterate the initial graph flow prompt word template according to the intermediate result and the actual visual workflow execution path diagram to generate the graph flow prompt word template.

3. The system according to claim 2, wherein: The initial template module includes: A background data unit, used to generate task background data based on the target task; Information variable unit, used to generate macro auxiliary information and global variables based on the target task; The attribute unit is used to generate the initial visual workflow execution path diagram and the attribute information of each node in the diagram; Format unit, used to generate output format definition.

4. The system according to claim 3, wherein: The attribute unit is also used to The connection relationship between nodes is defined by pseudo code to generate the initial visual workflow execution path diagram, wherein the connection relationship between nodes includes: conditional branch and loop jump logic.

5. The system according to claim 2, wherein: The template iteration module includes: A result comparison unit, configured to compare the intermediate result with an expected intermediate result; A path comparison unit, configured to compare the actual visual workflow execution path diagram with the initial visual workflow execution path diagram in the initial diagram flow prompt word template; An iterative unit is used to iterate the initial graph flow prompt word template according to the comparison result to generate the graph flow prompt word template.

6. The system according to claim 1, wherein: The information module includes: Content unit, used to obtain the input content of the user's current conversation; The context unit is used by the context manager to obtain the context information of the user's current conversation; An information unit is configured to generate the input information using the input content and the context information.

7. The system according to claim 1, wherein: The workflow execution module includes: A judgment unit, configured to jump to a corresponding next node according to a judgment result when the large model executes to a node containing a judgment function according to the visual workflow execution path diagram; and / or The tool unit is used for the large model to call and execute the corresponding external tool to generate the intermediate result when executing to the node containing the tool call according to the visual workflow execution path diagram.

8. The system according to claim 7, wherein: The judging unit is further configured to: When the large model executes to a node including a judgment function according to the visual workflow execution path diagram, it obtains the judgment rule preset in the attribute information of the node; Generate a judgment result according to the input information and the judgment rule; Jump to the corresponding next node based on the judgment result.

9. The system according to claim 1, wherein: The workflow execution module further includes: The management unit is configured to input the intermediate result into a context manager to generate the intermediate input information.

10. The system according to claim 1, wherein: The output module includes: A post-processing unit is used to perform output post-processing on the processing result according to the output format definition in the graph flow prompt word template and feed back the output to the user.

11. A large-scale intelligent workflow interaction method based on graph process prompt words, characterized in that: include: Obtaining input information of the current conversation, the input information including user input content and context information; Inputting a preset graph flow prompt word template and the input information into the large model, wherein the graph flow prompt word template at least includes a visual workflow execution path diagram consisting of multiple nodes and attribute information of each node; The large model starts execution from the initial node according to the visual workflow execution path diagram, and generates intermediate results according to the attribute information corresponding to the node during the execution process; Generate intermediate input information according to the intermediate result and input it into subsequent nodes; When the large model is executed to the termination node, the processing results are generated to feed back to the user.

Images