Large language model instruction optimization method and system based on task decomposition and electronic equipment

Abstract

The invention provides a large language model instruction optimization method and system based on task decomposition and electronic equipment. The performance of a large model on a graph reasoning task is remarkably enhanced. According to the method, firstly, a graph reasoning task is divided into a plurality of subtasks, and an independent instruction template is designed for each subtask, so that the understanding ability of a model to graph topological information is remarkably enhanced; according to the method, the graph structure information is effectively extracted from the natural language by utilizing the graph tool instruction module, and the graph understanding ability of the model is improved. Secondly, aiming at different scales of graphs, a classification method of within-limit graphs and over-limit graphs is adopted, and the problem that large-scale graphs cannot be directly input is solved. The within-limit diagram can be directly analyzed into a text format, and the over-limit diagram is provided through a file path, so that the capability and the flexibility of the model in processing different scales of diagrams are ensured. In addition, through a parameter instruction module, extraction and use of tool parameters are standardized, and accuracy and consistency of the tool parameters in the reasoning task are guaranteed.

Description

Technical Field

[0001] The present invention belongs to the field of large language models, and in particular relates to a large language model instruction optimization method, system, and electronic device based on task decomposition. Background Art

[0002] Large Language Models have demonstrated revolutionary performance improvements in various areas of natural language processing tasks. These models have powerful natural language understanding and generation capabilities through pre-training on massive text data, enabling them to accurately capture the intent and sentiment in the text and provide key contextual information for various downstream task decisions. In multiple natural language processing fields such as text generation, machine translation, speech recognition, and question-answering systems, the application of large language models has greatly improved user experience and decision-making efficiency. Although large language models have performed well in fields such as natural language processing, they still face great challenges in processing graph data. Graph structures exhibit high connectivity, rich combinatorial properties, and non-Euclidean characteristics, which makes their processing fundamentally different from traditional text or image data. Research shows that although large language models have the basic ability to understand graph attributes and solve graph reasoning tasks, their accuracy is still significantly insufficient due to two challenges.

[0003] To improve the reasoning ability of large language models, researchers have explored methods based on in-context learning (ICL). The core idea of ​​this method is to guide the model to quickly adapt to new tasks during the reasoning phase by providing task-related examples (demonstrations) without fine-tuning the model parameters. The key to contextual learning is how to design high-quality prompts and examples to stimulate the model's reasoning ability.

[0004] Talk like a Graph was initiated by Google Research to improve the exploration of large language models (LLMs) in graph reasoning tasks by encoding graph structure data into natural language text. The study explored different prompting strategies, including zero-shot prompts, few-shot contextual learning, etc., to optimize the model's reasoning ability in one step. NLgraph aims to evaluate whether language models can solve graph structure problems through natural language. The researchers proposed two prompting methods: Build-a-Graph prompts and algorithmic prompts; Build-a-Graph prompts encourage the model to map the graph structure to the concept space, while algorithmic prompts guide the model to re-examine the algorithmic steps of the task. Although contextual learning has made significant progress in reasoning tasks, it still faces challenges in generalizing to new domains and tasks. For example, the model is sensitive to the selection and ordering of examples, and human intervention may be required when dealing with complex tasks. Future research directions may include further optimizing example design, improving the dynamic matching mechanism of reasoning patterns, and exploring more efficient search strategies.

[0005] In order to further improve the graph processing capabilities of large language models, researchers began to explore the use of text instructions to guide the model to perform graph-related reasoning tasks. This exploration was initially based on the Chain of Thought (CoT) method, the core idea of ​​which is that large language models can solve complex graph problems through step-by-step reasoning. The chain thinking method encourages the model to show its reasoning process when solving problems, similar to how humans step by step show the steps of solving problems when solving mathematical problems. Many researchers have conducted extensive research and experiments based on the chain thinking method. They found that reasoning in a chain thinking style can significantly improve the performance of large language models on some basic graph reasoning tasks, such as detecting whether there is a loop in the graph (loop detection) and finding the shortest path between two points (shortest path problem). These tasks usually involve analyzing the structure of the graph and applying specific algorithms to find solutions.

[0006] However, when the task becomes more complex, such as solving the maximum flow problem in the network or performing topological sorting, the performance improvement brought by the chain thinking method is not always consistent. These tasks require not only a deep understanding of the structure of the graph, but also the application of more advanced algorithms and search strategies. To address this challenge, GraphWiz uses GPT-4 to generate the initial reasoning path, and combines multi-sample collection and reinforcement learning to improve the accuracy of the model output. This method demonstrates high accuracy and good generalization ability when dealing with various graph problems by selecting the best path from multiple possible solutions. The success of GraphWiz shows that by combining advanced large language models and carefully designed reasoning strategies, the performance of large language models on complex graph tasks can be significantly improved. Another method, GraphInstruct, enhances the reasoning performance of the model by providing a diverse graph generation process and detailed reasoning steps, as well as adopting a step mask training strategy. This method improves the performance of the model on graph tasks by hiding certain steps during training and forcing the model to learn how to reason independently. Although these methods have shown some effectiveness in graph processing tasks, studies have also found that when these methods are applied to tasks different from the training domain (Out of domain task), their performance often drops significantly. This suggests that despite advances in graph processing capabilities of large language models, generalization to new domains and tasks remains a challenge. Summary of the invention

[0007] Existing large language models have two major problems in graph reasoning: 1) insufficient graph understanding capabilities, which are manifested in the inability to accurately grasp the graph topology information in natural language; 2) limited graph processing capabilities, which are restricted by the reasoning ability of the generative model and cannot effectively execute relevant algorithms for solving graph problems. These problems seriously affect the accuracy and efficiency of the model in graph reasoning tasks.

[0008] In order to solve these problems, the present invention proposes a large language model instruction optimization method, system, and electronic device based on task decomposition. By splitting the graph reasoning task into multiple subtasks and designing an independent instruction template for each subtask, this method enhances the model's ability to understand graph topology information. At the same time, with the help of the concept of tool learning, the model's ability to handle graphs of different sizes is optimized, and the extraction and use of tool parameters are standardized. The core component is the graph tool instruction module

[0009] (GraphTool-Instruction), which includes three key technical parts: graph instruction (Graph-Instruction), task instruction (Task-Instruction) and parameter instruction (Parameter-Instruction), which are used for graph extraction, tool name identification and tool parameter extraction respectively.

[0010] In order to solve the above technical problems, a large language model instruction optimization method based on task decomposition of the present invention specifically includes the following steps:

[0011] Step 1: Obtain graph data, extract the required nodes, edges, and feature information in the graph, and then divide the graph into an in-limit graph and an out-limit graph according to the maximum token length threshold of the large language model; the in-limit graph is described in natural language, and the out-limit graph is converted into a graph structure file in a standardized format;

[0012] Step 2: Construct a task instruction tool set, represented by T = {t 1 ,t 2 ,…,t n}, where each tool t i Defined as a four-tuple, including four key attributes, including tool name, tool description, tool parameters and return type; add format constraints based on the task instruction toolset, define a templated calling format for each tool, and guide the output generated by the large language model to match the tool parameters and return type;

[0013] Based on the task instruction toolset and format constraints, the task instructions are composed to guide the large model to generate tool call statements related to the graph reasoning task to form output;

[0014] Step 3: Build a tool retriever to match the tool name generated in the task instruction with the predefined tool set to find the standardized template information of the tool;

[0015] Step 4: Use the tool templates and parameter instructions retrieved by the tool retriever to re-enter the large model to generate standardized tool parameters that meet the requirements of the tool template. The content input into the large language model includes task context, tool templates and parameter instructions to form a new tool call output.

[0016] The graph description in natural language form includes the definition of nodes and edges; the internal graph uses a two-prompt method to extract the information of weighted graphs and unweighted graphs. The first prompt guides the large model to extract the information of weighted graphs, and the second prompt extracts the information of unweighted graphs. After the extraction is completed, the graph structure and related information are converted into a standardized graph format.

[0017] The graph structure file records the node set, edge set and edge weight information of the graph, and provides it to the large language model through the file path; the over-limit graph uses a one-time prompt to guide the large model to extract the file path information of the over-limit graph.

[0018] The standard format of the tool call statement ToolCall is as follows:

[0019] ToolCall=Name(Graph=G,Param 1 =value 1 ,…,param i =value i )

[0020] Among them, Name indicates the tool name, Graph indicates the input graph object G, which is an in-limit graph or an out-limit graph, and Param i Indicates tool parameters, value i Indicates the actual parameter values ​​required by the tool when executing;

[0021] Therefore, the large model output LLM is expressed as:

[0022] ToolCall=LLM(I)

[0023] Wherein, I represents the task instruction.

[0024] The operation of the tool template retriever is expressed as:

[0025] T(Ne)=Retrieve(Name,T)

[0026] Specifically, the tool template retriever Retrieve will match the tool name Name in the tool call statement ToolCall generated by the task instruction with the predefined tool names in the toolset T. Once a matching tool name is found, the retriever will extract the complete tool information T(Name) corresponding to the name from the toolset.

[0027] The internal graph is parsed by regular expressions to obtain a standardized graph format.

[0028] The present invention also provides a large language model instruction optimization system based on task decomposition, comprising:

[0029] Graph instruction module: used for graph extraction, obtaining node, edge, and feature information in graph data, and then dividing the graph into in-limit graph and out-limit graph according to the maximum token length threshold of the large language model; the in-limit graph is described in natural language, and the out-limit graph is converted into a graph structure file in a standardized format;

[0030] Task instruction module: includes a task instruction toolset, where each tool is defined as a quadruple, including four key attributes, including tool name, tool description, tool parameters, and return type; and based on the task instruction toolset and format constraints, the task instruction is composed to guide the large language model to generate tool call statements related to the graph reasoning task to form output, and the output matches the tool parameters and return type;

[0031] Parameter instruction module: including a tool retriever, which matches the tool name generated in the task instruction with the predefined tool set to find the standardized template information of the tool;

[0032] Output execution module: The tool templates and parameter instructions retrieved by the tool retriever are re-input into the large model to generate standardized tool parameters that meet the requirements of the tool template. The content input into the large language model includes task context, tool templates and parameter instructions to form a new tool call output.

[0033] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above method.

[0034] Thanks to the following three designs, the large model enhanced by the graph tool instruction module has achieved the best reasoning effect in the field of graph reasoning.

[0035] Graph directives: This directive is designed to help large models extract graph structure information from a given task. In order to evaluate and enhance the model's ability to handle graphs of different sizes, the maximum token length of 4096, which is generally accepted by current large models, is used as a benchmark. Based on this threshold, the graph is divided into a WL-Graph and an EL-Graph. WL-Graphs can be directly input into the model in text form, while EL-Graphs need to be stored in files and provided to the model through paths. Therefore, two types of graph directives are designed. For WL-Graphs, information of weighted and unweighted graphs is extracted through two prompts and presented in the graph format of the NetworkX package in Python. The graph structure information in the output text is parsed using regular expressions, and the graph can be reconstructed in the tool. Due to the large size of the EL-Graph, it is difficult to fully extract the graph structure information through natural language, so the file path is used instead of the graph. For the EL-Graph, a one-time prompt is used to guide the model to identify and extract the file path, so that the tool can retrieve the graph structure information from the specified path.

[0036] Task Instructions: To build the task instructions, a toolset was created. For each tool, four properties were defined: tool name, tool description, tool parameters, and return type. This set is intended to inform the big model of the appropriate graph reasoning task for each tool. Based on the predefined toolset, some general descriptions of the expected format were added to constrain the output of the big model.

[0037] Parameter Instructions: For reasoning tasks that contain parameters, parameter instructions are specifically used to further standardize the format of parameters extracted by task instructions. First, a tool template retriever is proposed, which recognizes tool names based on previous task instructions and then retrieves the corresponding tool templates from the tool set. In the second step, the retrieved tool templates are combined with parameter instructions as new input to obtain highly accurate tool parameters.

[0038] The invention proposes a large language model instruction optimization method based on task decomposition, which significantly enhances the performance of large models in graph reasoning tasks. First, by splitting the graph reasoning task into multiple subtasks and designing an independent instruction template for each subtask, the model's ability to understand graph topology information is significantly enhanced. The method uses the graph tool instruction module to effectively extract graph structure information from natural language, thereby improving the model's graph understanding ability. Secondly, for graphs of different sizes, the classification method of limited graphs and over-limit graphs is adopted to solve the problem that large-scale graphs cannot be directly input. The limited graph can be directly parsed into text format, while the over-limit graph is provided through the file path, ensuring the model's ability and flexibility in processing graphs of different sizes. In addition, through the parameter instruction module, the extraction and use of tool parameters are standardized, ensuring the accuracy and consistency of tool parameters in reasoning tasks. This design improves the accuracy and efficiency of graph processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a schematic diagram of the structure of the tool instruction module in the method of the present invention. DETAILED DESCRIPTION

[0040] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of a large language model instruction optimization method based on task decomposition of the present invention in conjunction with the accompanying drawings.

[0041] This patent proposes a large language model instruction optimization method based on task decomposition. By splitting the graph reasoning task into multiple subtasks and designing independent instruction templates for each subtask, this method significantly enhances the model's ability to understand graph topology information. At the same time, with the help of the concept of tool learning, the model's ability to handle graphs of different sizes is optimized, and the extraction and use of tool parameters are standardized.

[0042] The core task of the present invention is to optimize graph structure reasoning tasks based on a large language model. Graph structure reasoning tasks are a type of complex reasoning tasks with a graph as the core data structure, which usually involves understanding and reasoning about nodes, edges and their relationships. The multi-task learning capability of the large model enables it to handle a variety of graph reasoning tasks under a unified framework, thereby reducing dependence on dedicated algorithms and improving the versatility and scalability of tasks. By means of task decomposition, the present invention can break down complex graph reasoning tasks into multiple subtasks, each of which is optimized through an independent instruction template, so that the large language model can more efficiently process the topological information and semantic information of graph structure data.

[0043] The method of the present invention can be widely used in tasks such as traffic network path query and planning, social network character relationship query, and computer network structure query. For example, in a traffic network, the method can be used for path query, shortest path planning, and multi-mode traffic path optimization, significantly improving the intelligence level of the traffic system. In a social network, it can be used to query the relationship path between users, analyze key figures, and identify social groups, helping to deeply understand the structure and dynamics of the social network. In a computer network or communication network, the method can efficiently complete network topology structure query, key node identification, and network fault path location, providing strong support for network management and maintenance.

[0044] The main contributions of this method are as follows: 1) Based on the idea of ​​task decomposition, a plug-and-play tool instruction template is designed: a large language model tool instruction template for graph reasoning tasks is proposed, including graph instructions (Graph-Instruction), task instructions (Task-Instruction) and parameter instructions (Parameter-Instruction). This instruction template can be seamlessly embedded in the current mainstream large language models (such as GPT, Llama, etc.), significantly improving their performance in graph reasoning tasks; 2) A processing mechanism for bounded graphs (WL-Graph) and over-bounded graphs (EL-Graph) is proposed: in view of the maximum token length limit of large language models, an innovative classification method for bounded graphs and over-bounded graphs is proposed, and a corresponding processing mechanism is designed, so that the model can effectively handle graph reasoning tasks of different scales; 3) A tool template retriever is introduced to standardize the tool parameter extraction process: through the tool template retriever combined with parameter instructions, the tool parameter extraction is highly standardized and accurate, further improving the performance of large language models in complex reasoning tasks.

[0045] like Figure 1As shown in the figure, this method is divided into three basic modules: Graph-Instruction, Task-Instruction and Parameter-Instruction. These components are respectively oriented to three subtasks: graph extraction, tool name recognition and tool parameter extraction. The specific technical details will be introduced below:

[0046] A large language model instruction optimization method based on task decomposition comprises the following steps:

[0047] Step 1: Obtain graph data, and automatically extract nodes, edges and features from the graph; in practical applications, the method of the present invention can be widely used in the fields of transportation network path query and planning, social network character relationship analysis, and network structure query. In a transportation network, nodes can be automatically generated by the geographic coordinates or unique identifiers (such as station numbers, airport codes, etc.) of transportation hubs, and edges are constructed based on the actual connection relationships of transportation routes (such as the starting and ending points of roads and railways) and related attributes (such as distance, travel time, etc.). Through this method, tasks such as path query, shortest path planning, and multi-modal transportation path optimization can be efficiently completed, thereby significantly improving the intelligence and operation efficiency of the transportation system; in social network scenarios, nodes are usually generated by the user's unique identifier (such as user ID), and edges are automatically extracted through interaction records between users (such as friend relationships, follow-up relationships, or message records), and weight information (such as interaction frequency or relationship strength) can be added. With this method, we can quickly query the relationship paths between users, identify key figures in social networks, and even analyze group structures and interaction patterns. In computer networks or communication networks, nodes can be generated by the unique identifier of a device or server (such as an IP address or MAC address), and edges can be constructed based on the connection relationship between devices (such as a physical link or a logical connection), and weight information (such as bandwidth, latency, or traffic) can be added. This method can be used to query network topology, locate key nodes, and quickly diagnose network fault paths, thereby providing strong support for network management and maintenance.

[0048] According to the maximum token length threshold of the large model, the graph is divided into a WL-Graph and an EL-Graph. The maximum token length threshold of the large model refers to the maximum length of the input text that the current large language model (such as Llama3-8B) can effectively process. To ensure that the model can effectively parse and process the graph structure information, the input graph data is classified based on the threshold. In the present invention, the input data form of the graph can include the following two types:

[0049] 1)) Graph description in natural language: This is a way to describe the graph structure in text form, usually including the definition of nodes and edges. For example, "The graph contains nodes 1, 2, ..., 20, and the graph contains the following edges: 1->2 (weight: 3), 2->4 (weight: 4), ..., 10->20 (weight: 1)". This description method is suitable for smaller graphs because its text length does not exceed the token limit of large models.

[0050] 2) Graph structure files: For large-scale graphs, their complex topological structures are difficult to fully describe using natural language, so they are stored in files. Files can be stored in standardized graph data formats, such as JSON, CSV, or GraphML. These files record information such as the graph's node set, edge set, and edge weights, and are provided to the model through file paths. The graph data in the file can be parsed and loaded by tools such as Python's NetworkX library for subsequent task processing.

[0051] For the internal-limited graph (WL-Graph), a two-shot method is used to extract information about weighted and unweighted graphs. The first prompt guides the large model to extract information about weighted graphs (such as the shortest path problem), and the second prompt is used to extract information about unweighted graphs (such as the loop detection problem). After the extraction is completed, the graph structure and related information will be converted into a standardized graph format supported by Python's NetworkX library for subsequent tool calls.

[0052] In the text containing graph information, the description of the edge may appear in the form of "1->2 (weight: 3)" or "(2,4,weight=4)", and the node number is usually a number. However, these descriptions are not uniform and cannot be directly read by the tool. Therefore, it is necessary to first parse the node set and edge set from the text and convert them into a standardized graph data structure. The node is usually a set of numbers, such as "1, 2, ..., 20", which can be directly identified as a complete list of nodes; the edge is represented in the form of a triple, including the starting point, the end point and an optional weight (such as "1->2 (weight: 3)" means that there is an edge with a weight of 3 between node 1 and node 2). To achieve this process, the original graph information can be normalized into a unified graph representation form through a large model, and then the detailed information of the nodes and edges can be extracted from the text using tools such as regular expressions. For example, for the output format "edges are:

[0053] [(1,2,weight=3),(2,4,weight=4)]", the regular expression can efficiently extract the start point, end point and weight information of the edge. Finally, the node and edge information will be organized into a standardized form to facilitate program operation and subsequent processing.

[0054] For EL-Graph, due to its large size, it is difficult to fully describe its structural information through natural language. Therefore, a one-time prompt is used to guide the large model to extract the file path information of the EL-Graph. The prompt content includes the storage location of the file, the file format, and how to retrieve and load the graph structure information from the file path through the tool. By storing the structural information of the EL-Graph in a file and providing it to the model in the form of a path, the tool can directly retrieve the graph data from the specified path, thus solving the input limitation problem caused by the large size of the EL-Graph.

[0055] Step 2: Forming task instructions: In order to guide the large model to accurately generate tool call statements related to graph reasoning tasks, we first need to build a task instruction toolset. The core of the toolset is to define four key properties for each tool. Tool name, tool description, tool parameters, and return type. The task instruction toolset can be represented as a set T = {t 1 ,t 2 ,…,t n}, where each tool t i Defined as a quaternion:

[0056] t i =(Name i ,Desc i ,Param i ,Return i )

[0057] Among them, Name represents the unique identification name of the tool, which is used by the large language model to identify and call the tool. For example, "ShortestPath" can represent a tool for calculating the shortest path in a graph. Desc represents the functional description of the tool, which is used to briefly explain the role of the tool and clarify its specific use in graph reasoning tasks, such as "calculating node centrality" or "generating the shortest path". Param represents the set of input parameters required by the tool, including parameter name, parameter type (such as string, integer, floating point number, etc.) and the specific meaning of the parameter. For example, the parameters may include "starting node (start_node)" and "target node (end_node)", which represent the starting point and end point of the path calculation respectively. Return represents the return type of the tool, which is used to describe the format and data type of the tool output. For example, the return value may be an integer (such as the length of the shortest path), a list (such as a sequence of nodes on the path), a dictionary (such as a mapping between nodes and their attributes), or a graph structure (such as a graph object of NetworkX).

[0058] By building a task instruction toolset, a clear tool calling specification is provided for the large model to ensure that the model can accurately understand the function and usage of each tool. At the same time, in order to make the tool call statements generated by the large model conform to the expected standard format, it is necessary to add clear constraints on the format based on the task instruction toolset. Specifically, by defining a templated call format for each tool, the output generated by the large model is guided to match the tool parameters and return types. The standard format of tool calls can be expressed as:

[0059] ToolCall=Name(Graph=G,Param 1 =value 1 ,…,Param i =value i )

[0060] Name represents the tool corresponding to the corresponding tool name, Graph represents the input graph object G, which is obtained by parsing the in-limit graph through regular expression or by reading the specified path file for the out-limit graph, Param represents the parameters required for tool execution, and value i Indicates the actual parameter values ​​required by the tool when executing;

[0061] By introducing format constraints on the data structure required by the tool, we ensure that the tool call statements generated by the large model are not only syntactically correct, but also meet the input requirements of tool execution. Based on the task instruction toolset T and format constraints, the task instruction I is composed to guide the large model to generate tool call statements related to the graph reasoning task. Specifically, the task instruction toolset T contains detailed information about all tools that can be provided to the large model, and the format constraints define the standardized format of the tool call statements in the form of natural language, clarifying the grammatical rules and parameter structure of the tool call to ensure that the call statements generated by the large model meet the input requirements of the tool. The output of the large model can be expressed as:

[0062] ToolCall=LLM(I)

[0063] ToolCall represents the tool call statement output by the large model that can be recognized and directly executed.

[0064] Step 3: Use the parameter instruction module to match the tool name generated in the task instruction with the predefined tool set to find the standardized template information of the tool: However, due to the limitations of the open source large model, especially the small parameter large model (such as Llama3-8B), although the model can generate statements that meet the tool call format based on the instruction template, there may still be problems such as parameter misalignment, omission, and duplication. Therefore, this method proposes a tool template retriever, whose core function is to match the tool name generated in the task instruction with the predefined tool set to find the standardized template information of the tool. The operation of the tool template retriever can be expressed as:

[0065] T(Name)=Retrieve(Name,T)

[0066] Specifically, the tool template retriever Retricve matches the tool name Name in the tool call statement ToolCall generated by the task instruction with the predefined tool names in the toolset T. Once a matching tool name is found, the retriever extracts the complete tool information T(Name) corresponding to the name from the toolset.

[0067] Through the retriever, the parameters and call specifications required by the tool can be accurately obtained, providing a basis for the subsequent standardized parameter extraction. In this step, the retrieved tool template T (Name) and parameter instructions are re-input into the large model to generate standardized tool parameters P that meet the requirements of the tool template. The content input into the large model includes task context, tool template and parameter instructions to form a new tool call output:

[0068] ToolCall new =Name(Graph=G,Param 1 =value 1 ,…,Param i =value i )

[0069] The present invention also provides a large language model instruction optimization system based on task decomposition, comprising:

[0070] Graph instruction module: used for graph extraction, obtaining node, edge, and feature information in graph data, and then dividing the graph into in-limit graph and out-limit graph according to the maximum token length threshold of the large language model; the in-limit graph is described in natural language, and the out-limit graph is converted into a graph structure file in a standardized format;

[0071] The graph description in natural language form includes the definition of nodes and edges; the internal graph uses a two-prompt method to extract the information of weighted graphs and unweighted graphs. The first prompt guides the large model to extract the information of weighted graphs, and the second prompt extracts the information of unweighted graphs. After the extraction is completed, the graph structure and related information are converted into a standardized graph format.

[0072] The graph structure file records the node set, edge set and edge weight information of the graph, and provides it to the large language model through the file path; the over-limit graph uses a one-time prompt to guide the large model to extract the file path information of the over-limit graph.

[0073] The internal graph is parsed by regular expressions to obtain a standardized graph format.

[0074] Task instruction module: includes a task instruction toolset, where each tool is defined as a quadruple, including four key attributes, including tool name, tool description, tool parameters, and return type; and based on the task instruction toolset and format constraints, the task instruction is composed to guide the large language model to generate tool call statements related to the graph reasoning task to form output, and the output matches the tool parameters and return type;

[0075] The standard format of the tool call statement ToolCall is as follows:

[0076] ToolCall=Name(Graph=G,Param 1 =value 1 ,…,Param i =value i )

[0077] Among them, Name indicates the tool name, Graph indicates the input graph object G, which is an in-limit graph or an out-limit graph, and Param i Indicates tool parameters, value i Indicates the actual parameter values ​​required by the tool when executing;

[0078] Therefore, the large model output LLM is expressed as:

[0079] ToolCall=LLM(I)

[0080] Wherein, I represents the task instruction.

[0081] Parameter instruction module: including a tool retriever, which matches the tool name generated in the task instruction with the predefined tool set to find the standardized template information of the tool;

[0082] The operation of the tool template retriever is expressed as:

[0083] T(Name)=Retrieve(Name,T)

[0084] Specifically, the tool template retriever Retrieve will match the tool name Name in the tool call statement ToolCall generated by the task instruction with the predefined tool names in the toolset T. Once a matching tool name is found, the retriever will extract the complete tool information T(Name) corresponding to the name from the toolset.

[0085] Output execution module: The tool templates and parameter instructions retrieved by the tool retriever are re-input into the large model to generate standardized tool parameters that meet the requirements of the tool template. The content input into the large language model includes task context, tool templates and parameter instructions to form a new tool call output.

[0086] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above method.

[0087] To verify the effectiveness of the method and model, a large number of experiments were conducted on the public datasets GraphInstruct and NLGraph. The results showed that this method has reached an advanced level in open source models and surpassed the closed-source model method based on Few-shot.

[0088] The loop detection, connectivity, bipartite graph determination, graph topology analysis, shortest path problem, triangle detection, and maximum flow problem tasks were selected from the GraphWiz dataset to verify the model capability. The connectivity, loop detection, Hamiltonian path, bipartite graph determination, shortest path problem, and degree calculation tasks were selected from the InstructGraph dataset to verify the model capability.

[0089] The answer accuracy is used as the evaluation indicator to calculate the percentage of questions correctly predicted by the large language model in the test data set to the total number of questions.

[0090] GPT-3.5-Turbo, GPT-4-Turbo and Cluade-3-sonnet are currently recognized as the closed-source large model series with the strongest reasoning ability. The above three models are selected and the reasoning ability of each large model for related graph tasks is enhanced based on the two-shot context prompt method. GraphWiz is selected as the representative of the chain thinking large model. At the same time, in order to better compare the effectiveness of this method (using Llama3-8B without fine-tuning), we introduced the open source large model Llama3-8B fine-tuned using the Lora method as a comparison.

[0091] Table 1. Comparison of graph reasoning results based on the GraphWiz dataset

[0092]

[0093] As shown in Table 1, with the large model Llama3-8B as the carrier, this method shows excellent graph reasoning ability on the GraphWiz dataset, with an average accuracy of 97.07%, significantly surpassing all baseline models. Compared with large language models based on text instructions (such as GPT-3.5-Turbo and GPT-4-Turbo), its average accuracy is improved by 68.57% and 58.25% respectively, showing the great advantage of this method in graph tasks. Even compared with the best performing baseline method GraphWiz (average accuracy of 53.71%), the graph tool instruction module has significant improvements on all tasks. For example, on complex tasks such as Topology Sorting and Shortest Path, the accuracy of the graph tool instruction module reached 97.50% and 98.00% respectively, far exceeding GraphWiz's 28.00% and 27.75%. These results show that even without fine-tuning, the graph tool instruction module can still solve graph reasoning problems with near-perfect performance, showing extremely high research value and application potential.

[0094] Table 2. Comparison of graph reasoning results based on the GraphInstruct dataset

[0095]

[0096]

[0097] On the InstructGraph dataset, the graph tool instruction module also demonstrated strong graph reasoning capabilities, with an average accuracy of 97.1%, significantly ahead of other models. In specific tasks, it performed particularly well in tasks such as Cycle Detection and Connection, with accuracy rates of 100% and 99.82%, respectively, which is a significant improvement over GraphWiz's 90.02% and 84.55%. On more challenging tasks such as Hamilton Path and Shortest Path, the graph tool instruction module achieved accuracies of 92.91% and 98.60%, respectively, far ahead of other models. This performance shows that the graph tool instruction module can demonstrate comprehensive and stable high performance in different types of graph tasks, further proving the effectiveness and adaptability of its tool instruction template. Even in the face of complex datasets and tasks, its performance is still close to perfect, demonstrating extremely high practical value.

[0098] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A large language model instruction optimization method based on task decomposition, characterized in that: The following steps are involved: Step 1: Obtain graph data, extract the required nodes, edges, and feature information in the graph, and then divide the graph into an in-limit graph and an out-limit graph according to the maximum token length threshold of the large language model; the in-limit graph is described in natural language, and the out-limit graph is converted into a graph structure file in a standardized format; Step 2: Construct a task instruction tool set, represented by T = {t1, t2, …, t n }, where each tool t i Defined as a four-tuple, including four key attributes: tool name, tool description, tool parameters, and return type; Add format constraints based on the task instruction toolset, define a templated calling format for each tool, and guide the output generated by the large language model to match the tool parameters and return types; Based on the task instruction toolset and format constraints, the task instructions are composed to guide the large model to generate tool call statements related to the graph reasoning task to form output; Step 3: Build a tool retriever to match the tool name generated in the task instruction with the predefined tool set to find the standardized template information of the tool; Step 4: Use the tool templates and parameter instructions retrieved by the tool retriever to re-enter the large model to generate standardized tool parameters that meet the requirements of the tool template. The content input into the large language model includes task context, tool templates and parameter instructions to form a new tool call output.

2. The large language model instruction optimization method based on task decomposition according to claim 1 is characterized in that: The graph description in natural language form includes the definition of nodes and edges; the internal graph uses a two-prompt method to extract the information of weighted graphs and unweighted graphs. The first prompt guides the large model to extract the information of weighted graphs, and the second prompt extracts the information of unweighted graphs. After the extraction is completed, the graph structure and related information are converted into a standardized graph format.

3. The method for optimizing large language model instructions based on task decomposition according to claim 2, characterized in that: The graph structure file records the node set, edge set and edge weight information of the graph, and provides it to the large language model through the file path; the over-limit graph uses a one-time prompt to guide the large model to extract the file path information of the over-limit graph.

4. The method for optimizing large language model instructions based on task decomposition according to claim 3, characterized in that: The standard format of the tool call statement TookCall is as follows: ToolCall=Name(Graph=G,Param1=value1,…,Param i =value i ) Among them, Name indicates the tool name, Graph indicates the input graph object G, which is an in-limit graph or an out-limit graph, and Param i Indicates tool parameters, value i Indicates the actual parameter values ​​required by the tool when executing; The large model output LLM is expressed as: ToolCall=LLM(I) Wherein, I represents the task instruction.

5. The method for optimizing large language model instructions based on task decomposition according to claim 4, characterized in that: The operation of the tool template retriever is expressed as: T(Name)=Retrieve(Name,T) Specifically, the tool template retriever retrieve will match the tool name Name in the tool call statement ToolCall generated by the task instruction with the predefined tool names in the toolset T. Once a matching tool name is found, the retriever will extract the complete tool information T(Name) corresponding to the name from the toolset.

6. The method for optimizing large language model instructions based on task decomposition according to claim 5, characterized in that: The internal graph is parsed by regular expressions to obtain a standardized graph format.

7. A large language model instruction optimization system based on task decomposition, characterized in that: include: Graph instruction module: used for graph extraction, obtaining node, edge, and feature information in graph data, and then dividing the graph into in-limit graph and out-limit graph according to the maximum token length threshold of the large language model; the in-limit graph is described in natural language, and the out-limit graph is converted into a graph structure file in a standardized format; Task instruction module: includes a task instruction toolset, where each tool is defined as a quadruple, including four key attributes, including tool name, tool description, tool parameters, and return type; and based on the task instruction toolset and format constraints, the task instruction is composed to guide the large language model to generate tool call statements related to the graph reasoning task to form output, and the output matches the tool parameters and return type; Parameter instruction module: including a tool retriever, which matches the tool name generated in the task instruction with the predefined tool set to find the standardized template information of the tool; Output execution module: The tool templates and parameter instructions retrieved by the tool retriever are re-input into the large model to generate standardized tool parameters that meet the requirements of the tool template. The content input into the large language model includes task context, tool templates and parameter instructions to form a new tool call output.

8. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store program data, and the processor is used to execute the program data to implement the large language model instruction optimization method based on task decomposition as described in any one of claims 1 to 6.

Images