Task evaluation method, task evaluation result display method and device, equipment and medium

Abstract

The invention relates to a task evaluation method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: receiving a query question sent by a terminal, and returning a task node corresponding to the query question to the terminal, so that the terminal displays the task node; executing the task represented by the task node to obtain a task execution result; determining a node evaluation index value of the task node based on the task execution result; and returning the node evaluation index value to the terminal to enable the terminal to update a display style of the task node based on the node evaluation index value, and displaying the node evaluation index value of the task node in response to a trigger operation on the task node. By adopting the method, the transparency and interpretability of the task execution process can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a task evaluation method, a method for displaying task evaluation results, an apparatus, a device, and a medium. Background Technology

[0002] With the development of Large Language Modeling (LLM), Multi-Agent, and Intelligent Workflow technologies, intelligent decision-making systems based on the collaborative execution of multiple task nodes are gradually becoming an important means of handling complex query tasks. Such systems typically perform task planning on the user-input query, breaking down the overall task into multiple independent or related task nodes. Each task node then performs semantic analysis, data processing, or inference operations to obtain the final query result. While multi-task node collaboration can improve system processing capabilities, it also makes the internal inference chain more complex, placing higher demands on the transparency and interpretability of the task execution process.

[0003] In traditional technologies, systems typically only display the structure or execution status of task nodes, such as static status information like "executed" or "execution failed," failing to reveal the role of each task node in the process of generating query results. This makes it difficult for users to effectively understand the decision-making logic that generates query results. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer device, and storage medium for displaying node evaluation and task evaluation results that can improve task interpretability, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a task evaluation method. The method includes:

[0006] The system receives a query question sent by the terminal and returns the task node corresponding to the query question to the terminal so that the terminal displays the task node.

[0007] Execute the task represented by the task node to obtain the task execution result;

[0008] The node evaluation index value of the task node is determined based on the task execution result;

[0009] The node evaluation index value is returned to the terminal so that the terminal updates the display style of the task node based on the node evaluation index value, and displays the node evaluation index value of the task node in response to the trigger operation of the task node.

[0010] Secondly, this application also provides a task evaluation apparatus. The apparatus includes:

[0011] The query question receiving module is used to receive query questions sent by the terminal and return the task node corresponding to the query question to the terminal so that the terminal can display the task node.

[0012] The task execution module is used to execute the task represented by the task node and obtain the task execution result;

[0013] The indicator value determination module is used to determine the node evaluation indicator value of the task node based on the task execution result;

[0014] The indicator value return module is used to return the node evaluation indicator value to the terminal, so that the terminal updates the display style of the task node based on the node evaluation indicator value, and displays the node evaluation indicator value of the task node in response to the trigger operation of the task node.

[0015] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0016] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0017] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described method.

[0018] The aforementioned task evaluation method, apparatus, computer equipment, storage medium, and computer program product receive a query question sent by a terminal, return the task node corresponding to the query question to the terminal so that the terminal displays the task node, executes the task represented by the task node, obtains the task execution result, determines the node evaluation index value of the task node based on the task execution result, and returns the node evaluation index value to the terminal to update the display style of the task node, and displays the node evaluation index value of the task node in response to the trigger operation of the task node. This allows users to intuitively identify the importance and reliability of the task node in the process of generating query results in the interface, thereby solving the problem that traditional technologies cannot reveal the mechanism of action of task nodes, making it difficult for users to understand the decision logic of query results, and improving the transparency and interpretability of the task execution process.

[0019] Sixthly, this application provides a method for displaying task evaluation results. The method includes:

[0020] The query question has been retrieved;

[0021] Display the task node corresponding to the query question;

[0022] After the task corresponding to the task node is completed, the display style of the task node is updated; the display style is related to the node evaluation index value corresponding to the task node; the node evaluation index value represents the evaluation result of the task execution status of the task node;

[0023] In response to a trigger operation on the task node, the node evaluation index value of the task node is displayed.

[0024] Seventhly, this application also provides a display device for task evaluation results. The device includes:

[0025] The query question retrieval module is used to retrieve query questions;

[0026] The task node display module is used to display the task node corresponding to the query question; after the task corresponding to the task node is completed, the display style of the task node is updated; the display style is related to the node evaluation index value corresponding to the task node; the node evaluation index value represents the evaluation result of the task execution status of the task node;

[0027] The information display module is used to display the node evaluation index value of the task node in response to the trigger operation of the task node.

[0028] Eighthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0029] Ninthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0030] In a tenth aspect, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described above.

[0031] The aforementioned method, apparatus, computer device, storage medium, and computer program product for displaying task evaluation results, after obtaining a query question, displays the task node corresponding to the query question. After the task corresponding to the task node is completed, the display style of the task node is updated based on the received node evaluation index value. In response to the triggering operation of the task node, the node evaluation index value of the task node is displayed. The node evaluation index value represents the evaluation result of the task node's task execution status, enabling users to intuitively identify the importance and reliability of the task node in the process of generating query results in the interface. This solves the problem in traditional technologies that cannot reveal the mechanism of task node operation, making it difficult for users to understand the decision logic of query results, and improves the transparency and interpretability of the task execution process. Attached Figure Description

[0032] Figure 1 This is a diagram illustrating the application environment of the task evaluation method in one embodiment;

[0033] Figure 2 This is a flowchart illustrating a task evaluation method in one embodiment;

[0034] Figure 3 This is a schematic diagram of a first instruction template in one embodiment;

[0035] Figure 4 This is a schematic diagram of a second instruction template in one embodiment;

[0036] Figure 5 This is a flowchart illustrating a method for displaying task evaluation results in one embodiment;

[0037] Figure 6 This is a schematic diagram of a query processing page in one embodiment;

[0038] Figure 7 This is a schematic diagram of the query processing page in another embodiment;

[0039] Figure 8 This is a schematic diagram of the node evaluation system in one embodiment;

[0040] Figure 9 This is a structural block diagram of a task evaluation device in one embodiment;

[0041] Figure 10 This is a structural block diagram of the task evaluation device in another embodiment;

[0042] Figure 11 This is a structural block diagram of a device for displaying task evaluation results in one embodiment;

[0043] Figure 12 A structural block diagram of a device for displaying task evaluation results in another embodiment;

[0044] Figure 13 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0046] The task evaluation method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be placed on the cloud or other servers. The above task evaluation method can be executed by terminal 102 or server 104 independently, or it can be executed through interaction between terminal 102 and server 104. In one embodiment, server 104 receives a query question sent by the terminal, returns the task node corresponding to the query question to the terminal, so that the terminal displays the task node, executes the task represented by the task node, obtains the task execution result, determines the node evaluation index value of the task node based on the task execution result, and returns the node evaluation index value to the terminal, so that the terminal updates the display style of the task node based on the node evaluation index value, and displays the node evaluation index value of the task node in response to a trigger operation on the task node.

[0047] The method for displaying task evaluation results provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated on server 104, or it can be placed in the cloud or on other servers. The method for displaying the task evaluation results described above can be executed by terminal 102 or server 104 individually, or it can be executed through interaction between terminal 102 and server 104. In one embodiment, after obtaining a query question, terminal 102 sends the query question to the server, so that the server returns the task node corresponding to the query question and executes the task represented by the task node to obtain the task execution result. Based on the task execution result, the node evaluation index value of the task node is determined, the node evaluation index value is returned, and the task node corresponding to the query question is displayed. After the task corresponding to the task node is completed, the display style of the task node is updated. The display style is related to the node evaluation index value corresponding to the task node. In response to a trigger operation on the task node, the node evaluation index value of the task node is displayed.

[0048] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0049] In one embodiment, such as Figure 2 As shown, a task evaluation method is provided, which can be applied to... Figure 1 Taking server 104 as an example, the following steps are included:

[0050] S202, Receive the query question sent by the terminal, and return the corresponding task node to the terminal so that the terminal can display the task node.

[0051] The query question refers to the content of the query entered by the user. The query question can cover user needs of any category or field, such as product comparison, knowledge Q&A, service consultation, process guidance, etc. For example, users can enter queries such as "Comparison between model A and model B mobile phones", "How to improve image clarity", "What to do if a certain software cannot be launched", etc.

[0052] A task node is a logical unit derived from a query question and used to perform specific processing operations. Each task node corresponds to an independent task that achieves the query question's objective. Task nodes are automatically generated by the server based on a preset task planning model or rules. A task node may include a task title and a task description, where the task description serves as a summary of the task's content.

[0053] Specifically, the terminal displays a query interface for inputting query questions. Users can input their query questions in the query interface displayed on the terminal. After the terminal obtains the query questions input by the user, it sends the query questions to the server. The server uses an intelligent agent to perform semantic parsing and task planning on the query questions, thereby obtaining multiple tasks to achieve the query question's objective. Based on these multiple tasks, the server generates corresponding task nodes and returns the task nodes to the terminal so that the terminal can display the task nodes.

[0054] The intelligent agent can be a type of intelligent agent built on a large language model (LLM), possessing the ability to autonomously understand, perceive, plan, remember, and use tools. In this embodiment of the disclosure, the intelligent agent can be implemented as a single intelligent agent based on a large language model, or it can be implemented by multiple intelligent agents based on a large language model cooperating with each other.

[0055] Task planning refers to the process by which an intelligent agent, after semantically parsing a query, breaks down the process of solving the query into several independently executable steps based on a pre-defined task generation model, task decomposition rules, or knowledge decision-making path. The result of task planning is the generation of multiple logically ordered and interrelated tasks.

[0056] A task refers to the specific execution steps that an intelligent agent breaks down into based on the goal of solving the query problem during the task planning process. Each task processes a specific part of the query problem and is used to complete the corresponding intermediate goal.

[0057] In an exemplary embodiment, the agent can employ a "thinking + action" reasoning pattern when determining the task for the query content. Thinking refers to the agent analyzing the current situation and considering what to do next, while action refers to the action performed by the agent, typically by invoking a tool. During the agent's task planning phase, the agent performs a round of thinking beforehand, generating various tasks to answer the query content. In the specific implementation, different query content can correspond to different prompt templates, and a prompt is constructed based on these templates. For example, when the query content is "How tall is Xiao A?", the corresponding prompt may include the following structure:

[0058] User: How tall is Little B?

[0059] The intelligent agent thinks: I need to search for Little B's height;

[0060] Intelligent agent actions: networked search;

[0061] The agent outputs: Little B's height;

[0062] Now let's start processing the new task: the user's query: How tall is Xiao A?

[0063] Guided by the aforementioned inference chain, the intelligent agent can automatically break down tasks such as "retrieving height information" and "extracting search results" and construct corresponding task nodes.

[0064] S204, execute the task represented by the task node and obtain the task execution result.

[0065] It should be noted that task execution refers to the actual execution of the operations corresponding to the task based on the processing objective represented by the task node. Task execution may include, but is not limited to, calling an intelligent agent to perform inference processing, calling external tools or interfaces to obtain data, retrieving database information, performing data operations or analysis, generating text content, performing logical judgments, and processing user context information.

[0066] Task execution results refer to the output obtained after the task is executed. Task execution results may include, but are not limited to, the following forms: text output generated by the agent (such as analysis conclusions, answer paragraphs, and suggestions); data returned by tool calls or interface calls (such as search results, query data, and calculation parameters); results of logical judgments (such as classification conclusions and status identifiers); and results of processing input data (such as summaries, extracted information, and structured information).

[0067] Specifically, the server determines the task type corresponding to the task node based on the task content it represents, and executes the processing operation matching that task. The server can execute tasks by calling an agent to perform inference operations, calling external tool interfaces to obtain data, performing database queries, performing logical judgments, running data analysis algorithms, or generating text content. After completing the above processing operations, the server obtains the task execution result corresponding to that task and uses this result as input for subsequent node evaluation or the processing of the next task node.

[0068] S206, Determine the node evaluation index value of the task node based on the task execution result.

[0069] Among them, the node evaluation index value refers to the evaluation value obtained by quantifying the degree of role, reliability or effectiveness of the task node in the process of solving the query problem after obtaining the task execution result corresponding to the task node, according to the preset evaluation rules or evaluation model. In other words, the node evaluation index value is used to characterize the quality performance or contribution performance of the task node.

[0070] Node evaluation metrics can include, but are not limited to, the following types: node contribution metrics, node confidence metrics, or other evaluation metrics. Node contribution metrics are used to characterize the importance of a task node in the final query result generation, for example, by analyzing the content and weight of the task execution results cited in the query results. Node confidence metrics are used to characterize the reliability, accuracy, or robustness of the task execution results of the task node, for example, by quantitative calculation based on the reflective text generated by the agent. Other evaluation metrics can be, for example, node execution stability metrics, task completion metrics, result credibility metrics, etc.

[0071] Specifically, after obtaining the task execution result corresponding to the task node, the server analyzes and processes the task execution result according to the preset evaluation model or evaluation rules to determine the node evaluation index value corresponding to the task node. The server can calculate the node contribution index value, node confidence index value, etc. based on the referenced content, semantic information, execution quality, reliability description or other quantifiable features in the task execution result. After the calculation is completed, the server associates and stores the obtained node evaluation index value with the task node so that it can be used to update the display style of the task node or to display the specific evaluation result when the user triggers the viewing operation.

[0072] S208, return the node evaluation index value to the terminal so that the terminal can update the display style of the task node based on the node evaluation index value, and display the node evaluation index value of the task node in response to the trigger operation on the task node.

[0073] The display style of a task node refers to the interface attribute that presents the visual effect of the task node, used to intuitively represent the status, importance, or evaluation result of the task node. The display style of a task node may include, but is not limited to, one or more of the following interface presentation forms: color attributes (such as color depth, color type, color gradient, etc.), brightness or transparency attributes, node size or node shape, node border style (such as thickness, solid or blurred, color, etc.), node animation effects (such as blinking, pulse, fade-in, flowing effects, etc.).

[0074] Triggering an action on a task node refers to an interactive operation performed on a specific task node on the terminal interface to view, expand, or obtain its node evaluation information. Triggering actions may include, but are not limited to: clicking on a task node, single or double-clicking on a task node area, performing a hover operation on a task node to trigger a floating tooltip, long-pressing a task node, or selecting a task node via keyboard, stylus, gestures, or other interactive methods.

[0075] Specifically, after determining the node evaluation metric value for any task node, the server returns the corresponding node evaluation metric value to the terminal. Upon receiving the node evaluation metric value returned by the server, the terminal can update the display style of the task node based on the node evaluation metric value, such as adjusting the task node's color, brightness, transparency, size, or animation effects, to intuitively reflect the importance or reliability of the task node in the query problem-solving process. When the user performs a trigger operation on the task node in the terminal interface, the terminal can display the node evaluation metric value of the task node in the interface, so that the user can view the specific evaluation results of the task node and further understand the role and contribution of the task node in the process of generating query results.

[0076] In the above task evaluation method, the receiving terminal sends a query question, returns the corresponding task node to the terminal so that the terminal can display the task node, execute the task represented by the task node, obtain the task execution result, determine the node evaluation index value of the task node based on the task execution result, and return the node evaluation index value to the terminal to update the display style of the task node. This is used to respond to the trigger operation of the task node and display the node evaluation index value of the task node, so that users can intuitively identify the importance and reliability of the task node in the process of generating query results in the interface. This solves the problem that traditional technologies cannot reveal the mechanism of task node operation, making it difficult for users to understand the decision logic of query results, and improves the transparency and interpretability of the task execution process.

[0077] In one embodiment, the node evaluation index value includes the node contribution index value; the process by which the server determines the node evaluation index value of the task node based on the task execution result includes the following steps: generating query results for the query question; determining the associated content corresponding to the target fragment in the query results in the task execution result; determining the importance score of the task node based on the query results and associated content; and determining the node contribution index value of the task node based on the importance score of the task node.

[0078] The node contribution index is a quantitative indicator that characterizes the degree of influence a task node has on the final query result. This index reflects the importance of the task node in the overall task chain and is typically calculated based on the degree to which the task execution result is referenced or used in the query results, as well as the importance weight of the referenced content. In other words, the higher the node contribution index value, the greater the significant contribution of the task node to the query results.

[0079] Query results refer to the final output content used to answer user queries, obtained after integrating, reasoning, or generating the task execution results corresponding to all task nodes. Query results can be presented in the form of reports, analytical text, structured content, question-and-answer text, or other formats.

[0080] A target fragment refers to a portion of the query results. This fragment typically contains key arguments, descriptive information, concluding content, or other semantic units that answer the user's question. In the node contribution evaluation process, the target fragment is used to locate the task execution results that the fragment depends on, in order to determine whether each task node provides supporting information for the fragment.

[0081] Related content refers to the information in the task execution results corresponding to a target segment in the query results that is referenced or used. Related content can include text descriptions, data results, model-generated content, analysis conclusions, etc. Related content is used to determine the specific degree of participation of the task node in the generation of query results and is the basic data for calculating importance scores.

[0082] Importance scoring refers to a quantitative score that determines the importance of a task node in generating query results based on the semantic, citation, or contribution relationships between query results and related content. Importance scoring can comprehensively consider factors such as the degree of support, irreplaceability, and contribution intensity of related content to the target fragment and the overall conclusion. The score can be a level, weight value, or numerical value, and is used as the basis for subsequent calculation of node contribution index values.

[0083] Specifically, after generating query results for the query question, the server parses the query results to identify the target fragment. Then, the server searches for related content corresponding to the target fragment in the task execution results of each task node to determine the source of the task execution results that each target fragment depends on. Based on the target fragment and its corresponding related content in the query results, the server performs an importance analysis on the task node that provides the related content and generates an importance score for the task node according to a preset importance assessment rule. After obtaining the importance score of each task node, the server determines the node contribution index value of each task node according to the ratio of the importance score of each task node to the importance scores of all task nodes, so as to characterize the degree of contribution of the task node to the generation of query results.

[0084] In the above embodiments, the server generates query results for the query question, determines the associated content corresponding to the target fragment in the query results within the task execution results, determines the importance score of the task node based on the query results and associated content, and determines the node contribution index value of the task node based on the importance score of the task node. This allows for a quantitative reflection of the actual contribution of each task node in the query result generation process, enabling a refined assessment of the importance of task nodes and improving the interpretability of the query task and the transparency of the result presentation.

[0085] In one embodiment, the query result also includes a reference identifier for the task execution result, and the number of task nodes is at least two. The process by which the server determines the associated content corresponding to the target fragment in the query result includes the following steps: obtaining the reference identifier of the target fragment in the query result; determining the target task node that provides data support for the target fragment based on the reference identifier among at least two task nodes; and extracting the corresponding associated content from the task execution result of the target task node based on the target fragment.

[0086] In this context, a reference identifier refers to specific marker information appended to the query results, used to indicate the task execution result corresponding to the task node referenced or dependent on by a target segment in the query results. Reference identifiers can take the form of numbers, annotation symbols, reference labels, or other forms, used to establish the correspondence between the content of the query results and the task execution results.

[0087] A target task node is a task node identified by a reference identifier among at least two task nodes that provides data or semantic support for a target fragment in the query results. The task execution result of the target task node contains the related content referenced by the target fragment and is the source information node that generated the target fragment.

[0088] Specifically, after generating the query results, the server parses the query results to obtain the reference identifiers carried by the target fragments in the query results. Subsequently, the server determines the target task node that provides data support for the target fragment among at least two task nodes, based on the reference relationships indicated by the reference identifiers. After determining the target task node, the server further extracts the associated content that matches the target fragment from the task execution results corresponding to the target task node based on the content characteristics of the target fragment, as an important basis for subsequently calculating the contribution of the target task node.

[0089] In one embodiment, after identifying the target task node, the server parses the task execution result corresponding to that target task node to obtain information content that can be used to support the query results. Subsequently, based on the semantic features of the target fragment, the server performs semantic comparison and relevance analysis on the task execution result to determine the result fragment that matches the target fragment semantically or in content, and extracts this result fragment as associated content corresponding to the target fragment. The associated content can be determined through keyword matching, similarity calculation, contextual relevance judgment, or semantic association analysis using a large language model, to ensure that the extracted associated content accurately reflects the supporting role of the target task node in the task execution result for the target fragment.

[0090] In the above embodiments, the server obtains the reference identifier of the target fragment in the query results, and determines the target task node that provides data support for the target fragment based on the reference identifier among at least two task nodes. Based on the target fragment, the server extracts the corresponding related content from the task execution results of the target task node, thereby achieving accurate positioning of the data source relationship between the query results and each task node. This enables the system to calculate the actual contribution of each task node in the query result generation process based on the real semantic support link, thereby significantly improving the accuracy and interpretability of node contribution evaluation.

[0091] In one embodiment, the process by which the terminal determines the importance score of a task node based on the query results and related content includes the following steps: generating a first instruction statement based on the query results, the target fragment, the related content, and the first instruction template; and performing an importance assessment task based on the first instruction statement to obtain the importance score of the target task node.

[0092] The first instruction template refers to a pre-defined text structure or prompt template used to guide the evaluation model (such as a large language model) in quantifying the importance of task nodes. This template contains placeholders for describing query results, target fragments, and related content, as well as rule-based statements to guide the model in evaluating according to pre-defined dimensions (such as argument coreness, value conversion degree, and irreplaceability).

[0093] The first instruction statement refers to the target instruction text generated after filling the actual data such as query results, target fragments and related content into the corresponding positions of the template based on the first instruction template. It is used to drive the evaluation model to perform importance analysis. The first instruction statement contains complete contextual information and clear scoring requirements, and can be directly used as input to the evaluation model to prompt the evaluation model to output the importance score of the corresponding task node.

[0094] Importance assessment task refers to the importance analysis process performed by an assessment model (such as a large language model) based on the first instruction statement. This task performs semantic understanding and judgment on the degree of role of the associated content provided by the task node in the query results, and generates an importance score for the target task node according to the preset assessment dimensions and scoring rules. Importance assessment task can output structured results, such as importance level, numerical score and brief reasoning, for subsequent node contribution calculation.

[0095] Specifically, after obtaining the query results, target fragments, and associated content corresponding to the target task nodes, the server fills the query results, target fragments, and associated content into the corresponding fields of the first instruction template according to the preset first instruction template, generating a first instruction statement to drive the evaluation model to perform analysis. The generated first instruction statement is input into the importance evaluation model (e.g., a large language model) to perform the importance evaluation task. The evaluation model performs semantic analysis on the degree of role of the associated content in the query results based on the context information in the first instruction statement and the evaluation dimensions defined in the template, and outputs the corresponding structured scoring results. The server extracts the importance score of the target task node from the scoring results for subsequent calculation of the node contribution index value.

[0096] like Figure 3 As shown, the first instruction template is:

[0097] # Role

[0098] You are a rigorous academic reviewer responsible for evaluating the importance of citations throughout research reports.

[0099] # Task

[0100] Assess the importance level of the following currently cited excerpt within the full report. Your assessment must consider the substantive contribution of the excerpt in conjunction with the cited content.

[0101] # Contextual Information

[0102] • Full report: {Insert full report}

[0103] • Current reference fragment: {Insert current reference fragment}

[0104] • Referenced content: {Insert referenced content}

[0105] #Evaluation Dimensions and Standards

[0106] Please analyze comprehensively:

[0107] 1. Core Argument: Is the information conveyed in this passage, derived from the cited content, the report's core argument (high), key evidence (medium), or background information (low)?

[0108] 2. Value Conversion: Does the current citation in the report directly quote the core conclusions / data from the cited content (high), or summarize / paraphrase it (medium), or only mention its peripheral information (low)?

[0109] 3. Irreplaceability: Would removing this section and the information it cites seriously impair the integrity and persuasiveness of the full report's conclusions (high), partially impair (medium), or slightly impair (low)?

[0110] #Importance Level Definition

[0111] • Key (weight=3): Rated "high" across multiple dimensions, it forms the cornerstone supporting the report's conclusions.

[0112] •Important (weight=2): Rated as “medium” in multiple dimensions, or a combination of “high” and “medium”, providing important support for the main argument.

[0113] • Auxiliary (weight=1): "Low" or "Medium" across all dimensions, and its removal does not affect the core logic. # Output Specification

[0114] Your final output must strictly contain only the following JSON objects:

[0115] {

[0116] "importance_level": "critical / important / auxiliary",

[0117] "reasoning": A brief analysis based on the above dimensions, not exceeding 100 words.

[0118] }

[0119] This first instruction template guides agents to assess the importance of cited content according to unified semantic analysis rules. The template explicitly defines the agent's role, assessment objectives, input format, analysis dimensions, and output specifications when performing importance assessment tasks, ensuring consistent evaluation standards and comparability across different task nodes. Furthermore, in this example, the first instruction template uses an "academic review expert" role, requiring the agent to conduct a comprehensive analysis based on the complete report, the current cited fragment, and the cited content. This enables the agent to determine the actual contribution of the cited content to the query results from three dimensions: argument coreness, value conversion degree, and irreplaceability. The template also pre-defines the logical definitions of importance levels (such as "critical," "important," and "auxiliary") and constrains the assessment results through a structured JSON output format, allowing the scoring results to be directly parsed by the server for subsequent contribution calculations. The first instruction statement generated by this template ensures a consistent evaluation process for importance assessment tasks across different query questions, task nodes, and citation relationships, thereby guaranteeing that the importance scoring of task nodes is scientific, stable, and quantifiable.

[0120] In the above embodiments, the server generates a first instruction statement based on the query results, target fragment, related content, and a first instruction template. Based on the first instruction statement, it performs an importance assessment task to obtain an importance score for the target task node. This enables a refined quantification of the importance of task nodes based on the actual contribution relationship at the semantic level, avoiding the contribution assessment bias caused by relying on simple counting or rule matching. It also enables accurate identification of the degree of influence of each task node during the query result generation process, improving the reliability and interpretability of the task assessment results.

[0121] In one embodiment, the number of task nodes is at least two; the process by which the terminal determines the node contribution index value of a task node based on the importance score of the task node includes the following steps: determining the total importance score based on the importance score of each task node; and determining the node contribution index value of each task node based on the total importance score and the importance score of each task node respectively.

[0122] The overall importance score refers to the total score obtained by summing the importance scores of at least two task nodes, which is used to reflect the overall importance level of all task nodes in the process of generating query results.

[0123] Specifically, after obtaining the importance scores for at least two task nodes, the server first sums the importance scores of each task node to obtain the overall importance score for all task nodes. Then, using the overall importance score as a normalization benchmark, the server calculates the ratio between the importance score of each task node and the overall importance score to obtain the node contribution index value for each task node. For example, the server can divide the importance score of a task node by the overall importance score and convert the ratio into a percentage to obtain the contribution ratio of that task node to the final query result.

[0124] In the above embodiments, the server determines the overall importance score based on the importance score of each task node, and determines the node contribution index value of each task node based on the overall importance score and the importance score of each task node. This enables the proportional calculation of the importance of different task nodes using a unified quantitative standard, and achieves an accurate characterization of the actual contribution of each task node in the query result generation process. This method avoids the contribution bias caused by the lack of a unified evaluation scale in traditional methods, and makes the node contribution evaluation results consistent, comparable, and interpretable. This is beneficial for the terminal to intuitively present key nodes and improve the interpretability of query tasks.

[0125] In one embodiment, the node evaluation index value includes the node confidence index value; the process by which the terminal determines the node evaluation index value of the task node based on the task execution result includes the following steps: obtaining the reflection text corresponding to the task node; the reflection text is generated after the task node completes the corresponding task; generating a second instruction statement based on the reflection text and the second instruction template; executing the confidence generation task based on the second instruction statement to obtain the node confidence index value of the task node.

[0126] The node confidence index value refers to a quantitative score obtained by analyzing the reflection text generated after a task node completes its corresponding task using a pre-defined confidence assessment model. It characterizes the reliability, consistency, and stability of the task execution result of that task node. The node confidence index value can be expressed numerically or in terms of levels, and can be used to measure the credibility of the node's output results, serving as an important basis for updating the task node's display style.

[0127] Reflective text refers to the self-evaluative natural language text generated by the agent based on the task execution results after a task node completes its corresponding task. This text typically includes descriptions of the execution process, output quality, potential uncertainties, possible limitations, and directions for improvement, reflecting the task node's subjective judgment and self-assessment of the reliability of its own output results.

[0128] The second instruction template refers to a pre-defined text structure or prompt template used to guide confidence assessment models (such as large language models) in analyzing reflective texts and generating confidence index values. The second instruction template includes placeholders for filling in the reflective text and scoring dimensions (such as depth of self-criticism, logical consistency, and clarity of improvement intentions), standardizing the input format for confidence assessment tasks and ensuring a consistent analytical framework across different task nodes.

[0129] The second instruction statement refers to the target instruction text used to drive the confidence assessment model to perform tasks, formed by filling the placeholders in the template with the reflection text of the corresponding task node, based on the second instruction template. The second instruction statement contains complete reflection text content and clear scoring requirements, and can be directly used as input for the confidence generation task to prompt the model to output the node confidence index value of the task node.

[0130] Specifically, after the server completes the task execution corresponding to the task node, it first drives the agent to generate reflection text to evaluate the output quality of this task based on the execution result of the task. The generated reflection text is then associated with and stored with the task node. According to the preset second instruction template, the reflection text is filled into the corresponding placeholder fields in the template to generate a second instruction statement for confidence assessment. The second instruction statement is then input into the confidence assessment model (e.g., a large language model) to execute the confidence generation task. The assessment model performs semantic analysis on the execution reliability of the task node based on the reflection content and scoring requirements in the second instruction statement and outputs a structured confidence result. The server extracts the node confidence index value of the task node from the assessment result.

[0131] like Figure 4 As shown, the second instruction template is used to evaluate the confidence level of the reflection text generated after the task node is completed. It is a preset instruction framework that drives the confidence level generation task. The second instruction template includes parts such as role setting, task definition, reflection text insertion position, evaluation dimension description, scoring principles, and output format constraints. Its specific form is as follows:

[0132] # Role

[0133] You are a rigorous quality evaluator responsible for assessing the reliability of an agent's work through self-reflection.

[0134] # Task

[0135] Analyze the self-reflection of the following agent and give it a confidence score of 0-100, representing the agent's confidence in the reliability and completeness of its output.

[0136] # Current Agent: {Insert description of the current Agent}

[0137] # Reflection Content

[0138] {Insert Reflection output text for the Agent node}

[0139] # Evaluation Dimensions and Standards

[0140] Please evaluate based on the following dimensions:

[0141] Depth of self-criticism: Did you honestly acknowledge uncertainties, knowledge limitations, or potential errors? (The deeper the reflection, the higher the score)

[0142] Logical consistency: Are the limitations or advantages mentioned in the reflection logically consistent with the output results? (The higher the consistency, the higher the score)

[0143] Clarity of Improvement Intent: Did the reflection propose specific and feasible directions for improvement or subsequent verification steps? (The clearer the intent, the higher the score)

[0144] # Scoring Principles

[0145] High score (81-100): Profound reflection, clear understanding of one's own boundaries, highly self-consistent logic, and high confidence in the results.

[0146] Middle part (41-80): The reflection has some content, but may be superficial or have slight logical doubts, with a moderate level of confidence.

[0147] Low score (0-40): The reflection is empty, avoids the problem, or has obvious logical contradictions with the output results, and has a low degree of confidence.

[0148] # Output Specification

[0149] Your final output must strictly contain only the following JSON objects:

[0150] {

[0151] "confidence_score": An integer from 0 to 100,

[0152] "reasoning": "The reasons for the rating based on the above dimensions, not exceeding 100 words."

[0153] }

[0154] In this example, the second instruction template is set up as a "quality evaluator". It requires the evaluation model (such as a large language model) to conduct a comprehensive analysis of the reflection content based on the node-based reflective text from multiple dimensions such as the depth of self-criticism, logical consistency and the clarity of improvement intention. This enables the evaluation model to comprehensively judge the reliability of the task node from the aspects of execution process, output quality and self-awareness. The second instruction template not only clarifies the semantic focus that the evaluation model needs to pay attention to, but also provides a unified standard for scoring principles, level division and output format. This makes the confidence results generated by different task nodes in different scenarios have consistent evaluation standards and comparability, which is conducive to building a stable and reliable node confidence index system.

[0155] In the above embodiments, the server obtains the reflection text corresponding to the task node, which is generated after the task node completes the corresponding task. Based on the reflection text and the second instruction template, a second instruction statement is generated. Based on the second instruction statement, a confidence generation task is executed to obtain the node confidence index value of the task node. This allows for a refined quantification of the reliability of the execution result based on the task node's self-quality assessment, improving the accuracy and semantic relevance of the confidence assessment result. This enables the system to present the node reliability level in a more intuitive way in the visualization display, and improves the transparency and interpretability of the query task execution process.

[0156] In one embodiment, the process of the terminal obtaining the reflection text corresponding to the task node includes the following steps: generating a third instruction statement based on the task execution result of the task node and the third instruction template; executing the reflection generation task based on the third instruction statement to obtain the node reflection text of the task node.

[0157] The third instruction template refers to a preset prompt structure used to guide the reflection generation model (such as a large language model) to automatically generate node reflection text based on the task execution results. This template contains multiple placeholders related to the quality of task execution, such as task objectives, execution process, generated content, potential uncertainties, and possible sources of errors. It is used to standardize the scope of reflection content generation, so that the reflection text generated by different task nodes has a consistent structure and semantic dimension, thereby facilitating subsequent confidence assessment.

[0158] The third instruction statement refers to the target input text generated by filling the corresponding position of the third instruction template with the task execution result after obtaining the task node's task execution result. The third instruction statement contains the key content of the task execution result and the reflection requirements defined in the template. It is the complete input information used to drive the reflection generation task and can enable the reflection generation model to generate interpretable reflection text based on the specific context.

[0159] Reflective generation tasks refer to the process of generating self-evaluation content by a reflective generation model (such as a large language model) based on a third instruction statement. This task automatically generates natural language reflective text describing the output quality, logical reliability, uncertainties, potential risks, and directions for improvement of the task by performing semantic analysis on the task execution results of the task nodes. This provides a semantic basis for subsequent calculation of the confidence index values ​​of the nodes.

[0160] Specifically, after the server completes the task execution corresponding to the task node, it first fills the corresponding placeholder of the third instruction template with the task execution result of the task node according to the preset third instruction template to generate a third instruction statement to drive the reflective generative model to perform self-evaluation. Then, the generated third instruction statement is sent as input to the reflective generative model (e.g., a large language model) to execute the reflective generative task. Based on the task execution content and reflection requirements in the third instruction statement, the reflective generative model performs semantic analysis on the execution process and output quality of the node to generate natural language reflective text that reflects the reliability and potential limitations of the node's execution.

[0161] In the above embodiments, the server generates a third instruction statement based on the task execution result of the task node and the third instruction template, and performs a reflection generation task based on the third instruction statement to obtain the node reflection text of the task node. This enables the automatic generation of structured and self-explanatory reflection information without human intervention, allowing the system to perform quantitative analysis of the execution quality of the task node based on objective semantic evidence. This improves the accuracy, consistency and automation level of node confidence assessment, and provides more sufficient data support for subsequent reliability demonstration and interpretability analysis.

[0162] In one embodiment, the above task evaluation method further includes the following steps: performing task planning processing on the query problem to obtain a task for completing the query problem; generating a node corresponding to the task to obtain a task node corresponding to the query problem.

[0163] Task planning and processing refers to the process by which the server, with the help of an intelligent agent, performs semantic parsing of the query and then automatically breaks down the overall solution process of the query into several independently executable tasks based on preset task decomposition rules, task generation models, or knowledge reasoning paths. Task planning and processing can include identifying the core objectives of the query, deriving the intermediate steps required to achieve those objectives, and determining the dependencies between the steps, thereby forming a structured and executable task flow. Through task planning and processing, complex queries can be decomposed into multiple logically clear and sequentially ordered tasks.

[0164] Specifically, after receiving the query from the terminal, the server first performs semantic parsing on the query through an intelligent agent to identify the overall goal of the query task, i.e., the query target. Then, based on a preset task planning model, task decomposition rules, or knowledge reasoning path, and in conjunction with the query target, the server performs task planning processing, automatically breaking down the query task's resolution process into several independently executable tasks. After obtaining each task, the server generates task nodes to represent each task based on the processing content, execution logic, and dependencies between tasks. Multiple task nodes are then organized into a task node set for the query task according to a hierarchical structure or logical order, and the task node set is returned to the terminal so that the terminal can display the structured execution process of the query task.

[0165] In the above embodiments, the server performs task planning processing on the query problem to complete the task of the query problem, generates the corresponding node of the task, and obtains the task node corresponding to the query problem. In this way, the complex query problem proposed by the user can be transformed into a structured and clear task execution process. This allows the system to decompose the query problem in a controllable, interpretable and executable manner, improves the accuracy and consistency of subsequent task execution, node evaluation calculation and visualization, and significantly enhances the transparency and interpretability of the entire query task process.

[0166] In one embodiment, the process of the server performing task planning processing on a query question to obtain tasks for completing the query question specifically includes the following steps: performing semantic parsing on the query question through a planning agent to obtain semantic parsing results; determining multiple tasks for completing the query target corresponding to the query question based on the semantic parsing results; executing the tasks represented by the task nodes to obtain task execution results, including: assigning corresponding execution agents to multiple tasks from multiple candidate execution agents; and executing the tasks corresponding to the task nodes through the execution agents to obtain task execution results.

[0167] Among them, the planning agent refers to an agent specifically responsible for semantic parsing of query questions. This agent uses natural language processing technology to understand the input query questions, analyze their goals and task breakdown requirements, and generate a structured task list to help the system transform complex problems into multiple executable tasks.

[0168] Candidate execution agents are a predefined set of agents capable of performing specific tasks. Each candidate execution agent possesses specific task processing capabilities and resources, such as data retrieval, reasoning, computational analysis, and text generation. They can select appropriate tools, interfaces, or algorithms to process tasks based on task type and execution requirements. Candidate execution agents are allocated according to their capabilities and task matching to ensure that tasks can be completed efficiently and accurately.

[0169] An execution agent is an agent responsible for executing the tasks assigned to it and returning the results. Each execution agent completes its task unit through specific execution logic (such as calling external tools, database queries, algorithm calculations, generating text, etc.) and feeds back the execution results to the server. It can be understood that the execution agent can complete tasks independently and can call different data sources, algorithms or interfaces according to different requirements of the task.

[0170] The query objective refers to the overall processing goal that the query question points to, that is, the total task that needs to be completed to meet the user's query needs. The query objective corresponds to the final solution goal that the user inputs for the query question, and it is usually composed of multiple logically related and collaborative tasks.

[0171] Specifically, after receiving the query from the terminal, the server first performs semantic parsing on the query using a planning agent to obtain a semantic parsing result. The semantic parsing result includes a structured understanding of the query's objective, core tasks, and related information. Based on the semantic parsing result, multiple tasks are determined to complete the query objective corresponding to the query. These tasks can process different aspects of the query, such as data querying, information reasoning, and logical judgment. These tasks are then assigned to corresponding execution agents from among multiple candidate execution agents. Each execution agent is responsible for executing a specific task based on its type and requirements. The execution agents can use relevant data sources, external interfaces, tools, or algorithms to process the task according to a preset execution strategy and generate task execution results.

[0172] In the above embodiments, the server performs semantic parsing on the query question by planning an intelligent agent, obtains the semantic parsing result, determines multiple tasks to complete the query target corresponding to the query question based on the semantic parsing result, assigns corresponding execution intelligent agents to each of the multiple tasks, and obtains the task execution result by executing the tasks corresponding to the task nodes through the execution intelligent agents. This achieves efficient decomposition and parallel processing of the query question. Through the cooperation of intelligent agents, the query task can be automatically decomposed into multiple independently executable sub-tasks, ensuring the efficiency and accuracy of task execution. Each execution intelligent agent performs specialized processing according to the task type, effectively improving the flexibility and response speed of task execution.

[0173] In one embodiment, such as Figure 5 As shown, a method for displaying task evaluation results is provided, which can be applied to... Figure 1 Taking server 104 as an example, the following steps are included:

[0174] S502, the query question has been obtained.

[0175] The query question refers to the content of the query entered by the user. The query question can cover user needs of any category or field, such as product comparison, knowledge Q&A, service consultation, process guidance, etc. For example, users can enter queries such as "Comparison between model A and model B mobile phones", "How to improve image clarity", "What to do if a certain software cannot be launched", etc.

[0176] Specifically, the terminal displays a query interface for inputting queries. Users can enter their queries on this interface. After receiving the query, the terminal sends it to the server. The server, through an intelligent agent, performs semantic parsing and task planning on the query, generating multiple tasks to achieve the query's objective. Based on these tasks, the server generates corresponding task nodes and returns them to the terminal for display. Furthermore, the server determines the task type based on the task content represented by the task node and executes processing operations matching that task. These operations can be performed by calling the intelligent agent to perform inference operations, calling external tool interfaces to obtain data, performing database queries, conducting logical judgments, running data analysis algorithms, or generating text content. The server completes the task execution in a specific manner. After performing the above processing operations, it obtains the task execution result corresponding to the task and uses this result as the input for subsequent node evaluation or the next task node processing. After obtaining the task execution result corresponding to the task node, the server analyzes and processes the result according to a preset evaluation model or evaluation rules to determine the node evaluation index value corresponding to the task node. The server can calculate the node contribution index value, node confidence index value, etc., based on the referenced content, semantic information, execution quality, reliability description, or other quantifiable features in the task execution result. After the calculation is completed, the server associates and stores the obtained node evaluation index value with the task node so that it can be used to update the display style of the task node or to display the specific evaluation result when the user triggers a viewing operation.

[0177] The intelligent agent can be a type of intelligent agent built on a large language model (LLM), possessing the ability to autonomously understand, perceive, plan, remember, and use tools. In this embodiment of the disclosure, the intelligent agent can be implemented as a single intelligent agent based on a large language model, or it can be implemented by multiple intelligent agents based on a large language model cooperating with each other.

[0178] Task planning refers to the process by which an intelligent agent, after semantically parsing a query, breaks down the process of solving the query into several independently executable steps based on a pre-defined task generation model, task decomposition rules, or knowledge decision-making path. The result of task planning is the generation of multiple logically ordered and interrelated tasks.

[0179] A task refers to the specific execution steps that an intelligent agent breaks down into based on the goal of solving the query problem during the task planning process. Each task processes a specific part of the query problem and is used to complete the corresponding intermediate goal.

[0180] A task node is a logical unit derived from a query question and used to perform specific processing operations. Each task node corresponds to an independent task that achieves the query question's objective. Task nodes are automatically generated by the server based on a preset task planning model or rules. A task node may include a task title and a task description, where the task description serves as a summary of the task's content.

[0181] Task execution refers to the actual execution of the operations corresponding to the task based on the processing objective represented by the task node. Task execution may include, but is not limited to, calling an intelligent agent to perform reasoning, calling external tools or interfaces to obtain data, retrieving database information, performing data operations or analysis, generating text content, performing logical judgments, and processing user context information.

[0182] Task execution results refer to the output obtained after the task is executed. Task execution results may include, but are not limited to, the following forms: text output generated by the agent (such as analysis conclusions, answer paragraphs, and suggestions); data returned by tool calls or interface calls (such as search results, query data, and calculation parameters); results of logical judgments (such as classification conclusions and status identifiers); and results of processing input data (such as summaries, extracted information, and structured information).

[0183] The node evaluation index value refers to the evaluation value obtained by quantifying the degree of role, reliability or effectiveness of the task node in the process of solving the query problem after obtaining the task execution result corresponding to the task node, according to the preset evaluation rules or evaluation model. In other words, the node evaluation index value is used to characterize the quality performance or contribution performance of the task node.

[0184] Node evaluation metrics can include, but are not limited to, the following types: node contribution metrics, node confidence metrics, or other evaluation metrics. Node contribution metrics are used to characterize the importance of a task node in the final query result generation, for example, by analyzing the content and weight of the task execution results cited in the query results. Node confidence metrics are used to characterize the reliability, accuracy, or robustness of the task execution results of the task node, for example, by quantitative calculation based on the reflective text generated by the agent. Other evaluation metrics can be, for example, node execution stability metrics, task completion metrics, result credibility metrics, etc.

[0185] S504 displays the task node corresponding to the query question.

[0186] Specifically, after receiving the task nodes returned by the server, the terminal displays the task nodes corresponding to the query question on the interface according to the structural information, hierarchical relationship, and preset interface display rules of the task nodes. The terminal can present multiple task nodes in a tree structure, process structure, graph node structure, or other visual layout, allowing users to intuitively view the task structure after the query question has been broken down. In addition, the terminal can also display the title, description, and other content of the task nodes when displaying them, so that users can understand the processing goal and scope of the task corresponding to that task node, thereby gaining a more comprehensive understanding of the task decomposition process of the query question.

[0187] like Figure 6 The diagram illustrates a query processing page in one embodiment. For the user's query "Comparison of Model A and Model B mobile phones," the server uses an intelligent agent to perform semantic analysis and task planning, breaking down the overall comparison requirement into several logically clear tasks, such as "Product Information Collection," "Product Filtering and Preliminary Information Collection," "Detailed Parameter and Specification Analysis," "Price Information Retrieval," "Product Parameter Comparison," "Product Performance Optimization Analysis," and "Result Generation." Based on the task node set returned by the server, the terminal displays each task node hierarchically, allowing users to intuitively identify the sequence and dependencies between different tasks. In the example interface, each task node is presented as a graph node, accompanied by a node title, execution time, and task description. For example, the "Product Information Collection" node collects configuration, function, appearance, and price information for Model A and Model B mobile phones; nodes such as "Parameter Comparison" and "Performance Optimization Analysis" are used for further structured comparative analysis. For task nodes with sub-nodes, such as "Product Filtering and Preliminary Information Collection," the terminal visualizes its subordinate tasks using a branch structure, allowing users to clearly understand the execution method and processing logic of the task chain. Through the above presentation method, the terminal can present the complex reasoning and multi-step processing of the query problem to the user in a clear and visual structure, so that the user can gain a more intuitive and transparent perception of the task planning and execution process without having to understand the reasoning mechanism of the underlying intelligent agent.

[0188] S506 After the task corresponding to the task node is completed, update the display style of the task node; the display style is related to the node evaluation index value corresponding to the task node.

[0189] The display style of a task node refers to the interface attribute that presents the visual effect of the task node, used to intuitively represent the status, importance, or evaluation result of the task node. The display style of a task node may include, but is not limited to, one or more of the following interface presentation forms: color attributes (such as color depth, color type, color gradient, etc.), brightness or transparency attributes, node size or node shape, node border style (such as thickness, solid or blurred, color, etc.), node animation effects (such as blinking, pulse, fade-in, flowing effects, etc.).

[0190] The display style being related to the node evaluation index value corresponding to the task node means that the interface display attributes of the task node are dynamically adjusted according to the node evaluation index value, so that the visual performance of the task node can reflect its importance, reliability or other evaluation results in the query problem solving process.

[0191] Specifically, after receiving the node evaluation index value of the task node returned by the server, the terminal determines the target display style corresponding to that node evaluation index value according to preset display mapping rules. For example, the terminal can select different colors, brightness, transparency, size, or animation effects based on the magnitude of the node evaluation index value to represent the importance or reliability of the task node in the query problem-solving process. After determining the target display style, the terminal updates the style of the corresponding task node in the interface so that the visual appearance of the task node matches its node evaluation index value, thereby enabling the user to intuitively identify the evaluation status of the task node through the interface display.

[0192] S508, in response to a trigger operation on a task node, displays the node evaluation metric value of the task node.

[0193] In this context, triggering an operation on a task node refers to an interactive operation performed on a specific task node on the terminal interface to view, expand, or obtain its node evaluation information. Triggering operations may include, but are not limited to: clicking on a task node, single or double-clicking on a task node area, performing a hover operation on a task node to trigger a floating tooltip, long-pressing a task node, or selecting a task node via keyboard, stylus, gestures, or other interactive methods.

[0194] Specifically, when a user performs a trigger operation on the task node in the terminal interface, the terminal can display the node evaluation index value of the task node in the interface, so that the user can view the specific evaluation results of the task node and further understand the role and contribution of the task node in the process of generating query results.

[0195] The above-mentioned method for displaying task evaluation results, after obtaining the query question, displays the task node corresponding to the query question. After the task corresponding to the task node is completed, the display style of the task node is updated based on the received node evaluation index value. In response to the trigger operation of the task node, the node evaluation index value of the task node is displayed. The node evaluation index value represents the evaluation result of the task node's task execution. This allows users to intuitively identify the importance and reliability of the task node in the process of generating query results in the interface. This solves the problem that traditional technologies cannot reveal the mechanism of task node operation, making it difficult for users to understand the decision logic of query results, and improves the transparency and interpretability of the task execution process.

[0196] In one embodiment, the process of the terminal displaying the task node corresponding to the query question includes the following steps: displaying the task node corresponding to the query question according to a preset display style; the process of the terminal updating the display style of the task node includes the following steps: determining the attribute value of the display attribute of the task node based on the node evaluation index value corresponding to the task node; updating the preset display style to the target display style corresponding to the attribute value of the display attribute.

[0197] The preset display style refers to the default visual presentation method used for the initial display of task nodes. The preset display style can include the basic appearance attributes of task nodes, such as default color, brightness, transparency, size, border style or layout.

[0198] Display attributes refer to adjustable visual features used to control the presentation of task nodes in the interface, reflecting the node evaluation index value corresponding to the task node. Display attributes may include, but are not limited to, color attributes (such as color type, color depth, color gradient), brightness attributes, transparency attributes, size attributes, border style attributes, or animation effect attributes, which are used to visually distinguish the evaluation status or importance of different task nodes.

[0199] Display attribute values ​​are numerical values ​​or levels determined based on the node evaluation metric values ​​corresponding to the task node, used to assign specific display effects to the task node. For example, the color attribute value can be a color code or color depth level representing a color gradient, the size attribute value can be a numerical range of the node's display size, and the transparency attribute value can be a transparency coefficient. The display attribute values ​​reflect the magnitude changes in the node's evaluation metric values, enabling different task nodes to be distinguished through visual differences.

[0200] The target display style refers to the final visual presentation of the task node after the attribute values ​​of the display attributes are determined. The target display style is determined by the corresponding display attributes and their attribute values, such as target color style, target brightness style, target transparency style, or target size style, so as to intuitively reflect the task evaluation results of the task node and enable users to quickly identify the importance or reliability of the node through the interface visual effects.

[0201] Specifically, after receiving the task nodes returned by the server, the terminal first performs initial rendering of each task node according to the preset display style, presenting the task nodes in the interface with default visual attributes such as color, size, transparency, or shape. Then, when the terminal receives the node evaluation index value corresponding to the task node returned by the server, the terminal calculates or looks up the attribute value of the corresponding display attribute based on the index value, such as determining the color gradient value, brightness intensity, transparency level, or node size ratio. After determining the above attribute values, the terminal replaces the original preset display style with the target display style corresponding to the attribute value to update the interface visual effect of the task node, so that the task node can present different display states according to the node evaluation index value, thereby making it easier for users to intuitively identify the importance or reliability of each task node.

[0202] In the above embodiments, the terminal displays the task node corresponding to the query question according to a preset display style, and determines the attribute value of the display attribute of the task node based on the node evaluation index value corresponding to the task node. The preset display style is then updated to the target display style corresponding to the attribute value of the display attribute. This enables the interface display of the task node to dynamically reflect its evaluation results, allowing users to intuitively identify the importance or reliability of the node through visual differences without having to view the underlying data. This improves the interpretability and interaction efficiency of the task execution process, and enhances the user's understanding and operating experience of the multi-node task structure.

[0203] In one embodiment, the display attributes include color attributes, and the preset display style is a preset color style. The process by which the terminal determines the attribute value of the display attributes of the task node based on the node evaluation index value corresponding to the task node includes the following steps: determining the color gradient value of the task node based on the node evaluation index value. The process by which the terminal updates the preset display style to the target display style corresponding to the attribute value of the display attributes includes the following steps: generating the target color style based on the color gradient value; and replacing the preset color style with the target color style.

[0204] Color attributes refer to the display attributes used to control the color characteristics of task nodes in the interface. They are used to intuitively reflect the differences in the node evaluation index values ​​(such as contribution and confidence) corresponding to the task nodes through color changes. Color attributes can include parameters such as color type, color depth, color brightness, hue value, saturation, lightness, or color gradient.

[0205] Preset color style refers to the default color configuration used to initially display task nodes before obtaining node evaluation index values. Specifically, it can be a uniform base color, such as light gray, default blue, or other system default colors, to ensure that task nodes are displayed with a consistent color style during the initial rendering stage, without reflecting any evaluation results.

[0206] Color gradient values ​​are numerical parameters used to map and display colors, calculated based on the node evaluation index values ​​corresponding to the task nodes. In other words, color gradient values ​​from low to high correspond to different positions of the color in the gradient range, such as from light to dark or from cool to warm colors, to characterize the differences in contribution or confidence of different nodes. They are the basic data used to generate the target color style.

[0207] The target color style refers to the final color display effect determined based on the color gradient value and applied to the task node. The target color style corresponds to the specific color of the color gradient value in the preset color mapping rules, such as dark red, bright yellow, or low-saturation blue.

[0208] Specifically, after receiving the node evaluation index value corresponding to the task node returned by the server, the terminal first converts the node evaluation index value into the corresponding color gradient value according to the preset color mapping rules. For example, the node evaluation index value with a higher value is mapped to a darker or brighter color gradient position in the color gradient range. Then, the terminal finds or calculates the corresponding target color style according to the determined color gradient value. For example, it generates a color code, hue value or brightness parameter that matches the gradient position, and thus forms the target color style. After generating the target color style, the original preset color style of the task node is replaced with the target color style to update the color display effect of the task node in the interface, so that the visual presentation of the task node can dynamically reflect its node evaluation index value.

[0209] In the above embodiments, the terminal determines the color gradient value of the task node based on the node evaluation index value, generates a target color style based on the color gradient value, and replaces the preset color style with the target color style. This allows the task node to intuitively reflect the differences in its node evaluation index value through color depth, brightness, or hue changes, enabling users to quickly perceive the importance or reliability of the task node without reading text descriptions. This significantly improves the visualization effect, information transmission efficiency, and system interpretability of the query task execution process.

[0210] In one embodiment, the preset display style includes a first preset display style and a second preset display style; the process of the terminal displaying the task node corresponding to the query question according to the preset display style includes the following steps: displaying the task node corresponding to the query question according to the first preset display style corresponding to the node contribution dimension; the above-mentioned method for displaying the task evaluation result further includes the following steps: in response to the indicator value switching operation, displaying the task node corresponding to the query question according to the second preset display style corresponding to the node confidence dimension.

[0211] The first preset display style refers to the default visual presentation method used when displaying task nodes in terms of node contribution. This style is designed based on the node contribution index value and may include color gradient, brightness difference, node size variation, border emphasis or other visual differentiation methods.

[0212] The node contribution dimension refers to the evaluation dimension used to assess and demonstrate the size or importance of the role played by each task node in the process of generating query results. This dimension is calculated based on indicators such as the reference weight, semantic value, or irreplaceability of the content provided by the node in the final query results, and is used to characterize the contribution level of the task node to the final generated results.

[0213] The second preset display style refers to the default visual presentation method used when displaying task nodes in terms of node confidence. This style is associated with the confidence index value and can express the reliability of the task node execution results through color depth, transparency, brightness or other visual differences.

[0214] The node confidence dimension is an evaluation dimension used to assess and display the reliability, consistency, and robustness of the task execution results corresponding to the task node. This dimension is based on the analysis of the reflective text generated by the agent and is used to quantify the credibility of the task node in terms of execution quality.

[0215] The indicator value switching operation refers to the interactive command on the terminal interface to switch the display dimension of a task node. For example, by clicking the switch button, selecting a drop-down option, or sliding the control bar, the display method of the task node can be switched from contribution-based display to confidence-based display, or vice versa. This operation triggers the terminal to re-render the task node with the corresponding preset display style, realizing a multi-dimensional visualization.

[0216] Specifically, after receiving the task node and its corresponding contribution index value returned by the server, the terminal first renders the task node according to the first preset display style corresponding to the node contribution dimension. For example, it determines visual attributes such as node color gradient, brightness level, or node size based on the contribution level, thereby displaying the task node in the interface in the form of contribution dimension. Subsequently, when the terminal detects that the user has performed an index value switching operation, the terminal regenerates the display effect of the task node according to the second preset display style corresponding to the node confidence dimension. For example, it determines the color depth, transparency, or brightness change based on the node confidence index value, replaces the original contribution display style with the confidence display style, and renders the task node with the new visual effect, allowing the user to switch and view the task evaluation results under different dimensions.

[0217] In the above embodiments, the terminal displays the task node corresponding to the query question according to the first preset display style corresponding to the node contribution dimension, and in response to the indicator value switching operation, displays the task node corresponding to the query question according to the second preset display style corresponding to the node confidence dimension. This allows users to flexibly switch between different evaluation dimensions to view the display effect of the task node, realize multi-dimensional visualization analysis of the task node, and enable the system to present the degree of role and reliability of the task node in a more intuitive and understandable way, thereby improving the user's understanding of the query task execution process and decision support capabilities.

[0218] In one embodiment, the process of a terminal displaying the node evaluation index value of a task node in response to a trigger operation on the task node includes the following steps: in response to a trigger operation on the task node, displaying the node evaluation index value of the task node and the evaluation description information corresponding to the task node.

[0219] The evaluation description information refers to the textual or structured explanation of the node evaluation index values ​​used to help users understand the task nodes. It includes the basis, semantic meaning, scoring reasons or analysis summary of the node contribution index value, node confidence index value or other evaluation index values. The evaluation description information can come from the analysis explanation output by the evaluation model (such as a large language model) or can be automatically generated by the server according to preset rules. It is used to explain the reasons for the high or low values ​​of node index values, the role and positioning of the node in the query results, the reliability analysis of the node execution results, etc.

[0220] Specifically, after the terminal detects a trigger operation (such as clicking, long-pressing, hovering, or other interactive actions) performed by the user on any task node, it obtains the node evaluation index value and evaluation description information corresponding to the task node from the server or reads it from the local cache. The terminal then displays the node evaluation index value of the task node in the interface in a preset display method (such as a pop-up prompt box, side information panel, or floating label), and at the same time presents the evaluation description information used to explain the index value, so that the user can understand the scoring basis, node execution quality, and its role in the overall task while viewing the numerical evaluation results.

[0221] In the above embodiments, the terminal displays the node evaluation index value of the task node and the evaluation description information corresponding to the task node in response to the trigger operation of the task node. This enables the user to obtain the quantitative evaluation results and semantic explanation of the task node in real time during the interaction, improves the readability and understandability of the evaluation information, and enables the user to more accurately judge the role and reliability of the task node in the query task execution process, further enhancing the interpretability of the system and the user operation experience.

[0222] In one embodiment, the number of task nodes is at least two, and the method for displaying the task evaluation result further includes the following steps: after the task corresponding to the first task node is completed in the at least two task nodes, the data flow object between the first task node and the second task node is displayed, and the second task node performs subsequent processing based on the task execution result of the first task node.

[0223] Among them, the first task node refers to the task node whose corresponding task has been completed and has produced a task execution result among at least two task nodes. Specifically, the first task node can be located in the preceding position of the task flow, and its task execution result can provide data support or logical basis for subsequent task nodes. It is the upstream node that forms the data flow relationship between nodes.

[0224] The second task node refers to a task node that continues to perform subsequent processing operations based on the task execution result of the first task node among at least two task nodes. The second task node is located downstream of the task flow, and its input depends on the output of the first task node. It is the target node or inflow node that forms a data flow path with the first task node.

[0225] A data flow object is a visual graphic element used in the interface to represent the data transfer relationship between the first task node and the second task node. The data flow object is used to show the flow direction and dependency relationship of the task execution results between task nodes. Its form can include, but is not limited to, data flow lines, lines with arrows, animated lines with dynamic gradient effects or moving trajectory points, dashed lines or highlighted paths, etc.

[0226] Specifically, after receiving information about multiple task nodes and their execution relationships from the server, the terminal determines the second task node that has a data dependency on the first task node when the task corresponding to the first task node is completed, based on the node dependency relationship provided by the server. The terminal then generates and displays a data flow object between the first and second task nodes on the interface, such as in the form of a line, arrow, or a flow line with animation effects, to represent the data transmission path from the first task node to the second task node. In addition, the terminal can also trigger the subsequent display logic of the second task node based on the task execution result of the first task node, so that the user can intuitively view the data flow relationship and processing order between task nodes.

[0227] like Figure 7The diagram above illustrates a query processing page in one embodiment. In this example, for the query "Comparison of Model A and Model B mobile phones," the server generates multiple task nodes with dependencies based on task planning results. These include "Product Information Collection," "Product Filtering and Preliminary Information Collection," "Detailed Parameter and Function Analysis," "Price Information Retrieval," "Product Parameter and Function Comparison," "Product Cost-Effectiveness Analysis," and "Generate Comparison Analysis Results." These task nodes are visualized in a tree structure according to the sequence of tasks, facilitating user understanding of the entire query processing path. In the illustrated process, after the "Product Information Collection" node completes execution and generates relevant task results, this node becomes the first task node, and its output is used by subsequent second task nodes such as "Product Filtering and Preliminary Information Collection" and "Detailed Parameter and Function Analysis." After detecting the completion of the first task node's task execution, the terminal draws corresponding data flow objects on the interface based on node dependencies to display the data between the first and second task nodes. The terminal generates arrowed lines, data flow lines, or dynamic point movement trajectory lines between nodes such as "Product Information Collection," "Product Filtering and Preliminary Information Collection," and "Detailed Parameters and Functional Analysis." This visually represents the process of task execution results flowing from upstream to downstream nodes. For example, after the "Product Filtering and Preliminary Information Collection" node completes its execution, it acts as the first task node, further transmitting the processing results to second task nodes such as "Detailed Parameters and Functional Analysis," "Price Information Retrieval," and "Product Parameter and Functional Comparison." The terminal also draws data flow objects between these nodes, allowing users to clearly see how information passes through multi-dimensional analysis nodes, ultimately converging at the "Product Cost-Effectiveness Analysis" node and flowing further to the "Generate Comparative Analysis Results" node. Through this data flow display method, users can not only see the hierarchical structure of task nodes but also obtain dynamic visual feedback on the data flow between nodes. This makes the task execution path, node dependencies, and data transmission logic more intuitive and transparent, helping to improve the understandability and explainability of the query processing process.

[0228] In the above embodiments, after the task corresponding to the first task node in at least two task nodes is completed, the terminal displays the data flow object between the first task node and the second task node. The second task node performs subsequent processing based on the task execution result of the first task node. This allows the data dependency relationship and processing order between task nodes to be presented in a visual way, so that users can intuitively grasp the execution path of the task chain without understanding the underlying scheduling logic. This improves the transparency and interpretability of the task execution process, helps users quickly identify key nodes and information flow in the task process, and improves the interactive experience and usability of the system.

[0229] In one embodiment, the node evaluation index value includes the node contribution index value; the data flow object is a data flow line; the above task evaluation result display method further includes the following steps: displaying a data flow animation at the corresponding position of the data flow line; the display intensity of the data flow animation is related to the node contribution index value of the first task node associated with the data flow line.

[0230] Among them, the data flow line refers to the visual connection object used to represent the data transmission path between task nodes. It is used to intuitively display the directional relationship of the first task node transmitting the task execution result to the second task node in the interface. The data flow line can be a straight line, a curve, a line with an arrow, a dashed line, or other visual line segment forms. Its function is to represent the logical dependency or data flow path between task nodes, so that users can clearly identify the upstream and downstream relationships in the task link.

[0231] Data flow animation refers to dynamic visualization effects based on data flow lines. For example, animation elements such as moving light spots, flowing particles, brightness gradients, and trajectory lines are superimposed on the data flow lines to simulate the process of data flowing between nodes.

[0232] The display intensity of a data stream animation refers to parameters used to characterize the dynamic effect of the animation in its visual presentation. These parameters include the animation's brightness, transparency, color saturation, speed, repetition frequency, and particle density. Display intensity is related to the node contribution index; the higher the node contribution, the brighter the data stream animation, the denser the particles, or the faster the flow, thus visually distinguishing the importance of different nodes in the data flow process.

[0233] Specifically, after displaying the data flow line between the first and second task nodes, the terminal generates a data flow animation effect to be displayed at the location of the data flow line based on the task node contribution index value returned by the server. The terminal can draw trajectory points, particle light flow, brightness gradient stripes, or other dynamic elements that move over time on the data flow line to simulate the process of data flowing from the first task node to the second task node. When rendering the data flow animation, the terminal can adjust the display intensity of the animation according to the contribution index value of the first task node. For example, it can increase the brightness, flow speed, or particle density of the data flow animation corresponding to high contribution nodes, so that the animation effect can better highlight the data flow path of high contribution nodes and intuitively show the importance of different nodes in the task processing process.

[0234] In the above embodiments, the terminal displays a data flow animation at the corresponding position of the data flow line, and the display intensity of the data flow animation is related to the node contribution index value of the first task node associated with the data flow line. This allows the terminal to highlight the differences in contribution of different task nodes in the data flow process in a dynamic visual form, enabling users to intuitively identify the data output intensity and influence of key nodes through visual changes such as animation brightness, speed, or particle density. This enhances the perceptibility and interpretability of the data flow relationship between task nodes, and improves the visualization effect and user understanding efficiency of the multi-node collaborative processing flow.

[0235] In one embodiment, the number of task nodes is at least two, and the method for displaying the task evaluation results further includes: in response to a node sorting operation, displaying a sorting result based on the node evaluation index value for each task node.

[0236] The node sorting operation refers to the interactive command used to adjust the display order of task nodes. This operation can be triggered in various ways, including but not limited to: clicking the sort button, selecting a sorting dimension (such as contribution sorting, confidence sorting), switching the sorting method (ascending order, descending order) in the sorting menu, triggering a sorting request through dragging or gesture operations in the visual interface, and automatically triggering sorting when preset conditions are met.

[0237] The sorting result refers to the sorting information obtained by sorting at least two task nodes according to the sorting method specified in the node sorting operation, based on the node evaluation index values ​​corresponding to each task node. The sorting result may include, but is not limited to, the following: the sorted list of task nodes, the updated node display order or interface layout (such as rearranging node positions, adjusting the node hierarchy in the tree diagram, updating the left-right or top-bottom display order of nodes), and sorting status information used for interface display (such as sorting direction indicators (↑ / ↓), and selected sorting dimension labels).

[0238] Specifically, after receiving a node sorting operation initiated by the user, the terminal obtains or reads the node evaluation index values ​​corresponding to each task node from the server according to the selected sorting dimension, and sorts at least two task nodes according to a preset sorting rule (e.g., from high to low or from low to high). Based on the sorting results, the terminal generates a new node arrangement order or interface layout, and updates the sorted task nodes in the interface, allowing the user to view each task node in the sorted order. The terminal can also present a sorting direction indicator or a current sorting dimension label in the sorting results to indicate the current sorting basis, making it easier for users to quickly understand the sorting logic and locate the node with the highest contribution or the risk node with the lowest confidence.

[0239] In the above embodiments, the terminal responds to the node sorting operation and displays the sorting results of each task node based on the node evaluation index value, so that users can quickly identify key nodes, auxiliary nodes or potential risk nodes according to the level of contribution or confidence, improve the interpretability of query tasks and information acquisition efficiency, and reduce the important cost of users manually searching and judging in complex task structures, further enhancing the visualization effect and interactive experience of task evaluation results.

[0240] In one embodiment, the method for displaying the task evaluation results further includes the following steps: in response to a node filtering operation, displaying the filtering results after filtering at least two task nodes based on node evaluation index values; the node filtering operation is used to instruct the terminal to filter task nodes according to preset filtering conditions.

[0241] The node filtering operation refers to the interactive command that filters the display range of task nodes. This operation can be triggered in various ways, including but not limited to: clicking the filter button or filter menu, selecting the filter dimension in the filter panel (such as filtering by contribution or by confidence level), entering or adjusting the filter threshold (such as setting the contribution level to be greater than 20% or the confidence level to be less than 60%), selecting the preset filter mode (such as "show key nodes", "show risk nodes", "show only high contribution nodes"), or selecting the filter range through gestures or sliders.

[0242] Preset filtering conditions refer to the filtering rules or constraint parameters used to determine whether a task node meets the display requirements. Filtering conditions may include: numerical thresholds based on node evaluation indicators, composite conditions based on combinations of multiple indicators, filtering based on sorting position, and filtering based on classification labels or features. For example, the filtering condition may be a contribution rate ≥ X%, or a filtering condition may be nodes with high contribution rate and high confidence, or a filtering condition may be to display the M nodes with the lowest confidence, or a filtering condition may be to select task nodes of the target type.

[0243] Specifically, after receiving a node filtering operation initiated by the user, the terminal obtains the node evaluation index value corresponding to each task node and compares the node evaluation index value with the preset filtering conditions. For task nodes that meet the filtering conditions, the terminal continues to display them in the normal style on the interface. For task nodes that do not meet the filtering conditions, the terminal performs processing methods such as hiding the display, reducing the transparency, weakening the color, collapsing the node, or other weakening the display to generate a set of filtered nodes. Subsequently, the terminal presents the filtering results on the interface, allowing the user to view only the target nodes that meet the filtering conditions, thereby quickly focusing on high contribution nodes, low confidence nodes, or other key nodes with specific characteristics.

[0244] In the above embodiments, the terminal displays the filtering results after filtering at least two task nodes based on node evaluation index values ​​in response to the node filtering operation. This enables users to quickly focus on key task nodes that meet specific contribution or confidence conditions in complex task structures, effectively reducing the interference of irrelevant information in the interface, improving the efficiency of users in identifying high-value nodes or potential risk nodes, and further enhancing the visualization and analysis capabilities of task evaluation results and the interpretability of task execution process.

[0245] In one embodiment, a node evaluation system is also provided, such as Figure 8 As shown, the node evaluation system includes:

[0246] 1. A multi-agent task execution module is used to automatically break down user-input queries according to a preset task planning strategy, generating multiple tasks. Multiple agents then execute these tasks to obtain the corresponding results. This module supports parallel or serial task scheduling to ensure that complex queries can be processed efficiently.

[0247] 2. The data results statistics module is used to collect, archive, and standardize the execution results of multiple tasks, forming a unified data structure that can be used for subsequent evaluation. This data structure may include, but is not limited to, information such as: task node identifiers, task output text, citation markers, and reflection text generation status.

[0248] 3. Data extraction module, used to extract two types of key data from the task execution results:

[0249] (1) Citation data: used to characterize the citation status of task execution results in the final query report, including the location of the citation fragment, the source of the corresponding task node, etc.;

[0250] (2) Reflection data: used to characterize the self-reflection text generated by the agent after the task node completes the task, and is the basic data for subsequent confidence assessment.

[0251] 4. The analysis engine module integrates a large language model (LLM) for comprehensive semantic analysis of task nodes based on citation and reflection data. The analysis engine module can call the contribution calculation model and the confidence calculation model respectively, generate corresponding analysis command statements according to preset prompt templates, and execute importance level assessment and confidence assessment tasks.

[0252] 5. The contribution calculation module is used to determine the importance score of each task node based on the semantic analysis results of the cited fragments and content using LLM, and further generate a node contribution index value. The node contribution index value is used to characterize the degree of influence of the task node on the formation of the conclusion in the final query report.

[0253] 6. Confidence Calculation Module: Based on the LLM's analysis results of the reflective text, this module calculates the node confidence index value for each task node according to a preset confidence calculation template. The node confidence index value reflects the reliability, consistency, and stability of the task execution results.

[0254] 7. A visual interactive front-end module is used to visually represent task nodes based on contribution and confidence index values. This module can apply visual effects such as dynamic coloring, brightness adjustment, transparency changes, and node size changes to nodes based on their index values. Simultaneously, the trigger area of ​​the task node supports user interaction, displaying corresponding index values ​​and evaluation descriptions when the user interacts with the node.

[0255] 8. Coloring Protocol Layer Module: This module defines the mapping rules for the visual attributes of nodes. It maps the contribution or confidence index values ​​to the corresponding color gradient, brightness, or transparency according to the set gradient color levels, thereby generating the final display style of the nodes.

[0256] 9. The dynamic coloring encoding module is used to perform real-time animation rendering of nodes and their data flow lines in the front-end visualization interface based on the output of the coloring protocol layer module. For example, when a task node has a high contribution, this module can enhance the color saturation of the node or thicken its associated data flow line.

[0257] 10. The rendering visualization encoder module is used to perform unified graphical rendering of task nodes, data flow objects and their dynamic effects, and present a visual layout of tree structure, process structure or graph node structure on the interface, so that users can intuitively understand the hierarchical relationship of task nodes, execution link and evaluation results.

[0258] 11. Tooltip module: When a user performs interactive operations such as hovering or clicking on a specific task node, the tooltip module displays the corresponding contribution index value, confidence index value, and generated evaluation description text, thereby improving the interpretability of the system.

[0259] Through the coordinated operation of the above modules, the node evaluation system provided in this embodiment can quantitatively analyze the role, importance and execution reliability of each task node in a multi-Agent task execution framework, and display it on the front-end interface in a dynamic and interactive manner, thereby significantly improving the transparency and understandability of complex task processing and enhancing the user's ability to grasp the system's reasoning chain.

[0260] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0261] Based on the same inventive concept, this application also provides a task evaluation apparatus for implementing the task evaluation method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more task evaluation apparatus embodiments provided below can be found in the limitations of the task evaluation method described above, and will not be repeated here.

[0262] In one embodiment, such as Figure 9 As shown, a task evaluation device is provided, including: a query question receiving module 902, a task execution module 904, an indicator value determination module 906, and an indicator value return module 908, wherein:

[0263] The query question receiving module 902 is used to receive query questions sent by the terminal and return the corresponding task node to the terminal so that the terminal can display the task node;

[0264] Task execution module 904 is used to execute the tasks represented by the task nodes and obtain the task execution results;

[0265] The indicator value determination module 906 is used to determine the node evaluation indicator value of the task node based on the task execution result;

[0266] The indicator value return module 908 is used to return the node evaluation indicator value to the terminal so that the terminal can update the display style of the task node based on the node evaluation indicator value, and display the node evaluation indicator value of the task node in response to the trigger operation of the task node.

[0267] In one embodiment, the node evaluation index value includes the node contribution index value; the index value determination module 906 is used to: generate query results for the query question; determine the associated content corresponding to the target fragment in the query results in the task execution results; determine the importance score of the task node based on the query results and associated content; and determine the node contribution index value of the task node based on the importance score of the task node.

[0268] In one embodiment, the query result also includes a reference identifier for the task execution result, and the number of task nodes is at least two; the indicator value determination module 906 is further configured to: obtain the reference identifier of the target fragment in the query result; determine the target task node that provides data support for the target fragment based on the reference identifier among the at least two task nodes; and extract the corresponding related content from the task execution result of the target task node based on the target fragment.

[0269] In one embodiment, the indicator value determination module 906 is used to: generate a first instruction statement based on the query results, target fragment, related content and a first instruction template; and perform an importance assessment task based on the first instruction statement to obtain an importance score for the target task node.

[0270] In one embodiment, the number of task nodes is at least two; the indicator value determination module 906 is used to: determine the total importance score based on the importance score of each task node; and determine the node contribution indicator value of each task node based on the total importance score and the importance score of each task node respectively.

[0271] In one embodiment, the node evaluation index value includes the node confidence index value; the index value determination module 906 is used to: obtain the reflection text corresponding to the task node; the reflection text is generated after the task node completes the corresponding task; generate a second instruction statement based on the reflection text and the second instruction template; and execute the confidence generation task based on the second instruction statement to obtain the node confidence index value of the task node.

[0272] In one embodiment, the indicator value determination module 906 is used to: generate a third instruction statement based on the task execution result of the task node and the third instruction template; and execute a reflection generation task based on the third instruction statement to obtain the node reflection text of the task node.

[0273] In one embodiment, such as Figure 10 As shown, the above-mentioned device also includes a task planning module 910, which is used to: perform task planning processing on the query task corresponding to the query question to obtain the task of the query task; generate the node corresponding to the task to obtain the task node of the query task.

[0274] In one embodiment, the task planning module 910 is further configured to: perform semantic parsing on the query question through a planning agent to obtain a semantic parsing result; determine multiple tasks for completing the query target corresponding to the query question based on the semantic parsing result; assign corresponding execution agents to the multiple tasks respectively; and the task execution module 904 is further configured to execute the tasks corresponding to the task nodes through the execution agents to obtain task execution results.

[0275] In one embodiment, such as Figure 11 As shown, a task evaluation result display device is provided, including: a query question acquisition module 1102, a task node display module 1104, and an information display module 1106, wherein:

[0276] The query question retrieval module 1102 is used to retrieve query questions;

[0277] The task node display module 1104 is used to display the task node corresponding to the query question; after the task corresponding to the task node is completed, the display style of the task node is updated; the display style is related to the node evaluation index value corresponding to the task node; the node evaluation index value represents the evaluation result of the task node's task execution status;

[0278] The information display module 1106 is used to display the node evaluation index value of the task node in response to the trigger operation of the task node.

[0279] In the above embodiments, after obtaining the query question, the task node corresponding to the query question is displayed. After the task corresponding to the task node is completed, the display style of the task node is updated based on the received node evaluation index value. In response to the trigger operation of the task node, the node evaluation index value of the task node is displayed. The node evaluation index value represents the evaluation result of the task node's task execution status, so that users can intuitively identify the importance and reliability of the task node in the process of generating query results in the interface. This solves the problem in traditional technology that the mechanism of task node operation cannot be revealed, making it difficult for users to understand the decision logic of query results, and improves the transparency and interpretability of the task execution process.

[0280] In one embodiment, the task node display module 1104 is configured to: display the task node corresponding to the query question according to a preset display style; the task node display module 1104 is configured to: determine the attribute value of the display attribute of the task node based on the node evaluation index value corresponding to the task node; and update the preset display style to the target display style corresponding to the attribute value of the display attribute.

[0281] In one embodiment, the display attributes include color attributes, and the preset display style is a preset color style; the task node display module 1104 is further configured to: determine the color gradient value of the task node based on the node evaluation index value; the task node display module 1104 is further configured to: generate a target color style based on the color gradient value; and replace the preset color style with the target color style.

[0282] In one embodiment, the preset display style includes a first preset display style and a second preset display style; the task node display module 1104 is used to: display the task node corresponding to the query question according to the first preset display style corresponding to the node contribution dimension; the device further includes a display switching module, used to: in response to the indicator value switching operation, display the task node corresponding to the query question according to the second preset display style corresponding to the node confidence dimension.

[0283] In one embodiment, the information display module 1106 is configured to: in response to a trigger operation on a task node, display the node evaluation index value of the task node and the evaluation description information corresponding to the task node.

[0284] In one embodiment, the number of task nodes is at least two, such as... Figure 12 As shown, the device also includes a data stream object display module 1108, which is used to: display the data stream object between the first task node and the second task node after the task corresponding to the first task node is completed in at least two task nodes, and the second task node performs subsequent processing based on the task execution result of the first task node.

[0285] In one embodiment, the node evaluation index value includes the node contribution index value; the data flow object is a data flow line; the data flow object display module 1108 is used to: display a data flow animation at the corresponding position of the data flow line; the display intensity of the data flow animation is related to the node contribution index value of the first task node associated with the data flow line.

[0286] Each module in the aforementioned task evaluation device and task evaluation result display device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0287] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 13As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data to be processed. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a task evaluation method and a method for displaying the task evaluation results.

[0288] Those skilled in the art will understand that Figure 13 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0289] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0290] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0291] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0292] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0293] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, etc., and are not limited to these.

[0294] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0295] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A task evaluation method, characterized in that, The method includes: The system receives a query question sent by the terminal and returns the task node corresponding to the query question to the terminal so that the terminal displays the task node. Execute the task represented by the task node to obtain the task execution result; The node evaluation index value of the task node is determined based on the task execution result; The node evaluation index value is returned to the terminal so that the terminal updates the display style of the task node based on the node evaluation index value, and displays the node evaluation index value of the task node in response to the trigger operation of the task node.

2. The method according to claim 1, characterized in that, The node evaluation index values ​​include node contribution index values; determining the node evaluation index values ​​of the task nodes based on the task execution results includes: Generate query results for the query question; In the task execution result, determine the associated content corresponding to the target fragment in the query result; Based on the query results and the associated content, the importance score of the task node is determined; Based on the importance score of the task node, the node contribution index value of the task node is determined.

3. The method according to claim 2, characterized in that, The query results also include reference identifiers for the task execution results, and the number of task nodes is at least two; determining the associated content corresponding to the target segment in the query results within the task execution results includes: Obtain the reference identifier of the target segment in the query results; Among at least two of the task nodes, the target task node that provides data support for the target fragment is determined based on the reference identifier; Based on the target fragment, the corresponding related content is extracted from the task execution result of the target task node.

4. The method according to claim 3, characterized in that, The process of determining the importance score of the task node based on the query results and the associated content includes: Based on the query results, the target fragment, the associated content, and the first instruction template, a first instruction statement is generated; Based on the first instruction statement, an importance assessment task is performed to obtain the importance score of the target task node.

5. The method according to claim 2, characterized in that, The number of task nodes is at least two; the determination of the node contribution index value of the task node based on the importance score of the task node includes: The overall importance score is determined based on the importance score of each of the aforementioned task nodes; The node contribution index value of each task node is determined based on the total importance score and the importance score of each task node.

6. The method according to claim 1, characterized in that, The node evaluation index value includes the node confidence index value; the determination of the node evaluation index value of the task node based on the task execution result includes: Obtain the reflection text corresponding to the task node; the reflection text is generated after the task node completes the corresponding task; A second instruction statement is generated based on the aforementioned reflection text and the second instruction template; The confidence generation task is executed based on the second instruction statement to obtain the node confidence index value of the task node.

7. The method according to claim 6, characterized in that, The step of obtaining the reflection text corresponding to the task node includes: A third instruction statement is generated based on the task execution result of the task node and the third instruction template. Based on the third instruction statement, the reflection generation task is executed to obtain the node reflection text of the task node.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The query problem is processed by task planning to obtain the task to complete the query problem. Generate the node corresponding to the task to obtain the task node corresponding to the query question.

9. The method according to claim 8, characterized in that, The step of performing task planning on the query problem to obtain tasks for completing the query problem includes: The query question is semantically parsed by a planning intelligent agent to obtain the semantic parsing result; Based on the semantic parsing results, multiple tasks are determined to complete the query target corresponding to the query question; The execution of the task represented by the task node, and the resulting task execution, include: Among multiple candidate execution agents, corresponding execution agents are assigned to the multiple tasks respectively; The execution agent executes the task corresponding to the task node to obtain the task execution result.

10. A method for displaying task evaluation results, characterized in that, The method includes: The query question has been retrieved; Display the task node corresponding to the query question; After the task corresponding to the task node is completed, the display style of the task node is updated; the display style is related to the node evaluation index value corresponding to the task node; the node evaluation index value represents the evaluation result of the task execution status of the task node; In response to a trigger operation on the task node, the node evaluation index value of the task node is displayed.

11. The method according to claim 10, characterized in that, The task node that displays the query question includes: The task node corresponding to the query question is displayed according to the preset display style; Updating the display style of the task node includes: Based on the node evaluation index value corresponding to the task node, determine the attribute value of the display attribute of the task node; Update the preset display style to the target display style corresponding to the attribute value of the display attribute.

12. The method according to claim 11, characterized in that, The display attributes include color attributes, and the preset display style is a preset color style; determining the attribute values ​​of the display attributes of the task node based on the node evaluation index value corresponding to the task node includes: Based on the node evaluation index value, determine the color gradient value of the task node; The step of updating the preset display style to the target display style corresponding to the attribute value of the display attribute includes: Generate the target color style based on the color gradient value; Replace the preset color style with the target color style.

13. The method according to claim 11, characterized in that, The preset display style includes a first preset display style and a second preset display style; displaying the task node corresponding to the query question according to the preset display style includes: Display the task node corresponding to the query question according to the first preset display style corresponding to the node contribution dimension; The method further includes: In response to the indicator value switching operation, the task node corresponding to the query question is displayed according to the second preset display style corresponding to the node confidence dimension.

14. The method according to claim 10, characterized in that, The step of displaying the node evaluation metric value of the task node in response to a trigger operation on the task node includes: In response to a trigger operation on the task node, the node evaluation index value of the task node and the corresponding evaluation description information of the task node are displayed.

15. The method according to claim 10, characterized in that, The number of task nodes is at least two, and the method further includes: After the task corresponding to the first task node is completed in at least two task nodes, the data flow object between the first task node and the second task node is displayed, and the second task node performs subsequent processing based on the task execution result of the first task node.

16. The method according to claim 15, characterized in that, The node evaluation index value includes the node contribution index value; the data flow object is a data flow direction line; the method further includes: At the corresponding position of the data flow line, a data flow animation is displayed; the display intensity of the data flow animation is related to the node contribution index value of the first task node associated with the data flow line.

17. A task evaluation device, characterized in that, The device includes: The query question receiving module is used to receive query questions sent by the terminal and return the task node corresponding to the query question to the terminal so that the terminal can display the task node. The task execution module is used to execute the task represented by the task node and obtain the task execution result; The indicator value determination module is used to determine the node evaluation indicator value of the task node based on the task execution result; The indicator value return module is used to return the node evaluation indicator value to the terminal, so that the terminal updates the display style of the task node based on the node evaluation indicator value, and displays the node evaluation indicator value of the task node in response to the trigger operation of the task node.

18. A display device for task evaluation results, characterized in that, The device includes: The query question retrieval module is used to retrieve query questions; The task node display module is used to display the task node corresponding to the query question; after the task corresponding to the task node is completed, the display style of the task node is updated; the display style is related to the node evaluation index value corresponding to the task node; the node evaluation index value represents the evaluation result of the task execution status of the task node; The information display module is used to display the node evaluation index value of the task node in response to the trigger operation of the task node.

19. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 16.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 16.

21. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 16.

Images