Multi-agent driven visual generation method

Abstract

The application discloses a kind of multi-agent driven visual generation method, which is based on the multi-agent system cooperated by answerability evaluation agent, analysis agent, visual optimization agent and coordinator, and includes the following steps: receiving user query question and structured dataset, scheduling answerability evaluation agent by coordinator, using two-stage judgment mechanism to judge whether the question can be answered by the current dataset;The coordinator schedules the visual optimization agent, uses a multimodal large model as a feedback source to perform task consistency evaluation, generates structured optimization suggestions and performs differential repair on the visualization code, iteratively optimizes until the requirements are met or the maximum number of iterations is reached;Finally output corresponding text answer and visualization result.The application solves the technical problems of how to effectively reduce the illusion risk, improve the code executable rate, and ensure the matching degree of the visualization result and the analysis task in the natural language interaction scenario.

Description

Technical Field

[0001] This invention belongs to the technical field of data visualization and large language model intelligent agents, specifically involving a multi-agent driven visualization generation method for the automated visualization generation of structured data. Background Technology

[0002] With the deepening of enterprise digital transformation, government affairs, finance, retail and other fields have accumulated massive amounts of structured business data. Data visualization, as a core means to intuitively present data patterns and support business decisions, is experiencing continuous growth in demand. Traditional data visualization generation relies on manual coding or drag-and-drop tool configuration, requiring operators to have data analysis capabilities, visualization design knowledge and programming basics. This results in high implementation barriers, long production cycles, and difficulty in adapting to high-frequency and diverse real-time data analysis needs. Automated visualization generation technology has become a core research direction for improving the efficiency of data value transformation.

[0003] Early automated visualization solutions were mostly based on rule mapping and template matching mechanisms. By predefined rules for matching field types with chart types, standardized statistical charts were automatically generated. These solutions were stable and efficient, but they had significant limitations. For example, they could only support simple statistical needs within a preset range, could not understand complex business queries in natural language, and could not dynamically adjust visual encoding and information hierarchy according to specific analysis objectives. The generated charts had a low degree of matching with actual business needs, making it difficult to meet the requirements of customized analysis scenarios.

[0004] The development of large language models has provided a new technical path for the automated generation of natural language-driven visualizations, but its direct application still suffers from several technical shortcomings. On the one hand, existing solutions generally lack a pre-processing data responsiveness verification step, often forcibly generating visualization results for queries that the dataset cannot support, easily leading to data illusions and distorted conclusions. On the other hand, most solutions couple data analysis logic and visualization generation logic within the same model inference, increasing the complexity of single-step reasoning, resulting in low executability of visualization code, and the generated charts generally suffer from problems such as a lack of emphasis on core information and a mismatch between visual encoding and task objectives. Furthermore, existing visualization optimization solutions based on large models often employ an iterative mode of full code regeneration, which is prone to problems such as code structure drift and alteration of original correct logic during iteration, resulting in insufficient stability and controllability of the optimization process. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-agent driven visualization generation method to solve technical problems such as how to effectively reduce the risk of illusion, improve code executability, ensure the matching degree between visualization results and analysis tasks, and maintain the controllability of iterative optimization in natural language interaction scenarios.

[0006] Based on a first key aspect of the present invention, a multi-agent-driven visualization generation method is provided. This method is based on a multi-agent system collaboratively executed by an answerability assessment agent, an analysis agent, a visualization optimization agent, and a coordinator, and includes the following steps:

[0007] The system receives user queries and structured datasets. A coordinator centrally schedules answerability assessment agents and uses a two-stage decision mechanism to determine whether the query can be answered by the current dataset. If the query is answered, the scheduled analysis agent decouples the data analysis and visualization generation tasks and outputs the text answer, initial visualization code, and corresponding initial charts. If the query is not answered, only basic statistical visualization results are generated.

[0008] The coordinator schedules the visualization optimization agent, uses a multimodal large model as the feedback source to perform task consistency evaluation, generates structured optimization suggestions, performs differential repair on the visualization code, and iterates optimization until the requirements are met or the preset maximum number of iterations is reached.

[0009] The final output includes the corresponding text answer and visualization results.

[0010] As a whole, the above solution realizes the division of labor and cooperation among various links at the top-level architecture level. The pre-emptive answerability judgment avoids the generation of illusions without data support from the source. The decoupled execution mode reduces the complexity of reasoning to improve code reliability. Differential iterative optimization ensures the matching degree between visualization and task. It systematically solves the core problem that existing automated visualization solutions are unable to simultaneously take into account the accuracy of results, the adaptability of requirements and the stability of operation.

[0011] Optionally, the responsiveness assessment agent employs a two-stage decision mechanism, the calculation process of which is as follows: The first stage involves joint inference to query the question and dataset fields, utilizing a large language model to analyze the question. With dataset Joint reasoning is performed to initially determine whether the task is executable and output the preliminary judgment result. This stage uses a large language model for two-dimensional scoring: the first dimension is field matching: analyzing whether the entities and attributes involved in the problem have corresponding fields in the dataset; the second dimension is operational feasibility: whether the data operations required by the problem can be implemented on the current data; when both dimensions result in "yes", the preliminary confidence level is determined. The value is 1, otherwise The value is 0, and the reason for the judgment is output. Only when the initial confidence level If the answer is less than a preset threshold, indicating an initial determination that the answer is unanswerable, a second-stage review is triggered. This stage verifies the logical consistency of the reasoning behind the determination and outputs the review result. When the verification passes The value is 1, otherwise The value is 0; when and When a question is deemed unanswerable, the overall judgment score is obtained using the following segmented calculation formula. : .

[0012] By transforming the responsiveness determination into a quantifiable two-dimensional scoring and review mechanism, and through the second-stage review and verification of the initial unresponsive results, the probability of misjudgment in the single-stage determination is effectively reduced. This not only avoids the erroneous interception of responsive questions, but also strictly prevents the forced generation of unresponsive questions. By controlling the authenticity of data from the input point, this solves the technical problems of existing solutions lacking pre-verification and easily generating unfounded visualization results that lead to data illusion.

[0013] Optionally, the analytical agent splits the execution process into two independent sub-processes: analysis generation and visualization generation, with data calculation logic and visual expression logic being separated from each other;

[0014] The two sub-processes share intermediate statistical results from data calculations;

[0015] For tasks that rely on key analysis results to determine visually emphasized objects, a sequential generation process is adopted, in which analysis generation is performed first, followed by visualization generation.

[0016] By separating data computation and visual representation into two categories of logic, the coupling complexity of single-step model reasoning is reduced, and code logic errors caused by multi-task coupling are reduced. At the same time, relying on shared intermediate results ensures data consistency, effectively improving the executability of the initial visualization code and the accuracy of data conclusions.

[0017] Optionally, when the visualization optimization agent performs task consistency evaluation, it first inputs the current chart image, visualization code, user question, and corresponding answer into a multimodal large language model to jointly judge whether the visual expression, statistical caliber, and task objective of the chart are consistent. That is, it evaluates whether the current chart can effectively support the recognition and understanding of task information. If the current chart can meet the task expression requirements, no further optimization is needed; otherwise, it proceeds to the next stage.

[0018] The above solutions establish objective and quantitative visualization quality evaluation standards, measuring the degree of matching between charts and analysis tasks from three core dimensions: data prominence, coding differentiation, and information indicativeness. This provides clear judgment criteria for iterative optimization, avoids subjective bias in optimization direction, ensures that visualization results always align with business analysis goals, and solves the problems of low visualization-task matching and lack of unified optimization standards in existing solutions.

[0019] Optionally, the code differential repair uses a partial editing method, without regenerating the complete visualization code, including:

[0020] Based solely on optimization suggestions, the code snippet to be modified is located, and a corresponding editable snippet is generated to replace the corresponding position in the original code.

[0021] The repair operation applies to visual encoding parameters, sorting and filtering logic, annotation statements, and local layout settings, and only replaces the chart type when the chart type does not match the task.

[0022] The optimization mode of replacing full code regeneration with partial code snippet replacement strictly limits the scope of the repair operation and adjusts the chart type only in necessary scenarios. This effectively avoids the problems of code structure drift and accidental tampering of the original correct logic during the iteration process, improves the stability and controllability of the optimization process, and reduces the probability of code execution failure after iteration.

[0023] Optionally, the coordinator employs centralized scheduling logic to uniformly manage the agent's calling order, state transitions, and error handling, including:

[0024] A maximum number of iterations is preset. After each iteration, the optimization suggestions are checked to see if they are empty. If they are empty, the iteration is terminated early.

[0025] If the code execution fails after differential repair, it will automatically revert to the previous round of valid visual code and trigger a constrained retry.

[0026] Skip the optimization process if the question is unanswerable.

[0027] The above solutions achieve controllable operation of the entire process through unified state transition management, control unnecessary resource consumption through the maximum number of iterations and the early termination mechanism for empty suggestions, improve the system's fault tolerance through code failure rollback and retry mechanism, and adapt to different scenarios through differentiated strategies for skipping optimization in unanswerable scenarios. These solutions solve the problems of poor fault tolerance and uncontrollable resource consumption in the existing visualization generation process.

[0028] Optionally, the visualization code is implemented using the Matplotlib and Seaborn libraries;

[0029] When a question is unanswerable, output at least one basic statistical visualization chart from the following categories: variable distribution, category composition, time trend, and missing data. Do not output inferential text answers without data support.

[0030] The above solutions ensure cross-environment consistency of visualization code execution by unifying the code dependencies between Matplotlib and Seaborn. By explicitly defining the rule of only outputting basic statistical visualizations and not generating inferential answers in unanswerable scenarios, the space for model illusions is further reduced, while providing users with basic understanding of the dataset and avoiding misleading users with unfounded outputs.

[0031] Based on a second key aspect of the present invention, a multi-agent collaborative visualization generation system for implementing the aforementioned method is provided, comprising an responsiveness assessment agent, an analysis agent, a visualization optimization agent, and a coordinator:

[0032] The responsiveness assessment agent receives user query questions and structured datasets, uses a two-stage mechanism to determine the responsiveness of the questions, and outputs the determination results.

[0033] The analytical agent is used to decouple the generation of text answers from the initial visualization code and charts when the question is answerable, and to generate basic statistical visualization results when the question is unanswerable.

[0034] A visualization optimization agent is used to perform task consistency evaluation on visualization results based on a multimodal large model, generate structured optimization suggestions, perform differential repair on visualization code, and perform iterative optimization.

[0035] The coordinator is used to centrally schedule the execution order and state transition of each agent, control the number of iterations, handle execution errors and rollback logic, and output text answers and visualization results.

[0036] The above solutions transform the technical logic at the methodological level into intelligent agents in the form of physical system functional modules, providing a structured implementation carrier for the engineering implementation of technical solutions. Each intelligent agent performs its own function, which can stably support the full-process visualization generation from problem verification to optimization output, ensuring the feasibility and reproducibility of the methodological solutions.

[0037] Based on a third key aspect of the present invention, an electronic device is provided, comprising one or more processors;

[0038] Storage device for storing one or more programs;

[0039] When one or more programs are executed by one or more processors, the one or more processors implement the aforementioned multi-agent driven visualization generation method.

[0040] Based on a fourth key aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed, implements the multi-agent driven visualization generation method as described above.

[0041] Compared with existing technologies, in terms of result accuracy and hallucination risk prevention, this invention establishes an entry-level verification barrier through a two-stage answerability determination mechanism, reduces the misjudgment rate by relying on dual-dimensional quantitative scoring and review logic, and strictly limits the output boundaries of unanswerable scenarios, outputting only basic statistical visualization charts and not generating inferential conclusions without data basis. This significantly reduces the space for the generation of data hallucinations from the source and ensures the consistency between the output results and the original dataset.

[0042] In terms of visualization generation quality and code execution reliability, this invention reduces the task complexity of single-step model inference by separating data analysis and visualization generation into independent sub-processes, thereby improving the executability of the initial visualization code and the accuracy of data conclusions. Simultaneously, it employs an iterative optimization mode of local differential repair to replace the traditional full code regeneration method, strictly limiting the scope of repair operations and avoiding code structure drift and tampering with correct logic during iteration. Combined with a quantitative task consistency evaluation standard, it achieves precise alignment between visualization effects and analysis objectives.

[0043] In terms of system operating efficiency and scenario adaptability, this invention constructs an operating system that combines controllability and robustness. The centralized coordinator, through unified state transition management and mechanisms such as setting a maximum number of iterations, early termination for empty suggestions, and automatic rollback and retry for code failures, controls unnecessary computational overhead and improves system fault tolerance in abnormal scenarios. A unified code dependency library ensures cross-environment execution consistency, and the multi-agent architecture can adapt to various query needs, from basic statistics to customized analysis. The overall solution can be flexibly implemented in real-time visualization scenarios of structured data in multiple fields such as government affairs, finance, and retail. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, obtaining other drawings based on these drawings without creative effort still falls within the scope of the present invention.

[0045] Figure 1 This illustration shows a flowchart of an execution method for multi-agent-driven visualization generation in one embodiment of the present invention.

[0046] Figure 2This illustration shows a framework diagram of a multi-agent-driven visualization generation method in one embodiment of the present invention;

[0047] Figure 3 An example of a coordinator scheduling algorithm in one embodiment of the present invention is shown. Detailed Implementation

[0048] The preferred embodiments of the present invention will be described in detail below to provide a clearer understanding of the purpose, features, and advantages of the invention. It should be understood that the following embodiments are not intended to limit the scope of the invention, but are merely illustrative of the essential spirit of the technical solution of the invention.

[0049] In the following description, certain specific details are set forth for the purpose of illustrating various disclosed embodiments in order to provide a thorough understanding of the various disclosed embodiments. However, those skilled in the art will recognize that embodiments may be practiced without one or more of these specific details. In other instances, well-known techniques associated with the invention may not have been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0050] Throughout this specification, references to "an embodiment" or "an embodiment" indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of "in an embodiment" or "an embodiment" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any manner in one or more embodiments.

[0051] The specific meanings of the technical terms that may be used in this invention are explained as follows:

[0052] Two-stage answerability determination mechanism: The core mechanism of the pre-verification of this invention refers to the initial confidence level and determination reasoning output by scoring the user query question and the structured dataset in two dimensions: field matching and operation executability. Then, the logical consistency of the results initially determined to be unanswerable is checked. Finally, a comprehensive determination score is obtained through a segmented calculation formula to confirm whether the question can be answered by the current dataset.

[0053] Analysis-Visualization Decoupling Generation: The core execution logic of the analytical agent in this invention refers to splitting data analysis reasoning and visualization code generation into two independent sub-processes. The two sub-processes share the intermediate statistical results of data calculation. For tasks that rely on analysis conclusions to determine visual emphasis objects, a sequential process of analysis first and then mapping is adopted.

[0054] Task Consistency Assessment: The quantitative evaluation mechanism of the visualization optimization process in this invention refers to scoring the visualization results from three dimensions: visual prominence of answer data, visual coding differentiation, and indicativeness of core information. The comprehensive matching score is calculated using an equal-weighted method to determine the degree of fit between the visualization results and the analysis task objectives.

[0055] Code Differential Repair: The core execution method of the visualization iterative optimization of this invention refers to not regenerating the complete visualization code, but only locating the code segment to be modified based on the optimization suggestions, and adjusting the visual coding parameters, annotations, layout and other contents by local editing and replacement, and changing the chart type only when the chart type is obviously mismatched with the task.

[0056] Centralized Coordinator: The core scheduling unit of the multi-agent system of this invention, responsible for the unified management of the calling order, state transition, iteration number and exception handling of each agent, and scheduling rules such as rollback and retry when code execution fails, early termination when optimization suggestions are empty, and skipping optimization in unanswerable scenarios, to ensure controllable operation of the entire process.

[0057] Basic statistical visualization results: This invention addresses the standardized output content for unanswerable questions by not generating inferential conclusions directly corresponding to the query, but only outputting statistical charts that reflect the overall characteristics of the dataset, including variable distribution, category composition, time trend, and missing data, thus avoiding the illusion of output without data support.

[0058] In one feasible embodiment, combined with Figure 1 As shown, this invention provides a multi-agent-driven visualization generation method. This method is based on a multi-agent system jointly executed by an answerability assessment agent, an analysis agent, a visualization optimization agent, and a coordinator, and includes the following steps S100-S300:

[0059] Step S100: Receive user query questions and structured datasets. The coordinator centrally schedules the answerability assessment agent and uses a two-stage decision mechanism to determine whether the question can be answered by the current dataset. If the question is determined to be answerable, the scheduled analysis agent decouples and performs data analysis and visualization generation tasks, outputting text answers, initial visualization code, and corresponding initial charts. If the question is determined to be unanswerable, only basic statistical visualization results are generated.

[0060] Step S200: The coordinator schedules the visualization optimization agent to perform task consistency evaluation using a multimodal large model as the feedback source, generates structured optimization suggestions, performs differential repair on the visualization code, and iterates optimization until the requirements are met or the preset maximum number of iterations is reached.

[0061] Step S300: Finally, output the corresponding text answer and visualization results.

[0062] The following detailed description is provided through a specific embodiment.

[0063] In this embodiment, a problem statement is first provided: given an analysis problem proposed by a user. and the dataset to be analyzed The goal of this embodiment is to generate text answers. And the data visualization results that support this conclusion. Visualization is used not only to present data features but also as an important source of evidence supporting analytical conclusions. Furthermore, the system's output format varies depending on the specific problem conditions.

[0064] 1) When analyzing the problem When an answer is available, output To provide a clear analytical conclusion regarding the problem, Visualizations used to reveal data patterns and support conclusions;

[0065] 2) When analyzing the problem When unable to answer, , Indicates the empty set, at this time To support basic visualizations for exploratory analysis, such as variable distribution, category composition, time trends, missing data, or correlations between variables, to help users understand the overall structure and analyzable scope of the data.

[0066] Therefore, this task can be formalized as a conditional generation problem:

[0067]

[0068] In the formula, This indicates the analysis question raised by the user; This represents the dataset to be analyzed;

[0069] To ensure the validity of the generated results, the output is... It should satisfy the following properties: 1) Question-answer alignment: text answer Able to accurately respond to analytical questions 1) Semantic intent; 2) Chart accuracy: Visualization results Able to provide supporting text answers 3) Data evidence; 4) Chart readability: Visualization results It demonstrates excellent expressive power in terms of label clarity, text overlap control, color contrast, consistency between coordinate axes and legends, logical sorting, and information density control. It helps users intuitively understand key patterns within the data.

[0070] Properties 1) and 2) are required to be satisfied only if the question is answerable.

[0071] In the following embodiments, the visualization method of the present invention adopts a centralized multi-agent architecture with unified scheduling by a coordinator, decoupling different subtasks into independent modules and achieving global collaboration through unified scheduling. Figure 2 As shown, this method breaks down complex data analysis and visualization tasks into multiple sub-tasks, which are handled by different agents, including: an answerability evaluation agent (AE agent), an analysis agent (ANA agent), and a visualization optimization agent (VO agent).

[0072] In this architecture, the AE Agent is responsible for determining whether the user's question can be answered on the given data; the ANA Agent is responsible for generating the text answer and the initial visualization results; and the VO Agent is responsible for evaluating the quality of the current chart from the perspective of task alignment and outputting structured optimization suggestions, thereby performing local repairs on the existing visualization code to achieve stable iterative optimization.

[0073] The coordinator, as the core control module, is responsible for subtask scheduling, execution status management, and feedback integration. Unlike distributed multi-agent systems, all agents do not directly negotiate with each other; instead, they exchange information and make decisions through the coordinator, thereby improving the overall stability and controllability of the system.

[0074] This centralized architecture also brings the following three advantages to this task: 1) Globally available code information: The code information generated in the analysis process can be accessed and reused uniformly by each module, avoiding repeated parsing; 2) State traceability: Supports complete execution history and intermediate state management; 3) Robust error recovery mechanism: Enables fine-grained retries and repairs in the event of partial failure.

[0075] The following embodiments illustrate an agent for assessing responsiveness.

[0076] In automated data analysis, not all user questions can be answered directly from the current data. Continuing to perform complex analysis on unanswerable questions may lead to misleading insights or even failure. Therefore, it is necessary to assess the answerability of questions before analysis and visualization generation. This embodiment designs an answerability assessment agent and employs a two-stage assessment mechanism to improve the reliability of the judgment.

[0077] Phase 1: Preliminary assessment.

[0078] The first stage utilizes large language models to analyze the problem. With dataset Joint reasoning is performed to initially determine whether the task is executable and output the preliminary judgment result. This stage mainly judges from two aspects: 1) Field matching: Analyze whether the entities and attributes involved in the problem have corresponding fields in the dataset. 2) Operation executability: Whether the data operations required by the problem (such as aggregation, grouping, trend analysis, etc.) can be implemented on the current data.

[0079] Phase Two: Judgment and Review.

[0080] Based on the initial assessment, this embodiment introduces a verification mechanism only for the "unanswerable" conclusion to reduce the risk of misjudgment. Specifically, the second stage does not re-output a complete answerability label, but instead determines whether the "unanswerable" conclusion given in the first stage is valid. This stage is mainly used to perform a logical consistency analysis of the reasons for the first stage's judgment.

[0081] Finally, the answerability assessment agent is defined as follows:

[0082] ;

[0083] In the formula Preliminary assessment results: This indicates that an answer is possible, but does not trigger a review process, and an agreement is made. ; This indicates that an answer is not possible. The reasons for determining the form of natural language are explained. This indicates the verification result of the "unanswerable" conclusion: when When, it indicates that the review module supports the first stage of unanswerable determination; when If this occurs, it indicates that the review module was unable to confirm the conclusion.

[0084] The coordinator executes the final decision based on the agent's output of the answerability assessment: when and At that time, the question is deemed unanswerable.

[0085] The analytical agent is illustrated in the following examples.

[0086] The analytical agent is only responsible for generating text answers and initial visualization code. To improve the accuracy of answer generation and reduce the coupling requirements of complex tasks on the capabilities of a single model, this embodiment splits the execution process into two relatively independent but interrelated sub-processes: analysis generation and visualization generation, separating the data computation logic from the visual expression logic.

[0087] This design aims to reduce the coupling requirements of complex tasks on the capabilities of a single agent, significantly reducing the inference burden on the model in a single subtask. To balance efficiency and consistency, analysis generation and visualization generation can be performed collaboratively while sharing intermediate statistical results; for tasks that require key analysis results to determine visually emphasized objects, a sequential generation strategy of "analysis first, then visualization" is preferred.

[0088] 1. Analyze task execution.

[0089] When the question is answerable, the large language model analyzes the user's query. With dataset Generate analytical code for calculating the answer. Its core objective is to construct an executable data processing flow to obtain an explicit answer in text form, i.e., a text answer. .

[0090] When a question is unanswerable, provide a text answer. Skip this stage and go directly to the visualization stage, generating basic statistical charts based solely on data pattern information, without performing the answer-solving process.

[0091] 2. Initial visualization generation.

[0092] During visualization tasks, the large language model responds to the analysis questions posed by the user. With dataset This step generates Python visualization code for graphical representation. To ensure execution consistency and the stability of subsequent difference repair, this embodiment limits the visualization libraries to Matplotlib and Seaborn. The goal of this stage is to generate initial charts that can express the main data relationships, providing a foundation for subsequent task-oriented evaluation and repair.

[0093] Finally, the definition of the analytical agent is as follows:

[0094] ;

[0095] In the formula, This represents the initially generated visualization code; This represents the initially generated visualization chart. The formula means that the user query question and the structured dataset are input into the analysis agent. After the agent's internal analysis and visualization generation are decoupled, it outputs three types of results: the initial visualization code, the initial visualization chart, and the text answer.

[0096] The following examples illustrate the visualization optimization agent.

[0097] Using visual saliency as a feedback signal can, to some extent, facilitate the visualization design process for designers. However, this type of method primarily focuses on visual optimization, neglecting the core objective of visualization: serving specific analytical tasks. Since different analytical tasks have varying requirements for information presentation, relying solely on visual saliency is insufficient to comprehensively measure the effectiveness of visualization. Therefore, this embodiment designs a visualization optimization agent oriented towards analytical tasks. Based on the initial visualization generation results, it optimizes the charts through an iterative mechanism of "evaluation-suggestion-repair" to make them more clearly and accurately support the user's task. This agent mainly includes three sub-processes: task consistency evaluation, optimization suggestion generation, and differential repair execution.

[0098] 1. Task consistency assessment.

[0099] The visualization optimization agent first inputs the current chart image, visualization code, user question, and corresponding answer into a multimodal large language model. It then jointly judges whether the visual expression, statistical caliber, and task objective of the chart are consistent, that is, it evaluates whether the current chart can effectively support the recognition and understanding of task information.

[0100] This stage primarily evaluates the following three aspects: 1) Whether the data corresponding to the answer is effectively highlighted visually; 2) Whether the visual encoding, such as color, size, and position, is sufficiently distinctive to support users in quickly identifying key information; 3) Whether clear visual indicators, such as labels, annotations, or highlights, are provided to guide users to locate the core content related to the question. If the current chart already meets the task's expression requirements, no further optimization is needed; otherwise, proceed to the next stage.

[0101] 2. Optimization suggestion generation.

[0102] When no optimization is needed, the VO Agent does not generate optimization suggestions and directly outputs the current result; otherwise, it generates further optimization suggestions for the current visualization to guide the subsequent automatic adjustment process. Specifically, the current visualization result, user question, and answer information are input into the Multimodal Large Language Model (MLLM), which combines visualization design principles and task objectives to generate structured optimization suggestions.

[0103] The generated optimization suggestions mainly include the following four aspects: 1) Whether the chart type matches the task type; 2) Whether it is necessary to enhance the visibility of key data objects through sorting, filtering, or local emphasis; 3) Whether it is necessary to adjust visual encoding such as color contrast, size mapping, and transparency to enhance differentiation; 4) Whether it is necessary to add indicative elements such as text labels, reference lines, and highlight marks to improve information location efficiency. The optimization suggestions are output in a structured form as the basis for subsequent code fixes.

[0104] 3. Differential repair execution.

[0105] To mitigate the risks of code structure drift and execution failure caused by free regeneration during iterative optimization, this embodiment introduces a differential editing method for performing local modifications on existing visualization code. Unlike the initial generation stage, this method does not regenerate the entire visualization code. Instead, based on structured optimization suggestions, it generates only locally edited fragments related to the modification suggestions and replaces them in the corresponding positions in the original code. This improves the chart's expressive effect on the analysis task while maintaining the stability of the original correct code structure as much as possible. The differential repair operation mainly affects visual encoding parameters, sorting and filtering logic, annotation and comment statements, and local layout settings; chart type replacement is only allowed when the current chart type is clearly mismatched with the task. The repaired code is re-executed, generating new chart results, and then re-enters the task consistency evaluation module, thus forming a closed-loop iterative optimization process.

[0106] Through the above design, the visualization optimization agent can transform task-level optimization requirements into executable code-level adjustments, thereby improving the ability of charts to express analysis tasks while ensuring the stability of the code structure.

[0107] Finally, the definition of the visualization optimization agent is as follows:

[0108] ;

[0109] In the formula, Indicates the first Visualize the input code during round iteration; Indicates the first The corresponding chart results;

[0110] When the consistency assessment result indicates that no remediation is required. , , When repair is needed, This represents the set of optimization suggestions generated in the current round. This represents the optimized visualization code. This indicates the chart result generated after optimization.

[0111] The coordinator is illustrated in the following embodiments.

[0112] The coordinator is the core control module for implementing the method of this invention. It is responsible for uniformly managing the calling order, state transitions, result determination, and feedback integration of each agent. This design avoids a single agent directly controlling the global process, thereby enhancing the system's controllability, consistency, and robustness.

[0113] An example algorithm for the coordinator execution process is as follows: Figure 3 As shown. First, the answerability assessment agent is invoked to determine the answerability of the user's question and obtain the determination result. (Lines 1-6). Subsequently, the system enters the analysis and visualization generation phase: when When the coordinator determines that the current question cannot be answered directly by data, the system only generates a visualization result. And set the text answer to empty, that is (Lines 8-9); Otherwise, the coordinator determines that the current task can continue, the system enters problem-driven analysis mode, and generates a text answer. With visualization results (Line 11).

[0114] After obtaining the initial result, the coordinator saves it as the result of the current stage (line 13), and then... Make a judgment (lines 14-16). Since the optimization objective of the VO Agent depends on the task consistency among the question, answer, and graph, and... At this time, the consistency goal cannot be achieved. Therefore, when If the result is positive, return the result directly to end the analysis (line 15); otherwise, proceed to the visualization and optimization stage (lines 17-24). The maximum number of iterations is set to 3 in this embodiment as a hyperparameter. During this stage, the coordinator iteratively calls the visualization optimization agent to evaluate the current visualization results and returns the optimized visualization results and a set of optimization suggestions. (Line 18). When If the current visualization result meets the requirements, the system directly returns the current result as the final output (lines 20-22). Otherwise, the coordinator updates the current stage result with the optimized result and continues to execute the next iteration. When the preset maximum number of iterations is reached, the system terminates the optimization process and returns the currently saved result (line 24). If the differentially edited code fails to execute, the system reverts to the previous visualization code and triggers a constrained retrieval.

[0115] The effectiveness of the method of the present invention is verified through experiments in the following embodiments.

[0116] The experiments in this invention are based on the Text2Vis dataset evaluation benchmark released by the University of York. This dataset is designed for text-to-visualization tasks, covering more than 20 types of charts and various data analysis queries such as trend analysis, correlation analysis, and outlier detection. It covers multiple fields such as finance, healthcare, politics, and energy, and contains a total of 1985 samples, including 1746 training samples and 239 test samples. It has both closed-ended and open-ended questions, which can comprehensively evaluate the model's analytical reasoning and visualization generation capabilities.

[0117] The experiment set up three baseline methods as comparison objects: the direct prompt-based LLM-only method, the two post-training fine-tuning models RL-Text2Vis and VisCoder-7b, and the two agent schemes Agentic Inference and AutoGen-based multi-agent schemes. The base models used were Qwen2.5-7B-Instruct and Qwen2.5-14B-Instruct, and the evaluation model used was Internvl-3.5-38B. All models were deployed through the lmdeploy framework and run on the corresponding number of A6000 GPUs configured according to the parameter scale.

[0118] The experiment constructs an evaluation system from five dimensions to systematically measure the comprehensive performance of the method. Answer matching is used to determine the semantic consistency between the generated text answer and the standard answer, and is performed by a multimodal large model for semantic matching. Code execution success rate is the percentage of error-free code execution that generates results. Visualization clarity and readability, as well as chart accuracy, are all scored on a multimodal scale of 1-5, respectively measuring the visual standardization of charts and the accuracy of data relationship expression. The final pass rate is a comprehensive indicator, requiring samples to simultaneously meet three conditions: successful code execution, answer matching, and both visualization scores not lower than 3.5, reflecting the end-to-end usability of the method.

[0119] Comparative experimental results show that the multi-agent driven visualization generation method proposed in this invention achieves the highest final pass rate under both parameter scales of the base model. With a 7B parameter setting, the final pass rate of the multi-agent driven visualization generation method is 26.75%, representing a relative improvement of approximately 59.8% compared to the suboptimal baseline AutoGen; with a 14B parameter setting, the final pass rate increases to 40.55%, representing a relative improvement of approximately 60% compared to the suboptimal baseline Agentic Inference.

[0120] The results demonstrate that the performance improvement of the method does not depend on stronger base model parameters, but rather on systematic improvements at the architecture level. The pre-emptive answerability evaluation mechanism effectively intercepts invalid queries without data support, ensuring performance stability and scalability under different base scales.

[0121] From the perspective of individual metrics, the core advantage of the multi-agent driven visualization generation method of this invention lies in its answer matching accuracy. Under 7B and 14B settings, the answer matching accuracy of the multi-agent driven visualization generation method of this invention reached 42.62% and 48.46% respectively, ranking first among all compared methods. This confirms the effectiveness of the decoupling design of analysis and visualization—this mechanism separates the reasoning load of data computation and visual expression, allowing the model to focus on semantic reasoning and data computation, thereby improving the accuracy of text answers.

[0122] In terms of visualization quality, the visualization clarity and chart accuracy scores of the multi-agent driven visualization generation method of this invention are 3.92 and 3.74 respectively under the 14B model, both of which are the highest scores; under the 7B model, its chart accuracy index remains leading, with only the visualization clarity slightly lower than VisCoder and RL-Text2Vis after special chart fine-tuning, but its overall performance is better.

[0123] The module ablation experiment was conducted based on the Qwen2.5-7B-Instruct model to verify the contribution of each core agent module to the overall performance. After removing the responsiveness assessment module, the answer matching rate dropped from 42.62% to 36.57%, a relative decrease of approximately 14.2%, and the final pass rate fell to 25.19%, indicating that the pre-judgment mechanism can avoid forced generation without data support and effectively reduce the logical illusion of the large model. After removing the decoupling mechanism of the analysis agent, all indicators showed a comprehensive decline, with the code execution success rate plummeting to 69.97% and the final pass rate dropping significantly to 14.01%, a relative decrease of approximately 47.6%, proving that the decoupling design is the core architectural foundation for ensuring code execution accuracy and answer quality. After removing the visualization optimization module, the two visualization scores showed a slight decline, and the final pass rate dropped to 26.45%, verifying the effect of multimodal feedback optimization on improving chart quality.

[0124] To verify the reliability of the open-source evaluation model, an experiment was conducted to verify the model's consistency. The evaluation results of GPT-4o and InternVL-3.5-38B were compared at both the sample and conclusion levels. At the sample level, the consistency rate for answer matching ranged from 0.952 to 0.976, with Cohen's Kappa coefficient between 0.894 and 0.916. The consistency rate for final pass rate remained between 0.925 and 0.950, with Kappa values ​​between 0.703 and 0.808. The overall mean Spearman rank correlation coefficients for the two visualization scores were 0.811 and 0.831, respectively, with mean absolute errors controlled below 0.5. At the conclusion level, the two evaluators ranked all compared methods identically. The multi-agent driven visualization generation method of this invention was rated first in all three indicators: answer matching, final pass rate, and chart correctness, demonstrating that this open-source model can serve as a reliable and low-cost alternative for automated evaluation.

[0125] In test cases examining differences in gender and online shopping behavior patterns, the multi-agent-driven visualization generation method of this invention generated text answers covering key statistical values ​​across all dimensions, demonstrating superior data granularity and accuracy compared to other methods. Regarding visualization generation, other methods generally suffer from issues such as chart type mismatch and overlapping / occlusion of long X-axis labels. In contrast, the multi-agent-driven visualization generation method of this invention, through differential repair of visualization optimization agents, automatically adjusts label layout to avoid text occlusion, resulting in generated results that most closely resemble manually labeled text. Overall experiments show that the multi-agent collaborative architecture design can simultaneously ensure answer accuracy, code executability, and visualization quality, effectively addressing the shortcomings of existing solutions in overall performance.

[0126] It should be understood that the program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0127] The acquisition, storage, and application of user personal information involved in the technical solution of this invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0128] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this invention does not impose any limitations on them.

[0129] The technical terms, principles, or means related to the technical solutions of the present invention mentioned in the above embodiments, which are not described in detail above, are all well-known technologies or common practices that are known to those skilled in the art.

[0130] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A multi-agent driven visualization generation method, characterized in that, This method is based on a multi-agent system that is collaboratively executed by an answerability assessment agent, an analysis agent, a visualization optimization agent, and a coordinator, and includes the following steps: The system receives user queries and structured datasets. A coordinator centrally schedules answerability assessment agents and uses a two-stage decision mechanism to determine whether the query can be answered by the current dataset. If the query is answered, the scheduled analysis agent decouples the data analysis and visualization generation tasks and outputs the text answer, initial visualization code, and corresponding initial charts. If the query is not answered, only basic statistical visualization results are generated. The coordinator schedules the visualization optimization agent, uses a multimodal large model as the feedback source to perform task consistency evaluation, generates structured optimization suggestions, performs differential repair on the visualization code, and iterates optimization until the requirements are met or the preset maximum number of iterations is reached. The final output includes the corresponding text answer and visualization results.

2. The multi-agent driven visual generation method of claim 1, wherein, The responsiveness assessment agent employs a two-stage decision mechanism, the calculation process of which is as follows: The first stage involves joint inference to query the question and dataset fields, utilizing a large language model to analyze the question. With dataset Joint reasoning is performed to initially determine whether the task is executable and output the preliminary judgment result. This stage uses a large language model for two-dimensional scoring: the first dimension is field matching: analyzing whether the entities and attributes involved in the problem have corresponding fields in the dataset; the second dimension is operation executability: whether the data operations required by the problem can be implemented on the current data; when both dimensions result in "yes", the preliminary confidence level is determined. The value is 1, otherwise The value is 0, and the reason for the judgment is output. Only when the initial confidence level If the answer is less than a preset threshold, indicating an initial determination that the answer is unanswerable, a second-stage review is triggered. This stage verifies the logical consistency of the reasoning behind the determination and outputs the review result. When the verification passes The value is 1, otherwise The value is 0; When and the problem is determined to be unanswerable, the overall determination score is obtained by calculating the score in the following segments : 。 3. The multi-agent driven visual generation method of claim 1, wherein, The analytical agent breaks down the execution process into two independent sub-processes: analysis generation and visualization generation, with data calculation logic and visual expression logic being separated from each other. The two sub-processes share intermediate statistical results from data calculations; For tasks that rely on key analysis results to determine visually emphasized objects, a sequential generation process is adopted, in which analysis generation is performed first, followed by visualization generation.

4. The multi-agent driven visual generation method of claim 1, wherein, When the visualization optimization agent performs a task consistency evaluation, it first inputs the current chart image, visualization code, user question, and corresponding answer into a multimodal large language model. The model then makes a joint judgment on whether the visual expression, statistical caliber, and task objective of the chart are consistent. In other words, it evaluates whether the current chart can effectively support the recognition and understanding of task information. If the current chart can meet the task expression requirements, no further optimization is needed; otherwise, it proceeds to the next stage.

5. The multi-agent driven visual generation method of claim 1, wherein, The code differential repair uses a partial editing method and does not regenerate the complete visualization code, including: Based solely on optimization suggestions, the code snippet to be modified is located, and a corresponding editable snippet is generated to replace the corresponding position in the original code. The repair operation applies to visual encoding parameters, sorting and filtering logic, annotation statements, and local layout settings, and only replaces the chart type when the chart type does not match the task.

6. The multi-agent driven visual generation method of claim 1, wherein, The coordinator employs centralized scheduling logic to uniformly manage the agent's calling order, state transitions, and error handling, including: A maximum number of iterations is preset. After each iteration, the optimization suggestions are checked to see if they are empty. If they are empty, the iteration is terminated early. If the code execution fails after differential repair, it will automatically revert to the previous round of valid visual code and trigger a constrained retry. Skip the optimization process if the question is unanswerable.

7. The multi-agent driven visual generation method of claim 1, wherein, The visualization code is implemented using the Matplotlib and Seaborn libraries; When a question is unanswerable, output at least one basic statistical visualization chart from the following categories: variable distribution, category composition, time trend, and missing data. Do not output inferential text answers without data support.

8. A multi-agent collaborative driven visual generation system for implementing the method of any one of claims 1-7, characterized in that, This includes an responsiveness assessment agent, an analysis agent, a visualization optimization agent, and a coordinator: The responsiveness assessment agent receives user query questions and structured datasets, uses a two-stage mechanism to determine the responsiveness of the questions, and outputs the determination results. The analytical agent is used to decouple the generation of text answers from the initial visualization code and charts when the question is answerable, and to generate basic statistical visualization results when the question is unanswerable. A visualization optimization agent is used to perform task consistency evaluation on visualization results based on a multimodal large model, generate structured optimization suggestions, perform differential repair on visualization code, and perform iterative optimization. The coordinator is used to centrally schedule the execution order and state transition of each agent, control the number of iterations, handle execution errors and rollback logic, and output text answers and visualization results.

9. An electronic device, comprising: Includes one or more processors; Storage device for storing one or more programs; When one or more programs are executed by one or more processors, the one or more processors implement the multi-agent driven visualization generation method as described in any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed, it implements the multi-agent driven visualization generation method as described in any one of claims 1-7.

Images