An exploratory data analysis method and system based on large language model interactive reasoning

Abstract

This invention discloses an exploratory data analysis method and system based on interactive reasoning using a large language model. The invention uses the user-proposed initial question as the root question node of the analysis tree, and executes the following steps starting from the root question node: Based on selecting a question node from the analysis tree, relevant existing analysis results are retrieved from the memory agent; the hypothesis generation agent performs deductive reasoning on the currently selected question node, proposing a new hypothesis as the hypothesis node for the currently selected question node; for the hypothesis node, the planning agent and execution agent interactively execute an inner loop to generate and execute a verification plan; the checking agent summarizes the execution results of the verification plan and stores them in the memory agent; the checking agent checks whether the data subset generated by verifying the hypothesis in the inner loop is suitable for bottom-up analysis; if suitable, the checking agent is called to mine the data subset and store it in the memory agent; the question proposal agent proposes a new question based on the analysis results of the current hypothesis node.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology and relates to an exploratory data analysis method and system based on interactive reasoning of a large language model. Background Technology

[0002] In scientific research, researchers generally emphasize the importance of causal explanations or theories. Through these theoretical explanations, we can understand events in the real world and make predictions and controls based on this understanding. While good theories are crucial, the starting point of a theory can be said to be the initial identification of the phenomenon that needs explanation. This scientific activity—phenomenon discovery—is the foundation of scientific progress. Logically, it precedes the development, refinement, and testing of theories, and its importance is also reflected in the fact that the Nobel Prize is awarded more for the discovery of phenomena than for the construction of explanatory theories.

[0003] For a long time, most scientific research has strongly favored the hypothesis-deductive confirmatory research method, also known as confirmatory data analysis (CDA). In this model, the entire theoretical framework is usually clearly defined before the research begins, leaving very little room for exploration and discovery. This research orientation is likely to inhibit the development of the fundamental scientific activity of phenomenon discovery. Exploratory data analysis (EDA) was proposed in this context as a natural complement to CDA.

[0004] EDA (Exploratory Data Analysis) primarily originates from the pioneering work of renowned statistician John Tukey and his colleagues, with his classic work, *Exploratory Data Analysis*, being the most representative. While some rudimentary EDA techniques existed before Tukey's work, this book was the first to systematically integrate the core concepts and technical framework of EDA. In a series of lectures, Tukey summarized several principles constituting EDA, including: a willingness to discover phenomena that are both unexpected and regular; maintaining flexibility in thinking; relying on simple mathematical operations; emphasizing "seeing" the results (i.e., visualization); employing robust statistical methods; viewing data as a combination of "fit + residuals"; and emphasizing model building. These concepts collectively constitute the essence of exploratory data analysis: an open, flexible, and discovery-oriented scientific approach to analysis.

[0005] In summary, the main difference between EDA and CDA lies in their focus: the former emphasizes the discovery of verifiable hypotheses (phenomena), while the latter focuses on verifying pre-defined hypotheses (phenomena). In other words, without a clear analytical intent or direction from the analyst, EDA aims to extract valuable patterns and regularities from the data by combining observations of the data itself with the analyst's prior knowledge. CDA, on the other hand, is based on the analyst having a clear analytical intent (which may stem from their existing knowledge or specific interests) and aims to verify that intent using specific data analysis tools.

[0006] Before the rise of deep learning, pattern recommendation systems designed specifically for exploratory data analysis (EDA) were the primary tools for automating EDA. These systems typically sit between the dataset and the user interface, recommending efficient data analysis operations (such as visualization, drill-down analysis, SQL queries, and other general analytical suggestions) without requiring specific user input or parameters. In fact, to date, these systems remain the most widely used and mature automated EDA solutions in real-world production environments.

[0007] Recommender systems can be broadly categorized into data-driven and log-driven systems. Data-driven recommender systems rely entirely on currently available data to generate recommendations. They use predefined interestness metrics to evaluate the utility of specific data operations (such as SQL queries, data visualization, data summarization, etc.) or directly assess the interest of certain subsets of data. Log-driven recommender systems take the state of the current user's ongoing data analysis session as input and, based on previous exploration sessions by the same or other users, generate potentially promising recommendations for subsequent operations. These systems utilize a log database containing past exploration sessions and make recommendations based on the assumption that if two sessions have similar prefixes, their subsequent operations are likely to be similar.

[0008] In exploratory data analysis (EDA), measuring the interest of various data analysis operations is a key challenge in automating the EDA process. Therefore, researchers have proposed training specialized deep learning models to capture the complex measures of interest in EDA. These methods can be divided into two categories: value modeling and policy modeling. Value modeling methods train a scoring model to rate data analysis operations. These methods can be integrated into data-driven pattern recommendation systems to improve recommendation quality by reordering candidate operations. Policy modeling methods, on the other hand, do not explicitly model the value of data analysis operations; instead, they directly train a policy model to predict the next action that aligns with the user's interests.

[0009] Large Language Models (LLMs), supported by massive training datasets and a large number of parameters, have acquired extensive knowledge and possess capabilities such as instruction following and context learning, demonstrating potential close to human intelligence. Therefore, they are particularly suitable for automated exploratory data analysis (EDA). Existing work has explored the application of LLMs in automated data analysis tasks. However, these works lack terminology, are poorly connected, and have not yet formed a comprehensive research framework.

[0010] Some work utilizes LLMs to build automated data storytelling systems, aiming to transform raw data into logical and organized data stories (including multiple modalities such as text descriptions and visualizations). This task setup is similar to EDA. However, these systems typically employ a linear workflow, where all steps are performed sequentially without any loops. Therefore, insights derived from the data are not used to guide LLMs in generating further analysis but are directly output as the final result.

[0011] Furthermore, some systems combine LLMs with pattern recommendation systems, using large model agents (LLM agents) to operate the pattern recommendation systems for data analysis. This approach effectively combines bottom-up pattern discovery (induction) and top-down reasoning analysis (deduction). However, these systems rely entirely on the recommendation system for insights and can only perform a few predefined analytical actions, thus significantly limiting the scope of action of large model agents in EDA tasks.

[0012] Exploratory data analysis (EDA) emphasizes pattern discovery and data understanding, serving as a primary step in driving further in-depth analysis. Currently, insights gained through EDA are becoming increasingly important in decision-making across various fields. However, conducting exploratory data analysis still faces three main challenges, which are precisely the difficulties that the aforementioned background techniques cannot fully resolve.

[0013] Challenge 1: Observing and interpreting data in a specific domain requires researchers to possess expertise in that field. Even for medium-sized databases (tables), there are numerous potential data patterns, and enumerating all possible patterns is impractical. For example, for a two-dimensional data table with k columns (fields), there are k(k-1) / 2 binary correlations between fields alone, while the number of multivariate correlations grows exponentially. Many of these potential data patterns are obvious based on human knowledge, lack relevance to the current analytical task, or do not exhibit any meaningful characteristics. To identify or pre-determine which data patterns are most likely to contribute valuable findings, researchers must have a thorough understanding of the semantics and characteristics of the data itself and possess expertise in the relevant domain.

[0014] Challenge Two: Data analysis requires researchers to possess sufficient statistical knowledge and skills. Researchers, based on their own knowledge and observation of data, can discover some (highly probable) valuable data patterns. However, these are merely intuitive understandings and require rigorous statistical analysis for verification before they can be used to guide further analysis or support decision-making. Performing statistical analysis not only requires researchers to possess statistical knowledge and understanding of statistical methods (such as the chi-square test for analyzing variable independence), but also requires proficiency in common data analysis tools such as SQL, Python, and MATLAB. Furthermore, because the structure and semantics of data itself, as well as the analytical needs of users, are flexible and varied, the choice of data analysis methods is difficult to define as a specific model or a fixed set of procedures.

[0015] Challenge Three: Researchers face a trade-off between exploration and utilization in the EDA process. To discover meaningful patterns in data, researchers can adopt two basic strategies: first, propose high-probability hypotheses based on their existing knowledge and verify them through statistical analysis; second, directly observe and confirm significant patterns from the data. The former utilizes the researcher's existing knowledge, which we call deductive analysis or top-down analysis; the latter explores the hidden knowledge in the data, which we call inductive analysis or bottom-up analysis. The basic goal of EDA dictates that these two analytical modes need to work together, and neither is dispensable. EDA is often a multi-round iterative analytical process. In the process of conducting research, humans usually follow a thinking pattern that alternates between deductive and inductive reasoning: first, based on their own knowledge, they roughly define a subset of data with high analytical value (deduction); then, they gain insights by observing the data and update their own knowledge (induction), thereby guiding the next analytical goal (deduction).

[0016] The main difficulties faced by pattern recommendation systems lie in challenges one and two. Because these systems lack intelligence, they struggle to flexibly adjust their analysis plans based on users' analytical needs and interests. They typically can only perform a predefined set of analytical operations, or require users to engage in numerous complex and tedious interactions. Furthermore, pattern recommendation systems struggle to determine which patterns are valuable and which are not based on the nature of the data itself and the user's analytical intent; therefore, the data analysis results they return are often characterized by low information density and inconsistent quality.

[0017] In recent years, large language models (LLMs) have made significant progress in reasoning, planning, context learning, and tool invocation. LLM-based agents have shown considerable potential in understanding user intent and generating actions to achieve user goals. Therefore, multi-agent systems built on large language models are considered promising for addressing the aforementioned challenges. However, existing multi-agent systems still struggle to solve challenge three, for the following reasons.

[0018] 1. These systems rely on detailed analytical intent provided by the user, i.e., the so-called "hypotheses" of confirmatory data analysis. They can generate analysis plans, execute analysis operations, and collect results based on the user's provided intent, but they cannot independently raise valuable analytical questions, thus requiring continuous user interaction (human-in-the-loop). When the user's analytical intent is unclear, such systems struggle to conduct effective data analysis, or require the user to provide clearer analytical intent through questioning or other means. This contradicts the fundamental goals of exploratory data analysis.

[0019] 2. Such systems typically execute a one-off, single-round analysis process, rather than initiating further multi-round analyses based on the results. They usually return the analysis results to the user and wait for further analytical requests, rather than utilizing the existing results to guide further analysis. In contrast, exploratory data analysis is a closed-loop process combining inductive and deductive reasoning, requiring analysts to propose further analytical goals and plans based on the existing results.

[0020] Such systems fail to effectively combine bottom-up and top-down analytical paradigms. They typically develop analytical plans based on user-defined analytical needs rather than independently proposing analytical directions based on observations of the data itself. Consequently, the analytical results they return tend to fall within the user's expectations, and they are unable to autonomously discover valuable potential clues within the data. This is essentially a top-down analytical paradigm with low data utilization and insufficient exploratory capabilities. Summary of the Invention

[0021] The present invention addresses the limitations of existing technologies by providing an exploratory data analysis method and system based on interactive reasoning using large language models (LLMs). This invention proposes a multi-agent system based on large language models (LLMs), which primarily utilizes interactive reasoning through large model agents to achieve automated and high-quality exploratory data analysis. Users only need to interact with the system in natural language, expressing their analytical intentions, without requiring any data analysis skills or related domain knowledge.

[0022] The technical solution of this invention is as follows: An exploratory data analysis method based on interactive reasoning using a large language model is characterized by taking the initial question raised by the user as the root question node of the analysis tree, and performing the following steps starting from the root question node: Select a problem node from the analysis tree; retrieve relevant existing analysis results from the memory agent based on the currently selected problem node; the hypothesis generation agent performs deductive reasoning on the currently selected problem node and proposes new hypotheses as the hypothesis node for the currently selected problem node; For a hypothesis node selected from the analytic tree, the planning agent and the execution agent interact to execute the inner loop, generating and executing the verification plan; The inspection agent summarizes the results of the verification plan execution and stores them in the memory agent; The inner loop of the check agent verifies whether the data subset generated by the hypothesis is suitable for bottom-up analysis; if it is suitable, the check agent is called; otherwise, it checks whether the current analysis tree has reached the maximum depth. If the maximum depth has been reached, the analysis ends; otherwise, the problem proposal agent is called. The inspection agent performs bottom-up insight mining on the data subset and stores the mining results in the memory agent; The problem proposal agent proposes one or more new problems as problem nodes for the currently selected hypothesis node based on the analysis results of the current hypothesis node and in combination with relevant memories retrieved from the memory agent. Repeat the above steps until the preset conditions are met, at which point the loop terminates. The memory agent summarizes the questions, hypotheses, verification conclusions, and observations raised during the iterative analysis into an analysis report, which is then returned to the user as a response to the initial question.

[0023] Preferably, the inner loop steps include: The planning agent generates data query requirements in natural language and translates them into SQL statements; the execution agent executes the SQL queries in the selected database and returns a subset of data. The planning agent generates the Python code required for data analysis; executes the Python code and collects the output information; if the execution is error-free, the planning agent analyzes and reflects on the execution results of the Python code, checking for potential errors in the Python code; if potential errors are found, the potential errors in the Python code are corrected and the code is re-executed; otherwise, the checking agent is called to summarize the execution results of the Python code and store them in the memory agent; otherwise, based on the error message, the planning agent corrects the Python code and re-executes it.

[0024] Preferably, the inspection agent summarizes the data subset.

[0025] Preferably, the preset conditions include reaching a preset maximum computational budget limit or no new problems or assumptions being generated.

[0026] Preferably, the analysis tree consists of problem nodes and hypothesis nodes. Each problem node includes multiple hypothesis nodes as its child nodes, and each path from the root problem node to the leaf node is a thought chain formed by alternating problems and hypotheses.

[0027] Preferably, the problem proposal agent, hypothesis generation agent, and planning agent are treated as a whole as the deductive end, and the inspection agent is used as the inductive end; the deductive end is used to determine which parts of the original data to analyze, thereby improving the efficiency of data exploration; the inductive end is used to collect information from the data, thereby improving the utilization rate of the data.

[0028] Preferably, the deductive end calls the execution agent, the inductive end checks the execution result of the execution agent and injects it into the memory agent, and the deductive end uses the knowledge accumulated in the memory agent as guidance.

[0029] An exploratory data analysis system based on interactive reasoning using a large language model is characterized by comprising a problem proposal agent, a hypothesis generation agent, a planning agent, an execution agent, an inspection agent, and a memory agent. The problem proposal agent and the hypothesis generation agent are used to guide the agent's exploration path in the data space during exploratory data analysis. Specifically, the problem proposal agent retrieves relevant existing analysis results from the memory agent based on the problem node selected from the analysis tree; the hypothesis generation agent performs deductive reasoning on the currently selected problem node to propose new hypotheses as the hypothesis node for the currently selected problem node; and the problem proposal agent proposes one or more new questions as the problem node for the currently selected hypothesis node based on the analysis results of the current hypothesis node and in conjunction with relevant memories retrieved from the memory agent. The planning agent and the execution agent collaborate to verify the hypothesis; wherein, the planning agent is responsible for generating the verification plan, and the execution agent is responsible for calling external tools to execute the verification plan and collecting feedback; for a hypothesis node selected from the analysis tree, the planning agent and the execution agent interactively execute an inner loop to generate and execute the verification plan; The inspection agent is responsible for summarizing the execution results of the verification plan and storing them in the memory agent; and checking whether the data subset generated by verifying the hypothesis in the inner loop is suitable for bottom-up analysis; if it is suitable, the inspection agent is called; otherwise, it checks whether the current analysis tree has reached the maximum depth; if it has reached the maximum depth, the analysis ends; otherwise, the problem proposal agent is called; bottom-up insight mining is performed on the data subset, and the mining results are stored in the memory agent. The memory agent is used to record the questions proposed by the question proposal agent, the hypotheses proposed by the hypothesis generation agent and their corresponding verification results, and the processing results of the check agent.

[0030] A server is characterized by comprising a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the methods described above.

[0031] A computer-readable storage medium having a computer program stored thereon, characterized in that the computer program implements the above-described method when executed by a processor.

[0032] This patent, as a multi-agent system based on a large language model for automating exploratory data analysis, is characterized by the following: 1. It can autonomously discover valuable data patterns through data exploration and analysis without requiring users to provide clear and explicit analytical intentions, demonstrating a high degree of intelligence; 2. It can conduct multi-round, closed-loop data analysis processes based on existing data analysis results without user intervention. It can iteratively explore the data until all the knowledge hidden in the data is discovered, with a high degree of automation. 3. By combining top-down (deductive) and bottom-up (inductive) data analysis paradigms, a reasonable balance is struck between utilizing prior knowledge and exploring implicit knowledge within the data, resulting in high data utilization.

[0033] The beneficial effects of this invention are as follows: To address Challenge 1, this system employs a multi-agent system based on Large Language Models (LLMs). The large model agents autonomously propose valuable research questions and hypotheses to guide the exploratory data analysis process. This process implicitly introduces highly human-aligned interest metrics, avoiding meaningless, valueless, and superficial data analysis operations. LLMs possess extensive global knowledge and can understand user intent, data structure and semantics, and other guiding information expressed in natural language, thus facilitating value judgments relevant to specific research contexts. For domain-specific analytical tasks, the domain knowledge of vertical domain large models can be utilized, or the necessary knowledge can be injected into general domain large models through Retrival Augmented Generation (RAG). Traditional non-large model solutions struggle to learn highly complex interest metrics.

[0034] To address Challenge 2, this system utilizes powerful and flexible SQL and Python as the primary tools for data analysis. These tools can perform most statistical analysis operations, thus meeting the requirements of exploratory data analysis. However, generating corresponding code based on complex and ever-changing user intent remains a challenge. Unlike traditional solutions, this system leverages large model agents (LLMs) to generate and iteratively improve code. LLMs demonstrate powerful capabilities in code generation and tool invocation, thereby enhancing the flexibility and automation of statistical analysis. By generating and executing code, LLMs can autonomously determine and execute a sufficiently broad set of data analysis operations, rather than being limited to a predefined set of analysis operations. Furthermore, if needed, specialized analysis tools can be designed for specific analysis tasks and invoked using the context learning and instruction-following capabilities of LLMs.

[0035] To address Challenge 3, this system utilizes different large-scale model agents to perform top-down inference analysis and bottom-up pattern mining, respectively. Specifically, the problem proposal agent and hypothesis generation agent, starting from user intent and combining domain knowledge both within and outside the model, infer valuable analytical objectives (i.e., research questions and hypotheses) relevant to the current task. On the other hand, the planning agent and execution agent analyze and explore the data, while the checking agent discovers interesting patterns by observing the analysis results. These discoveries serve as knowledge sources for the next round of iterations, allowing agents to conduct multiple rounds of iterative exploration of the data. Attached Figure Description

[0036] Figure 1 This is a flowchart of the method of the present invention.

[0037] Figure 2 This is a system framework diagram.

[0038] Figure 3This is a flowchart of a method according to an embodiment of the present invention.

[0039] Figure 4 This is a system diagram of the present invention. Detailed Implementation

[0040] To make the above features and advantages of the present invention more apparent and understandable, specific embodiments are described below in conjunction with the accompanying drawings.

[0041] like Figure 1 As shown, an embodiment of the present invention provides an exploratory data analysis method based on interactive reasoning using a large language model. The method is characterized by using the initial question raised by the user as the root question node of the analysis tree, and executing the following steps starting from the root question node: Select a problem node from the analysis tree; retrieve relevant existing analysis results from the memory agent based on the currently selected problem node; the hypothesis generation agent performs deductive reasoning on the currently selected problem node and proposes new hypotheses as the hypothesis node for the currently selected problem node; For a hypothesis node selected from the analytic tree, the planning agent and the execution agent interact to execute the inner loop, generating and executing the verification plan; The inspection agent summarizes the results of the verification plan execution and stores them in the memory agent; The inner loop of the check agent verifies whether the data subset generated by the hypothesis is suitable for bottom-up analysis; if it is suitable, the check agent is called; otherwise, it checks whether the current analysis tree has reached the maximum depth. If the maximum depth has been reached, the analysis ends; otherwise, the problem proposal agent is called. The inspection agent performs bottom-up insight mining on the data subset and stores the mining results in the memory agent; The problem proposal agent proposes one or more new problems as problem nodes for the currently selected hypothesis node based on the analysis results of the current hypothesis node and in combination with relevant memories retrieved from the memory agent. Repeat the above steps until the preset conditions are met, at which point the loop terminates. The memory agent summarizes the questions, hypotheses, verification conclusions, and observations raised during the iterative analysis into an analysis report, which is then returned to the user as a response to the initial question.

[0042] like Figure 2 As shown, an optional embodiment of the present invention provides a multi-agent system composed of multiple Large Model Agents (LLM Agents), including six agents: a problem proposal agent, a hypothesis generation agent, a planning agent, an execution agent, an inspection agent, and a memory agent. These agents iteratively interact and autonomously conduct multiple rounds of exploratory data analysis. The definitions and functions of each agent are introduced below, followed by the specific flow of the system operation.

[0043] 1. Problem Proposal Agent and Hypothesis Generation Agent We formalize the top-down data analysis process as a loop of "questioning - hypothesis making - hypothesis testing". The Questioner (question proposal agent) and Proposer (hypothesis generation agent) are the core of this process. The former proposes further research questions based on already verified hypotheses, while the latter makes hypotheses about the proposed research questions. Their interaction dominates the agent's exploration path in the data space during EDA, thus forming the core loop of the framework proposed in this invention.

[0044] Specifically, starting with the initial question posed by the user (i.e., the User Query, which can be a very broad analytical requirement or empty), the interaction between the Questioner and Proposer gradually constructs an "analysis tree." The analysis tree is the primary data structure upon which this invention is based for executing a multi-round closed-loop data analysis process combining induction and deduction. The analysis tree consists of question nodes and hypothesis nodes; each question node has multiple hypothesis nodes as its child nodes, and vice versa. Therefore, any path from the root node to a leaf node is a thought chain formed by alternating questions and hypotheses. This aligns with the general pattern of human exploratory data analysis: making hypotheses about the proposed questions, conducting data analysis guided by these hypotheses, and proposing new questions based on the data analysis results, thus repeating the cycle. Each time a new question is proposed or a new hypothesis is verified, the Shared Memory module updates the shared memory with useful information to guide subsequent data analysis processes.

[0045] The quality of the questions posed by the Questioner and the hypotheses proposed by the Proposer has a decisive impact on the effectiveness of top-down data analysis. Therefore, improving the ability of intelligent agents to raise research questions and make reasonable hypotheses is one of the key technical challenges and difficulties of AI for Data and even AI for Science. Taking question formulation as an example, this invention believes that its quality includes the following two aspects: Feasibility: This refers to whether the problem can be answered by analyzing existing data. The feasibility of a problem depends on whether the information needed to solve it lies entirely within the semantic boundaries of the given database.

[0046] Importance: This refers to the value of a problem to the user. A problem that is of interest to the user and has research value is valuable. The general research value of a problem is determined by domain knowledge, while the value of a problem to a specific user is determined by the user's intention.

[0047] Furthermore, judging quality requires intelligent agents to possess certain specific knowledge. For example, to determine whether a problem has research value, an intelligent agent needs a thorough understanding of the relevant domain's expertise; to determine whether a problem can be answered based on the current database, an intelligent agent needs a comprehensive understanding of the database's structure and semantics. Because the specific contexts faced by data analysis are diverse, potentially involving various professional fields and database tables, this required knowledge cannot be pre-loaded into the EDA system. Even for general-domain models with broad knowledge, handling certain vertical tasks (such as medical-related data analysis) remains challenging.

[0048] For the problem of feasibility assessment, the solution of this invention is: first, the database tables and columns are annotated manually, or the meaning of the tables and columns is inferred by the large model. Then, these annotations are placed in the context of the large model, and its context learning and reasoning abilities are used to assess the feasibility of specific problems and hypotheses.

[0049] To address the problem of a lack of domain knowledge, the solution of this invention is to inject professional knowledge into a general domain model by using Retrival Augmented Generation (RAG), or to draw on the idea of ​​Mixed Expert Models (MoEs) to call different vertical domain models according to the specific application scenario.

[0050] 2. Planning and execution agency After the intelligent agent proposes high-quality research questions and corresponding hypotheses, the Planner and Executor collaborate to verify (prove or disprove) the hypotheses. The former is responsible for generating the Verification Plan, while the latter is responsible for calling external tools to execute the Verification Plan and collecting feedback. The two need to work closely together and interact multiple times to iteratively optimize the execution plan.

[0051] The primary design choice in generating the validation plan is how to define and invoke the toolset. A toolset is a predefined set of operations used to interact with the data. Recent work has widely utilized the planning and reasoning capabilities of Large Language Models (LLMs) to select appropriate tools and determine the invocation parameters, while the definition of the toolset varies. For example, some works define a series of functions for interacting with tables (selecting rows, selecting columns, aggregation, sorting, etc.), each receiving specific parameters. The LLM agent processes the raw table by iteratively invoking these functions, progressively simplifying it until the desired result is obtained. Other works allow the LLM agent to choose from three actions: directly answering questions based on the table content, generating SQL statements to process the table, or generating Python code to process the table.

[0052] Considering the highly flexible questions and hypotheses posed by the Questioner and Proposer, this invention selects SQL and Python code as its toolset to leverage their powerful and flexible data processing capabilities to accomplish various data analysis tasks. Specifically, the system implemented in this invention defines the execution plan as follows: The execution plan consists of three parts: a data query request expressed in natural language, specifying the subset of data needed to verify a given hypothesis; SQL statements used to extract the required data from the database and perform preliminary processing (aggregation, sorting, etc.); and a Python script used to perform relatively complex processing and analysis on the retrieved data.

[0053] Another important design choice is the interaction flow between the Planner and the Executor. The execution plan generated by the Planner cannot be guaranteed to be correct, so it is necessary to adjust the execution plan using feedback collected by the Executor. The framework proposed in this invention requires the Planner to first generate an initial execution plan, then submit it to the Executor for execution, adjust the execution plan based on the execution results, and submit it to the Executor for execution again. This process is repeated continuously until the execution plan is error-free or the maximum number of repetitions is reached. During this process, there are two scenarios where the execution plan needs adjustment: first, an error occurs during execution, resulting in no execution result; second, no error occurs during execution, but the execution plan itself contains logical fallacies, leading to incorrect execution results. Taking Python code as an example: for the former, the original code can be corrected using the error messages provided by the code interpreter; for the latter, potential errors can be checked through self-reflection of the large language model.

[0054] 3. Check the agent The Inspector's task is twofold. First, the Inspector is responsible for providing the final answer to the hypothesis. In other words, after the Planner and Executor collaborate to produce an error-free execution plan and its results, the Inspector is responsible for analyzing whether the results support, contradict, or fail to answer the corresponding hypothesis, and summarizing the entire validation plan. Second, the Inspector is responsible for extracting other useful information from the hypothesis validation process. In other words, the Inspector will examine the data and intermediate results involved in the execution process, extracting data insights from them.

[0055] The Inspector's second task—examining the data and intermediate results involved in the execution process and extracting data insights—is a key manifestation of the bottom-up analysis approach within the EDA framework proposed in this invention, and is crucial for maximizing data utilization. During hypothesis testing, the system extracts a subset of data (views) from the database and utilizes data analysis tools (such as Python) to acquire certain knowledge. However, this top-down analysis serves a predetermined purpose (i.e., the hypothesis) and often supports or refutes a given hypothesis by calculating certain statistics. As data is filtered, aggregated, and compressed, a significant amount of information contained in the original data is lost. In particular, if certain interesting patterns cannot be derived beforehand and presented as hypotheses, they may be overlooked in the analysis. The goal of the Questioner, Proposer, and Planner is to guide the data analysis process with hypotheses, but these hypotheses are inevitably constrained by the capabilities and cognitive biases of intelligent agents, causing many potentially interesting patterns to remain undiscovered. Therefore, the Inspector needs to directly observe the data content obtained during hypothesis testing and derive valuable insights from it. The meaning of "valuable" here is similar to that of "important" as defined earlier.

[0056] There are various design options for extracting valuable insights from raw data from a bottom-up perspective. As an example of this invention, various pattern recommendation systems can be invoked to perform pattern mining on the data subset extracted during data analysis, and then the Inspector can select the data patterns obtained through mining.

[0057] As a preferred embodiment of the present invention, the data subset content can be directly placed in the context of the large model and provided to the Inspector, directly prompting it to extract insights from it. These data subsets are extracted, aggregated, and preliminarily processed from massive amounts of raw data during the gradual construction of the analysis tree. They are data subsets with high potential analytical value, and their data volume is greatly reduced compared to the original data, while the information density is greatly increased, significantly reducing the difficulty of bottom-up insight mining.

[0058] It is worth noting that the EDA framework proposed in this invention can be viewed as an alternation of deductive and inductive steps. If we consider Questioner, Proposer, and Planner as a whole, they constitute the "deductive end" of the framework, while Inspector constitutes the "inductive end." The deductive end determines which parts of the raw data to analyze, thereby improving the efficiency of data exploration; the inductive end collects as much information as possible from the data, thereby improving data utilization. The deductive and inductive ends are connected through two paths: one is the Executor, where the deductive end calls the Executor, and the inductive end checks its results; the other is Shared Memory, where the inductive end injects knowledge into Shared Memory, and the deductive end uses the accumulated knowledge therein as guidance.

[0059] 4. Memory Proxy Shared Memory is responsible for recording and organizing information derived from previously analyzed data in the EDA process. Specifically, questions posed by the Questioner, hypotheses proposed by the Proposer (and their validation results), and insights extracted by the Inspector are all recorded in Shared Memory. The Questioner and Proposer retrieve "memories" from Shared Memory as guidance for formulating questions or hypotheses. This avoids generating duplicate questions or hypotheses and allows the intelligent agent to be inspired by previous research findings. Shared Memory is key to achieving effective feedback between actions and reasoning.

[0060] Not limited to this invention, memory modules are crucial components for a large portion of large model-based intelligent agents (LLM-based agents). They are responsible for storing information perceived from the environment and using recorded memories to facilitate future behavioral decisions. Memory modules help agents accumulate experience, evolve, and behave more consistently, rationally, and efficiently. The design of agent memory modules often draws on research findings in cognitive science regarding human memory processes. Structurally, human memory typically follows a process from sensory memory (registering perceptual input), to short-term memory (temporarily retaining information), and then to long-term memory (integrating information over a long period). Researchers are also inspired by these human memory mechanisms in the design of agent memory structures. Specifically, short-term memory typically corresponds to input information defined by the context window of the Transformer architecture; while long-term memory resembles an external vector storage system, allowing agents to quickly query and retrieve information as needed. Format-wise, the format of the memory storage medium is also an important design choice for agent memory modules, such as natural language memory or embedding-based memory.

[0061] The Shared Memory proposed in this invention supports four operations: initialization, update, read, and summary. The initialization operation saves basic information to Shared Memory, which may include user intent (Query), database structure information (Schema), knowledge related to the current task (Knowledge), and even preliminary database exploration results, etc. The update operation updates Shared Memory with newly obtained information from the data. The read operation retrieves historical information from Shared Memory. The summary operation filters, refines, and summarizes the information in Shared Memory, generates a data analysis report, and returns it to the user as the final EDA output.

[0062] The operating process of this system is as follows: Figure 3 As shown, the process consists of an outer loop and an inner loop. The outer loop, or "question posing - hypothesis making - hypothesis testing - result collection" loop, is the main loop executed by the system. Through this process, the system effectively combines top-down deductive reasoning (i.e., the "question posing - hypothesis making - hypothesis testing" process) with bottom-up pattern discovery (i.e., checking the agent's summarization and extraction of interesting information from the data analysis results). The inner loop refers to the process of generating and executing the verification plan, which is also the stage where the actual data analysis operations are performed. By continuously collecting feedback information and improving the code, the success rate of data analysis operations can be greatly improved.

[0063] The initial analysis requirement provided by the user (a broad question indicating the user's basic analytical intent) serves as the root question node of the analysis tree. After initialization, the analysis tree contains only this root question node. Based on this initial analysis tree, the system performs the following steps (outer loop): S1. Select a problem node from the parse tree. Retrieve relevant existing analysis results from the memory agent; the hypothesis generation agent combines internal and external knowledge of the model with relevant memories to perform deductive reasoning for the currently selected problem node, and proposes new hypotheses as child nodes (hypothesis nodes) of the currently selected problem node, thereby updating the parse tree.

[0064] S2. Based on the current assumptions, the planning agent and execution agent interact to execute the inner loop, generating and executing the verification plan. This step requires the following sub-steps: S21. The planning agent generates data query requirements in natural language format; S22. The planning agent translates natural language query requests into SQL statements; S23. The execution agent executes an SQL query in the selected database and returns a subset of data; S24. Plan the agent to generate the Python code required for data analysis (i.e., Python code to verify the correctness of the current hypothetical node); S25. Execute the Python code using the agent and collect the output information; if the execution is error-free, proceed to step S27; otherwise, proceed to step S26. S26. Based on the error message, plan the agent to correct the Python code, and jump to step S25; S27. The planning agent analyzes and reflects on the execution results of the Python code, checking for potential errors in the Python code. If a potential error is found, proceed to step S28; otherwise, proceed to step S3 to call the checking agent. S28. Plan the agent to correct potential errors in the Python code, then proceed to step S25.

[0065] S3. The inspection agent summarizes the execution results of the Python code (i.e., the execution results of the verification plan) and stores the summary results in the memory agent.

[0066] S4. Examine the subset of data obtained in step S2 of the agent observation and analyze whether it is suitable for bottom-up analysis. If it is suitable, proceed to step S5; otherwise, proceed to step S6.

[0067] S5. Examine the proxy call pattern recommendation system or directly observe the data, perform bottom-up insight mining on the data subset, and store the results in the memory proxy.

[0068] S6. If the current analytic tree has reached its maximum depth, the process ends; otherwise, the problem proposal agent proposes one or more new problems as child nodes (problem nodes) of the currently selected hypothesis node based on the analysis results of the current hypothesis and in combination with the relevant memories retrieved in the memory agent, thereby updating the analytic tree, and then jumps to step S1.

[0069] The loop terminates when the system reaches its preset maximum computational budget limit, or when the system determines that no new questions or hypotheses can be generated. Afterward, the memory agent summarizes the questions, hypotheses, verification conclusions, and observations raised during the iterative analysis into an analysis report, which is returned to the user as the final result, serving as a response to the user's initial analysis questions.

[0070] like Figure 4 As shown, an embodiment of the present invention provides an exploratory data analysis system based on interactive reasoning of a large language model, characterized in that it includes a problem proposal agent, a hypothesis generation agent, a planning agent, an execution agent, an inspection agent, and a memory agent; The problem proposal agent and the hypothesis generation agent are used to guide the agent's exploration path in the data space during exploratory data analysis. Specifically, the problem proposal agent retrieves relevant existing analysis results from the memory agent based on the problem node selected from the analysis tree; the hypothesis generation agent performs deductive reasoning on the currently selected problem node to propose new hypotheses as the hypothesis node for the currently selected problem node; and the problem proposal agent proposes one or more new questions as the problem node for the currently selected hypothesis node based on the analysis results of the current hypothesis node and in conjunction with relevant memories retrieved from the memory agent. The planning agent and the execution agent collaborate to verify the hypothesis; wherein, the planning agent is responsible for generating the verification plan, and the execution agent is responsible for calling external tools to execute the verification plan and collecting feedback; for a hypothesis node selected from the analysis tree, the planning agent and the execution agent interactively execute an inner loop to generate and execute the verification plan; The inspection agent is responsible for summarizing the execution results of the verification plan and storing them in the memory agent; and checking whether the data subset generated by verifying the hypothesis in the inner loop is suitable for bottom-up analysis; if it is suitable, the inspection agent is called; otherwise, it checks whether the current analysis tree has reached the maximum depth; if it has reached the maximum depth, the analysis ends; otherwise, the problem proposal agent is called; bottom-up insight mining is performed on the data subset, and the mining results are stored in the memory agent. The memory agent is used to record the questions proposed by the question proposal agent, the hypotheses proposed by the hypothesis generation agent and their corresponding verification results, and the processing results of the check agent.

[0071] An embodiment of the present invention provides a server, characterized in that it includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the above-described method.

[0072] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program implements the above-described method when executed by a processor.

[0073] Although the above preferred embodiments have been described in detail, they are merely for illustrating the technical solutions of the embodiments of the present invention and not for limiting them. Those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions, but should not depart from the principles and scope of the technical solutions of the embodiments of the present invention. Any equivalent substitutions and modifications that do not depart from the principles and core features of the embodiments of the present invention should be included within the scope of the present invention, which is defined by the claims.

Claims

1. An exploratory data analysis method based on large language model interactive inference, characterized in that, The initial question posed by the user is taken as the root question node of the analysis tree, and the following steps are performed starting from the root question node: Select a problem node from the analysis tree; retrieve relevant existing analysis results from the memory agent based on the currently selected problem node; the hypothesis generation agent performs deductive reasoning on the currently selected problem node and proposes new hypotheses as the hypothesis node for the currently selected problem node; For a hypothesis node selected from the analytic tree, the planning agent and the execution agent interact to execute the inner loop, generating and executing the verification plan; The inspection agent summarizes the results of the verification plan execution and stores them in the memory agent; Check whether the subset of data generated in the proxy detection inner loop is suitable for bottom-up analysis, verifying the hypothesis. If suitable, invoke the check agent; Otherwise, check if the current parse tree has reached its maximum depth. If it has, end the analysis; otherwise, call the problem proposal agent. The inspection agent performs bottom-up insight mining on the data subset and stores the mining results in the memory agent; The problem proposal agent proposes one or more new problems as problem nodes for the currently selected hypothesis node based on the analysis results of the current hypothesis node and in combination with relevant memories retrieved from the memory agent. Repeat the above steps until the preset conditions are met, at which point the loop terminates. The memory agent summarizes the questions, hypotheses, verification conclusions, and observations raised during the iterative analysis process into an analysis report, which is returned to the user as a response to the initial questions.

2. The method of claim 1, wherein, The steps of the inner loop include: The planning agent generates data query requirements in natural language and translates them into SQL statements; the execution agent executes the SQL queries in the selected database and returns a subset of data. The planning agent generates the Python code required for data analysis; executes the Python code and collects the output information; if the execution is error-free, the planning agent analyzes and reflects on the execution results of the Python code, checking for potential errors in the Python code; if potential errors are found, the potential errors in the Python code are corrected and the code is re-executed; otherwise, the checking agent is called to summarize the execution results of the Python code and store them in the memory agent; otherwise, based on the error message, the planning agent corrects the Python code and re-executes it.

3. The method of claim 2, wherein, The inspection agent summarizes the subset of data.

4. The method according to claim 1 or 2 or 3, characterized in that, The preset conditions include reaching the preset maximum computational budget limit or no new problems or assumptions being generated.

5. The method according to claim 1 or 2 or 3, characterized in that, The analysis tree consists of problem nodes and hypothesis nodes. Each problem node includes multiple hypothesis nodes as its child nodes. Each path from the root problem node to the leaf node is a thought chain formed by alternating problems and hypotheses.

6. The method according to claim 1, 2, or 3, characterized in that, The problem proposal agent, hypothesis generation agent, and planning agent are treated as a whole as the deductive end, and the inspection agent is used as the inductive end; the deductive end is used to determine which parts of the original data to analyze, thereby improving the efficiency of data exploration. The summarizing end is used to collect information from the data, thereby improving the utilization rate of the data.

7. The method according to claim 6, characterized in that, The deductive end calls the execution agent, the inductive end checks the execution result of the execution agent and injects it into the memory agent, and the deductive end uses the knowledge accumulated in the memory agent as guidance.

8. An exploratory data analysis system based on interactive reasoning using a large language model, characterized in that, This includes problem-proposing agents, hypothesis-generating agents, planning agents, execution agents, inspection agents, and memory agents; The problem proposal agent and the hypothesis generation agent are used to guide the agent's exploration path in the data space during exploratory data analysis. Specifically, the problem proposal agent retrieves relevant existing analysis results from the memory agent based on the problem node selected from the analysis tree; the hypothesis generation agent performs deductive reasoning on the currently selected problem node to propose new hypotheses as the hypothesis node for the currently selected problem node; and the problem proposal agent proposes one or more new questions as the problem node for the currently selected hypothesis node based on the analysis results of the current hypothesis node and in conjunction with relevant memories retrieved from the memory agent. The planning agent and the execution agent collaborate to verify the hypothesis; wherein, the planning agent is responsible for generating the verification plan, and the execution agent is responsible for calling external tools to execute the verification plan and collecting feedback; for a hypothesis node selected from the analysis tree, the planning agent and the execution agent interactively execute an inner loop to generate and execute the verification plan; The inspection agent is responsible for summarizing the execution results of the verification plan and storing them in the memory agent; and checking whether the data subset generated by verifying the hypothesis in the inner loop is suitable for bottom-up analysis; if it is suitable, the inspection agent is called; otherwise, it checks whether the current analysis tree has reached the maximum depth; if it has reached the maximum depth, the analysis ends; otherwise, the problem proposal agent is called; bottom-up insight mining is performed on the data subset, and the mining results are stored in the memory agent. The memory agent is used to record the questions proposed by the question proposal agent, the hypotheses proposed by the hypothesis generation agent and their corresponding verification results, and the processing results of the check agent.

9. A server, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images