A method and system for adaptive data analysis

Abstract

This disclosure relates to a method and system for adaptive data analysis. The method includes: generating natural language description information for a data analysis task; determining multiple subtasks in a workflow corresponding to the data analysis task based on the natural language description information; generating flow control information for the workflow; scheduling the execution of the multiple subtasks based on the flow control information, including: determining an algorithm execution task for the multiple subtasks; generating executable code for each of the multiple subtasks using a Large Language Model (LLM) based on the algorithm execution task; generating analysis results for the multiple subtasks in response to successful execution of the executable code; and obtaining a final analysis report based on the analysis results.

Description

Technical Field

[0001] This disclosure relates to the field of data analysis technology, and more specifically to a method and system for adaptive data analysis. Background Technology

[0002] Current mainstream automated data analysis solutions, including general-purpose large model code interpreters, AutoML tools, and data analysis agents, typically employ patterns such as "single-agent free generation," "unidirectional linear execution," or "static pipeline orchestration." While these solutions possess certain data processing, modeling, and report generation capabilities, they still suffer from the following shortcomings: (1) The planning process relies too much on implicit reasoning within the model. Existing systems usually rely on the large model to determine the analysis sequence and method selection on its own, lacking a mechanism to explicitly inject standard steps, method dependencies and process constraints in statistical analysis into the planning process. Therefore, problems such as process omissions, unstable sequence and insufficient overall controllability are likely to occur.

[0003] (2) Insufficient ability to jointly understand data context and task scenario. Existing systems often rely mainly on user natural language instructions for analysis, lacking a joint understanding of structured data features and external auxiliary document information. Therefore, it is difficult to accurately identify the domain of the task, the focus of analysis, and the appropriate way to express the results.

[0004] (3) Unclear system-level coordination and module-level execution. Although some solutions have the ability to collaborate with multiple modules, they usually lack a clear division of labor that combines global planning with local autonomy, making it difficult to simultaneously ensure the consistency of the overall process and the flexible adaptability of modules.

[0005] (4) Insufficient robustness of code execution. When the preprocessing or modeling code fails to execute, the existing system usually terminates directly or only performs a simple retry, lacking a closed-loop mechanism for joint retrieval and repair based on error information, original code and external knowledge.

[0006] (5) The report generation lacks a structured organizational mechanism. Existing systems often directly splice intermediate results into a report, lacking a chapter-level organizational process based on the outline and intermediate products, resulting in insufficient logical coherence, traceability and editability of the final report.

[0007] Therefore, there is an urgent need for a method and system to further improve existing automated data analysis solutions. Summary of the Invention

[0008] This summary is provided to introduce, in a simplified form, some concepts that will be further described in the following detailed description section. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0009] To address the problems in existing technologies, this disclosure provides a method and system for adaptive data analysis. In the technical solution of this disclosure, firstly, multi-source contextual fusion is performed to generate an analysis profile, including analysis task description information from document information and CSV data tables. Then, a structured prior-driven workflow control mechanism schedules multiple sub-tasks within the workflow corresponding to the analysis task. Furthermore, a Suggest-Code-Refine (SCR) execution mechanism and a Retrieval-Augmented Generation (RAG) code generation mechanism are combined to implement specific algorithm execution and anomaly repair, ultimately completing the analysis result output and report assembly.

[0010] Specifically, in a first aspect of the present invention, a method for adaptive data analysis is disclosed, the method comprising: Receive description information of the data analysis task and extract task feature information from the description information; Generate natural language description information for the data analysis task based on the task feature information; Based on the natural language description information, determine multiple sub-tasks in the workflow corresponding to the data analysis task; The standard steps, step dependencies, sequence constraints, and step execution result determination logic corresponding to the multiple sub-tasks are encoded into structured priors to generate the workflow control information based on the structured priors. The execution of the plurality of sub-tasks is scheduled based on the process control information, including for each of the plurality of sub-tasks: Determine the algorithm to be used for this subtask; Based on the algorithm, the task execution uses a Large Language Model (LLM) to generate executable code for that subtask; and Execute the executable code and repeat the following steps if execution fails until success: Initiate searches to multiple knowledge sources to obtain repair context information; The executable code is regenerated and executed based on the repair context information; In response to the successful execution of the executable code, the analysis results of the multiple subtasks are generated; and A final analysis report is obtained based on the analysis results.

[0011] In one optional implementation, the explanatory information includes document information and a comma-separated values ​​(CSV) data table regarding the data analysis task. The document information includes auxiliary documents, historical records, background information, a directory structure, and task description text. The extraction of task feature information includes: extracting textual semantic features from the document information; and extracting metadata features from the CSV data table. The task feature information includes both the metadata features and the textual semantic features. In this implementation, joint modeling based on "text-side context + data-side context" can improve the accuracy of task scenario identification and provide a more realistic contextual basis for subsequent workflow scheduling and code generation.

[0012] In one optional implementation, scheduling the execution of the plurality of subtasks further includes, for each of the plurality of subtasks: executing the next subtask if the subtask executes successfully; or backtracking to execute the previous subtask if the subtask fails. This implementation supports non-linear loops throughout the entire analysis task flow; that is, the analysis task flow is no longer a unidirectional linear link flow, but a dynamic flow that allows backtracking, iteration, and optimization based on the execution results of each task node.

[0013] In one optional implementation, generating the executable code includes: constructing a retrieval query based on the algorithm execution task; initiating a retrieval from multiple knowledge sources based on the retrieval query to obtain retrieval results, including API documentation, sample code, and domain knowledge, wherein the multiple knowledge sources include local knowledge sources and online knowledge sources; and generating the executable code based on the retrieval results, according to the natural language description information and the flow control information, and regenerating and executing the executable code includes: in response to the executable code execution failure, capturing error context information, including error type, call stack, variable context, dependency version information, and runtime environment characteristics; constructing a retrieval query based on the error context information; initiating a retrieval from the multiple knowledge sources based on the retrieval query; obtaining API documentation and solution information for repairing the executable code through the retrieval and using it together with the error context information as the repair context information to regenerate the executable code; and executing the repaired executable code in a sandbox execution environment. In this implementation, when the model's internal knowledge is insufficient, the external library version changes, or the calling parameters are mismatched, relevant knowledge can be obtained through external retrieval and the code can be corrected. This significantly improves the success rate and robustness of complex data analysis tasks and enables each task node to have adaptive adjustment and continuous optimization capabilities.

[0014] In one optional implementation, the analysis results include intermediate products, charts, and text content. Obtaining a final analysis report based on the analysis results includes: performing retrieval, filtering, and chapter-level structured organization on the analysis results based on the explanatory information. If organization fails, the following operations are repeated until successful: initiating searches to multiple knowledge sources to obtain repair context information; reorganizing the analysis results based on the repair context information; and generating the final analysis report based on the organization. In this implementation, the final analysis report generated through the above-described retrieval-enhanced content organization method not only summarizes the overall process results but also maintains the traceability of content sources and the logical consistency of chapter organization, making the report more suitable for scientific research analysis, business decision-making, and formal reporting scenarios.

[0015] In a second aspect of the invention, a system for adaptive data analysis is disclosed, the system comprising: The analysis task description module is configured as follows: Receive description information of the data analysis task and extract task feature information from the description information; Generate natural language description information for the data analysis task based on the task feature information; The task flow control module is configured as follows: Based on the natural language description information, determine multiple sub-tasks in the workflow corresponding to the data analysis task; The standard steps, step dependencies, sequence constraints, and step execution result determination logic corresponding to the multiple subtasks are encoded into structured priors to generate the workflow's process control information based on these structured priors; and The execution of the plurality of subtasks is scheduled based on the process control information, including determining the algorithm execution task for each of the plurality of subtasks. The task code generation module is configured to, for each of the plurality of subtasks: Based on the algorithm, the task is executed using a Large Language Model (LLM) to generate executable code for the subtask. Execute the executable code and repeat the following steps if execution fails until success: Initiate searches to multiple knowledge sources to obtain repair context information; Based on the repair context information, the executable code is regenerated and executed; and The analysis report assembly module is configured as follows: In response to the successful execution of the executable code, the analysis results of the multiple subtasks are generated; A final analysis report is obtained based on the analysis results.

[0016] In one optional implementation, the explanatory information includes document information and a comma-separated CSV data table regarding the data analysis task. The document information includes auxiliary documents, historical records, background information, a directory structure, and task description text. The analysis description module is further configured to extract task feature information by: extracting textual semantic features from the document information; and extracting metadata features from the CSV data table. The task feature information includes both the metadata features and the textual semantic features. In this implementation, joint modeling based on "text-side context + data-side context" can improve the accuracy of task scenario identification and provide a more realistic context for subsequent workflow scheduling and code generation.

[0017] In an optional implementation, the task flow control module is further configured to schedule the execution of the plurality of subtasks by performing the following operations for each subtask: executing the next subtask if the subtask executes successfully; or backtracking to execute the previous subtask if the subtask fails. In this implementation, non-linear loops of the entire analysis task flow can be supported; that is, the analysis task flow is no longer a unidirectional linear link flow, but a dynamic flow that allows backtracking, iteration, and optimization based on the execution results of each task node.

[0018] In an optional implementation, the task code generation module is further configured to generate the executable code by: constructing a retrieval query based on the algorithm to execute a task; initiating a retrieval from the plurality of knowledge sources based on the retrieval query to obtain retrieval results, including API documentation, sample code, and domain knowledge, wherein the plurality of knowledge sources include local knowledge sources and online knowledge sources; and generating the executable code based on the retrieval results, according to the natural language description information and the flow control information. The task code generation module is further configured to regenerate and execute the executable code by: capturing error context information, including error type, call stack, variable context, dependency version information, and runtime environment characteristics, in response to the executable code execution failure; constructing a retrieval query based on the error context information; initiating a retrieval from the plurality of knowledge sources based on the retrieval query; obtaining API documentation and solution information for repairing the executable code through the retrieval and using it together with the error context information as the repair context information to regenerate the executable code; and executing the repaired executable code in a sandbox execution environment. In this implementation, when the model's internal knowledge is insufficient, the external library version changes, or the calling parameters are mismatched, relevant knowledge can be obtained through external retrieval and the code can be corrected. This significantly improves the success rate and robustness of complex data analysis tasks and enables each task node to have adaptive adjustment and continuous optimization capabilities.

[0019] In one optional implementation, the analysis results include intermediate products, charts, and text content. The analysis report assembly module is further configured to obtain a final analysis report through the following operations: based on the explanatory information, performing retrieval, filtering, and chapter-level structured organization on the analysis results, wherein in the event of organization failure, the following operations are repeated until organization succeeds: initiating searches to multiple knowledge sources to obtain repair context information; reorganizing the analysis results based on the repair context information; and generating the final analysis report based on the organization. In this implementation, the final analysis report generated through the above-described retrieval-enhanced content organization method not only summarizes the overall process results but also maintains the traceability of content sources and the logical consistency of chapter organization, making the report more suitable for scientific research analysis, business decision-making, and formal reporting scenarios.

[0020] In a third aspect of the invention, a computer-readable storage medium is disclosed, characterized in that the computer-readable storage medium stores instructions that, when executed, implement the method described in the first aspect.

[0021] Other aspects, features, and embodiments of the invention will become apparent to those skilled in the art after reading the following description of specific exemplary embodiments of the invention in conjunction with the accompanying drawings. Although features of the invention may be discussed below with reference to certain embodiments and drawings, all embodiments of the invention may include one or more of the advantageous features discussed herein. In other words, while one or more embodiments may be discussed having certain advantageous features, one or more of such features may also be used according to the various embodiments of the invention discussed herein. Similarly, although exemplary embodiments may be discussed below as embodiments of devices, apparatuses, or methods, it should be understood that such exemplary embodiments can be implemented in various systems, devices, apparatuses, and methods. Attached Figure Description

[0022] To gain a more detailed understanding of the features described above in this disclosure, reference can be made to a more specific description of the above-briefly summarized aspects, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings illustrate only certain typical aspects of this disclosure and should not be considered as limiting its scope, as other equivalent aspects are permissible in this description.

[0023] Figure 1 This is a schematic block diagram of a system for adaptive data analysis according to an embodiment of the present disclosure.

[0024] Figure 2 This is a flowchart of a method for an adaptive data analysis system according to an embodiment of the present disclosure. Detailed Implementation

[0025] The various embodiments will now be described in more detail with reference to the accompanying drawings, which form part of this invention and illustrate specific exemplary embodiments. However, the embodiments may be implemented in many different forms and should not be construed as limiting the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of these embodiments to those skilled in the art. The embodiments may be implemented as methods, apparatus, or devices. Therefore, these embodiments may be implemented in hardware, entirely in software, or in a combination of software and hardware aspects. Therefore, the following detailed description is not intended to be limiting.

[0026] The steps in each flowchart can be performed by hardware (e.g., processor, engine, memory, circuitry), software (e.g., operating system, application, driver, machine / processor executable instructions), or a combination thereof. As will be understood by those skilled in the art, the methods involved in each embodiment may include more or fewer steps than shown.

[0027] Currently, while data analysis tasks generally follow a relatively stable process paradigm, their actual execution still heavily relies on statistical knowledge, inter-stage dependencies, data characteristics, and user analysis goals. Existing systems either overly depend on fixed templates and lack flexibility, or rely entirely on large-scale models for free planning and lack stability and verifiability. Therefore, it is difficult to balance automation, analytical rigor, execution stability, and user control.

[0028] Based on this, the technical solution disclosed herein aims to construct an intelligent data analysis system that can perform end-to-end automatic analysis, accept structured constraints, external knowledge enhancement, and user feedback correction, and mainly includes the following characteristics: (1) Explicitly embed standard steps, method dependencies and process constraints in statistical analysis into the planning process to improve the completeness, rationality and controllability of the analysis process; (2) By simultaneously analyzing auxiliary documents, historical records and structured data tables, the system can automatically understand the current analysis scenario and improve the accuracy of task identification; (3) By combining system-level planning with module-level execution, we can improve the adaptability of each analysis link to different data characteristics and task objectives while ensuring global consistency. (4) Introduce a search-enhanced generation (RAG) mechanism during the code generation and execution phase, so that the system can automatically retrieve API documentation, solutions and external knowledge when errors occur, and complete code repair; (5) Introduce a retrieval-enhanced content organization mechanism based on intermediate analysis results during the report generation stage to improve the logic, traceability and editability of the final report.

[0029] The various aspects of the present invention will now be described in detail.

[0030] Figure 1 This is a schematic block diagram of a system 100 for adaptive data analysis according to an embodiment of the present disclosure.

[0031] In one embodiment of this disclosure, such as Figure 1 As shown, the system 100 for adaptive data analysis may include an analysis task description module 102, a task flow control module 104, a task code generation module 106, and an analysis report assembly module 108. These modules will be described in detail below.

[0032] In the above embodiments, the task description module 102 can be configured to receive description information of the data analysis task and extract task feature information from the description information.

[0033] In one optional implementation, the explanatory information may include document information and a CSV data table regarding the data analysis task. The document information includes supporting documents, historical records, background information, a table of contents (also known as a preset outline), and task description text (including task objectives, etc.). In another optional implementation, the task description module 102 may also be configured to extract textual semantic features from the document information. As those skilled in the art will understand, the document information is not limited to the above, but may also include any other suitable information from which textual semantic features can be extracted, such as data manuals, business materials, etc.

[0034] In an optional implementation, the task description module 102 can also be configured to extract metadata features from the CSV data table. These metadata features may include, but are not limited to, column names, field types, distribution characteristics, time attributes, category attributes, and table structure information. Furthermore, the task feature information may include both the metadata features and the textual semantic features. In other words, the task description module 102 can be configured to extract task feature information from the description information by extracting textual semantic features from the document information and metadata features from the CSV data table.

[0035] In the embodiments described above, the task description module 102 can be further configured to generate natural language description information for the data analysis task based on the task feature information. In an optional implementation, the task description module 102 can use any suitable inference engine or inference model to output natural language description information (also referred to as natural language profiling) for the current analysis task. This natural language description information can refer to a natural language summary generated by the inference engine that describes industry background, preferred styles, and task intent, and may include, but is not limited to, industry background, analysis objectives, priority indicators, recommended analysis methods, and result expression style. As those skilled in the art will understand, the task description module 102 can use any suitable inference model to generate natural language description information, and is not limited to any specific model.

[0036] Therefore, in the above implementation, by simultaneously receiving document information and CSV data tables, jointly extracting text semantic features and data meta-features, and inputting them into the profiling inference engine to generate a natural language profiling description, the accuracy of task scenario recognition can be improved, and the system can adaptively adjust the subsequent analysis process and result expression method according to different domain backgrounds, data structures and user preferences.

[0037] Continue describing Figure 1In the above embodiments of this disclosure, the task flow control module 104 can be configured to determine multiple subtasks in the workflow corresponding to the data analysis task based on the natural language description information generated by the analysis task description module 102. In one optional implementation, subtasks may include, but are not limited to, data cleaning, exploratory analysis (e.g., preliminary analysis of data, such as counting rows, columns, and the proportion of missing values), modeling and validation (e.g., building predictive models (e.g., linear regression models, machine learning models, etc.) for generating subtasks in the workflow, i.e., training models for predictive tasks (each subtask can be considered a predictive task from the model's perspective) and validating their effects). In other optional implementations, subtasks may also include feature engineering, model diagnostics, result interpretation, anomaly detection, etc., depending on the specific application scenario and not limited to any particular subtask.

[0038] In the above embodiments of this disclosure, the task flow control module 104 can also be configured to encode the standard steps, step dependencies and sequence constraints, and step execution result determination logic corresponding to the multiple subtasks into structured priors to generate the workflow flow control information based on the structured priors. In an optional embodiment, the standard steps can refer to standard operating procedures (SOPs) and may include fixed (inherent) logical steps in the statistical analysis domain, such as cleaning, description, modeling, and diagnosis. In an optional embodiment, the structured prior can also be called Structure as Prior, and can mean encoding domain expert knowledge, such as statistical analysis SOPs, into the skeleton structure of the control flow and using it as constraints and process node requirements for large model reasoning and process scheduling, rather than having the model generate the process from scratch. Thus, the system realizes process scheduling under structured prior constraints, improving the completeness, rationality, and controllability of the analysis steps.

[0039] In one optional implementation, the structured prior can be embodied in a statistical workflow engine and injected into the process control information as rule templates, state transition rules, control prompts, etc., to achieve unified scheduling of each task node (corresponding to each subtask) in the workflow corresponding to the data analysis task. In other optional implementations, the structured prior can also be embodied in a rule base, state machine, process template, knowledge graph, or other structured carrier that can express step dependencies, as long as it can impose constraints on process state control, and is not limited to any specific carrier.

[0040] In the embodiments described above, the task flow control module 104 can also be configured to schedule the execution of multiple subtasks based on the flow control information, including determining an algorithm execution task for each of the multiple subtasks for distribution to the subsequent task code generation module 106. In an optional embodiment, the task flow control module 104 can be further configured to schedule the execution of the multiple subtasks by performing the following operations for each subtask: executing the next subtask if the subtask is executed successfully; or reverting to the execution of the previous subtask if the subtask fails. In this embodiment, the task flow control module 104 can also be further configured to continue executing the subtask until it succeeds if the subtask fails, which will be described in detail below. Therefore, the system can determine subsequent actions based on the feedback status or indicators after the current node is executed, including scheduling the next task node, continuing to execute the current path, or backtracking to the previous node for reprocessing if the effect is not satisfactory. This supports the non-linear loop of the entire analysis task process, meaning that the analysis task process is no longer a unidirectional linear link process, but a dynamic process that allows backtracking, iteration, and optimization based on the execution results of each task node. Furthermore, it can avoid process omissions and logical divergences caused by the current large model's free planning, thereby improving the completeness, controllability, and statistical rationality of the analysis process.

[0041] In the above embodiments of this disclosure, the task code generation module 106 can be configured to generate executable code for each of the plurality of subtasks using LLM based on the algorithm execution task issued by the task flow control module 104. In an optional embodiment, the task code generation module 106 can also be configured to generate executable code in a RAG manner through the following operations: 1) Construct retrieval queries based on the algorithm. As those skilled in the art will understand, any suitable method can be used to construct retrieval queries, and is not limited to any particular method; 2) Based on this search query, searches are initiated from multiple knowledge sources to obtain search results, including API documentation, sample code, and domain knowledge. These multiple knowledge sources include local knowledge sources and online knowledge sources. This enables the generation of code that better meets the task requirements when calling code that is known to the used LLM, such as specific algorithms, library functions, or parameters (e.g., when there is insufficient knowledge within the LLM, external library version changes, or mismatched call parameters). Knowledge sources can include, but are not limited to, documents, web pages, and files, and can also be extended to internal / external enterprise knowledge bases, version control repositories, sample code repositories, or any other suitable external knowledge sources; and 3) Based on the search results, generate the executable code according to natural language description information and flow control information.

[0042] In another alternative implementation, the task code generation module 106 may specifically generate the executable code based on the profile context information such as task objectives in the natural language description information and information such as process node requirements in the process control information. As those skilled in the art will understand, the task code generation module 106 may generate the executable code based on any suitable information and using any suitable model, and is not limited to any specific information and specific model.

[0043] In the embodiments described above, the task code generation module 106 can be configured to execute the generated executable code. In an optional embodiment, the task code generation module 106 can execute the executable code in a sandbox execution environment or any other suitable execution environment. In this embodiment, the task code generation module 106 can also be configured to repeat the following operations until successful execution in the event of execution failure: initiate a search to multiple knowledge sources to obtain repair context information; regenerate and execute the executable code based on the repair context information. In an optional embodiment, the task code generation module 106 is further configured to regenerate and execute the executable code to implement the RAG mechanism through the following operations: 1) In response to executable code execution failure, capture error context information. This error context information may include error type, call stack, variable context, dependency version information, and runtime environment characteristics. This information can be collectively referred to as traceback and environment information. Traceback information may include stack trace information output when the program encounters an error, including error type, location, and call chain, etc. 2) Construct a search query based on the error context information. As will be understood by those skilled in the art, any suitable method can be used to construct the search query, and it is not limited to any particular method; 3) Based on this search query, searches are initiated against these multiple knowledge sources. This also enables the generation of executable code that is more likely to correct the current error when calling code that is known to the LLM being used, such as specific algorithms, library functions, or parameters (e.g., when there is insufficient knowledge within the LLM, external library version changes, or mismatched call parameters causing the LLM to fail / exclude code execution). 4) The retrieval retrieves API documentation and solution information for fixing the executable code, which, along with the error context information, is used as the repair context to regenerate the executable code. In this operation, the code generated by the LLM serves as the base code, and the repair context information obtained through the retrieval is provided to the LLM so that it can modify the base code based on the repair context information to regenerate the repaired executable code; and 5) Execute the repaired executable code in the sandbox execution environment. If execution still fails, repeat steps 1) to 5), repeating the process until successful execution. Alternatively, in another optional implementation, a maximum number of correction rounds can be set. When the number of loops reaches this maximum number of correction rounds, the loop stops, and the executable code is no longer regenerated.

[0044] Thus, as shown above, system 100 implements the SCR mechanism, namely Suggest-Code-Refine: For a data analysis task to be executed, the task flow control module 104 determines multiple sub-tasks (Suggest) in the workflow corresponding to the data analysis task. The task code generation module 106 generates code for each sub-task in a RAG manner. Based on the execution result feedback, the generated code is iteratively corrected (Refine). In the event of execution failure / exception, a closed-loop mechanism of "error / exception capture - enhanced retrieval - repair and rewriting - re-execution" is implemented, thereby improving the code execution success rate and exception recovery capability, and enhancing the system's robustness in scenarios involving complex algorithm calls, use of uncommon libraries, and version changes. In addition, this also enables each task node to have adaptive adjustment and continuous optimization capabilities.

[0045] Continue describing Figure 1 In the above embodiments of this disclosure, the analysis report assembly module 108 can be configured to generate analysis results for the plurality of subtasks in response to successful execution of executable code. In one optional implementation, the analysis results include intermediate products (e.g., a statistical value used for charting, such as an average, as a prerequisite for the chart), charts, and text content. In another optional implementation, the task code generation module 106 may also perform the operation of generating analysis results in response to successful execution of executable code for the subtasks.

[0046] In the embodiments described above, the analysis report assembly module 108 can also be configured to obtain a final analysis report based on the analysis results. In an optional embodiment, the analysis report assembly module 108 can be further configured to obtain the final analysis report by: performing retrieval, filtering, and chapter-level structured organization on the analysis results based on the description information of the data analysis task, and generating the final analysis report based on this organization. Specifically, the analysis report assembly module 108 can perform chapter-level structured organization based on the directory structure in the description information. In an optional embodiment, the analysis report assembly module can be further configured to repeat the following operations until organization succeeds in the event of organization failure: initiating searches to multiple knowledge sources to obtain repair context information; and reorganizing the analysis results based on the repair context information. In this embodiment, similar to the above-described retrieval-enhanced code generation, the analysis report assembly module 108 here implements retrieval-enhanced report organization, rather than simply splicing together the analysis results of various sub-tasks. As those skilled in the art will understand, any suitable method can be used to determine whether the above-described chapter-level structured organization is successful, and it is not limited to any particular method. Therefore, through the aforementioned enhanced content organization, the system can improve the logical coherence, traceability, and editability of the final report, making the final report more suitable for scientific research analysis, business decision-making, and formal reporting scenarios.

[0047] Figure 2 This is a flowchart of a method 200 for adaptive data analysis according to an embodiment of the present disclosure.

[0048] like Figure 2 As shown, method 200 begins at step 202, receiving descriptive information for a data analysis task and extracting task feature information from that descriptive information. In an optional implementation, the descriptive information includes document information about the data analysis task and a comma-separated CSV data table. The document information includes auxiliary documents, historical records, background information, a directory structure, and task description text. The extraction of task feature information includes: extracting textual semantic features from the document information; and extracting metadata features from the CSV data table, wherein the task feature information includes the metadata features and the textual semantic features.

[0049] Next, method 200 continues to step 204, generating natural language description information for the data analysis task based on the task feature information. In an optional implementation, the natural language description information may refer to a natural language summary generated by the inference engine that describes the industry background, preferred styles, and task intent, and may include, but is not limited to: industry background, analysis objectives, priority indicators, recommended analysis methods, and result expression style.

[0050] Subsequently, method 200 continues to step 206, determining multiple subtasks in the workflow corresponding to the data analysis task based on the natural language description information. In one optional implementation, the subtasks may include, but are not limited to, data cleaning, exploratory analysis, modeling and validation. In other optional implementations, the subtasks may also include feature engineering, model diagnostics, result interpretation, anomaly detection, etc.

[0051] Next, method 200 continues to step 208, encoding the standard steps, step dependencies, sequence constraints, and step execution result determination logic corresponding to the multiple subtasks into structured priors to generate the workflow's process control information based on these structured priors. In an optional implementation, the structured priors can be embodied in a statistical workflow engine and injected into the process control information as rule templates, state transition rules, control prompts, etc.

[0052] Then, method 200 continues to step 210, scheduling the execution of the plurality of subtasks based on the flow control information. This step may include, for each of the plurality of subtasks: determining an algorithm execution task for the subtask; generating executable code for the subtask using LLM based on the algorithm execution task; and executing the executable code and, in the event of execution failure, repeating the following operations until execution is successful: initiating a search to multiple knowledge sources to obtain repair context information, and regenerating and executing the executable code based on the repair context information. In an optional implementation, this step may also include, for each of the plurality of subtasks: executing the next subtask if the subtask executes successfully; or, in the event of execution failure, reverting to the execution of the previous subtask. In one alternative implementation, generating the executable code may include constructing a retrieval query based on the algorithm's task execution; initiating searches to multiple knowledge sources based on the retrieval query to obtain search results, including API documentation, sample code, and domain knowledge; and generating the executable code based on the search results, according to the natural language description information and the flow control information, and regenerating and executing the executable code includes: capturing error context information in response to the executable code execution failure; constructing a retrieval query based on the error context information; initiating searches to the multiple knowledge sources based on the retrieval query; obtaining API documentation and solution information for repairing the executable code through the search and using it together with the error context information as the repair context information to regenerate the executable code; and executing the repaired executable code in a sandbox execution environment.

[0053] Subsequently, method 200 continues to step 212, generating analysis results for the multiple subtasks in response to successful execution of the executable code. In an optional implementation, the analysis results include intermediate products, charts, and text content.

[0054] Finally, method 200 continues to step 214, obtaining a final analysis report based on the analysis results. In an optional implementation, this step may include: performing retrieval, filtering, and chapter-level structured organization on the analysis results based on the explanatory information, wherein in the event of organization failure, the following operations are repeated until organization is successful: initiating searches to multiple knowledge sources to obtain repair context information and reorganizing the analysis results based on the repair context information; generating the final analysis report based on the organization.

[0055] In step 214, method 200 ends.

[0056] In summary, the technical solution disclosed herein does not simply generate code and output results directly from a large model. Instead, it completes the entire intelligent data analysis process from task understanding, process orchestration, algorithm execution to report output through a three-layer collaboration of multi-source context fusion, structured prior-driven control, and retrieval-enhanced generation and self-healing execution.

[0057] The embodiments of the present invention have been described above with reference to block diagrams and / or operational descriptions of methods and apparatus according to embodiments of the present invention. The functions / actions indicated in the blocks may appear in a different order than shown in any flowchart. For example, depending on the functions / actions involved, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order.

[0058] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for adaptive data analysis, comprising: Receive description information of the data analysis task and extract task feature information from the description information; Generate natural language description information for the data analysis task based on the task feature information; Based on the natural language description information, determine multiple sub-tasks in the workflow corresponding to the data analysis task; The standard steps, step dependencies, sequence constraints, and step execution result determination logic corresponding to the multiple sub-tasks are encoded into structured priors to generate the workflow control information based on the structured priors. The execution of the plurality of sub-tasks is scheduled based on the process control information, including for each of the plurality of sub-tasks: Determine the algorithm to be used for this subtask; Based on the algorithm, the task is executed using a Large Language Model (LLM) to generate executable code for the subtask. as well as Execute the executable code and repeat the following steps if execution fails until success: Initiate searches to multiple knowledge sources to obtain repair context information; The executable code is regenerated and executed based on the repair context information; In response to the successful execution of the executable code, the analysis results of the multiple subtasks are generated; as well as A final analysis report is obtained based on the analysis results.

2. The method as described in claim 1, characterized in that, The explanatory information includes document information about the data analysis task and a comma-separated CSV data table. The document information includes auxiliary documents, historical records, background information, a directory structure, and task description text. The extracted task feature information includes: Extract textual semantic features from the document information; Extract metadata features from the CSV data table, wherein the task feature information includes the metadata features and the text semantic features.

3. The method as described in claim 1, characterized in that, Scheduling the execution of the plurality of subtasks also includes, for each of the plurality of subtasks: If the current subtask executes successfully, the next subtask will be executed; or If the subtask fails, roll back to the previous subtask.

4. The method as described in claim 1, characterized in that: Generating the executable code includes: The algorithm is used to execute tasks and construct retrieval queries. Based on the search query, a search is initiated against the multiple knowledge sources to obtain search results, including API documentation, sample code, and domain knowledge. The multiple knowledge sources include local knowledge sources and online knowledge sources. Based on the search results, the executable code is generated according to the natural language description information and the flow control information, and Regenerating and executing the executable code includes: In response to the failure of the executable code execution, error context information is captured, including error type, call stack, variable context, dependency version information, and runtime environment characteristics; Construct a search query based on the error context information; Based on the search query, a search is initiated to the multiple knowledge sources; The API documentation and solution information for repairing the executable code are obtained through the retrieval and used together with the error context information as the repair context information to regenerate the executable code; and Execute the patched executable code in a sandbox execution environment.

5. The method as described in claim 1, characterized in that, The analysis results include intermediate products, charts, and text content, wherein the final analysis report obtained based on the analysis results includes: Based on the aforementioned information, the analysis results are retrieved, filtered, and organized into a chapter-level structure. If organization fails, the following steps are repeated until successful: Initiate searches to multiple knowledge sources to obtain repair context information; The analysis results are reorganized based on the repair context information; The final analysis report is generated based on the organization.

6. A system for adaptive data analysis, comprising: The analysis task description module is configured as follows: Receive description information of the data analysis task and extract task feature information from the description information; Generate natural language description information for the data analysis task based on the task feature information; The task flow control module is configured as follows: Based on the natural language description information, determine multiple sub-tasks in the workflow corresponding to the data analysis task; The standard steps, step dependencies, sequence constraints, and step execution result determination logic corresponding to the multiple sub-tasks are encoded into structured priors to generate the workflow control information based on the structured priors. as well as The execution of the plurality of subtasks is scheduled based on the process control information, including determining the algorithm execution task for each of the plurality of subtasks. The task code generation module is configured to, for each of the plurality of subtasks: Based on the algorithm, the task is executed using a Large Language Model (LLM) to generate executable code for the subtask. Execute the executable code and repeat the following steps if execution fails until success: Initiate searches to multiple knowledge sources to obtain repair context information; The executable code is regenerated and executed based on the repair context information; as well as The analysis report assembly module is configured as follows: In response to the successful execution of the executable code, the analysis results of the multiple subtasks are generated; A final analysis report is obtained based on the analysis results.

7. The system as described in claim 6, characterized in that, The explanatory information includes document information about the data analysis task and a comma-separated CSV data table. The document information includes auxiliary documents, historical records, background information, a directory structure, and task description text. The analysis description module is further configured to extract the task feature information through the following operations: Extract textual semantic features from the document information; Extract metadata features from the CSV data table, wherein the task feature information includes the metadata features and the text semantic features.

8. The system as described in claim 6, characterized in that, The task flow control module is further configured to schedule the execution of the plurality of subtasks by performing the following operations for each subtask: If the current subtask executes successfully, the next subtask will be executed; or If the subtask fails, roll back to the previous subtask.

9. The system as described in claim 6, characterized in that: The task code generation module is further configured to generate the executable code through the following operations: The algorithm is used to execute tasks and construct retrieval queries. Based on the search query, a search is initiated to the multiple knowledge sources to obtain search results, including API documentation, sample code, and domain knowledge. The multiple knowledge sources include local knowledge sources and online knowledge sources. as well as Based on the search results, the executable code is generated according to the natural language description information and the flow control information, and The task code generation module is further configured to regenerate and execute the executable code by performing the following operations: In response to the failure of the executable code execution, error context information is captured, including error type, call stack, variable context, dependency version information, and runtime environment characteristics; Construct a search query based on the error context information; Based on the search query, a search is initiated to the multiple knowledge sources; The API documentation and solution information for repairing the executable code are obtained through the retrieval and used together with the error context information as the repair context information to regenerate the executable code; as well as Execute the patched executable code in a sandbox execution environment.

10. The system as described in claim 6, characterized in that, The analysis results include intermediate products, charts, and text content, wherein the analysis report assembly module is further configured to obtain the final analysis report through the following operations: Based on the aforementioned information, the analysis results are retrieved, filtered, and organized into a chapter-level structure. If organization fails, the following steps are repeated until successful: Initiate searches to multiple knowledge sources to obtain repair context information; The analysis results are reorganized based on the repair context information; The final analysis report is generated based on the organization.

Images