A case knowledge extraction and retrieval method for data analysis intelligent agent

Abstract

The application discloses a case knowledge extraction and retrieval method for a data analysis intelligent agent, and comprises the following steps: performing chapter-level preprocessing and analysis logic analysis on field scientific literature, and aligning the full text into an analysis planning domain, an execution operation domain, and a result verification and correction domain; extracting an intention node, an operation node, a cognitive constraint node and a state observation node therefrom, and establishing four types of hyper-edges to construct an inferable procedural knowledge hypergraph, which forms a case knowledge base after being symbolically encapsulated; in the retrieval stage, an analysis task is received and a context vector dynamically fused by a macroscopic intention vector, a microscopic state vector and a confusion degree vector is generated, multi-dimensional matching positioning is performed through intention anchoring, state anchoring and path anchoring, and a decision support package is generated through forward reasoning along the hyper-edges. The application realizes logically coherent extraction and situation-adaptive retrieval of case knowledge, and improves the decision accuracy of the data analysis intelligent agent in complex tasks.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and data processing technology, and in particular to a method for extracting and retrieving case knowledge for data analysis intelligent agents. Background Technology

[0002] Data analytics agents are intelligent systems capable of autonomously completing analytical tasks such as data exploration, statistical modeling, result interpretation, and report generation based on natural language instructions. With the rapid development of large language model technology, these agents are showing broad application prospects in scenarios such as business analysis, economic research, corporate consulting, and scientific research data analysis.

[0003] However, current data analysis agents generally face a lack of domain-specific experience and knowledge when performing complex analytical tasks. While general-purpose large language models perform well on routine reasoning tasks, they lack a deep understanding of domain-specific analytical process design, algorithm applicability judgment, variable relationship constraints, and result validity verification rules. This leads to problems such as inappropriate modeling path selection, misuse of statistical algorithms, and biased conclusions in practical applications, severely limiting their reliability in specialized analytical scenarios.

[0004] Domain-specific scientific literature, research reports, and industry case studies contain a wealth of practical experience accumulated by expert analysts, including complete sequences of analytical steps, indicator design rules, algorithm selection criteria, and anomaly handling strategies. Effectively extracting and reusing this case knowledge in the decision-making process of intelligent agents can significantly enhance their analytical capabilities. However, efficiently and completely extracting this knowledge from long-text scientific literature faces significant technical challenges. Traditional text processing methods based on fixed blocks tend to fragment the analytical logic flow across paragraphs, leading to breaks in contextual information. Conventional information extraction techniques often focus on isolated entity relationships, failing to capture the reasoning links and decision dependencies that run throughout the text, resulting in the omission or fragmentation of key empirical knowledge.

[0005] In case retrieval and matching, existing technologies largely rely on simple textual semantic similarity calculations. However, case matching for data analysis tasks is highly context-dependent: a truly valuable similar case not only needs to be similar in terms of analytical objectives but also needs to deeply align with the current task in multiple dimensions, such as data characteristics, completed operational history, and current decision-making dilemmas. Simple semantic matching struggles to capture this multi-dimensional contextual alignment requirement, often resulting in retrieved cases with limited reference value.

[0006] In summary, there is an urgent need for a technical solution that can systematically and logically extract case knowledge from long-text scientific literature and dynamically and accurately match similar cases based on multi-dimensional contexts during the analysis process. Summary of the Invention

[0007] The purpose of this invention is to provide a method for extracting and retrieving case knowledge for data analysis agents, in order to solve the problems of fragmented and context-severed extraction of long text case knowledge in the prior art, as well as the difficulty of matching static retrieval with dynamic analysis scenarios.

[0008] Technical solution: A method for extracting and retrieving case knowledge for data analysis intelligent agents, comprising the following steps:

[0009] Acquire scientific literature data in the field, perform chapter-level preprocessing, and obtain paragraph sequences with structural tags;

[0010] The paragraph sequence is analyzed and parsed logically. The full text is parsed into the analysis planning domain, the execution operation domain, and the result verification and correction domain through a domain-fine-tuned language model, achieving three-segment alignment and obtaining a set of domain-annotated paragraphs.

[0011] Nodes are extracted from the domain-annotated paragraph set and hyperedge relationships are identified to construct a procedural knowledge hypergraph;

[0012] Based on the procedural knowledge hypergraph, the cognitive constraint nodes are symbolically encapsulated to generate constraint functions that can be automatically verified by machines, and the complete hypergraph of each case is stored in the structured case knowledge base.

[0013] Receive data analysis tasks, parse task requirements, and generate dynamic context vectors that evolve with the analysis process;

[0014] Multidimensional anchoring matching is performed between the dynamic context vector and the procedural knowledge hypergraph in the structured case knowledge base to determine the location of the current task context in each hypergraph;

[0015] Starting from the anchor node, forward reasoning is performed along the hyperedge to extract candidate operation sequences, constraint checklists, and alternative path previews, generating a contextualized decision support package for use by the data analysis agent.

[0016] Beneficial effects: By constructing a reasonable procedural knowledge hypergraph and introducing a multi-dimensional dynamic context matching mechanism, this invention achieves logically coherent extraction of case knowledge and high-precision, context-adaptive retrieval, thereby improving the decision-making accuracy of data analysis agents in complex tasks. Attached Figure Description

[0017] Figure 1 This is a flowchart of the overall solution of the present invention.

[0018] Figure 2 This is a flowchart of the process for constructing a procedural knowledge hypergraph according to the present invention.

[0019] Figure 3 This is a flowchart illustrating how the present invention generates dynamic context vectors that evolve as the analysis process progresses.

[0020] Figure 4 This is a flowchart illustrating the generation of a contextualized decision support package according to the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] like Figures 1-4 As shown, this embodiment describes in detail the data processing flow of a case knowledge extraction and retrieval method for data analysis intelligent agents, specifically including:

[0023] S1: Acquire scientific literature data in the field, perform chapter-level preprocessing, and obtain paragraph sequences with structural tags.

[0024] This embodiment aims to transform scattered, multi-source raw long text data into standardized, structured paragraph sequences that can be processed by subsequent models.

[0025] Specifically, the first step is to collect and aggregate multi-source heterogeneous scientific literature data. The data collected includes, but is not limited to, scientific papers from academic databases, technical reports published by research institutions, analytical case documents from industry consulting firms, and internal review reports from companies. The acquired raw literature data is in various formats, covering unstructured or formatted documents such as PDF, HTML, and Word.

[0026] Next, the acquired original document collection was formatted and its metadata extracted. Using a document parsing engine, files of different formats were uniformly converted into plain text streams. Simultaneously, core metadata such as title, author, publication date, source, and abstract of each document was extracted to establish a document-level index.

[0027] Furthermore, the standardized full-text document undergoes cleaning and paragraph-level segmentation. The text cleaning process filters out noise content irrelevant to the main text analysis logic, including headers and footers, automatically numbered figures, tables, and formulas, citation lists, acknowledgments, and original code blocks in appendices. After cleaning, the entire document is segmented into paragraphs based on line breaks, indentation formats, and chapter title numbering patterns.

[0028] Preferably, based on paragraph segmentation, each paragraph is assigned a structural tag. The generation of structural tags employs a combination of rule matching and heuristic algorithms: for example, common academic literature chapter titles such as "Abstract," "Introduction," "Methods," "Experiments," "Discussion," and "Conclusion" are identified and converted into standard structural tags; for paragraphs without explicit titles, their tags are inferred from their location and the tags of adjacent paragraphs, and then tagged accordingly. Ultimately, this results in a sequence of paragraphs with structural tags, consisting of multiple data pairs, each containing a text paragraph and its corresponding structural tag.

[0029] S2: Perform logical parsing on the paragraph sequence, and use a domain-fine-tuned language model to parse the full text into an analysis planning domain, an execution operation domain, and a result verification and correction domain, achieving three-segment alignment and obtaining a set of domain-annotated paragraphs.

[0030] This embodiment functionally re-divides the entire argumentation process of the literature from the perspective of the analyst's logical thinking.

[0031] In this embodiment, the obtained paragraph sequence with structural labels is first input into a domain-fine-tuned large language model. The large language model is based on an open-source general-purpose large model and uses a corpus consisting of a large number of data analysis case papers for efficient parameter fine-tuning, enabling it to recognize data analysis-specific verbal steps.

[0032] Specifically, the large language model performs full-text semantic understanding of the input text, executing a functional parsing. The parsing task is defined as dividing each paragraph of the entire text into three mutually exclusive and interconnected macro-domains based on its logical role in the analysis and argumentation. The first category of paragraphs, describing the problems faced in the analysis, the proposed research hypotheses, the designed analysis plan, or the selected algorithm path, is identified and marked as the analysis planning domain. The second category of paragraphs, describing specific data processing actions, cleaning operations, statistical modeling implementation processes, or algorithm execution details, is identified and marked as the execution operation domain. The third category of paragraphs, describing the verification and diagnosis of processing results, the discovery of counterexamples or anomalies, and the modification of the original plan or the triggering of a new analysis loop, is identified and marked as the result verification and correction domain.

[0033] Furthermore, after completing the paragraph-level domain labeling, the model also needs to identify and explicitly establish cross-paragraph jump relationships between domains. For example, after a paragraph in the execution operation domain completes a normality test, its test result may lead the process to jump to a paragraph in the result verification and correction domain, which describes the algorithm replacement decision taken due to non-normal data. This jump relationship will be recorded as a directed link. Ultimately, what is obtained is no longer a linear sequence of paragraphs, but a set of domain-labeled paragraphs where all paragraphs are assigned the planning, execution, or verification domain labels and contain logical jump links.

[0034] S3: Extract nodes from the domain-annotated paragraph set and identify hyperedge relationships to construct a procedural knowledge hypergraph.

[0035] This embodiment compiles the logically aligned set of domain-labeled paragraphs obtained in the previous stage into a structured, computable knowledge representation, namely a procedural knowledge hypergraph. The procedural knowledge hypergraph not only stores knowledge content but also encodes the reasoning and decision-making logic during the analysis process.

[0036] First, targeted extraction of four types of core knowledge nodes is performed. Using a sequence labeling model, the following atomic knowledge units are identified and classified from the domain-labeled paragraph set: The first type is the intent node representing a local analytical objective, such as "whether there is a significant difference between the mean values ​​of the treatment group and the control group on the core indicators"; the second type is the operation node representing a specific analysis or treatment method, such as "Shapiro-Wilk normality test" or "propensity score matching"; the third type is the cognitive constraint node representing the preconditions or inherent contract of the algorithm that must be met before performing an operation, such as "sample data are independent and identically distributed" or "analysis of variance requires homogeneity of variance among groups"; the fourth type is the state observation node representing the key intermediate outputs or final conclusions generated in the analysis process, such as a specific p-value, a qualitative conclusion of a model diagnostic plot, or a correlation coefficient matrix.

[0037] Secondly, identify and establish multidimensional hyperedge relationships connecting the aforementioned nodes. Based on the semantic logic within paragraphs and cross-domain jump links, construct four types of hyperedges: First, based on the description of the operation taken to achieve a certain intention, establish a means-end hyperedge pointing from the intention node to the operation node; second, based on the description that an operation can only be executed when specific conditions are met, establish a condition-action hyperedge pointing from the cognitive constraint node to the operation node; third, based on the description that an operation produces a certain state information after execution, establish an action-output hyperedge pointing from the operation node to the state observation node; fourth, and most importantly, based on the description that the observation of a certain state triggers a new intention or changes the subsequent operation path, establish an observation-decision hyperedge pointing from the state observation node to the new intention node or operation node.

[0038] Further improvements to this embodiment involve performing a global consistency check on the hypergraph after initial construction. Using coreference resolution technology, nodes scattered across different chapters of the literature but actually pointing to the same variable, the same analysis method, or the same rule are merged. Semantic alignment ensures that the nodes associated with each hyperedge across paragraphs are semantically logically consistent. Through this fusion, previously fragmented knowledge elements are integrated into a logically coherent case-based cognitive graph that can be forward-deduce along the hyperedges.

[0039] S4: Based on the procedural knowledge hypergraph, the cognitive constraint nodes are symbolically encapsulated to generate constraint functions that can be automatically verified by the machine, and the complete hypergraph of each case is stored in the structured case knowledge base.

[0040] After constructing the skeleton of the procedural knowledge hypergraph, this embodiment performs symbolic transformation from natural language to executable code on the cognitive constraint nodes with the most decision-making value, and completes the persistent storage of knowledge.

[0041] Specifically, it iterates through all cognitive constraint nodes in each case cognitive graph. The text encapsulated within these nodes, such as "Variable X must follow a normal distribution" or "The ratio of sample sizes in each group should not exceed 1:4," is currently still an unstructured natural language description.

[0042] For each cognitive constraint node, rule parsing and structured reconstruction of the condition expression are performed. Using predefined rule templates and a semantic parser, the natural language rule is parsed into one or more atomic conditions connected by logical operators, and the data fields, statistics, and preset thresholds on which the condition judgment depends are identified, ultimately forming a structured condition expression.

[0043] Furthermore, this structured conditional expression is compiled into a constraint verification function that can be automatically invoked by the machine. The function's input interface is explicitly defined as receiving a snapshot of the data environment in the current analysis task. Internally, the function performs corresponding statistical tests or data checks according to the conditional expression, ultimately outputting a Boolean value and additional verification detail logs. For example, a cognitive constraint node describing "homogeneity of variance" will be transformed into a verification function encapsulating a Levene test or Bartlett test. After symbolic encapsulation, the complete case cognitive graph, including the original node, hyperedges, and the newly generated constraint verification function, is stored as a record in the structured case knowledge base, completing the organization and storage of the knowledge extraction results.

[0044] S5: Receives data analysis tasks, parses task requirements, and generates dynamic context vectors that evolve with the analysis process.

[0045] This embodiment aims to construct a dynamic representation that can visualize the current cognitive state and data environment of an intelligent agent in real time.

[0046] In one alternative implementation, the user submits a natural language description analysis task through an interactive interface and uploads a structured dataset to be analyzed.

[0047] First, multi-dimensional feature parsing is performed on the received analysis task description. A multi-task learning model is used to simultaneously extract features from three dimensions: first, analysis requirement features, which characterize the macro-level objectives of the task, such as "causal inference," "predictive modeling," or "anomaly attribution"; second, data features, automatically extracted directly from the uploaded dataset, including field names, data types of each field, distribution of missing values, basic statistics, and other pattern information; and third, variable relationship features, where if the user explicitly specifies dependent and independent variables, this variable role binding relationship is parsed and recorded.

[0048] Next, based on the aforementioned features and the agent's operational state, a dynamic context vector is generated and continuously updated. At the start of the task, the analyzed requirement features are encoded into a high-dimensional macro-intention vector using an embedding model fine-tuned for task type classification. Simultaneously, the extracted data features, along with the encoded sequence of operations completed by the agent at this moment and real-time statistical summary information from the current data table, are jointly encoded using a state encoder to form a micro-state vector.

[0049] In a preferred embodiment, this vector is not static. Each time the agent completes an operation, such as performing missing value imputation, an incremental update of the microstate vector is triggered. Furthermore, the probability distribution of the policy network output when the agent decides on its next action is continuously monitored. When the perplexity of this distribution exceeds a preset threshold, indicating that the agent lacks confidence in its next action and is in a state of cognitive uncertainty, the system automatically triggers a knowledge enhancement request and encodes the semantic representation of the current candidate action space into a perplexity vector. Finally, the macro-intention vector, microstate vector, and perplexity vector obtained in the above manner are dynamically weighted, concatenated, and fused to output an instantaneous dynamic context vector that fully expresses "what the agent wants to do, the environment, and the difficulties encountered."

[0050] S6: Perform multi-dimensional anchoring matching between the dynamic context vector and the procedural knowledge hypergraph in the structured case knowledge base to determine the location of the current task context in each hypergraph.

[0051] This embodiment uses the generated dynamic context vector as a query probe to perform a context localization operation in the structured case knowledge base that differs from traditional similarity ranking.

[0052] Specifically, a multi-dimensional anchoring matching process with three parallel channels is performed. The first channel is intent anchoring: the macro intent vector in the dynamic context vector is extracted separately and its vector similarity is calculated with the intent nodes in all case cognitive graphs in the knowledge base, recalling a set of candidate graph regions that are highly similar to the macro analysis target.

[0053] The second channel is state anchoring: the micro-state vector portion of the dynamic context vector is parsed into a series of symbolic state features. For example, the parts of the vector related to field type and distribution pattern are parsed into semantic labels such as "there is a continuous dependent variable". Using these symbolic labels, precise matching is performed with state observation nodes in the graph to recall case locations that observe similar data states or produce similar intermediate results.

[0054] The third channel is path anchoring: the historical sequence of multiple operation steps completed by the agent up to the current moment is matched with the sub-paths of operation nodes in the hypergraph. Preferably, this matching adopts a hierarchical strategy: first, a global path fuzzy matching is performed on the core operation steps to filter out a range of candidate cases with similar process skeletons; then, fine-grained node-level precise comparison is performed within this range.

[0055] Furthermore, by combining the matching scores from the three channels mentioned above, a weighted comprehensive anchoring score is calculated. The one or more nodes or regions with the highest scores are identified as the precise location points of the current task context in the corresponding case hypergraph. Through this multi-dimensional anchoring, instead of simply finding a "similar case," the knowledge location is found to be "the decision-making moment that is currently in a similar analysis stage, facing similar data states, and having similar operational history."

[0056] S7: Starting from the anchor node, perform forward reasoning along the hyperedge to extract candidate operation sequences, constraint checklists, and alternative path previews, and generate a contextualized decision support package for the data analysis agent to call.

[0057] After completing the context localization, this embodiment takes the anchor node as the starting point, uses the connection structure of the hypergraph to perform navigational knowledge forward reasoning, and finally generates structured guidance information that can be directly acted upon by the intelligent agent.

[0058] First, starting from the anchored intention node or state observation node, a forward walk is performed along its emitted means-end hyperedge to directly extract a set of candidate executable operation nodes used in historical successful cases in the current context, providing a reference for the agent's algorithm selection.

[0059] For each operation in the candidate operation node set, traverse backwards through the condition-action hyperedges pointing to it, extract all the preceding cognitive constraint nodes, and summarize them into a list of rules that must be verified.

[0060] Simultaneously, a forward-looking path simulation is performed. Continuing forward from the candidate operation node, its subsequent action-output hyperedges and observation-decision hyperedges are traversed to construct a possible decision tree. The trunk of this tree represents the execution of candidate operations, while its branches represent various possible state observations after execution, as well as the corresponding subsequent correction paths, thus forming a preview of alternative analysis paths.

[0061] Finally, the decision support package is organized and generated. For example, for the candidate operation node set, the BM25 keyword retrieval mechanism is introduced, combining the algorithm name, indicator name, and other key text associated with the node for precise recall. The pure vector retrieval results are supplemented and reordered to output the final recommended operation sequence. For each item in the rule list, the generated symbolic constraint verification function is called to perform immediate compliance verification on the real-time data in the current microstate. Items that fail verification are automatically associated with the corrective measures recorded in their respective cases, forming an immediate constraint checklist containing verification results and corrective suggestions. Finally, the recommended operation sequence, the immediate constraint checklist, the alternative analysis path preview, and the referenced case metadata are packaged together to generate a contextualized decision support package and output, completing the knowledge enhancement of the data analysis agent.

[0062] According to another aspect of this application, the steps for constructing a procedural knowledge hypergraph include:

[0063] S3.1: From the domain-labeled paragraph set, extract four types of nodes: intent nodes representing local analysis goals, operation nodes representing specific analysis methods, cognitive constraint nodes representing preconditions and algorithm contracts, and state observation nodes representing intermediate or final analysis outputs.

[0064] This step, based on the output set of domain-annotated paragraphs, performs the first stage of knowledge hypergraph construction: the targeted extraction of four types of core knowledge nodes. Unlike traditional named entity recognition or relation extraction, node extraction here aims to capture the basic cognitive units that constitute the analytical reasoning process from a specialized data analysis perspective.

[0065] Specifically, a hybrid model of sequence labeling and text generation, trained on a domain-labeled paragraph set, is used to scan and classify text fragments within each paragraph sentence by sentence. The first type of extracted object is the intent node, whose text source is usually distributed in the analysis planning domain. The model is trained to identify local analysis objectives that appear in the form of interrogative sentences, target statements, or hypothetical statements, such as "Is there a significant difference between the medians of the two groups?", "The objective is to build a customer churn early warning model with a prediction accuracy of no less than 85%", or "Assuming that the launch of a new product has a positive impact on user activity". The extracted intent nodes contain natural language description text and contextual information of the paragraph in which they are located.

[0066] The second type of extraction object is the operation node, whose text source is mainly distributed in the execution operation domain. The model identifies specific, executable analysis methods or data processing actions from them, including the name of statistical test method, data cleaning function, feature transformation operation, machine learning algorithm name and its core parameter configuration, etc.

[0067] The third type of extraction object is cognitive constraint nodes, which is the key feature that distinguishes this scheme from conventional information extraction. Cognitive constraint nodes can be distributed in the analysis and planning domain and the execution operation domain, and their textual form is usually expressed as conditional adverbial clauses, hypothetical statements, or restrictive statements. The model is specifically trained to capture two types of constraint information: first, preconditions, that is, the mathematical or logical state that the data must satisfy before performing an operation; second, algorithmic contracts, that is, the inherent scope of application and assumptions of a method. For example, "assuming the sample comes from a normal distribution" or "this method is sensitive to outliers and needs to be pre-processed" will both be extracted as cognitive constraint nodes.

[0068] The fourth type of extracted objects are state observation nodes, whose text sources are mainly distributed in the result verification and correction domain, but may also appear in the execution operation domain. The model identifies text fragments representing key outputs in the analysis process, including specific statistical values, qualitative descriptions of test conclusions, values ​​of model performance indicators, and core judgments of the visualization diagnostic chart. Each extracted node is assigned a globally unique identifier and retains its original text, paragraph, and domain label metadata.

[0069] S3.2: Based on the semantic logic within the paragraph, identify and establish the means-end hyperedge between the intent node and the operation node.

[0070] After node extraction is completed, this step begins to establish the first type of logical hyperedge connecting the nodes, namely the means-end hyperedge, which is used to encode the correspondence of "what analytical means were used to achieve a certain analytical goal".

[0071] In one specific implementation, the hyperedge recognition model uses a constructed cross-paragraph jump link as its skeleton to align relationships between nodes at a fine-grained level. The model traverses each intent node, searching for operation nodes bound to the intent in its paragraph and its logical downstream paragraphs. The semantic matching strategy considers both explicit connectives and implicit semantic associations: for explicit connectives, the model identifies guiding language patterns such as "Therefore, we adopt...", "To test this hypothesis, we execute...", and "...is used to solve the above problem"; for implicit semantic associations, it uses attention interaction between the semantic vectors of intent nodes and the semantic vectors of candidate operation nodes to calculate their logical coupling degree. When a highly coupled intent-operation pair is matched, a means-end hyperedge is established from the intent node to the operation node. Since an analytical intent may correspond to multiple optional operation paths, the hypergraph structure naturally supports divergent hyperedges connecting multiple operation nodes starting from an intent node, thus preserving the diversity of method choices in the case. Each established hyperedge records its associated source node, target node, relationship type, and confidence level.

[0072] S3.3: Based on the conditional judgment statement, identify and establish the condition-action hyperedge between the cognitive constraint node and the operation node.

[0073] This step establishes a second type of logical hyperedge, namely the condition-action hyperedge, to encode the constraint knowledge that "an operation can only be executed if certain preconditions are met or its algorithmic contract is satisfied."

[0074] Specifically, the hyperedge recognition model performs context backtracking on each operation node, checking whether there are cognitive constraint nodes in its preceding text that form a conditional relationship with it. At the linguistic level, the model captures typical conditional markers, such as phrases like "provided that... is satisfied," "the prerequisite is...", "requires...", "this method assumes...", etc. When such a conditional relationship is identified between a cognitive constraint node and an operation node, a condition-action hyperedge is established pointing from the cognitive constraint node to the operation node. According to a further improvement of this embodiment, the constraint knowledge contained in the cognitive constraint node is explicitly linked to the specific operation of its constraint. After symbolically encapsulating the cognitive constraint node, this hyperedge will serve as a signal path to trigger automatic verification. For example, a cognitive constraint node describing "analysis of variance requires homogeneity of variance among groups" is connected to both the "one-way ANOVA" operation node and the "two-way ANOVA" operation node through a condition-action hyperedge, meaning that regardless of which operation is included as a candidate, the constraint must be verified.

[0075] S3.4: Based on the output description of the analysis process, identify and establish the action-output hyperedge between the operation node and the state observation node.

[0076] This step establishes a third type of logical hyperedge, namely the action-output hyperedge, to encode "what intermediate results or final conclusions are produced after performing a certain operation".

[0077] In a preferred embodiment, the model searches for state observation nodes downstream of each operation node that have an output relationship, following the established logical flow direction. The output relationship is represented in the text as: result report sentences immediately following the operation, statements about the model's output values, descriptions of qualitative judgments in diagnostic charts, etc. The model determines whether a state observation node is a direct output of the corresponding operation node by analyzing the subject-verb-object structure and discourse progression logic of the sentence. An action-output hyperedge is established, pointing from the operation node to the state observation node. This hyperedge plays a crucial role in subsequent retrieval and forward inference: when the agent's current task context matches a state observation node through state anchoring, it can trace back along this hyperedge to the upstream operation that generated that state; and during forward inference, it can also predict the possible observed states downstream after executing candidate operations.

[0078] S3.5: Based on the state feedback and decision correction description, identify and establish the observation-decision hyperedge between the state observation node and the new intention node or operation node.

[0079] This step establishes the fourth type, which is also the most valuable for dynamic decision-making: the observation-decision hyperedge. This hyperedge is used to encode the dynamic decision adjustments made by expert analysts after observing an intermediate state.

[0080] Specifically, the model focuses on the result verification and correction domains, as well as paragraphs that have logical transitions to the verification domain. Pattern recognition is divided into two categories: The first category involves observations triggering new analytical intentions. For example, if a paragraph describes "the test results of residual autocorrelation indicate that the model specification is biased, and further investigation of the impact of omitted variables is needed," then an observation-decision hyperedge is established from the observation node of "significant residual autocorrelation" to the new intention node of "investigating the impact of omitted variables." The second category involves observations directly triggering operational-level corrections. For example, if "due to the severe right-skewed distribution of the data, the mean-based t-test is abandoned, and the Mann-Whitney U test is directly adopted," then an observation-decision hyperedge is established from the observation node of "severely right-skewed data" to the operational node of "Mann-Whitney U test." Establishing these hyperedges ensures that the knowledge hypergraph not only records successful analytical paths but, more importantly, records the branches and correction paths encountered when problems arise, providing a direct structural foundation for generating previews of alternative paths.

[0081] According to another aspect of this application, the step of generating a dynamic context vector that evolves with the analysis process includes:

[0082] S5.1: Receive the user's input description of the analysis task and the uploaded dataset.

[0083] This step serves as the entry point for human-computer interaction, completing the collection and standardization of raw information for the analysis task.

[0084] In a typical application scenario, users input an analysis task description in natural language text through the interactive interface of the data analysis agent. This description can be a brief statement of the analysis intent or a detailed description of the analysis requirements, including background, objectives, and expectations. Simultaneously, users submit a structured dataset to be analyzed via a file upload interface. The dataset format supports common tabular data files, including but not limited to CSV, Excel, and database query result export tables. Upon receiving the dataset, basic validation and metadata extraction are performed: verifying the dataset's file integrity, encoding compliance, and basic format validity; extracting the dataset's field name list, data type inferences for each field, row and column size statistics, and a preliminary overview of missing value distribution. After successful validation, a metadata snapshot of the dataset is generated and temporarily stored in the task session context. The analysis task description text and the dataset metadata together constitute the raw input for subsequent requirement parsing.

[0085] S5.2: Perform semantic parsing on the analysis task description to extract analysis requirement features, data features, and variable relationship features.

[0086] After obtaining user input, this step performs multi-dimensional feature analysis on the analysis task description and dataset to provide initial feature materials for the construction of dynamic context vectors.

[0087] Specifically, a multi-task learning architecture consisting of a shared encoder and three task-specific heads is employed for feature extraction. The encoder jointly encodes the analysis task description text and metadata such as field names in the dataset into a context-aware hidden representation. The first task head is an analysis requirement classifier, which maps the hidden representation to a predefined analysis requirement type space. This type system covers common analysis intent categories such as exploratory data analysis, difference testing, association analysis, causal inference, predictive modeling, clustering, and anomaly attribution. The output analysis requirement feature is a probability distribution vector in this type space, while extracting one or more labels with the highest probability in the type space as macro-intent labels. The second task head is a data feature extractor, which combines pattern information such as field type distribution, missing proportion, and summary statistics of numerical fields obtained directly from the dataset to output a structured data feature description object. The third task head is a variable relationship parser, which identifies the variable roles explicitly specified in the user description, such as "using sales as the dependent variable" or "comparing by region," parses and outputs variable relationship features, and records the binding relationships between dependent variables, independent variables, grouping variables, weight variables, and data fields.

[0088] S5.3: Encode the analytical demand characteristics into a macro-intent vector.

[0089] After obtaining the multidimensional feature parsing results, this step further encodes the analyzed demand features into a high-dimensional dense macro-intent vector, which is used for semantic alignment with intent nodes in the case knowledge hypergraph during the retrieval stage.

[0090] According to one aspect of this application, this encoding is performed using an embedding model fine-tuned through comparative learning of analytical intent. This embedding model is based on a pre-trained sentence-level semantic embedding model and is fine-tuned using a dataset consisting of intent description texts and their corresponding intent category labels from a large number of data analysis cases. The goal of fine-tuning is to narrow the distance between samples belonging to the same intent category in the vector space, while widening the distance between samples belonging to different categories. The encoding process involves concatenating the output probability distribution vector of the analytical requirement with the user's original analytical task description text, inputting this concatenated vector into the fine-tuned embedding model, and having its Transformer encoder generate a dense vector of fixed dimensions, which is the macroscopic intent vector. This vector lies on a manifold in the semantic space related to the data analysis intent and can be effectively compared with the cosine similarity of intent node embedding vectors from case literature.

[0091] S5.4: Jointly encode the sequence of operations completed by the agent, the statistical summary of the current data table, and the data structure to form a microstate vector. This microstate vector is updated in real time with each action of the agent.

[0092] This step constructs the micro-state portion of the dynamic context vector that evolves continuously with the task progress, in order to characterize the precise data environment and historical behavioral trajectory of the agent.

[0093] In a preferred embodiment, the microstate vector is encoded by a state encoder neural network. The encoder's input includes three information channels: the first channel is the operation sequence channel, which receives the embedded representation sequence of all operations performed by the agent since the start of the task. This sequence is encoded and compressed into a fixed-length history vector using a lightweight Transformer or bidirectional long short-term memory network. The second channel is the statistical summary channel, which receives the summative statistics of each numerical field in the current data table, such as mean, standard deviation, quartiles, skewness, and kurtosis, as well as the category frequency distribution of categorical fields. This is encoded using a fully connected network. The third channel is the data structure channel, which receives pattern information such as the field names, current data types, and missing value ratios of the current table. This is encoded using another fully connected network. The output vectors of the three channels are concatenated and then fused into a unified microstate vector through a fully connected layer. At the initial moment of task startup, the operation sequence channel is empty and replaced by a zero vector, while the statistical summary and data structure channels are encoded based on the user-uploaded original dataset. As the agent performs data analysis operations step by step, it automatically captures new data table snapshots and new operation records after each operation is completed, triggering incremental updates in the three channels mentioned above. The micro-state vector evolves in real time, precisely synchronizing with the agent's analysis progress.

[0094] S5.5: Monitor the decision probability distribution of the agent when deciding on the next action. When the perplexity exceeds the preset threshold, trigger knowledge enhancement and encode the semantic information of the current candidate action space into a perplexity vector.

[0095] This step introduces an adaptive knowledge enhancement triggering mechanism based on the agent's cognitive uncertainty, ensuring that the dynamic context vector can express "where the agent encountered difficulties".

[0096] Specifically, the agent typically maintains a policy network or decision module to sample the next action from the action space based on the current state. During each round of decision-making, the policy network outputs a probability distribution for each action in the candidate action space. The system continuously monitors this probability distribution and calculates its perplexity index. Perplexity is defined here as a weighted combination of the exponent of the number of candidate actions and the entropy of the probability distribution, used to measure the policy network's certainty regarding the current decision. When the perplexity is below a preset threshold, it indicates that the agent is confident in the next action and can execute the decision normally; when the perplexity exceeds the preset threshold, it indicates that the agent lacks sufficient experience and knowledge to make a reliable decision in the current situation, at which point the system automatically triggers a knowledge enhancement process. After triggering, all candidate actions in the current candidate action space are transformed into embedding vectors one by one through an action semantic encoder. Then, the embedding vectors of all candidate actions are pooled and aggregated to generate a perplexity vector. This vector encodes the nature of the decision dilemma faced by the agent and the spatial distribution of candidate directions, providing crucial cognitive gap localization signals for subsequent retrieval.

[0097] S5.6: Dynamically weight and merge the macro-intention vector, micro-state vector, and confusion vector to obtain the dynamic context vector at the current moment.

[0098] After obtaining the three sub-vectors, this step performs the final dynamic context vector fusion to generate a high-dimensional probe that uniformly represents the current cognitive state of the task.

[0099] In this embodiment, the fusion of dynamic context vectors does not employ fixed concatenation weights, but instead introduces a lightweight gated attention fusion module. This module takes as input the concatenation of a macro-intent vector, a micro-state vector, and a perplexity vector, and generates three scalar attention weights through a two-layer fully connected network, each corresponding to one of the three sub-vectors. The weight generation mechanism allows the fusion process to adaptively adjust the contribution of each part according to the current specific context: in the initial stage of the task, the macro-intent vector typically holds the dominant weight because no historical operations have yet occurred, and the micro-state is not yet rich; as the analysis deepens, the rich data environment information carried by the micro-state vector gradually increases its weight; when the agent triggers knowledge enhancement and the perplexity vector appears, it is given a higher instantaneous weight to emphasize the prominent position of the current decision dilemma in the context representation. Finally, the three sub-vectors are multiplied by their corresponding attention weights and then concatenated to form the dynamic context vector at the current moment, which serves as the unified query vector for performing multi-dimensional anchor matching.

[0100] According to another aspect of this application, the steps for generating a contextualized decision support package include:

[0101] For the candidate operation node set, combined with the BM25 keyword retrieval mechanism, the algorithm name and indicator name associated with the node are precisely recalled and weighted by relevance to obtain the recommended operation sequence.

[0102] This step introduces a sparse keyword retrieval mechanism for hybrid ranking, improving the recall accuracy for specific analysis methods and indicator names.

[0103] Specifically, BM25 keyword retrieval uses the algorithm name, method name, core indicator name, and technical terms associated with each operation node in the candidate operation node set as index fields, and key terms from the user's original analysis task description and terms with significant features in the current data table as query terms. It calculates the exact match score between each candidate operation node and the current task at the lexical level. The BM25 score is then weighted and fused with the structured score based on graph forward walk and the matching score based on multi-dimensional anchoring to obtain the final ranking score for each candidate operation. The recommended operation sequence is output in descending order of the final score. This hybrid retrieval mechanism effectively compensates for the potential fuzzy matching problem of technical terms in pure vector semantic retrieval, ensuring that the recommended operation terms are accurate and directly correspond to specific algorithm library calls or data processing functions.

[0104] For each cognitive constraint node in the rule list, the symbolic constraint function is invoked to perform real-time verification on the data in the current micro-state, generating an real-time constraint checklist containing verification results and correction suggestions.

[0105] This step transforms the static rule list into a dynamic, real-time verification result for the current data, enabling automatic compliance verification of "rules-data".

[0106] In this embodiment, the system traverses each cognitive constraint node in the rule list, locates and calls its corresponding symbolic constraint verification function through a function registry. During the call, a real-time data table snapshot and calculated intermediate statistics from the current task session context are passed as input parameters to the verification function. The verification function executes preset statistical tests or data verification logic, returning a result object containing Boolean verification conclusions and detailed verification logs. The system collects the verification results of all constraint nodes and constructs an instant constraint checklist. Each item in the checklist clearly labels the constraint description, verification result, verification details, and correction suggestions for failed constraints. The correction suggestions originate from alternative operation schemes pointed to by the observation-decision hyperedge associated with the constraint node in the original case graph, or from countermeasures explicitly recorded in literature. Through this instant verification, the agent can obtain a complete compliance check report before executing recommended operations, effectively avoiding erroneous modeling and conclusion bias caused by ignoring statistical assumptions.

[0107] The recommended operation sequence, immediate constraint checklist, alternative analysis path preview, and case sources are summarized, packaged into a contextualized decision support package, and output.

[0108] After completing all reasoning and verification steps, this step organizes and packages the final output, generating a complete structured knowledge enhancement package that can be directly called by the data analysis agent.

[0109] Specifically, the contextualized decision support package is encapsulated using a standardized data exchange format, and its internal logical structure is divided into four modules. The first module is the recommended operation sequence, containing a list of operation schemes arranged by priority. Each operation scheme includes a method name, a reference implementation path, a confidence score, and suggestions for key parameters. The second module is an immediate constraint checklist, containing all generated constraint verification results. Items that fail verification are highlighted, and corresponding correction or alternative suggestions are provided. The third module is a preview of alternative analysis paths, presenting the possible subsequent development paths and triggering conditions for each recommended operation in the form of a structured decision tree. The fourth module is a metadata and sourcing module, recording the case sources cited for each recommended knowledge, including document titles, specific paragraph citations, and the overall analysis step sequence of the case, to ensure the interpretability and traceability of the decision support. After packaging, the contextualized decision support package is output to the decision module of the data analysis agent through the system's internal message bus or API interface, allowing it to directly call upon and reference it in core stages such as algorithm selection, analysis path planning, and result analysis.

[0110] This invention effectively overcomes the core bottlenecks of traditional methods in the extraction and reuse of case knowledge by innovatively constructing a procedural knowledge hypergraph and introducing a multi-dimensional dynamic context vector retrieval mechanism. At the knowledge extraction level, it abandons the drawbacks of traditional block-based methods that easily lead to contextual fragmentation. It adopts a three-stage alignment strategy based on the "analysis and planning domain - execution operation domain - result verification and correction domain" to systematically extract atomic nodes such as intent, operation, cognitive constraints, and state observation from long-text scientific literature. Furthermore, it reconstructs their logical dependencies through four types of hyperedges: means-end, condition-action, action-output, and observation-decision. This forms a coherent knowledge structure that can fully represent the expert's analytical reasoning link and decision causality, improving the completeness and logical consistency of case knowledge construction. At the retrieval and matching level, it breaks through the limitations of single static semantic matching and constructs a contextual vector model that dynamically fuses macroscopic intent vectors, microscopic state vectors, and perplexity vectors. This model can depict the cognitive state and decision-making dilemmas of the agent in the analysis process in real time. Through multi-dimensional collaborative matching of intent anchoring, state anchoring, and path anchoring, it accurately projects the current task context to the corresponding decision node in the case knowledge hypergraph, realizing a paradigm shift from "retrieving similar documents" to "locating similar decision moments." The final generated contextualized decision support package, which includes recommended operation sequences, immediate constraint checklists, and alternative path previews, directly provides the agent with executable, verifiable, and traceable expert-level analysis guidance, effectively reducing the risk of erroneous modeling and algorithm misuse, and improving the execution quality and decision credibility of complex data analysis tasks.

[0111] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. A method for extracting and retrieving case knowledge for data analysis intelligent agents, characterized in that, include: Acquire scientific literature data in the field, perform chapter-level preprocessing, and obtain paragraph sequences with structural tags; The paragraph sequence is analyzed and parsed logically. The full text is parsed into the analysis planning domain, the execution operation domain, and the result verification and correction domain through a domain-fine-tuned language model, achieving three-segment alignment and obtaining a set of domain-annotated paragraphs. Nodes are extracted from the domain-annotated paragraph set and hyperedge relationships are identified to construct a procedural knowledge hypergraph; Based on the procedural knowledge hypergraph, the cognitive constraint nodes are symbolically encapsulated to generate constraint functions that can be automatically verified by machines, and the complete hypergraph of each case is stored in the structured case knowledge base. Receive data analysis tasks, parse task requirements, and generate dynamic context vectors that evolve with the analysis process; Multidimensional anchoring matching is performed between the dynamic context vector and the procedural knowledge hypergraph in the structured case knowledge base to determine the location of the current task context in each hypergraph; Starting from the anchor node, forward reasoning is performed along the hyperedge to extract candidate operation sequences, constraint checklists, and alternative path previews, generating a contextualized decision support package for use by the data analysis agent.

2. The method according to claim 1, characterized in that, The paragraph sequence is analyzed logically, and the full text is parsed into an analysis planning domain, an execution operation domain, and a result verification and correction domain through a domain-fine-tuned language model, achieving three-part alignment, including: Input paragraph sequences into a domain-fine-tuned large language model for full-text semantic understanding; Identify and label the first type of paragraphs describing the analysis objectives and scheme design as the analysis planning domain; Identify and label the second type of paragraphs describing specific data processing and modeling operations as execution operation domains; Identify and label the third type of paragraphs describing result verification, counterexample analysis, and path correction as the result verification and correction domain; Identify and establish cross-paragraph jump relationships between different domains to achieve context alignment of the global logical flow.

3. The method according to claim 1, characterized in that, The steps involved in constructing a procedural knowledge hypergraph include: From the domain-annotated segment set, four types of nodes are extracted: intent nodes representing local analysis goals, operation nodes representing specific analysis methods, cognitive constraint nodes representing preconditions and algorithm contracts, and state observation nodes representing intermediate or final analysis outputs. Based on the semantic logic within the paragraph, identify and establish the means-end hyperedge between the intent node and the operation node; Based on the conditional statements, identify and establish condition-action hyperedges between cognitive constraint nodes and operation nodes; Based on the output description of the analysis process, identify and establish action-output hyperedges between operation nodes and state observation nodes; Based on the description of state feedback and decision correction, the observation-decision hyperedge between the state observation node and the new intention node or operation node is identified and established, forming a navigable knowledge hypergraph structure.

4. The method according to claim 3, characterized in that, The steps involved in constructing a procedural knowledge hypergraph also include: Perform a global consistency check on the initially constructed procedural knowledge hypergraph; By resolving coreference and semantic alignment, nodes scattered across different chapters but pointing to the same analysis steps, variables, or rules are linked and integrated to form a logically coherent case cognitive map.

5. The method according to claim 1, characterized in that, The steps to generate a dynamic context vector that evolves as the analysis progresses include: Receive user input of an analysis task description and uploaded dataset; Semantic parsing is performed on the analysis task description to extract analysis requirement features, data features, and variable relationship features; Encode the analytical demand characteristics into a macro-intention vector; The sequence of operations completed by the agent, the statistical summary of the current data table, and the data structure are jointly encoded to form a microstate vector; the microstate vector is updated in real time with each operation of the agent. The system monitors the decision probability distribution of the agent when deciding on the next action. When the perplexity exceeds a preset threshold, it triggers knowledge enhancement and encodes the semantic information of the current candidate action space into a perplexity vector. The macro-intention vector, micro-state vector, and confusion vector are dynamically weighted and concatenated to obtain the dynamic context vector at the current moment.

6. The method according to claim 1, characterized in that, Multidimensional anchoring matching is performed between dynamic context vectors and procedural knowledge hypergraphs in a structured case knowledge base, including: The macro-intent vector in the dynamic context vector is matched and anchored with the intent nodes in the hypergraph through vector similarity calculation; After parsing the micro-state vector in the dynamic context vector into symbolic state features, it is precisely matched and anchored with the state observation nodes in the hypergraph. The sequence of operations completed by the agent is matched and anchored with the sub-paths of the operation nodes in the hypergraph. Based on the comprehensive score of the three types of anchoring, the precise location point of the current task situation in each case hypergraph is determined.

7. The method according to claim 1, characterized in that, Perform forward inference along the superedge to extract candidate operation sequences, constraint checklists, and alternative path previews, including: Starting from the anchored intent node, perform a forward walk along the means-end hyperedge to extract a set of executable candidate operation nodes; For each operation node in the candidate operation node set, trace back its associated condition-action hyperedges to extract all cognitive constraint nodes and form a list of rules that must be verified. Simulations are performed along the action-output hyperedge and observation-decision hyperedge following the operation node to predict the possible state observation node and its branch path after executing the candidate operation, forming a preview of the alternative analysis path.

8. The method according to claim 7, characterized in that, The steps to generate a contextualized decision support package include: For the candidate operation node set, combined with the BM25 keyword retrieval mechanism, the algorithm name and indicator name associated with the node are precisely recalled and weighted by relevance to obtain the recommended operation sequence. For each cognitive constraint node in the rule list, the symbolic constraint function is invoked to perform real-time verification on the data in the current micro-state, generating an real-time constraint checklist containing verification results and correction suggestions; The recommended operation sequence, immediate constraint checklist, alternative analysis path preview, and case sources are summarized, packaged into a contextualized decision support package, and output.

9. The method according to claim 1, characterized in that, The steps for symbolic encapsulation of cognitive constraint nodes include: The natural language rules contained in the cognitive constraint nodes are parsed into structured conditional expressions; The conditional expression is transformed into a constraint validation function that can be called by the machine; the constraint validation function is used to perform compliance checks on the current state of the input data and return the validation results.

10. The method according to claim 6, characterized in that, Sequence matching anchoring employs a hierarchical matching strategy: Perform global path matching between the completed core operation sequence and the operation node sequence in the hypergraph to determine the range of candidate cases; Within the candidate case range, local fine-tuning is performed by combining intent node anchoring and state observation node anchoring to determine the final location point.

Images