A Large Language Model Table Analysis Method Based on Three-Layer Ontology Structure and Problem-Aware Pruning
By adopting a table analysis method based on a three-layer ontology structure, the problem of inaccurate understanding of industrial-grade table data is solved, achieving highly accurate and reliable analysis results with verifiable overall effects.
Patent Information
- Application Number
- CN202611091217.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-22
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies suffer from complex problems when processing industrial-grade tabular data, such as column name abbreviations, homonyms, differences in definitions, mixed use of units, and missing/outlier values. These issues result in a lack of systematic safeguards for the model in understanding fields, selecting objects, generating code, and performing error correction, leading to unreliable results.
A large language model table analysis method based on a three-layer ontology structure and problem-aware pruning is adopted. By receiving input tables and converting them into standardized table objects, it performs type identification, distribution statistics and anomaly analysis, identifies effective relationships between fields, generates risk warnings, dynamically prunes knowledge subgraphs and executes code in a security sandbox to achieve a self-correcting closed loop.
It improves the accuracy of table field identification and problem understanding, reduces the risk of contextual redundancy and reasoning defocus, enhances the interpretability and traceability of analysis conclusions, and ensures the credibility and executability of results.
Smart Images

Figure CN123115177A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence, natural language processing, and knowledge graph technology, and in particular to a large language model table analysis method based on a three-layer ontology structure and question-aware pruning. Background Technology
[0002] During enterprise digital transformation, business systems accumulate a large amount of structured tabular data (such as Excel, CSV, and reports), which is widely used in scenarios such as business analysis, financial auditing, and risk control monitoring. In recent years, Large Language Models (LLM) have been introduced into tabular question answering and automated data analysis. Common practices include: serializing the table and inputting it into the model to directly answer or generate code; introducing Retrieval Augmentation (RAG) to supplement business documents / data dictionaries; introducing knowledge graphs to inject field relationships using triples; and some solutions have also built a closed loop of "generation-execution-error reporting-regeneration".
[0003] However, the flattened sequence input makes the model overly reliant on the literal meaning of column names, lacking validation based on the actual data distribution. When faced with abbreviations, ambiguities, and differences in definitions, it is prone to column selection errors, unit misreading, and enumeration misjudgments. Furthermore, it lacks a systematic profile of field types, value ranges, and missing / abnormal patterns, making the analytical premise unreliable and increasing the risk of illusions from the source. RAG or knowledge graph injection often employs a "matching-as-concatenation" approach. Due to limitations in context length and insufficient relevance control, a large amount of irrelevant information is introduced, leading to redundancy, distraction, and inference bias (answering the wrong question or generalizing the output). Furthermore, static rules struggle to reflect dynamic data changes, resulting in a disconnect between knowledge and reality. The dirty data characteristics of industrial spreadsheets lead to frequent runtime errors in the generated code; even if it runs successfully, biased results may occur due to non-standardized units, missed filters, or inconsistent definitions. Existing error correction methods mostly rely on trial and error with prompt words, lacking structured constraints and verifiable repair paths, often resulting in repeated failures or using try / except to cover up problems, making it difficult to guarantee error correction convergence and reliable results.
[0004] In summary, industrial-grade tables often suffer from complex issues such as column name abbreviations, homonyms, differing definitions, mixed units, and missing / outlier values. Many business rules are implicitly embedded in cross-column relationships and data distribution. Current technologies primarily rely on text-based input, and external knowledge enhancement struggles to accurately address the current problems. This results in a lack of systematic safeguards throughout the entire model lifecycle—from understanding fields to selecting objects, generating code, performing error correction, and validating output. Consequently, both code executability and result reliability are unstable, hindering practical implementation. Summary of the Invention
[0005] To overcome the problems of unverifiable semantic understanding, difficulty in focusing and easy loss of focus in knowledge enhancement, non-convergence of execution loops, and unreliable results in existing structured table data analysis techniques, this invention proposes a large language model table analysis method based on a three-layer ontology structure and problem-aware pruning. This method improves the accuracy of table field identification and problem understanding, enhances the ability to understand table structural relationships, achieves knowledge focus on the current problem, reduces contextual redundancy and the risk of reasoning defocus, and enhances the interpretability, traceability, and engineering application value of the analysis conclusions, with verifiable overall effect support.
[0006] To achieve the above objectives, the present invention adopts the following technical solution, including: This invention proposes a large language model table analysis method based on a three-layer ontology structure and problem-aware pruning, comprising the following steps: S1: Receive input form T and user issues q Convert the original table into a standardized table object. T′ ; S2: Targeting T′ Each field undergoes type identification, distribution statistics, missing data analysis, anomaly analysis, and pattern induction to obtain a set of field profiles. , For the first j Column-by-column image vector; S3: Target T Based on rule discovery, semantic extraction, and fact verification, valid relationships are identified between fields to obtain a set of structural relationships. R ; S4: Profile collection based on fields P and structural relationship set R Generate a set of risk warnings W ; S5: Calculate the columns of the table and user questions q The relevance score is used to prune the set of structural relationships from high to low. R and risk warning set W Get user issues q Corresponding clipping knowledge subgraph and warning set Encapsulate and trim knowledge subgraphs and warning set A prompt was received; S6: Invoke the inference model to generate analysis code based on the prompt. It is executed in a security sandbox; if the code execution fails or there is an inconsistency in the result, an exception summary is extracted. and key log fragments Tail_tConstruction repair prompt Guide the inference model to perform local repairs and execute again; if the stopping condition is met, terminate the closed loop and output the final result or failure description.
[0007] Preferably, the column profile vector in step S2 includes: missing rate, proportion of unique values, confidence score for each column data type, skewness dominance rate, and business conflict rate; the column data types include at least: numeric, date, and identity types; Skewed dominance is used to reflect the proportion of unique values in non-empty records, which is the ratio of the frequency of unique values to the total number of non-empty cells; business conflict rate represents the proportion of cells that exceed the legal value range.
[0008] Preferred candidate types for table column data The confidence score is the weighted sum of the regularity pattern hit rate, parsing success rate, and statistical pattern matching score of the data in that column; Regular expression pattern hit rate is used to evaluate whether column data conforms to the type in literal form. The proportion; Parsing success rate is used to evaluate the proportion of column data that can be legally converted during program processing. Statistical morphological matching score is used to assess whether the macroscopic distribution characteristics of column data conform to the type. The essential attributes; for numerical types, the statistical form matching score is assigned "1" or "0" based on whether the mean of the column data is a valid value; for date types, the statistical form matching score adopts the form ratio of the column data that conforms to the time sequence pattern; for identifier types, the statistical form matching score reuses the proportion of non-empty unique values.
[0009] Preferably, in step S3, candidate topologies are discovered from numerical and identifier relationships using a rule-based algorithm, and semantic triples are extracted from field names, sample values, context descriptions, and statistical profiles using a pre-trained large model; both candidate topologies and semantic triples are guaranteed to be relations. , For header entity fields, For tail entity fields or expressions; It is a relational predicate; Then, the following steps are used to fuse and verify the candidate topologies and semantic triples to obtain a fact graph representing the final data table topology: Step S31: Align and deduplicate candidate topologies and semantic triples by entities and relations to form a candidate relation set. Merge conflicting items in the candidate relation set into a single entity pair to be verified. A single entity pair to be verified contains multiple candidate relations with the same head and tail entities but different relation predicates. ; Step S32: The unified validator calls the type matching operator, statistical evidence operator, business rule verification operator, and topology consistency detection operator respectively to calculate the candidate relations. Type matching score Statistical evidence score Business rule scores and topology consistency score Then, the four factors are weighted and summed to obtain the comprehensive verification score. ; Step S33: Filter out the comprehensive verification score Less than the preset score threshold Candidate Relationship Only retain when Candidate relationships are selected, and only the candidate relationship with the highest comprehensive verification score that is greater than the score threshold is retained in each single entity pair to be verified. Candidate relations are selected; the candidate relations that are retained are combined to form a fact graph, which is represented as a set of structural relations R.
[0010] Preferably, the relationships in a triple include derivation relationships, primary attribute dependency relationships, hierarchical affiliation relationships, and caliber consistency relationships.
[0011] Preferably, the method for detecting derivation relations is as follows: first, list multiple derivation relations and arrange them in order, then traverse each derivation relation until a certain derivation relation is satisfied, or traverse all derivation relations; The detection method for multiplicative derivation relations is as follows: First, construct three candidate column vectors. , The residual ratio sequence of C ={ } and calculate the coefficient of variation If the coefficient of variation satisfies the multiplicative derivation condition, then the multiplicative derivation relation is satisfied. ;
[0012] in, The number of valid samples participating in the test; , and These represent the data from columns A, B, and C for the i-th detection sample, respectively. The conditions for multiplication derivation are: ; The allowable fluctuation threshold, The allowable mean offset threshold; For residual ratio sequence The mean; For residual ratio sequence The ratio of the standard deviation to the mean; The detection method for additive derivation relations is as follows: construct an absolute error sequence { }, , Let $\frac{i}{i}$ be the absolute error of the i-th detected sample; calculate the sequence ${\frac{i}{i}$. }and{ The ratio of the medians of} is used as the normalized median error. ,if Then it is determined that an additive derivation relationship exists. ; The set permission threshold.
[0013] Preferably, in step S5, attribute columns are first filtered by label conflict detection, and then the risk function of the structural relationship set in R is calculated based on the attribute columns. Then, based on the relevance of each column to the user's question, perceptual clipping is performed to obtain a clipped knowledge subgraph. Then extract and crop the knowledge subgraph. Corresponding risk function Issue a risk warning;
[0014] in, For example The risk function, The missing rate, The skewed dominance rate, For business conflict rate; The degree of label conflict, i.e., column Attribute conflict rate with different attribute columns The mean; The degree of structural inconsistency; For normalized weights; Define the number of attribute fields corresponding to data in an attribute column as the unique value of the data under that attribute column, and record data with a unique value greater than 1 as conflicting data; calculate the proportion of conflicting data with the attribute column on the deduplicated data set of the column data as the attribute conflict rate between the column and the attribute column; The tag conflict detection method is as follows: traverse the attribute fields of the preparatory attribute column. If there is another column I that satisfies the condition that the attribute conflict rate between I and the preparatory attribute column is greater than a set threshold, then the preparatory attribute column cannot be used as an attribute column. Structural inconsistency The calculation method is as follows: First, extract the fields containing... A set of topological relationships, then traverse the topological relationships. For each pairwise combination of candidate relations, calculate the relation conflict penalty function; sum the relation conflict penalty functions over all relations to obtain the degree of structural inconsistency. .
[0015] Preferred, the first j List and user issues q Relevance score for:
[0016] in, Fields representing columns With the question Semantic similarity between keywords; Represents a column portrait vector With the question The degree of matching between the required operations; Representation field In the structural relationship diagram, the connection strength with the core field; the core field refers to the connection with the user's current question. q The table field that is most relevant and critical in terms of semantics or the required operation; express The extent to which the relevant risk warnings affect the current issue.
[0017] Preferably, in step S5, the user question q The corresponding prompt is located in the task command area. Key Field Profiling Area Cutting knowledge sub-images Warning set and constraint description area Composed of splicing elements; Task Command Area This includes: the inference model's professional identity in this session, and user questions. The code generation instructions for natural language mapping and the structural specifications that the model output must follow; Key Fields Profile Area Includes cropping knowledge subgraph Names, deterministic types, and column profile vectors of all columns. ; Constraint Description Area It describes the available variables and execution within the security sandbox, and is composed of the names of the DataFrame variables already loaded in the security sandbox, the necessary data variables exposed by the sandbox, and the prohibited behaviors of the security functions and strong directives.
[0018] The present invention proposes a system for executing the large language model table analysis method based on a three-layer ontology structure and problem-aware pruning, comprising: an ontology construction model, a reasoning model, a security sandbox, and a self-correcting closed-loop module; Ontology construction model for receiving input forms T and user issues qGenerate a set of field profiles Cutting knowledge sub-images and warning set And call the Prompt assembly template to encapsulate and trim the knowledge subgraph. and warning set A prompt was received; The ontology construction model includes: The statistical anchoring layer is used to perform type identification, distribution statistics, missing data analysis, anomaly analysis, and pattern induction on each field of a standardized table, outputting a set of field profiles. P ; The structural inference layer is used to identify valid relationships between fields based on rule discovery, semantic extraction, and fact verification, and outputs a set of structural relationships R. The semantic cognition layer is used to generate a set of risk warnings based on the set of field profiles and the set of structural relationships; and adopts a problem-aware pruning mechanism to calculate the relevance scores of fields, relationships and warnings based on user problems, output pruned knowledge subgraphs and assemble prompts. The inference model is used to generate code, perform interpretation, and fix errors in response to the prompt. The security sandbox execution module is used to execute the code generated by the inference model and return the execution results or exception information. The self-correcting closed-loop module is used to extract exception summaries and key log fragments when code execution fails or results are abnormal, construct repair prompts to guide the model to perform local repairs and execute again until the stopping conditions are met.
[0019] The advantages of this invention are: (1) This invention proposes a three-layer ontology structure for enhancing table cognition, specifically designed for large language model table analysis tasks. This structure transforms the original two-dimensional table from a "textual input object" into a "structured, reasonable object." The three-layer ontology structure includes a statistical anchoring layer, a structural inference layer, and a semantic cognition layer. The statistical anchoring layer expresses the objective data characteristics of the fields, the structural inference layer expresses the effective relationships between the fields, and the semantic cognition layer expresses risk warnings and constraint information relevant to the current analysis. Through this three-layer ontology structure, subsequent problem understanding, field selection, code generation, result analysis, and natural language interpretation no longer rely solely on the large language model's free guessing based on column names and context, but are instead built upon a structured cognitive foundation.
[0020] (2) This invention proposes a structured table understanding mechanism based on statistical anchoring and with fact verification as its core. The mechanism first performs type identification, missing value analysis, unique value analysis, distribution statistics, skewness analysis and business conflict detection on each column of the input table to form a field profile; then, candidate relationships are generated based on rule discovery and language extraction, and then only valid structural relationships consistent with the actual data of the current table are retained through fact consistency verification.
[0021] (3) This invention dynamically trims and assembles ontology knowledge based on the current user's question. In the prior art, the common practice is to directly concatenate field descriptions, rule descriptions, knowledge graph relationships, or external search results into the prompt words. However, due to the lack of question relevance control, this can easily lead to contextual redundancy, distraction, and loss of focus in reasoning. This invention, instead of injecting all knowledge indiscriminately, dynamically selects the most relevant parts from field profiles, structural relationships, and risk warnings based on the current user's question, constructs a local ontology subgraph, and then assembles it into prompt information according to a preset partitioning method for use by the reasoning model, thus avoiding the problems of contextual redundancy, distraction, and loss of focus in reasoning.
[0022] (4) This invention realizes a closed-loop self-correction mechanism based on secure execution, refined error extraction, and termination control. After the model generates analysis code, the code is first executed in a restricted security sandbox. If the execution fails, the complete and lengthy log is not returned as is. Instead, the abnormal information is refined and only key error clues and necessary context are retained. Then, the original problem, local ontology knowledge, and existing code are combined to construct repair prompts, guide the model to perform targeted repair, and execute again. Termination control is achieved through the maximum number of repair rounds, repeated detection of abnormal signatures, or execution success conditions.
[0023] (5) In summary, the present invention has established systematic safeguards in key aspects such as table field understanding, structural relationship identification, knowledge focus, code execution, self-correction and repair, and result interpretation. It can effectively solve the problems of inaccurate understanding, knowledge loss, unstable execution and unreliable results in the prior art, and has significant technological progress and practical application value. Attached Figure Description
[0024] Figure 1 This is a flowchart of the overall system architecture. Figure 2 A schematic diagram for generating a three-layer body; Figure 3 This is a timing diagram for the closed-loop execution and self-correction of inference code. Detailed Implementation
[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0026] like Figure 1 As shown, this invention employs a dual-path collaborative architecture, including an ontology construction model and a reasoning model. The former is used to generate field profiles, candidate relationships, and risk warnings, while the latter is used for code generation, execution interpretation, and error correction. The method of this invention includes at least the following steps: Step S1: Receive input form T and user issues q The original table is parsed, fields are extracted, missing data markers are standardized, and the format is normalized to obtain a standardized table object. T′ ; Step S2: Statistical analysis of anchoring layer pairs T′ Each field undergoes type identification, distribution statistics, missing data analysis, anomaly analysis, and pattern induction to obtain a set of field profiles. , For the first j Column-by-column image vector; Step S3: The structural inference layer identifies valid relationships between fields based on rule discovery, semantic extraction, and fact verification, and obtains a set of structural relationships R; Step S4: The semantic cognition layer profiles the data based on the field collection. P The set of structural relationships R generates a set of risk warnings. W ; Step S5: Based on the problem-aware pruning mechanism, according to the user's problem q Calculate the relevance scores of fields, relationships, and warnings to obtain a pruned knowledge subgraph. and based on Assembly Analysis Precautions (Prompt) q ); Step S6: Call the inference model to generate analysis code It is executed within a restricted security sandbox; if code execution fails or inconsistencies occur in the results, an exception summary is extracted. and key log fragments Tail_t Construction repair prompt The model is guided to perform local repairs and then executed again; if the stopping condition is met, the closed loop is terminated and the final result or failure description is output.
[0027] Specifically, the overall technical concept of this invention can be summarized as follows: using objective data distribution to constrain semantics, using structural topology to constrain reasoning, and using execution feedback to constrain repair. Wherein: The statistical anchoring layer answers what this column actually is (its output is used to constrain the structure inference layer), whether it is reliable, and whether there are extreme values or mixed types. The structural inference layer answers whether there are stable topological relationships such as derivation, constraints, and primary key-attribute binding between columns (its output is used to generate semantic risk warnings). The semantic cognitive layer answers which risks should be prioritized for analysis given the current data reality. The problem-aware pruning mechanism answers which knowledge must be retained and which knowledge should be suppressed for the current problem (field profiling, structural relationships, and risk warnings all participate in the problem relevance scoring). The inference model generates code that answers how the system achieves targeted repair and convergence based on truncated anomaly cues. Therefore, this invention essentially establishes a complete method chain for transforming static tables into executable cognitive objects.
[0028] The statistical anchoring layer, structural inference layer, and semantic cognition layer proposed in this invention constitute a three-layer ontology from bottom to top. For example... Figure 2 As shown, in the three-layer ontology structure, the statistical signals acquired by the bottom statistical anchoring layer not only serve as the output of this layer but also provide constraints to the upper layers; structural relationships are formed in the middle structural inference layer, supporting the generation of semantic warnings; and finally, the semantic cognition layer outputs analysis notes. The workflow of the statistical anchoring layer includes field statistical feature extraction, type and anomaly identification, and distribution and skewness analysis. The workflow of the structural inference layer includes: rule relationship discovery, LLM relationship extraction, fact verification filtering, and effective structural graph generation.
[0029] The statistical anchoring layer is used to perform column-by-column scanning, type identification, distribution statistics, missing data analysis, anomaly detection, and pattern induction on the table. It extracts the objective data facts of each column in the original table into field profiles that can be read, constrained, and referenced, so that the subsequent inference model's understanding of field semantics is based on verifiable data, rather than guessing based solely on field names or language experience.
[0030] Specifically, let the input table be a two-dimensional data matrix:
[0031] in, Indicates the total number of rows in the record. Indicates the total number of columns in the field. Indicates the first Line 1 The values retrieved from the column cells. Indicates the first number of the table Column data, , and These represent the values in the cells of the first row and the jth column, the second row and the jth column, and the Nth row and the jth column, respectively.
[0032] The statistical anchoring layer represents the field profiles of a table as column profile vectors. The column profile vector of column j... for:
[0033] in, Indicates the missing rate; Indicates the proportion of unique values; , and These represent the confidence scores for the column to be of numeric, date, and identity type, respectively. Indicates the skewed dominance rate; Indicates the business conflict rate; represents a vector; j represents the column index.
[0034] The missing rate is the percentage of empty cells in a column, used to describe the completeness of the column data. It is defined as:
[0035] in, This represents a set of null values, empty strings, unparseable values, or default missing markers.
[0036] When the missing rate of a certain field is too high, it means that subsequent filtering, aggregation or calculation based on that column may produce errors. Therefore, the column with a high missing rate should be subject to constraints that require caution or completion / filtering first.
[0037] The unique value ratio is used to distinguish between primary key / serial number columns, enumeration category columns, and continuous measurement columns, and is defined as follows:
[0038] in, Indicates the first A non-empty unique set. A larger value indicates lower repetition of data in the column. This represents the set of all non-empty data in column j after removing duplicates (also denoted as the set of non-empty unique values), that is, All elements are unique and have appeared in column j of the table; |for The number of elements in the data.
[0039] To enhance the robustness of field type determination, this invention does not employ a single rule for determination, but instead constructs a type scoring function based on multi-source evidence fusion to calculate the confidence score. , and The system will ultimately select the type with the highest confidence score as the primary type for that column.
[0040] Column types include: 1. Numerical data, such as columns like "Unit Price", "Actual Payment Amount", "Inventory Quantity", and "Account Balance" in a commodity trading system.
[0041] 2. Date-based, such as columns like "User Registration Time", "Order Creation Time", "Logistics Receipt Node", and "Historical Modification Stamp" in log systems or business transaction records.
[0042] 3. Identifier type, such as columns like 'Globally Unique User ID (UID)', 'Merchant ID', 'Logistics Tracking Number', and 'Device Hardware MAC Address' in distributed systems or relational databases.
[0043] For any candidate type Its confidence score is defined as:
[0044] in, The weights are empirical and satisfy: ; The regular expression pattern hit rate is used to evaluate whether the data in the j-th column conforms to the type in terms of literal form. The proportion; To measure the success rate of parsing, this is used to evaluate the proportion of data in column j that can be legally converted during program processing; Furthermore, the closer it is to 1, the more it conforms to the time sequence characteristics; The statistical morphological matching score is used to assess whether the macroscopic distribution characteristics of the data in column j conform to the type. The essential attributes of; Indicates type A pre-defined set of regular expression rules. For numeric types. , Includes regular expressions for recognizing various numeric formats, such as: pure integers, floating-point decimals, thousands separators, etc.; also includes features for date types. , Includes regular expressions for recognizing common industry time formats, such as: year-month-day (hour:minute:second), year / month / day, etc.; specifically for date types. , Includes regular expressions for identifying highly unique, fixed-encoding, or hash-based features, such as standard UUIDs, fixed-length alphanumeric and numeric fixed-length primary keys.
[0045] Representing binary numbers: .
[0046] in, This indicates an explicit type conversion logic event, triggered when the text in a non-empty cell... Successfully converted to type This event is valid if the data is in the corresponding standard format and no exception is thrown. ,on the contrary, 0; It calculates the arithmetic mean of a non-empty set. If the result is a significant number, it indicates that the column has a numerical computable form. This indicates that when the candidate type is numeric. If the arithmetic mean of the j-th column is a valid value, then the statistical morphological matching score is... ,on the contrary ; This represents the data in column j, c. j A non-empty data set; When the candidate type is date At that time, the statistical morphological matching score Used to evaluate the monotonicity of time series, specifically calculating the proportion of data that conform to the time series pattern; This represents the sequence of adjacent first-order differences calculated after sorting a non-empty dataset. For statistics The number of samples with a median score greater than 0 (i.e., exhibiting a monotonically increasing trend over time); N is the number of rows in the table. The column missing rate; When the candidate type is identifier Statistical morphological matching score Direct reuse of non-empty unique value proportion .
[0047] Based on confidence score It can be determined that the first j The main type of the column is:
[0048] To identify whether the distribution of column values is strongly dominated by a minority of samples, this invention defines the skewness dominance rate. This reflects the proportion of a single value in non-empty records:
[0049] in, This represents the set of all non-empty data in column j after deduplication. Represents a set The maximum frequency of an element in column j means, specifically, finding the frequency of the most frequent value among all non-empty unique values in that column. N is the number of rows in the table. Let be the missing rate of column j.
[0050] Business conflict rate is defined as:
[0051] in, This indicates the legal value range determined by domain rules, enumeration dictionary, historical statistical intervals, or user configuration.
[0052] In summary, the output of the statistical anchoring layer is not a simple list of column names, but a multi-dimensional objective profile. This profile serves as a constraint for the structural inference layer and also as a direct basis for the semantic cognition layer to generate early warning information.
[0053] The structural inference layer is used to determine the relationships that exist between columns. Let the candidate structural relationships be denoted as triples:
[0054] in, For header entity fields, For tail entity fields or expressions; The relational predicate must include at least one or more of the following: derived_from, depends_on, determines, belongs_to, and aggregates_over. The system first analyzes the numerical and identifying features between fields using a rule-based algorithm to extract candidate relations (i.e., candidate triples constituting the candidate topology). Second, it uses a pre-trained large model for ontology construction to extract semantic triples from field names, sample values, context descriptions, and statistical profiles. Finally, a unified validator fuses and verifies the two types of relations (candidate topology and semantic triples) to obtain a fact graph representing the final data table topology.
[0055] The specific steps of the unified validator in fusing and validating two types of relations (candidate topology and semantic triples) include: Step S31: Align and deduplicate the candidate topologies and semantic triples by entities and relations to form a candidate relation set. Merge conflicting items in the candidate relation set into a single entity pair to be verified. A single entity pair to be verified contains multiple contradictory candidate relations, i.e., those with the same head and tail entities but different relation predicates. .
[0056] In this step, the candidate topology is first split into candidate triples, and then the candidate triples and semantic triples are mapped to the same standard triple representation space. Candidate relationships are formed, and duplicates are removed to obtain a set of candidate relationships.
[0057] Step S32: Calculate the overall verification score for all candidate relations.
[0058] The unified validator invokes the type matching operator, statistical evidence operator, business rule validation operator, and topology consistency detection operator respectively to calculate the candidate relations. Type matching score Statistical evidence score Business rule scores and topology consistency score Then, the four factors are weighted and summed to obtain the comprehensive verification score. .Right now:
[0059] in, For normalized weights, satisfying .
[0060] The formula used to measure whether the field types at both ends of a relation match is as follows:
[0061] in, ; If the field type of the header entity h If it fully conforms to the definition of the relational predicate r, then The output is 1; if the type matches after implicit conversion (such as converting a character numeric value to a number), then... The output is 0.5; if there is a type conflict, then... The output is 0; If the field type of the tail entity field or expression t is... If it fully conforms to the definition of the relational predicate r, then If the type matches after implicit conversion (such as converting a character numeric value to a number), then The output is 0.5; if there is a type conflict, for example, the predicate r is aggregates_over requiring the tail entity to be numeric, but the actual... If it is text type, then The output is 0.
[0062] This indicates whether the rule-based algorithm has found evidence consistent with the triplet. The specific formula is as follows:
[0063] in, Represents tail entity Independent information entropy, , Represents the tail entity The set of non-empty unique values in; Representative value Marginal probability of appearing in the tail entity column; Indicates a known head entity Under the condition of tail entity Conditional entropy; , Represents the head entity The set of non-empty unique values in; Representative at and The joint probability of simultaneous occurrence; Representative at and The conditional probability of occurrence.
[0064] This represents hard validation at the business rule level:
[0065] in, Indicates the first The execution result of the business verification rule has a value of The penalty coefficient between them.
[0066] This indicates the relationship Does adding the current relation graph create a contradiction with the existing structure? The current relation graph refers to the basic relation graph formed by the remaining non-conflicting relations in the candidate relation set after removing the conflicting terms to be verified.
[0067] The specific formula is as follows: :
[0068] The unified validator examines the graph using either a topological sorting algorithm or a depth-first search (DFS). If it finds that introducing a candidate relation would cause a logical infinite loop in the global fact graph, it assigns a validator a validator. This allows them to be completely filtered out as untrustworthy candidate relationships in the overall score.
[0069] Step S33: Filter out the comprehensive verification score Less than the preset score threshold Candidate Relationship Only retain when Candidate relations. If multiple mutually exclusive candidate relations are all greater than... Then only keep The highest scorer completes the conflict resolution, forming the final fact map, i.e., the set of structural relationships R. A preset score threshold is set. The specific possible value is 0.65.
[0070] In a preferred embodiment, the structure inference layer output includes at least the following types of relationships: Firstly, the derivation relationship, that is, whether a column is obtained from other columns through addition, subtraction, multiplication, division or proportional transformation; Secondly, the dependency relationship of the main attribute, that is, whether a certain ID class field is stable determines a certain tag attribute; Third, hierarchical affiliation, such as a city belonging to a province, or a product belonging to a category; Fourth, consistency of standards, for example, tax amount + untaxed amount = tax-inclusive amount.
[0071] In this way, the structural layer obtained by the system is no longer a pure language graph, but a fact graph bound to the current input table.
[0072] In the structural inference layer, derivation relations can be detected using a pre-defined derivation column detection algorithm. The following lists the detection algorithms for multiplicative and additive derivation relations.
[0073] Suppose we select three candidate column vectors from a set of numeric fields. , C, of which:
[0074] in, The number of valid samples participating in the test; , and The first test sample consists of data from columns A, B, and C. , and The second test sample comes from data in columns A, B, and C, respectively. , and The nth test sample comes from columns A, B, and C, respectively.
[0075] The detection of multiplicative derivation relations includes the following steps: SA1. First, construct the sample-level residual ratio:
[0076] in, It is a very small positive number, used to avoid the denominator being zero.
[0077] SA2, Define the coefficient of variation of the residual ratio:
[0078] in, For the residual ratio sequence { The coefficient of variation of} and Representing the residual ratio sequence { The mean and standard deviation of}.
[0079] SA3. Determine if the hard check conditions are met: ; The allowable fluctuation threshold, The allowable mean offset threshold; If the condition is met, then a multiplicative derivation relationship exists. ; If the condition is not met, other derivation relations will be checked according to their priority, such as additive derivation relations.
[0080] The detection of additive derivation relationships includes the following steps: SA1', First, define the absolute error sequence { }:
[0081] in, Let be the absolute error of the i-th detected sample; SA2', Define the normalized median error: for{ }The median of the sequence, for{ }The median of the sequence; To prevent extremely small numbers with a denominator of 0.
[0082] SA3' Determine if the hard check condition is met: ; The set permission threshold; If the condition is met, then an additive derivation relationship can be determined to exist. ; If the condition is not met, other derivation relations are checked according to their priority order. If all derivation relations have been traversed and none are satisfied, it is determined that no derivation relation exists.
[0083] In application examples, derivation relationship detection can be further extended to subtractive, proportional, and linear combination relationships.
[0084] The structure inference layer is also used to perform label collision detection: set up For the candidate ID column, For any attribute column. For each ID value Define it in the attribute column The number of unique values is:
[0085] in, This represents the ID values in the candidate ID column. The hidden information matches a certain attribute of any attribute column G; Used to count the number of duplicates after deduplication. Assume... Represents the ID column (such as a national ID number). It is a specific ID value. Represents attribute columns (such as gender), if A certain ID number If a gender appears at least once, and all genders corresponding to column G are female, then... =1; if A certain ID number It appears at least multiple times, corresponding to column G. The gender can be male or female. =2.
[0086] The improvement of this invention lies in the introduction of "normalized absolute error" based on the median for hard verification.
[0087] If there exists a certain candidate ID in column I... satisfy:
[0088] This indicates that the same ID has multiple different values in the attribute column, meaning there is a tag conflict.
[0089] Furthermore, in order to measure the overall severity of conflicts in a certain attribute column, this invention defines the attribute conflict rate:
[0090] in, Represents the set of unique values for the ID column. for The number of samples in the dataset. If If it is higher, it indicates that the field It is not suitable as a stable attribute label for direct use in statistical analysis.
[0091] In this implementation, the system not only records whether there is a conflict, but also records which attribute columns, which specific IDs, and which corresponding values the conflict occurs in. Figure 2 The semantic warnings generated from the output of the mid-structure inference layer are based on these derivation relationships and label conflict results.
[0092] After the statistical anchoring layer and structural inference layer are completed, the system has mastered two key types of knowledge about the table: one is what each column looks like, represented as a column profile vector. Another type involves understanding the relationships between columns, represented by derivation relationships. However, directly inputting all this information into the reasoning model can easily lead to lengthy contexts, loss of focus, and distraction. Therefore, this invention further designs a semantic cognition layer and a problem-aware pruning mechanism to transform comprehensive tabular knowledge into concise and precise cognitive fragments tailored to the current problem.
[0093] Let R be the set of structural relations, then for columns Constructable risk function :
[0094] in, The missing rate, The skewed dominance rate, For business conflict rate; The degree of label conflict is calculated when column j is the ID column, indicating the different attribute columns. The mean; To indicate the degree of structural inconsistency, for example, when a field is identified as both a derived column and incorrectly used as a primary key dependency source, its... It will rise. For normalized weights.
[0095]
[0096] in, Indicates fields The set of all topological relationships. This is a predefined relation conflict penalty function. When a relation... and If the logic is mutually exclusive, output a higher score (e.g., 1.0); otherwise, output 0.
[0097] The system according to The size of the problem, combined with its semantics, generates a corresponding cognitive warning. Subsequently, the system enters the problem-aware pruning phase. Let the user's problem be denoted as... The system then constructs a problem relevance score for each column. :
[0098] in, Indicate column names and questions Semantic similarity between keywords; Represents a column portrait vector With the question The degree of matching between the required operations; express The connection strength between a field and a core field in the structural relationship graph (i.e., the relationship graph corresponding to the structural relationship set R) is specifically determined by the field in the structural relationship graph. The reciprocal of the sum of the Euclidean distance to user problem q plus 1; It is for columns The correlation penalty score between the generated cognitive warning and question q; For the corresponding preset weight coefficients, satisfying Core fields This refers to the most relevant and critical table field that is semantically or operationally related to the user's current question q.
[0099]
[0100] in, Representation field Column names and column profile vectors and user issues The similarity of intent. This indicates that in the set of structural relations R output by the structural inference layer, the fields... The importance of a field, or centrality for short, can be measured by the normalized sum of its in-degree and out-degree. A field's centrality is determined by the fact that it is depended upon by many other fields (e.g., order ID) or derived from many other fields (e.g., total profit). higher. and To adjust the balance weight between semantic-driven and structure-driven approaches; For a collection of fields.
[0101] express The extent to which the relevant risk warnings affect the current issue.
[0102] The system scores the relevance of the problem. Sort from highest to lowest and select the top rank. The fields; at the same time, from the structural relationship set Extracting from this The topological relationships between the fields are extracted. Each field corresponds to a cognitive warning, thus forming a tailored knowledge subgraph for the current problem. and warning set .
[0103] Both are collectively referred to as Analysis Notes. Then, the Analysis Notes are encapsulated according to a predefined prompt template to form a Prompt:
[0104] in, For the task instruction area, the inference model is explicitly required to output code and explanation, and its formula is:
[0105] in, Define the professional identity of the inference model in this session. User issues The specific code generation instructions for natural language mapping, The model output must adhere to a strict structure. The model is required to output not only executable code, but also structured thinking logs to facilitate white-box auditing and subsequent error backtracking.
[0106] The key field profiling area injects a strongly typed data contract into the model, and its internal structure can be represented as follows:
[0107] in, This indicates the number of items retained after problem relevance trimming. The names of the highly relevant columns. Directly referencing the deterministic type calculated from the underlying layer , The column profile vector generated by the statistical anchoring layer is cited. .
[0108] This section defines the structured relationship area after trimming, providing the model with the logical topology within the data tables to prevent the model from experiencing "relationship illusions" in complex table structures. Its construction is as follows:
[0109] Among them, the comprehensive verification score Exceeding the threshold Fact graph triples; This indicates taking the union of sets.
[0110] The formula for the risk warning zone most relevant to the current problem is:
[0111] in, Indicates the preceding The risk score for each relevant column, for those exceeding the safety threshold. The field, call The function generates mandatory natural language warnings.
[0112] The area for specifying available variables and execution constraints within the sandbox is defined by the following formula:
[0113] in, It explicitly tells the model the names of the DataFrame variables that are currently loaded in the security sandbox. The sandbox is declared to expose only necessary data variables and security functions. Prohibited behaviors are declared with strong directives, including disallowing access to external networks, system commands, and unauthorized file resources. This indicates a splicing operation.
[0114] The system uses the Prompt assembly module to encapsulate user analysis results into a prompt template, forming a prompt Prompt; it can also encapsulate repair prompts into a prompt template, forming a repaired prompt Prompt, which will be referred to as FixPrompt.
[0115] After completing the question-aware clipping, the system sends the completed prompt to the Reasoning LLM, which generates executable code for analyzing the current table.
[0116] like Figure 3 As shown, the Prompt processing procedure includes the following steps: S61. The Prompt assembly module sends a prompt to the Reasoning LLM inference model. The inference model returns code, which is then executed in the security sandbox. S62. Determine if the code executes correctly; If an execution error occurs, the error extraction module returns the error information and extracts key error clues as repair clues, which are then sent to the Prompt assembly module. The Prompt assembly module combines the repair clues to update the repair prompt Prompt, obtains the repaired prompt Prompt, and then returns to step S61. If the execution is successful, request the generation of the final conclusion and output the natural language result.
[0117] In practice, the code can be Python code and executed in a restricted security sandbox. The security sandbox only exposes necessary data variables and security functions, and does not allow access to external networks, system commands, or unauthorized file resources, thereby ensuring that the execution process is secure and controllable.
[0118] Let the first The code for generating the round is Its execution log is The system first executes in the sandbox. If the execution is successful, the standard output and necessary intermediate indicators will be returned, and the final natural language conclusion will be generated; if the execution fails, the error extraction stage will be entered to obtain the corrected prompt.
[0119] To avoid lengthy stack traces interfering with the model's attention, the system performs tail truncation on the exception log, extracting only the most discriminative exception summary. and key segments at the end :
[0120] in, This indicates that key information within the last 300 characters of the log or a preset length range should be retained.
[0121] In the code self-correction loop, if an exception occurs during the execution of the security sandbox, the system extracts the exception summary. and key segments at the end Afterwards, a repair prompt needs to be constructed. (That is, the prompt updated based on the fix clues):
[0122] Among them, the memory retention area The aim is to maintain the inference model's initial understanding of the current task and data base, preventing "context drift" during multiple rounds of repair. Its construction is as follows:
[0123] in, To reiterate what the current problem is, and Force reloading of previously used key field relationships and risk warnings that must continue to be followed.
[0124] Abnormal Site Reporting Area The formula provides the model with precise location of the execution failure in the current round:
[0125] in, Injection The failed code generated in the round clearly indicates where the current code failed. Provides an exception summary (such as KeyError, TypeError) generated by the error extraction module, which is the core information of the error.
[0126] Repair behavior command area To define the operational boundaries for the repair action, the core content is: Patch constraints use natural language instructions to require the model to fix things locally as much as possible (e.g., "Please only modify the line of code that caused the exception, and keep other verified logic") rather than completely rewriting it. This controls the divergence of the large model during the fix process and accelerates the closure convergence. We reiterate the resource access restrictions in the sandbox to ensure that the patched code still meets basic security requirements.
[0127] In a preferred embodiment, the system is configured with the following closed-loop stopping condition:
[0128] in, This represents the maximum number of repair cycles. Indicates the first The signature function for round exceptions is used to determine whether the same type of error occurs repeatedly.
[0129] If the system detects two consecutive identical erroneous signatures, it determines that the model has entered the invalid interval of repeated patching. At this point, it terminates further iterations and returns the current failure reason and suggested manual checkpoints to the user. Therefore, this invention forms a closed-loop path of knowledge constraint generation—secure execution verification—structured anomaly extraction—repair under knowledge constraints—re-execution, improving the code capability of large language models in table analysis scenarios from occasional runnable code to executable inference with convergence guarantees. Figure 3 It can be used as a direct illustration for this section.
[0130] The following describes the implementation process of this invention using a real-world table-based question-and-answer scenario. The input table is the learning duration and exam score data table shown in Table 1, which contains two fields: Hours: learning duration; Scores: exam scores. The user's question is: "Is the relationship between learning duration and exam scores linear? After removing a small number of extreme points, how sensitive is the estimated slope to extreme points?"
[0131] Table 1. Data on Study Duration and Exam Scores
[0132] In this embodiment, the system first parses and standardizes the input table to obtain a standardized table object T′. Then, it enters the statistical anchoring layer, constructing field profiles for the Hours and Scores columns respectively: For the Hours column, the system identifies its primary type as numeric, with a missing rate close to 0 and a high proportion of unique values, making it suitable for direct participation in regression analysis and outlier sensitivity analysis; for the Scores column, the system identifies its primary type as numeric, with a missing rate close to 0, making it suitable as a regression dependent variable. Subsequently, in the structural inference layer, the system identifies candidate relationships of Scores depends_on Hours, and, considering the current problem, determines whether the relationship between the two columns is linear and whether the regression slope is sensitive to a small number of extreme points, in addition to simply requiring the calculation of the correlation coefficient. In the semantic cognition layer, the system generates the following risk warnings: First, sample size warning: the current sample size is small, and extreme points may have a significant impact on slope estimation; Second, method warning: the correlation coefficient alone is insufficient to answer the questions of linearity and slope sensitivity; Third, consistency warning: the final textual conclusions must be based on the code execution output, and numerical conclusions that have not been verified by execution must not be given in advance; Fourth, extreme point definition warning: the method of extreme point identification must be clearly explained.
[0133] During the problem-aware pruning phase, the system retains the two key fields Hours and Scores, and the two task requirements of linear relationship judgment and slope sensitivity analysis, while suppressing the injection of irrelevant knowledge. Then, Prompt(q) is assembled, requiring the inference model to at least complete: (1) Fit a full sample linear regression and output the slope; (2) Specify which criteria will be used to identify extreme points; (3) After removing extreme points, refit and output the new slope; (4) Compare the difference in slope and the magnitude of change before and after; (5) Generate natural language conclusions based solely on the execution results.
[0134] Subsequently, the inference model generates analysis code, which is then executed in a secure sandbox. In this embodiment, a typical error is prone to occur in ordinary large language models: a seemingly reasonable set of slope and sensitivity conclusions is given in the text, followed by a piece of code; however, after the code is actually executed, the resulting values may be inconsistent with the textual conclusions. Specifically: the preceding analysis claimed a correlation coefficient of approximately 0.98, a slope of approximately 9.78, a post-sampling slope of approximately 10.32, and moderate sensitivity, while the actual code execution results showed a correlation coefficient of 1.00, an original slope of 6.84, a post-sampling slope still of 6.84, and a difference of 0.00. This kind of contradiction between textual inference conclusions and code execution results is precisely the problem that the closed-loop consistency verification of this invention aims to solve.
[0135] Therefore, under the present invention, the system does not directly accept the aforementioned natural language guessing results, but further performs consistency checks. If the correlation coefficient, slope value, or sensitive / insensitive judgment in the text is found to be inconsistent with the actual execution result, the system extracts an anomaly summary, constructs a repair prompt, and requires the model to: (1) Remove intermediate speculations that are not supported by the execution; (2) Rewrite the answer based solely on the actual execution results; (3) Clearly explain the criteria for determining extreme points; (4) The linearity judgment and the slope sensitivity judgment are stated separately.
[0136] In this way, the final output of the system is no longer a guess followed by calculation, but a calculation followed by statement, thus ensuring that the result is consistent with the execution.
[0137] To further verify the effectiveness of the proposed large language model table analysis method based on a three-layer ontology structure and problem-aware pruning in practical tasks, this embodiment conducts comparative verification on a publicly available benchmark task (including Table 1). In one embodiment, the verification uses the Data Science Cognitive Benchmark as the test benchmark, and selects Qwen3-8B and Qwen3-32B from the publicly available benchmarks as base models, using their publicly available scores as the control baseline. Based on this, two progressive enhancement schemes are designed around the core technical route of this invention: the first is a three-layer ontology enhancement scheme, which introduces a statistical anchoring layer, a structural inference layer, and a semantic cognition layer into the original table analysis chain to verify the effect of hierarchical cognitive modeling on field understanding, structural relationship identification, and risk constraints; the second is a three-layer ontology + problem-aware pruning scheme, which, based on the three-layer ontology, further dynamically prunes and assembles field profiles, structural relationships, and risk warnings related to the current problem to verify the effect of the knowledge focusing mechanism oriented towards the current task on the overall analysis performance.
[0138] In the evaluation data, the evaluation metrics used in the benchmark are L0, L1, L2, L3, L4, and the comprehensive metric Total. L0 measures problem construction and context identification capabilities; L1 measures behavior selection and decision adaptation capabilities; L2 measures data perception and semantic alignment capabilities; L3 measures cognitive reasoning and verification / reflection capabilities; L4 measures result effectiveness and delivery usability; and Total represents the comprehensive performance of the above capabilities. The comparison results are shown in Table 2.
[0139] Table 2. Performance Comparison of LLM Table Analysis Based on Different Cognitive Enhancement Structures
[0140] Compared with existing methods that directly serialize two-dimensional tables and input them into large language models, or rely solely on static field descriptions, general retrieval enhancements, and error feedback-based trial-and-error repair to assist analysis, the large language model table analysis method proposed in this invention, based on a three-layer ontology structure and problem-aware pruning, improves the accuracy of table field identification and problem understanding, enhances the ability to understand table structural relationships, achieves knowledge focus on the current problem, reduces contextual redundancy and the risk of reasoning defocus, improves the executability of code generation and the credibility of analysis results, and enhances the interpretability, traceability, and engineering application value of analysis conclusions, with verifiable overall effect support.
[0141] Of course, those skilled in the art will recognize that the present invention is not limited to the details of the exemplary embodiments described above, but also includes the same or similar structures that can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.
[0142] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
[0143] The technologies, shapes, and structures not described in detail in this invention are all known technologies.
Claims
1. A large language model table analysis method based on a three-layer ontology structure and problem-aware pruning, characterized in that, Includes the following steps: S1 : Receive input table T and user question q convert the original table into a standardized table object T′ ; S2: Targeting T′ Each field undergoes type identification, distribution statistics, missing data analysis, anomaly analysis, and pattern induction to obtain a set of field profiles. , For the first j Column-by-column image vector; S3: Target T Based on rule discovery, semantic extraction, and fact verification, valid relationships are identified between fields to obtain a set of structural relationships. R ; S4: Profile collection based on fields P and structural relationship set R Generate a set of risk warnings W ; S5: Calculate the columns of the table and user questions q The relevance score is used to prune the set of structural relationships from high to low. R and risk warning set W Get user issues q Corresponding clipping knowledge subgraph and warning set Encapsulate and trim knowledge subgraphs and warning set A prompt was received; S6: Invoke the inference model to generate analysis code based on the prompt. And execute it within a security sandbox; If code execution fails or there is an inconsistency in the result, extract the exception summary. and key log fragments Tail_t Construction repair prompt Guide the inference model to perform local repairs and execute again; If the stopping condition is met, the closed loop is terminated and the final result or failure description is output.
2. The large language model table analysis method based on a three-layer ontology structure and problem-aware pruning as described in claim 1, characterized in that, The column profile vector in step S2 includes: missing rate, proportion of unique values, confidence score for each column data type, skewness of dominance, and business conflict rate; the column data types include at least: numeric, date, and identity. Skewed dominance is used to reflect the proportion of unique values in non-empty records, which is the ratio of the frequency of unique values to the total number of non-empty cells; business conflict rate represents the proportion of cells that exceed the legal value range.
3. The large language model table analysis method based on a three-layer ontology structure and problem-aware pruning as described in claim 2, characterized in that, Candidate types of table column data The confidence score is the weighted sum of the regularity pattern hit rate, parsing success rate, and statistical pattern matching score of the data in that column; Regular expression pattern hit rate is used to evaluate whether column data conforms to the type in literal form. The proportion; Parsing success rate is used to evaluate the proportion of column data that can be legally converted during program processing. Statistical morphological matching score is used to assess whether the macroscopic distribution characteristics of column data conform to the type. The essential attributes; for numerical types, the statistical form matching score is assigned "1" or "0" based on whether the mean of the column data is a valid value; for date types, the statistical form matching score adopts the form ratio of the column data that conforms to the time sequence pattern; for identifier types, the statistical form matching score reuses the proportion of non-empty unique values.
4. The large language model table analysis method based on a three-layer ontology structure and problem-aware pruning as described in claim 1, characterized in that, In step S3, candidate topologies are discovered from numerical and identifier relationships using rule-based algorithms, and semantic triples are extracted from field names, sample values, context descriptions, and statistical profiles using a pre-trained large model. Both the candidate topology and semantic triples are guaranteed to be relations. , For header entity fields, For tail entity fields or expressions; It is a relational predicate; Then, the following steps are used to fuse and verify the candidate topologies and semantic triples to obtain a fact graph representing the final data table topology: Step S31: Align and deduplicate candidate topologies and semantic triples by entities and relations to form a candidate relation set. Merge conflicting items in the candidate relation set into a single entity pair to be verified. A single entity pair to be verified contains multiple candidate relations with the same head and tail entities but different relation predicates. ; Step S32: The unified validator calls the type matching operator, statistical evidence operator, business rule verification operator, and topology consistency detection operator respectively to calculate the candidate relations. Type matching score Statistical evidence score Business rule scores and topology consistency score Then, the four factors are weighted and summed to obtain the comprehensive verification score. ; Step S33: Filter out the comprehensive verification score Less than the preset score threshold Candidate Relationship Only retain when Candidate relationships are selected, and only the candidate relationship with the highest comprehensive verification score that is greater than the score threshold is retained in each single entity pair to be verified. Candidate relations are selected; the candidate relations that are retained are combined to form a fact graph, which is represented as a set of structural relations R.
5. The large language model table analysis method based on a three-layer ontology structure and problem-aware pruning as described in claim 4, characterized in that, The relationships in a triple include derivation relationships, primary attribute dependency relationships, hierarchical attribution relationships, and caliber consistency relationships.
6. The large language model table analysis method based on a three-layer ontology structure and problem-aware pruning as described in claim 5, characterized in that, The method for detecting derivation relations is as follows: First, list multiple derivation relations and arrange them in order, then traverse each derivation relation until a certain derivation relation is satisfied, or traverse all derivation relations. The detection method for multiplicative derivation relations is as follows: First, construct three candidate column vectors. , The residual ratio sequence of C ={ } and calculate the coefficient of variation If the coefficient of variation satisfies the multiplicative derivation condition, then the multiplicative derivation relation is satisfied. ; in, The number of valid samples participating in the test; , and These represent the data from columns A, B, and C for the i-th detection sample, respectively. The conditions for multiplication derivation are: ; The allowable fluctuation threshold, The allowable mean offset threshold; For residual ratio sequence The mean; For residual ratio sequence The ratio of the standard deviation to the mean; The detection method for additive derivation relations is as follows: construct an absolute error sequence { }, , Let $\frac{i}{i}$ be the absolute error of the i-th detected sample; calculate the sequence ${\frac{i}{i}$. }and{ The ratio of the medians of} is used as the normalized median error. ,if Then it is determined that an additive derivation relationship exists. ; The set permission threshold.
7. The large language model table analysis method based on a three-layer ontology structure and problem-aware pruning as described in claim 1, characterized in that, In step S5, attribute columns are first filtered by label conflict detection, and then the risk function of the structural relationship set in R is calculated based on the attribute columns. ; Then, based on the relevance of each column to the user's question, perceptual clipping is performed to obtain a clipped knowledge subgraph. Then extract and crop the knowledge subgraph. Corresponding risk function Issue a risk warning; in, For example The risk function, The missing rate, The skewed dominance rate, For business conflict rate; The degree of label conflict, i.e., column Attribute conflict rate with different attribute columns The mean; The degree of structural inconsistency; For normalized weights; Define the number of attribute fields corresponding to data in an attribute column as the unique value of the data under that attribute column, and record data with a unique value greater than 1 as conflicting data; calculate the proportion of conflicting data with the attribute column on the deduplicated data set of the column data as the attribute conflict rate between the column and the attribute column; The tag conflict detection method is as follows: traverse the attribute fields of the preparatory attribute column. If there is another column I that satisfies the condition that the attribute conflict rate between I and the preparatory attribute column is greater than a set threshold, then the preparatory attribute column cannot be used as an attribute column. Structural inconsistency The calculation method is as follows: First, extract the fields containing... A set of topological relationships, then traverse the topological relationships. For each pairwise combination of candidate relations, calculate the relation conflict penalty function; sum the relation conflict penalty functions over all relations to obtain the degree of structural inconsistency. .
8. The large language model table analysis method based on a three-layer ontology structure and problem-aware pruning as described in claim 7, characterized in that, No. j List and user issues q Relevance score for: in, Fields representing columns With the question Semantic similarity between keywords; Represents a column portrait vector With the question The degree of matching between the required operations; Representation field In the structural relationship diagram, the connection strength with the core field; the core field refers to the connection with the user's current question. q The table field that is most relevant and critical in terms of semantics or the required operation; express The extent to which the relevant risk warnings affect the current issue.
9. The large language model table analysis method based on a three-layer ontology structure and problem-aware pruning as described in claim 7, characterized in that, In step S5, the user question q The corresponding prompt is located in the task command area. Key Field Profiling Area Cutting knowledge sub-images Warning set and constraint description area Composed of splicing elements; Task Command Area This includes: the inference model's professional identity in this session, and user questions. The code generation instructions for natural language mapping and the structural specifications that the model output must follow; Key Fields Profile Area Includes cropping knowledge subgraph Names, deterministic types, and column profile vectors of all columns. ; Constraint Description Area It describes the available variables and execution within the security sandbox, and is composed of the names of the DataFrame variables already loaded in the security sandbox, the necessary data variables exposed by the sandbox, and the prohibited behaviors of the security functions and strong directives.
10. A system for performing the large language model table analysis method based on a three-layer ontology structure and problem-aware pruning as described in any one of claims 1-9, characterized in that, include: Ontology construction model, inference model, security sandbox, and self-correcting closed-loop module; Ontology construction model for receiving input forms T and user issues q Generate a set of field profiles Cutting knowledge sub-images and warning set And call the Prompt assembly template to encapsulate and trim the knowledge subgraph. and warning set A prompt was received; The ontology construction model includes: The statistical anchoring layer is used to perform type identification, distribution statistics, missing data analysis, anomaly analysis, and pattern induction on each field of a standardized table, outputting a set of field profiles. P ; The structural inference layer is used to identify valid relationships between fields based on rule discovery, semantic extraction, and fact verification, and outputs a set of structural relationships R. The semantic cognition layer is used to generate a set of risk warnings based on the set of field profiles and the set of structural relationships; and adopts a problem-aware pruning mechanism to calculate the relevance scores of fields, relationships and warnings based on user problems, output pruned knowledge subgraphs and assemble prompts. The inference model is used to generate code, perform interpretation, and fix errors in response to the prompt. The security sandbox execution module is used to execute the code generated by the inference model and return the execution results or exception information. The self-correcting closed-loop module is used to extract exception summaries and key log fragments when code execution fails or results are abnormal, construct repair prompts to guide the model to perform local repairs and execute again until the stopping conditions are met.
