An ontology-enhanced method, terminal, and medium for automated structured data analysis.
By employing an ontology-enhanced structured data automatic analysis method, we have solved the challenges of semantic understanding, multi-table association, and graph construction in data analysis during enterprise digital transformation. This method enables efficient and reliable data analysis and intelligent question answering, eliminates the illusion of large models, and improves the automation level and reliability of data analysis results.
Patent Information
- Application Number
- CN202611106296.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-24
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies in the process of enterprise digital transformation suffer from problems such as low degree of automation in semantic understanding, difficulty in discovering multi-table relationships, risk of illusion in large model analysis, and lack of dynamic graph construction capabilities. In particular, when faced with non-standard naming, meaningless table headers, or ambiguous fields, they cannot efficiently and automatically perform data analysis and knowledge graph construction.
We employ an ontology-enhanced structured data automatic analysis method, which uses a hybrid semantic mapping mechanism for column semantic annotation, combines multi-strategy evaluation to establish cross-table relationships, dynamically constructs a knowledge graph, and eliminates the illusion of a large model through adaptive analysis strategies and entity reverse verification mechanisms, thereby achieving highly reliable data analysis without human intervention.
It enables automatic association and graph construction of heterogeneous data without human intervention, accurately identifies the true semantics of non-standard fields, automatically mines implicit association keys, reduces the cost of full-scale inference, and ensures the reliability and authenticity of data analysis results.
Smart Images

Figure CN123115307A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer data processing and knowledge engineering technology, specifically to an ontology-enhanced method, terminal, and medium for automatic analysis of structured data. Background Technology
[0002] During enterprise digital transformation, a large amount of high-value data exists in the form of independent "data silos" such as CSV and Excel files. Existing data analysis technologies mainly face the following four bottlenecks: 1. Low level of automation in semantic understanding: Traditional methods rely on manually defined rules or regular expression matching to identify the meaning of columns (e.g., "col_01" represents "user ID"). Faced with non-standard naming, meaningless headers, or ambiguous fields (e.g., "Price" may refer to cost price, selling price, or discount price), traditional ETL tools cannot automatically understand them, and directly using a general large model for full data inference is too costly and limited by the context window.
[0003] 2. Difficulty in Discovering Multi-Table Joins: In loosely structured groups of documents lacking foreign key constraints, automatically discovering join keys between tables is extremely difficult. Existing technologies mostly rely on exact matching of column names (e.g., all must be named "user_id"). When column names differ ("uid" vs. "id") or data has different formats, joins cannot be automatically established, making cross-table joint analysis impossible.
[0004] 3. The Illusion Risk of Large-Scale Model Analysis: Generative AI (AIGC) often exhibits a phenomenon of "serious nonsense" (i.e., illusion) when handling data analysis tasks, fabricating values or entities that do not exist in the data. Due to the lack of rigorous verification mechanisms for the underlying data, its output results cannot be used for serious business decisions.
[0005] 4. Lack of dynamic graph construction capabilities: Traditional knowledge graph construction requires experts to predefine the ontology (Schema). When faced with multi-source data with unknown structure, there is a lack of a mechanism that can dynamically generate the ontology based on the data content, automatically align with industry standards (such as Schema.org), and respond to queries in real time. Summary of the Invention
[0006] To address the technical problems existing in the prior art, this invention provides an automatic analysis method, terminal, and medium for structured data based on ontology enhancement. It can automatically perform high-precision semantic annotation and cross-table association on multi-source heterogeneous structured data to dynamically construct a knowledge graph without the need for manual rule definition. Furthermore, it can adaptively analyze data according to query intent and ultimately eliminate the illusion of a large model through an entity reverse verification mechanism, thereby achieving highly reliable and fully automated data analysis and intelligent question answering.
[0007] To achieve the above objectives, the present invention provides the following technical solution: This invention discloses an automatic analysis method for structured data based on ontology enhancement, comprising steps S1 to S6.
[0008] S1. Obtain multi-source structured data and extract the feature information of each data table in the structured data; S2. Based on feature information, a hybrid semantic mapping mechanism is used to semantically annotate the columns of the data table, obtain the candidate ontology classes and the comprehensive scores of the candidate ontology classes for the corresponding columns, and determine the final semantic label when the comprehensive score meets the preset conditions. S3. By combining multiple correlation evaluation strategies, calculate the total correlation score of cross-table field pairs, and establish cross-table relationships between data tables based on the total correlation score; S4. Identify the primary keys of each data table and generate globally unique identifiers for data rows. Combine semantic tags and cross-table relationships to construct and store a dynamic knowledge graph. S5. Receive the user's natural language query, assess the fuzziness of the query and match an adaptive analysis strategy, and generate a preliminary answer containing key information by combining a dynamic knowledge graph; S6. Extract key information from the initial answer, initiate a reverse query verification in the dynamic knowledge graph, and output the final answer based on the verification results.
[0009] As a further improvement to the above scheme, in step S2, the specific process of semantically annotating the columns of the data table based on the feature information using a hybrid semantic mapping mechanism includes: Extract metadata features and content sample features from each column of the data table, and concatenate them to generate a text description; The text description is encoded into a vector using a pre-trained semantic vector model, and an approximate nearest neighbor search is performed in a pre-set standard ontology vector library to recall a preset number of candidate ontology classes and their corresponding similarity scores. Construct prompt words containing metadata features, content sample features, context information, and candidate ontology classes, input them into a large language model for logical reasoning, and output the confidence score corresponding to the most matching candidate ontology class; The similarity score and the confidence score are weighted and fused to calculate a comprehensive score. If the comprehensive score exceeds a preset semantic threshold, the column is marked as the corresponding ontology class.
[0010] As a further improvement to the above scheme, the metadata features include the column name and data type of the corresponding column; the content sample features include representative values in the corresponding column that are ranked in descending order of frequency of occurrence and are among the top values of a preset quantity, as well as random sample values; the context information includes the column names of other columns belonging to the same data table; and the concatenation to generate a text description refers to combining the column name, data type, high-frequency value, and random sample value of the corresponding column according to a preset text template into a natural language description containing complete semantics.
[0011] As a further improvement to the above scheme, step S3 specifically includes: For the cross-table field pairs to be detected, the following correlation metrics are calculated in parallel: Calculate the index alignment based on the matching ratio of row number comparison values; Calculate the intersection-union ratio of the value sets to obtain the Jaccard set similarity; Construct a frequency distribution histogram and calculate cosine similarity to obtain distribution similarity; Determine whether two columns of semantic labels belong to the same ontology attribute to obtain semantic consistency weight; The total correlation score is obtained by weighted summation of the above-mentioned correlation indicators. If the total correlation score exceeds the preset correlation threshold, it is determined that the two fields constitute a correlation key and a connection edge is established in the graph.
[0012] As a further improvement to the above scheme, step S4 specifically includes: Calculate the uniqueness ratio, regular expression matching degree, and column name keyword matching degree for each column to obtain the overall primary key score for a single column. When the overall score of a single primary key is lower than a preset threshold, the combined uniqueness of the fields is calculated to identify composite primary keys. Generate a globally unique identifier for each row of data in a format that includes namespace, table name, row hash value and primary key value. Convert the table data into triples containing entities, attributes and relations and store them in the graph database to complete the construction of a dynamic knowledge graph.
[0013] As a further improvement to the above scheme, step S5 specifically includes: Receive natural language query questions from users, analyze the features of qualifying words in the natural language query questions, and calculate the fuzziness score; The response is tiered based on the ambiguity score: if the ambiguity is low, it is converted into a deterministic query statement and executed directly; if the ambiguity is medium, the hypothesis completion strategy is triggered, and the query conditions are automatically supplemented based on the data distribution characteristics and declared in the answer; if the ambiguity is high, the multi-scenario simulation strategy is triggered, and multiple sets of hypothesis parameters are generated, the results are calculated separately, and the results are compared and displayed. Based on the parsed query intent, the corresponding subgraph data is retrieved from the dynamic knowledge graph. The query question, subgraph data, and corresponding assumptions are then input into the large language model to generate a preliminary answer.
[0014] As a further improvement to the above scheme, step S6 specifically includes: The generated preliminary response is parsed to extract key information containing entity identifiers and numerical indicators; Generate a reverse query statement to check whether the entity identifier actually exists in the triples of the dynamic knowledge graph, in order to perform existence verification; Check whether the numerical indicators can be calculated from the attribute values in the dynamic knowledge graph to verify the consistency of the calculation; The final answer carrying a reference to the data source is output only if both the existence verification and the computational consistency verification pass; if either verification fails, the current answer is intercepted and a correction or downgrade strategy is triggered.
[0015] As a further improvement to the above scheme, step S1 specifically includes: Scan the specified data source directory to load multi-source structured files, and clean each loaded structured file, removing completely empty lines and columns; Identify the header positions of the cleaned structured files and count the number of rows, columns, and null values in each data table to generate table-level metadata; The table-level metadata is used as the feature information for subsequent column semantic annotation and cross-table association network construction.
[0016] The present invention also discloses a computer terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the ontology-enhanced automatic analysis method for structured data as described above.
[0017] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the ontology-enhanced structured data automatic analysis method as described above.
[0018] Compared with the prior art, the beneficial effects of the present invention are: 1. The structured data automatic analysis method based on ontology enhancement disclosed in this invention establishes cross-table associations by annotating multi-source data through hybrid semantic mapping and multi-strategy evaluation, thereby constructing a dynamic knowledge graph. Combined with adaptive analysis strategies and entity reverse query verification mechanisms, it realizes automatic association and graph construction of heterogeneous data without manual intervention, effectively eliminates the illusion problem of large model question answering, and improves the reliability of data analysis results.
[0019] 2. In the data cognition and graph construction stage, this invention accurately identifies the true semantics of non-standard fields by splicing metadata and sample features to perform weighted fusion scoring of vectors and large models; by comprehensively calculating multiple similarity indicators, it automatically mines the implicit association keys between tables without foreign key constraints; and by identifying primary keys through multi-dimensional indicators to generate unique identifiers, it achieves the effect of reducing the cost of full-scale inference while getting rid of predefined structure dependence and automatically converting tables with unknown structures from multiple sources into underlying knowledge graphs.
[0020] 3. In the intelligent interaction and verification stage, this invention adapts to the uncertainty of user questions by calculating the ambiguity of the query question and triggering corresponding hierarchical response strategies (such as deterministic query, hypothesis completion, or multi-scenario simulation). At the same time, by extracting entities and values from the preliminary answers of the large model, it initiates reverse dual verification of existence and computational consistency in the graph, realizing the establishment of a rigorous verification loop between the model output and the underlying data, intercepting fabricated data that deviates from factual basis, and ensuring the authenticity and credibility of the final result. Attached Figure Description
[0021] Figure 1 This is a flowchart of the ontology-enhanced automatic structured data analysis method in Embodiment 1 of the present invention.
[0022] Figure 2 This is a flowchart illustrating the specific implementation of the ontology-enhanced automatic structured data analysis method in Embodiment 1 of the present invention.
[0023] Figure 3 This is a flowchart of the hybrid semantic mapping mechanism in Embodiment 1 of the present invention.
[0024] Figure 4 This is a flowchart of the multi-strategy cross-table association discovery process in Embodiment 1 of the present invention.
[0025] Figure 5 This is a flowchart of the anti-hallucination question-and-answer verification closed loop in Embodiment 1 of the present invention.
[0026] Figure 6 This is a schematic diagram of the computer terminal structure in Embodiment 2 of the present invention. Detailed Implementation
[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0028] Example 1
[0029] Please see Figure 1 and Figure 2 This embodiment provides an automatic structured data analysis method based on ontology enhancement, including steps S1 to S6. S1 to S4 constitute the first stage, data cognition and graph construction process (automated backend processing); S5 to S6 constitute the second stage, intelligent interaction and verification process (frontend user response).
[0030] The goal of the first phase is to transform raw, semantically null CSV / Excel files into knowledge graphs with rich semantics and relationships.
[0031] S1. Obtain multi-source structured data and extract feature information of each data table in the structured data.
[0032] Typically, enterprise information systems contain a large number of independent data files in the form of CSV, Excel, or relational database exported files. These multi-source structured data often lack unified standards and foreign key constraints, forming "data silos" that are difficult to use directly.
[0033] As the data access and preprocessing stage of the overall system implementation process, step S1's core lies in automatically parsing the underlying data and extracting preliminary features, laying the foundation for subsequent ontology mapping. Step S1 specifically includes: Scan the specified data source directory to load multi-source structured files, and clean each loaded structured file, removing completely empty lines and columns; Identify the header positions of the cleaned structured files and count the number of rows, columns, and null values in each data table to generate table-level metadata; The table-level metadata is used as the feature information for subsequent column semantic annotation and cross-table association network construction.
[0034] S2. Based on the feature information, a hybrid semantic mapping mechanism is used to semantically annotate the columns of the data table, obtain the candidate ontology classes and the comprehensive scores of the candidate ontology classes for the corresponding columns, and determine the final semantic label when the comprehensive score meets the preset conditions.
[0035] Traditional methods rely on manually defined rules or regular expression matching, and directly using general-purpose large models for full-data inference is too costly and easily limited by context windows. As one of the core innovations of this invention, step S2 proposes a hybrid semantic mapping mechanism, effectively combining lightweight vector retrieval with heavyweight large-scale model inference, ultimately outputting the semantic label for each column (e.g., Col_A mapped to schema:Person). Please refer to... Figure 3 The specific process of semantic annotation includes: Extract metadata features and content sample features from each column of the data table, and concatenate them to generate a text description; throughout the implementation process, this text description serves as a bridge connecting data columns with no clear meaning with the industry standard ontology; The text description is encoded into vectors using a pre-trained semantic vector model (such as BERT), and an approximate nearest neighbor search (ANN) is performed in a pre-built standard ontology vector library (built on Schema.org) to recall the top-K candidate ontology classes with the highest similarity and their corresponding similarity scores. Construct prompt words containing metadata features, content sample features, context information, and candidate ontology classes, input them into a large language model for logical reasoning, and output the confidence score corresponding to the most matching candidate ontology class; The similarity score and the confidence score are weighted and fused to calculate a comprehensive score. If the comprehensive score exceeds a preset semantic threshold, the column is marked as the corresponding ontology class.
[0036] The metadata features include the column name and data type of the corresponding column; the content sample features include representative values in the corresponding column that are ranked in descending order of frequency of occurrence and are among the top values in a preset quantity, as well as random sample values; the context information includes the column names of other columns belonging to the same data table; and the concatenation to generate a text description refers to combining the column name, data type, high-frequency value, and random sample value of the corresponding column into a natural language description containing complete semantics according to a preset text template.
[0037] S3. By combining multiple correlation evaluation strategies, calculate the total correlation score of cross-table field pairs, and establish cross-table relationships between data tables based on the total correlation score.
[0038] To address the pain point of existing technologies in automatically discovering join keys between loosely linked groups of files without foreign key constraints, step S3, as the cross-table join discovery stage in the overall implementation process, replaces simple column name exact matching with a multi-dimensional feature scoring strategy. This achieves automated discovery of join keys, thereby establishing connections between tables. Please refer to [link to relevant documentation]. Figure 4 Step S3 specifically includes: For the cross-table field pairs to be detected, the following correlation metrics are calculated in parallel: The index alignment is calculated based on the matching ratio of row number comparison values. Specifically, the columns of the two tables to be compared are strictly matched one-to-one according to the physical row numbers or default indexes of the original files. The proportion of rows with completely identical values in the same row is then counted. This metric is specifically used to identify related data that has not been shuffled but has been forcibly split into two files.
[0039] The Jaccard similarity is calculated by taking the intersection-union ratio (IU) of two sets of values. Jaccard similarity (IU) is a classic statistical metric used to assess the degree of overlap between two sets of values. Specifically, it involves extracting all unique, deduplicated values from two columns of data to form two sets. The Jaccard similarity is then calculated by dividing the number of elements in the intersection by the number of elements in the union. This metric effectively overcomes data misalignment issues caused by incorrect row numbers; a high score is achieved as long as the two columns contain highly overlapping high-frequency IDs or entity names.
[0040] A frequency distribution histogram is constructed, and cosine similarity is calculated to obtain distribution similarity. Cosine similarity is often used to measure the degree of overlap between two data distributions in terms of direction or trend. For numerical fields, frequency is counted by binning within numerical intervals; for textual fields, frequency is counted by enumerating values by category, thus transforming the data distribution of each column into feature vectors (i.e., the frequency distribution histogram). The cosine similarity between these two feature vectors is then calculated. This metric is used to discover potentially related fields that, even if their specific values differ, exhibit a high degree of consistency in their business distribution patterns (such as monetary distribution or age distribution).
[0041] Determine whether two columns of semantic labels belong to the same ontology attribute to obtain semantic consistency weight; The total correlation score is obtained by weighted summation of the above multiple correlation indicators. If the total correlation score exceeds the preset correlation threshold, it is determined that the two fields constitute a correlation key and a connection edge (such as owl:sameAs) is established in the graph.
[0042] S4. Identify the primary keys of each data table and generate globally unique identifiers (URIs, Uniform Resource Identifiers) for each data row. Combine the semantic tags and the cross-table relationships to construct and store a dynamic knowledge graph.
[0043] To address the shortcomings of existing technologies in constructing dynamic graphs for data with unknown structures, step S4 eliminates the need for an expert team to pre-define the ontology structure, automatically aligning with industry standards (such as Schema.org) and achieving a dynamic transformation from a loose two-dimensional relational table structure to a graph network structure. Step S4 specifically includes: Calculate the uniqueness ratio, ID regular expression matching degree, and column name keyword matching degree for each column to obtain the overall primary key score for each column. Specifically, the uniqueness ratio is calculated by comparing the number of unique values after deduplication in a single column to the total number of rows in the table. If this ratio approaches 1.0, the column has extremely high primary key potential. The regular expression matching degree is calculated by scanning the data sample of the column using a pre-defined regular expression library to evaluate whether its format conforms to typical identifier characteristics (e.g., mixed encoding containing letters and numbers, UUID format, equal-length strings, etc.) and outputting a format matching score. The column name keyword matching degree is calculated by performing word segmentation and semantic analysis on the column names to check whether they contain high-frequency primary key feature words such as "id," "code," "number," and "primary key," assigning corresponding semantic scores.
[0044] When the overall score of a single primary key is lower than a preset threshold, the combined uniqueness of the field combinations is calculated to identify composite primary keys. By traversing two or more adjacent columns or combinations of columns with specific semantics, the combined uniqueness ratio of the combined fields is calculated. If the combined uniqueness ratio meets the threshold requirement, the field combination is identified as a composite primary key.
[0045] Graph databases (such as Neo4j) are specifically designed to store network topology data consisting of "entity-relationship-entity" and attribute nodes, and can provide more efficient multi-hop path penetration and relation query performance than traditional relational databases; Each row of data is generated with a globally unique identifier in the format (Namespace:TableName_RowHash_PKValue) including the namespace, table name, row hash value, and primary key value. This ensures no conflicts between entities across tables. The globally unique identifier guarantees uniqueness and provides a unique basis for inference during large model execution, thereby reducing the illusion of inference in large models. The table data is then converted into RDF triples containing entities, attributes, and relationships and stored in a graph database to complete the construction of a dynamic knowledge graph.
[0046] The goal of the second phase is to receive user questions, generate answers using the constructed graph, and ensure that the answers are authentic and reliable.
[0047] S5. Receive the user's natural language query question, assess the fuzziness of the query question and match an adaptive analysis strategy, and generate a preliminary answer containing key information by combining the dynamic knowledge graph.
[0048] In human-computer interaction, human natural language questions often possess ambiguity, omission, and conditional vagueness. Directly converting these into machine queries can easily lead to query failures or return erroneous results that deviate from the user's true intent. As another core innovation of this invention, "an adaptive question-answering strategy based on fuzziness evaluation," step S5, through a refined hierarchical response mechanism, significantly enhances the system's analytical robustness and exploratory divergence capabilities when facing uncertain query intents. Please refer to [link / reference]. Figure 5 Step S5 specifically includes: Receive the user's natural language query question, analyze the features of the qualifying words in the natural language query question, and calculate the fuzziness score (0-1). The analysis of qualifier features in natural language query questions and the calculation of ambiguity scores specifically include: inputting the user's natural language query question into a pre-built intent analysis big language model; requiring the model to identify whether the question contains explicit qualifier features such as time, location, quantity, and entity name through prompts; and outputting a confidence decay value between 0 and 1 as the ambiguity score based on the degree of missing qualifier features and the clarity of reference. For example, when the query conditions are completely explicit (e.g., sales figures in Beijing in 2023), the score approaches 0; when broad terms (e.g., "recently," "approximately") or key qualifiers are missing, the score increases towards 1.
[0049] The response is tiered based on the ambiguity score: for low ambiguity, it is converted into a deterministic query statement (SPARQL / SQL) and executed directly; for medium ambiguity, a hypothesis completion strategy is triggered, and the query conditions are automatically supplemented based on the data distribution characteristics (e.g., "high frequency" is defined as ">5 times", "recent" = "30 days"), and declared in the response; for high ambiguity, a multi-scenario simulation strategy is triggered, generating multiple sets of hypothesis parameters (Scenario A / B / C), calculating the results respectively, and comparing and displaying them. Based on the parsed query intent, the corresponding subgraph data is retrieved from the dynamic knowledge graph. The query question, subgraph data, and corresponding assumptions are constructed into prompts and input into the large language model to generate an initial answer.
[0050] S6. Extract key information from the preliminary answer, initiate a reverse query verification in the dynamic knowledge graph, and output the final answer based on the verification result.
[0051] Generative artificial intelligence, when processing text and reasoning tasks, inevitably produces an illusion phenomenon—generating seemingly reasonable and rigorous data that actually lacks factual basis—because its essence is predicting the next word based on probability distribution. To address this critical weakness of unreliable data analysis results directly output by large language models, step S6 introduces an "entity reverse verification mechanism." This mechanism establishes a rigorous data verification loop at the underlying level, ensuring the authenticity and credibility of the final output, thus enabling large model technology to be safely applied to serious business decision-making scenarios. Step S6 specifically includes: The generated preliminary response is parsed to extract key information containing entity identifiers and numerical indicators; Generate a reverse query statement to check whether the entity identifier actually exists in the triples of the dynamic knowledge graph, in order to perform existence verification; Check whether the numerical indicators can be calculated from the attribute values in the dynamic knowledge graph to verify the consistency of the calculation; The final answer carrying a reference to the data source is output only if both the existence verification and the computational consistency verification pass; if either verification fails, it is marked as "suspected hallucination", the current answer is intercepted and a correction or downgrade strategy is triggered.
[0052] In summary, the method proposed in this invention has the following advantages: 1. The entire process is automated, reducing the manual ETL and modeling work of several days to minutes, achieving "plug and play".
[0053] 2. The hybrid mapping mechanism enables machines to understand abbreviations, polysemous words, and business terms, significantly improving accuracy.
[0054] 3. By using an entity reference verification mechanism, the illusion problem of generative AI is fundamentally suppressed, and the results are credible and traceable.
[0055] 4. Business personnel do not need to master SQL; they can complete complex analyses involving multiple table joins using only natural language.
[0056] 5. The system does not rely on a predefined schema and can adapt to tabular data with various unknown structures.
[0057] To more clearly demonstrate how this invention works, a specific "sales data analysis" scenario will be used for demonstration below.
[0058] (I) Preparation of Scene Data
[0059] Suppose a user uploaded two loosely structured CSV files with non-standard column names: Table 1. Customer Table (client.csv): Column names are u_code (ID), name (name), and loc (city). Example data: C_001, Zhang San, Beijing.
[0060] Table 2 Transaction Table (trans.csv): Column names are tid (transaction number), cid (foreign key), amt (amount). Data example: T_101, C_001, 1000.
[0061] Note: The column names in the two tables are inconsistent. u_code vs cid ), and there is no explicit foreign key definition.
[0062] (II) Process Demonstration 1: Data Preprocessing and Map Construction (Backend)
[0063] 1. Semantic mapping: The system analyzes Table 1 u_code, and LLM identifies it as a "personnel ID" based on content features.
[0064] The system analyzes Table 2 amt and identifies it as "amount".
[0065] 2. Association Discovery: The system compares Table 1 u_code and Table 2 cid.
[0066] Although the column names are different, Jaccard similarity calculations found that the value sets of the two columns highly overlapped (0.9) and the distribution similarity was extremely high.
[0067] The system determines that the two are related keys and establishes a connection.
[0068] 3. Atlas generation: Generate Zhang San's URI: data:client_row1_C001.
[0069] The URI for generating the order is: data:trans_row1_T101.
[0070] Write relationship: <Order T101> hasCustomer <Zhang San>.
[0071] (III) Process Demonstration 2: Anti-Hallucination Intelligent Question and Answer (Front-end)
[0072] A user asked: "What is the total spending of customers in Beijing?"
[0073] 1. Ambiguity assessment: If the question is clear (low ambiguity), proceed to a precise query.
[0074] 2. Reasoning and Calculation: The system retrieves the customer (Zhang) with loc="Beijing" in the graph and finds his order (T_101) along the associated edge.
[0075] LLM calculation: Zhang's total order amount = 1000.
[0076] 3. Citation verification (key step): LLM generated the answer: "Total consumption of 1000 yuan, from Zhang."
[0077] System reverse verification: Does "Zhang Mou" exist in the graph? (Yes); Does the sum of the order amounts equal 1000? (Yes).
[0078] 4. Final output: "Total consumption of Beijing customers is 1000 yuan. [Source: table trans, row 1]".
[0079] Example 2
[0080] This embodiment provides a computer terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the ontology-enhanced structured data automatic analysis method as described in Embodiment 1.
[0081] like Figure 6 As shown, the computer terminal provided in this embodiment includes: at least one processor 101, and a memory 102 connected to at least one processor 101. This embodiment does not limit the specific connection medium between the processor 101 and the memory 102. Figure 6 The example shown is the connection between processor 101 and memory 102 via bus 100. Bus 100 is... Figure 6 The connections between other components are shown in bold lines and are for illustrative purposes only, not as limiting information. Bus 100 can be divided into address bus, data bus, control bus, etc., for ease of representation. Figure 6 The bus is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. Alternatively, the processor 101 may also be called a controller; there is no restriction on the name.
[0082] In this embodiment, the memory 102 stores instructions that can be executed by at least one processor 101. The at least one processor 101 can execute the aforementioned method by executing the instructions stored in the memory 102.
[0083] The processor 101 is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory 102 and calling data stored in memory 102, the processor can perform various functions and process data, thereby monitoring the device as a whole.
[0084] In one possible design, processor 101 may include one or more processing units. Processor 101 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into processor 101. In some embodiments, processor 101 and memory 102 may be implemented on the same chip; in some embodiments, they may also be implemented on separate chips.
[0085] Processor 101 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the ontology-enhanced structured data automatic analysis method disclosed in Embodiment 1 can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules in processor 101.
[0086] Memory 102, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 102 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 102 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In this embodiment, memory 102 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.
[0087] By designing and programming the processor 101, the code corresponding to the ontology-enhanced structured data automatic analysis method described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute the code during runtime. Figure 1 The steps of the ontology-enhanced structured data automatic analysis method are shown. How to design and program the processor 101 is a technique well-known to those skilled in the art and will not be described further here.
[0088] Example 3
[0089] This embodiment provides a computer-readable storage medium storing a computer program thereon. When the program is executed by a processor, it implements the steps of the ontology-enhanced structured data automatic analysis method as described in Embodiment 1.
[0090] The computer-readable storage medium may include flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage medium may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, smart memory card, secure digital card, flash memory card, etc., provided on the computer device. Of course, the storage medium may include both internal storage units and external storage devices of the computer device. In this embodiment, the memory is typically used to store the operating system and various application software installed on the computer device. In addition, the memory can also be used to temporarily store various types of data that have been output or will be output.
[0091] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for automatic analysis of structured data based on ontology enrichment, characterized in that, include: S1. Obtain multi-source structured data and extract feature information of each data table in the structured data; S2. Based on the feature information, a hybrid semantic mapping mechanism is used to semantically annotate the columns of the data table, obtain the candidate ontology classes and the comprehensive scores of the candidate ontology classes for the corresponding columns, and determine the final semantic label when the comprehensive score meets the preset conditions. S3. By combining multiple correlation evaluation strategies, calculate the total correlation score of cross-table field pairs, and establish cross-table relationships between data tables based on the total correlation score; S4. Identify the primary keys of each data table and generate globally unique identifiers for data rows. Combine the semantic tags and the cross-table relationships to construct and store a dynamic knowledge graph. S5. Receive the user's natural language query question, assess the fuzziness of the query question and match an adaptive analysis strategy, and generate a preliminary answer containing key information by combining the dynamic knowledge graph; S6. Extract key information from the preliminary answer, initiate a reverse query verification in the dynamic knowledge graph, and output the final answer based on the verification result.
2. The ontology-enhanced structured data automatic analysis method according to claim 1, characterized in that, In step S2, the specific process of semantically annotating the columns of the data table based on the feature information using a hybrid semantic mapping mechanism includes: Extract metadata features and content sample features from each column of the data table, and concatenate them to generate a text description; The text description is encoded into a vector using a pre-trained semantic vector model, and an approximate nearest neighbor search is performed in a pre-set standard ontology vector library to recall a preset number of candidate ontology classes and their corresponding similarity scores. Construct prompt words containing metadata features, content sample features, context information, and candidate ontology classes, input them into a large language model for logical reasoning, and output the confidence score corresponding to the most matching candidate ontology class; The similarity score and the confidence score are weighted and fused to calculate a comprehensive score. If the comprehensive score exceeds a preset semantic threshold, the column is marked as the corresponding ontology class.
3. The ontology-enhanced structured data automatic analysis method according to claim 2, characterized in that, The metadata features include the column name and data type of the corresponding column; the content sample features include representative values in the corresponding column that are ranked in descending order of frequency of occurrence and are among the top values of a preset quantity, as well as random sample values; the context information includes the column names of other columns belonging to the same data table; The term "concatenating and generating text description" refers to combining the column names, data types, high-frequency values, and random sampled values of the corresponding columns according to a preset text template into a natural language description containing complete semantics.
4. The ontology-enhanced structured data automatic analysis method according to claim 1, characterized in that, Step S3 specifically includes: For the cross-table field pairs to be detected, the following correlation metrics are calculated in parallel: Calculate the index alignment based on the matching ratio of row number comparison values; Calculate the intersection-union ratio of the value sets to obtain the Jaccard set similarity; Construct a frequency distribution histogram and calculate cosine similarity to obtain distribution similarity; Determine whether two columns of semantic labels belong to the same ontology attribute to obtain semantic consistency weight; The total correlation score is obtained by weighted summation of the above-mentioned correlation indicators. If the total correlation score exceeds the preset correlation threshold, it is determined that the two fields constitute a correlation key and a connection edge is established in the graph.
5. The ontology-enhanced structured data automatic analysis method according to claim 1, characterized in that, Step S4 specifically includes: Calculate the uniqueness ratio, regular expression matching degree, and column name keyword matching degree for each column to obtain the overall primary key score for a single column. When the overall score of a single primary key is lower than a preset threshold, the combined uniqueness of the fields is calculated to identify composite primary keys. Generate a globally unique identifier for each row of data in a format that includes namespace, table name, row hash value and primary key value. Convert the table data into triples containing entities, attributes and relations and store them in the graph database to complete the construction of a dynamic knowledge graph.
6. The ontology-enhanced structured data automatic analysis method according to claim 1, characterized in that, Step S5 specifically includes: Receive natural language query questions from users, analyze the features of qualifying words in the natural language query questions, and calculate the fuzziness score; The response is tiered based on the ambiguity score: if the ambiguity is low, it is converted into a deterministic query statement and executed directly; if the ambiguity is medium, the hypothesis completion strategy is triggered, and the query conditions are automatically supplemented based on the data distribution characteristics and declared in the answer; if the ambiguity is high, the multi-scenario simulation strategy is triggered, and multiple sets of hypothesis parameters are generated, the results are calculated separately, and the results are compared and displayed. Based on the parsed query intent, the corresponding subgraph data is retrieved from the dynamic knowledge graph. The query question, subgraph data, and corresponding assumptions are then input into the large language model to generate a preliminary answer.
7. The ontology-enhanced structured data automatic analysis method of claim 1, wherein, Step S6 specifically includes: The generated preliminary response is parsed to extract key information containing entity identifiers and numerical indicators; Generate a reverse query statement to check whether the entity identifier actually exists in the triples of the dynamic knowledge graph, in order to perform existence verification; Check whether the numerical indicators can be calculated from the attribute values in the dynamic knowledge graph to verify the consistency of the calculation; The final answer carrying a reference to the data source is output only if both the existence verification and the computational consistency verification pass; if either verification fails, the current answer is intercepted and a correction or downgrade strategy is triggered.
8. The ontology-enhanced structured data automatic analysis method of claim 1, wherein, Step S1 specifically includes: Scan the specified data source directory to load multi-source structured files, and clean each loaded structured file, removing completely empty lines and columns; Identify the header positions of the cleaned structured files and count the number of rows, columns, and null values in each data table to generate table-level metadata; The table-level metadata is used as the feature information for subsequent column semantic annotation and cross-table association network construction.
9. A computer terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the ontology-enhanced automatic analysis method for structured data as described in any one of claims 1 to 8.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the program is executed by the processor, it implements the steps of the ontology-enhanced automatic analysis method for structured data as described in any one of claims 1 to 8.
